* [PATCH v6 00/34] GPU scheduler for i915 driver
@ 2016-04-20 17:13 John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 01/34] drm/i915: Add total count to context status debugfs output John.C.Harrison
` (41 more replies)
0 siblings, 42 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Implemented a batch buffer submission scheduler for the i915 DRM driver.
The general theory of operation is that when batch buffers are
submitted to the driver, the execbuffer() code assigns a unique seqno
value and then packages up all the information required to execute the
batch buffer at a later time. This package is given over to the
scheduler which adds it to an internal node list. The scheduler also
scans the list of objects associated with the batch buffer and
compares them against the objects already in use by other buffers in
the node list. If matches are found then the new batch buffer node is
marked as being dependent upon the matching node. The same is done for
the context object. The scheduler also bumps up the priority of such
matching nodes on the grounds that the more dependencies a given batch
buffer has the more important it is likely to be.
The scheduler aims to have a given (tuneable) number of batch buffers
in flight on the hardware at any given time. If fewer than this are
currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added
to the queue and the driver returns back to user land.
As each batch buffer completes, it raises an interrupt which wakes up
the scheduler. Note that it is possible for multiple buffers to
complete before the IRQ handler gets to run. Further, the seqno values
of the individual buffers are not necessary incrementing as the
scheduler may have re-ordered their submission. However, the scheduler
keeps the list of executing buffers in order of hardware submission.
Thus it can scan through the list until a matching seqno is found and
then mark all in flight nodes from that point on as completed.
A deferred work queue is also poked by the interrupt handler. When
this wakes up it can do more involved processing such as actually
removing completed nodes from the queue and freeing up the resources
associated with them (internal memory allocations, DRM object
references, context reference, etc.). The work handler also checks the
in flight count and calls the submission code if a new slot has
appeared.
When the scheduler's submit code is called, it scans the queued node
list for the highest priority node that has no unmet dependencies.
Note that the dependency calculation is complex as it must take
inter-ring dependencies and potential preemptions into account. Note
also that in the future this will be extended to include external
dependencies such as the Android Native Sync file descriptors and/or
the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to execbuff_final() for
submission to the hardware. The in flight count is then re-checked and
a new node popped from the list if appropriate.
The scheduler also allows high priority batch buffers (e.g. from a
desktop compositor) to jump ahead of whatever is already running if
the underlying hardware supports pre-emption. In this situation, any
work that was pre-empted is returned to the queued list ready to be
resubmitted when no more high priority work is outstanding.
Various IGT tests are in progress to test the scheduler's operation
and will follow.
v2: Updated for changes in struct fence patch series and other changes
to underlying tree (e.g. removal of cliprects). Also changed priority
levels to be signed +/-1023 range and reduced mutex lock usage.
v3: More reuse of cached pointers rather than repeated dereferencing
(David Gordon).
Moved the dependency generation code out to a seperate function for
easier readability. Also added in support for the read-read
optimisation.
Major simplification of the DRM file close handler.
Fixed up an overzealous WARN.
Removed unnecessary flushing of the scheduler queue when waiting for a
request.
v4: Removed user land fence/sync integration as this is dependent upon
the de-staging of the Android sync code. That de-staging is now being
done by someone else. The sync support will be added back in to the
scheduler in a separate patch series which must wait for the
de-staging to be landed first.
Added support for killing batches from contexts that were banned after
the batches were submitted to the scheduler.
Changed various comments to fix typos, update to reflect changes to
the code, correct formatting and line wrapping, etc. Also wrapped
various long lines and twiddled white space to keep the style checker
happy.
Changed a bunch of BUG_ONs to WARN_ONs as apparently the latter are
preferred.
Used the correct array memory allocator function (kmalloc_array
instead of kmalloc).
Fixed a variable type (review comment by Joonas).
Fixed a WARN_ON firing incorrectly when removing killed nodes from the
scheduler's queue.
Dropped register definition update patch from this series. The changes
are all for pre-emption so it makes more sense for it to be part of
that series instead.
v5: Reverted power management changes as they apparently conflict with
mutex acquisition. Converted the override mask module parameter to a
set of boolean enable flags (just one in this patch set, but others
are added later for controlling pre-emption). [Chris Wilson]
Removed lots of whitespace from i915_scheduler.c and re-ordered it to
remove all forward declarations. Squashed down the i915_scheduler.c
sections of various patches into the initial 'start of scheduler'
patch. Thus the later patches simply hook in existing code into
various parts of the driver rather than adding the code as well. Added
documentation to various functions. Re-worked the submit function in
terms of mutex locking, error handling and exit paths. Split the
delayed work handler function in half. Made use of the kernel 'clamp'
macro. [Joonas Lahtinen]
Dropped the 'immediate submission' override option. This was a half
way house between full scheduler and direct submission and was only
really useful during early debug.
Added a re-install of the scheduler's interrupt hook around GPU reset.
[Zhipeng Gong]
Used lighter weight spinlocks.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' and 'assert_scheduler_lock_held()'
helper macros. Renamed 'i915_gem_execbuff_release_batch_obj' to
'i915_gem_execbuf_release_batch_obj'. Updated to use 'to_i915()'
instead of dev_private. Converted all enum labels to uppercase.
Removed various unnecessary WARNs. Renamed 'saved_objects' to just
'objs'. More code refactoring. Removed even more white space. Added
an i915_scheduler_destroy() function instead of doing explicit clean
up of scheduler internals from i915_driver_unload(). Changed extra
boolean i915_wait_request() parameter to a flags word and consumed the
original boolean parameter too. Also, replaced the
i915_scheduler_is_request_tracked() function with
i915_scheduler_is_mutex_required() and
i915_scheduler_is_request_batch_buffer() as the need for the former
has gone away and it was really being used to ask the latter two
questions in a convoluted manner. Wrapped boolean 'flush' parameter to
intel_engine_idle() with an _flush() macro.
[review feedback from Joonas Lahtinen]
Moved scheduler modue parameter declaration to correct place in
i915_params struct. [review feedback from Matt Roper]
Added an admin only check when setting the tuning parameters via
debugfs to prevent rogue user code trying to break the system with
strange settings. [review feedback from Jesse Barnes]
Added kerneldoc for intel_engine_idle().
Added running totals of 'flying' and 'queued' nodes rather than
re-calculating each time as a minor CPU performance optimisation.
Removed support for out of order seqno completion. All the prep work
patch series (seqno to request conversion, late seqno assignment,
etc.) that has now been done means that the scheduler no longer
generates out of order seqno completions. Thus all the complex code
for coping with such is no longer required and can be removed.
Fixed a bug in scheduler bypass mode introduced in the clean up code
refactoring of v5. The clean up function was seeing the node in the
wrong state and thus refusing to process it.
Improved the throttle by file handle feature by chaning from a simple
'return to userland when full' scheme with a 'sleep on request'
scheme. The former could lead to the busy polling and wasting lots of
CPU time as user land continuously retried the execbuf IOCTL in a
tight loop. Now the driver will sleep (without holding the mutex lock)
on the oldest request outstanding for that file and then automatically
retry. This is closer to the pre-scheduler behaviour of stalling on a
full ring buffer.
[Patches against drm-intel-nightly tree fetched 13/04/2016 with struct
fence conversion patches applied]
Dave Gordon (2):
drm/i915: Cache request pointer in *_submission_final()
drm/i915: Add scheduling priority to per-context parameters
John Harrison (32):
drm/i915: Add total count to context status debugfs output
drm/i915: Prelude to splitting i915_gem_do_execbuffer in two
drm/i915: Split i915_dem_do_execbuffer() in half
drm/i915: Re-instate request->uniq because it is extremely useful
drm/i915: Start of GPU scheduler
drm/i915: Disable hardware semaphores when GPU scheduler is enabled
drm/i915: Force MMIO flips when scheduler enabled
drm/i915: Added scheduler hook when closing DRM file handles
drm/i915: Added scheduler hook into i915_gem_request_notify()
drm/i915: Added deferred work handler for scheduler
drm/i915: Redirect execbuffer_final() via scheduler
drm/i915: Keep the reserved space mechanism happy
drm/i915: Added tracking/locking of batch buffer objects
drm/i915: Hook scheduler node clean up into retire requests
drm/i915: Added scheduler support to __wait_request() calls
drm/i915: Added scheduler support to page fault handler
drm/i915: Added scheduler flush calls to ring throttle and idle functions
drm/i915: Add scheduler hook to GPU reset
drm/i915: Added a module parameter to allow the scheduler to be disabled
drm/i915: Support for 'unflushed' ring idle
drm/i915: Defer seqno allocation until actual hardware submission time
drm/i915: Added trace points to scheduler
drm/i915: Added scheduler queue throttling by DRM file handle
drm/i915: Added debugfs interface to scheduler tuning parameters
drm/i915: Add early exit to execbuff_final() if insufficient ring space
drm/i915: Added scheduler statistic reporting to debugfs
drm/i915: Add scheduler support functions for TDR
drm/i915: Enable GPU scheduler by default
drm/i915: Add support for retro-actively banning batch buffers
drm/i915: Allow scheduler to manage inter-ring object synchronisation
drm/i915: Added debug state dump facilities to scheduler
drm/i915: Scheduler state dump via debugfs
drivers/gpu/drm/i915/Makefile | 1 +
drivers/gpu/drm/i915/i915_debugfs.c | 336 +++++-
drivers/gpu/drm/i915/i915_dma.c | 5 +
drivers/gpu/drm/i915/i915_drv.c | 9 +
drivers/gpu/drm/i915/i915_drv.h | 58 +-
drivers/gpu/drm/i915/i915_gem.c | 156 ++-
drivers/gpu/drm/i915/i915_gem_context.c | 24 +
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 297 +++--
drivers/gpu/drm/i915/i915_params.c | 4 +
drivers/gpu/drm/i915/i915_params.h | 1 +
drivers/gpu/drm/i915/i915_scheduler.c | 1709 ++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 180 +++
drivers/gpu/drm/i915/i915_trace.h | 225 +++-
drivers/gpu/drm/i915/intel_display.c | 10 +-
drivers/gpu/drm/i915/intel_lrc.c | 161 ++-
drivers/gpu/drm/i915/intel_lrc.h | 1 +
drivers/gpu/drm/i915/intel_ringbuffer.c | 69 +-
drivers/gpu/drm/i915/intel_ringbuffer.h | 5 +-
include/uapi/drm/i915_drm.h | 1 +
19 files changed, 3118 insertions(+), 134 deletions(-)
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* [PATCH v6 01/34] drm/i915: Add total count to context status debugfs output
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 02/34] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two John.C.Harrison
` (40 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When there are lots and lots and even more lots of contexts (e.g. when
running with execlists) it is useful to be able to immediately see
what the total context count is.
v4: Re-typed a variable (review feedback from Joonas)
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 29ff1b3..e89781a 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -1961,7 +1961,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
struct intel_engine_cs *engine;
struct intel_context *ctx;
enum intel_engine_id id;
- int ret;
+ int ret, count = 0;
ret = mutex_lock_interruptible(&dev->struct_mutex);
if (ret)
@@ -1976,6 +1976,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
describe_ctx(m, ctx);
if (ctx == dev_priv->kernel_context)
seq_printf(m, "(kernel context) ");
+ count++;
if (i915.enable_execlists) {
seq_putc(m, '\n');
@@ -1999,6 +2000,8 @@ static int i915_context_status(struct seq_file *m, void *unused)
seq_putc(m, '\n');
}
+ seq_printf(m, "Total: %d contexts\n", count);
+
mutex_unlock(&dev->struct_mutex);
return 0;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 02/34] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 01/34] drm/i915: Add total count to context status debugfs output John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 03/34] drm/i915: Split i915_dem_do_execbuffer() in half John.C.Harrison
` (39 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler decouples the submission of batch buffers to the driver
with their submission to the hardware. This basically means splitting
the execbuffer() function in half. This change rearranges some code
ready for the split to occur.
v5: Dropped runtime PM calls as they conflict with the mutex lock.
Instead of being done at the lowest submission level, they are now
left right at the top driver entry level. [feedback from Chris Wilson]
v6: Updated to newer nightly (lots of ring -> engine renaming).
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 41 ++++++++++++++++++------------
drivers/gpu/drm/i915/intel_lrc.c | 18 +++++++++----
2 files changed, 38 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 40d333c..f25fd2c 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -970,10 +970,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
if (flush_domains & I915_GEM_DOMAIN_GTT)
wmb();
- /* Unconditionally invalidate gpu caches and ensure that we do flush
- * any residual writes from the previous batch.
- */
- return intel_ring_invalidate_all_caches(req);
+ return 0;
}
static bool
@@ -1238,17 +1235,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
u32 instp_mask;
int ret;
- ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
- if (ret)
- return ret;
-
- ret = i915_switch_context(params->request);
- if (ret)
- return ret;
-
- WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<engine->id),
- "%s didn't clear reload\n", engine->name);
-
instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
instp_mask = I915_EXEC_CONSTANTS_MASK;
switch (instp_mode) {
@@ -1282,6 +1268,30 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
return -EINVAL;
}
+ ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
+ if (ret)
+ return ret;
+
+ i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+ /* To be split into two functions here... */
+
+ /*
+ * Unconditionally invalidate gpu caches and ensure that we do flush
+ * any residual writes from the previous batch.
+ */
+ ret = intel_ring_invalidate_all_caches(params->request);
+ if (ret)
+ return ret;
+
+ /* Switch to the correct context for the batch */
+ ret = i915_switch_context(params->request);
+ if (ret)
+ return ret;
+
+ WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<engine->id),
+ "%s didn't clear reload\n", engine->name);
+
if (engine == &dev_priv->engine[RCS] &&
instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(params->request, 4);
@@ -1318,7 +1328,6 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
- i915_gem_execbuffer_move_to_active(vmas, params->request);
i915_gem_execbuffer_retire_commands(params);
return 0;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index a351a0e..5866342 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -690,10 +690,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
if (flush_domains & I915_GEM_DOMAIN_GTT)
wmb();
- /* Unconditionally invalidate gpu caches and ensure that we do flush
- * any residual writes from the previous batch.
- */
- return logical_ring_invalidate_all_caches(req);
+ return 0;
}
int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -989,6 +986,18 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
if (ret)
return ret;
+ i915_gem_execbuffer_move_to_active(vmas, params->request);
+
+ /* To be split into two functions here... */
+
+ /*
+ * Unconditionally invalidate gpu caches and ensure that we do flush
+ * any residual writes from the previous batch.
+ */
+ ret = logical_ring_invalidate_all_caches(params->request);
+ if (ret)
+ return ret;
+
if (engine == &dev_priv->engine[RCS] &&
instp_mode != dev_priv->relative_constants_mode) {
ret = intel_logical_ring_begin(params->request, 4);
@@ -1013,7 +1022,6 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
- i915_gem_execbuffer_move_to_active(vmas, params->request);
i915_gem_execbuffer_retire_commands(params);
return 0;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 03/34] drm/i915: Split i915_dem_do_execbuffer() in half
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 01/34] drm/i915: Add total count to context status debugfs output John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 02/34] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 04/34] drm/i915: Cache request pointer in *_submission_final() John.C.Harrison
` (38 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Split the execbuffer() function in half. The first half collects and
validates all the information required to process the batch buffer. It
also does all the object pinning, relocations, active list management,
etc - basically anything that must be done upfront before the IOCTL
returns and allows the user land side to start changing/freeing
things. The second half does the actual ring submission.
This change implements the split but leaves the back half being called
directly from the end of the front half.
v2: Updated due to changes in underlying tree - addition of sync fence
support and removal of cliprects.
v3: Moved local 'ringbuf' variable to make later patches in the
series a bit neater.
v4: Corrected a typo in the commit message and downgraded a BUG_ON to
a WARN_ON as the latter is preferred. Also removed all the external
sync/fence support as that will now be a separate patch series.
v5: Updated for runtime PM changes.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Renamed 'i915_gem_execbuff_release_batch_obj' to
'i915_gem_execbuf_release_batch_obj' and updated to use 'to_i915()'
instead of dev_private. [review feedback from Joonas Lahtinen]
Added an explicit remove_from_client() call to the failure path to fix
a race condition with invalid requests the client list.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_drv.h | 11 +++
drivers/gpu/drm/i915/i915_gem.c | 2 +
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 113 +++++++++++++++++++++--------
drivers/gpu/drm/i915/intel_lrc.c | 57 ++++++++++-----
drivers/gpu/drm/i915/intel_lrc.h | 1 +
5 files changed, 137 insertions(+), 47 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 9519b11..f978539 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1701,10 +1701,18 @@ struct i915_execbuffer_params {
struct drm_device *dev;
struct drm_file *file;
uint32_t dispatch_flags;
+ uint32_t args_flags;
uint32_t args_batch_start_offset;
+ uint32_t args_batch_len;
+ uint32_t args_num_cliprects;
+ uint32_t args_DR1;
+ uint32_t args_DR4;
uint64_t batch_obj_vm_offset;
struct intel_engine_cs *engine;
struct drm_i915_gem_object *batch_obj;
+ struct drm_clip_rect *cliprects;
+ uint32_t instp_mask;
+ int instp_mode;
struct intel_context *ctx;
struct drm_i915_gem_request *request;
};
@@ -1991,6 +1999,7 @@ struct drm_i915_private {
int (*execbuf_submit)(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
+ int (*execbuf_final)(struct i915_execbuffer_params *params);
int (*init_engines)(struct drm_device *dev);
void (*cleanup_engine)(struct intel_engine_cs *engine);
void (*stop_engine)(struct intel_engine_cs *engine);
@@ -2896,9 +2905,11 @@ int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
struct drm_i915_gem_request *req);
void i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params);
+void i915_gem_execbuf_release_batch_obj(struct drm_i915_gem_object *batch_obj);
int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params);
int i915_gem_execbuffer(struct drm_device *dev, void *data,
struct drm_file *file_priv);
int i915_gem_execbuffer2(struct drm_device *dev, void *data,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index d1f8c47..21fce67 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -5383,11 +5383,13 @@ int i915_gem_init(struct drm_device *dev)
if (!i915.enable_execlists) {
dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
+ dev_priv->gt.execbuf_final = i915_gem_ringbuffer_submission_final;
dev_priv->gt.init_engines = i915_gem_init_engines;
dev_priv->gt.cleanup_engine = intel_cleanup_engine;
dev_priv->gt.stop_engine = intel_stop_engine;
} else {
dev_priv->gt.execbuf_submit = intel_execlists_submission;
+ dev_priv->gt.execbuf_final = intel_execlists_submission_final;
dev_priv->gt.init_engines = intel_logical_rings_init;
dev_priv->gt.cleanup_engine = intel_logical_ring_cleanup;
dev_priv->gt.stop_engine = intel_logical_ring_stop;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index f25fd2c..6e500bf 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1230,41 +1230,38 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_device *dev = params->dev;
struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
- u64 exec_start, exec_len;
- int instp_mode;
- u32 instp_mask;
int ret;
- instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
- instp_mask = I915_EXEC_CONSTANTS_MASK;
- switch (instp_mode) {
+ params->instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
+ params->instp_mask = I915_EXEC_CONSTANTS_MASK;
+ switch (params->instp_mode) {
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
+ if (params->instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
- if (instp_mode != dev_priv->relative_constants_mode) {
+ if (params->instp_mode != dev_priv->relative_constants_mode) {
if (INTEL_INFO(dev)->gen < 4) {
DRM_DEBUG("no rel constants on pre-gen4\n");
return -EINVAL;
}
if (INTEL_INFO(dev)->gen > 5 &&
- instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
+ params->instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
return -EINVAL;
}
/* The HW changed the meaning on this bit on gen6 */
if (INTEL_INFO(dev)->gen >= 6)
- instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
+ params->instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
}
break;
default:
- DRM_DEBUG("execbuf with unknown constants: %d\n", instp_mode);
+ DRM_DEBUG("execbuf with unknown constants: %d\n", params->instp_mode);
return -EINVAL;
}
@@ -1274,7 +1271,33 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- /* To be split into two functions here... */
+ ret = dev_priv->gt.execbuf_final(params);
+ if (ret)
+ return ret;
+
+ /*
+ * Free everything that was stored in the QE structure (until the
+ * scheduler arrives and does it instead):
+ */
+ if (params->dispatch_flags & I915_DISPATCH_SECURE)
+ i915_gem_execbuf_release_batch_obj(params->batch_obj);
+
+ return 0;
+}
+
+/*
+ * This is the main function for sending a batch to the engine.
+ * It is called from the scheduler, with the struct_mutex already held.
+ */
+int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
+{
+ struct drm_i915_private *dev_priv = to_i915(params->dev);
+ struct intel_engine_cs *engine = params->engine;
+ u64 exec_start, exec_len;
+ int ret;
+
+ /* The mutex must be acquired before calling this function */
+ WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
@@ -1293,7 +1316,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
"%s didn't clear reload\n", engine->name);
if (engine == &dev_priv->engine[RCS] &&
- instp_mode != dev_priv->relative_constants_mode) {
+ params->instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(params->request, 4);
if (ret)
return ret;
@@ -1301,19 +1324,19 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
intel_ring_emit(engine, MI_NOOP);
intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
intel_ring_emit_reg(engine, INSTPM);
- intel_ring_emit(engine, instp_mask << 16 | instp_mode);
+ intel_ring_emit(engine, params->instp_mask << 16 | params->instp_mode);
intel_ring_advance(engine);
- dev_priv->relative_constants_mode = instp_mode;
+ dev_priv->relative_constants_mode = params->instp_mode;
}
- if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
- ret = i915_reset_gen7_sol_offsets(dev, params->request);
+ if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
+ ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
if (ret)
return ret;
}
- exec_len = args->batch_len;
+ exec_len = params->args_batch_len;
exec_start = params->batch_obj_vm_offset +
params->args_batch_start_offset;
@@ -1642,24 +1665,47 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->file = file;
params->engine = engine;
params->dispatch_flags = dispatch_flags;
+ params->args_flags = args->flags;
+ params->args_batch_len = args->batch_len;
+ params->args_num_cliprects = args->num_cliprects;
+ params->args_DR1 = args->DR1;
+ params->args_DR4 = args->DR4;
params->batch_obj = batch_obj;
params->ctx = ctx;
params->request = req;
ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
+ if (ret)
+ goto err_client;
+
+ /* the request owns the ref now */
+ i915_gem_context_unreference(ctx);
-err_batch_unpin:
/*
- * FIXME: We crucially rely upon the active tracking for the (ppgtt)
- * batch vma for correctness. For less ugly and less fragility this
- * needs to be adjusted to also track the ggtt batch vma properly as
- * active.
+ * The eb list is no longer required. The scheduler has extracted all
+ * the information than needs to persist.
*/
+ eb_destroy(eb);
+
+ /*
+ * Don't clean up everything that is now saved away in the queue.
+ * Just unlock and return immediately.
+ */
+ mutex_unlock(&dev->struct_mutex);
+
+ intel_runtime_pm_put(dev_priv);
+
+ return 0;
+
+err_client:
+ if (req->file_priv)
+ i915_gem_request_remove_from_client(req);
+
+err_batch_unpin:
if (dispatch_flags & I915_DISPATCH_SECURE)
- i915_gem_object_ggtt_unpin(batch_obj);
+ i915_gem_execbuf_release_batch_obj(batch_obj);
err:
- /* the request owns the ref now */
i915_gem_context_unreference(ctx);
eb_destroy(eb);
@@ -1668,12 +1714,8 @@ err:
* must be freed again. If it was submitted then it is being tracked
* on the active request list and no clean up is required here.
*/
- if (ret && !IS_ERR_OR_NULL(req)) {
- if (req->file_priv)
- i915_gem_request_remove_from_client(req);
-
+ if (ret && !IS_ERR_OR_NULL(req))
i915_gem_request_cancel(req);
- }
mutex_unlock(&dev->struct_mutex);
@@ -1684,6 +1726,17 @@ pre_mutex_err:
return ret;
}
+void i915_gem_execbuf_release_batch_obj(struct drm_i915_gem_object *batch_obj)
+{
+ /*
+ * FIXME: We crucially rely upon the active tracking for the (ppgtt)
+ * batch vma for correctness. For less ugly and less fragility this
+ * needs to be adjusted to also track the ggtt batch vma properly as
+ * active.
+ */
+ i915_gem_object_ggtt_unpin(batch_obj);
+}
+
/*
* Legacy execbuffer just creates an exec2 list from the original exec object
* list array and passes it to the real function.
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 5866342..6ac063d 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -945,35 +945,31 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_device *dev = params->dev;
struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
- struct intel_ringbuffer *ringbuf = params->ctx->engine[engine->id].ringbuf;
- u64 exec_start;
- int instp_mode;
- u32 instp_mask;
int ret;
- instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
- instp_mask = I915_EXEC_CONSTANTS_MASK;
- switch (instp_mode) {
+ params->instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
+ params->instp_mask = I915_EXEC_CONSTANTS_MASK;
+ switch (params->instp_mode) {
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
+ if (params->instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
- if (instp_mode != dev_priv->relative_constants_mode) {
- if (instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
+ if (params->instp_mode != dev_priv->relative_constants_mode) {
+ if (params->instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
return -EINVAL;
}
/* The HW changed the meaning on this bit on gen6 */
- instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
+ params->instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
}
break;
default:
- DRM_DEBUG("execbuf with unknown constants: %d\n", instp_mode);
+ DRM_DEBUG("execbuf with unknown constants: %d\n", params->instp_mode);
return -EINVAL;
}
@@ -988,7 +984,34 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- /* To be split into two functions here... */
+ ret = dev_priv->gt.execbuf_final(params);
+ if (ret)
+ return ret;
+
+ /*
+ * Free everything that was stored in the QE structure (until the
+ * scheduler arrives and does it instead):
+ */
+ if (params->dispatch_flags & I915_DISPATCH_SECURE)
+ i915_gem_execbuf_release_batch_obj(params->batch_obj);
+
+ return 0;
+}
+
+/*
+ * This is the main function for sending a batch to the engine.
+ * It is called from the scheduler, with the struct_mutex already held.
+ */
+int intel_execlists_submission_final(struct i915_execbuffer_params *params)
+{
+ struct drm_i915_private *dev_priv = to_i915(params->dev);
+ struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+ struct intel_engine_cs *engine = params->engine;
+ u64 exec_start;
+ int ret;
+
+ /* The mutex must be acquired before calling this function */
+ WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
@@ -999,7 +1022,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
return ret;
if (engine == &dev_priv->engine[RCS] &&
- instp_mode != dev_priv->relative_constants_mode) {
+ params->instp_mode != dev_priv->relative_constants_mode) {
ret = intel_logical_ring_begin(params->request, 4);
if (ret)
return ret;
@@ -1007,14 +1030,14 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
intel_logical_ring_emit(ringbuf, MI_NOOP);
intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
intel_logical_ring_emit_reg(ringbuf, INSTPM);
- intel_logical_ring_emit(ringbuf, instp_mask << 16 | instp_mode);
+ intel_logical_ring_emit(ringbuf, params->instp_mask << 16 | params->instp_mode);
intel_logical_ring_advance(ringbuf);
- dev_priv->relative_constants_mode = instp_mode;
+ dev_priv->relative_constants_mode = params->instp_mode;
}
exec_start = params->batch_obj_vm_offset +
- args->batch_start_offset;
+ params->args_batch_start_offset;
ret = engine->emit_bb_start(params->request, exec_start, params->dispatch_flags);
if (ret)
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index 9affda2..3d0a4c9 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -121,6 +121,7 @@ struct i915_execbuffer_params;
int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas);
+int intel_execlists_submission_final(struct i915_execbuffer_params *params);
void intel_execlists_retire_requests(struct intel_engine_cs *engine);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 04/34] drm/i915: Cache request pointer in *_submission_final()
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (2 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 03/34] drm/i915: Split i915_dem_do_execbuffer() in half John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 05/34] drm/i915: Re-instate request->uniq because it is extremely useful John.C.Harrison
` (37 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: Dave Gordon <david.s.gordon@intel.com>
Keep a local copy of the request pointer in the _final() functions
rather than dereferencing the params block repeatedly.
v3: New patch in series.
v6: Updated to newer nightly (lots of ring -> engine renaming).
For: VIZ-1587
Signed-off-by: Dave Gordon <david.s.gordon@intel.com>
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 17 +++++++++--------
drivers/gpu/drm/i915/intel_lrc.c | 11 ++++++-----
2 files changed, 15 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 6e500bf..2b902e3 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1292,6 +1292,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
{
struct drm_i915_private *dev_priv = to_i915(params->dev);
+ struct drm_i915_gem_request *req = params->request;
struct intel_engine_cs *engine = params->engine;
u64 exec_start, exec_len;
int ret;
@@ -1303,12 +1304,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
- ret = intel_ring_invalidate_all_caches(params->request);
+ ret = intel_ring_invalidate_all_caches(req);
if (ret)
return ret;
/* Switch to the correct context for the batch */
- ret = i915_switch_context(params->request);
+ ret = i915_switch_context(req);
if (ret)
return ret;
@@ -1317,7 +1318,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (engine == &dev_priv->engine[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_ring_begin(params->request, 4);
+ ret = intel_ring_begin(req, 4);
if (ret)
return ret;
@@ -1331,7 +1332,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
}
if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
- ret = i915_reset_gen7_sol_offsets(params->dev, params->request);
+ ret = i915_reset_gen7_sol_offsets(params->dev, req);
if (ret)
return ret;
}
@@ -1343,13 +1344,13 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (exec_len == 0)
exec_len = params->batch_obj->base.size;
- ret = engine->dispatch_execbuffer(params->request,
- exec_start, exec_len,
- params->dispatch_flags);
+ ret = engine->dispatch_execbuffer(req,
+ exec_start, exec_len,
+ params->dispatch_flags);
if (ret)
return ret;
- trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+ trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 6ac063d..69df948 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1005,7 +1005,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
int intel_execlists_submission_final(struct i915_execbuffer_params *params)
{
struct drm_i915_private *dev_priv = to_i915(params->dev);
- struct intel_ringbuffer *ringbuf = params->request->ringbuf;
+ struct drm_i915_gem_request *req = params->request;
+ struct intel_ringbuffer *ringbuf = req->ringbuf;
struct intel_engine_cs *engine = params->engine;
u64 exec_start;
int ret;
@@ -1017,13 +1018,13 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
- ret = logical_ring_invalidate_all_caches(params->request);
+ ret = logical_ring_invalidate_all_caches(req);
if (ret)
return ret;
if (engine == &dev_priv->engine[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_logical_ring_begin(params->request, 4);
+ ret = intel_logical_ring_begin(req, 4);
if (ret)
return ret;
@@ -1039,11 +1040,11 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
exec_start = params->batch_obj_vm_offset +
params->args_batch_start_offset;
- ret = engine->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+ ret = engine->emit_bb_start(req, exec_start, params->dispatch_flags);
if (ret)
return ret;
- trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
+ trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 05/34] drm/i915: Re-instate request->uniq because it is extremely useful
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (3 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 04/34] drm/i915: Cache request pointer in *_submission_final() John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 06/34] drm/i915: Start of GPU scheduler John.C.Harrison
` (36 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The seqno value cannot always be used when debugging issues via trace
points. This is because it can be reset back to start, especially
during TDR type tests. Also, when the scheduler arrives the seqno is
only valid while a given request is executing on the hardware. While
the request is simply queued waiting for submission, it's seqno value
will be zero (meaning invalid).
v4: Wrapped a long line to keep the style checker happy.
v5: Added uniq to the dispatch trace point [Svetlana Kukanova]
For: VIZ-5115
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Tomas Elf <tomas.elf@intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 5 +++++
drivers/gpu/drm/i915/i915_gem.c | 4 +++-
drivers/gpu/drm/i915/i915_trace.h | 32 ++++++++++++++++++++++----------
3 files changed, 30 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index f978539..7492ce7 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2012,6 +2012,8 @@ struct drm_i915_private {
struct intel_encoder *dig_port_map[I915_MAX_PORTS];
+ uint32_t request_uniq;
+
/*
* NOTE: This is the dri1/ums dungeon, don't add stuff here. Your patch
* will be rejected. Instead look for a better place.
@@ -2287,6 +2289,9 @@ struct drm_i915_gem_request {
*/
u32 seqno;
+ /* Unique identifier which can be used for trace points & debug */
+ uint32_t uniq;
+
/** Position in the ringbuffer of the start of the request */
u32 head;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 21fce67..a632276 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3008,7 +3008,8 @@ static void i915_gem_request_fence_value_str(struct fence *req_fence,
req = container_of(req_fence, typeof(*req), fence);
- snprintf(str, size, "%d [%d]", req->fence.seqno, req->seqno);
+ snprintf(str, size, "%d [%d:%d]", req->fence.seqno, req->uniq,
+ req->seqno);
}
static const struct fence_ops i915_gem_request_fops = {
@@ -3085,6 +3086,7 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
req->i915 = dev_priv;
req->engine = engine;
+ req->uniq = dev_priv->request_uniq++;
req->ctx = ctx;
i915_gem_context_reference(req->ctx);
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index b7c7031..59a6266 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -458,6 +458,7 @@ TRACE_EVENT(i915_gem_ring_sync_to,
__field(u32, dev)
__field(u32, sync_from)
__field(u32, sync_to)
+ __field(u32, uniq_to)
__field(u32, seqno)
),
@@ -465,13 +466,14 @@ TRACE_EVENT(i915_gem_ring_sync_to,
__entry->dev = from->dev->primary->index;
__entry->sync_from = from->id;
__entry->sync_to = to_req->engine->id;
+ __entry->uniq_to = to_req->uniq;
__entry->seqno = i915_gem_request_get_seqno(req);
),
- TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u",
+ TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u, to_uniq=%u",
__entry->dev,
__entry->sync_from, __entry->sync_to,
- __entry->seqno)
+ __entry->seqno, __entry->uniq_to)
);
TRACE_EVENT(i915_gem_ring_dispatch,
@@ -481,6 +483,7 @@ TRACE_EVENT(i915_gem_ring_dispatch,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, seqno)
__field(u32, flags)
),
@@ -490,13 +493,15 @@ TRACE_EVENT(i915_gem_ring_dispatch,
i915_gem_request_get_engine(req);
__entry->dev = engine->dev->primary->index;
__entry->ring = engine->id;
+ __entry->uniq = req->uniq;
__entry->seqno = i915_gem_request_get_seqno(req);
__entry->flags = flags;
i915_trace_irq_get(engine, req);
),
- TP_printk("dev=%u, ring=%u, seqno=%u, flags=%x",
- __entry->dev, __entry->ring, __entry->seqno, __entry->flags)
+ TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, flags=%x",
+ __entry->dev, __entry->ring, __entry->uniq,
+ __entry->seqno, __entry->flags)
);
TRACE_EVENT(i915_gem_ring_flush,
@@ -506,6 +511,7 @@ TRACE_EVENT(i915_gem_ring_flush,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, invalidate)
__field(u32, flush)
),
@@ -513,12 +519,13 @@ TRACE_EVENT(i915_gem_ring_flush,
TP_fast_assign(
__entry->dev = req->engine->dev->primary->index;
__entry->ring = req->engine->id;
+ __entry->uniq = req->uniq;
__entry->invalidate = invalidate;
__entry->flush = flush;
),
- TP_printk("dev=%u, ring=%x, invalidate=%04x, flush=%04x",
- __entry->dev, __entry->ring,
+ TP_printk("dev=%u, ring=%x, request=%u, invalidate=%04x, flush=%04x",
+ __entry->dev, __entry->ring, __entry->uniq,
__entry->invalidate, __entry->flush)
);
@@ -529,6 +536,7 @@ DECLARE_EVENT_CLASS(i915_gem_request,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, seqno)
),
@@ -537,11 +545,13 @@ DECLARE_EVENT_CLASS(i915_gem_request,
i915_gem_request_get_engine(req);
__entry->dev = engine->dev->primary->index;
__entry->ring = engine->id;
+ __entry->uniq = req ? req->uniq : 0;
__entry->seqno = i915_gem_request_get_seqno(req);
),
- TP_printk("dev=%u, ring=%u, seqno=%u",
- __entry->dev, __entry->ring, __entry->seqno)
+ TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u",
+ __entry->dev, __entry->ring, __entry->uniq,
+ __entry->seqno)
);
DEFINE_EVENT(i915_gem_request, i915_gem_request_add,
@@ -590,6 +600,7 @@ TRACE_EVENT(i915_gem_request_wait_begin,
TP_STRUCT__entry(
__field(u32, dev)
__field(u32, ring)
+ __field(u32, uniq)
__field(u32, seqno)
__field(bool, blocking)
),
@@ -605,13 +616,14 @@ TRACE_EVENT(i915_gem_request_wait_begin,
i915_gem_request_get_engine(req);
__entry->dev = engine->dev->primary->index;
__entry->ring = engine->id;
+ __entry->uniq = req ? req->uniq : 0;
__entry->seqno = i915_gem_request_get_seqno(req);
__entry->blocking =
mutex_is_locked(&engine->dev->struct_mutex);
),
- TP_printk("dev=%u, ring=%u, seqno=%u, blocking=%s",
- __entry->dev, __entry->ring,
+ TP_printk("dev=%u, ring=%u, uniq=%u, seqno=%u, blocking=%s",
+ __entry->dev, __entry->ring, __entry->uniq,
__entry->seqno, __entry->blocking ? "yes (NB)" : "no")
);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 06/34] drm/i915: Start of GPU scheduler
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (4 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 05/34] drm/i915: Re-instate request->uniq because it is extremely useful John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-06-10 16:24 ` Tvrtko Ursulin
2016-04-20 17:13 ` [PATCH v6 07/34] drm/i915: Disable hardware semaphores when GPU scheduler is enabled John.C.Harrison
` (35 subsequent siblings)
41 siblings, 1 reply; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Initial creation of scheduler source files. Note that this patch
implements most of the scheduler functionality but does not hook it in
to the driver yet. It also leaves the scheduler code in 'pass through'
mode so that even when it is hooked in, it will not actually do very
much. This allows the hooks to be added one at a time in bite size
chunks and only when the scheduler is finally enabled at the end does
anything start happening.
The general theory of operation is that when batch buffers are
submitted to the driver, the execbuffer() code packages up all the
information required to execute the batch buffer at a later time. This
package is given over to the scheduler which adds it to an internal
node list. The scheduler also scans the list of objects associated
with the batch buffer and compares them against the objects already in
use by other buffers in the node list. If matches are found then the
new batch buffer node is marked as being dependent upon the matching
node. The same is done for the context object. The scheduler also
bumps up the priority of such matching nodes on the grounds that the
more dependencies a given batch buffer has the more important it is
likely to be.
The scheduler aims to have a given (tuneable) number of batch buffers
in flight on the hardware at any given time. If fewer than this are
currently executing when a new node is queued, then the node is passed
straight through to the submit function. Otherwise it is simply added
to the queue and the driver returns back to user land.
The scheduler is notified when each batch buffer completes and updates
its internal tracking accordingly. At the end of the completion
interrupt processing, if any scheduler tracked batches were processed,
the scheduler's deferred worker thread is woken up. This can do more
involved processing such as actually removing completed nodes from the
queue and freeing up the resources associated with them (internal
memory allocations, DRM object references, context reference, etc.).
The work handler also checks the in flight count and calls the
submission code if a new slot has appeared.
When the scheduler's submit code is called, it scans the queued node
list for the highest priority node that has no unmet dependencies.
Note that the dependency calculation is complex as it must take
inter-ring dependencies and potential preemptions into account. Note
also that in the future this will be extended to include external
dependencies such as the Android Native Sync file descriptors and/or
the linux dma-buff synchronisation scheme.
If a suitable node is found then it is sent to execbuff_final() for
submission to the hardware. The in flight count is then re-checked and
a new node popped from the list if appropriate. All nodes that are not
submitted have their priority bumped. This ensures that low priority
tasks do not get starved out by busy higher priority ones - everything
will eventually get its turn to run.
Note that this patch does not implement pre-emptive scheduling. Only
basic scheduling by re-ordering batch buffer submission is currently
implemented. Pre-emption of actively executing batch buffers comes in
the next patch series.
v2: Changed priority levels to +/-1023 due to feedback from Chris
Wilson.
Removed redundant index from scheduler node.
Changed time stamps to use jiffies instead of raw monotonic. This
provides lower resolution but improved compatibility with other i915
code.
Major re-write of completion tracking code due to struct fence
conversion. The scheduler no longer has it's own private IRQ handler
but just lets the existing request code handle completion events.
Instead, the scheduler now hooks into the request notify code to be
told when a request has completed.
Reduced driver mutex locking scope. Removal of scheduler nodes no
longer grabs the mutex lock.
v3: Refactor of dependency generation to make the code more readable.
Also added in read-read optimisation support - i.e., don't treat a
shared read-only buffer as being a dependency.
Allowed the killing of queued nodes rather than only flying ones.
v4: Updated the commit message to better reflect the current state of
the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array
memory allocator function (kmalloc_array instead of kmalloc).
Corrected the format of some comments. Wrapped some lines differently
to keep the style checker happy.
Fixed a WARN_ON when killing nodes. The dependency removal code checks
that nodes being destroyed do not have any oustanding dependencies
(which would imply they should not have been executed yet). In the
case of nodes being destroyed, e.g. due to context banning, then this
might well be the case - they have not been executed and do indeed
have outstanding dependencies.
Re-instated the code to disble interrupts when not in use. The
underlying problem causing broken IRQ reference counts seems to have
been fixed now.
v5: Shuffled various functions around to remove forward declarations
as apparently these are frowned upon. Removed lots of white space as
apparently having easy to read code is also frowned upon. Split the
direct submission scheduler bypass code out into a separate function.
Squashed down the i915_scheduler.c sections of various patches into
this patch. Thus the later patches simply hook in existing code into
various parts of the driver rather than adding the code as well. Added
documentation to various functions. Re-worked the submit function in
terms of mutex locking, error handling and exit paths. Split the
delayed work handler function in half. Made use of the kernel 'clamp'
macro. [Joonas Lahtinen]
Added runtime PM calls as these must be done at the top level before
acquiring the driver mutex lock. [Chris Wilson]
Removed some obsolete debug code that had been forgotten about.
Moved more clean up code into the 'i915_gem_scheduler_clean_node()'
function rather than replicating it in mutliple places.
Used lighter weight spinlocks.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' and 'assert_scheduler_lock_held()'
helper macros. Renamed 'i915_gem_execbuff_release_batch_obj' to
'i915_gem_execbuf_release_batch_obj'. Updated to use 'to_i915()'
instead of dev_private. Converted all enum labels to uppercase.
Removed various unnecessary WARNs. Renamed 'saved_objects' to just
'objs'. Split code for counting incomplete nodes out into a separate
function. Removed even more white space. Added a destroy() function.
[review feedback from Joonas Lahtinen]
Added running totals of 'flying' and 'queued' nodes rather than
re-calculating each time as a minor CPU performance optimisation.
Removed support for out of order seqno completion. All the prep work
patch series (seqno to request conversion, late seqno assignment,
etc.) that has now been done means that the scheduler no longer
generates out of order seqno completions. Thus all the complex code
for coping with such is no longer required and can be removed.
Fixed a bug in scheduler bypass mode introduced in the clean up code
refactoring of v5. The clean up function was seeing the node in the
wrong state and thus refusing to process it.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/Makefile | 1 +
drivers/gpu/drm/i915/i915_dma.c | 3 +
drivers/gpu/drm/i915/i915_drv.h | 6 +
drivers/gpu/drm/i915/i915_gem.c | 5 +
drivers/gpu/drm/i915/i915_scheduler.c | 867 ++++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 113 +++++
6 files changed, 995 insertions(+)
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index e9cdeb5..289fa73 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -10,6 +10,7 @@ ccflags-y := -Werror
i915-y := i915_drv.o \
i915_irq.o \
i915_params.o \
+ i915_scheduler.o \
i915_suspend.o \
i915_sysfs.o \
intel_csr.o \
diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
index b377753..2ad4071 100644
--- a/drivers/gpu/drm/i915/i915_dma.c
+++ b/drivers/gpu/drm/i915/i915_dma.c
@@ -37,6 +37,7 @@
#include "i915_drv.h"
#include "i915_vgpu.h"
#include "i915_trace.h"
+#include "i915_scheduler.h"
#include <linux/pci.h>
#include <linux/console.h>
#include <linux/vt.h>
@@ -1448,6 +1449,8 @@ int i915_driver_unload(struct drm_device *dev)
intel_csr_ucode_fini(dev_priv);
+ i915_scheduler_destroy(dev_priv);
+
/* Free error state after interrupts are fully disabled. */
cancel_delayed_work_sync(&dev_priv->gpu_error.hangcheck_work);
i915_destroy_error_state(dev);
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 7492ce7..7b62e2c 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1717,6 +1717,8 @@ struct i915_execbuffer_params {
struct drm_i915_gem_request *request;
};
+struct i915_scheduler;
+
/* used in computing the new watermarks state */
struct intel_wm_config {
unsigned int num_pipes_active;
@@ -1994,6 +1996,8 @@ struct drm_i915_private {
struct i915_runtime_pm pm;
+ struct i915_scheduler *scheduler;
+
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
struct {
int (*execbuf_submit)(struct i915_execbuffer_params *params,
@@ -2335,6 +2339,8 @@ struct drm_i915_gem_request {
/** process identifier submitting this request */
struct pid *pid;
+ struct i915_scheduler_queue_entry *scheduler_qe;
+
/**
* The ELSP only accepts two elements at a time, so we queue
* context/tail pairs on a given queue (ring->execlist_queue) until the
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a632276..b7466cb 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -32,6 +32,7 @@
#include "i915_vgpu.h"
#include "i915_trace.h"
#include "intel_drv.h"
+#include "i915_scheduler.h"
#include <linux/shmem_fs.h>
#include <linux/slab.h>
#include <linux/swap.h>
@@ -5405,6 +5406,10 @@ int i915_gem_init(struct drm_device *dev)
*/
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+ ret = i915_scheduler_init(dev);
+ if (ret)
+ goto out_unlock;
+
ret = i915_gem_init_userptr(dev);
if (ret)
goto out_unlock;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
new file mode 100644
index 0000000..9d628b9
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -0,0 +1,867 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+#include "intel_drv.h"
+#include "i915_scheduler.h"
+
+#define for_each_scheduler_node(node, id) \
+ list_for_each_entry((node), &scheduler->node_queue[(id)], link)
+
+#define assert_scheduler_lock_held(scheduler) \
+ do { \
+ WARN_ONCE(!spin_is_locked(&(scheduler)->lock), "Spinlock not locked!"); \
+ } while(0)
+
+/**
+ * i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
+ * @dev: DRM device
+ */
+bool i915_scheduler_is_enabled(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+
+ return dev_priv->scheduler != NULL;
+}
+
+/**
+ * i915_scheduler_init - Initialise the scheduler.
+ * @dev: DRM device
+ * Returns zero on success or -ENOMEM if memory allocations fail.
+ */
+int i915_scheduler_init(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ int e;
+
+ if (scheduler)
+ return 0;
+
+ scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
+ if (!scheduler)
+ return -ENOMEM;
+
+ spin_lock_init(&scheduler->lock);
+
+ for (e = 0; e < I915_NUM_ENGINES; e++) {
+ INIT_LIST_HEAD(&scheduler->node_queue[e]);
+ scheduler->counts[e].flying = 0;
+ scheduler->counts[e].queued = 0;
+ }
+
+ /* Default tuning values: */
+ scheduler->priority_level_min = -1023;
+ scheduler->priority_level_max = 1023;
+ scheduler->priority_level_bump = 50;
+ scheduler->priority_level_preempt = 900;
+ scheduler->min_flying = 2;
+
+ dev_priv->scheduler = scheduler;
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_destroy - Get rid of the scheduler.
+ * @dev: DRM device
+ */
+void i915_scheduler_destroy(struct drm_i915_private *dev_priv)
+{
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ int e;
+
+ if (!scheduler)
+ return;
+
+ for (e = 0; e < I915_NUM_ENGINES; e++)
+ WARN(!list_empty(&scheduler->node_queue[e]), "Destroying with list entries on engine %d!", e);
+
+ kfree(scheduler);
+ dev_priv->scheduler = NULL;
+}
+
+/*
+ * Add a popped node back in to the queue. For example, because the engine
+ * was hung when execfinal() was called and thus the engine submission needs
+ * to be retried later.
+ */
+static void i915_scheduler_node_requeue(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *node)
+{
+ assert_scheduler_lock_held(scheduler);
+
+ WARN_ON(!I915_SQS_IS_FLYING(node));
+
+ /* Seqno will be reassigned on relaunch */
+ node->params.request->seqno = 0;
+ node->status = I915_SQS_QUEUED;
+ scheduler->counts[node->params.engine->id].flying--;
+ scheduler->counts[node->params.engine->id].queued++;
+}
+
+/*
+ * Give up on a node completely. For example, because it is causing the
+ * engine to hang or is using some resource that no longer exists.
+ */
+static void i915_scheduler_node_kill(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *node)
+{
+ assert_scheduler_lock_held(scheduler);
+
+ WARN_ON(I915_SQS_IS_COMPLETE(node));
+
+ if (I915_SQS_IS_FLYING(node))
+ scheduler->counts[node->params.engine->id].flying--;
+ else
+ scheduler->counts[node->params.engine->id].queued--;
+
+ node->status = I915_SQS_DEAD;
+}
+
+/* Mark a node as in flight on the hardware. */
+static void i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
+{
+ struct drm_i915_private *dev_priv = to_i915(node->params.dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *engine = node->params.engine;
+
+ assert_scheduler_lock_held(scheduler);
+
+ WARN_ON(node->status != I915_SQS_POPPED);
+
+ /*
+ * Add the node (which should currently be in state popped) to the
+ * front of the queue. This ensure that flying nodes are always held
+ * in hardware submission order.
+ */
+ list_add(&node->link, &scheduler->node_queue[engine->id]);
+
+ node->status = I915_SQS_FLYING;
+
+ scheduler->counts[engine->id].flying++;
+
+ if (!(scheduler->flags[engine->id] & I915_SF_INTERRUPTS_ENABLED)) {
+ bool success = true;
+
+ success = engine->irq_get(engine);
+ if (success)
+ scheduler->flags[engine->id] |= I915_SF_INTERRUPTS_ENABLED;
+ }
+}
+
+static inline uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
+ struct intel_engine_cs *engine)
+{
+ return scheduler->counts[engine->id].flying;
+}
+
+static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
+{
+ struct i915_scheduler_queue_entry *node;
+ int i;
+
+ assert_scheduler_lock_held(scheduler);
+
+ /*
+ * Ensure circular dependencies don't cause problems and that a bump
+ * by object usage only bumps each using buffer once:
+ */
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
+ for_each_scheduler_node(node, i)
+ node->bumped = false;
+ }
+}
+
+static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *target,
+ uint32_t bump)
+{
+ uint32_t new_priority;
+ int i, count;
+
+ if (target->priority >= scheduler->priority_level_max)
+ return 1;
+
+ if (target->bumped)
+ return 0;
+
+ new_priority = target->priority + bump;
+ if ((new_priority <= target->priority) ||
+ (new_priority > scheduler->priority_level_max))
+ target->priority = scheduler->priority_level_max;
+ else
+ target->priority = new_priority;
+
+ count = 1;
+ target->bumped = true;
+
+ for (i = 0; i < target->num_deps; i++) {
+ if (!target->dep_list[i])
+ continue;
+
+ if (target->dep_list[i]->bumped)
+ continue;
+
+ count += i915_scheduler_priority_bump(scheduler,
+ target->dep_list[i],
+ bump);
+ }
+
+ return count;
+}
+
+/*
+ * Nodes are considered valid dependencies if they are queued on any engine
+ * or if they are in flight on a different engine. In flight on the same
+ * engine is no longer interesting for non-premptive nodes as the engine
+ * serialises execution. For pre-empting nodes, all in flight dependencies
+ * are valid as they must not be jumped by the act of pre-empting.
+ *
+ * Anything that is neither queued nor flying is uninteresting.
+ */
+static inline bool i915_scheduler_is_dependency_valid(
+ struct i915_scheduler_queue_entry *node, uint32_t idx)
+{
+ struct i915_scheduler_queue_entry *dep;
+
+ dep = node->dep_list[idx];
+ if (!dep)
+ return false;
+
+ if (I915_SQS_IS_QUEUED(dep))
+ return true;
+
+ if (I915_SQS_IS_FLYING(dep)) {
+ if (node->params.engine != dep->params.engine)
+ return true;
+ }
+
+ return false;
+}
+
+static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *engine,
+ struct i915_scheduler_queue_entry **pop_node)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *best = NULL;
+ struct i915_scheduler_queue_entry *node;
+ int ret;
+ int i;
+ bool any_queued = false;
+ bool has_local, has_remote, only_remote = false;
+
+ assert_scheduler_lock_held(scheduler);
+
+ *pop_node = NULL;
+ ret = -ENODATA;
+
+ for_each_scheduler_node(node, engine->id) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+ any_queued = true;
+
+ has_local = false;
+ has_remote = false;
+ for (i = 0; i < node->num_deps; i++) {
+ if (!i915_scheduler_is_dependency_valid(node, i))
+ continue;
+
+ if (node->dep_list[i]->params.engine == node->params.engine)
+ has_local = true;
+ else
+ has_remote = true;
+ }
+
+ if (has_remote && !has_local)
+ only_remote = true;
+
+ if (!has_local && !has_remote) {
+ if (!best ||
+ (node->priority > best->priority))
+ best = node;
+ }
+ }
+
+ if (best) {
+ list_del(&best->link);
+
+ INIT_LIST_HEAD(&best->link);
+ best->status = I915_SQS_POPPED;
+
+ scheduler->counts[engine->id].queued--;
+
+ ret = 0;
+ } else {
+ /* Can only get here if:
+ * (a) there are no buffers in the queue
+ * (b) all queued buffers are dependent on other buffers
+ * e.g. on a buffer that is in flight on a different engine
+ */
+ if (only_remote) {
+ /* The only dependent buffers are on another engine. */
+ ret = -EAGAIN;
+ } else if (any_queued) {
+ /* It seems that something has gone horribly wrong! */
+ WARN_ONCE(true, "Broken dependency tracking on engine %d!\n",
+ (int) engine->id);
+ }
+ }
+
+ *pop_node = best;
+ return ret;
+}
+
+/*
+ * NB: The driver mutex lock must be held before calling this function. It is
+ * only really required during the actual back end submission call. However,
+ * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
+ * And releasing the one before acquiring the other leads to other code
+ * being run and interfering.
+ */
+static int i915_scheduler_submit(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ int ret, count = 0, flying;
+
+ WARN_ON(!mutex_is_locked(&engine->dev->struct_mutex));
+
+ spin_lock_irq(&scheduler->lock);
+
+ WARN_ON(scheduler->flags[engine->id] & I915_SF_SUBMITTING);
+ scheduler->flags[engine->id] |= I915_SF_SUBMITTING;
+
+ /* First time around, complain if anything unexpected occurs: */
+ ret = i915_scheduler_pop_from_queue_locked(engine, &node);
+ if (ret)
+ goto error;
+
+ do {
+ WARN_ON(node->params.engine != engine);
+ WARN_ON(node->status != I915_SQS_POPPED);
+ count++;
+
+ /*
+ * The call to pop above will have removed the node from the
+ * list. So add it back in and mark it as in flight.
+ */
+ i915_scheduler_node_fly(node);
+
+ spin_unlock_irq(&scheduler->lock);
+ ret = dev_priv->gt.execbuf_final(&node->params);
+ spin_lock_irq(&scheduler->lock);
+
+ /*
+ * Handle failed submission but first check that the
+ * watchdog/reset code has not nuked the node while we
+ * weren't looking:
+ */
+ if (ret && (node->status != I915_SQS_DEAD)) {
+ bool requeue = true;
+
+ /*
+ * Oh dear! Either the node is broken or the engine is
+ * busy. So need to kill the node or requeue it and try
+ * again later as appropriate.
+ */
+
+ switch (-ret) {
+ case ENODEV:
+ case ENOENT:
+ /* Fatal errors. Kill the node. */
+ requeue = false;
+ i915_scheduler_node_kill(scheduler, node);
+ break;
+
+ case EAGAIN:
+ case EBUSY:
+ case EIO:
+ case ENOMEM:
+ case ERESTARTSYS:
+ case EINTR:
+ /* Supposedly recoverable errors. */
+ break;
+
+ default:
+ /*
+ * Assume the error is recoverable and hope
+ * for the best.
+ */
+ MISSING_CASE(-ret);
+ break;
+ }
+
+ if (requeue) {
+ i915_scheduler_node_requeue(scheduler, node);
+ /*
+ * No point spinning if the engine is currently
+ * unavailable so just give up and come back
+ * later.
+ */
+ break;
+ }
+ }
+
+ /* Keep launching until the sky is sufficiently full. */
+ flying = i915_scheduler_count_flying(scheduler, engine);
+ if (flying >= scheduler->min_flying)
+ break;
+
+ /* Grab another node and go round again... */
+ ret = i915_scheduler_pop_from_queue_locked(engine, &node);
+ } while (ret == 0);
+
+ /* Don't complain about not being able to submit extra entries */
+ if (ret == -ENODATA)
+ ret = 0;
+
+ /*
+ * Bump the priority of everything that was not submitted to prevent
+ * starvation of low priority tasks by a spamming high priority task.
+ */
+ i915_scheduler_priority_bump_clear(scheduler);
+ for_each_scheduler_node(node, engine->id) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ i915_scheduler_priority_bump(scheduler, node,
+ scheduler->priority_level_bump);
+ }
+
+ /* On success, return the number of buffers submitted. */
+ if (ret == 0)
+ ret = count;
+
+error:
+ scheduler->flags[engine->id] &= ~I915_SF_SUBMITTING;
+ spin_unlock_irq(&scheduler->lock);
+ return ret;
+}
+
+static void i915_generate_dependencies(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *node,
+ uint32_t engine)
+{
+ struct i915_scheduler_obj_entry *this, *that;
+ struct i915_scheduler_queue_entry *test;
+ int i, j;
+ bool found;
+
+ for_each_scheduler_node(test, engine) {
+ if (I915_SQS_IS_COMPLETE(test))
+ continue;
+
+ /*
+ * Batches on the same engine for the same
+ * context must be kept in order.
+ */
+ found = (node->params.ctx == test->params.ctx) &&
+ (node->params.engine == test->params.engine);
+
+ /*
+ * Batches working on the same objects must
+ * be kept in order.
+ */
+ for (i = 0; (i < node->num_objs) && !found; i++) {
+ this = node->objs + i;
+
+ for (j = 0; j < test->num_objs; j++) {
+ that = test->objs + j;
+
+ if (this->obj != that->obj)
+ continue;
+
+ /* Only need to worry about writes */
+ if (this->read_only && that->read_only)
+ continue;
+
+ found = true;
+ break;
+ }
+ }
+
+ if (found) {
+ node->dep_list[node->num_deps] = test;
+ node->num_deps++;
+ }
+ }
+}
+
+static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_entry *qe)
+{
+ struct drm_i915_private *dev_priv = to_i915(qe->params.dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ int ret;
+
+ scheduler->flags[qe->params.engine->id] |= I915_SF_SUBMITTING;
+ ret = dev_priv->gt.execbuf_final(&qe->params);
+ scheduler->flags[qe->params.engine->id] &= ~I915_SF_SUBMITTING;
+
+ /*
+ * Don't do any clean up on failure because the caller will
+ * do it all anyway.
+ */
+ if (ret)
+ return ret;
+
+ /* Need to release any resources held by the node: */
+ qe->status = I915_SQS_COMPLETE;
+ i915_scheduler_clean_node(qe);
+
+ return 0;
+}
+
+static inline uint32_t i915_scheduler_count_incomplete(struct i915_scheduler *scheduler)
+{
+ int e, incomplete = 0;
+
+ for (e = 0; e < I915_NUM_ENGINES; e++)
+ incomplete += scheduler->counts[e].queued + scheduler->counts[e].flying;
+
+ return incomplete;
+}
+
+/**
+ * i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
+ * scheduler.
+ * @qe: The batch buffer request to be queued.
+ * The expectation is the qe passed in is a local stack variable. This
+ * function will copy its contents into a freshly allocated list node. The
+ * new node takes ownership of said contents so the original qe should simply
+ * be discarded and not cleaned up (i.e. don't free memory it points to or
+ * dereference objects it holds). The node is added to the scheduler's queue
+ * and the batch buffer will be submitted to the hardware at some future
+ * point in time (which may be immediately, before returning or may be quite
+ * a lot later).
+ */
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
+{
+ struct drm_i915_private *dev_priv = to_i915(qe->params.dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *engine = qe->params.engine;
+ struct i915_scheduler_queue_entry *node;
+ bool not_flying;
+ int i, e;
+ int incomplete;
+
+ /* Bypass the scheduler and send the buffer immediately? */
+ if (1/*!i915.enable_scheduler*/)
+ return i915_scheduler_queue_execbuffer_bypass(qe);
+
+ node = kmalloc(sizeof(*node), GFP_KERNEL);
+ if (!node)
+ return -ENOMEM;
+
+ *node = *qe;
+ INIT_LIST_HEAD(&node->link);
+ node->status = I915_SQS_QUEUED;
+ node->stamp = jiffies;
+ i915_gem_request_reference(node->params.request);
+
+ WARN_ON(node->params.request->scheduler_qe);
+ node->params.request->scheduler_qe = node;
+
+ /*
+ * Need to determine the number of incomplete entries in the list as
+ * that will be the maximum size of the dependency list.
+ *
+ * Note that the allocation must not be made with the spinlock acquired
+ * as kmalloc can sleep. However, the unlock/relock is safe because no
+ * new entries can be queued up during the unlock as the i915 driver
+ * mutex is still held. Entries could be removed from the list but that
+ * just means the dep_list will be over-allocated which is fine.
+ */
+ spin_lock_irq(&scheduler->lock);
+ incomplete = i915_scheduler_count_incomplete(scheduler);
+
+ /* Temporarily unlock to allocate memory: */
+ spin_unlock_irq(&scheduler->lock);
+ if (incomplete) {
+ node->dep_list = kmalloc_array(incomplete,
+ sizeof(*node->dep_list),
+ GFP_KERNEL);
+ if (!node->dep_list) {
+ kfree(node);
+ return -ENOMEM;
+ }
+ } else
+ node->dep_list = NULL;
+
+ spin_lock_irq(&scheduler->lock);
+ node->num_deps = 0;
+
+ if (node->dep_list) {
+ for (e = 0; e < I915_NUM_ENGINES; e++)
+ i915_generate_dependencies(scheduler, node, e);
+
+ WARN_ON(node->num_deps > incomplete);
+ }
+
+ node->priority = clamp(node->priority,
+ scheduler->priority_level_min,
+ scheduler->priority_level_max);
+
+ if ((node->priority > 0) && node->num_deps) {
+ i915_scheduler_priority_bump_clear(scheduler);
+
+ for (i = 0; i < node->num_deps; i++)
+ i915_scheduler_priority_bump(scheduler,
+ node->dep_list[i], node->priority);
+ }
+
+ list_add_tail(&node->link, &scheduler->node_queue[engine->id]);
+
+ not_flying = i915_scheduler_count_flying(scheduler, engine) <
+ scheduler->min_flying;
+
+ scheduler->counts[engine->id].queued++;
+
+ spin_unlock_irq(&scheduler->lock);
+
+ if (not_flying)
+ i915_scheduler_submit(engine);
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_notify_request - Notify the scheduler that the given
+ * request has completed on the hardware.
+ * @req: Request structure which has completed
+ * @preempt: Did it complete pre-emptively?
+ * A sequence number has popped out of the hardware and the request handling
+ * code has mapped it back to a request and will mark that request complete.
+ * It also calls this function to notify the scheduler about the completion
+ * so the scheduler's node can be updated appropriately.
+ * Returns true if the request is scheduler managed, false if not. The return
+ * value is combined for all freshly completed requests and if any were true
+ * then i915_scheduler_wakeup() is called so the scheduler can do further
+ * processing (submit more work) at the end.
+ */
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
+{
+ struct drm_i915_private *dev_priv = to_i915(req->engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node = req->scheduler_qe;
+ unsigned long flags;
+
+ if (!node)
+ return false;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+
+ WARN_ON(!I915_SQS_IS_FLYING(node));
+
+ /* Node was in flight so mark it as complete. */
+ if (req->cancelled)
+ node->status = I915_SQS_DEAD;
+ else
+ node->status = I915_SQS_COMPLETE;
+
+ scheduler->counts[req->engine->id].flying--;
+
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return true;
+}
+
+static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
+ struct i915_scheduler_queue_entry *remove)
+{
+ struct i915_scheduler_queue_entry *node;
+ int i, r;
+ int count = 0;
+
+ /*
+ * Ensure that a node is not being removed which is still dependent
+ * upon other (not completed) work. If that happens, it implies
+ * something has gone very wrong with the dependency tracking! Note
+ * that there is no need to worry if this node has been explicitly
+ * killed for some reason - it might be being killed before it got
+ * sent to the hardware.
+ */
+ if (remove->status != I915_SQS_DEAD) {
+ for (i = 0; i < remove->num_deps; i++)
+ if ((remove->dep_list[i]) &&
+ (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
+ count++;
+ WARN_ON(count);
+ }
+
+ /*
+ * Remove this node from the dependency lists of any other node which
+ * might be waiting on it.
+ */
+ for (r = 0; r < I915_NUM_ENGINES; r++) {
+ for_each_scheduler_node(node, r) {
+ for (i = 0; i < node->num_deps; i++) {
+ if (node->dep_list[i] != remove)
+ continue;
+
+ node->dep_list[i] = NULL;
+ }
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_wakeup - wake the scheduler's worker thread
+ * @dev: DRM device
+ * Called at the end of seqno interrupt processing if any request has
+ * completed that corresponds to a scheduler node.
+ */
+void i915_scheduler_wakeup(struct drm_device *dev)
+{
+ /* XXX: Need to call i915_scheduler_remove() via work handler. */
+}
+
+/**
+ * i915_scheduler_clean_node - free up any allocations/references
+ * associated with the given scheduler queue entry.
+ * @node: Queue entry structure which is complete
+ * After a give batch buffer completes on the hardware, all the information
+ * required to resubmit it is no longer required. However, the node entry
+ * itself might still be required for tracking purposes for a while longer.
+ * This function should be called as soon as the node is known to be complete
+ * so that these resources may be freed even though the node itself might
+ * hang around.
+ */
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
+{
+ if (!I915_SQS_IS_COMPLETE(node)) {
+ WARN(!node->params.request->cancelled,
+ "Cleaning active node: %d!\n", node->status);
+ return;
+ }
+
+ if (node->params.batch_obj) {
+ /*
+ * The batch buffer must be unpinned before it is unreferenced
+ * otherwise the unpin fails with a missing vma!?
+ */
+ if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
+ i915_gem_execbuf_release_batch_obj(node->params.batch_obj);
+
+ node->params.batch_obj = NULL;
+ }
+
+ /* And anything else owned by the node: */
+ if (node->params.cliprects) {
+ kfree(node->params.cliprects);
+ node->params.cliprects = NULL;
+ }
+}
+
+static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
+ struct intel_engine_cs *engine,
+ struct list_head *remove)
+{
+ struct i915_scheduler_queue_entry *node, *node_next;
+ bool do_submit;
+
+ spin_lock_irq(&scheduler->lock);
+
+ INIT_LIST_HEAD(remove);
+ list_for_each_entry_safe(node, node_next, &scheduler->node_queue[engine->id], link) {
+ if (!I915_SQS_IS_COMPLETE(node))
+ break;
+
+ list_del(&node->link);
+ list_add(&node->link, remove);
+
+ /* Strip the dependency info while the mutex is still locked */
+ i915_scheduler_remove_dependent(scheduler, node);
+
+ continue;
+ }
+
+ /*
+ * Release the interrupt reference count if there are no longer any
+ * nodes to worry about.
+ */
+ if (list_empty(&scheduler->node_queue[engine->id]) &&
+ (scheduler->flags[engine->id] & I915_SF_INTERRUPTS_ENABLED)) {
+ engine->irq_put(engine);
+ scheduler->flags[engine->id] &= ~I915_SF_INTERRUPTS_ENABLED;
+ }
+
+ /* Launch more packets now? */
+ do_submit = (scheduler->counts[engine->id].queued > 0) &&
+ (scheduler->counts[engine->id].flying < scheduler->min_flying);
+
+ spin_unlock_irq(&scheduler->lock);
+
+ return do_submit;
+}
+
+void i915_scheduler_process_work(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ bool do_submit;
+ struct list_head remove;
+
+ if (list_empty(&scheduler->node_queue[engine->id]))
+ return;
+
+ /* Remove completed nodes. */
+ do_submit = i915_scheduler_remove(scheduler, engine, &remove);
+
+ if (!do_submit && list_empty(&remove))
+ return;
+
+ /* Need to grab the pm lock outside of the mutex lock */
+ if (do_submit)
+ intel_runtime_pm_get(dev_priv);
+
+ mutex_lock(&engine->dev->struct_mutex);
+
+ if (do_submit)
+ i915_scheduler_submit(engine);
+
+ while (!list_empty(&remove)) {
+ node = list_first_entry(&remove, typeof(*node), link);
+ list_del(&node->link);
+
+ /* Free up all the DRM references */
+ i915_scheduler_clean_node(node);
+
+ /* And anything else owned by the node: */
+ node->params.request->scheduler_qe = NULL;
+ i915_gem_request_unreference(node->params.request);
+ kfree(node->dep_list);
+ kfree(node);
+ }
+
+ mutex_unlock(&engine->dev->struct_mutex);
+
+ if (do_submit)
+ intel_runtime_pm_put(dev_priv);
+}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
new file mode 100644
index 0000000..c895c4c
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -0,0 +1,113 @@
+/*
+ * Copyright (c) 2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef _I915_SCHEDULER_H_
+#define _I915_SCHEDULER_H_
+
+enum i915_scheduler_queue_status {
+ /* Limbo: */
+ I915_SQS_NONE = 0,
+ /* Not yet submitted to hardware: */
+ I915_SQS_QUEUED,
+ /* Popped from queue, ready to fly: */
+ I915_SQS_POPPED,
+ /* Sent to hardware for processing: */
+ I915_SQS_FLYING,
+ /* Finished processing on the hardware: */
+ I915_SQS_COMPLETE,
+ /* Killed by watchdog or catastrophic submission failure: */
+ I915_SQS_DEAD,
+ /* Limit value for use with arrays/loops */
+ I915_SQS_MAX
+};
+
+#define I915_SQS_IS_QUEUED(node) (((node)->status == I915_SQS_QUEUED))
+#define I915_SQS_IS_FLYING(node) (((node)->status == I915_SQS_FLYING))
+#define I915_SQS_IS_COMPLETE(node) (((node)->status == I915_SQS_COMPLETE) || \
+ ((node)->status == I915_SQS_DEAD))
+
+struct i915_scheduler_obj_entry {
+ struct drm_i915_gem_object *obj;
+ bool read_only;
+};
+
+struct i915_scheduler_queue_entry {
+ /* Any information required to submit this batch buffer to the hardware */
+ struct i915_execbuffer_params params;
+
+ /* -1023 = lowest priority, 0 = default, 1023 = highest */
+ int32_t priority;
+ bool bumped;
+
+ /* Objects referenced by this batch buffer */
+ struct i915_scheduler_obj_entry *objs;
+ int num_objs;
+
+ /* Batch buffers this one is dependent upon */
+ struct i915_scheduler_queue_entry **dep_list;
+ int num_deps;
+
+ enum i915_scheduler_queue_status status;
+ unsigned long stamp;
+
+ /* List of all scheduler queue entry nodes */
+ struct list_head link;
+};
+
+struct i915_scheduler_node_states {
+ uint32_t flying;
+ uint32_t queued;
+};
+
+struct i915_scheduler {
+ struct list_head node_queue[I915_NUM_ENGINES];
+ uint32_t flags[I915_NUM_ENGINES];
+ spinlock_t lock;
+
+ /* Node counts: */
+ struct i915_scheduler_node_states counts[I915_NUM_ENGINES];
+
+ /* Tuning parameters: */
+ int32_t priority_level_min;
+ int32_t priority_level_max;
+ int32_t priority_level_bump;
+ int32_t priority_level_preempt;
+ uint32_t min_flying;
+};
+
+/* Flag bits for i915_scheduler::flags */
+enum {
+ I915_SF_INTERRUPTS_ENABLED = (1 << 0),
+ I915_SF_SUBMITTING = (1 << 1),
+};
+
+bool i915_scheduler_is_enabled(struct drm_device *dev);
+int i915_scheduler_init(struct drm_device *dev);
+void i915_scheduler_destroy(struct drm_i915_private *dev_priv);
+void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
+int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
+bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
+void i915_scheduler_wakeup(struct drm_device *dev);
+
+#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 07/34] drm/i915: Disable hardware semaphores when GPU scheduler is enabled
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (5 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 06/34] drm/i915: Start of GPU scheduler John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 08/34] drm/i915: Force MMIO flips when scheduler enabled John.C.Harrison
` (34 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Hardware sempahores require seqno values to be continuously
incrementing. However, the scheduler's reordering of batch buffers
means that the seqno values going through the hardware could be out of
order. Thus semaphores can not be used.
On the other hand, the scheduler superceeds the need for hardware
semaphores anyway. Having one ring stall waiting for something to
complete on another ring is inefficient if that ring could be working
on some other, independent task. This is what the scheduler is meant
to do - keep the hardware as busy as possible by reordering batch
buffers to avoid dependency stalls.
v4: Downgraded a BUG_ON to WARN_ON as the latter is preferred.
v5: Squashed the i915_scheduler.c portions down into the 'start of
scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_drv.c | 9 +++++++++
drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++++
2 files changed, 13 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 29b4e79..1c5bdd0 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -34,6 +34,7 @@
#include "i915_drv.h"
#include "i915_trace.h"
#include "intel_drv.h"
+#include "i915_scheduler.h"
#include <linux/apple-gmux.h>
#include <linux/console.h>
@@ -532,6 +533,14 @@ void intel_detect_pch(struct drm_device *dev)
bool i915_semaphore_is_enabled(struct drm_device *dev)
{
+ /* Hardware semaphores are not compatible with the scheduler due to the
+ * seqno values being potentially out of order. However, semaphores are
+ * also not required as the scheduler will handle interring dependencies
+ * and try do so in a way that does not cause dead time on the hardware.
+ */
+ if (i915_scheduler_is_enabled(dev))
+ return false;
+
if (INTEL_INFO(dev)->gen < 6)
return false;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 5f209ba..e3f2237 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -33,6 +33,7 @@
#include <drm/i915_drm.h>
#include "i915_trace.h"
#include "intel_drv.h"
+#include "i915_scheduler.h"
int __intel_ring_space(int head, int tail, int size)
{
@@ -1478,6 +1479,9 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];
int ret;
+ /* Arithmetic on sequence numbers is unreliable with a scheduler. */
+ WARN_ON(i915_scheduler_is_enabled(signaller->dev));
+
/* Throughout all of the GEM code, seqno passed implies our current
* seqno is >= the last seqno executed. However for hardware the
* comparison is strictly greater than.
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 08/34] drm/i915: Force MMIO flips when scheduler enabled
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (6 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 07/34] drm/i915: Disable hardware semaphores when GPU scheduler is enabled John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 09/34] drm/i915: Added scheduler hook when closing DRM file handles John.C.Harrison
` (33 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
MMIO flips are the preferred mechanism now but more importantly, pipe
based flips cause issues for the scheduler. Specifically, submitting
work to the rings around the side of the scheduler could cause that
work to be lost if the scheduler generates a pre-emption event on that
ring.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/intel_display.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index e550e5b..9407934 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -47,6 +47,7 @@
#include <linux/dma_remapping.h>
#include <linux/reservation.h>
#include <linux/dma-buf.h>
+#include "i915_scheduler.h"
/* Primary plane formats for gen <= 3 */
static const uint32_t i8xx_primary_formats[] = {
@@ -11244,6 +11245,8 @@ static bool use_mmio_flip(struct intel_engine_cs *engine,
return true;
else if (i915.enable_execlists)
return true;
+ else if (i915_scheduler_is_enabled(engine->dev))
+ return true;
else if (obj->base.dma_buf &&
!reservation_object_test_signaled_rcu(obj->base.dma_buf->resv,
false))
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 09/34] drm/i915: Added scheduler hook when closing DRM file handles
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (7 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 08/34] drm/i915: Force MMIO flips when scheduler enabled John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 10/34] drm/i915: Added scheduler hook into i915_gem_request_notify() John.C.Harrison
` (32 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler decouples the submission of batch buffers to the driver
with submission of batch buffers to the hardware. Thus it is possible
for an application to close its DRM file handle while there is still
work outstanding. That means the scheduler needs to know about file
close events so it can remove the file pointer from such orphaned
batch buffers and not attempt to dereference it later.
v3: Updated to not wait for outstanding work to complete but merely
remove the file handle reference. The wait was getting excessively
complicated with inter-ring dependencies, pre-emption, and other such
issues.
v4: Changed some white space to keep the style checker happy.
v5: Added function documentation and removed apparently objectionable
white space. [Joonas Lahtinen]
Used lighter weight spinlocks.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' helper macro. Updated to use
'to_i915()' instead of dev_private. Removed the return value from
i915_scheduler_closefile() as it is not used for anything. [review
feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_dma.c | 2 ++
drivers/gpu/drm/i915/i915_scheduler.c | 45 +++++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
3 files changed, 48 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
index 2ad4071..75dc313 100644
--- a/drivers/gpu/drm/i915/i915_dma.c
+++ b/drivers/gpu/drm/i915/i915_dma.c
@@ -1509,6 +1509,8 @@ void i915_driver_lastclose(struct drm_device *dev)
void i915_driver_preclose(struct drm_device *dev, struct drm_file *file)
{
+ i915_scheduler_closefile(dev, file);
+
mutex_lock(&dev->struct_mutex);
i915_gem_context_close(dev, file);
i915_gem_release(dev, file);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 9d628b9..6dd9838 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -865,3 +865,48 @@ void i915_scheduler_process_work(struct intel_engine_cs *engine)
if (do_submit)
intel_runtime_pm_put(dev_priv);
}
+
+/**
+ * i915_scheduler_closefile - notify the scheduler that a DRM file handle
+ * has been closed.
+ * @dev: DRM device
+ * @file: file being closed
+ *
+ * Goes through the scheduler's queues and removes all connections to the
+ * disappearing file handle that still exist. There is an argument to say
+ * that this should also flush such outstanding work through the hardware.
+ * However, with pre-emption, TDR and other such complications doing so
+ * becomes a locking nightmare. So instead, just warn with a debug message
+ * if the application is leaking uncompleted work and make sure a null
+ * pointer dereference will not follow.
+ */
+void i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *engine;
+
+ if (!scheduler)
+ return;
+
+ spin_lock_irq(&scheduler->lock);
+
+ for_each_engine(engine, dev_priv) {
+ for_each_scheduler_node(node, engine->id) {
+ if (node->params.file != file)
+ continue;
+
+ if (!I915_SQS_IS_COMPLETE(node))
+ DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+ node->params.request->uniq,
+ node->params.request->seqno,
+ node->status,
+ engine->name);
+
+ node->params.file = NULL;
+ }
+ }
+
+ spin_unlock_irq(&scheduler->lock);
+}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index c895c4c..40398bb 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -105,6 +105,7 @@ enum {
bool i915_scheduler_is_enabled(struct drm_device *dev);
int i915_scheduler_init(struct drm_device *dev);
void i915_scheduler_destroy(struct drm_i915_private *dev_priv);
+void i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file);
void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 10/34] drm/i915: Added scheduler hook into i915_gem_request_notify()
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (8 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 09/34] drm/i915: Added scheduler hook when closing DRM file handles John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 11/34] drm/i915: Added deferred work handler for scheduler John.C.Harrison
` (31 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler needs to know when requests have completed so that it
can keep its own internal state up to date and can submit new requests
to the hardware from its queue.
v2: Updated due to changes in request handling. The operation is now
reversed from before. Rather than the scheduler being in control of
completion events, it is now the request code itself. The scheduler
merely receives a notification event. It can then optionally request
it's worker thread be woken up after all completion processing is
complete.
v4: Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
v5: Squashed the i915_scheduler.c portions down into the 'start of
scheduler' patch. [Joonas Lahtinen]
v6: Updated to newer nightly (lots of ring -> engine renaming).
Changed an '|=' to an 'if() ='. [review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index b7466cb..14dc641 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2918,6 +2918,7 @@ void i915_gem_request_notify(struct intel_engine_cs *engine, bool fence_locked)
{
struct drm_i915_gem_request *req, *req_next;
unsigned long flags;
+ bool wake_sched = false;
u32 seqno;
if (list_empty(&engine->fence_signal_list)) {
@@ -2956,6 +2957,15 @@ void i915_gem_request_notify(struct intel_engine_cs *engine, bool fence_locked)
*/
list_del_init(&req->signal_link);
+ /*
+ * NB: Must notify the scheduler before signalling
+ * the node. Otherwise the node can get retired first
+ * and call scheduler_clean() while the scheduler
+ * thinks it is still active.
+ */
+ if (i915_scheduler_notify_request(req))
+ wake_sched = true;
+
if (!req->cancelled) {
fence_signal_locked(&req->fence);
trace_i915_gem_request_complete(req);
@@ -2972,6 +2982,13 @@ void i915_gem_request_notify(struct intel_engine_cs *engine, bool fence_locked)
if (!fence_locked)
spin_unlock_irqrestore(&engine->fence_lock, flags);
+
+ /* Necessary? Or does the fence_signal() call do an implicit wakeup? */
+ wake_up_all(&engine->irq_queue);
+
+ /* Final scheduler processing after all individual updates are done. */
+ if (wake_sched)
+ i915_scheduler_wakeup(engine->dev);
}
static const char *i915_gem_request_get_driver_name(struct fence *req_fence)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 11/34] drm/i915: Added deferred work handler for scheduler
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (9 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 10/34] drm/i915: Added scheduler hook into i915_gem_request_notify() John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-06-10 16:29 ` Tvrtko Ursulin
2016-04-20 17:13 ` [PATCH v6 12/34] drm/i915: Redirect execbuffer_final() via scheduler John.C.Harrison
` (30 subsequent siblings)
41 siblings, 1 reply; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler needs to do interrupt triggered work that is too complex
to do in the interrupt handler. Thus it requires a deferred work
handler to process such tasks asynchronously.
v2: Updated to reduce mutex lock usage. The lock is now only held for
the minimum time within the remove function rather than for the whole
of the worker thread's operation.
v5: Removed objectionable white space and added some documentation.
[Joonas Lahtinen]
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added an i915_scheduler_destroy() function instead of doing explicit
clean up of scheduler internals from i915_driver_unload().
[review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++
drivers/gpu/drm/i915/i915_gem.c | 2 ++
drivers/gpu/drm/i915/i915_scheduler.c | 28 ++++++++++++++++++++++++++--
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
4 files changed, 39 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 7b62e2c..ed9d829 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1296,6 +1296,16 @@ struct i915_gem_mm {
struct delayed_work retire_work;
/**
+ * New scheme is to get an interrupt after every work packet
+ * in order to allow the low latency scheduling of pending
+ * packets. The idea behind adding new packets to a pending
+ * queue rather than directly into the hardware ring buffer
+ * is to allow high priority packets to over take low priority
+ * ones.
+ */
+ struct work_struct scheduler_work;
+
+ /**
* When we detect an idle GPU, we want to turn on
* powersaving features. So once we see that there
* are no more requests outstanding and no more
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 14dc641..50c45f3 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -5546,6 +5546,8 @@ i915_gem_load_init(struct drm_device *dev)
i915_gem_retire_work_handler);
INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
i915_gem_idle_work_handler);
+ INIT_WORK(&dev_priv->mm.scheduler_work,
+ i915_scheduler_work_handler);
init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 6dd9838..2dc5597 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -95,6 +95,8 @@ void i915_scheduler_destroy(struct drm_i915_private *dev_priv)
if (!scheduler)
return;
+ cancel_work_sync(&dev_priv->mm.scheduler_work);
+
for (e = 0; e < I915_NUM_ENGINES; e++)
WARN(!list_empty(&scheduler->node_queue[e]), "Destroying with list entries on engine %d!", e);
@@ -738,7 +740,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
*/
void i915_scheduler_wakeup(struct drm_device *dev)
{
- /* XXX: Need to call i915_scheduler_remove() via work handler. */
+ struct drm_i915_private *dev_priv = to_i915(dev);
+
+ queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
}
/**
@@ -820,7 +824,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
return do_submit;
}
-void i915_scheduler_process_work(struct intel_engine_cs *engine)
+static void i915_scheduler_process_work(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = to_i915(engine->dev);
struct i915_scheduler *scheduler = dev_priv->scheduler;
@@ -867,6 +871,26 @@ void i915_scheduler_process_work(struct intel_engine_cs *engine)
}
/**
+ * i915_scheduler_work_handler - scheduler's work handler callback.
+ * @work: Work structure
+ * A lot of the scheduler's work must be done asynchronously in response to
+ * an interrupt or other event. However, that work cannot be done at
+ * interrupt time or in the context of the event signaller (which might in
+ * fact be an interrupt). Thus a worker thread is required. This function
+ * will cause the thread to wake up and do its processing.
+ */
+void i915_scheduler_work_handler(struct work_struct *work)
+{
+ struct intel_engine_cs *engine;
+ struct drm_i915_private *dev_priv;
+
+ dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
+
+ for_each_engine(engine, dev_priv)
+ i915_scheduler_process_work(engine);
+}
+
+/**
* i915_scheduler_closefile - notify the scheduler that a DRM file handle
* has been closed.
* @dev: DRM device
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 40398bb..b8d4a343 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -110,5 +110,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
+void i915_scheduler_work_handler(struct work_struct *work);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 12/34] drm/i915: Redirect execbuffer_final() via scheduler
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (10 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 11/34] drm/i915: Added deferred work handler for scheduler John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 13/34] drm/i915: Keep the reserved space mechanism happy John.C.Harrison
` (29 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Updated the execbuffer() code to pass the packaged up batch buffer
information to the scheduler rather than calling execbuffer_final()
directly. The scheduler queue() code is currently a stub which simply
chains on to _final() immediately.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Renamed 'i915_gem_execbuff_release_batch_obj' to
'i915_gem_execbuf_release_batch_obj'. [review feedback from Joonas
Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 18 +++++++-----------
drivers/gpu/drm/i915/intel_lrc.c | 12 ++++--------
2 files changed, 11 insertions(+), 19 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 2b902e3..f7f3eb0 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -33,6 +33,7 @@
#include "intel_drv.h"
#include <linux/dma_remapping.h>
#include <linux/uaccess.h>
+#include "i915_scheduler.h"
#define __EXEC_OBJECT_HAS_PIN (1<<31)
#define __EXEC_OBJECT_HAS_FENCE (1<<30)
@@ -1227,6 +1228,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas)
{
+ struct i915_scheduler_queue_entry *qe;
struct drm_device *dev = params->dev;
struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -1271,17 +1273,11 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- ret = dev_priv->gt.execbuf_final(params);
+ qe = container_of(params, typeof(*qe), params);
+ ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
return ret;
- /*
- * Free everything that was stored in the QE structure (until the
- * scheduler arrives and does it instead):
- */
- if (params->dispatch_flags & I915_DISPATCH_SECURE)
- i915_gem_execbuf_release_batch_obj(params->batch_obj);
-
return 0;
}
@@ -1471,8 +1467,8 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
struct intel_engine_cs *engine;
struct intel_context *ctx;
struct i915_address_space *vm;
- struct i915_execbuffer_params params_master; /* XXX: will be removed later */
- struct i915_execbuffer_params *params = ¶ms_master;
+ struct i915_scheduler_queue_entry qe;
+ struct i915_execbuffer_params *params = &qe.params;
const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
u32 dispatch_flags;
int ret;
@@ -1538,7 +1534,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
else
vm = &ggtt->base;
- memset(¶ms_master, 0x00, sizeof(params_master));
+ memset(&qe, 0x00, sizeof(qe));
eb = eb_create(args);
if (eb == NULL) {
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 69df948..6bf0e3f 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -137,6 +137,7 @@
#include <drm/i915_drm.h>
#include "i915_drv.h"
#include "intel_mocs.h"
+#include "i915_scheduler.h"
#define GEN9_LR_CONTEXT_RENDER_SIZE (22 * PAGE_SIZE)
#define GEN8_LR_CONTEXT_RENDER_SIZE (20 * PAGE_SIZE)
@@ -942,6 +943,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas)
{
+ struct i915_scheduler_queue_entry *qe;
struct drm_device *dev = params->dev;
struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
@@ -984,17 +986,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
- ret = dev_priv->gt.execbuf_final(params);
+ qe = container_of(params, typeof(*qe), params);
+ ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
return ret;
- /*
- * Free everything that was stored in the QE structure (until the
- * scheduler arrives and does it instead):
- */
- if (params->dispatch_flags & I915_DISPATCH_SECURE)
- i915_gem_execbuf_release_batch_obj(params->batch_obj);
-
return 0;
}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 13/34] drm/i915: Keep the reserved space mechanism happy
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (11 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 12/34] drm/i915: Redirect execbuffer_final() via scheduler John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 14/34] drm/i915: Added tracking/locking of batch buffer objects John.C.Harrison
` (28 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Ring space is reserved when constructing a request to ensure that the
subsequent 'add_request()' call cannot fail due to waiting for space
on a busy or broken GPU. However, the scheduler jumps in to the middle
of the execbuffer process between request creation and request
submission. Thus it needs to cancel the reserved space when the
request is simply added to the scheduler's queue and not yet
submitted. Similarly, it needs to re-reserve the space when it finally
does want to send the batch buffer to the hardware.
v3: Updated to use locally cached request pointer.
v5: Updated due to changes to earlier patches in series - for runtime
PM calls and splitting bypass mode into a separate function.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 20 ++++++++++++++------
drivers/gpu/drm/i915/i915_scheduler.c | 4 ++++
drivers/gpu/drm/i915/intel_lrc.c | 13 +++++++++++--
3 files changed, 29 insertions(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index f7f3eb0..5df7ae2 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1296,18 +1296,22 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ ret = intel_ring_reserve_space(req);
+ if (ret)
+ goto error;
+
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
ret = intel_ring_invalidate_all_caches(req);
if (ret)
- return ret;
+ goto error;
/* Switch to the correct context for the batch */
ret = i915_switch_context(req);
if (ret)
- return ret;
+ goto error;
WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<engine->id),
"%s didn't clear reload\n", engine->name);
@@ -1316,7 +1320,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
params->instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(req, 4);
if (ret)
- return ret;
+ goto error;
intel_ring_emit(engine, MI_NOOP);
intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
@@ -1330,7 +1334,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (params->args_flags & I915_EXEC_GEN7_SOL_RESET) {
ret = i915_reset_gen7_sol_offsets(params->dev, req);
if (ret)
- return ret;
+ goto error;
}
exec_len = params->args_batch_len;
@@ -1344,13 +1348,17 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
exec_start, exec_len,
params->dispatch_flags);
if (ret)
- return ret;
+ goto error;
trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
- return 0;
+error:
+ if (ret)
+ intel_ring_reserved_space_cancel(req->ringbuf);
+
+ return ret;
}
/**
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 2dc5597..2fb3f52 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -519,6 +519,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
struct i915_scheduler *scheduler = dev_priv->scheduler;
int ret;
+ intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
+
scheduler->flags[qe->params.engine->id] |= I915_SF_SUBMITTING;
ret = dev_priv->gt.execbuf_final(&qe->params);
scheduler->flags[qe->params.engine->id] &= ~I915_SF_SUBMITTING;
@@ -584,6 +586,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
node->stamp = jiffies;
i915_gem_request_reference(node->params.request);
+ intel_ring_reserved_space_cancel(node->params.request->ringbuf);
+
WARN_ON(node->params.request->scheduler_qe);
node->params.request->scheduler_qe = node;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 6bf0e3f..0a4ef61 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1010,13 +1010,17 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ ret = intel_logical_ring_reserve_space(req);
+ if (ret)
+ goto err;
+
/*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
ret = logical_ring_invalidate_all_caches(req);
if (ret)
- return ret;
+ goto err;
if (engine == &dev_priv->engine[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
@@ -1038,13 +1042,18 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
ret = engine->emit_bb_start(req, exec_start, params->dispatch_flags);
if (ret)
- return ret;
+ goto err;
trace_i915_gem_ring_dispatch(req, params->dispatch_flags);
i915_gem_execbuffer_retire_commands(params);
return 0;
+
+err:
+ intel_ring_reserved_space_cancel(params->request->ringbuf);
+
+ return ret;
}
void intel_execlists_retire_requests(struct intel_engine_cs *engine)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 14/34] drm/i915: Added tracking/locking of batch buffer objects
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (12 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 13/34] drm/i915: Keep the reserved space mechanism happy John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 15/34] drm/i915: Hook scheduler node clean up into retire requests John.C.Harrison
` (27 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler needs to track interdependencies between batch buffers.
These are calculated by analysing the object lists of the buffers and
looking for commonality. The scheduler also needs to keep those
buffers locked long after the initial IOCTL call has returned to user
land.
v3: Updated to support read-read optimisation.
v5: Updated due to changes to earlier patches in series for splitting
bypass mode into a separate function and consoliding the clean up code.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Renamed 'saved_objects' to just 'objs'. [review feedback from Joonas
Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 45 ++++++++++++++++++++++++++++--
drivers/gpu/drm/i915/i915_scheduler.c | 15 ++++++++++
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 5df7ae2..e105aae 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1479,7 +1479,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
struct i915_execbuffer_params *params = &qe.params;
const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
u32 dispatch_flags;
- int ret;
+ int ret, i;
bool need_relocs;
if (!i915_gem_check_execbuffer(args))
@@ -1552,6 +1552,12 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
goto pre_mutex_err;
}
+ qe.objs = kzalloc(sizeof(*qe.objs) * args->buffer_count, GFP_KERNEL);
+ if (!qe.objs) {
+ ret = -ENOMEM;
+ goto err;
+ }
+
/* Look up object handles */
ret = eb_lookup_vmas(eb, exec, args, vm, file);
if (ret)
@@ -1676,9 +1682,32 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->args_DR1 = args->DR1;
params->args_DR4 = args->DR4;
params->batch_obj = batch_obj;
- params->ctx = ctx;
params->request = req;
+ /*
+ * Save away the list of objects used by this batch buffer for the
+ * purpose of tracking inter-buffer dependencies.
+ */
+ for (i = 0; i < args->buffer_count; i++) {
+ struct drm_i915_gem_object *obj;
+
+ /*
+ * NB: 'drm_gem_object_lookup()' increments the object's
+ * reference count and so must be matched by a
+ * 'drm_gem_object_unreference' call.
+ */
+ obj = to_intel_bo(drm_gem_object_lookup(dev, file,
+ exec[i].handle));
+ qe.objs[i].obj = obj;
+ qe.objs[i].read_only = obj->base.pending_write_domain == 0;
+
+ }
+ qe.num_objs = i;
+
+ /* Lock and save the context object as well. */
+ i915_gem_context_reference(ctx);
+ params->ctx = ctx;
+
ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
if (ret)
goto err_client;
@@ -1714,6 +1743,18 @@ err:
i915_gem_context_unreference(ctx);
eb_destroy(eb);
+ /* Need to release the objects: */
+ if (qe.objs) {
+ for (i = 0; i < qe.num_objs; i++)
+ drm_gem_object_unreference(&qe.objs[i].obj->base);
+
+ kfree(qe.objs);
+ }
+
+ /* Context too */
+ if (params->ctx)
+ i915_gem_context_unreference(params->ctx);
+
/*
* If the request was created but not successfully submitted then it
* must be freed again. If it was submitted then it is being tracked
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 2fb3f52..cdb86da 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -762,6 +762,8 @@ void i915_scheduler_wakeup(struct drm_device *dev)
*/
void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
{
+ int i;
+
if (!I915_SQS_IS_COMPLETE(node)) {
WARN(!node->params.request->cancelled,
"Cleaning active node: %d!\n", node->status);
@@ -779,6 +781,19 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
node->params.batch_obj = NULL;
}
+ /* Release the locked buffers: */
+ for (i = 0; i < node->num_objs; i++)
+ drm_gem_object_unreference(&node->objs[i].obj->base);
+ kfree(node->objs);
+ node->objs = NULL;
+ node->num_objs = 0;
+
+ /* Context too: */
+ if (node->params.ctx) {
+ i915_gem_context_unreference(node->params.ctx);
+ node->params.ctx = NULL;
+ }
+
/* And anything else owned by the node: */
if (node->params.cliprects) {
kfree(node->params.cliprects);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 15/34] drm/i915: Hook scheduler node clean up into retire requests
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (13 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 14/34] drm/i915: Added tracking/locking of batch buffer objects John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 16/34] drm/i915: Added scheduler support to __wait_request() calls John.C.Harrison
` (26 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler keeps its own lock on various DRM objects in order to
guarantee safe access long after the original execbuff IOCTL has
completed. This is especially important when pre-emption is enabled as
the batch buffer might need to be submitted to the hardware multiple
times. This patch hooks the clean up of these locks into the request
retire function. The request can only be retired after it has
completed on the hardware and thus is no longer eligible for
re-submission. Thus there is no point holding on to the locks beyond
that time.
v3: Updated to not WARN when cleaning a node that is being cancelled.
The clean will happen later so skipping it at the point of
cancellation is fine.
v5: Squashed the i915_scheduler.c portions down into the 'start of
scheduler' patch. [Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 50c45f3..cf0316d 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1483,6 +1483,9 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
fence_signal_locked(&request->fence);
}
+ if (request->scheduler_qe)
+ i915_scheduler_clean_node(request->scheduler_qe);
+
i915_gem_request_unreference(request);
}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 16/34] drm/i915: Added scheduler support to __wait_request() calls
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (14 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 15/34] drm/i915: Hook scheduler node clean up into retire requests John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 17/34] drm/i915: Added scheduler support to page fault handler John.C.Harrison
` (25 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler can cause batch buffers, and hence requests, to be
submitted to the ring out of order and asynchronously to their
submission to the driver. Thus at the point of waiting for the
completion of a given request, it is not even guaranteed that the
request has actually been sent to the hardware yet. Even it is has
been sent, it is possible that it could be pre-empted and thus
'unsent'.
This means that it is necessary to be able to submit requests to the
hardware during the wait call itself. Unfortunately, while some
callers of __wait_request() release the mutex lock first, others do
not (and apparently can not). Hence there is the ability to deadlock
as the wait stalls for submission but the asynchronous submission is
stalled for the mutex lock.
This change hooks the scheduler in to the __wait_request() code to
ensure correct behaviour. That is, flush the target batch buffer
through to the hardware and do not deadlock waiting for something that
cannot currently be submitted. Instead, the wait call must return
EAGAIN at least as far back as necessary to release the mutex lock and
allow the scheduler's asynchronous processing to get in and handle the
pre-emption operation and eventually (re-)submit the work.
v3: Removed the explicit scheduler flush from i915_wait_request().
This is no longer necessary and was causing unintended changes to the
scheduler priority level which broke a validation team test.
v4: Corrected the format of some comments to keep the style checker
happy.
v5: Added function description. [Joonas Lahtinen]
v6: Updated to newer nightly (lots of ring -> engine renaming).
Updated to use 'to_i915()' instead of dev_private. Changed extra
boolean wait_request() parameter to a flags word and consumed the
original boolean parameter too. Also, replaced the
i915_scheduler_is_request_tracked() function with
i915_scheduler_is_mutex_required() as the need for the former has gone
away and it was really being used to ask the latter question in a
convoluted manner. [review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 7 ++++-
drivers/gpu/drm/i915/i915_gem.c | 49 ++++++++++++++++++++++++++-------
drivers/gpu/drm/i915/i915_scheduler.c | 26 +++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
drivers/gpu/drm/i915/intel_display.c | 5 ++--
drivers/gpu/drm/i915/intel_ringbuffer.c | 8 ++++--
6 files changed, 81 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index ed9d829..6fa2541 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3159,9 +3159,14 @@ void __i915_add_request(struct drm_i915_gem_request *req,
__i915_add_request(req, NULL, false)
int __i915_wait_request(struct drm_i915_gem_request *req,
unsigned reset_counter,
- bool interruptible,
+ uint32_t flags,
s64 *timeout,
struct intel_rps_client *rps);
+
+/* flags used by users of __i915_wait_request */
+#define I915_WAIT_REQUEST_INTERRUPTIBLE (1 << 0)
+#define I915_WAIT_REQUEST_LOCKED (1 << 1)
+
int __must_check i915_wait_request(struct drm_i915_gem_request *req);
int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
int __must_check
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index cf0316d..8f7bf6f 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1234,7 +1234,9 @@ static int __i915_spin_request(struct drm_i915_gem_request *req, int state)
* __i915_wait_request - wait until execution of request has finished
* @req: duh!
* @reset_counter: reset sequence associated with the given request
- * @interruptible: do an interruptible wait (normally yes)
+ * @flags: flags to define the nature of wait
+ * I915_WAIT_INTERRUPTIBLE - do an interruptible wait (normally yes)
+ * I915_WAIT_LOCKED - caller is holding struct_mutex
* @timeout: in - how long to wait (NULL forever); out - how much time remaining
*
* Note: It is of utmost importance that the passed in seqno and reset_counter
@@ -1249,20 +1251,22 @@ static int __i915_spin_request(struct drm_i915_gem_request *req, int state)
*/
int __i915_wait_request(struct drm_i915_gem_request *req,
unsigned reset_counter,
- bool interruptible,
+ uint32_t flags,
s64 *timeout,
struct intel_rps_client *rps)
{
struct intel_engine_cs *engine = i915_gem_request_get_engine(req);
struct drm_device *dev = engine->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
+ bool interruptible = flags & I915_WAIT_REQUEST_INTERRUPTIBLE;
int state = interruptible ? TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE;
uint32_t seqno;
DEFINE_WAIT(wait);
unsigned long timeout_expire;
s64 before = 0; /* Only to silence a compiler warning. */
- int ret;
+ int ret = 0;
+ might_sleep();
WARN(!intel_irqs_enabled(dev_priv), "IRQs disabled");
if (i915_gem_request_completed(req))
@@ -1317,6 +1321,19 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
break;
}
+ if (flags & I915_WAIT_REQUEST_LOCKED) {
+ /*
+ * If this request is being processed by the scheduler
+ * then it is unsafe to sleep with the mutex lock held
+ * as the scheduler may require the lock in order to
+ * progress the request.
+ */
+ if (i915_scheduler_is_mutex_required(req)) {
+ ret = -EAGAIN;
+ break;
+ }
+ }
+
if (i915_gem_request_completed(req)) {
ret = 0;
break;
@@ -1521,6 +1538,7 @@ i915_wait_request(struct drm_i915_gem_request *req)
struct drm_i915_private *dev_priv;
bool interruptible;
int ret;
+ uint32_t flags;
BUG_ON(req == NULL);
@@ -1534,9 +1552,13 @@ i915_wait_request(struct drm_i915_gem_request *req)
if (ret)
return ret;
+ flags = I915_WAIT_REQUEST_LOCKED;
+ if (interruptible)
+ flags |= I915_WAIT_REQUEST_INTERRUPTIBLE;
+
ret = __i915_wait_request(req,
atomic_read(&dev_priv->gpu_error.reset_counter),
- interruptible, NULL, NULL);
+ flags, NULL, NULL);
if (ret)
return ret;
@@ -1648,7 +1670,8 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
mutex_unlock(&dev->struct_mutex);
for (i = 0; ret == 0 && i < n; i++)
- ret = __i915_wait_request(requests[i], reset_counter, true,
+ ret = __i915_wait_request(requests[i], reset_counter,
+ I915_WAIT_REQUEST_INTERRUPTIBLE,
NULL, rps);
mutex_lock(&dev->struct_mutex);
@@ -3589,7 +3612,8 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
for (i = 0; i < n; i++) {
if (ret == 0)
- ret = __i915_wait_request(req[i], reset_counter, true,
+ ret = __i915_wait_request(req[i], reset_counter,
+ I915_WAIT_REQUEST_INTERRUPTIBLE,
args->timeout_ns > 0 ? &args->timeout_ns : NULL,
to_rps_client(file));
i915_gem_request_unreference(req[i]);
@@ -3620,11 +3644,15 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (!i915_semaphore_is_enabled(obj->base.dev)) {
struct drm_i915_private *i915 = to_i915(obj->base.dev);
+ uint32_t flags;
+
+ flags = I915_WAIT_REQUEST_LOCKED;
+ if (i915->mm.interruptible)
+ flags |= I915_WAIT_REQUEST_INTERRUPTIBLE;
+
ret = __i915_wait_request(from_req,
atomic_read(&i915->gpu_error.reset_counter),
- i915->mm.interruptible,
- NULL,
- &i915->rps.semaphores);
+ flags, NULL, &i915->rps.semaphores);
if (ret)
return ret;
@@ -4610,7 +4638,8 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
if (target == NULL)
return 0;
- ret = __i915_wait_request(target, reset_counter, true, NULL, NULL);
+ ret = __i915_wait_request(target, reset_counter,
+ I915_WAIT_REQUEST_INTERRUPTIBLE, NULL, NULL);
if (ret == 0)
queue_delayed_work(dev_priv->wq, &dev_priv->mm.retire_work, 0);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index cdb86da..cf8b50a 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -910,6 +910,32 @@ void i915_scheduler_work_handler(struct work_struct *work)
}
/**
+ * i915_scheduler_is_mutex_required - query if it is safe to hold the mutex
+ * lock while waiting for the given request.
+ * @req: request to be queried
+ *
+ * Looks up the given request in the scheduler's internal queue and reports
+ * on whether the scheduler will need to acquire the driver's mutex lock in
+ * order for the that request to complete.
+ */
+bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req)
+{
+ struct drm_i915_private *dev_priv = to_i915(req->engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!scheduler)
+ return false;
+
+ if (req->scheduler_qe == NULL)
+ return false;
+
+ if (I915_SQS_IS_QUEUED(req->scheduler_qe))
+ return true;
+
+ return false;
+}
+
+/**
* i915_scheduler_closefile - notify the scheduler that a DRM file handle
* has been closed.
* @dev: DRM device
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index b8d4a343..38e860d 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -111,5 +111,6 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
void i915_scheduler_work_handler(struct work_struct *work);
+bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
#endif /* _I915_SCHEDULER_H_ */
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 9407934..f1d730a 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11371,7 +11371,7 @@ static void intel_mmio_flip_work_func(struct work_struct *work)
if (mmio_flip->req) {
WARN_ON(__i915_wait_request(mmio_flip->req,
mmio_flip->crtc->reset_counter,
- false, NULL,
+ 0, NULL,
&mmio_flip->i915->rps.mmioflips));
i915_gem_request_unreference(mmio_flip->req);
}
@@ -13439,7 +13439,8 @@ static int intel_atomic_prepare_commit(struct drm_device *dev,
continue;
ret = __i915_wait_request(intel_plane_state->wait_req,
- reset_counter, true,
+ reset_counter,
+ I915_WAIT_REQUEST_INTERRUPTIBLE,
NULL, NULL);
/* Swallow -EIO errors to allow updates during hw lockup. */
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index e3f2237..6021655 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2362,6 +2362,7 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)
int intel_engine_idle(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *req;
+ uint32_t flags;
/* Wait upon the last request to be completed */
if (list_empty(&engine->request_list))
@@ -2371,11 +2372,14 @@ int intel_engine_idle(struct intel_engine_cs *engine)
struct drm_i915_gem_request,
list);
+ flags = I915_WAIT_REQUEST_LOCKED;
+ if (to_i915(engine->dev)->mm.interruptible)
+ flags |= I915_WAIT_REQUEST_INTERRUPTIBLE;
+
/* Make sure we do not trigger any retires */
return __i915_wait_request(req,
atomic_read(&to_i915(engine->dev)->gpu_error.reset_counter),
- to_i915(engine->dev)->mm.interruptible,
- NULL, NULL);
+ flags, NULL, NULL);
}
int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 17/34] drm/i915: Added scheduler support to page fault handler
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (15 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 16/34] drm/i915: Added scheduler support to __wait_request() calls John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 18/34] drm/i915: Added scheduler flush calls to ring throttle and idle functions John.C.Harrison
` (24 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
GPU page faults can now require scheduler operation in order to
complete. For example, in order to free up sufficient memory to handle
the fault the handler must wait for a batch buffer to complete that
has not even been sent to the hardware yet. Thus EAGAIN no longer
means a GPU hang, it can occur under normal operation.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 8f7bf6f..5fb4472 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1998,10 +1998,15 @@ out:
}
case -EAGAIN:
/*
- * EAGAIN means the gpu is hung and we'll wait for the error
- * handler to reset everything when re-faulting in
+ * EAGAIN can mean the gpu is hung and we'll have to wait for
+ * the error handler to reset everything when re-faulting in
* i915_mutex_lock_interruptible.
+ *
+ * It can also indicate various other nonfatal errors for which
+ * the best response is to give other threads a chance to run,
+ * and then retry the failing operation in its entirety.
*/
+ /*FALLTHRU*/
case 0:
case -ERESTARTSYS:
case -EINTR:
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 18/34] drm/i915: Added scheduler flush calls to ring throttle and idle functions
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (16 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 17/34] drm/i915: Added scheduler support to page fault handler John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 19/34] drm/i915: Add scheduler hook to GPU reset John.C.Harrison
` (23 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When requesting that all GPU work is completed, it is now necessary to
get the scheduler involved in order to flush out work that queued and
not yet submitted.
v2: Updated to add support for flushing the scheduler queue by time
stamp rather than just doing a blanket flush.
v3: Moved submit_max_priority() to this patch from an earlier patch
is it is no longer required in the other.
v4: Corrected the format of a comment to keep the style checker happy.
Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
v5: Shuffled functions around to remove forward prototypes, removed
similarly offensive white space and added documentation. Re-worked the
mutex locking around the submit function. [Joonas Lahtinen]
Used lighter weight spinlocks.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' helper macro. Updated to use
'to_i915()' instead of dev_private. Converted all enum labels to
uppercase. [review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 22 +++++
drivers/gpu/drm/i915/i915_scheduler.c | 178 ++++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 3 +
3 files changed, 203 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 5fb4472..3ee0283 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3874,6 +3874,10 @@ int i915_gpu_idle(struct drm_device *dev)
/* Flush everything onto the inactive list. */
for_each_engine(engine, dev_priv) {
+ ret = i915_scheduler_flush(engine, true);
+ if (ret < 0)
+ return ret;
+
if (!i915.enable_execlists) {
struct drm_i915_gem_request *req;
@@ -4612,6 +4616,7 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
struct drm_i915_gem_request *request, *target = NULL;
unsigned reset_counter;
int ret;
+ struct intel_engine_cs *engine;
ret = i915_gem_wait_for_error(&dev_priv->gpu_error);
if (ret)
@@ -4621,6 +4626,23 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
if (ret)
return ret;
+ for_each_engine(engine, dev_priv) {
+ /*
+ * Flush out scheduler entries that are getting 'stale'. Note
+ * that the following recent_enough test will only check
+ * against the time at which the request was submitted to the
+ * hardware (i.e. when it left the scheduler) not the time it
+ * was submitted to the driver.
+ *
+ * Also, there is not much point worring about busy return
+ * codes from the scheduler flush call. Even if more work
+ * cannot be submitted right now for whatever reason, we
+ * still want to throttle against stale work that has already
+ * been submitted.
+ */
+ i915_scheduler_flush_stamp(engine, recent_enough, false);
+ }
+
spin_lock(&file_priv->mm.lock);
list_for_each_entry(request, &file_priv->mm.request_list, client_list) {
if (time_after_eq(request->emitted_jiffies, recent_enough))
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index cf8b50a..580b9d2 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -342,6 +342,10 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *engine,
* attempting to acquire a mutex while holding a spin lock is a Bad Idea.
* And releasing the one before acquiring the other leads to other code
* being run and interfering.
+ *
+ * Hence any caller that does not already have the mutex lock for other
+ * reasons should call i915_scheduler_submit_unlocked() instead in order to
+ * obtain the lock first.
*/
static int i915_scheduler_submit(struct intel_engine_cs *engine)
{
@@ -464,6 +468,22 @@ error:
return ret;
}
+static int i915_scheduler_submit_unlocked(struct intel_engine_cs *engine)
+{
+ struct drm_device *dev = engine->dev;
+ int ret;
+
+ ret = i915_mutex_lock_interruptible(dev);
+ if (ret)
+ return ret;
+
+ ret = i915_scheduler_submit(engine);
+
+ mutex_unlock(&dev->struct_mutex);
+
+ return ret;
+}
+
static void i915_generate_dependencies(struct i915_scheduler *scheduler,
struct i915_scheduler_queue_entry *node,
uint32_t engine)
@@ -909,6 +929,164 @@ void i915_scheduler_work_handler(struct work_struct *work)
i915_scheduler_process_work(engine);
}
+static int i915_scheduler_submit_max_priority(struct intel_engine_cs *engine,
+ bool is_locked)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ int ret, count = 0;
+ bool found;
+
+ do {
+ found = false;
+ spin_lock_irq(&scheduler->lock);
+ for_each_scheduler_node(node, engine->id) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ if (node->priority < scheduler->priority_level_max)
+ continue;
+
+ found = true;
+ break;
+ }
+ spin_unlock_irq(&scheduler->lock);
+
+ if (!found)
+ break;
+
+ if (is_locked)
+ ret = i915_scheduler_submit(engine);
+ else
+ ret = i915_scheduler_submit_unlocked(engine);
+ if (ret < 0)
+ return ret;
+
+ count += ret;
+ } while (found);
+
+ return count;
+}
+
+/**
+ * i915_scheduler_flush_stamp - force requests of a given age through the
+ * scheduler.
+ * @engine: Engine to be flushed
+ * @target: Jiffy based time stamp to flush up to
+ * @is_locked: Is the driver mutex lock held?
+ * DRM has a throttle by age of request facility. This requires waiting for
+ * outstanding work over a given age. This function helps that by forcing
+ * queued batch buffers over said age through the system.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
+ unsigned long target,
+ bool is_locked)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv;
+ struct i915_scheduler *scheduler;
+ int flush_count = 0;
+
+ if (!engine)
+ return -EINVAL;
+
+ dev_priv = to_i915(engine->dev);
+ scheduler = dev_priv->scheduler;
+
+ if (!scheduler)
+ return 0;
+
+ if (is_locked && (scheduler->flags[engine->id] & I915_SF_SUBMITTING)) {
+ /*
+ * Scheduler is busy already submitting another batch,
+ * come back later rather than going recursive...
+ */
+ return -EAGAIN;
+ }
+
+ spin_lock_irq(&scheduler->lock);
+ i915_scheduler_priority_bump_clear(scheduler);
+ for_each_scheduler_node(node, engine->id) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ if (node->stamp > target)
+ continue;
+
+ flush_count = i915_scheduler_priority_bump(scheduler,
+ node, scheduler->priority_level_max);
+ }
+ spin_unlock_irq(&scheduler->lock);
+
+ if (flush_count) {
+ DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", engine->name, flush_count);
+ flush_count = i915_scheduler_submit_max_priority(engine, is_locked);
+ }
+
+ return flush_count;
+}
+
+/**
+ * i915_scheduler_flush - force all requests through the scheduler.
+ * @engine: Engine to be flushed
+ * @is_locked: Is the driver mutex lock held?
+ * For various reasons it is sometimes necessary to the scheduler out, e.g.
+ * due to engine reset.
+ * Returns zero on success or -EAGAIN if the scheduler is busy (e.g. waiting
+ * for a pre-emption event to complete) but the mutex lock is held which
+ * would prevent the scheduler's asynchronous processing from completing.
+ */
+int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked)
+{
+ struct i915_scheduler_queue_entry *node;
+ struct drm_i915_private *dev_priv;
+ struct i915_scheduler *scheduler;
+ bool found;
+ int ret;
+ uint32_t count = 0;
+
+ if (!engine)
+ return -EINVAL;
+
+ dev_priv = to_i915(engine->dev);
+ scheduler = dev_priv->scheduler;
+
+ if (!scheduler)
+ return 0;
+
+ WARN_ON(is_locked && (scheduler->flags[engine->id] & I915_SF_SUBMITTING));
+
+ do {
+ found = false;
+ spin_lock_irq(&scheduler->lock);
+ for_each_scheduler_node(node, engine->id) {
+ if (!I915_SQS_IS_QUEUED(node))
+ continue;
+
+ found = true;
+ break;
+ }
+ spin_unlock_irq(&scheduler->lock);
+
+ if (found) {
+ if (is_locked)
+ ret = i915_scheduler_submit(engine);
+ else
+ ret = i915_scheduler_submit_unlocked(engine);
+ if (ret < 0)
+ return ret;
+
+ count += ret;
+ }
+ } while (found);
+
+ return count;
+}
+
/**
* i915_scheduler_is_mutex_required - query if it is safe to hold the mutex
* lock while waiting for the given request.
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 38e860d..92dd5df 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -111,6 +111,9 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
void i915_scheduler_work_handler(struct work_struct *work);
+int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
+int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
+ unsigned long stamp, bool is_locked);
bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 19/34] drm/i915: Add scheduler hook to GPU reset
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (17 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 18/34] drm/i915: Added scheduler flush calls to ring throttle and idle functions John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 20/34] drm/i915: Added a module parameter to allow the scheduler to be disabled John.C.Harrison
` (22 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When the watchdog resets the GPU, all interrupts get disabled despite
the reference count remaining. As the scheduler probably had
interrupts enabled during the reset (it would have been waiting for
the bad batch to complete), it must be poked to tell it that the
interrupt has been disabled.
v5: New patch in series.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Updated to use 'to_i915()' instead of dev_private. Converted all enum
labels to uppercase. [review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 2 ++
drivers/gpu/drm/i915/i915_scheduler.c | 11 +++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
3 files changed, 14 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 3ee0283..ebc7f0a 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3327,6 +3327,8 @@ static void i915_gem_reset_engine_cleanup(struct drm_i915_private *dev_priv,
}
intel_ring_init_seqno(engine, engine->last_submitted_seqno);
+
+ i915_scheduler_reset_cleanup(engine);
}
void i915_gem_reset(struct drm_device *dev)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 580b9d2..9c3150a 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -821,6 +821,17 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
}
}
+void i915_scheduler_reset_cleanup(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (scheduler->flags[engine->id] & I915_SF_INTERRUPTS_ENABLED) {
+ engine->irq_put(engine);
+ scheduler->flags[engine->id] &= ~I915_SF_INTERRUPTS_ENABLED;
+ }
+}
+
static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
struct intel_engine_cs *engine,
struct list_head *remove)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 92dd5df..4e7c0a7 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -106,6 +106,7 @@ bool i915_scheduler_is_enabled(struct drm_device *dev);
int i915_scheduler_init(struct drm_device *dev);
void i915_scheduler_destroy(struct drm_i915_private *dev_priv);
void i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file);
+void i915_scheduler_reset_cleanup(struct intel_engine_cs *engine);
void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 20/34] drm/i915: Added a module parameter to allow the scheduler to be disabled
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (18 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 19/34] drm/i915: Add scheduler hook to GPU reset John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 21/34] drm/i915: Support for 'unflushed' ring idle John.C.Harrison
` (21 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
It can be useful to be able to disable the GPU scheduler via a module
parameter for debugging purposes.
v5: Converted from a multi-feature 'overrides' mask to a single
'enable' boolean. Further features (e.g. pre-emption) will now be
separate 'enable' booleans added later. [Chris Wilson]
v6: Moved scheduler parameter declaration to correct place in
i915_params struct. [review feedback from Matt Roper]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
CC: Matt Roper <matthew.d.roper@intel.com>
---
drivers/gpu/drm/i915/i915_params.c | 4 ++++
drivers/gpu/drm/i915/i915_params.h | 1 +
drivers/gpu/drm/i915/i915_scheduler.c | 5 ++++-
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index 1285063..45288a2 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -58,6 +58,7 @@ struct i915_params i915 __read_mostly = {
.guc_log_level = -1,
.enable_dp_mst = true,
.inject_load_failure = 0,
+ .enable_scheduler = 0,
};
module_param_named(modeset, i915.modeset, int, 0400);
@@ -210,3 +211,6 @@ MODULE_PARM_DESC(enable_dp_mst,
module_param_named_unsafe(inject_load_failure, i915.inject_load_failure, uint, 0400);
MODULE_PARM_DESC(inject_load_failure,
"Force an error after a number of failure check points (0:disabled (default), N:force failure at the Nth failure check point)");
+
+module_param_named_unsafe(enable_scheduler, i915.enable_scheduler, int, 0600);
+MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable [default], 1 = enable)");
diff --git a/drivers/gpu/drm/i915/i915_params.h b/drivers/gpu/drm/i915/i915_params.h
index 02bc278..44b08b3 100644
--- a/drivers/gpu/drm/i915/i915_params.h
+++ b/drivers/gpu/drm/i915/i915_params.h
@@ -50,6 +50,7 @@ struct i915_params {
int mmio_debug;
int edp_vswing;
unsigned int inject_load_failure;
+ int enable_scheduler;
/* leave bools at the end to not create holes */
bool enable_hangcheck;
bool fastboot;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 9c3150a..13084fb 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -42,6 +42,9 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
+ if (!i915.enable_scheduler)
+ return false;
+
return dev_priv->scheduler != NULL;
}
@@ -593,7 +596,7 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
int incomplete;
/* Bypass the scheduler and send the buffer immediately? */
- if (1/*!i915.enable_scheduler*/)
+ if (!i915.enable_scheduler)
return i915_scheduler_queue_execbuffer_bypass(qe);
node = kmalloc(sizeof(*node), GFP_KERNEL);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 21/34] drm/i915: Support for 'unflushed' ring idle
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (19 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 20/34] drm/i915: Added a module parameter to allow the scheduler to be disabled John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 22/34] drm/i915: Defer seqno allocation until actual hardware submission time John.C.Harrison
` (20 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When the seqno wraps around zero, the entire GPU is forced to be idle
for some reason (possibly only to work around issues with hardware
semaphores but no-one seems too sure!). This causes a problem if the
force idle occurs at an inopportune moment such as in the middle of
submitting a batch buffer. Specifically, it would lead to recursive
submits - submitting work requires a new seqno, the new seqno requires
idling the ring, idling the ring requires submitting work, submitting
work requires a new seqno...
This change adds a 'flush' parameter to the idle function call which
specifies whether the scheduler queues should be flushed out. I.e. is
the call intended to just idle the ring as it is right now (no flush)
or is it intended to force all outstanding work out of the system
(with flush).
In the seqno wrap case, pending work is not an issue because the next
operation will be to submit it. However, in other cases, the intention
is to make sure everything that could be done has been done.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added kerneldoc for intel_engine_idle().
Wrapped boolean 'flush' parameter with an _flush() macro.
[review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem.c | 2 +-
drivers/gpu/drm/i915/intel_lrc.c | 2 +-
drivers/gpu/drm/i915/intel_ringbuffer.c | 31 +++++++++++++++++++++++++++++--
drivers/gpu/drm/i915/intel_ringbuffer.h | 4 +++-
4 files changed, 34 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index ebc7f0a..1737f13 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3896,7 +3896,7 @@ int i915_gpu_idle(struct drm_device *dev)
i915_add_request_no_flush(req);
}
- ret = intel_engine_idle(engine);
+ ret = intel_engine_idle_flush(engine);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 0a4ef61..d67b08b 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1091,7 +1091,7 @@ void intel_logical_ring_stop(struct intel_engine_cs *engine)
if (!intel_engine_initialized(engine))
return;
- ret = intel_engine_idle(engine);
+ ret = intel_engine_idle_flush(engine);
if (ret && !i915_reset_in_progress(&to_i915(engine->dev)->gpu_error))
DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
engine->name, ret);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 6021655..f5bcd24 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2359,10 +2359,37 @@ static void __wrap_ring_buffer(struct intel_ringbuffer *ringbuf)
intel_ring_update_space(ringbuf);
}
-int intel_engine_idle(struct intel_engine_cs *engine)
+/**
+ * __intel_engine_idle - Force the engine to be idle.
+ * @engine: Engine to be idled
+ * @flush: Should queued scheduler work also be flushed
+ * Waits for all outstanding requests that have been sent to the given engine
+ * to complete. Can optionally also force all unsent requests that are queued
+ * in the scheduler to be sent first.
+ * Returns zero on success otherwise a negative error code.
+ *
+ * NB: Flushing can lead to recursion if called at the wrong time. E.g. flush
+ * causes the scheduler to submit requests to the hardware, submitting
+ * requests requires allocating a new seqno, when the seqno wraps around it
+ * idles the engine, idling with flush causes the scheduler to submit requests...
+ */
+int __intel_engine_idle(struct intel_engine_cs *engine, bool flush)
{
struct drm_i915_gem_request *req;
uint32_t flags;
+ int ret;
+
+ /*
+ * NB: Must not flush the scheduler if this idle request is from
+ * within an execbuff submission (i.e. due to 'get_seqno' calling
+ * 'wrap_seqno' calling 'idle'). As that would lead to recursive
+ * flushes!
+ */
+ if (flush) {
+ ret = i915_scheduler_flush(engine, true);
+ if (ret)
+ return ret;
+ }
/* Wait upon the last request to be completed */
if (list_empty(&engine->request_list))
@@ -3202,7 +3229,7 @@ intel_stop_engine(struct intel_engine_cs *engine)
if (!intel_engine_initialized(engine))
return;
- ret = intel_engine_idle(engine);
+ ret = intel_engine_idle_flush(engine);
if (ret && !i915_reset_in_progress(&to_i915(engine->dev)->gpu_error))
DRM_ERROR("failed to quiesce %s whilst cleaning up: %d\n",
engine->name, ret);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index c75c5e1..2e7daef 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -474,7 +474,9 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
int intel_ring_space(struct intel_ringbuffer *ringbuf);
bool intel_engine_stopped(struct intel_engine_cs *engine);
-int __must_check intel_engine_idle(struct intel_engine_cs *engine);
+#define intel_engine_idle(engine) __intel_engine_idle((engine), false)
+#define intel_engine_idle_flush(engine) __intel_engine_idle((engine), true)
+int __must_check __intel_engine_idle(struct intel_engine_cs *engine, bool flush);
void intel_ring_init_seqno(struct intel_engine_cs *engine, u32 seqno);
int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 22/34] drm/i915: Defer seqno allocation until actual hardware submission time
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (20 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 21/34] drm/i915: Support for 'unflushed' ring idle John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 23/34] drm/i915: Added trace points to scheduler John.C.Harrison
` (19 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The seqno value is now only used for the final test for completion of
a request. It is no longer used to track the request through the
software stack. Thus it is no longer necessary to allocate the seqno
immediately with the request. Instead, it can be done lazily and left
until the request is actually sent to the hardware. This is particular
advantageous with a GPU scheduler as the requests can then be
re-ordered between their creation and their hardware submission
without having out of order seqnos.
v2: i915_add_request() can't fail!
Combine with 'drm/i915: Assign seqno at start of exec_final()'
Various bits of code during the execbuf code path need a seqno value
to be assigned to the request. This change makes this assignment
explicit at the start of submission_final() rather than relying on an
auto-generated seqno to have happened already. This is in preparation
for a future patch which changes seqno values to be assigned lazily
(during add_request).
v3: Updated to use locally cached request pointer.
v4: Changed some white space and comment formatting to keep the style
checker happy.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 1 +
drivers/gpu/drm/i915/i915_gem.c | 23 ++++++++++++++++++++++-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 14 ++++++++++++++
drivers/gpu/drm/i915/intel_lrc.c | 14 ++++++++++++++
4 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 6fa2541..e9aaacc 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2302,6 +2302,7 @@ struct drm_i915_gem_request {
* has finished processing this request.
*/
u32 seqno;
+ u32 reserved_seqno;
/* Unique identifier which can be used for trace points & debug */
uint32_t uniq;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 1737f13..467d7da 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2659,6 +2659,11 @@ i915_gem_get_seqno(struct drm_device *dev, u32 *seqno)
/* reserve 0 for non-seqno */
if (dev_priv->next_seqno == 0) {
+ /*
+ * Why is the full re-initialisation required? Is it only for
+ * hardware semaphores? If so, could skip it in the case where
+ * semaphores are disabled?
+ */
int ret = i915_gem_init_seqno(dev, 0);
if (ret)
return ret;
@@ -2716,6 +2721,12 @@ void __i915_add_request(struct drm_i915_gem_request *request,
WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
}
+ /* Make the request's seqno 'live': */
+ if (!request->seqno) {
+ request->seqno = request->reserved_seqno;
+ WARN_ON(request->seqno != dev_priv->last_seqno);
+ }
+
trace_i915_gem_request_add(request);
request->head = request_start;
@@ -2978,6 +2989,9 @@ void i915_gem_request_notify(struct intel_engine_cs *engine, bool fence_locked)
list_for_each_entry_safe(req, req_next, &engine->fence_signal_list, signal_link) {
if (!req->cancelled) {
+ /* How can this happen? */
+ WARN_ON(req->seqno == 0);
+
if (!i915_seqno_passed(seqno, req->seqno))
break;
}
@@ -3129,7 +3143,14 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
if (req == NULL)
return -ENOMEM;
- ret = i915_gem_get_seqno(engine->dev, &req->seqno);
+ /*
+ * Assign an identifier to track this request through the hardware
+ * but don't make it live yet. It could change in the future if this
+ * request gets overtaken. However, it still needs to be allocated
+ * in advance because the point of submission must not fail and seqno
+ * allocation can fail.
+ */
+ ret = i915_gem_get_seqno(engine->dev, &req->reserved_seqno);
if (ret)
goto err;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index e105aae..5450219 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1296,6 +1296,20 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Make sure the request's seqno is the latest and greatest: */
+ if (req->reserved_seqno != dev_priv->last_seqno) {
+ ret = i915_gem_get_seqno(engine->dev, &req->reserved_seqno);
+ if (ret)
+ return ret;
+ }
+ /*
+ * And make it live because some of the execbuff submission code
+ * requires the seqno to be available up front.
+ */
+ WARN_ON(req->seqno);
+ req->seqno = req->reserved_seqno;
+ WARN_ON(req->seqno != dev_priv->last_seqno);
+
ret = intel_ring_reserve_space(req);
if (ret)
goto error;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index d67b08b..b01571e 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1010,6 +1010,20 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Make sure the request's seqno is the latest and greatest: */
+ if (req->reserved_seqno != dev_priv->last_seqno) {
+ ret = i915_gem_get_seqno(engine->dev, &req->reserved_seqno);
+ if (ret)
+ return ret;
+ }
+ /*
+ * And make it live because some of the execbuff submission code
+ * requires the seqno to be available up front.
+ */
+ WARN_ON(req->seqno);
+ req->seqno = req->reserved_seqno;
+ WARN_ON(req->seqno != dev_priv->last_seqno);
+
ret = intel_logical_ring_reserve_space(req);
if (ret)
goto err;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 23/34] drm/i915: Added trace points to scheduler
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (21 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 22/34] drm/i915: Defer seqno allocation until actual hardware submission time John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 24/34] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
` (18 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Added trace points to the scheduler to track all the various events,
node state transitions and other interesting things that occur.
v2: Updated for new request completion tracking implementation.
v3: Updated for changes to node kill code.
v4: Wrapped some long lines to keep the style checker happy.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Dropped 'min_seqno' value from 'i915_scheduler_remove' tracepoint as
it has also been removed from the code.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +
drivers/gpu/drm/i915/i915_scheduler.c | 24 +++-
drivers/gpu/drm/i915/i915_trace.h | 193 +++++++++++++++++++++++++++++
drivers/gpu/drm/i915/intel_lrc.c | 2 +
4 files changed, 219 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 5450219..a08638a 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1273,6 +1273,8 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
+ trace_i915_gem_ring_queue(engine, params);
+
qe = container_of(params, typeof(*qe), params);
ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 13084fb..a3a7a82 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -124,6 +124,8 @@ static void i915_scheduler_node_requeue(struct i915_scheduler *scheduler,
node->status = I915_SQS_QUEUED;
scheduler->counts[node->params.engine->id].flying--;
scheduler->counts[node->params.engine->id].queued++;
+ trace_i915_scheduler_unfly(node->params.engine, node);
+ trace_i915_scheduler_node_state_change(node->params.engine, node);
}
/*
@@ -137,12 +139,14 @@ static void i915_scheduler_node_kill(struct i915_scheduler *scheduler,
WARN_ON(I915_SQS_IS_COMPLETE(node));
- if (I915_SQS_IS_FLYING(node))
+ if (I915_SQS_IS_FLYING(node)) {
scheduler->counts[node->params.engine->id].flying--;
- else
+ trace_i915_scheduler_unfly(node->params.engine, node);
+ } else
scheduler->counts[node->params.engine->id].queued--;
node->status = I915_SQS_DEAD;
+ trace_i915_scheduler_node_state_change(node->params.engine, node);
}
/* Mark a node as in flight on the hardware. */
@@ -166,6 +170,8 @@ static void i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
node->status = I915_SQS_FLYING;
scheduler->counts[engine->id].flying++;
+ trace_i915_scheduler_fly(engine, node);
+ trace_i915_scheduler_node_state_change(engine, node);
if (!(scheduler->flags[engine->id] & I915_SF_INTERRUPTS_ENABLED)) {
bool success = true;
@@ -317,6 +323,7 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *engine,
best->status = I915_SQS_POPPED;
scheduler->counts[engine->id].queued--;
+ trace_i915_scheduler_node_state_change(engine, best);
ret = 0;
} else {
@@ -335,6 +342,8 @@ static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *engine,
}
}
+ trace_i915_scheduler_pop_from_queue(engine, best);
+
*pop_node = best;
return ret;
}
@@ -542,6 +551,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
struct i915_scheduler *scheduler = dev_priv->scheduler;
int ret;
+ trace_i915_scheduler_queue(qe->params.engine, qe);
+
intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
scheduler->flags[qe->params.engine->id] |= I915_SF_SUBMITTING;
@@ -668,6 +679,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
scheduler->min_flying;
scheduler->counts[engine->id].queued++;
+ trace_i915_scheduler_queue(engine, node);
+ trace_i915_scheduler_node_state_change(engine, node);
spin_unlock_irq(&scheduler->lock);
@@ -698,6 +711,8 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
struct i915_scheduler_queue_entry *node = req->scheduler_qe;
unsigned long flags;
+ trace_i915_scheduler_landing(req);
+
if (!node)
return false;
@@ -712,6 +727,7 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
node->status = I915_SQS_COMPLETE;
scheduler->counts[req->engine->id].flying--;
+ trace_i915_scheduler_node_state_change(req->engine, node);
spin_unlock_irqrestore(&scheduler->lock, flags);
@@ -872,6 +888,8 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
do_submit = (scheduler->counts[engine->id].queued > 0) &&
(scheduler->counts[engine->id].flying < scheduler->min_flying);
+ trace_i915_scheduler_remove(engine, do_submit);
+
spin_unlock_irq(&scheduler->lock);
return do_submit;
@@ -907,6 +925,8 @@ static void i915_scheduler_process_work(struct intel_engine_cs *engine)
node = list_first_entry(&remove, typeof(*node), link);
list_del(&node->link);
+ trace_i915_scheduler_destroy(engine, node);
+
/* Free up all the DRM references */
i915_scheduler_clean_node(node);
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index 59a6266..2edaaf6 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -9,6 +9,7 @@
#include "i915_drv.h"
#include "intel_drv.h"
#include "intel_ringbuffer.h"
+#include "i915_scheduler.h"
#undef TRACE_SYSTEM
#define TRACE_SYSTEM i915
@@ -815,6 +816,198 @@ TRACE_EVENT(switch_mm,
__entry->dev, __entry->ring, __entry->to, __entry->vm)
);
+TRACE_EVENT(i915_scheduler_queue,
+ TP_PROTO(struct intel_engine_cs *engine,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(engine, node),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = engine->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("engine=%d, uniq=%d, seqno=%d",
+ __entry->engine, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_fly,
+ TP_PROTO(struct intel_engine_cs *engine,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(engine, node),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = engine->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("engine=%d, uniq=%d, seqno=%d",
+ __entry->engine, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_unfly,
+ TP_PROTO(struct intel_engine_cs *engine,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(engine, node),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = engine->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("engine=%d, uniq=%d, seqno=%d",
+ __entry->engine, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_landing,
+ TP_PROTO(struct drm_i915_gem_request *req),
+ TP_ARGS(req),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ __field(u32, status)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = req->engine->id;
+ __entry->uniq = req->uniq;
+ __entry->seqno = req->seqno;
+ __entry->status = req->scheduler_qe ?
+ req->scheduler_qe->status : ~0U;
+ ),
+
+ TP_printk("engine=%d, uniq=%d, seqno=%d, status=%d",
+ __entry->engine, __entry->uniq, __entry->seqno,
+ __entry->status)
+);
+
+TRACE_EVENT(i915_scheduler_remove,
+ TP_PROTO(struct intel_engine_cs *engine,
+ bool do_submit),
+ TP_ARGS(engine, do_submit),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(bool, do_submit)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = engine->id;
+ __entry->do_submit = do_submit;
+ ),
+
+ TP_printk("engine=%d, do_submit=%d", __entry->engine, __entry->do_submit)
+);
+
+TRACE_EVENT(i915_scheduler_destroy,
+ TP_PROTO(struct intel_engine_cs *engine,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(engine, node),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = engine->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("engine=%d, uniq=%d, seqno=%d",
+ __entry->engine, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_pop_from_queue,
+ TP_PROTO(struct intel_engine_cs *engine,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(engine, node),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = engine->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node ? node->params.request->seqno : 0;
+ ),
+
+ TP_printk("engine=%d, uniq=%d, seqno=%d",
+ __entry->engine, __entry->uniq, __entry->seqno)
+);
+
+TRACE_EVENT(i915_scheduler_node_state_change,
+ TP_PROTO(struct intel_engine_cs *engine,
+ struct i915_scheduler_queue_entry *node),
+ TP_ARGS(engine, node),
+
+ TP_STRUCT__entry(
+ __field(u32, engine)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ __field(u32, status)
+ ),
+
+ TP_fast_assign(
+ __entry->engine = engine->id;
+ __entry->uniq = node ? node->params.request->uniq : 0;
+ __entry->seqno = node->params.request->seqno;
+ __entry->status = node->status;
+ ),
+
+ TP_printk("engine=%d, uniq=%d, seqno=%d, status=%d",
+ __entry->engine, __entry->uniq, __entry->seqno,
+ __entry->status)
+);
+
+TRACE_EVENT(i915_gem_ring_queue,
+ TP_PROTO(struct intel_engine_cs *ring,
+ struct i915_execbuffer_params *params),
+ TP_ARGS(ring, params),
+
+ TP_STRUCT__entry(
+ __field(u32, ring)
+ __field(u32, uniq)
+ __field(u32, seqno)
+ ),
+
+ TP_fast_assign(
+ __entry->ring = ring->id;
+ __entry->uniq = params->request->uniq;
+ __entry->seqno = params->request->seqno;
+ ),
+
+ TP_printk("ring=%d, uniq=%d, seqno=%d", __entry->ring,
+ __entry->uniq, __entry->seqno)
+);
+
#endif /* _I915_TRACE_H_ */
/* This part must be outside protection */
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index b01571e..252fc24 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -986,6 +986,8 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
i915_gem_execbuffer_move_to_active(vmas, params->request);
+ trace_i915_gem_ring_queue(engine, params);
+
qe = container_of(params, typeof(*qe), params);
ret = i915_scheduler_queue_execbuffer(qe);
if (ret)
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 24/34] drm/i915: Added scheduler queue throttling by DRM file handle
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (22 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 23/34] drm/i915: Added trace points to scheduler John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-05-06 13:19 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 25/34] drm/i915: Added debugfs interface to scheduler tuning parameters John.C.Harrison
` (17 subsequent siblings)
41 siblings, 1 reply; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler decouples the submission of batch buffers to the driver
from their subsequent submission to the hardware. This means that an
application which is continuously submitting buffers as fast as it can
could potentialy flood the driver. To prevent this, the driver now
tracks how many buffers are in progress (queued in software or
executing in hardware) and limits this to a given (tunable) number. If
this number is exceeded then the queue to the driver will return
EAGAIN and thus prevent the scheduler's queue becoming arbitrarily
large.
v3: Added a missing decrement of the file queue counter.
v4: Updated a comment.
v5: Updated due to changes to earlier patches in series - removing
forward declarations and white space. Also added some documentation.
[Joonas Lahtinen]
v6: Updated to newer nightly (lots of ring -> engine renaming).
Replace the simple 'return to userland when full' scheme with a 'sleep
on request' scheme. The former could lead to the busy polling and
wasting lots of CPU time as user land continuously retried the execbuf
IOCTL in a tight loop. Now the driver will sleep (without holding the
mutex lock) on the oldest request outstanding for that file and then
automatically retry. This is closer to the pre-scheduler behaviour of
stalling on a full ring buffer.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 2 +
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 8 ++
drivers/gpu/drm/i915/i915_scheduler.c | 118 +++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 2 +
4 files changed, 130 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index e9aaacc..25b8fd6 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -376,6 +376,8 @@ struct drm_i915_file_private {
} rps;
unsigned int bsd_ring;
+
+ u32 scheduler_queue_length;
};
/* Used by dp and fdi links */
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index a08638a..1f8486e 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1818,6 +1818,10 @@ i915_gem_execbuffer(struct drm_device *dev, void *data,
return -EINVAL;
}
+ /* Throttle batch requests per device file */
+ if (i915_scheduler_file_queue_wait(file))
+ return -EAGAIN;
+
/* Copy in the exec list from userland */
exec_list = drm_malloc_ab(sizeof(*exec_list), args->buffer_count);
exec2_list = drm_malloc_ab(sizeof(*exec2_list), args->buffer_count);
@@ -1908,6 +1912,10 @@ i915_gem_execbuffer2(struct drm_device *dev, void *data,
return -EINVAL;
}
+ /* Throttle batch requests per device file */
+ if (i915_scheduler_file_queue_wait(file))
+ return -EAGAIN;
+
exec2_list = drm_malloc_gfp(args->buffer_count,
sizeof(*exec2_list),
GFP_TEMPORARY);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index a3a7a82..0908370 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -80,6 +80,7 @@ int i915_scheduler_init(struct drm_device *dev)
scheduler->priority_level_bump = 50;
scheduler->priority_level_preempt = 900;
scheduler->min_flying = 2;
+ scheduler->file_queue_max = 64;
dev_priv->scheduler = scheduler;
@@ -496,6 +497,28 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *engine)
return ret;
}
+/**
+ * i915_scheduler_file_queue_inc - Increment the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_inc(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
+ file_priv->scheduler_queue_length++;
+}
+
+/**
+ * i915_scheduler_file_queue_dec - Decrement the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_dec(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
+ file_priv->scheduler_queue_length--;
+}
+
static void i915_generate_dependencies(struct i915_scheduler *scheduler,
struct i915_scheduler_queue_entry *node,
uint32_t engine)
@@ -675,6 +698,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
list_add_tail(&node->link, &scheduler->node_queue[engine->id]);
+ i915_scheduler_file_queue_inc(node->params.file);
+
not_flying = i915_scheduler_count_flying(scheduler, engine) <
scheduler->min_flying;
@@ -871,6 +896,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
/* Strip the dependency info while the mutex is still locked */
i915_scheduler_remove_dependent(scheduler, node);
+ /* Likewise clean up the file pointer. */
+ if (node->params.file) {
+ i915_scheduler_file_queue_dec(node->params.file);
+ node->params.file = NULL;
+ }
+
continue;
}
@@ -963,6 +994,92 @@ void i915_scheduler_work_handler(struct work_struct *work)
i915_scheduler_process_work(engine);
}
+/**
+ * i915_scheduler_file_queue_wait - Waits for space in the per file queue.
+ * @file: File object to process.
+ * This allows throttling of applications by limiting the total number of
+ * outstanding requests to a specified level. Once that limit is reached,
+ * this call will stall waiting on the oldest outstanding request. If it can
+ * not stall for any reason it returns true to mean that the queue is full
+ * and no more requests should be accepted.
+ */
+bool i915_scheduler_file_queue_wait(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+ struct drm_i915_private *dev_priv = file_priv->dev_priv;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct drm_i915_gem_request *req;
+ struct i915_scheduler_queue_entry *node;
+ unsigned reset_counter;
+ int ret;
+ struct intel_engine_cs *engine;
+
+ if (file_priv->scheduler_queue_length < scheduler->file_queue_max)
+ return false;
+
+ do {
+ spin_lock_irq(&scheduler->lock);
+
+ /*
+ * Find the first (i.e. oldest) request for this file. In the
+ * case where an app is using multiple engines, this search
+ * might be skewed by engine. However, worst case is an app has
+ * queued ~60 requests to a high indexed engine and then one
+ * request to a low indexed engine. In such a case, the driver
+ * will wait for longer than necessary but operation will
+ * still be correct and that case is not rare enough to add
+ * jiffy based inter-engine checks.
+ */
+ req = NULL;
+ for_each_engine(engine, dev_priv) {
+ for_each_scheduler_node(node, engine->id) {
+ if (I915_SQS_IS_COMPLETE(node))
+ continue;
+
+ if (node->params.file != file)
+ continue;
+
+ req = node->params.request;
+ break;
+ }
+
+ if (req)
+ break;
+ }
+
+ if (!req) {
+ spin_unlock_irq(&scheduler->lock);
+ return false;
+ }
+
+ i915_gem_request_reference(req);
+
+ spin_unlock_irq(&scheduler->lock);
+
+ ret = i915_gem_check_wedge(&dev_priv->gpu_error, false);
+ if (ret)
+ goto err_unref;
+
+ reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
+
+ ret = __i915_wait_request(req, reset_counter,
+ I915_WAIT_REQUEST_INTERRUPTIBLE, NULL, NULL);
+ if (ret)
+ goto err_unref;
+
+ /* Make sure the request's resources actually get cleared up */
+ i915_scheduler_process_work(req->engine);
+
+ i915_gem_request_unreference(req);
+ } while(file_priv->scheduler_queue_length >= scheduler->file_queue_max);
+
+ return false;
+
+err_unref:
+ i915_gem_request_unreference(req);
+ return true;
+}
+
static int i915_scheduler_submit_max_priority(struct intel_engine_cs *engine,
bool is_locked)
{
@@ -1185,6 +1302,7 @@ void i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
node->status,
engine->name);
+ i915_scheduler_file_queue_dec(node->params.file);
node->params.file = NULL;
}
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 4e7c0a7..5c33c83 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -94,6 +94,7 @@ struct i915_scheduler {
int32_t priority_level_bump;
int32_t priority_level_preempt;
uint32_t min_flying;
+ uint32_t file_queue_max;
};
/* Flag bits for i915_scheduler::flags */
@@ -116,5 +117,6 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
unsigned long stamp, bool is_locked);
bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
+bool i915_scheduler_file_queue_wait(struct drm_file *file);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 25/34] drm/i915: Added debugfs interface to scheduler tuning parameters
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (23 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 24/34] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 26/34] drm/i915: Add early exit to execbuff_final() if insufficient ring space John.C.Harrison
` (16 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
There are various parameters within the scheduler which can be tuned
to improve performance, reduce memory footprint, etc. This change adds
support for altering these via debugfs.
v2: Updated for priorities now being signed values.
v5: Squashed priority bumping entries into this patch rather than a
separate patch all of their own.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Updated to use 'to_i915()' instead of dev_private. [review feedback
from Joonas Lahtinen]
Added an admin only check when setting the parameters to prevent rogue
user code trying to break the system with strange settings. [review
feedback from Jesse Barnes]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: Jesse Barnes <jbarnes@virtuousgeek.org>
---
drivers/gpu/drm/i915/i915_debugfs.c | 187 ++++++++++++++++++++++++++++++++++++
1 file changed, 187 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index e89781a..980bb20 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -39,6 +39,7 @@
#include "intel_ringbuffer.h"
#include <drm/i915_drm.h>
#include "i915_drv.h"
+#include "i915_scheduler.h"
enum {
ACTIVE_LIST,
@@ -1125,6 +1126,186 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_next_seqno_fops,
i915_next_seqno_get, i915_next_seqno_set,
"0x%llx\n");
+static int
+i915_scheduler_priority_min_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_min;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_min_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ scheduler->priority_level_min = (int32_t) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_min_fops,
+ i915_scheduler_priority_min_get,
+ i915_scheduler_priority_min_set,
+ "%lld\n");
+
+static int
+i915_scheduler_priority_max_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_max;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_max_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ scheduler->priority_level_max = (int32_t) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_max_fops,
+ i915_scheduler_priority_max_get,
+ i915_scheduler_priority_max_set,
+ "%lld\n");
+
+static int
+i915_scheduler_priority_bump_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_bump;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_bump_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ scheduler->priority_level_bump = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_bump_fops,
+ i915_scheduler_priority_bump_get,
+ i915_scheduler_priority_bump_set,
+ "%lld\n");
+
+static int
+i915_scheduler_priority_preempt_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->priority_level_preempt;
+ return 0;
+}
+
+static int
+i915_scheduler_priority_preempt_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ scheduler->priority_level_preempt = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_priority_preempt_fops,
+ i915_scheduler_priority_preempt_get,
+ i915_scheduler_priority_preempt_set,
+ "%lld\n");
+
+static int
+i915_scheduler_min_flying_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->min_flying;
+ return 0;
+}
+
+static int
+i915_scheduler_min_flying_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ scheduler->min_flying = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_min_flying_fops,
+ i915_scheduler_min_flying_get,
+ i915_scheduler_min_flying_set,
+ "%llu\n");
+
+static int
+i915_scheduler_file_queue_max_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = (u64) scheduler->file_queue_max;
+ return 0;
+}
+
+static int
+i915_scheduler_file_queue_max_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ scheduler->file_queue_max = (u32) val;
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
+ i915_scheduler_file_queue_max_get,
+ i915_scheduler_file_queue_max_set,
+ "%llu\n");
+
static int i915_frequency_info(struct seq_file *m, void *unused)
{
struct drm_info_node *node = m->private;
@@ -5426,6 +5607,12 @@ static const struct i915_debugfs_files {
{"i915_gem_drop_caches", &i915_drop_caches_fops},
{"i915_error_state", &i915_error_state_fops},
{"i915_next_seqno", &i915_next_seqno_fops},
+ {"i915_scheduler_priority_min", &i915_scheduler_priority_min_fops},
+ {"i915_scheduler_priority_max", &i915_scheduler_priority_max_fops},
+ {"i915_scheduler_priority_bump", &i915_scheduler_priority_bump_fops},
+ {"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
+ {"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
+ {"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 26/34] drm/i915: Add early exit to execbuff_final() if insufficient ring space
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (24 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 25/34] drm/i915: Added debugfs interface to scheduler tuning parameters John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 27/34] drm/i915: Added scheduler statistic reporting to debugfs John.C.Harrison
` (15 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
One of the major purposes of the GPU scheduler is to avoid stalling
the CPU when the GPU is busy and unable to accept more work. This
change adds support to the ring submission code to allow a ring space
check to be performed before attempting to submit a batch buffer to
the hardware. If insufficient space is available then the scheduler
can go away and come back later, letting the CPU get on with other
work, rather than stalling and waiting for the hardware to catch up.
v3: Updated to use locally cached request pointer.
v4: Line wrapped some comments differently to keep the style checker
happy. Downgraded a BUG_ON to a WARN_ON as the latter is preferred.
Removed some obsolete, commented out code.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Updated to use 'to_i915()' instead of dev_private. [review feedback
from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 41 +++++++++++++++++------
drivers/gpu/drm/i915/intel_lrc.c | 54 +++++++++++++++++++++++++++---
drivers/gpu/drm/i915/intel_ringbuffer.c | 26 ++++++++++++++
drivers/gpu/drm/i915/intel_ringbuffer.h | 1 +
4 files changed, 107 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 1f8486e..bacee5c 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1148,25 +1148,19 @@ i915_reset_gen7_sol_offsets(struct drm_device *dev,
{
struct intel_engine_cs *engine = req->engine;
struct drm_i915_private *dev_priv = dev->dev_private;
- int ret, i;
+ int i;
if (!IS_GEN7(dev) || engine != &dev_priv->engine[RCS]) {
DRM_DEBUG("sol reset is gen7/rcs only\n");
return -EINVAL;
}
- ret = intel_ring_begin(req, 4 * 3);
- if (ret)
- return ret;
-
for (i = 0; i < 4; i++) {
intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
intel_ring_emit_reg(engine, GEN7_SO_WRITE_OFFSET(i));
intel_ring_emit(engine, 0);
}
- intel_ring_advance(engine);
-
return 0;
}
@@ -1294,6 +1288,7 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
struct intel_engine_cs *engine = params->engine;
u64 exec_start, exec_len;
int ret;
+ uint32_t min_space;
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
@@ -1317,6 +1312,34 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
goto error;
/*
+ * It would be a bad idea to run out of space while writing commands
+ * to the ring. One of the major aims of the scheduler is to not
+ * stall at any point for any reason. However, doing an early exit
+ * half way through submission could result in a partial sequence
+ * being written which would leave the engine in an unknown state.
+ * Therefore, check in advance that there will be enough space for
+ * the entire submission whether emitted by the code below OR by any
+ * other functions that may be executed before the end of final().
+ *
+ * NB: This test deliberately overestimates, because that's easier
+ * than tracing every potential path that could be taken!
+ *
+ * Current measurements suggest that we may need to emit up to 186
+ * dwords, so this is rounded up to 256 here. Then double that to get
+ * the free space requirement, because the block is not allowed to
+ * span the transition from the end to the beginning of the ring.
+ */
+#define I915_BATCH_EXEC_MAX_LEN 256 /* max dwords emitted here */
+ min_space = I915_BATCH_EXEC_MAX_LEN * 2 * sizeof(uint32_t);
+ ret = intel_ring_test_space(req->ringbuf, min_space);
+ if (ret)
+ goto error;
+
+ ret = intel_ring_begin(req, I915_BATCH_EXEC_MAX_LEN);
+ if (ret)
+ goto error;
+
+ /*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
@@ -1334,10 +1357,6 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
if (engine == &dev_priv->engine[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_ring_begin(req, 4);
- if (ret)
- goto error;
-
intel_ring_emit(engine, MI_NOOP);
intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
intel_ring_emit_reg(engine, INSTPM);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 252fc24..b9258ee 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -231,6 +231,27 @@ enum {
static int intel_lr_context_pin(struct intel_context *ctx,
struct intel_engine_cs *engine);
+/*
+ * Test to see if the ring has sufficient space to submit a given piece
+ * of work without causing a stall
+ */
+static int logical_ring_test_space(struct intel_ringbuffer *ringbuf,
+ int min_space)
+{
+ if (ringbuf->space < min_space) {
+ /* Need to update the actual ring space. Otherwise, the system
+ * hangs forever testing a software copy of the space value that
+ * never changes!
+ */
+ intel_ring_update_space(ringbuf);
+
+ if (ringbuf->space < min_space)
+ return -EAGAIN;
+ }
+
+ return 0;
+}
+
/**
* intel_sanitize_enable_execlists() - sanitize i915.enable_execlists
* @dev: DRM device.
@@ -1008,6 +1029,7 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
struct intel_engine_cs *engine = params->engine;
u64 exec_start;
int ret;
+ uint32_t min_space;
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
@@ -1031,6 +1053,34 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
goto err;
/*
+ * It would be a bad idea to run out of space while writing commands
+ * to the ring. One of the major aims of the scheduler is to not
+ * stall at any point for any reason. However, doing an early exit
+ * half way through submission could result in a partial sequence
+ * being written which would leave the engine in an unknown state.
+ * Therefore, check in advance that there will be enough space for
+ * the entire submission whether emitted by the code below OR by any
+ * other functions that may be executed before the end of final().
+ *
+ * NB: This test deliberately overestimates, because that's easier
+ * than tracing every potential path that could be taken!
+ *
+ * Current measurements suggest that we may need to emit up to 186
+ * dwords, so this is rounded up to 256 here. Then double that to get
+ * the free space requirement, because the block is not allowed to
+ * span the transition from the end to the beginning of the ring.
+ */
+#define I915_BATCH_EXEC_MAX_LEN 256 /* max dwords emitted here */
+ min_space = I915_BATCH_EXEC_MAX_LEN * 2 * sizeof(uint32_t);
+ ret = logical_ring_test_space(ringbuf, min_space);
+ if (ret)
+ goto err;
+
+ ret = intel_logical_ring_begin(req, I915_BATCH_EXEC_MAX_LEN);
+ if (ret)
+ goto err;
+
+ /*
* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
@@ -1040,10 +1090,6 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
if (engine == &dev_priv->engine[RCS] &&
params->instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_logical_ring_begin(req, 4);
- if (ret)
- return ret;
-
intel_logical_ring_emit(ringbuf, MI_NOOP);
intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
intel_logical_ring_emit_reg(ringbuf, INSTPM);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index f5bcd24..6ea27c6 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2571,6 +2571,32 @@ int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
return 0;
}
+/*
+ * Test to see if the ring has sufficient space to submit a given piece
+ * of work without causing a stall
+ */
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space)
+{
+ struct drm_i915_private *dev_priv = to_i915(ringbuf->engine->dev);
+
+ /* There is a separate LRC version of this code. */
+ WARN_ON(i915.enable_execlists);
+
+ if (ringbuf->space < min_space) {
+ /* Need to update the actual ring space. Otherwise, the system
+ * hangs forever testing a software copy of the space value that
+ * never changes!
+ */
+ ringbuf->head = I915_READ_HEAD(ringbuf->engine);
+ ringbuf->space = intel_ring_space(ringbuf);
+
+ if (ringbuf->space < min_space)
+ return -EAGAIN;
+ }
+
+ return 0;
+}
+
void intel_ring_init_seqno(struct intel_engine_cs *engine, u32 seqno)
{
struct drm_i915_private *dev_priv = to_i915(engine->dev);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 2e7daef..067d635 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -450,6 +450,7 @@ void intel_cleanup_engine(struct intel_engine_cs *engine);
int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
+int intel_ring_test_space(struct intel_ringbuffer *ringbuf, int min_space);
int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
static inline void intel_ring_emit(struct intel_engine_cs *engine,
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 27/34] drm/i915: Added scheduler statistic reporting to debugfs
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (25 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 26/34] drm/i915: Add early exit to execbuff_final() if insufficient ring space John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-05-06 13:21 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 28/34] drm/i915: Add scheduler support functions for TDR John.C.Harrison
` (14 subsequent siblings)
41 siblings, 1 reply; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
It is useful for know what the scheduler is doing for both debugging
and performance analysis purposes. This change adds a bunch of
counters and such that keep track of various scheduler operations
(batches submitted, completed, flush requests, etc.). The data can
then be read in userland via the debugfs mechanism.
v2: Updated to match changes to scheduler implementation.
v3: Updated for changes to kill code and flush code.
v4: Removed the fence/sync code as that will be part of a separate
patch series. Wrapped a long line to keep the style checker happy.
v5: Updated to remove forward declarations and white space. Added
documentation. [Joonas Lahtinen]
Used lighter weight spinlocks.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' and 'assert_scheduler_lock_held()'
helper macros. Updated to use 'to_i915()' instead of dev_private.
Converted all enum labels to uppercase. Removed even more white space.
Moved the enum to string conversion function to debugfs.c rather than
scheduler.c [review feedback from Joonas Lahtinen]
Added running totals of 'flying' and 'queued' nodes rather than
re-calculating each time as a minor CPU performance optimisation.
Added stats to the new file queue wait implementation.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 111 +++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 3 +
drivers/gpu/drm/i915/i915_scheduler.c | 85 +++++++++++++++++++++-
drivers/gpu/drm/i915/i915_scheduler.h | 35 +++++++++
4 files changed, 231 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 980bb20..1d04cde 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -3609,6 +3609,116 @@ static int i915_drrs_status(struct seq_file *m, void *unused)
return 0;
}
+static const char *i915_scheduler_queue_status_str(
+ enum i915_scheduler_queue_status status)
+{
+ static char str[50];
+
+ switch (status) {
+ case I915_SQS_NONE:
+ return "None";
+
+ case I915_SQS_QUEUED:
+ return "Queued";
+
+ case I915_SQS_POPPED:
+ return "Popped";
+
+ case I915_SQS_FLYING:
+ return "Flying";
+
+ case I915_SQS_COMPLETE:
+ return "Complete";
+
+ case I915_SQS_DEAD:
+ return "Dead";
+
+ case I915_SQS_MAX:
+ return "Invalid";
+
+ default:
+ break;
+ }
+
+ sprintf(str, "[Unknown_%d!]", status);
+ return str;
+}
+
+static int i915_scheduler_info(struct seq_file *m, void *unused)
+{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_device *dev = node->minor->dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_stats *stats = scheduler->stats;
+ struct i915_scheduler_stats_nodes node_stats[I915_NUM_ENGINES];
+ struct intel_engine_cs *engine;
+ char str[50 * (I915_NUM_ENGINES + 1)], name[50], *ptr;
+ int ret, i, e;
+
+ ret = mutex_lock_interruptible(&dev->mode_config.mutex);
+ if (ret)
+ return ret;
+
+#define PRINT_VAR(name, fmt, var) \
+ do { \
+ sprintf(str, "%-22s", name); \
+ ptr = str + strlen(str); \
+ for_each_engine_id(engine, dev_priv, e) { \
+ sprintf(ptr, " %10" fmt, var); \
+ ptr += strlen(ptr); \
+ } \
+ seq_printf(m, "%s\n", str); \
+ } while (0)
+
+ PRINT_VAR("Engine name:", "s", dev_priv->engine[e].name);
+ PRINT_VAR(" Engine seqno", "d", engine->get_seqno(engine));
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Batch submissions:\n");
+ PRINT_VAR(" Queued", "u", stats[e].queued);
+ PRINT_VAR(" Submitted", "u", stats[e].submitted);
+ PRINT_VAR(" Completed", "u", stats[e].completed);
+ PRINT_VAR(" Expired", "u", stats[e].expired);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Flush counts:\n");
+ PRINT_VAR(" By object", "u", stats[e].flush_obj);
+ PRINT_VAR(" By request", "u", stats[e].flush_req);
+ PRINT_VAR(" By stamp", "u", stats[e].flush_stamp);
+ PRINT_VAR(" Blanket", "u", stats[e].flush_all);
+ PRINT_VAR(" Entries bumped", "u", stats[e].flush_bump);
+ PRINT_VAR(" Entries submitted", "u", stats[e].flush_submit);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Miscellaneous:\n");
+ PRINT_VAR(" ExecEarly retry", "u", stats[e].exec_early);
+ PRINT_VAR(" ExecFinal requeue", "u", stats[e].exec_again);
+ PRINT_VAR(" ExecFinal killed", "u", stats[e].exec_dead);
+ PRINT_VAR(" Hung flying", "u", stats[e].kill_flying);
+ PRINT_VAR(" Hung queued", "u", stats[e].kill_queued);
+ PRINT_VAR(" File queue wait", "u", stats[e].file_wait);
+ PRINT_VAR(" File queue stall", "u", stats[e].file_stall);
+ PRINT_VAR(" File queue lost", "u", stats[e].file_lost);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Queue contents:\n");
+ for_each_engine(engine, dev_priv)
+ i915_scheduler_query_stats(engine, node_stats + engine->id);
+
+ for (i = 0; i < (I915_SQS_MAX + 1); i++) {
+ sprintf(name, " %s", i915_scheduler_queue_status_str(i));
+ PRINT_VAR(name, "d", node_stats[e].counts[i]);
+ }
+ seq_putc(m, '\n');
+
+#undef PRINT_VAR
+
+ mutex_unlock(&dev->mode_config.mutex);
+
+ return 0;
+}
+
struct pipe_crc_info {
const char *name;
struct drm_device *dev;
@@ -5585,6 +5695,7 @@ static const struct drm_info_list i915_debugfs_list[] = {
{"i915_semaphore_status", i915_semaphore_status, 0},
{"i915_shared_dplls_info", i915_shared_dplls_info, 0},
{"i915_dp_mst_info", i915_dp_mst_info, 0},
+ {"i915_scheduler_info", i915_scheduler_info, 0},
{"i915_wa_registers", i915_wa_registers, 0},
{"i915_ddb_info", i915_ddb_info, 0},
{"i915_sseu_status", i915_sseu_status, 0},
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index bacee5c..823283d 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1804,6 +1804,9 @@ pre_mutex_err:
/* intel_gpu_busy should also get a ref, so it will free when the device
* is really idle. */
intel_runtime_pm_put(dev_priv);
+
+ dev_priv->scheduler->stats[engine->id].exec_early++;
+
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 0908370..74b33f9 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -142,9 +142,12 @@ static void i915_scheduler_node_kill(struct i915_scheduler *scheduler,
if (I915_SQS_IS_FLYING(node)) {
scheduler->counts[node->params.engine->id].flying--;
+ scheduler->stats[node->params.engine->id].kill_flying++;
trace_i915_scheduler_unfly(node->params.engine, node);
- } else
+ } else {
scheduler->counts[node->params.engine->id].queued--;
+ scheduler->stats[node->params.engine->id].kill_queued++;
+ }
node->status = I915_SQS_DEAD;
trace_i915_scheduler_node_state_change(node->params.engine, node);
@@ -390,6 +393,8 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
*/
i915_scheduler_node_fly(node);
+ scheduler->stats[engine->id].submitted++;
+
spin_unlock_irq(&scheduler->lock);
ret = dev_priv->gt.execbuf_final(&node->params);
spin_lock_irq(&scheduler->lock);
@@ -413,6 +418,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
case ENOENT:
/* Fatal errors. Kill the node. */
requeue = false;
+ scheduler->stats[engine->id].exec_dead++;
i915_scheduler_node_kill(scheduler, node);
break;
@@ -423,6 +429,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
case ERESTARTSYS:
case EINTR:
/* Supposedly recoverable errors. */
+ scheduler->stats[engine->id].exec_again++;
break;
default:
@@ -431,6 +438,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
* for the best.
*/
MISSING_CASE(-ret);
+ scheduler->stats[engine->id].exec_again++;
break;
}
@@ -574,12 +582,15 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
struct i915_scheduler *scheduler = dev_priv->scheduler;
int ret;
+ scheduler->stats[qe->params.engine->id].queued++;
+
trace_i915_scheduler_queue(qe->params.engine, qe);
intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
scheduler->flags[qe->params.engine->id] |= I915_SF_SUBMITTING;
ret = dev_priv->gt.execbuf_final(&qe->params);
+ scheduler->stats[qe->params.engine->id].submitted++;
scheduler->flags[qe->params.engine->id] &= ~I915_SF_SUBMITTING;
/*
@@ -593,6 +604,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
qe->status = I915_SQS_COMPLETE;
i915_scheduler_clean_node(qe);
+ scheduler->stats[qe->params.engine->id].expired++;
+
return 0;
}
@@ -704,6 +717,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
scheduler->min_flying;
scheduler->counts[engine->id].queued++;
+ scheduler->stats[engine->id].queued++;
+
trace_i915_scheduler_queue(engine, node);
trace_i915_scheduler_node_state_change(engine, node);
@@ -746,10 +761,13 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
WARN_ON(!I915_SQS_IS_FLYING(node));
/* Node was in flight so mark it as complete. */
- if (req->cancelled)
+ if (req->cancelled) {
node->status = I915_SQS_DEAD;
- else
+ scheduler->stats[req->engine->id].kill_flying++;
+ } else {
node->status = I915_SQS_COMPLETE;
+ scheduler->stats[req->engine->id].completed++;
+ }
scheduler->counts[req->engine->id].flying--;
trace_i915_scheduler_node_state_change(req->engine, node);
@@ -892,6 +910,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
list_del(&node->link);
list_add(&node->link, remove);
+ scheduler->stats[engine->id].expired++;
/* Strip the dependency info while the mutex is still locked */
i915_scheduler_remove_dependent(scheduler, node);
@@ -1048,12 +1067,14 @@ bool i915_scheduler_file_queue_wait(struct drm_file *file)
}
if (!req) {
+ scheduler->stats[engine->id].file_lost++;
spin_unlock_irq(&scheduler->lock);
return false;
}
i915_gem_request_reference(req);
+ scheduler->stats[engine->id].file_wait++;
spin_unlock_irq(&scheduler->lock);
ret = i915_gem_check_wedge(&dev_priv->gpu_error, false);
@@ -1077,9 +1098,60 @@ bool i915_scheduler_file_queue_wait(struct drm_file *file)
err_unref:
i915_gem_request_unreference(req);
+
+ spin_lock_irq(&scheduler->lock);
+ scheduler->stats[engine->id].file_wait--;
+ scheduler->stats[engine->id].file_stall++;
+ spin_unlock_irq(&scheduler->lock);
+
return true;
}
+/**
+ * i915_scheduler_query_stats - return various scheduler statistics
+ * @engine: Engine to report on
+ * @stats: Stats structure to be filled in
+ * For various reasons (debugging, performance analysis, curiosity) it is
+ * useful to see statistics about what the scheduler is doing. This function
+ * returns the stats that have been gathered in a data structure. The
+ * expectation is that this will be returned to the user via debugfs.
+ */
+int i915_scheduler_query_stats(struct intel_engine_cs *engine,
+ struct i915_scheduler_stats_nodes *stats)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+
+ memset(stats, 0x00, sizeof(*stats));
+
+ spin_lock_irq(&scheduler->lock);
+
+ for_each_scheduler_node(node, engine->id) {
+ if (node->status >= I915_SQS_MAX) {
+ DRM_DEBUG_DRIVER("Invalid node state: %d! [uniq = %d, seqno = %d]\n",
+ node->status, node->params.request->uniq,
+ node->params.request->seqno);
+
+ stats->counts[I915_SQS_MAX]++;
+ continue;
+ }
+
+ stats->counts[node->status]++;
+ }
+
+ WARN(stats->counts[I915_SQS_QUEUED] != scheduler->counts[engine->id].queued,
+ "Queued count mis-match: %d vs %d!\n",
+ stats->counts[I915_SQS_QUEUED], scheduler->counts[engine->id].queued);
+ WARN(stats->counts[I915_SQS_FLYING] != scheduler->counts[engine->id].flying,
+ "Flying count mis-match: %d vs %d!\n",
+ stats->counts[I915_SQS_FLYING], scheduler->counts[engine->id].flying);
+
+ spin_unlock_irq(&scheduler->lock);
+
+ return 0;
+}
+
static int i915_scheduler_submit_max_priority(struct intel_engine_cs *engine,
bool is_locked)
{
@@ -1160,6 +1232,7 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
}
spin_lock_irq(&scheduler->lock);
+ scheduler->stats[engine->id].flush_stamp++;
i915_scheduler_priority_bump_clear(scheduler);
for_each_scheduler_node(node, engine->id) {
if (!I915_SQS_IS_QUEUED(node))
@@ -1170,12 +1243,15 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
flush_count = i915_scheduler_priority_bump(scheduler,
node, scheduler->priority_level_max);
+ scheduler->stats[engine->id].flush_bump += flush_count;
}
spin_unlock_irq(&scheduler->lock);
if (flush_count) {
DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", engine->name, flush_count);
flush_count = i915_scheduler_submit_max_priority(engine, is_locked);
+ if (flush_count > 0)
+ scheduler->stats[engine->id].flush_submit += flush_count;
}
return flush_count;
@@ -1211,6 +1287,8 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked)
WARN_ON(is_locked && (scheduler->flags[engine->id] & I915_SF_SUBMITTING));
+ scheduler->stats[engine->id].flush_all++;
+
do {
found = false;
spin_lock_irq(&scheduler->lock);
@@ -1228,6 +1306,7 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked)
ret = i915_scheduler_submit(engine);
else
ret = i915_scheduler_submit_unlocked(engine);
+ scheduler->stats[engine->id].flush_submit++;
if (ret < 0)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 5c33c83..75d7962 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -80,6 +80,36 @@ struct i915_scheduler_node_states {
uint32_t queued;
};
+struct i915_scheduler_stats {
+ /* Batch buffer counts: */
+ uint32_t queued;
+ uint32_t submitted;
+ uint32_t completed;
+ uint32_t expired;
+
+ /* Other stuff: */
+ uint32_t flush_obj;
+ uint32_t flush_req;
+ uint32_t flush_stamp;
+ uint32_t flush_all;
+ uint32_t flush_bump;
+ uint32_t flush_submit;
+
+ uint32_t exec_early;
+ uint32_t exec_again;
+ uint32_t exec_dead;
+ uint32_t kill_flying;
+ uint32_t kill_queued;
+
+ uint32_t file_wait;
+ uint32_t file_stall;
+ uint32_t file_lost;
+};
+
+struct i915_scheduler_stats_nodes {
+ uint32_t counts[I915_SQS_MAX + 1];
+};
+
struct i915_scheduler {
struct list_head node_queue[I915_NUM_ENGINES];
uint32_t flags[I915_NUM_ENGINES];
@@ -95,6 +125,9 @@ struct i915_scheduler {
int32_t priority_level_preempt;
uint32_t min_flying;
uint32_t file_queue_max;
+
+ /* Statistics: */
+ struct i915_scheduler_stats stats[I915_NUM_ENGINES];
};
/* Flag bits for i915_scheduler::flags */
@@ -117,6 +150,8 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
unsigned long stamp, bool is_locked);
bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
+int i915_scheduler_query_stats(struct intel_engine_cs *engine,
+ struct i915_scheduler_stats_nodes *stats);
bool i915_scheduler_file_queue_wait(struct drm_file *file);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 28/34] drm/i915: Add scheduler support functions for TDR
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (26 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 27/34] drm/i915: Added scheduler statistic reporting to debugfs John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 29/34] drm/i915: Enable GPU scheduler by default John.C.Harrison
` (13 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The TDR code needs to know what the scheduler is up to in order to
work out whether a ring is really hung or not.
v4: Removed some unnecessary braces to keep the style checker happy.
v5: Removed white space and added documentation. [Joonas Lahtinen]
Also updated for new module parameter.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' helper macro. Updated to use
'to_i915()' instead of dev_private. [review feedback from Joonas
Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_scheduler.c | 33 +++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
2 files changed, 34 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 74b33f9..e7377bb 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -1388,3 +1388,36 @@ void i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
spin_unlock_irq(&scheduler->lock);
}
+
+/**
+ * i915_scheduler_is_engine_flying - does the given engine have in flight batches?
+ * @engine: Engine to query
+ * Used by TDR to distinguish hung engines (not moving but with work to do)
+ * from idle engines (not moving because there is nothing to do). Returns true
+ * if the given engine has batches currently executing on the hardware.
+ */
+bool i915_scheduler_is_engine_flying(struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ unsigned long flags;
+ bool found = false;
+
+ /* With the scheduler in bypass mode, no information can be returned. */
+ if (!i915.enable_scheduler)
+ return true;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+
+ for_each_scheduler_node(node, engine->id) {
+ if (I915_SQS_IS_FLYING(node)) {
+ found = true;
+ break;
+ }
+ }
+
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return found;
+}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 75d7962..bdb9403 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -145,6 +145,7 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
void i915_scheduler_wakeup(struct drm_device *dev);
+bool i915_scheduler_is_engine_flying(struct intel_engine_cs *engine);
void i915_scheduler_work_handler(struct work_struct *work);
int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 29/34] drm/i915: Enable GPU scheduler by default
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (27 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 28/34] drm/i915: Add scheduler support functions for TDR John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 30/34] drm/i915: Add scheduling priority to per-context parameters John.C.Harrison
` (12 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Now that all the scheduler patches have been applied, it is safe to enable.
v5: Updated for new module parameter.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_params.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_params.c b/drivers/gpu/drm/i915/i915_params.c
index 45288a2..067f13a 100644
--- a/drivers/gpu/drm/i915/i915_params.c
+++ b/drivers/gpu/drm/i915/i915_params.c
@@ -58,7 +58,7 @@ struct i915_params i915 __read_mostly = {
.guc_log_level = -1,
.enable_dp_mst = true,
.inject_load_failure = 0,
- .enable_scheduler = 0,
+ .enable_scheduler = 1,
};
module_param_named(modeset, i915.modeset, int, 0400);
@@ -213,4 +213,4 @@ MODULE_PARM_DESC(inject_load_failure,
"Force an error after a number of failure check points (0:disabled (default), N:force failure at the Nth failure check point)");
module_param_named_unsafe(enable_scheduler, i915.enable_scheduler, int, 0600);
-MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable [default], 1 = enable)");
+MODULE_PARM_DESC(enable_scheduler, "Enable scheduler (0 = disable, 1 = enable [default])");
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 30/34] drm/i915: Add scheduling priority to per-context parameters
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (28 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 29/34] drm/i915: Enable GPU scheduler by default John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 31/34] drm/i915: Add support for retro-actively banning batch buffers John.C.Harrison
` (11 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: Dave Gordon <david.s.gordon@intel.com>
Added an interface for user land applications/libraries/services to
set their GPU scheduler priority. This extends the existing context
parameter IOCTL interface to add a scheduler priority parameter. The
range is +/-1023 with +ve numbers meaning higher priority. Only
system processes may set a higher priority than the default (zero),
normal applications may only lower theirs.
v2: New patch in series.
v6: Updated to use 'to_i915()' instead of dev_private.
[review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: Dave Gordon <David.S.Gordon@Intel.com>
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 14 ++++++++++++++
drivers/gpu/drm/i915/i915_gem_context.c | 24 ++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 3 +++
include/uapi/drm/i915_drm.h | 1 +
4 files changed, 42 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 25b8fd6..4d5d059 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -815,6 +815,19 @@ struct i915_ctx_hang_stats {
bool banned;
};
+/*
+ * User-settable GFX scheduler priorities are on a scale of -1023 (I don't
+ * care about running) to +1023 (I'm the most important thing in existence)
+ * with zero being the default. Any process may decrease its scheduling
+ * priority, but only a sufficiently privileged process may increase it
+ * beyond zero.
+ */
+
+struct i915_ctx_sched_info {
+ /* Scheduling priority */
+ int32_t priority;
+};
+
struct i915_fence_timeline {
char name[32];
unsigned fence_context;
@@ -855,6 +868,7 @@ struct intel_context {
int flags;
struct drm_i915_file_private *file_priv;
struct i915_ctx_hang_stats hang_stats;
+ struct i915_ctx_sched_info sched_info;
struct i915_hw_ppgtt *ppgtt;
/* Legacy ring buffer submission */
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 6b30de9..6330b39 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -961,6 +961,9 @@ int i915_gem_context_getparam_ioctl(struct drm_device *dev, void *data,
else
args->value = to_i915(dev)->ggtt.base.total;
break;
+ case I915_CONTEXT_PARAM_PRIORITY:
+ args->value = (__u64) ctx->sched_info.priority;
+ break;
default:
ret = -EINVAL;
break;
@@ -998,6 +1001,7 @@ int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data,
else
ctx->hang_stats.ban_period_seconds = args->value;
break;
+
case I915_CONTEXT_PARAM_NO_ZEROMAP:
if (args->size) {
ret = -EINVAL;
@@ -1006,6 +1010,26 @@ int i915_gem_context_setparam_ioctl(struct drm_device *dev, void *data,
ctx->flags |= args->value ? CONTEXT_NO_ZEROMAP : 0;
}
break;
+
+ case I915_CONTEXT_PARAM_PRIORITY:
+ {
+ int32_t priority = (int32_t) args->value;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ if (args->size)
+ ret = -EINVAL;
+ else if ((priority > scheduler->priority_level_max) ||
+ (priority < scheduler->priority_level_min))
+ ret = -EINVAL;
+ else if ((priority > 0) &&
+ !capable(CAP_SYS_ADMIN))
+ ret = -EPERM;
+ else
+ ctx->sched_info.priority = priority;
+ break;
+ }
+
default:
ret = -EINVAL;
break;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 823283d..fc663fa 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1719,6 +1719,9 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->batch_obj = batch_obj;
params->request = req;
+ /* Start with the context's priority level */
+ qe.priority = ctx->sched_info.priority;
+
/*
* Save away the list of objects used by this batch buffer for the
* purpose of tracking inter-buffer dependencies.
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index a5524cc..0e87551 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -1167,6 +1167,7 @@ struct drm_i915_gem_context_param {
#define I915_CONTEXT_PARAM_BAN_PERIOD 0x1
#define I915_CONTEXT_PARAM_NO_ZEROMAP 0x2
#define I915_CONTEXT_PARAM_GTT_SIZE 0x3
+#define I915_CONTEXT_PARAM_PRIORITY 0x4
__u64 value;
};
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 31/34] drm/i915: Add support for retro-actively banning batch buffers
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (29 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 30/34] drm/i915: Add scheduling priority to per-context parameters John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 32/34] drm/i915: Allow scheduler to manage inter-ring object synchronisation John.C.Harrison
` (10 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
If a given context submits too many hanging batch buffers then it will
be banned and no further batch buffers will be accepted for it.
However, it is possible that a large number of buffers may already
have been accepted and are sat in the scheduler waiting to be
executed. This patch adds a late ban check to ensure that these will
also be discarded.
v4: New patch in series.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 6 ++++++
drivers/gpu/drm/i915/intel_lrc.c | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index fc663fa..789f668 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1293,6 +1293,12 @@ int i915_gem_ringbuffer_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Check the context wasn't banned between submission and execution: */
+ if (params->ctx->hang_stats.banned) {
+ DRM_DEBUG("Trying to execute for banned context!\n");
+ return -ENOENT;
+ }
+
/* Make sure the request's seqno is the latest and greatest: */
if (req->reserved_seqno != dev_priv->last_seqno) {
ret = i915_gem_get_seqno(engine->dev, &req->reserved_seqno);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index b9258ee..02808f7 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1034,6 +1034,12 @@ int intel_execlists_submission_final(struct i915_execbuffer_params *params)
/* The mutex must be acquired before calling this function */
WARN_ON(!mutex_is_locked(¶ms->dev->struct_mutex));
+ /* Check the context wasn't banned between submission and execution: */
+ if (params->ctx->hang_stats.banned) {
+ DRM_DEBUG("Trying to execute for banned context!\n");
+ return -ENOENT;
+ }
+
/* Make sure the request's seqno is the latest and greatest: */
if (req->reserved_seqno != dev_priv->last_seqno) {
ret = i915_gem_get_seqno(engine->dev, &req->reserved_seqno);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 32/34] drm/i915: Allow scheduler to manage inter-ring object synchronisation
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (30 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 31/34] drm/i915: Add support for retro-actively banning batch buffers John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 33/34] drm/i915: Added debug state dump facilities to scheduler John.C.Harrison
` (9 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler has always tracked batch buffer dependencies based on
DRM object usage. This means that it will not submit a batch on one
ring that has outstanding dependencies still executing on other rings.
This is exactly the same synchronisation performed by
i915_gem_object_sync() using hardware semaphores where available and
CPU stalls where not (e.g. in execlist mode and/or on Gen8 hardware).
Unfortunately, when a batch buffer is submitted to the driver the
_object_sync() call happens first. Thus in case where hardware
semaphores are disabled, the driver has already stalled until the
dependency has been resolved.
This patch adds an optimisation to _object_sync() to ignore the
synchronisation in the case where it will subsequently be handled by
the scheduler. This removes the driver stall and (in the single
application case) provides near hardware semaphore performance even
when hardware semaphores are disabled. In a busy system where there is
other work that can be executed on the stalling ring, it provides
better than hardware semaphore performance as it removes the stall
from both the driver and from the hardware. There is also a theory
that this method should improve power usage as hardware semaphores are
apparently not very power efficient - the stalled ring does not go
into as low a power a state as when it is genuinely idle.
The optimisation is to check whether both ends of the synchronisation
are batch buffer requests. If they are, then the scheduler will have
the inter-dependency tracked and managed. If one or other end is not a
batch buffer request (e.g. a page flip) then the code falls back to
the CPU stall or hardware semaphore as appropriate.
To check whether the existing usage is a batch buffer, the code simply
calls the 'are you tracking this request' function of the scheduler on
the object's last_read_req member. To check whether the new usage is a
batch buffer, a flag is passed in from the caller.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Replaced the i915_scheduler_is_request_tracked() function with
i915_scheduler_is_request_batch_buffer() as the need for the former
has gone away and it was really being used to ask the latter question
in a convoluted manner. [review feedback from Joonas Lahtinen]
Issue: VIZ-5566
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 2 +-
drivers/gpu/drm/i915/i915_gem.c | 16 +++++++++++++---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 19 +++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 1 +
drivers/gpu/drm/i915/intel_display.c | 2 +-
drivers/gpu/drm/i915/intel_lrc.c | 2 +-
7 files changed, 37 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 4d5d059..cb346ab 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3101,7 +3101,7 @@ static inline void i915_gem_object_unpin_map(struct drm_i915_gem_object *obj)
int __must_check i915_mutex_lock_interruptible(struct drm_device *dev);
int i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct intel_engine_cs *to,
- struct drm_i915_gem_request **to_req);
+ struct drm_i915_gem_request **to_req, bool to_batch);
void i915_vma_move_to_active(struct i915_vma *vma,
struct drm_i915_gem_request *req);
int i915_gem_dumb_create(struct drm_file *file_priv,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 467d7da..818113d 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -3658,7 +3658,7 @@ static int
__i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct intel_engine_cs *to,
struct drm_i915_gem_request *from_req,
- struct drm_i915_gem_request **to_req)
+ struct drm_i915_gem_request **to_req, bool to_batch)
{
struct intel_engine_cs *from;
int ret;
@@ -3670,6 +3670,14 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (i915_gem_request_completed(from_req))
return 0;
+ /*
+ * The scheduler will manage inter-ring object dependencies
+ * as long as both to and from requests are scheduler managed
+ * (i.e. batch buffers).
+ */
+ if (to_batch && i915_scheduler_is_request_batch_buffer(from_req))
+ return 0;
+
if (!i915_semaphore_is_enabled(obj->base.dev)) {
struct drm_i915_private *i915 = to_i915(obj->base.dev);
uint32_t flags;
@@ -3728,6 +3736,8 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
* @to_req: request we wish to use the object for. See below.
* This will be allocated and returned if a request is
* required but not passed in.
+ * @to_batch: is the sync request on behalf of batch buffer submission?
+ * If so then the scheduler can (potentially) manage the synchronisation.
*
* This code is meant to abstract object synchronization with the GPU.
* Calling with NULL implies synchronizing the object with the CPU
@@ -3758,7 +3768,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
int
i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct intel_engine_cs *to,
- struct drm_i915_gem_request **to_req)
+ struct drm_i915_gem_request **to_req, bool to_batch)
{
const bool readonly = obj->base.pending_write_domain == 0;
struct drm_i915_gem_request *req[I915_NUM_ENGINES];
@@ -3780,7 +3790,7 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
req[n++] = obj->last_read_req[i];
}
for (i = 0; i < n; i++) {
- ret = __i915_gem_object_sync(obj, to, req[i], to_req);
+ ret = __i915_gem_object_sync(obj, to, req[i], to_req, to_batch);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 789f668..70ad67c 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -954,7 +954,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->engine, &req);
+ ret = i915_gem_object_sync(obj, req->engine, &req, true);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index e7377bb..fd53833 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -1344,6 +1344,25 @@ bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req)
}
/**
+ * i915_scheduler_is_request_batch_buffer - is this request related to a
+ * batch buffer object?
+ * @req: request to be queried
+ *
+ * Returns true if the given request is for a batch buffer. False means it
+ * is for something else - page flip, context initialisation, etc.
+ */
+bool i915_scheduler_is_request_batch_buffer(struct drm_i915_gem_request *req)
+{
+ if (req->scheduler_qe == NULL)
+ return false;
+
+ if (req->scheduler_qe->params.batch_obj == NULL)
+ return false;
+
+ return true;
+}
+
+/**
* i915_scheduler_closefile - notify the scheduler that a DRM file handle
* has been closed.
* @dev: DRM device
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index bdb9403..0df16e7 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -151,6 +151,7 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
unsigned long stamp, bool is_locked);
bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
+bool i915_scheduler_is_request_batch_buffer(struct drm_i915_gem_request *req);
int i915_scheduler_query_stats(struct intel_engine_cs *engine,
struct i915_scheduler_stats_nodes *stats);
bool i915_scheduler_file_queue_wait(struct drm_file *file);
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index f1d730a..0e5da38 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11607,7 +11607,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
* into the display plane and skip any waits.
*/
if (!mmio_flip) {
- ret = i915_gem_object_sync(obj, engine, &request);
+ ret = i915_gem_object_sync(obj, engine, &request, false);
if (ret)
goto cleanup_pending;
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 02808f7..c1263e6 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -698,7 +698,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->engine, &req);
+ ret = i915_gem_object_sync(obj, req->engine, &req, true);
if (ret)
return ret;
}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 33/34] drm/i915: Added debug state dump facilities to scheduler
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (31 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 32/34] drm/i915: Allow scheduler to manage inter-ring object synchronisation John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 34/34] drm/i915: Scheduler state dump via debugfs John.C.Harrison
` (8 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
When debugging batch buffer submission issues, it is useful to be able
to see what the current state of the scheduler is. This change adds
functions for decoding the internal scheduler state and reporting it.
v3: Updated a debug message with the new state_str() function.
v4: Wrapped some long lines to keep the style checker happy. Removed
the fence/sync code as that will now be part of a separate patch series.
v5: Removed forward declarations and white space. Added documentation.
[Joonas Lahtinen]
Also squashed in later patch to add seqno information from the start.
It was only being added in a separate patch due to historical reasons
which have since gone away.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' helper macro. Updated to use
'to_i915()' instead of dev_private. Converted all enum labels to
uppercase. Moved the enum to string conversion function to debugfs.c
rather than scheduler.c [review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 2 +-
drivers/gpu/drm/i915/i915_scheduler.c | 270 +++++++++++++++++++++++++++++++++-
drivers/gpu/drm/i915/i915_scheduler.h | 15 ++
3 files changed, 284 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 1d04cde..9ac486f 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -3609,7 +3609,7 @@ static int i915_drrs_status(struct seq_file *m, void *unused)
return 0;
}
-static const char *i915_scheduler_queue_status_str(
+const char *i915_scheduler_queue_status_str(
enum i915_scheduler_queue_status status)
{
static char str[50];
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index fd53833..92ca786 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -48,6 +48,85 @@ bool i915_scheduler_is_enabled(struct drm_device *dev)
return dev_priv->scheduler != NULL;
}
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node)
+{
+ static char str[50];
+ char *ptr = str;
+
+ *(ptr++) = node->bumped ? 'B' : '-',
+ *(ptr++) = i915_gem_request_completed(node->params.request) ? 'C' : '-';
+
+ *ptr = 0;
+
+ return str;
+}
+
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status)
+{
+ switch (status) {
+ case I915_SQS_NONE:
+ return 'N';
+
+ case I915_SQS_QUEUED:
+ return 'Q';
+
+ case I915_SQS_POPPED:
+ return 'X';
+
+ case I915_SQS_FLYING:
+ return 'F';
+
+ case I915_SQS_COMPLETE:
+ return 'C';
+
+ case I915_SQS_DEAD:
+ return 'D';
+
+ default:
+ break;
+ }
+
+ return '?';
+}
+
+const char *i915_scheduler_flag_str(uint32_t flags)
+{
+ static char str[100];
+ char *ptr = str;
+
+ *ptr = 0;
+
+#define TEST_FLAG(flag, msg) \
+ do { \
+ if (flags & (flag)) { \
+ strcpy(ptr, msg); \
+ ptr += strlen(ptr); \
+ flags &= ~(flag); \
+ } \
+ } while (0)
+
+ TEST_FLAG(I915_SF_INTERRUPTS_ENABLED, "IntOn|");
+ TEST_FLAG(I915_SF_SUBMITTING, "Submitting|");
+ TEST_FLAG(I915_SF_DUMP_FORCE, "DumpForce|");
+ TEST_FLAG(I915_SF_DUMP_DETAILS, "DumpDetails|");
+ TEST_FLAG(I915_SF_DUMP_DEPENDENCIES, "DumpDeps|");
+ TEST_FLAG(I915_SF_DUMP_SEQNO, "DumpSeqno|");
+
+#undef TEST_FLAG
+
+ if (flags) {
+ sprintf(ptr, "Unknown_0x%X!", flags);
+ ptr += strlen(ptr);
+ }
+
+ if (ptr == str)
+ strcpy(str, "-");
+ else
+ ptr[-1] = 0;
+
+ return str;
+};
+
/**
* i915_scheduler_init - Initialise the scheduler.
* @dev: DRM device
@@ -1107,6 +1186,193 @@ err_unref:
return true;
}
+static int i915_scheduler_dump_locked(struct intel_engine_cs *engine,
+ const char *msg)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+ int flying = 0, queued = 0, complete = 0, other = 0;
+ static int old_flying = -1, old_queued = -1, old_complete = -1;
+ bool b_dump;
+ char brkt[2] = { '<', '>' };
+
+ if (!engine)
+ return -EINVAL;
+
+ for_each_scheduler_node(node, engine->id) {
+ if (I915_SQS_IS_QUEUED(node))
+ queued++;
+ else if (I915_SQS_IS_FLYING(node))
+ flying++;
+ else if (I915_SQS_IS_COMPLETE(node))
+ complete++;
+ else
+ other++;
+ }
+
+ b_dump = (flying != old_flying) ||
+ (queued != old_queued) ||
+ (complete != old_complete);
+ if (scheduler->flags[engine->id] & I915_SF_DUMP_FORCE) {
+ if (!b_dump) {
+ b_dump = true;
+ brkt[0] = '{';
+ brkt[1] = '}';
+ }
+
+ scheduler->flags[engine->id] &= ~I915_SF_DUMP_FORCE;
+ }
+
+ if (b_dump) {
+ old_flying = flying;
+ old_queued = queued;
+ old_complete = complete;
+ DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d, "
+ "Flags = %s, Next = %d:%d %c%s%c\n",
+ engine->name, queued, flying, complete, other,
+ i915_scheduler_flag_str(scheduler->flags[engine->id]),
+ dev_priv->request_uniq, dev_priv->next_seqno,
+ brkt[0], msg, brkt[1]);
+ } else {
+ /*DRM_DEBUG_DRIVER("<%s> Q:%02d, F:%02d, C:%02d, O:%02d"
+ ", Flags = %s, Next = %d:%d [%s]\n",
+ engine->name,
+ queued, flying, complete, other,
+ i915_scheduler_flag_str(scheduler->flags[engine->id]),
+ dev_priv->request_uniq, dev_priv->next_seqno, msg); */
+
+ return 0;
+ }
+
+ if (scheduler->flags[engine->id] & I915_SF_DUMP_SEQNO) {
+ uint32_t seqno;
+
+ seqno = engine->get_seqno(engine);
+
+ DRM_DEBUG_DRIVER("<%s> Seqno = %d\n", engine->name, seqno);
+ }
+
+ if (scheduler->flags[engine->id] & I915_SF_DUMP_DETAILS) {
+ int i, deps;
+ uint32_t count, counts[I915_SQS_MAX];
+
+ memset(counts, 0x00, sizeof(counts));
+
+ for_each_scheduler_node(node, engine->id) {
+ if (node->status < I915_SQS_MAX) {
+ count = counts[node->status]++;
+ } else {
+ DRM_DEBUG_DRIVER("<%s> Unknown status: %d!\n",
+ engine->name, node->status);
+ count = -1;
+ }
+
+ deps = 0;
+ for (i = 0; i < node->num_deps; i++)
+ if (i915_scheduler_is_dependency_valid(node, i))
+ deps++;
+
+ DRM_DEBUG_DRIVER("<%s> %c:%02d> uniq = %d, seqno"
+ " = %d/%s, deps = %d / %d, %s [pri = "
+ "%4d]\n", engine->name,
+ i915_scheduler_queue_status_chr(node->status),
+ count,
+ node->params.request->uniq,
+ node->params.request->seqno,
+ node->params.engine->name,
+ deps, node->num_deps,
+ i915_qe_state_str(node),
+ node->priority);
+
+ if ((scheduler->flags[engine->id] & I915_SF_DUMP_DEPENDENCIES)
+ == 0)
+ continue;
+
+ for (i = 0; i < node->num_deps; i++)
+ if (node->dep_list[i])
+ DRM_DEBUG_DRIVER("<%s> |-%c:"
+ "%02d%c uniq = %d, seqno = %d/%s, %s [pri = %4d]\n",
+ engine->name,
+ i915_scheduler_queue_status_chr(node->dep_list[i]->status),
+ i,
+ i915_scheduler_is_dependency_valid(node, i)
+ ? '>' : '#',
+ node->dep_list[i]->params.request->uniq,
+ node->dep_list[i]->params.request->seqno,
+ node->dep_list[i]->params.engine->name,
+ i915_qe_state_str(node->dep_list[i]),
+ node->dep_list[i]->priority);
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * i915_scheduler_dump - dump the scheduler's internal state to the debug log.
+ * @engine: Engine to dump info for
+ * @msg: A reason why it is being dumped
+ * For debugging purposes, it can be very useful to see the internal state of
+ * the scheduler for a given engine.
+ */
+int i915_scheduler_dump(struct intel_engine_cs *engine, const char *msg)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+ ret = i915_scheduler_dump_locked(engine, msg);
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return ret;
+}
+
+static int i915_scheduler_dump_all_locked(struct drm_device *dev,
+ const char *msg)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct intel_engine_cs *engine;
+ int r, ret = 0;
+
+ for_each_engine(engine, dev_priv) {
+ scheduler->flags[engine->id] |= I915_SF_DUMP_FORCE |
+ I915_SF_DUMP_DETAILS |
+ I915_SF_DUMP_SEQNO |
+ I915_SF_DUMP_DEPENDENCIES;
+ r = i915_scheduler_dump_locked(engine, msg);
+ if (ret == 0)
+ ret = r;
+ }
+
+ return ret;
+}
+
+/**
+ * i915_scheduler_dump_all - dump the scheduler's internal state to the debug
+ * log.
+ * @dev: DRM device
+ * @msg: A reason why it is being dumped
+ * For debugging purposes, it can be very useful to see the internal state of
+ * the scheduler.
+ */
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&scheduler->lock, flags);
+ ret = i915_scheduler_dump_all_locked(dev, msg);
+ spin_unlock_irqrestore(&scheduler->lock, flags);
+
+ return ret;
+}
+
/**
* i915_scheduler_query_stats - return various scheduler statistics
* @engine: Engine to report on
@@ -1394,10 +1660,10 @@ void i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
continue;
if (!I915_SQS_IS_COMPLETE(node))
- DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%d on %s\n",
+ DRM_DEBUG_DRIVER("Closing file handle with outstanding work: %d:%d/%s on %s\n",
node->params.request->uniq,
node->params.request->seqno,
- node->status,
+ i915_qe_state_str(node),
engine->name);
i915_scheduler_file_queue_dec(node->params.file);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 0df16e7..7c0edf5 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -41,6 +41,9 @@ enum i915_scheduler_queue_status {
/* Limit value for use with arrays/loops */
I915_SQS_MAX
};
+char i915_scheduler_queue_status_chr(enum i915_scheduler_queue_status status);
+const char *i915_scheduler_queue_status_str(
+ enum i915_scheduler_queue_status status);
#define I915_SQS_IS_QUEUED(node) (((node)->status == I915_SQS_QUEUED))
#define I915_SQS_IS_FLYING(node) (((node)->status == I915_SQS_FLYING))
@@ -74,6 +77,7 @@ struct i915_scheduler_queue_entry {
/* List of all scheduler queue entry nodes */
struct list_head link;
};
+const char *i915_qe_state_str(struct i915_scheduler_queue_entry *node);
struct i915_scheduler_node_states {
uint32_t flying;
@@ -132,9 +136,17 @@ struct i915_scheduler {
/* Flag bits for i915_scheduler::flags */
enum {
+ /* Internal state */
I915_SF_INTERRUPTS_ENABLED = (1 << 0),
I915_SF_SUBMITTING = (1 << 1),
+
+ /* Dump/debug flags */
+ I915_SF_DUMP_FORCE = (1 << 8),
+ I915_SF_DUMP_DETAILS = (1 << 9),
+ I915_SF_DUMP_DEPENDENCIES = (1 << 10),
+ I915_SF_DUMP_SEQNO = (1 << 11),
};
+const char *i915_scheduler_flag_str(uint32_t flags);
bool i915_scheduler_is_enabled(struct drm_device *dev);
int i915_scheduler_init(struct drm_device *dev);
@@ -150,6 +162,9 @@ void i915_scheduler_work_handler(struct work_struct *work);
int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
unsigned long stamp, bool is_locked);
+int i915_scheduler_dump(struct intel_engine_cs *engine,
+ const char *msg);
+int i915_scheduler_dump_all(struct drm_device *dev, const char *msg);
bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
bool i915_scheduler_is_request_batch_buffer(struct drm_i915_gem_request *req);
int i915_scheduler_query_stats(struct intel_engine_cs *engine,
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 34/34] drm/i915: Scheduler state dump via debugfs
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (32 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 33/34] drm/i915: Added debug state dump facilities to scheduler John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH 1/1] drm/i915: Add wrapper for context priority interface John.C.Harrison
` (7 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
Added a facility for triggering the scheduler state dump via a debugfs
entry.
v2: New patch in series.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Updated to use 'to_i915()' instead of dev_private. Converted all enum
labels to uppercase. [review feedback from Joonas Lahtinen]
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 33 +++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.c | 9 +++++----
drivers/gpu/drm/i915/i915_scheduler.h | 6 ++++++
3 files changed, 44 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 9ac486f..5328dc1 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -1306,6 +1306,38 @@ DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_file_queue_max_fops,
i915_scheduler_file_queue_max_set,
"%llu\n");
+static int
+i915_scheduler_dump_flags_get(void *data, u64 *val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ *val = scheduler->dump_flags;
+
+ return 0;
+}
+
+static int
+i915_scheduler_dump_flags_set(void *data, u64 val)
+{
+ struct drm_device *dev = data;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+
+ scheduler->dump_flags = lower_32_bits(val) & I915_SF_DUMP_MASK;
+
+ if (val & 1)
+ i915_scheduler_dump_all(dev, "DebugFS");
+
+ return 0;
+}
+
+DEFINE_SIMPLE_ATTRIBUTE(i915_scheduler_dump_flags_fops,
+ i915_scheduler_dump_flags_get,
+ i915_scheduler_dump_flags_set,
+ "0x%llx\n");
+
static int i915_frequency_info(struct seq_file *m, void *unused)
{
struct drm_info_node *node = m->private;
@@ -5724,6 +5756,7 @@ static const struct i915_debugfs_files {
{"i915_scheduler_priority_preempt", &i915_scheduler_priority_preempt_fops},
{"i915_scheduler_min_flying", &i915_scheduler_min_flying_fops},
{"i915_scheduler_file_queue_max", &i915_scheduler_file_queue_max_fops},
+ {"i915_scheduler_dump_flags", &i915_scheduler_dump_flags_fops},
{"i915_display_crc_ctl", &i915_display_crc_ctl_fops},
{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 92ca786..7afb528 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -160,6 +160,10 @@ int i915_scheduler_init(struct drm_device *dev)
scheduler->priority_level_preempt = 900;
scheduler->min_flying = 2;
scheduler->file_queue_max = 64;
+ scheduler->dump_flags = I915_SF_DUMP_FORCE |
+ I915_SF_DUMP_DETAILS |
+ I915_SF_DUMP_SEQNO |
+ I915_SF_DUMP_DEPENDENCIES;
dev_priv->scheduler = scheduler;
@@ -1339,10 +1343,7 @@ static int i915_scheduler_dump_all_locked(struct drm_device *dev,
int r, ret = 0;
for_each_engine(engine, dev_priv) {
- scheduler->flags[engine->id] |= I915_SF_DUMP_FORCE |
- I915_SF_DUMP_DETAILS |
- I915_SF_DUMP_SEQNO |
- I915_SF_DUMP_DEPENDENCIES;
+ scheduler->flags[engine->id] |= scheduler->dump_flags & I915_SF_DUMP_MASK;
r = i915_scheduler_dump_locked(engine, msg);
if (ret == 0)
ret = r;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 7c0edf5..1840536 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -129,6 +129,7 @@ struct i915_scheduler {
int32_t priority_level_preempt;
uint32_t min_flying;
uint32_t file_queue_max;
+ uint32_t dump_flags;
/* Statistics: */
struct i915_scheduler_stats stats[I915_NUM_ENGINES];
@@ -145,6 +146,11 @@ enum {
I915_SF_DUMP_DETAILS = (1 << 9),
I915_SF_DUMP_DEPENDENCIES = (1 << 10),
I915_SF_DUMP_SEQNO = (1 << 11),
+
+ I915_SF_DUMP_MASK = I915_SF_DUMP_FORCE |
+ I915_SF_DUMP_DETAILS |
+ I915_SF_DUMP_DEPENDENCIES |
+ I915_SF_DUMP_SEQNO,
};
const char *i915_scheduler_flag_str(uint32_t flags);
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 1/1] drm/i915: Add wrapper for context priority interface
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (33 preceding siblings ...)
2016-04-20 17:13 ` [PATCH v6 34/34] drm/i915: Scheduler state dump via debugfs John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH 1/2] igt/gem_ctx_param_basic: Updated to support scheduler " John.C.Harrison
` (6 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
There is an EGL extension to set execution priority per context. This
can be implemented via the i915 per context priority parameter. This
patch adds a wrapper to connect the two together in a way that can be
updated as necessary without breaking one side or the other.
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
include/drm/i915_drm.h | 1 +
intel/intel_bufmgr.h | 5 ++++
intel/intel_bufmgr_gem.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 76 insertions(+)
diff --git a/include/drm/i915_drm.h b/include/drm/i915_drm.h
index c4ce6b2..8d9f063 100644
--- a/include/drm/i915_drm.h
+++ b/include/drm/i915_drm.h
@@ -1167,6 +1167,7 @@ struct drm_i915_gem_context_param {
#define I915_CONTEXT_PARAM_BAN_PERIOD 0x1
#define I915_CONTEXT_PARAM_NO_ZEROMAP 0x2
#define I915_CONTEXT_PARAM_GTT_SIZE 0x3
+#define I915_CONTEXT_PARAM_PRIORITY 0x4
__u64 value;
};
diff --git a/intel/intel_bufmgr.h b/intel/intel_bufmgr.h
index a1abbcd..49d4125 100644
--- a/intel/intel_bufmgr.h
+++ b/intel/intel_bufmgr.h
@@ -209,6 +209,11 @@ int drm_intel_gem_bo_wait(drm_intel_bo *bo, int64_t timeout_ns);
drm_intel_context *drm_intel_gem_context_create(drm_intel_bufmgr *bufmgr);
void drm_intel_gem_context_destroy(drm_intel_context *ctx);
+int drm_intel_gem_context_get_priority(drm_intel_context *ctx, int *priority);
+#define DRM_INTEL_CTX_SET_PRIORITY_EGL_LOW (0x10001)
+#define DRM_INTEL_CTX_SET_PRIORITY_EGL_MEDIUM (0x10002)
+#define DRM_INTEL_CTX_SET_PRIORITY_EGL_HIGH (0x10003)
+int drm_intel_gem_context_set_priority(drm_intel_context *ctx, int priority);
int drm_intel_gem_bo_context_exec(drm_intel_bo *bo, drm_intel_context *ctx,
int used, unsigned int flags);
diff --git a/intel/intel_bufmgr_gem.c b/intel/intel_bufmgr_gem.c
index 0a4012b..591ccd5 100644
--- a/intel/intel_bufmgr_gem.c
+++ b/intel/intel_bufmgr_gem.c
@@ -3154,6 +3154,76 @@ drm_intel_gem_context_destroy(drm_intel_context *ctx)
}
int
+drm_intel_gem_context_get_priority(drm_intel_context *ctx, int *priority)
+{
+ drm_intel_bufmgr_gem *bufmgr_gem;
+ struct drm_i915_gem_context_param ctxparam;
+ int ret;
+
+ if ((ctx == NULL) || (priority == NULL))
+ return -EINVAL;
+
+ memclear(ctxparam);
+
+ bufmgr_gem = (drm_intel_bufmgr_gem *)ctx->bufmgr;
+ ctxparam.ctx_id = ctx->ctx_id;
+ ctxparam.param = I915_CONTEXT_PARAM_PRIORITY;
+ ret = drmIoctl(bufmgr_gem->fd, DRM_IOCTL_I915_GEM_CONTEXT_GETPARAM,
+ &ctxparam);
+ if (ret != 0) {
+ int err = errno;
+ *priority = 0;
+ fprintf(stderr, "DRM_IOCTL_I915_GEM_CONTEXT_GETPARAM failed: %s\n",
+ strerror(err));
+ return -err;
+ }
+
+ *priority = (int) ctxparam.value;
+
+ return 0;
+}
+
+int
+drm_intel_gem_context_set_priority(drm_intel_context *ctx, int priority)
+{
+ drm_intel_bufmgr_gem *bufmgr_gem;
+ struct drm_i915_gem_context_param ctxparam;
+ int ret;
+
+ if (ctx == NULL)
+ return -EINVAL;
+
+ memclear(ctxparam);
+
+ switch (priority) {
+ case DRM_INTEL_CTX_SET_PRIORITY_EGL_LOW:
+ priority = -500;
+ break;
+ case DRM_INTEL_CTX_SET_PRIORITY_EGL_MEDIUM:
+ priority = -0;
+ break;
+ case DRM_INTEL_CTX_SET_PRIORITY_EGL_HIGH:
+ priority = 500;
+ break;
+ }
+
+ bufmgr_gem = (drm_intel_bufmgr_gem *)ctx->bufmgr;
+ ctxparam.ctx_id = ctx->ctx_id;
+ ctxparam.param = I915_CONTEXT_PARAM_PRIORITY;
+ ctxparam.value = priority;
+ ret = drmIoctl(bufmgr_gem->fd, DRM_IOCTL_I915_GEM_CONTEXT_SETPARAM,
+ &ctxparam);
+ if (ret != 0) {
+ int err = errno;
+ fprintf(stderr, "DRM_IOCTL_I915_GEM_CONTEXT_SETPARAM failed: %s\n",
+ strerror(err));
+ return -err;
+ }
+
+ return 0;
+}
+
+int
drm_intel_get_reset_stats(drm_intel_context *ctx,
uint32_t *reset_count,
uint32_t *active,
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 1/2] igt/gem_ctx_param_basic: Updated to support scheduler priority interface
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (34 preceding siblings ...)
2016-04-20 17:13 ` [PATCH 1/1] drm/i915: Add wrapper for context priority interface John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH 2/2] igt/gem_scheduler: Add gem_scheduler test John.C.Harrison
` (5 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The GPU scheduler has added an execution priority level to the context
object. There is an IOCTL interface to allow user apps/libraries to
set this priority. This patch updates the context paramter IOCTL test
to include the new interface.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
lib/ioctl_wrappers.h | 1 +
tests/gem_ctx_param_basic.c | 108 +++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 108 insertions(+), 1 deletion(-)
diff --git a/lib/ioctl_wrappers.h b/lib/ioctl_wrappers.h
index 8fe35b0..00cf324 100644
--- a/lib/ioctl_wrappers.h
+++ b/lib/ioctl_wrappers.h
@@ -109,6 +109,7 @@ struct local_i915_gem_context_param {
#define LOCAL_CONTEXT_PARAM_BAN_PERIOD 0x1
#define LOCAL_CONTEXT_PARAM_NO_ZEROMAP 0x2
#define LOCAL_CONTEXT_PARAM_GTT_SIZE 0x3
+#define LOCAL_CONTEXT_PARAM_PRIORITY 0x4
uint64_t value;
};
void gem_context_require_ban_period(int fd);
diff --git a/tests/gem_ctx_param_basic.c b/tests/gem_ctx_param_basic.c
index b75800c..1a10c01 100644
--- a/tests/gem_ctx_param_basic.c
+++ b/tests/gem_ctx_param_basic.c
@@ -147,10 +147,116 @@ igt_main
TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
}
+ ctx_param.param = LOCAL_CONTEXT_PARAM_PRIORITY;
+
+ igt_subtest("priority-root-set") {
+ ctx_param.context = ctx;
+ ctx_param.value = 2048;
+ TEST_FAIL(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM, EINVAL);
+ ctx_param.value = -2048;
+ TEST_FAIL(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM, EINVAL);
+ ctx_param.value = 512;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ ctx_param.value = -512;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ ctx_param.value = 0;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ }
+
+ igt_subtest("priority-root-set-egl") {
+ drm_intel_context *context;
+ drm_intel_bufmgr *bufmgr;
+ int ret, priority;
+
+ bufmgr = drm_intel_bufmgr_gem_init(fd, 4096);
+ context = drm_intel_gem_context_create(bufmgr);
+ igt_require(context);
+
+ ret = drm_intel_gem_context_set_priority(context, DRM_INTEL_CTX_SET_PRIORITY_EGL_LOW);
+ igt_assert_eq(ret, 0);
+ ret = drm_intel_gem_context_get_priority(context, &priority);
+ igt_assert_eq(ret, 0);
+ igt_assert_eq(priority, -500);
+
+ ret = drm_intel_gem_context_set_priority(context, DRM_INTEL_CTX_SET_PRIORITY_EGL_MEDIUM);
+ igt_assert_eq(ret, 0);
+ ret = drm_intel_gem_context_get_priority(context, &priority);
+ igt_assert_eq(ret, 0);
+ igt_assert_eq(priority, 0);
+
+ ret = drm_intel_gem_context_set_priority(context, DRM_INTEL_CTX_SET_PRIORITY_EGL_HIGH);
+ igt_assert_eq(ret, 0);
+ ret = drm_intel_gem_context_get_priority(context, &priority);
+ igt_assert_eq(ret, 0);
+ igt_assert_eq(priority, 500);
+
+ drm_intel_gem_context_destroy(context);
+ drm_intel_bufmgr_destroy(bufmgr);
+ }
+
+ igt_subtest("priority-non-root-set") {
+ igt_fork(child, 1) {
+ igt_drop_root();
+
+ ctx_param.context = ctx;
+ ctx_param.value = 512;
+ TEST_FAIL(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM, EPERM);
+ ctx_param.value = -512;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ ctx_param.value = 0;
+ TEST_SUCCESS(LOCAL_IOCTL_I915_GEM_CONTEXT_SETPARAM);
+ }
+
+ igt_waitchildren();
+ }
+
+ igt_subtest("priority-non-root-set-egl") {
+ igt_fork(child, 1) {
+ igt_drop_root();
+
+ drm_intel_context *context;
+ drm_intel_bufmgr *bufmgr;
+ int ret, priority;
+
+ bufmgr = drm_intel_bufmgr_gem_init(fd, 4096);
+ context = drm_intel_gem_context_create(bufmgr);
+ igt_require(context);
+
+ ret = drm_intel_gem_context_set_priority(context, DRM_INTEL_CTX_SET_PRIORITY_EGL_LOW);
+ igt_assert_eq(ret, 0);
+ ret = drm_intel_gem_context_get_priority(context, &priority);
+ igt_assert_eq(ret, 0);
+ igt_assert_eq(priority, -500);
+
+ ret = drm_intel_gem_context_set_priority(context, DRM_INTEL_CTX_SET_PRIORITY_EGL_MEDIUM);
+ igt_assert_eq(ret, 0);
+ ret = drm_intel_gem_context_get_priority(context, &priority);
+ igt_assert_eq(ret, 0);
+ igt_assert_eq(priority, 0);
+
+ ret = drm_intel_gem_context_set_priority(context, DRM_INTEL_CTX_SET_PRIORITY_EGL_LOW);
+ igt_assert_eq(ret, 0);
+ ret = drm_intel_gem_context_get_priority(context, &priority);
+ igt_assert_eq(ret, 0);
+ igt_assert_eq(priority, -500);
+
+ ret = drm_intel_gem_context_set_priority(context, DRM_INTEL_CTX_SET_PRIORITY_EGL_HIGH);
+ igt_assert_eq(ret, -EPERM);
+ ret = drm_intel_gem_context_get_priority(context, &priority);
+ igt_assert_eq(ret, 0);
+ igt_assert_eq(priority, -500);
+
+ drm_intel_gem_context_destroy(context);
+ drm_intel_bufmgr_destroy(bufmgr);
+ }
+
+ igt_waitchildren();
+ }
+
/* NOTE: This testcase intentionally tests for the next free parameter
* to catch ABI extensions. Don't "fix" this testcase without adding all
* the tests for the new param first. */
- ctx_param.param = LOCAL_CONTEXT_PARAM_GTT_SIZE + 1;
+ ctx_param.param = LOCAL_CONTEXT_PARAM_PRIORITY + 1;
igt_subtest("invalid-param-get") {
ctx_param.context = ctx;
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 2/2] igt/gem_scheduler: Add gem_scheduler test
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (35 preceding siblings ...)
2016-04-20 17:13 ` [PATCH 1/2] igt/gem_ctx_param_basic: Updated to support scheduler " John.C.Harrison
@ 2016-04-20 17:13 ` John.C.Harrison
2016-04-21 9:43 ` ✓ Fi.CI.BAT: success for GPU scheduler for i915 driver (rev2) Patchwork
` (4 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-04-20 17:13 UTC (permalink / raw)
To: Intel-GFX
From: Derek Morton <derek.j.morton@intel.com>
---
lib/Makefile.sources | 2 +
lib/igt.h | 1 +
lib/igt_bb_factory.c | 391 +++++++++++++++++++++++++++++++++++++++++++++
lib/igt_bb_factory.h | 47 ++++++
tests/Makefile.sources | 1 +
tests/gem_scheduler.c | 421 +++++++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 863 insertions(+)
create mode 100644 lib/igt_bb_factory.c
create mode 100644 lib/igt_bb_factory.h
create mode 100644 tests/gem_scheduler.c
diff --git a/lib/Makefile.sources b/lib/Makefile.sources
index 1316fd2..c450db2 100644
--- a/lib/Makefile.sources
+++ b/lib/Makefile.sources
@@ -7,6 +7,8 @@ libintel_tools_la_SOURCES = \
i915_reg.h \
i915_pciids.h \
igt.h \
+ igt_bb_factory.c \
+ igt_bb_factory.h \
igt_debugfs.c \
igt_debugfs.h \
igt_aux.c \
diff --git a/lib/igt.h b/lib/igt.h
index d751f24..be87915 100644
--- a/lib/igt.h
+++ b/lib/igt.h
@@ -37,6 +37,7 @@
#include "igt_kms.h"
#include "igt_pm.h"
#include "igt_stats.h"
+#include "igt_bb_factory.h"
#include "instdone.h"
#include "intel_batchbuffer.h"
#include "intel_chipset.h"
diff --git a/lib/igt_bb_factory.c b/lib/igt_bb_factory.c
new file mode 100644
index 0000000..064378a
--- /dev/null
+++ b/lib/igt_bb_factory.c
@@ -0,0 +1,391 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ * Derek Morton <derek.j.morton@intel.com>
+ *
+ */
+
+#include "igt.h"
+#include "intel_batchbuffer.h"
+#include <stdint.h>
+#include <inttypes.h>
+#include <time.h>
+
+#define SEC_TO_NSEC (1000 * 1000 * 1000)
+#define DWORDS_TO_BYTES(x) ((x)*4)
+
+#define MI_STORE_REGISTER_MEM(LENGTH) ((0x024 << 23) | ((LENGTH - 2) & 0xff))
+#define MI_MATH(NrInst) ((0x01A << 23) | ((NrInst - 1) & 0x3f))
+#define MI_CONDITIONAL_BATCH_BUFFER_END ((0x036 << 23) | (1 << 21) | 2)
+#define MI_COPY_MEM_MEM ((0x02E << 23) | (3))
+
+#define ALU_LOAD(TO, FROM) ((0x080 << 20) | ((TO) << 10) | (FROM))
+#define ALU_SUB ( 0x101 << 20)
+#define ALU_STORE(TO, FROM) ((0x180 << 20) | ((TO) << 10) | (FROM))
+
+#define TIMESTAMP_offset (0x358) /* Elapsed time from system start */
+#define CTX_TIMESTAMP_offset (0x3A8) /* Elapsed Time from context creation */
+#define ALU_GPU_R0_LSB_offset (0x600)
+#define ALU_GPU_R0_MSB_offset (0x604)
+#define ALU_GPU_R1_LSB_offset (0x608)
+#define ALU_GPU_R1_MSB_offset (0x60C)
+#define ALU_GPU_R2_LSB_offset (0x610)
+#define ALU_GPU_R2_MSB_offset (0x614)
+
+#define ALU_R0_ENCODING (0x00)
+#define ALU_R1_ENCODING (0x01)
+#define ALU_SRCA_ENCODING (0x20)
+#define ALU_SRCB_ENCODING (0x21)
+#define ALU_ACCU_ENCODING (0x31)
+
+/**
+ * SECTION:igt_bb_factory
+ * @short_description: Utility functions for creating batch buffers
+ * @title: Batch Buffer Factory
+ * @include: igt.h
+ *
+ * This library implements functions for creating batch buffers which may be
+ * useful to multiple tests.
+ */
+
+static void check_gen_8(int fd)
+{
+ static bool checked = false;
+ if(!checked) {
+ igt_require(intel_gen(intel_get_drm_devid(fd)) >= 8);
+ checked = true;
+ }
+}
+
+static int bb_address_size_dw(int fd)
+{
+ if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+ return 2;
+ else
+ return 1;
+}
+
+static uint32_t get_register_offset(int ringid)
+{
+ switch (ringid) {
+ case I915_EXEC_RENDER:
+ return 0x02000;
+ case I915_EXEC_BSD:
+ return 0x12000;
+ case I915_EXEC_BLT:
+ return 0x22000;
+ case I915_EXEC_VEBOX:
+ return 0x1A000;
+ default:
+ igt_assert_f(0, "Invalid ringid %d passed to get_register_offset()\n", ringid);
+ }
+}
+
+/**
+ * igt_create_delay_bb:
+ * @fd: file descriptor for i915 driver instance
+ * @bufmgr: Buffer manager to be used for creation of batch buffers
+ * ringid: Ring to create batch buffer for. e.g. I915_EXEC_RENDER
+ * loops: Number of times to loop
+ * dest: Buffer to use for saving the current loop count and timestamp.
+ *
+ * This creates a batch buffer which will itterate a loop a specified number
+ * of times. Intended for creating batch buffers which take an arbitarily
+ * long time to execute. This can be useful to keep a ring busy while other
+ * batch buffers are queued when testing batch execution order.
+ *
+ * The dest buffer will have a number of Dwords written by the batch buffer
+ * when it runs. They are:
+ * DW0 & DW1 - Counter LSB. Will be 0 if the batch buffer finished succesfully.
+ * DW2 Timestamp - Elapsed time since system start when batch buffer ran.
+ *
+ * Returns:
+ * The struct intel_batchbuffer created.
+ */
+struct intel_batchbuffer *igt_create_delay_bb(int fd, drm_intel_bufmgr *bufmgr,
+ int ringid, uint32_t loops, drm_intel_bo *dest)
+{
+ struct intel_batchbuffer *batch;
+ int addr_size_dw;
+ uint32_t regOffset;
+
+ check_gen_8(fd);
+
+ addr_size_dw = bb_address_size_dw(fd);
+ regOffset = get_register_offset(ringid);
+ batch = intel_batchbuffer_alloc(bufmgr, intel_get_drm_devid(fd));
+ igt_assert(batch);
+
+ BEGIN_BATCH(32, 5);
+ /* store current timestamp in DW2 */
+ OUT_BATCH(MI_STORE_REGISTER_MEM(addr_size_dw + 2));
+ OUT_BATCH(regOffset + TIMESTAMP_offset);
+ OUT_RELOC(dest, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, DWORDS_TO_BYTES(2));
+
+ /* Load R0 with loops */
+ OUT_BATCH(MI_LOAD_REGISTER_IMM);
+ OUT_BATCH(regOffset + ALU_GPU_R0_LSB_offset);
+ OUT_BATCH(loops);
+ OUT_BATCH(MI_LOAD_REGISTER_IMM);
+ OUT_BATCH(regOffset + ALU_GPU_R0_MSB_offset);
+ OUT_BATCH(0x00000000);
+ /* Load R1 with 1 */
+ OUT_BATCH(MI_LOAD_REGISTER_IMM);
+ OUT_BATCH(regOffset + ALU_GPU_R1_LSB_offset);
+ OUT_BATCH(0x00000001);
+ OUT_BATCH(MI_LOAD_REGISTER_IMM);
+ OUT_BATCH(regOffset + ALU_GPU_R1_MSB_offset);
+ OUT_BATCH(0x00000000);
+ /* Copy R0 / R1 into SRCA / SRCB, Perform R0 - R1, Store result in R0 */
+ /* e.g. R0 -= 1 */
+ OUT_BATCH(MI_MATH(4));
+ OUT_BATCH(ALU_LOAD(ALU_SRCA_ENCODING, ALU_R0_ENCODING));
+ OUT_BATCH(ALU_LOAD(ALU_SRCB_ENCODING, ALU_R1_ENCODING));
+ OUT_BATCH(ALU_SUB);
+ OUT_BATCH(ALU_STORE(ALU_R0_ENCODING, ALU_ACCU_ENCODING));
+ /* Copy R0 to dest
+ * On Gen8 MI_CONDITIONAL_BATCH_BUFFER_END BSD ring Compare address
+ * points to 2 Dwords, a mask (DW0) and data (DW1) which are ANDed
+ * together.
+ * On Gen9+, and the other rings on Gen8 Compare address points to
+ * just Data (DW0). For simplicity always copy R0 LSB to DW0 and DW1.
+ */
+ OUT_BATCH(MI_STORE_REGISTER_MEM(addr_size_dw + 2));
+ OUT_BATCH(regOffset + ALU_GPU_R0_LSB_offset);
+ OUT_RELOC(dest, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, 0);
+ OUT_BATCH(MI_STORE_REGISTER_MEM(addr_size_dw + 2));
+ OUT_BATCH(regOffset + ALU_GPU_R0_LSB_offset);
+ OUT_RELOC(dest, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, DWORDS_TO_BYTES(1));
+ /* Repeat until R0 == 0 */
+ OUT_BATCH(MI_CONDITIONAL_BATCH_BUFFER_END);
+ OUT_BATCH(0x00000000);
+ OUT_RELOC(dest, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, 0);
+ OUT_BATCH(MI_BATCH_BUFFER_START | (addr_size_dw - 1));
+ OUT_RELOC(batch->bo, I915_GEM_DOMAIN_INSTRUCTION, 0, DWORDS_TO_BYTES(15));
+ /* Should never get here, but end if it happens */
+
+ OUT_BATCH(MI_BATCH_BUFFER_END);
+ ADVANCE_BATCH();
+
+ return batch;
+}
+
+/**
+ * igt_create_timestamp_bb:
+ * @fd: file descriptor for i915 driver instance
+ * @bufmgr: Buffer manager to be used for creation of batch buffers
+ * ringid: Ring to create batch buffer for. e.g. I915_EXEC_RENDER
+ * dest: Buffer to use for saving the timestamps.
+ * load: Buffer to access. Set NULL if not required.
+ * write: If true and load is not NULL, will also write a timestamp to load
+ * buffer. If false and load is not NULL, will read from load buffer into dest.
+ * Intended for dependency checking.
+ *
+ * This creates a batch buffer which writes timestamps into a buffer object.
+ *
+ * The dest buffer will have a number of Dwords written by the batch buffer
+ * when it runs. They are:
+ * DW0 Reported timestamp - Elapsed time since system start.
+ * DW1 Context timestamp - Elapsed time since context was created.
+ * DW2 Value copied from DW0 of load if write == false
+ *
+ * Returns:
+ * The struct intel_batchbuffer created.
+ */
+struct intel_batchbuffer *igt_create_timestamp_bb(int fd, drm_intel_bufmgr *bufmgr,
+ int ringid, drm_intel_bo *dest, drm_intel_bo *load, bool write)
+{
+ struct intel_batchbuffer *batch;
+ int addr_size_dw;
+ uint32_t regOffset;
+
+ check_gen_8(fd);
+
+ addr_size_dw = bb_address_size_dw(fd);
+ regOffset = get_register_offset(ringid);
+ batch = intel_batchbuffer_alloc(bufmgr, intel_get_drm_devid(fd));
+ igt_assert(batch);
+
+ BEGIN_BATCH(6, 2);
+ /* store current reported timestamp in DW0 */
+ OUT_BATCH(MI_STORE_REGISTER_MEM(addr_size_dw + 2));
+ OUT_BATCH(regOffset + TIMESTAMP_offset);
+ OUT_RELOC(dest, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, DWORDS_TO_BYTES(0));
+
+ /* store current context timestamp in DW1 */
+ OUT_BATCH(MI_STORE_REGISTER_MEM(addr_size_dw + 2));
+ OUT_BATCH(regOffset + CTX_TIMESTAMP_offset);
+ OUT_RELOC(dest, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, DWORDS_TO_BYTES(1));
+
+ ADVANCE_BATCH();
+
+ if(load != NULL) {
+ if(write) {
+ BEGIN_BATCH(3, 1);
+ OUT_BATCH(MI_STORE_REGISTER_MEM(addr_size_dw + 2));
+ OUT_BATCH(regOffset + TIMESTAMP_offset);
+ OUT_RELOC(load, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, DWORDS_TO_BYTES(0));
+ ADVANCE_BATCH();
+ }
+ else {
+ BEGIN_BATCH(3, 2);
+ OUT_BATCH(MI_COPY_MEM_MEM);
+ OUT_RELOC(dest, I915_GEM_DOMAIN_INSTRUCTION, I915_GEM_DOMAIN_INSTRUCTION, DWORDS_TO_BYTES(2));
+ OUT_RELOC(load, I915_GEM_DOMAIN_INSTRUCTION, 0, DWORDS_TO_BYTES(0));
+ ADVANCE_BATCH();
+ }
+ }
+
+ BEGIN_BATCH(1, 0);
+ OUT_BATCH(MI_BATCH_BUFFER_END);
+ ADVANCE_BATCH();
+
+ return batch;
+}
+
+/**
+ * igt_create_noop_bb:
+ * @fd: file descriptor for i915 driver instance
+ * @bufmgr: Buffer manager to be used for creation of batch buffers
+ * ringid: Ring to create batch buffer for. e.g. I915_EXEC_RENDER
+ * noops: Number of MI_NOOP instructions to add to the batch buffer.
+ *
+ * This creates a batch buffer with a specified number of MI_NOOP instructions.
+ *
+ * Returns:
+ * The struct intel_batchbuffer created.
+ */
+struct intel_batchbuffer *igt_create_noop_bb(int fd, drm_intel_bufmgr *bufmgr,
+ int ringid, int noops)
+{
+ struct intel_batchbuffer *batch;
+ int loop;
+
+ batch = intel_batchbuffer_alloc(bufmgr, intel_get_drm_devid(fd));
+ igt_assert(batch);
+
+ BEGIN_BATCH(noops + 1, 0);
+ for(loop = 0; loop < noops; loop++)
+ OUT_BATCH(MI_NOOP);
+ OUT_BATCH(MI_BATCH_BUFFER_END);
+ ADVANCE_BATCH();
+
+ return batch;
+}
+
+/* Store calibrated values so they only need calculating once.
+ * I915_EXEC_RING_MASK allows 3 bits for ring ids so allow for storing 8 values */
+static uint32_t calibrated_ring_value[8] = {0, 0, 0, 0, 0, 0, 0, 0};
+
+/**
+ * igt_calibrate_delay_bb:
+ * @fd: file descriptor for i915 driver instance
+ * @bufmgr: Buffer manager to be used for creation of batch buffers
+ * ringid: Ring to calibrate. e.g. I915_EXEC_RENDER
+ *
+ * This calculates the value of loops that would need to be passed to
+ * igt_create_delay_bb() to create a delay of about 1 second on the specified
+ * ring.
+ *
+ * Returns:
+ * uint32_t to be passed to igt_create_delay_bb().
+ */
+uint32_t igt_calibrate_delay_bb(int fd, drm_intel_bufmgr *bufmgr, int ringid)
+{
+ uint32_t *buf;
+ struct intel_batchbuffer *bb;
+ struct timespec start, end;
+ uint64_t duration;
+ uint64_t calibrated;
+ uint32_t regOffset = get_register_offset(ringid);
+ drm_intel_bo *target_bo;
+
+ igt_assert(ringid < 8);
+ if(calibrated_ring_value[ringid] != 0)
+ return calibrated_ring_value[ringid];
+
+ target_bo = drm_intel_bo_alloc(bufmgr, "target bo", 4096, 4096);
+ igt_assert(target_bo);
+
+ /* Put some non zero values in the target bo */
+ drm_intel_bo_map(target_bo, 1);
+ buf = target_bo->virtual;
+ buf[0] = 0xff;
+ drm_intel_bo_unmap(target_bo);
+
+ bb = igt_create_delay_bb(fd, bufmgr, ringid, 0x100000, target_bo);
+
+ gem_quiescent_gpu(fd);
+ clock_gettime(CLOCK_MONOTONIC, &start);
+ intel_batchbuffer_flush_on_ring(bb, ringid);
+ /* This will not return until the bo has finished executing */
+ drm_intel_bo_map(target_bo, 0);
+ clock_gettime(CLOCK_MONOTONIC, &end);
+
+ buf = target_bo->virtual;
+ /* buf[0] in the target buffer should be 0 if the batch buffer completed */
+ igt_assert_f(buf[0] == 0, "buf[0] expected 0x0, got 0x%x\n", buf[0]);
+
+ duration = (((end.tv_sec - start.tv_sec) * SEC_TO_NSEC) + end.tv_nsec) - start.tv_nsec;
+ calibrated = (((uint64_t)(0x100000) * SEC_TO_NSEC) / duration);
+ igt_debug("Uncalibrated run took %" PRIu64 ".%04" PRIu64 "s\n",
+ duration / SEC_TO_NSEC,
+ (duration % SEC_TO_NSEC) / 100000);
+ drm_intel_bo_unreference(target_bo);
+ intel_batchbuffer_free(bb);
+
+ /* Sanity check. If duration < 10ms, something has clearly gone wrong */
+ igt_assert(duration > (SEC_TO_NSEC / 100));
+
+ if (calibrated > 0xffffffff) {
+ igt_warn("Truncating to max uint32\n");
+ return 0xffffffff;
+ }
+
+ calibrated_ring_value[ringid] = (uint32_t)calibrated;
+ return (uint32_t)calibrated;
+}
+
+/**
+ * igt_compare_timestamps:
+ * @ts1: timestamp 1
+ * @ts2: timestamp 2
+ *
+ * This compares two uint32_t timestamps. To handle wrapping it assumes the
+ * difference between the two timestamps is less than 1/4 the max elapsed time
+ * represented by the counters.
+ * It also assumes the timestamps are samples from the same counter.
+ *
+ * Returns:
+ * True if ts2 > ts1, allowing for counter wrapping, false otherwise.
+ */
+
+bool igt_compare_timestamps(uint32_t ts1, uint32_t ts2)
+{
+ if (ts2 > ts1)
+ return true;
+ else if ((ts1 > 0x80000000) && (ts2 < 0x40000000))
+ return true; /* Assuming timestamp counter wrapped */
+ else
+ return false;
+}
diff --git a/lib/igt_bb_factory.h b/lib/igt_bb_factory.h
new file mode 100644
index 0000000..3ab7f13
--- /dev/null
+++ b/lib/igt_bb_factory.h
@@ -0,0 +1,47 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ * Derek Morton <derek.j.morton@intel.com>
+ *
+ */
+
+#ifndef IGT_BB_FACTORY_H
+#define IGT_BB_FACTORY_H
+
+#include "intel_batchbuffer.h"
+#include <stdint.h>
+
+struct intel_batchbuffer *igt_create_delay_bb(int fd, drm_intel_bufmgr *bufmgr,
+ int ringid, uint32_t loops, drm_intel_bo *dest);
+
+struct intel_batchbuffer *igt_create_timestamp_bb(int fd, drm_intel_bufmgr *bufmgr,
+ int ringid, drm_intel_bo *dest, drm_intel_bo *load, bool write);
+
+struct intel_batchbuffer *igt_create_noop_bb(int fd, drm_intel_bufmgr *bufmgr,
+ int ringid, int noops);
+
+uint32_t igt_calibrate_delay_bb(int fd, drm_intel_bufmgr *bufmgr, int ringid);
+
+bool igt_compare_timestamps(uint32_t ts1, uint32_t ts2);
+
+#endif /* IGT_BB_FACTORY_H */
diff --git a/tests/Makefile.sources b/tests/Makefile.sources
index c52be2c..6acf2f6 100644
--- a/tests/Makefile.sources
+++ b/tests/Makefile.sources
@@ -69,6 +69,7 @@ TESTS_progs_M = \
gem_request_retire \
gem_reset_stats \
gem_ringfill \
+ gem_scheduler \
gem_set_tiling_vs_blt \
gem_shrink \
gem_softpin \
diff --git a/tests/gem_scheduler.c b/tests/gem_scheduler.c
new file mode 100644
index 0000000..b108343
--- /dev/null
+++ b/tests/gem_scheduler.c
@@ -0,0 +1,421 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ * Derek Morton <derek.j.morton@intel.com>
+ *
+ */
+
+#include "igt.h"
+#include <unistd.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <inttypes.h>
+#include <time.h>
+#include <sys/stat.h>
+#include <sys/ioctl.h>
+#include <fcntl.h>
+
+IGT_TEST_DESCRIPTION("Check scheduler behaviour. Basic tests ensure independant "
+ "batch buffers of the same priority are executed in "
+ "submission order. Priority tests ensure higher priority "
+ "batch buffers are executed first. Read-read tests ensure "
+ "batch buffers with a read dependency to the same buffer "
+ "object do not block each other. Write-write dependency "
+ "tests ensure batch buffers with a write dependency to a "
+ "buffer object will be executed in submission order but "
+ "will not block execution of other independant batch "
+ "buffers.");
+
+#define SEC_TO_NSEC (1000 * 1000 * 1000)
+
+struct ring {
+ const char *name;
+ int id;
+} rings[] = {
+ { "render", I915_EXEC_RENDER },
+ { "bsd", I915_EXEC_BSD },
+ { "blt", I915_EXEC_BLT },
+ { "vebox", I915_EXEC_VEBOX },
+ { NULL, 0 },
+};
+
+/* Basic test. Check batch buffers of the same priority and with no dependencies
+ * are executed in the order they are submitted.
+ */
+static void run_test_basic(int in_flight, int ringid, bool priority)
+{
+ int fd[3];
+ int loop;
+ drm_intel_bufmgr *bufmgr[3];
+ uint32_t *delay_buf, *ts1_buf, *ts2_buf;
+ struct intel_batchbuffer *ts1_bb, *ts2_bb;
+ struct intel_batchbuffer **in_flight_bbs;
+ uint32_t calibrated_1s;
+ drm_intel_bo *delay_bo, *ts1_bo, *ts2_bo;
+
+ in_flight_bbs = malloc(in_flight * sizeof(struct intel_batchbuffer *));
+ igt_assert(in_flight_bbs);
+
+ /* Need multiple i915 fd's. Scheduler will not change execution order of
+ * batch buffers from the same context.
+ */
+ for(loop=0; loop < 3; loop++) {
+ struct intel_batchbuffer *noop_bb;
+ fd[loop] = drm_open_driver(DRIVER_INTEL);
+ igt_assert(fd[loop] >= 0);
+ bufmgr[loop] = drm_intel_bufmgr_gem_init(fd[loop], 4096);
+ igt_assert(bufmgr[loop]);
+ drm_intel_bufmgr_gem_enable_reuse(bufmgr[loop]);
+ /* Send a noop batch buffer to force any deferred initialisation */
+ noop_bb = igt_create_noop_bb(fd[loop], bufmgr[loop], ringid, 5);
+ intel_batchbuffer_flush_on_ring(noop_bb, ringid);
+ intel_batchbuffer_free(noop_bb);
+ }
+
+ if(priority) {
+ struct local_i915_gem_context_param param;
+ param.context = 0; /* Default context */
+ param.size = 0;
+ param.param = LOCAL_CONTEXT_PARAM_PRIORITY;
+ param.value = 1000;
+ gem_context_set_param(fd[2], ¶m);
+ }
+
+ /* Create buffer objects */
+ delay_bo = drm_intel_bo_alloc(bufmgr[0], "delay bo", 4096, 4096);
+ igt_assert(delay_bo);
+ ts1_bo = drm_intel_bo_alloc(bufmgr[1], "ts1 bo", 4096, 4096);
+ igt_assert(ts1_bo);
+ ts2_bo = drm_intel_bo_alloc(bufmgr[2], "ts2 bo", 4096, 4096);
+ igt_assert(ts2_bo);
+
+ /* Put some non zero values in the delay bo */
+ drm_intel_bo_map(delay_bo, 1);
+ delay_buf = delay_bo->virtual;
+ delay_buf[0] = 0xff;
+ drm_intel_bo_unmap(delay_bo);
+
+ calibrated_1s = igt_calibrate_delay_bb(fd[0], bufmgr[0], ringid);
+
+ /* Batch buffers to fill the ring */
+ in_flight_bbs[0] = igt_create_delay_bb(fd[0], bufmgr[0], ringid, calibrated_1s, delay_bo);
+ for(loop = 1; loop < in_flight; loop++)
+ in_flight_bbs[loop] = igt_create_noop_bb(fd[0], bufmgr[0], ringid, 5);
+
+ /* Extra batch buffers in the scheduler queue */
+ ts1_bb = igt_create_timestamp_bb(fd[1], bufmgr[1], ringid, ts1_bo, NULL, false);
+ ts2_bb = igt_create_timestamp_bb(fd[2], bufmgr[2], ringid, ts2_bo, NULL, false);
+
+ /* Flush batchbuffers */
+ for(loop = 0; loop < in_flight; loop++)
+ intel_batchbuffer_flush_on_ring(in_flight_bbs[loop], ringid);
+ intel_batchbuffer_flush_on_ring(ts1_bb, ringid);
+ intel_batchbuffer_flush_on_ring(ts2_bb, ringid);
+
+ /* This will not return until the bo has finished executing */
+ drm_intel_bo_map(delay_bo, 0);
+ drm_intel_bo_map(ts1_bo, 0);
+ drm_intel_bo_map(ts2_bo, 0);
+
+ delay_buf = delay_bo->virtual;
+ ts1_buf = ts1_bo->virtual;
+ ts2_buf = ts2_bo->virtual;
+
+ igt_debug("Delay Timestamp = 0x%08" PRIx32 "\n", delay_buf[2]);
+ igt_debug("TS1 Timestamp = 0x%08" PRIx32 "\n", ts1_buf[0]);
+ igt_debug("TS2 Timestamp = 0x%08" PRIx32 "\n", ts2_buf[0]);
+
+ /* buf[0] in the target buffer should be 0 if the batch buffer completed */
+ igt_assert_f(delay_buf[0] == 0,
+ "delay_buf[0] expected 0x0, got 0x%" PRIx32 "\n", delay_buf[0]);
+
+ igt_assert_f(igt_compare_timestamps(delay_buf[2], ts1_buf[0]),
+ "Delay ts (0x%08" PRIx32 ") > TS1 ts (0x%08" PRIx32 ")\n",
+ delay_buf[2], ts1_buf[0]);
+ if(priority)
+ igt_assert_f(igt_compare_timestamps(ts2_buf[0], ts1_buf[0]),
+ "TS2 ts (0x%08" PRIx32 ") > TS1 ts (0x%08" PRIx32 ")\n",
+ ts2_buf[0], ts1_buf[0]);
+ else
+ igt_assert_f(igt_compare_timestamps(ts1_buf[0], ts2_buf[0]),
+ "TS1 ts (0x%08" PRIx32 ") > TS2 ts (0x%08" PRIx32 ")\n",
+ ts1_buf[0], ts2_buf[0]);
+
+ /* Cleanup */
+ for(loop = 0; loop < in_flight; loop++)
+ intel_batchbuffer_free(in_flight_bbs[loop]);
+ intel_batchbuffer_free(ts1_bb);
+ intel_batchbuffer_free(ts2_bb);
+
+ drm_intel_bo_unreference(delay_bo);
+ drm_intel_bo_unreference(ts1_bo);
+ drm_intel_bo_unreference(ts2_bo);
+ for(loop = 0; loop < 3; loop++) {
+ drm_intel_bufmgr_destroy(bufmgr[loop]);
+ close(fd[loop]);
+ }
+ free(in_flight_bbs);
+}
+
+/* Dependency test.
+ * write=0, Submit batch buffers with read dependencies to all rings. Delay one
+ * with a long executing batch buffer. Check the others are not held up.
+ * write=1, Submit batch buffers with write dependencies to all rings. Delay one
+ * with a long executing batch buffer. Also submit batch buffers with no
+ * dependencies to all rings. Batch buffers with write dependencies should be
+ * executed in submission order. The batch buffers with no dependencies should
+ * not be held up.
+ */
+static void run_test_dependency(int in_flight, int ring, bool write)
+{
+ int fd[4], fd2[4];
+ int loop;
+ int prime_fd;
+ uint32_t *delay_buf, *ts_buf[4], *ts2_buf[4], *shared_buf;
+ uint32_t calibrated_1s;
+ drm_intel_bufmgr *bufmgr[4], *bufmgr2[4];
+ struct intel_batchbuffer *ts_bb[4], *ts2_bb[4], **in_flight_bbs;
+ drm_intel_bo *delay_bo, *ts_bo[4], *ts2_bo[4], *shared_bo[4];
+
+ in_flight_bbs = malloc(in_flight * sizeof(struct intel_batchbuffer *));
+ igt_assert(in_flight_bbs);
+
+ /* Need multiple i915 fd's. Scheduler will not change execution order of
+ * batch buffers from the same context.
+ */
+ for(loop=0; loop < 4; loop++) {
+ struct intel_batchbuffer *noop_bb;
+ fd[loop] = drm_open_driver(DRIVER_INTEL);
+ igt_assert(fd[loop] >= 0);
+ bufmgr[loop] = drm_intel_bufmgr_gem_init(fd[loop], 4096);
+ igt_assert(bufmgr[loop]);
+ drm_intel_bufmgr_gem_enable_reuse(bufmgr[loop]);
+ /* Send a noop batch buffer to force any deferred initialisation */
+ noop_bb = igt_create_noop_bb(fd[loop], bufmgr[loop], rings[loop].id, 5);
+ intel_batchbuffer_flush_on_ring(noop_bb, rings[loop].id);
+ intel_batchbuffer_free(noop_bb);
+ if(write) {
+ struct intel_batchbuffer *noop_bb2;
+ fd2[loop] = drm_open_driver(DRIVER_INTEL);
+ igt_assert(fd2[loop] >= 0);
+ bufmgr2[loop] = drm_intel_bufmgr_gem_init(fd2[loop], 4096);
+ igt_assert(bufmgr2[loop]);
+ drm_intel_bufmgr_gem_enable_reuse(bufmgr2[loop]);
+ /* Send a noop batch buffer to force any deferred initialisation */
+ noop_bb2 = igt_create_noop_bb(fd2[loop], bufmgr2[loop], rings[loop].id, 5);
+ intel_batchbuffer_flush_on_ring(noop_bb2, rings[loop].id);
+ intel_batchbuffer_free(noop_bb2);
+ }
+ }
+
+ /* Create buffer objects */
+ delay_bo = drm_intel_bo_alloc(bufmgr[ring], "delay bo", 4096, 4096);
+ igt_assert(delay_bo);
+ for(loop = 0; loop < 4; loop++) {
+ ts_bo[loop] = drm_intel_bo_alloc(bufmgr[loop], "ts bo", 4096, 4096);
+ igt_assert(ts_bo[loop]);
+ if(write) {
+ ts2_bo[loop] = drm_intel_bo_alloc(bufmgr2[loop], "ts bo", 4096, 4096);
+ igt_assert(ts2_bo[loop]);
+ }
+ }
+
+ /* Create shared buffer object */
+ shared_bo[0] = drm_intel_bo_alloc(bufmgr[0], "shared bo", 4096, 4096);
+ igt_assert(shared_bo[0]);
+
+ drm_intel_bo_gem_export_to_prime(shared_bo[0], &prime_fd);
+ for(loop = 1; loop < 4; loop++) {
+ shared_bo[loop] = drm_intel_bo_gem_create_from_prime(bufmgr[loop],
+ prime_fd, 4096);
+ igt_assert(shared_bo[loop]);
+ }
+ close(prime_fd);
+
+ /* Put some non zero values in the delay and shared bo */
+ drm_intel_bo_map(delay_bo, 1);
+ delay_buf = delay_bo->virtual;
+ delay_buf[0] = 0xff;
+ drm_intel_bo_unmap(delay_bo);
+ drm_intel_bo_map(shared_bo[0], 1);
+ shared_buf = shared_bo[0]->virtual;
+ shared_buf[0] = 0xff00ff00;
+ drm_intel_bo_unmap(shared_bo[0]);
+
+ calibrated_1s = igt_calibrate_delay_bb(fd[ring], bufmgr[ring], rings[ring].id);
+
+ /* Batch buffers to fill the ring */
+ in_flight_bbs[0] = igt_create_delay_bb(fd[ring], bufmgr[ring], rings[ring].id, calibrated_1s, delay_bo);
+ for(loop = 1; loop < in_flight; loop++)
+ in_flight_bbs[loop] = igt_create_noop_bb(fd[ring], bufmgr[ring], rings[ring].id, 5);
+
+ for(loop = 0; loop < 4; loop++) {
+ ts_bb[loop] = igt_create_timestamp_bb(fd[loop], bufmgr[loop], rings[loop].id, ts_bo[loop], shared_bo[loop], write);
+ if(write)
+ ts2_bb[loop] = igt_create_timestamp_bb(fd2[loop], bufmgr2[loop], rings[loop].id, ts2_bo[loop], NULL, false);
+ }
+
+ /* Flush batchbuffers */
+ for(loop = 0; loop < in_flight; loop++)
+ intel_batchbuffer_flush_on_ring(in_flight_bbs[loop], rings[ring].id);
+
+ intel_batchbuffer_flush_on_ring(ts_bb[ring], rings[ring].id);
+ for(loop = 0; loop < 4; loop++)
+ if(loop != ring)
+ intel_batchbuffer_flush_on_ring(ts_bb[loop], rings[loop].id);
+
+ if(write) {
+ intel_batchbuffer_flush_on_ring(ts2_bb[ring], rings[ring].id);
+ for(loop = 0; loop < 4; loop++)
+ if(loop != ring)
+ intel_batchbuffer_flush_on_ring(ts2_bb[loop], rings[loop].id);
+ }
+
+ /* This will not return until the bo has finished executing */
+ drm_intel_bo_map(delay_bo, 0);
+ delay_buf = delay_bo->virtual;
+ for(loop = 0; loop < 4; loop++) {
+ drm_intel_bo_map(ts_bo[loop], 0);
+ ts_buf[loop] = ts_bo[loop]->virtual;
+ if(write) {
+ drm_intel_bo_map(ts2_bo[loop], 0);
+ ts2_buf[loop] = ts2_bo[loop]->virtual;
+ }
+ }
+
+ /* buf[0] in the target buffer should be 0 if the batch buffer completed */
+ igt_assert_f(delay_buf[0] == 0, "delay_buf[0] expected 0x0, got 0x%" PRIx32 "\n", delay_buf[0]);
+
+ igt_debug("%6s delay timestamp = 0x%08" PRIx32 "\n", rings[ring].name, delay_buf[2]);
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[0].name, ts_buf[0][0]);
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[1].name, ts_buf[1][0]);
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[2].name, ts_buf[2][0]);
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[3].name, ts_buf[3][0]);
+ if(write) {
+ igt_debug("Independant batch buffers\n");
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[0].name, ts2_buf[0][0]);
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[1].name, ts2_buf[1][0]);
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[2].name, ts2_buf[2][0]);
+ igt_debug("%6s batch timestamp = 0x%08" PRIx32 "\n", rings[3].name, ts2_buf[3][0]);
+ }
+
+ for(loop = 0; loop < 4; loop++) {
+ if(loop != ring) {
+ if(write) {
+ /* Write dependency, delayed ring should run first */
+ igt_assert_f(igt_compare_timestamps(ts_buf[ring][0], ts_buf[loop][0]),
+ "%s ran before %s - 0x%08" PRIx32 " vs 0x%08" PRIx32 "\n",
+ rings[loop].name, rings[ring].name,
+ ts_buf[loop][0], ts_buf[ring][0]);
+ /* Second bb without dependency should run first */
+ igt_assert_f(igt_compare_timestamps(ts2_buf[loop][0], ts_buf[loop][0]),
+ "(%s) independant bb was held up - 0x%08" PRIx32 " vs 0x%08" PRIx32 "\n",
+ rings[loop].name, ts_buf[loop][0], ts2_buf[loop][0]);
+ }
+ else
+ /* Read dependency, delayed ring should run last */
+ igt_assert_f(igt_compare_timestamps(ts_buf[loop][0], ts_buf[ring][0]),
+ "%s ran after %s - 0x%08" PRIx32 " vs 0x%08" PRIx32 "\n",
+ rings[loop].name, rings[ring].name,
+ ts_buf[loop][0], ts_buf[ring][0]);
+ }
+ }
+
+ /* Cleanup */
+ for(loop = 0; loop < in_flight; loop++)
+ intel_batchbuffer_free(in_flight_bbs[loop]);
+
+ for(loop = 0; loop < 4; loop++) {
+ intel_batchbuffer_free(ts_bb[loop]);
+ drm_intel_bo_unreference(ts_bo[loop]);
+ drm_intel_bo_unreference(shared_bo[loop]);
+ if(write) {
+ intel_batchbuffer_free(ts2_bb[loop]);
+ drm_intel_bo_unreference(ts2_bo[loop]);
+ }
+ }
+
+ drm_intel_bo_unreference(delay_bo);
+
+ for(loop = 0; loop < 4; loop++) {
+ drm_intel_bufmgr_destroy(bufmgr[loop]);
+ close(fd[loop]);
+ if(write) {
+ drm_intel_bufmgr_destroy(bufmgr2[loop]);
+ close(fd2[loop]);
+ }
+ }
+
+ free(in_flight_bbs);
+}
+
+igt_main
+{
+ int loop;
+ int in_flight;
+
+
+ igt_fixture {
+ int debug_fd;
+ int l;
+ char buf[4];
+ /* Get nbr of batch buffers that the scheduler will queue in the
+ * HW. If this debugfs file does not exist there is no scheduler
+ * so skip the test.
+ */
+ debug_fd = igt_debugfs_open("i915_scheduler_min_flying", O_RDONLY);
+ igt_skip_on(debug_fd == -1);
+ l = read(debug_fd, buf, sizeof(buf)-1);
+ igt_assert(l > 0);
+ igt_assert(l < sizeof(buf));
+ buf[l] = '\0';
+ if(sscanf(buf, "0x%2x", &in_flight) != 1)
+ igt_assert_f(sscanf(buf, "%2d", &in_flight) == 1,
+ "Error reading from i915_scheduler_min_flying\n");
+ close(debug_fd);
+ igt_debug("in flight = %d\n", in_flight);
+ }
+
+ for (loop=0; rings[loop].name != NULL; loop++)
+ igt_subtest_f("%s-basic", rings[loop].name) {
+ run_test_basic(in_flight, rings[loop].id, false);
+ }
+
+ for (loop=0; rings[loop].name != NULL; loop++)
+ igt_subtest_f("%s-priority", rings[loop].name) {
+ run_test_basic(in_flight, rings[loop].id, true);
+ }
+
+ for (loop=0; rings[loop].name != NULL; loop++)
+ igt_subtest_f("%s-read-read", rings[loop].name) {
+ run_test_dependency(in_flight, loop, false);
+ }
+
+ for (loop=0; rings[loop].name != NULL; loop++)
+ igt_subtest_f("%s-write-write", rings[loop].name) {
+ run_test_dependency(in_flight, loop, true);
+ }
+
+}
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* ✓ Fi.CI.BAT: success for GPU scheduler for i915 driver (rev2)
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (36 preceding siblings ...)
2016-04-20 17:13 ` [PATCH 2/2] igt/gem_scheduler: Add gem_scheduler test John.C.Harrison
@ 2016-04-21 9:43 ` Patchwork
2016-04-22 15:37 ` [PATCH v6 00/34] GPU scheduler for i915 driver John Harrison
` (3 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: Patchwork @ 2016-04-21 9:43 UTC (permalink / raw)
To: John.C.Harrison; +Cc: intel-gfx
== Series Details ==
Series: GPU scheduler for i915 driver (rev2)
URL : https://patchwork.freedesktop.org/series/3585/
State : success
== Summary ==
Series 3585v2 GPU scheduler for i915 driver
http://patchwork.freedesktop.org/api/1.0/series/3585/revisions/2/mbox/
bdw-ultra total:194 pass:170 dwarn:0 dfail:0 fail:1 skip:23
bsw-nuc-2 total:193 pass:154 dwarn:0 dfail:0 fail:0 skip:39
byt-nuc total:193 pass:155 dwarn:0 dfail:0 fail:0 skip:38
ilk-hp8440p total:194 pass:137 dwarn:0 dfail:0 fail:0 skip:57
ivb-t430s total:194 pass:166 dwarn:0 dfail:0 fail:0 skip:28
skl-i7k-2 total:194 pass:168 dwarn:0 dfail:0 fail:1 skip:25
skl-nuci5 total:194 pass:183 dwarn:0 dfail:0 fail:0 skip:11
bdw-nuci7 failed to connect after reboot
snb-x220t failed to collect. IGT log at Patchwork_1965/snb-x220t/igt.log
Results at /archive/results/CI_IGT_test/Patchwork_1965/
a09f2af0c7698d8b10f95f12bd0aa245dd53017e drm-intel-nightly: 2016y-04m-21d-07h-18m-40s UTC integration manifest
227a282 drm/i915: Add total count to context status debugfs output
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [PATCH v6 00/34] GPU scheduler for i915 driver
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (37 preceding siblings ...)
2016-04-21 9:43 ` ✓ Fi.CI.BAT: success for GPU scheduler for i915 driver (rev2) Patchwork
@ 2016-04-22 15:37 ` John Harrison
2016-04-23 9:57 ` ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver (rev2) Patchwork
` (2 subsequent siblings)
41 siblings, 0 replies; 50+ messages in thread
From: John Harrison @ 2016-04-22 15:37 UTC (permalink / raw)
To: Intel-GFX
This patch series can be pulled from
'ssh://people.freedesktop.org/~johnharr/scheduler' as the 'scheduler'
branch.
On 20/04/2016 18:13, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Implemented a batch buffer submission scheduler for the i915 DRM driver.
>
> The general theory of operation is that when batch buffers are
> submitted to the driver, the execbuffer() code assigns a unique seqno
> value and then packages up all the information required to execute the
> batch buffer at a later time. This package is given over to the
> scheduler which adds it to an internal node list. The scheduler also
> scans the list of objects associated with the batch buffer and
> compares them against the objects already in use by other buffers in
> the node list. If matches are found then the new batch buffer node is
> marked as being dependent upon the matching node. The same is done for
> the context object. The scheduler also bumps up the priority of such
> matching nodes on the grounds that the more dependencies a given batch
> buffer has the more important it is likely to be.
>
> The scheduler aims to have a given (tuneable) number of batch buffers
> in flight on the hardware at any given time. If fewer than this are
> currently executing when a new node is queued, then the node is passed
> straight through to the submit function. Otherwise it is simply added
> to the queue and the driver returns back to user land.
>
> As each batch buffer completes, it raises an interrupt which wakes up
> the scheduler. Note that it is possible for multiple buffers to
> complete before the IRQ handler gets to run. Further, the seqno values
> of the individual buffers are not necessary incrementing as the
> scheduler may have re-ordered their submission. However, the scheduler
> keeps the list of executing buffers in order of hardware submission.
> Thus it can scan through the list until a matching seqno is found and
> then mark all in flight nodes from that point on as completed.
>
> A deferred work queue is also poked by the interrupt handler. When
> this wakes up it can do more involved processing such as actually
> removing completed nodes from the queue and freeing up the resources
> associated with them (internal memory allocations, DRM object
> references, context reference, etc.). The work handler also checks the
> in flight count and calls the submission code if a new slot has
> appeared.
>
> When the scheduler's submit code is called, it scans the queued node
> list for the highest priority node that has no unmet dependencies.
> Note that the dependency calculation is complex as it must take
> inter-ring dependencies and potential preemptions into account. Note
> also that in the future this will be extended to include external
> dependencies such as the Android Native Sync file descriptors and/or
> the linux dma-buff synchronisation scheme.
>
> If a suitable node is found then it is sent to execbuff_final() for
> submission to the hardware. The in flight count is then re-checked and
> a new node popped from the list if appropriate.
>
> The scheduler also allows high priority batch buffers (e.g. from a
> desktop compositor) to jump ahead of whatever is already running if
> the underlying hardware supports pre-emption. In this situation, any
> work that was pre-empted is returned to the queued list ready to be
> resubmitted when no more high priority work is outstanding.
>
> Various IGT tests are in progress to test the scheduler's operation
> and will follow.
>
> v2: Updated for changes in struct fence patch series and other changes
> to underlying tree (e.g. removal of cliprects). Also changed priority
> levels to be signed +/-1023 range and reduced mutex lock usage.
>
> v3: More reuse of cached pointers rather than repeated dereferencing
> (David Gordon).
>
> Moved the dependency generation code out to a seperate function for
> easier readability. Also added in support for the read-read
> optimisation.
>
> Major simplification of the DRM file close handler.
>
> Fixed up an overzealous WARN.
>
> Removed unnecessary flushing of the scheduler queue when waiting for a
> request.
>
> v4: Removed user land fence/sync integration as this is dependent upon
> the de-staging of the Android sync code. That de-staging is now being
> done by someone else. The sync support will be added back in to the
> scheduler in a separate patch series which must wait for the
> de-staging to be landed first.
>
> Added support for killing batches from contexts that were banned after
> the batches were submitted to the scheduler.
>
> Changed various comments to fix typos, update to reflect changes to
> the code, correct formatting and line wrapping, etc. Also wrapped
> various long lines and twiddled white space to keep the style checker
> happy.
>
> Changed a bunch of BUG_ONs to WARN_ONs as apparently the latter are
> preferred.
>
> Used the correct array memory allocator function (kmalloc_array
> instead of kmalloc).
>
> Fixed a variable type (review comment by Joonas).
>
> Fixed a WARN_ON firing incorrectly when removing killed nodes from the
> scheduler's queue.
>
> Dropped register definition update patch from this series. The changes
> are all for pre-emption so it makes more sense for it to be part of
> that series instead.
>
> v5: Reverted power management changes as they apparently conflict with
> mutex acquisition. Converted the override mask module parameter to a
> set of boolean enable flags (just one in this patch set, but others
> are added later for controlling pre-emption). [Chris Wilson]
>
> Removed lots of whitespace from i915_scheduler.c and re-ordered it to
> remove all forward declarations. Squashed down the i915_scheduler.c
> sections of various patches into the initial 'start of scheduler'
> patch. Thus the later patches simply hook in existing code into
> various parts of the driver rather than adding the code as well. Added
> documentation to various functions. Re-worked the submit function in
> terms of mutex locking, error handling and exit paths. Split the
> delayed work handler function in half. Made use of the kernel 'clamp'
> macro. [Joonas Lahtinen]
>
> Dropped the 'immediate submission' override option. This was a half
> way house between full scheduler and direct submission and was only
> really useful during early debug.
>
> Added a re-install of the scheduler's interrupt hook around GPU reset.
> [Zhipeng Gong]
>
> Used lighter weight spinlocks.
>
> v6: Updated to newer nightly (lots of ring -> engine renaming).
>
> Added 'for_each_scheduler_node()' and 'assert_scheduler_lock_held()'
> helper macros. Renamed 'i915_gem_execbuff_release_batch_obj' to
> 'i915_gem_execbuf_release_batch_obj'. Updated to use 'to_i915()'
> instead of dev_private. Converted all enum labels to uppercase.
> Removed various unnecessary WARNs. Renamed 'saved_objects' to just
> 'objs'. More code refactoring. Removed even more white space. Added
> an i915_scheduler_destroy() function instead of doing explicit clean
> up of scheduler internals from i915_driver_unload(). Changed extra
> boolean i915_wait_request() parameter to a flags word and consumed the
> original boolean parameter too. Also, replaced the
> i915_scheduler_is_request_tracked() function with
> i915_scheduler_is_mutex_required() and
> i915_scheduler_is_request_batch_buffer() as the need for the former
> has gone away and it was really being used to ask the latter two
> questions in a convoluted manner. Wrapped boolean 'flush' parameter to
> intel_engine_idle() with an _flush() macro.
> [review feedback from Joonas Lahtinen]
>
> Moved scheduler modue parameter declaration to correct place in
> i915_params struct. [review feedback from Matt Roper]
>
> Added an admin only check when setting the tuning parameters via
> debugfs to prevent rogue user code trying to break the system with
> strange settings. [review feedback from Jesse Barnes]
>
> Added kerneldoc for intel_engine_idle().
>
> Added running totals of 'flying' and 'queued' nodes rather than
> re-calculating each time as a minor CPU performance optimisation.
>
> Removed support for out of order seqno completion. All the prep work
> patch series (seqno to request conversion, late seqno assignment,
> etc.) that has now been done means that the scheduler no longer
> generates out of order seqno completions. Thus all the complex code
> for coping with such is no longer required and can be removed.
>
> Fixed a bug in scheduler bypass mode introduced in the clean up code
> refactoring of v5. The clean up function was seeing the node in the
> wrong state and thus refusing to process it.
>
> Improved the throttle by file handle feature by chaning from a simple
> 'return to userland when full' scheme with a 'sleep on request'
> scheme. The former could lead to the busy polling and wasting lots of
> CPU time as user land continuously retried the execbuf IOCTL in a
> tight loop. Now the driver will sleep (without holding the mutex lock)
> on the oldest request outstanding for that file and then automatically
> retry. This is closer to the pre-scheduler behaviour of stalling on a
> full ring buffer.
>
> [Patches against drm-intel-nightly tree fetched 13/04/2016 with struct
> fence conversion patches applied]
>
> Dave Gordon (2):
> drm/i915: Cache request pointer in *_submission_final()
> drm/i915: Add scheduling priority to per-context parameters
>
> John Harrison (32):
> drm/i915: Add total count to context status debugfs output
> drm/i915: Prelude to splitting i915_gem_do_execbuffer in two
> drm/i915: Split i915_dem_do_execbuffer() in half
> drm/i915: Re-instate request->uniq because it is extremely useful
> drm/i915: Start of GPU scheduler
> drm/i915: Disable hardware semaphores when GPU scheduler is enabled
> drm/i915: Force MMIO flips when scheduler enabled
> drm/i915: Added scheduler hook when closing DRM file handles
> drm/i915: Added scheduler hook into i915_gem_request_notify()
> drm/i915: Added deferred work handler for scheduler
> drm/i915: Redirect execbuffer_final() via scheduler
> drm/i915: Keep the reserved space mechanism happy
> drm/i915: Added tracking/locking of batch buffer objects
> drm/i915: Hook scheduler node clean up into retire requests
> drm/i915: Added scheduler support to __wait_request() calls
> drm/i915: Added scheduler support to page fault handler
> drm/i915: Added scheduler flush calls to ring throttle and idle functions
> drm/i915: Add scheduler hook to GPU reset
> drm/i915: Added a module parameter to allow the scheduler to be disabled
> drm/i915: Support for 'unflushed' ring idle
> drm/i915: Defer seqno allocation until actual hardware submission time
> drm/i915: Added trace points to scheduler
> drm/i915: Added scheduler queue throttling by DRM file handle
> drm/i915: Added debugfs interface to scheduler tuning parameters
> drm/i915: Add early exit to execbuff_final() if insufficient ring space
> drm/i915: Added scheduler statistic reporting to debugfs
> drm/i915: Add scheduler support functions for TDR
> drm/i915: Enable GPU scheduler by default
> drm/i915: Add support for retro-actively banning batch buffers
> drm/i915: Allow scheduler to manage inter-ring object synchronisation
> drm/i915: Added debug state dump facilities to scheduler
> drm/i915: Scheduler state dump via debugfs
>
> drivers/gpu/drm/i915/Makefile | 1 +
> drivers/gpu/drm/i915/i915_debugfs.c | 336 +++++-
> drivers/gpu/drm/i915/i915_dma.c | 5 +
> drivers/gpu/drm/i915/i915_drv.c | 9 +
> drivers/gpu/drm/i915/i915_drv.h | 58 +-
> drivers/gpu/drm/i915/i915_gem.c | 156 ++-
> drivers/gpu/drm/i915/i915_gem_context.c | 24 +
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 297 +++--
> drivers/gpu/drm/i915/i915_params.c | 4 +
> drivers/gpu/drm/i915/i915_params.h | 1 +
> drivers/gpu/drm/i915/i915_scheduler.c | 1709 ++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 180 +++
> drivers/gpu/drm/i915/i915_trace.h | 225 +++-
> drivers/gpu/drm/i915/intel_display.c | 10 +-
> drivers/gpu/drm/i915/intel_lrc.c | 161 ++-
> drivers/gpu/drm/i915/intel_lrc.h | 1 +
> drivers/gpu/drm/i915/intel_ringbuffer.c | 69 +-
> drivers/gpu/drm/i915/intel_ringbuffer.h | 5 +-
> include/uapi/drm/i915_drm.h | 1 +
> 19 files changed, 3118 insertions(+), 134 deletions(-)
> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver (rev2)
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (38 preceding siblings ...)
2016-04-22 15:37 ` [PATCH v6 00/34] GPU scheduler for i915 driver John Harrison
@ 2016-04-23 9:57 ` Patchwork
2016-04-25 9:54 ` [PATCH v6 00/34] GPU scheduler for i915 driver Chris Wilson
2016-05-09 9:49 ` ✗ Fi.CI.BAT: warning for GPU scheduler for i915 driver (rev4) Patchwork
41 siblings, 0 replies; 50+ messages in thread
From: Patchwork @ 2016-04-23 9:57 UTC (permalink / raw)
To: John.C.Harrison; +Cc: intel-gfx
== Series Details ==
Series: GPU scheduler for i915 driver (rev2)
URL : https://patchwork.freedesktop.org/series/3585/
State : failure
== Summary ==
Series 3585v2 GPU scheduler for i915 driver
http://patchwork.freedesktop.org/api/1.0/series/3585/revisions/2/mbox/
Test kms_flip:
Subgroup basic-flip-vs-wf_vblank:
pass -> FAIL (snb-x220t)
Test kms_pipe_crc_basic:
Subgroup hang-read-crc-pipe-c:
skip -> INCOMPLETE (snb-dellxps)
bdw-nuci7 total:193 pass:181 dwarn:0 dfail:0 fail:0 skip:12
bdw-ultra total:193 pass:170 dwarn:0 dfail:0 fail:0 skip:23
bsw-nuc-2 total:192 pass:153 dwarn:0 dfail:0 fail:0 skip:39
byt-nuc total:192 pass:154 dwarn:0 dfail:0 fail:0 skip:38
ilk-hp8440p total:193 pass:136 dwarn:0 dfail:0 fail:0 skip:57
ivb-t430s total:193 pass:165 dwarn:0 dfail:0 fail:0 skip:28
skl-i7k-2 total:193 pass:168 dwarn:0 dfail:0 fail:0 skip:25
skl-nuci5 total:193 pass:182 dwarn:0 dfail:0 fail:0 skip:11
snb-dellxps total:45 pass:34 dwarn:0 dfail:0 fail:0 skip:10
snb-x220t total:193 pass:154 dwarn:0 dfail:0 fail:2 skip:37
Results at /archive/results/CI_IGT_test/Patchwork_2009/
340c485ad98d0ec0369a3b18d4a09938f3f5537d drm-intel-nightly: 2016y-04m-22d-17h-32m-25s UTC integration manifest
0d53aec drm/i915: Add total count to context status debugfs output
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [PATCH v6 00/34] GPU scheduler for i915 driver
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (39 preceding siblings ...)
2016-04-23 9:57 ` ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver (rev2) Patchwork
@ 2016-04-25 9:54 ` Chris Wilson
2016-04-25 11:55 ` John Harrison
2016-04-26 13:20 ` Daniel Vetter
2016-05-09 9:49 ` ✗ Fi.CI.BAT: warning for GPU scheduler for i915 driver (rev4) Patchwork
41 siblings, 2 replies; 50+ messages in thread
From: Chris Wilson @ 2016-04-25 9:54 UTC (permalink / raw)
To: John.C.Harrison; +Cc: Intel-GFX
On Wed, Apr 20, 2016 at 06:13:18PM +0100, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Implemented a batch buffer submission scheduler for the i915 DRM driver.
>
> The general theory of operation is that when batch buffers are
> submitted to the driver, the execbuffer() code assigns a unique seqno
> value and then packages up all the information required to execute the
> batch buffer at a later time. This package is given over to the
> scheduler which adds it to an internal node list.
That is called the request. Please use it properly. We pass a request to
the engine. The engine can hook up the scheduler in which case it may
defer execution of that request until the scheduler says it is ready.
But that is a conversation inside the engine, not intermingled with GEM.
GEM needs to only care about its unique seqno along the timeline. We can
leave space for the scheduler to inject a second seqno if that is
convenient, but since the scheduler can keep a global list of inflight
requests, its retirement is also globally ordered even if we only
inspect each request wrt to its own timeline.
> The scheduler also
> scans the list of objects associated with the batch buffer and
> compares them against the objects already in use by other buffers in
> the node list.
This part moves the policy of the GEM API and enshrines it into the
scheduler. Let GEM convert the implicit rules it has into explicit
dependencies, on to which we can add the explicit dependencies passed in
by the user. GEM has the information it needs to maintain its API, let's
not let that silliness spread.
> If matches are found then the new batch buffer node is
> marked as being dependent upon the matching node. The same is done for
> the context object. The scheduler also bumps up the priority of such
> matching nodes on the grounds that the more dependencies a given batch
> buffer has the more important it is likely to be.
Sounds like a rough cut at PI avoidance. This is easily abusable. It
sounds like you want a prio/nice split. Is that a good idea to introduce
in the first series?
> The scheduler aims to have a given (tuneable) number of batch buffers
> in flight on the hardware at any given time. If fewer than this are
> currently executing when a new node is queued, then the node is passed
> straight through to the submit function. Otherwise it is simply added
> to the queue and the driver returns back to user land.
Keep it in the scheduler, I see you decided to add EAGAIN to GEM execbuf
when currently execbuf *blocks* in this situation. Don't use EAGAIN here
since you just make userspace busyspin over a very, very heavyweight
ioctl, and you have the power here to throttle userspace without
blocking.
> As each batch buffer completes, it raises an interrupt which wakes up
> the scheduler. Note that it is possible for multiple buffers to
> complete before the IRQ handler gets to run. Further, the seqno values
> of the individual buffers are not necessary incrementing as the
> scheduler may have re-ordered their submission. However, the scheduler
> keeps the list of executing buffers in order of hardware submission.
> Thus it can scan through the list until a matching seqno is found and
> then mark all in flight nodes from that point on as completed.
No. You haven't built your timelines correctly. The idea behind the
timeline is that a request can wait upon its seqno and check it against
its own timeline, which is ordered only with other requests on its
timeline. Because of this, it is independent of whatever order the
scheduler executes the timelines in, each timeline is ordered.
A request can simply wait for its timeline to advance, completely
ignorant of the scheduler. (Request signaling may be driven by the
scheduler, but that is a lowlevel, not GEM or dma-buf/fence,
implementation detail. And only if the scheduler is coupled into the
user-interupt, but on execlists it will be using the context-switch
interrupt to driver itself, and for ringbuffer mode we have a choice of
user-interrupt or using pipe-control/dw-notify to keep the paths
separate.)
> A deferred work queue is also poked by the interrupt handler.
Wait you did all that in the interrupt handler. NO.
> If a suitable node is found then it is sent to execbuff_final() for
> submission to the hardware.
That is an absolutely atrocious implementation. Don't pass it back up
the layers when you already have all the information you need packaged in
the request to submit it to hardware.
execbuf -> engine->add_request -> [scheduler] -> engine->submit_request
(conceptually, since what should happen is execlists gets it
context-switch interrupt and then asks what set of requests to submit
next. In a ringbuffer mode, a kthread/ktasklet would run off the
interrupt ask which request to execute next and do the write into the
ring for the switch into the new context and execution of the next batch,
and then write the RING_TAIL.)
At the fundamental level it looks like you have not introduced timelines
correctly or introduced the scheduler as a separate entity for deciding
which request to execute next (or if this request should preempt execution).
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [PATCH v6 00/34] GPU scheduler for i915 driver
2016-04-25 9:54 ` [PATCH v6 00/34] GPU scheduler for i915 driver Chris Wilson
@ 2016-04-25 11:55 ` John Harrison
2016-04-26 13:20 ` Daniel Vetter
1 sibling, 0 replies; 50+ messages in thread
From: John Harrison @ 2016-04-25 11:55 UTC (permalink / raw)
To: Chris Wilson, Intel-GFX
On 25/04/2016 10:54, Chris Wilson wrote:
> On Wed, Apr 20, 2016 at 06:13:18PM +0100, John.C.Harrison@Intel.com wrote:
>> From: John Harrison <John.C.Harrison@Intel.com>
>>
>> Implemented a batch buffer submission scheduler for the i915 DRM driver.
>>
>> The general theory of operation is that when batch buffers are
>> submitted to the driver, the execbuffer() code assigns a unique seqno
>> value and then packages up all the information required to execute the
>> batch buffer at a later time. This package is given over to the
>> scheduler which adds it to an internal node list.
> That is called the request. Please use it properly. We pass a request to
> the engine. The engine can hook up the scheduler in which case it may
> defer execution of that request until the scheduler says it is ready.
> But that is a conversation inside the engine, not intermingled with GEM.
It looks like I forgot to update that part of the description. The
scheduler and its cover letter were written before the conversion of the
driver from using seqnos everywhere to using requests. Indeed, that
conversion was implemented as prep-work for the scheduler. So, yes the
description should be updated to match the code and say 'assigns a
request structure' not a seqno.
> GEM needs to only care about its unique seqno along the timeline. We can
> leave space for the scheduler to inject a second seqno if that is
> convenient, but since the scheduler can keep a global list of inflight
> requests, its retirement is also globally ordered even if we only
> inspect each request wrt to its own timeline.
Currently, there is no 'own timeline'. The seqno timeline is global to
the entire driver and not per engine, per context, or whatever. The
conversion to more useful timelines still needs to be implemented. Until
then, the scheduler must cope with the single global one.
>
>> The scheduler also
>> scans the list of objects associated with the batch buffer and
>> compares them against the objects already in use by other buffers in
>> the node list.
> This part moves the policy of the GEM API and enshrines it into the
> scheduler. Let GEM convert the implicit rules it has into explicit
> dependencies, on to which we can add the explicit dependencies passed in
> by the user. GEM has the information it needs to maintain its API, let's
> not let that silliness spread.
So how should this be done? The scheduler needs some mechanism for
ensuring that requests are only submitted to the hardware in a suitable
order. And that information needs to persist for as long as the request
still exists in an uncompleted state.
>
>> If matches are found then the new batch buffer node is
>> marked as being dependent upon the matching node. The same is done for
>> the context object. The scheduler also bumps up the priority of such
>> matching nodes on the grounds that the more dependencies a given batch
>> buffer has the more important it is likely to be.
> Sounds like a rough cut at PI avoidance. This is easily abusable. It
> sounds like you want a prio/nice split. Is that a good idea to introduce
> in the first series?
Not sure how you mean that it can be abused?
>> The scheduler aims to have a given (tuneable) number of batch buffers
>> in flight on the hardware at any given time. If fewer than this are
>> currently executing when a new node is queued, then the node is passed
>> straight through to the submit function. Otherwise it is simply added
>> to the queue and the driver returns back to user land.
> Keep it in the scheduler, I see you decided to add EAGAIN to GEM execbuf
> when currently execbuf *blocks* in this situation. Don't use EAGAIN here
> since you just make userspace busyspin over a very, very heavyweight
> ioctl, and you have the power here to throttle userspace without
> blocking.
Not sure where you are looking. The above description does not talk
about returning EAGAIN at any point. It says the batch buffer is either
sent directly to the hardware (if the case of idle hardware) or added to
the scheduler's queue (in the case of busy hardware). Either way, the
execbuf IOCTL returns success to the user. There is a later patch in the
series which adds throttling to prevent a user from submitting
arbitrarily large numbers of slow batches to the scheduler and
effectively consuming unbounded amounts of memory due to the scheduler's
queue not being limited by the size of a ring buffer. However, the
throttling is done by waiting on an outstanding request so as to block
until the queue depth has been reduced. It only returns an error to the
user in the case where the wait could not be done for some reason (e.g.
wedged GPU).
>> As each batch buffer completes, it raises an interrupt which wakes up
>> the scheduler. Note that it is possible for multiple buffers to
>> complete before the IRQ handler gets to run. Further, the seqno values
>> of the individual buffers are not necessary incrementing as the
>> scheduler may have re-ordered their submission. However, the scheduler
>> keeps the list of executing buffers in order of hardware submission.
>> Thus it can scan through the list until a matching seqno is found and
>> then mark all in flight nodes from that point on as completed.
> No. You haven't built your timelines correctly. The idea behind the
> timeline is that a request can wait upon its seqno and check it against
> its own timeline, which is ordered only with other requests on its
> timeline. Because of this, it is independent of whatever order the
> scheduler executes the timelines in, each timeline is ordered.
>
> A request can simply wait for its timeline to advance, completely
> ignorant of the scheduler. (Request signaling may be driven by the
> scheduler, but that is a lowlevel, not GEM or dma-buf/fence,
> implementation detail. And only if the scheduler is coupled into the
> user-interupt, but on execlists it will be using the context-switch
> interrupt to driver itself, and for ringbuffer mode we have a choice of
> user-interrupt or using pipe-control/dw-notify to keep the paths
> separate.)
Again, you are assuming per context timelines which the driver does not
yet have. That is planned work but is not yet implemented.
>
>> A deferred work queue is also poked by the interrupt handler.
> Wait you did all that in the interrupt handler. NO.
Actually, the comment is again out of date. Since the seqno -> request
conversion series, the scheduler no longer has to worry about out of
order seqno values from the hardware. So all of the above IRQ mess no
longer occurs.
>
>> If a suitable node is found then it is sent to execbuff_final() for
>> submission to the hardware.
> That is an absolutely atrocious implementation. Don't pass it back up
> the layers when you already have all the information you need packaged in
> the request to submit it to hardware.
>
> execbuf -> engine->add_request -> [scheduler] -> engine->submit_request
>
> (conceptually, since what should happen is execlists gets it
> context-switch interrupt and then asks what set of requests to submit
> next. In a ringbuffer mode, a kthread/ktasklet would run off the
> interrupt ask which request to execute next and do the write into the
> ring for the switch into the new context and execution of the next batch,
> and then write the RING_TAIL.)
>
> At the fundamental level it looks like you have not introduced timelines
> correctly or introduced the scheduler as a separate entity for deciding
> which request to execute next (or if this request should preempt execution).
When this code was first written, there was not even the possibility of
implementing per context timelines. Since that first version, there has
been a lot of complex prep work (anti-OLR series, seqno to request
conversion, struct fence conversion) which all makes the scheduler
series a lot simpler. During that process, there has been amply
opportunity to comment on the direction it was taking and to explain
what could be improved and how.
The agreement on per context timelines is that while it would make
things simpler still, it is yet more delay and could be left until after
the current scheduler series has landed. Otherwise it is a never ending
task to simply keep rebasing the scheduler patches onto a rapidly
changing target let alone write entire extra patch series and rework the
scheduler accordingly. Plus, we have multiple customers that require the
scheduler functionality (and the pre-emption support that builds on top)
so we need to get something shipping now rather than in another X years
time.
> -Chris
>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [PATCH v6 00/34] GPU scheduler for i915 driver
2016-04-25 9:54 ` [PATCH v6 00/34] GPU scheduler for i915 driver Chris Wilson
2016-04-25 11:55 ` John Harrison
@ 2016-04-26 13:20 ` Daniel Vetter
2016-05-05 11:54 ` John Harrison
1 sibling, 1 reply; 50+ messages in thread
From: Daniel Vetter @ 2016-04-26 13:20 UTC (permalink / raw)
To: Chris Wilson, John.C.Harrison, Intel-GFX
On Mon, Apr 25, 2016 at 10:54:27AM +0100, Chris Wilson wrote:
> > As each batch buffer completes, it raises an interrupt which wakes up
> > the scheduler. Note that it is possible for multiple buffers to
> > complete before the IRQ handler gets to run. Further, the seqno values
> > of the individual buffers are not necessary incrementing as the
> > scheduler may have re-ordered their submission. However, the scheduler
> > keeps the list of executing buffers in order of hardware submission.
> > Thus it can scan through the list until a matching seqno is found and
> > then mark all in flight nodes from that point on as completed.
>
> No. You haven't built your timelines correctly. The idea behind the
> timeline is that a request can wait upon its seqno and check it against
> its own timeline, which is ordered only with other requests on its
> timeline. Because of this, it is independent of whatever order the
> scheduler executes the timelines in, each timeline is ordered.
>
> A request can simply wait for its timeline to advance, completely
> ignorant of the scheduler. (Request signaling may be driven by the
> scheduler, but that is a lowlevel, not GEM or dma-buf/fence,
> implementation detail. And only if the scheduler is coupled into the
> user-interupt, but on execlists it will be using the context-switch
> interrupt to driver itself, and for ringbuffer mode we have a choice of
> user-interrupt or using pipe-control/dw-notify to keep the paths
> separate.)
This is rather crucial, since that expectations that other drivers can
rely on fence->seqno being ordered correctly within one timeline. And e.g.
amdgpu does what Chris describes and collapses fences on one timeline to
just one.
We do have to fix this before we can enable the scheduler.
The related issues with using struct fence (request) more is that we need
that also for android integration. On mutli-gpu desktops we already have
different kinds of fences, but real soon we'll also have different kinds
of fences (gem requests and kms vblank/flip complete events) on just one
gpu, and on android.
[more cut]
> At the fundamental level it looks like you have not introduced timelines
> correctly or introduced the scheduler as a separate entity for deciding
> which request to execute next (or if this request should preempt execution).
Reworking the scheduler to take request and in-fences, and correctly use
timelines is definitely the way to go.
/me out
Cheers, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [PATCH v6 00/34] GPU scheduler for i915 driver
2016-04-26 13:20 ` Daniel Vetter
@ 2016-05-05 11:54 ` John Harrison
0 siblings, 0 replies; 50+ messages in thread
From: John Harrison @ 2016-05-05 11:54 UTC (permalink / raw)
To: Daniel Vetter, Chris Wilson, Intel-GFX
On 26/04/2016 14:20, Daniel Vetter wrote:
> On Mon, Apr 25, 2016 at 10:54:27AM +0100, Chris Wilson wrote:
>>> As each batch buffer completes, it raises an interrupt which wakes up
>>> the scheduler. Note that it is possible for multiple buffers to
>>> complete before the IRQ handler gets to run. Further, the seqno values
>>> of the individual buffers are not necessary incrementing as the
>>> scheduler may have re-ordered their submission. However, the scheduler
>>> keeps the list of executing buffers in order of hardware submission.
>>> Thus it can scan through the list until a matching seqno is found and
>>> then mark all in flight nodes from that point on as completed.
>> No. You haven't built your timelines correctly. The idea behind the
>> timeline is that a request can wait upon its seqno and check it against
>> its own timeline, which is ordered only with other requests on its
>> timeline. Because of this, it is independent of whatever order the
>> scheduler executes the timelines in, each timeline is ordered.
>>
>> A request can simply wait for its timeline to advance, completely
>> ignorant of the scheduler. (Request signaling may be driven by the
>> scheduler, but that is a lowlevel, not GEM or dma-buf/fence,
>> implementation detail. And only if the scheduler is coupled into the
>> user-interupt, but on execlists it will be using the context-switch
>> interrupt to driver itself, and for ringbuffer mode we have a choice of
>> user-interrupt or using pipe-control/dw-notify to keep the paths
>> separate.)
> This is rather crucial, since that expectations that other drivers can
> rely on fence->seqno being ordered correctly within one timeline. And e.g.
> amdgpu does what Chris describes and collapses fences on one timeline to
> just one.
>
> We do have to fix this before we can enable the scheduler.
As I have stated several times, this is not broken. The fence series
might not introduce per context seqnos for the entire driver but it does
introduce a per context timeline specifically for the fence. Thus it is
perfectly safe to collapse fences that exist on the same timeline as
their internal seqno values are guaranteed to be properly ordered. The
fact that the fence's software seqno is not the same value as the
driver's hardware seqno is irrelevant to any external view and will not
cause a problem.
When the driver has been updated to support per context hardware seqnos,
it would be trivial to drop the fence's software timeline and switch to
using the hardware one. That has always been the plan.
> The related issues with using struct fence (request) more is that we need
> that also for android integration. On mutli-gpu desktops we already have
> different kinds of fences, but real soon we'll also have different kinds
> of fences (gem requests and kms vblank/flip complete events) on just one
> gpu, and on android.
>
> [more cut]
>
>> At the fundamental level it looks like you have not introduced timelines
>> correctly or introduced the scheduler as a separate entity for deciding
>> which request to execute next (or if this request should preempt execution).
> Reworking the scheduler to take request and in-fences, and correctly use
> timelines is definitely the way to go.
The scheduler has been taking 'in-fences' since it's start on Android.
That was one of the reasons the whole piece of work was first begun.
Before the de-staging patches were dropped from the struct fence series
(due to being picked up by external contributors) this scheduler patch
series included adding support for passing an external fence in to the
execbuf code path to have the driver wait on before allowing that work
to begin execution. Once the external work has fully landed and the
scheduler patches have been rebased ontop, that support will be back in
and is actually really quite trivial. Any resource which has a 'am I
ready yet' test can be used as a dependency check on a request when the
scheduler is decided what to execute next. Adding in new internal or
external dependencies is pretty easy.
>
> /me out
>
> Cheers, Daniel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* [PATCH v6 24/34] drm/i915: Added scheduler queue throttling by DRM file handle
2016-04-20 17:13 ` [PATCH v6 24/34] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
@ 2016-05-06 13:19 ` John.C.Harrison
0 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-05-06 13:19 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
The scheduler decouples the submission of batch buffers to the driver
from their subsequent submission to the hardware. This means that an
application which is continuously submitting buffers as fast as it can
could potentialy flood the driver. To prevent this, the driver now
tracks how many buffers are in progress (queued in software or
executing in hardware) and limits this to a given (tunable) number. If
this number is exceeded then the queue to the driver will return
EAGAIN and thus prevent the scheduler's queue becoming arbitrarily
large.
v3: Added a missing decrement of the file queue counter.
v4: Updated a comment.
v5: Updated due to changes to earlier patches in series - removing
forward declarations and white space. Also added some documentation.
[Joonas Lahtinen]
v6: Updated to newer nightly (lots of ring -> engine renaming).
Replace the simple 'return to userland when full' scheme with a 'sleep
on request' scheme. The former could lead to the busy polling and
wasting lots of CPU time as user land continuously retried the execbuf
IOCTL in a tight loop. Now the driver will sleep (without holding the
mutex lock) on the oldest request outstanding for that file and then
automatically retry. This is closer to the pre-scheduler behaviour of
stalling on a full ring buffer.
v6.1: Moved throttle point to later common location. Required for a
subsequent patch that needs the engine to have been determined already.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 2 +
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 4 +
drivers/gpu/drm/i915/i915_scheduler.c | 118 +++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_scheduler.h | 2 +
4 files changed, 126 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index e9aaacc..25b8fd6 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -376,6 +376,8 @@ struct drm_i915_file_private {
} rps;
unsigned int bsd_ring;
+
+ u32 scheduler_queue_length;
};
/* Used by dp and fdi links */
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index a08638a..5a674ad 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1538,6 +1538,10 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
dispatch_flags |= I915_DISPATCH_RS;
}
+ /* Throttle batch requests per device file */
+ if (i915_scheduler_file_queue_wait(file))
+ return -EAGAIN;
+
intel_runtime_pm_get(dev_priv);
ret = i915_mutex_lock_interruptible(dev);
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index a3a7a82..3569ddd 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -80,6 +80,7 @@ int i915_scheduler_init(struct drm_device *dev)
scheduler->priority_level_bump = 50;
scheduler->priority_level_preempt = 900;
scheduler->min_flying = 2;
+ scheduler->file_queue_max = 64;
dev_priv->scheduler = scheduler;
@@ -496,6 +497,28 @@ static int i915_scheduler_submit_unlocked(struct intel_engine_cs *engine)
return ret;
}
+/**
+ * i915_scheduler_file_queue_inc - Increment the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_inc(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
+ file_priv->scheduler_queue_length++;
+}
+
+/**
+ * i915_scheduler_file_queue_dec - Decrement the file's request queue count.
+ * @file: File object to process.
+ */
+static void i915_scheduler_file_queue_dec(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+
+ file_priv->scheduler_queue_length--;
+}
+
static void i915_generate_dependencies(struct i915_scheduler *scheduler,
struct i915_scheduler_queue_entry *node,
uint32_t engine)
@@ -675,6 +698,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
list_add_tail(&node->link, &scheduler->node_queue[engine->id]);
+ i915_scheduler_file_queue_inc(node->params.file);
+
not_flying = i915_scheduler_count_flying(scheduler, engine) <
scheduler->min_flying;
@@ -871,6 +896,12 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
/* Strip the dependency info while the mutex is still locked */
i915_scheduler_remove_dependent(scheduler, node);
+ /* Likewise clean up the file pointer. */
+ if (node->params.file) {
+ i915_scheduler_file_queue_dec(node->params.file);
+ node->params.file = NULL;
+ }
+
continue;
}
@@ -963,6 +994,92 @@ void i915_scheduler_work_handler(struct work_struct *work)
i915_scheduler_process_work(engine);
}
+/**
+ * i915_scheduler_file_queue_wait - Waits for space in the per file queue.
+ * @file: File object to process.
+ * This allows throttling of applications by limiting the total number of
+ * outstanding requests to a specified level. Once that limit is reached,
+ * this call will stall waiting on the oldest outstanding request. If it can
+ * not stall for any reason it returns true to mean that the queue is full
+ * and no more requests should be accepted.
+ */
+bool i915_scheduler_file_queue_wait(struct drm_file *file)
+{
+ struct drm_i915_file_private *file_priv = file->driver_priv;
+ struct drm_i915_private *dev_priv = file_priv->dev_priv;
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct drm_i915_gem_request *req;
+ struct i915_scheduler_queue_entry *node;
+ unsigned reset_counter;
+ int ret;
+ struct intel_engine_cs *engine;
+
+ if (file_priv->scheduler_queue_length < scheduler->file_queue_max)
+ return false;
+
+ do {
+ spin_lock_irq(&scheduler->lock);
+
+ /*
+ * Find the first (i.e. oldest) request for this file. In the
+ * case where an app is using multiple engines, this search
+ * might be skewed by engine. However, worst case is an app has
+ * queued ~60 requests to a high indexed engine and then one
+ * request to a low indexed engine. In such a case, the driver
+ * will wait for longer than necessary but operation will
+ * still be correct and that case is not rare enough to add
+ * jiffy based inter-engine checks.
+ */
+ req = NULL;
+ for_each_engine(engine, dev_priv) {
+ for_each_scheduler_node(node, engine->id) {
+ if (I915_SQS_IS_COMPLETE(node))
+ continue;
+
+ if (node->params.file != file)
+ continue;
+
+ req = node->params.request;
+ break;
+ }
+
+ if (req)
+ break;
+ }
+
+ if (!req) {
+ spin_unlock_irq(&scheduler->lock);
+ return false;
+ }
+
+ i915_gem_request_reference(req);
+
+ spin_unlock_irq(&scheduler->lock);
+
+ ret = i915_gem_check_wedge(&dev_priv->gpu_error, false);
+ if (ret)
+ goto err_unref;
+
+ reset_counter = atomic_read(&dev_priv->gpu_error.reset_counter);
+
+ ret = __i915_wait_request(req, reset_counter,
+ I915_WAIT_REQUEST_INTERRUPTIBLE, NULL, NULL);
+ if (ret)
+ goto err_unref;
+
+ /* Make sure the request's resources actually get cleared up */
+ i915_scheduler_process_work(req->engine);
+
+ i915_gem_request_unreference(req);
+ } while(file_priv->scheduler_queue_length >= scheduler->file_queue_max);
+
+ return false;
+
+err_unref:
+ i915_gem_request_unreference(req);
+ return true;
+}
+
static int i915_scheduler_submit_max_priority(struct intel_engine_cs *engine,
bool is_locked)
{
@@ -1185,6 +1302,7 @@ void i915_scheduler_closefile(struct drm_device *dev, struct drm_file *file)
node->status,
engine->name);
+ i915_scheduler_file_queue_dec(node->params.file);
node->params.file = NULL;
}
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 4e7c0a7..5c33c83 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -94,6 +94,7 @@ struct i915_scheduler {
int32_t priority_level_bump;
int32_t priority_level_preempt;
uint32_t min_flying;
+ uint32_t file_queue_max;
};
/* Flag bits for i915_scheduler::flags */
@@ -116,5 +117,6 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
unsigned long stamp, bool is_locked);
bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
+bool i915_scheduler_file_queue_wait(struct drm_file *file);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH v6 27/34] drm/i915: Added scheduler statistic reporting to debugfs
2016-04-20 17:13 ` [PATCH v6 27/34] drm/i915: Added scheduler statistic reporting to debugfs John.C.Harrison
@ 2016-05-06 13:21 ` John.C.Harrison
0 siblings, 0 replies; 50+ messages in thread
From: John.C.Harrison @ 2016-05-06 13:21 UTC (permalink / raw)
To: Intel-GFX
From: John Harrison <John.C.Harrison@Intel.com>
It is useful for know what the scheduler is doing for both debugging
and performance analysis purposes. This change adds a bunch of
counters and such that keep track of various scheduler operations
(batches submitted, completed, flush requests, etc.). The data can
then be read in userland via the debugfs mechanism.
v2: Updated to match changes to scheduler implementation.
v3: Updated for changes to kill code and flush code.
v4: Removed the fence/sync code as that will be part of a separate
patch series. Wrapped a long line to keep the style checker happy.
v5: Updated to remove forward declarations and white space. Added
documentation. [Joonas Lahtinen]
Used lighter weight spinlocks.
v6: Updated to newer nightly (lots of ring -> engine renaming).
Added 'for_each_scheduler_node()' and 'assert_scheduler_lock_held()'
helper macros. Updated to use 'to_i915()' instead of dev_private.
Converted all enum labels to uppercase. Removed even more white space.
Moved the enum to string conversion function to debugfs.c rather than
scheduler.c [review feedback from Joonas Lahtinen]
Added running totals of 'flying' and 'queued' nodes rather than
re-calculating each time as a minor CPU performance optimisation.
Added stats to the new file queue wait implementation.
v6.1: Added a target engine parameter to the throttle function. This
is used for tracking per engine throttle stats introduced in the v6
update. This fixes a random memory corruption bug caused by using an
invalid engine pointer.
For: VIZ-1587
Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 111 +++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 5 +-
drivers/gpu/drm/i915/i915_scheduler.c | 89 +++++++++++++++++++++--
drivers/gpu/drm/i915/i915_scheduler.h | 38 +++++++++-
4 files changed, 237 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 980bb20..1d04cde 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -3609,6 +3609,116 @@ static int i915_drrs_status(struct seq_file *m, void *unused)
return 0;
}
+static const char *i915_scheduler_queue_status_str(
+ enum i915_scheduler_queue_status status)
+{
+ static char str[50];
+
+ switch (status) {
+ case I915_SQS_NONE:
+ return "None";
+
+ case I915_SQS_QUEUED:
+ return "Queued";
+
+ case I915_SQS_POPPED:
+ return "Popped";
+
+ case I915_SQS_FLYING:
+ return "Flying";
+
+ case I915_SQS_COMPLETE:
+ return "Complete";
+
+ case I915_SQS_DEAD:
+ return "Dead";
+
+ case I915_SQS_MAX:
+ return "Invalid";
+
+ default:
+ break;
+ }
+
+ sprintf(str, "[Unknown_%d!]", status);
+ return str;
+}
+
+static int i915_scheduler_info(struct seq_file *m, void *unused)
+{
+ struct drm_info_node *node = (struct drm_info_node *) m->private;
+ struct drm_device *dev = node->minor->dev;
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_stats *stats = scheduler->stats;
+ struct i915_scheduler_stats_nodes node_stats[I915_NUM_ENGINES];
+ struct intel_engine_cs *engine;
+ char str[50 * (I915_NUM_ENGINES + 1)], name[50], *ptr;
+ int ret, i, e;
+
+ ret = mutex_lock_interruptible(&dev->mode_config.mutex);
+ if (ret)
+ return ret;
+
+#define PRINT_VAR(name, fmt, var) \
+ do { \
+ sprintf(str, "%-22s", name); \
+ ptr = str + strlen(str); \
+ for_each_engine_id(engine, dev_priv, e) { \
+ sprintf(ptr, " %10" fmt, var); \
+ ptr += strlen(ptr); \
+ } \
+ seq_printf(m, "%s\n", str); \
+ } while (0)
+
+ PRINT_VAR("Engine name:", "s", dev_priv->engine[e].name);
+ PRINT_VAR(" Engine seqno", "d", engine->get_seqno(engine));
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Batch submissions:\n");
+ PRINT_VAR(" Queued", "u", stats[e].queued);
+ PRINT_VAR(" Submitted", "u", stats[e].submitted);
+ PRINT_VAR(" Completed", "u", stats[e].completed);
+ PRINT_VAR(" Expired", "u", stats[e].expired);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Flush counts:\n");
+ PRINT_VAR(" By object", "u", stats[e].flush_obj);
+ PRINT_VAR(" By request", "u", stats[e].flush_req);
+ PRINT_VAR(" By stamp", "u", stats[e].flush_stamp);
+ PRINT_VAR(" Blanket", "u", stats[e].flush_all);
+ PRINT_VAR(" Entries bumped", "u", stats[e].flush_bump);
+ PRINT_VAR(" Entries submitted", "u", stats[e].flush_submit);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Miscellaneous:\n");
+ PRINT_VAR(" ExecEarly retry", "u", stats[e].exec_early);
+ PRINT_VAR(" ExecFinal requeue", "u", stats[e].exec_again);
+ PRINT_VAR(" ExecFinal killed", "u", stats[e].exec_dead);
+ PRINT_VAR(" Hung flying", "u", stats[e].kill_flying);
+ PRINT_VAR(" Hung queued", "u", stats[e].kill_queued);
+ PRINT_VAR(" File queue wait", "u", stats[e].file_wait);
+ PRINT_VAR(" File queue stall", "u", stats[e].file_stall);
+ PRINT_VAR(" File queue lost", "u", stats[e].file_lost);
+ seq_putc(m, '\n');
+
+ seq_puts(m, "Queue contents:\n");
+ for_each_engine(engine, dev_priv)
+ i915_scheduler_query_stats(engine, node_stats + engine->id);
+
+ for (i = 0; i < (I915_SQS_MAX + 1); i++) {
+ sprintf(name, " %s", i915_scheduler_queue_status_str(i));
+ PRINT_VAR(name, "d", node_stats[e].counts[i]);
+ }
+ seq_putc(m, '\n');
+
+#undef PRINT_VAR
+
+ mutex_unlock(&dev->mode_config.mutex);
+
+ return 0;
+}
+
struct pipe_crc_info {
const char *name;
struct drm_device *dev;
@@ -5585,6 +5695,7 @@ static const struct drm_info_list i915_debugfs_list[] = {
{"i915_semaphore_status", i915_semaphore_status, 0},
{"i915_shared_dplls_info", i915_shared_dplls_info, 0},
{"i915_dp_mst_info", i915_dp_mst_info, 0},
+ {"i915_scheduler_info", i915_scheduler_info, 0},
{"i915_wa_registers", i915_wa_registers, 0},
{"i915_ddb_info", i915_ddb_info, 0},
{"i915_sseu_status", i915_sseu_status, 0},
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index caf7e00..150b699 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1558,7 +1558,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
}
/* Throttle batch requests per device file */
- if (i915_scheduler_file_queue_wait(file))
+ if (i915_scheduler_file_queue_wait(file, engine))
return -EAGAIN;
intel_runtime_pm_get(dev_priv);
@@ -1808,6 +1808,9 @@ pre_mutex_err:
/* intel_gpu_busy should also get a ref, so it will free when the device
* is really idle. */
intel_runtime_pm_put(dev_priv);
+
+ dev_priv->scheduler->stats[engine->id].exec_early++;
+
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
index 3569ddd..123f1dc 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.c
+++ b/drivers/gpu/drm/i915/i915_scheduler.c
@@ -142,9 +142,12 @@ static void i915_scheduler_node_kill(struct i915_scheduler *scheduler,
if (I915_SQS_IS_FLYING(node)) {
scheduler->counts[node->params.engine->id].flying--;
+ scheduler->stats[node->params.engine->id].kill_flying++;
trace_i915_scheduler_unfly(node->params.engine, node);
- } else
+ } else {
scheduler->counts[node->params.engine->id].queued--;
+ scheduler->stats[node->params.engine->id].kill_queued++;
+ }
node->status = I915_SQS_DEAD;
trace_i915_scheduler_node_state_change(node->params.engine, node);
@@ -390,6 +393,8 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
*/
i915_scheduler_node_fly(node);
+ scheduler->stats[engine->id].submitted++;
+
spin_unlock_irq(&scheduler->lock);
ret = dev_priv->gt.execbuf_final(&node->params);
spin_lock_irq(&scheduler->lock);
@@ -413,6 +418,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
case ENOENT:
/* Fatal errors. Kill the node. */
requeue = false;
+ scheduler->stats[engine->id].exec_dead++;
i915_scheduler_node_kill(scheduler, node);
break;
@@ -423,6 +429,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
case ERESTARTSYS:
case EINTR:
/* Supposedly recoverable errors. */
+ scheduler->stats[engine->id].exec_again++;
break;
default:
@@ -431,6 +438,7 @@ static int i915_scheduler_submit(struct intel_engine_cs *engine)
* for the best.
*/
MISSING_CASE(-ret);
+ scheduler->stats[engine->id].exec_again++;
break;
}
@@ -574,12 +582,15 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
struct i915_scheduler *scheduler = dev_priv->scheduler;
int ret;
+ scheduler->stats[qe->params.engine->id].queued++;
+
trace_i915_scheduler_queue(qe->params.engine, qe);
intel_ring_reserved_space_cancel(qe->params.request->ringbuf);
scheduler->flags[qe->params.engine->id] |= I915_SF_SUBMITTING;
ret = dev_priv->gt.execbuf_final(&qe->params);
+ scheduler->stats[qe->params.engine->id].submitted++;
scheduler->flags[qe->params.engine->id] &= ~I915_SF_SUBMITTING;
/*
@@ -593,6 +604,8 @@ static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_en
qe->status = I915_SQS_COMPLETE;
i915_scheduler_clean_node(qe);
+ scheduler->stats[qe->params.engine->id].expired++;
+
return 0;
}
@@ -704,6 +717,8 @@ int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
scheduler->min_flying;
scheduler->counts[engine->id].queued++;
+ scheduler->stats[engine->id].queued++;
+
trace_i915_scheduler_queue(engine, node);
trace_i915_scheduler_node_state_change(engine, node);
@@ -746,10 +761,13 @@ bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
WARN_ON(!I915_SQS_IS_FLYING(node));
/* Node was in flight so mark it as complete. */
- if (req->cancelled)
+ if (req->cancelled) {
node->status = I915_SQS_DEAD;
- else
+ scheduler->stats[req->engine->id].kill_flying++;
+ } else {
node->status = I915_SQS_COMPLETE;
+ scheduler->stats[req->engine->id].completed++;
+ }
scheduler->counts[req->engine->id].flying--;
trace_i915_scheduler_node_state_change(req->engine, node);
@@ -892,6 +910,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
list_del(&node->link);
list_add(&node->link, remove);
+ scheduler->stats[engine->id].expired++;
/* Strip the dependency info while the mutex is still locked */
i915_scheduler_remove_dependent(scheduler, node);
@@ -997,13 +1016,15 @@ void i915_scheduler_work_handler(struct work_struct *work)
/**
* i915_scheduler_file_queue_wait - Waits for space in the per file queue.
* @file: File object to process.
+ * @engine: Engine being submitted to
* This allows throttling of applications by limiting the total number of
* outstanding requests to a specified level. Once that limit is reached,
* this call will stall waiting on the oldest outstanding request. If it can
* not stall for any reason it returns true to mean that the queue is full
* and no more requests should be accepted.
*/
-bool i915_scheduler_file_queue_wait(struct drm_file *file)
+bool i915_scheduler_file_queue_wait(struct drm_file *file,
+ struct intel_engine_cs *target)
{
struct drm_i915_file_private *file_priv = file->driver_priv;
struct drm_i915_private *dev_priv = file_priv->dev_priv;
@@ -1048,12 +1069,14 @@ bool i915_scheduler_file_queue_wait(struct drm_file *file)
}
if (!req) {
+ scheduler->stats[target->id].file_lost++;
spin_unlock_irq(&scheduler->lock);
return false;
}
i915_gem_request_reference(req);
+ scheduler->stats[target->id].file_wait++;
spin_unlock_irq(&scheduler->lock);
ret = i915_gem_check_wedge(&dev_priv->gpu_error, false);
@@ -1077,9 +1100,60 @@ bool i915_scheduler_file_queue_wait(struct drm_file *file)
err_unref:
i915_gem_request_unreference(req);
+
+ spin_lock_irq(&scheduler->lock);
+ scheduler->stats[target->id].file_wait--;
+ scheduler->stats[target->id].file_stall++;
+ spin_unlock_irq(&scheduler->lock);
+
return true;
}
+/**
+ * i915_scheduler_query_stats - return various scheduler statistics
+ * @engine: Engine to report on
+ * @stats: Stats structure to be filled in
+ * For various reasons (debugging, performance analysis, curiosity) it is
+ * useful to see statistics about what the scheduler is doing. This function
+ * returns the stats that have been gathered in a data structure. The
+ * expectation is that this will be returned to the user via debugfs.
+ */
+int i915_scheduler_query_stats(struct intel_engine_cs *engine,
+ struct i915_scheduler_stats_nodes *stats)
+{
+ struct drm_i915_private *dev_priv = to_i915(engine->dev);
+ struct i915_scheduler *scheduler = dev_priv->scheduler;
+ struct i915_scheduler_queue_entry *node;
+
+ memset(stats, 0x00, sizeof(*stats));
+
+ spin_lock_irq(&scheduler->lock);
+
+ for_each_scheduler_node(node, engine->id) {
+ if (node->status >= I915_SQS_MAX) {
+ DRM_DEBUG_DRIVER("Invalid node state: %d! [uniq = %d, seqno = %d]\n",
+ node->status, node->params.request->uniq,
+ node->params.request->seqno);
+
+ stats->counts[I915_SQS_MAX]++;
+ continue;
+ }
+
+ stats->counts[node->status]++;
+ }
+
+ WARN(stats->counts[I915_SQS_QUEUED] != scheduler->counts[engine->id].queued,
+ "Queued count mis-match: %d vs %d!\n",
+ stats->counts[I915_SQS_QUEUED], scheduler->counts[engine->id].queued);
+ WARN(stats->counts[I915_SQS_FLYING] != scheduler->counts[engine->id].flying,
+ "Flying count mis-match: %d vs %d!\n",
+ stats->counts[I915_SQS_FLYING], scheduler->counts[engine->id].flying);
+
+ spin_unlock_irq(&scheduler->lock);
+
+ return 0;
+}
+
static int i915_scheduler_submit_max_priority(struct intel_engine_cs *engine,
bool is_locked)
{
@@ -1160,6 +1234,7 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
}
spin_lock_irq(&scheduler->lock);
+ scheduler->stats[engine->id].flush_stamp++;
i915_scheduler_priority_bump_clear(scheduler);
for_each_scheduler_node(node, engine->id) {
if (!I915_SQS_IS_QUEUED(node))
@@ -1170,12 +1245,15 @@ int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
flush_count = i915_scheduler_priority_bump(scheduler,
node, scheduler->priority_level_max);
+ scheduler->stats[engine->id].flush_bump += flush_count;
}
spin_unlock_irq(&scheduler->lock);
if (flush_count) {
DRM_DEBUG_DRIVER("<%s> Bumped %d entries\n", engine->name, flush_count);
flush_count = i915_scheduler_submit_max_priority(engine, is_locked);
+ if (flush_count > 0)
+ scheduler->stats[engine->id].flush_submit += flush_count;
}
return flush_count;
@@ -1211,6 +1289,8 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked)
WARN_ON(is_locked && (scheduler->flags[engine->id] & I915_SF_SUBMITTING));
+ scheduler->stats[engine->id].flush_all++;
+
do {
found = false;
spin_lock_irq(&scheduler->lock);
@@ -1228,6 +1308,7 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked)
ret = i915_scheduler_submit(engine);
else
ret = i915_scheduler_submit_unlocked(engine);
+ scheduler->stats[engine->id].flush_submit++;
if (ret < 0)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
index 5c33c83..464c051 100644
--- a/drivers/gpu/drm/i915/i915_scheduler.h
+++ b/drivers/gpu/drm/i915/i915_scheduler.h
@@ -80,6 +80,36 @@ struct i915_scheduler_node_states {
uint32_t queued;
};
+struct i915_scheduler_stats {
+ /* Batch buffer counts: */
+ uint32_t queued;
+ uint32_t submitted;
+ uint32_t completed;
+ uint32_t expired;
+
+ /* Other stuff: */
+ uint32_t flush_obj;
+ uint32_t flush_req;
+ uint32_t flush_stamp;
+ uint32_t flush_all;
+ uint32_t flush_bump;
+ uint32_t flush_submit;
+
+ uint32_t exec_early;
+ uint32_t exec_again;
+ uint32_t exec_dead;
+ uint32_t kill_flying;
+ uint32_t kill_queued;
+
+ uint32_t file_wait;
+ uint32_t file_stall;
+ uint32_t file_lost;
+};
+
+struct i915_scheduler_stats_nodes {
+ uint32_t counts[I915_SQS_MAX + 1];
+};
+
struct i915_scheduler {
struct list_head node_queue[I915_NUM_ENGINES];
uint32_t flags[I915_NUM_ENGINES];
@@ -95,6 +125,9 @@ struct i915_scheduler {
int32_t priority_level_preempt;
uint32_t min_flying;
uint32_t file_queue_max;
+
+ /* Statistics: */
+ struct i915_scheduler_stats stats[I915_NUM_ENGINES];
};
/* Flag bits for i915_scheduler::flags */
@@ -117,6 +150,9 @@ int i915_scheduler_flush(struct intel_engine_cs *engine, bool is_locked);
int i915_scheduler_flush_stamp(struct intel_engine_cs *engine,
unsigned long stamp, bool is_locked);
bool i915_scheduler_is_mutex_required(struct drm_i915_gem_request *req);
-bool i915_scheduler_file_queue_wait(struct drm_file *file);
+int i915_scheduler_query_stats(struct intel_engine_cs *engine,
+ struct i915_scheduler_stats_nodes *stats);
+bool i915_scheduler_file_queue_wait(struct drm_file *file,
+ struct intel_engine_cs *engine);
#endif /* _I915_SCHEDULER_H_ */
--
1.9.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 50+ messages in thread
* ✗ Fi.CI.BAT: warning for GPU scheduler for i915 driver (rev4)
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
` (40 preceding siblings ...)
2016-04-25 9:54 ` [PATCH v6 00/34] GPU scheduler for i915 driver Chris Wilson
@ 2016-05-09 9:49 ` Patchwork
41 siblings, 0 replies; 50+ messages in thread
From: Patchwork @ 2016-05-09 9:49 UTC (permalink / raw)
To: John.C.Harrison; +Cc: intel-gfx
== Series Details ==
Series: GPU scheduler for i915 driver (rev4)
URL : https://patchwork.freedesktop.org/series/3585/
State : warning
== Summary ==
Series 3585v4 GPU scheduler for i915 driver
http://patchwork.freedesktop.org/api/1.0/series/3585/revisions/4/mbox/
Test kms_pipe_crc_basic:
Subgroup hang-read-crc-pipe-a:
incomplete -> PASS (snb-dellxps)
Subgroup nonblocking-crc-pipe-b:
pass -> SKIP (hsw-brixbox)
Subgroup suspend-read-crc-pipe-c:
skip -> PASS (hsw-brixbox)
bdw-nuci7-2 total:219 pass:206 dwarn:0 dfail:0 fail:0 skip:13
bdw-ultra total:219 pass:193 dwarn:0 dfail:0 fail:0 skip:26
bsw-nuc-2 total:218 pass:174 dwarn:0 dfail:0 fail:2 skip:42
byt-nuc total:218 pass:174 dwarn:0 dfail:0 fail:3 skip:41
hsw-brixbox total:219 pass:192 dwarn:0 dfail:0 fail:0 skip:27
hsw-gt2 total:219 pass:197 dwarn:0 dfail:0 fail:1 skip:21
ilk-hp8440p total:219 pass:155 dwarn:0 dfail:0 fail:1 skip:63
ivb-t430s total:219 pass:188 dwarn:0 dfail:0 fail:0 skip:31
skl-i7k-2 total:219 pass:191 dwarn:0 dfail:0 fail:0 skip:28
skl-nuci5 total:219 pass:207 dwarn:0 dfail:0 fail:0 skip:12
snb-dellxps total:219 pass:176 dwarn:0 dfail:0 fail:0 skip:43
snb-x220t total:219 pass:176 dwarn:0 dfail:0 fail:1 skip:42
Results at /archive/results/CI_IGT_test/Patchwork_2147/
4cf0071a8ecbb693f9364c66503fe5b4a7d9aa22 drm-intel-nightly: 2016y-05m-09d-08h-37m-34s UTC integration manifest
28f00ae drm/i915: Add total count to context status debugfs output
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [PATCH v6 06/34] drm/i915: Start of GPU scheduler
2016-04-20 17:13 ` [PATCH v6 06/34] drm/i915: Start of GPU scheduler John.C.Harrison
@ 2016-06-10 16:24 ` Tvrtko Ursulin
0 siblings, 0 replies; 50+ messages in thread
From: Tvrtko Ursulin @ 2016-06-10 16:24 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
Hi,
Just a few random comments/questions. (not a full review!)
On 20/04/16 18:13, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> Initial creation of scheduler source files. Note that this patch
> implements most of the scheduler functionality but does not hook it in
> to the driver yet. It also leaves the scheduler code in 'pass through'
> mode so that even when it is hooked in, it will not actually do very
> much. This allows the hooks to be added one at a time in bite size
> chunks and only when the scheduler is finally enabled at the end does
> anything start happening.
>
> The general theory of operation is that when batch buffers are
> submitted to the driver, the execbuffer() code packages up all the
> information required to execute the batch buffer at a later time. This
> package is given over to the scheduler which adds it to an internal
> node list. The scheduler also scans the list of objects associated
> with the batch buffer and compares them against the objects already in
> use by other buffers in the node list. If matches are found then the
> new batch buffer node is marked as being dependent upon the matching
> node. The same is done for the context object. The scheduler also
> bumps up the priority of such matching nodes on the grounds that the
> more dependencies a given batch buffer has the more important it is
> likely to be.
>
> The scheduler aims to have a given (tuneable) number of batch buffers
> in flight on the hardware at any given time. If fewer than this are
> currently executing when a new node is queued, then the node is passed
> straight through to the submit function. Otherwise it is simply added
> to the queue and the driver returns back to user land.
>
> The scheduler is notified when each batch buffer completes and updates
> its internal tracking accordingly. At the end of the completion
> interrupt processing, if any scheduler tracked batches were processed,
> the scheduler's deferred worker thread is woken up. This can do more
> involved processing such as actually removing completed nodes from the
> queue and freeing up the resources associated with them (internal
> memory allocations, DRM object references, context reference, etc.).
> The work handler also checks the in flight count and calls the
> submission code if a new slot has appeared.
>
> When the scheduler's submit code is called, it scans the queued node
> list for the highest priority node that has no unmet dependencies.
> Note that the dependency calculation is complex as it must take
> inter-ring dependencies and potential preemptions into account. Note
> also that in the future this will be extended to include external
> dependencies such as the Android Native Sync file descriptors and/or
> the linux dma-buff synchronisation scheme.
>
> If a suitable node is found then it is sent to execbuff_final() for
> submission to the hardware. The in flight count is then re-checked and
> a new node popped from the list if appropriate. All nodes that are not
> submitted have their priority bumped. This ensures that low priority
> tasks do not get starved out by busy higher priority ones - everything
> will eventually get its turn to run.
>
> Note that this patch does not implement pre-emptive scheduling. Only
> basic scheduling by re-ordering batch buffer submission is currently
> implemented. Pre-emption of actively executing batch buffers comes in
> the next patch series.
>
> v2: Changed priority levels to +/-1023 due to feedback from Chris
> Wilson.
>
> Removed redundant index from scheduler node.
>
> Changed time stamps to use jiffies instead of raw monotonic. This
> provides lower resolution but improved compatibility with other i915
> code.
>
> Major re-write of completion tracking code due to struct fence
> conversion. The scheduler no longer has it's own private IRQ handler
> but just lets the existing request code handle completion events.
> Instead, the scheduler now hooks into the request notify code to be
> told when a request has completed.
>
> Reduced driver mutex locking scope. Removal of scheduler nodes no
> longer grabs the mutex lock.
>
> v3: Refactor of dependency generation to make the code more readable.
> Also added in read-read optimisation support - i.e., don't treat a
> shared read-only buffer as being a dependency.
>
> Allowed the killing of queued nodes rather than only flying ones.
>
> v4: Updated the commit message to better reflect the current state of
> the code. Downgraded some BUG_ONs to WARN_ONs. Used the correct array
> memory allocator function (kmalloc_array instead of kmalloc).
> Corrected the format of some comments. Wrapped some lines differently
> to keep the style checker happy.
>
> Fixed a WARN_ON when killing nodes. The dependency removal code checks
> that nodes being destroyed do not have any oustanding dependencies
> (which would imply they should not have been executed yet). In the
> case of nodes being destroyed, e.g. due to context banning, then this
> might well be the case - they have not been executed and do indeed
> have outstanding dependencies.
>
> Re-instated the code to disble interrupts when not in use. The
> underlying problem causing broken IRQ reference counts seems to have
> been fixed now.
>
> v5: Shuffled various functions around to remove forward declarations
> as apparently these are frowned upon. Removed lots of white space as
> apparently having easy to read code is also frowned upon. Split the
> direct submission scheduler bypass code out into a separate function.
> Squashed down the i915_scheduler.c sections of various patches into
> this patch. Thus the later patches simply hook in existing code into
> various parts of the driver rather than adding the code as well. Added
> documentation to various functions. Re-worked the submit function in
> terms of mutex locking, error handling and exit paths. Split the
> delayed work handler function in half. Made use of the kernel 'clamp'
> macro. [Joonas Lahtinen]
>
> Added runtime PM calls as these must be done at the top level before
> acquiring the driver mutex lock. [Chris Wilson]
>
> Removed some obsolete debug code that had been forgotten about.
>
> Moved more clean up code into the 'i915_gem_scheduler_clean_node()'
> function rather than replicating it in mutliple places.
>
> Used lighter weight spinlocks.
>
> v6: Updated to newer nightly (lots of ring -> engine renaming).
>
> Added 'for_each_scheduler_node()' and 'assert_scheduler_lock_held()'
> helper macros. Renamed 'i915_gem_execbuff_release_batch_obj' to
> 'i915_gem_execbuf_release_batch_obj'. Updated to use 'to_i915()'
> instead of dev_private. Converted all enum labels to uppercase.
> Removed various unnecessary WARNs. Renamed 'saved_objects' to just
> 'objs'. Split code for counting incomplete nodes out into a separate
> function. Removed even more white space. Added a destroy() function.
> [review feedback from Joonas Lahtinen]
>
> Added running totals of 'flying' and 'queued' nodes rather than
> re-calculating each time as a minor CPU performance optimisation.
>
> Removed support for out of order seqno completion. All the prep work
> patch series (seqno to request conversion, late seqno assignment,
> etc.) that has now been done means that the scheduler no longer
> generates out of order seqno completions. Thus all the complex code
> for coping with such is no longer required and can be removed.
>
> Fixed a bug in scheduler bypass mode introduced in the clean up code
> refactoring of v5. The clean up function was seeing the node in the
> wrong state and thus refusing to process it.
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/Makefile | 1 +
> drivers/gpu/drm/i915/i915_dma.c | 3 +
> drivers/gpu/drm/i915/i915_drv.h | 6 +
> drivers/gpu/drm/i915/i915_gem.c | 5 +
> drivers/gpu/drm/i915/i915_scheduler.c | 867 ++++++++++++++++++++++++++++++++++
> drivers/gpu/drm/i915/i915_scheduler.h | 113 +++++
> 6 files changed, 995 insertions(+)
> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.c
> create mode 100644 drivers/gpu/drm/i915/i915_scheduler.h
>
> diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
> index e9cdeb5..289fa73 100644
> --- a/drivers/gpu/drm/i915/Makefile
> +++ b/drivers/gpu/drm/i915/Makefile
> @@ -10,6 +10,7 @@ ccflags-y := -Werror
> i915-y := i915_drv.o \
> i915_irq.o \
> i915_params.o \
> + i915_scheduler.o \
> i915_suspend.o \
> i915_sysfs.o \
> intel_csr.o \
> diff --git a/drivers/gpu/drm/i915/i915_dma.c b/drivers/gpu/drm/i915/i915_dma.c
> index b377753..2ad4071 100644
> --- a/drivers/gpu/drm/i915/i915_dma.c
> +++ b/drivers/gpu/drm/i915/i915_dma.c
> @@ -37,6 +37,7 @@
> #include "i915_drv.h"
> #include "i915_vgpu.h"
> #include "i915_trace.h"
> +#include "i915_scheduler.h"
> #include <linux/pci.h>
> #include <linux/console.h>
> #include <linux/vt.h>
> @@ -1448,6 +1449,8 @@ int i915_driver_unload(struct drm_device *dev)
>
> intel_csr_ucode_fini(dev_priv);
>
> + i915_scheduler_destroy(dev_priv);
> +
> /* Free error state after interrupts are fully disabled. */
> cancel_delayed_work_sync(&dev_priv->gpu_error.hangcheck_work);
> i915_destroy_error_state(dev);
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 7492ce7..7b62e2c 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -1717,6 +1717,8 @@ struct i915_execbuffer_params {
> struct drm_i915_gem_request *request;
> };
>
> +struct i915_scheduler;
> +
> /* used in computing the new watermarks state */
> struct intel_wm_config {
> unsigned int num_pipes_active;
> @@ -1994,6 +1996,8 @@ struct drm_i915_private {
>
> struct i915_runtime_pm pm;
>
> + struct i915_scheduler *scheduler;
> +
> /* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
> struct {
> int (*execbuf_submit)(struct i915_execbuffer_params *params,
> @@ -2335,6 +2339,8 @@ struct drm_i915_gem_request {
> /** process identifier submitting this request */
> struct pid *pid;
>
> + struct i915_scheduler_queue_entry *scheduler_qe;
> +
> /**
> * The ELSP only accepts two elements at a time, so we queue
> * context/tail pairs on a given queue (ring->execlist_queue) until the
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index a632276..b7466cb 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -32,6 +32,7 @@
> #include "i915_vgpu.h"
> #include "i915_trace.h"
> #include "intel_drv.h"
> +#include "i915_scheduler.h"
> #include <linux/shmem_fs.h>
> #include <linux/slab.h>
> #include <linux/swap.h>
> @@ -5405,6 +5406,10 @@ int i915_gem_init(struct drm_device *dev)
> */
> intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
>
> + ret = i915_scheduler_init(dev);
> + if (ret)
> + goto out_unlock;
> +
> ret = i915_gem_init_userptr(dev);
> if (ret)
> goto out_unlock;
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> new file mode 100644
> index 0000000..9d628b9
> --- /dev/null
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -0,0 +1,867 @@
> +/*
> + * Copyright (c) 2014 Intel Corporation
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + *
> + */
> +
> +#include "i915_drv.h"
> +#include "intel_drv.h"
> +#include "i915_scheduler.h"
> +
> +#define for_each_scheduler_node(node, id) \
> + list_for_each_entry((node), &scheduler->node_queue[(id)], link)
> +
> +#define assert_scheduler_lock_held(scheduler) \
> + do { \
> + WARN_ONCE(!spin_is_locked(&(scheduler)->lock), "Spinlock not locked!"); \
> + } while(0)
> +
> +/**
> + * i915_scheduler_is_enabled - Returns true if the scheduler is enabled.
> + * @dev: DRM device
> + */
> +bool i915_scheduler_is_enabled(struct drm_device *dev)
> +{
> + struct drm_i915_private *dev_priv = to_i915(dev);
> +
> + return dev_priv->scheduler != NULL;
> +}
> +
> +/**
> + * i915_scheduler_init - Initialise the scheduler.
> + * @dev: DRM device
> + * Returns zero on success or -ENOMEM if memory allocations fail.
> + */
> +int i915_scheduler_init(struct drm_device *dev)
> +{
> + struct drm_i915_private *dev_priv = to_i915(dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + int e;
> +
> + if (scheduler)
> + return 0;
Probably a GEM_BUG_ON or something.
> +
> + scheduler = kzalloc(sizeof(*scheduler), GFP_KERNEL);
> + if (!scheduler)
> + return -ENOMEM;
> +
> + spin_lock_init(&scheduler->lock);
> +
> + for (e = 0; e < I915_NUM_ENGINES; e++) {
> + INIT_LIST_HEAD(&scheduler->node_queue[e]);
> + scheduler->counts[e].flying = 0;
> + scheduler->counts[e].queued = 0;
> + }
> +
> + /* Default tuning values: */
> + scheduler->priority_level_min = -1023;
> + scheduler->priority_level_max = 1023;
> + scheduler->priority_level_bump = 50;
> + scheduler->priority_level_preempt = 900;
> + scheduler->min_flying = 2;
> +
> + dev_priv->scheduler = scheduler;
> +
> + return 0;
> +}
> +
> +/**
> + * i915_scheduler_destroy - Get rid of the scheduler.
> + * @dev: DRM device
> + */
> +void i915_scheduler_destroy(struct drm_i915_private *dev_priv)
> +{
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + int e;
> +
> + if (!scheduler)
> + return;
> +
> + for (e = 0; e < I915_NUM_ENGINES; e++)
> + WARN(!list_empty(&scheduler->node_queue[e]), "Destroying with list entries on engine %d!", e);
> +
> + kfree(scheduler);
> + dev_priv->scheduler = NULL;
> +}
> +
> +/*
> + * Add a popped node back in to the queue. For example, because the engine
> + * was hung when execfinal() was called and thus the engine submission needs
> + * to be retried later.
> + */
> +static void i915_scheduler_node_requeue(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *node)
> +{
> + assert_scheduler_lock_held(scheduler);
> +
> + WARN_ON(!I915_SQS_IS_FLYING(node));
> +
> + /* Seqno will be reassigned on relaunch */
> + node->params.request->seqno = 0;
> + node->status = I915_SQS_QUEUED;
> + scheduler->counts[node->params.engine->id].flying--;
> + scheduler->counts[node->params.engine->id].queued++;
> +}
> +
> +/*
> + * Give up on a node completely. For example, because it is causing the
> + * engine to hang or is using some resource that no longer exists.
> + */
> +static void i915_scheduler_node_kill(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *node)
> +{
> + assert_scheduler_lock_held(scheduler);
> +
> + WARN_ON(I915_SQS_IS_COMPLETE(node));
> +
> + if (I915_SQS_IS_FLYING(node))
> + scheduler->counts[node->params.engine->id].flying--;
> + else
> + scheduler->counts[node->params.engine->id].queued--;
> +
> + node->status = I915_SQS_DEAD;
> +}
> +
> +/* Mark a node as in flight on the hardware. */
> +static void i915_scheduler_node_fly(struct i915_scheduler_queue_entry *node)
> +{
> + struct drm_i915_private *dev_priv = to_i915(node->params.dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct intel_engine_cs *engine = node->params.engine;
> +
> + assert_scheduler_lock_held(scheduler);
> +
> + WARN_ON(node->status != I915_SQS_POPPED);
> +
> + /*
> + * Add the node (which should currently be in state popped) to the
> + * front of the queue. This ensure that flying nodes are always held
> + * in hardware submission order.
> + */
> + list_add(&node->link, &scheduler->node_queue[engine->id]);
> +
> + node->status = I915_SQS_FLYING;
> +
> + scheduler->counts[engine->id].flying++;
> +
> + if (!(scheduler->flags[engine->id] & I915_SF_INTERRUPTS_ENABLED)) {
> + bool success = true;
> +
> + success = engine->irq_get(engine);
> + if (success)
> + scheduler->flags[engine->id] |= I915_SF_INTERRUPTS_ENABLED;
> + }
> +}
> +
> +static inline uint32_t i915_scheduler_count_flying(struct i915_scheduler *scheduler,
> + struct intel_engine_cs *engine)
> +{
> + return scheduler->counts[engine->id].flying;
> +}
Maybe num_flying would be a more obvious name, or at least less worry
inducing.
> +
> +static void i915_scheduler_priority_bump_clear(struct i915_scheduler *scheduler)
> +{
> + struct i915_scheduler_queue_entry *node;
> + int i;
> +
> + assert_scheduler_lock_held(scheduler);
> +
> + /*
> + * Ensure circular dependencies don't cause problems and that a bump
> + * by object usage only bumps each using buffer once:
> + */
> + for (i = 0; i < I915_NUM_ENGINES; i++) {
> + for_each_scheduler_node(node, i)
> + node->bumped = false;
> + }
> +}
> +
> +static int i915_scheduler_priority_bump(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *target,
> + uint32_t bump)
> +{
> + uint32_t new_priority;
> + int i, count;
> +
> + if (target->priority >= scheduler->priority_level_max)
> + return 1;
> +
> + if (target->bumped)
> + return 0;
> +
> + new_priority = target->priority + bump;
> + if ((new_priority <= target->priority) ||
> + (new_priority > scheduler->priority_level_max))
> + target->priority = scheduler->priority_level_max;
> + else
> + target->priority = new_priority;
> +
> + count = 1;
> + target->bumped = true;
> +
> + for (i = 0; i < target->num_deps; i++) {
> + if (!target->dep_list[i])
> + continue;
> +
> + if (target->dep_list[i]->bumped)
> + continue;
> +
> + count += i915_scheduler_priority_bump(scheduler,
> + target->dep_list[i],
> + bump);
> + }
> +
> + return count;
> +}
> +
> +/*
> + * Nodes are considered valid dependencies if they are queued on any engine
> + * or if they are in flight on a different engine. In flight on the same
> + * engine is no longer interesting for non-premptive nodes as the engine
> + * serialises execution. For pre-empting nodes, all in flight dependencies
> + * are valid as they must not be jumped by the act of pre-empting.
> + *
> + * Anything that is neither queued nor flying is uninteresting.
> + */
> +static inline bool i915_scheduler_is_dependency_valid(
> + struct i915_scheduler_queue_entry *node, uint32_t idx)
> +{
> + struct i915_scheduler_queue_entry *dep;
> +
> + dep = node->dep_list[idx];
> + if (!dep)
> + return false;
> +
> + if (I915_SQS_IS_QUEUED(dep))
> + return true;
> +
> + if (I915_SQS_IS_FLYING(dep)) {
> + if (node->params.engine != dep->params.engine)
> + return true;
> + }
> +
> + return false;
> +}
> +
> +static int i915_scheduler_pop_from_queue_locked(struct intel_engine_cs *engine,
> + struct i915_scheduler_queue_entry **pop_node)
> +{
> + struct drm_i915_private *dev_priv = to_i915(engine->dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *best = NULL;
> + struct i915_scheduler_queue_entry *node;
> + int ret;
> + int i;
> + bool any_queued = false;
> + bool has_local, has_remote, only_remote = false;
> +
> + assert_scheduler_lock_held(scheduler);
> +
> + *pop_node = NULL;
Looks to be not needed since there is *pop_node = best below.
> + ret = -ENODATA;
> +
> + for_each_scheduler_node(node, engine->id) {
> + if (!I915_SQS_IS_QUEUED(node))
> + continue;
> + any_queued = true;
> +
> + has_local = false;
> + has_remote = false;
> + for (i = 0; i < node->num_deps; i++) {
> + if (!i915_scheduler_is_dependency_valid(node, i))
> + continue;
> +
> + if (node->dep_list[i]->params.engine == node->params.engine)
> + has_local = true;
> + else
> + has_remote = true;
Would it be worth breaking early from this loop if has_local && has_remote?
> + }
> +
> + if (has_remote && !has_local)
> + only_remote = true;
> +
> + if (!has_local && !has_remote) {
> + if (!best ||
> + (node->priority > best->priority))
> + best = node;
> + }
> + }
> +
> + if (best) {
> + list_del(&best->link);
> +
> + INIT_LIST_HEAD(&best->link);
list_del_init can replace the two lines above.
> + best->status = I915_SQS_POPPED;
> +
> + scheduler->counts[engine->id].queued--;
> +
> + ret = 0;
> + } else {
> + /* Can only get here if:
> + * (a) there are no buffers in the queue
> + * (b) all queued buffers are dependent on other buffers
> + * e.g. on a buffer that is in flight on a different engine
> + */
> + if (only_remote) {
> + /* The only dependent buffers are on another engine. */
> + ret = -EAGAIN;
> + } else if (any_queued) {
> + /* It seems that something has gone horribly wrong! */
> + WARN_ONCE(true, "Broken dependency tracking on engine %d!\n",
> + (int) engine->id);
> + }
> + }
> +
> + *pop_node = best;
> + return ret;
> +}
> +
> +/*
> + * NB: The driver mutex lock must be held before calling this function. It is
> + * only really required during the actual back end submission call. However,
> + * attempting to acquire a mutex while holding a spin lock is a Bad Idea.
> + * And releasing the one before acquiring the other leads to other code
> + * being run and interfering.
> + */
> +static int i915_scheduler_submit(struct intel_engine_cs *engine)
> +{
> + struct drm_i915_private *dev_priv = to_i915(engine->dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *node;
> + int ret, count = 0, flying;
> +
> + WARN_ON(!mutex_is_locked(&engine->dev->struct_mutex));
> +
> + spin_lock_irq(&scheduler->lock);
> +
> + WARN_ON(scheduler->flags[engine->id] & I915_SF_SUBMITTING);
> + scheduler->flags[engine->id] |= I915_SF_SUBMITTING;
> +
> + /* First time around, complain if anything unexpected occurs: */
> + ret = i915_scheduler_pop_from_queue_locked(engine, &node);
> + if (ret)
> + goto error;
> +
> + do {
> + WARN_ON(node->params.engine != engine);
> + WARN_ON(node->status != I915_SQS_POPPED);
> + count++;
> +
> + /*
> + * The call to pop above will have removed the node from the
> + * list. So add it back in and mark it as in flight.
> + */
> + i915_scheduler_node_fly(node);
> +
> + spin_unlock_irq(&scheduler->lock);
> + ret = dev_priv->gt.execbuf_final(&node->params);
> + spin_lock_irq(&scheduler->lock);
> +
> + /*
> + * Handle failed submission but first check that the
> + * watchdog/reset code has not nuked the node while we
> + * weren't looking:
> + */
> + if (ret && (node->status != I915_SQS_DEAD)) {
> + bool requeue = true;
> +
> + /*
> + * Oh dear! Either the node is broken or the engine is
> + * busy. So need to kill the node or requeue it and try
> + * again later as appropriate.
> + */
> +
> + switch (-ret) {
> + case ENODEV:
> + case ENOENT:
> + /* Fatal errors. Kill the node. */
> + requeue = false;
> + i915_scheduler_node_kill(scheduler, node);
> + break;
> +
> + case EAGAIN:
> + case EBUSY:
> + case EIO:
> + case ENOMEM:
> + case ERESTARTSYS:
> + case EINTR:
> + /* Supposedly recoverable errors. */
> + break;
> +
> + default:
> + /*
> + * Assume the error is recoverable and hope
> + * for the best.
> + */
> + MISSING_CASE(-ret);
> + break;
> + }
> +
> + if (requeue) {
> + i915_scheduler_node_requeue(scheduler, node);
> + /*
> + * No point spinning if the engine is currently
> + * unavailable so just give up and come back
> + * later.
> + */
> + break;
> + }
> + }
> +
> + /* Keep launching until the sky is sufficiently full. */
> + flying = i915_scheduler_count_flying(scheduler, engine);
> + if (flying >= scheduler->min_flying)
> + break;
> +
> + /* Grab another node and go round again... */
> + ret = i915_scheduler_pop_from_queue_locked(engine, &node);
> + } while (ret == 0);
> +
> + /* Don't complain about not being able to submit extra entries */
> + if (ret == -ENODATA)
> + ret = 0;
> +
> + /*
> + * Bump the priority of everything that was not submitted to prevent
> + * starvation of low priority tasks by a spamming high priority task.
> + */
> + i915_scheduler_priority_bump_clear(scheduler);
> + for_each_scheduler_node(node, engine->id) {
> + if (!I915_SQS_IS_QUEUED(node))
> + continue;
> +
> + i915_scheduler_priority_bump(scheduler, node,
> + scheduler->priority_level_bump);
> + }
bump_clear will iterate queues for all engines and then you iterate it
again. Would this be equivalent:
for (i = 0; i < I915_NUM_ENGINES; i++) {
for_each_scheduler_node(node, i) {
node->bumped = false;
if (i == engine->id && I915_SQS_IS_QUEUED(node))
i915_scheduler_priority_bump(scheduler, node,
scheduler->priority_level_bump);
}
}
Advantage is one loop fewer.
> +
> + /* On success, return the number of buffers submitted. */
> + if (ret == 0)
> + ret = count;
> +
> +error:
> + scheduler->flags[engine->id] &= ~I915_SF_SUBMITTING;
> + spin_unlock_irq(&scheduler->lock);
> + return ret;
> +}
> +
> +static void i915_generate_dependencies(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *node,
> + uint32_t engine)
> +{
> + struct i915_scheduler_obj_entry *this, *that;
> + struct i915_scheduler_queue_entry *test;
> + int i, j;
> + bool found;
> +
> + for_each_scheduler_node(test, engine) {
> + if (I915_SQS_IS_COMPLETE(test))
> + continue;
> +
> + /*
> + * Batches on the same engine for the same
> + * context must be kept in order.
> + */
> + found = (node->params.ctx == test->params.ctx) &&
> + (node->params.engine == test->params.engine);
> +
> + /*
> + * Batches working on the same objects must
> + * be kept in order.
> + */
> + for (i = 0; (i < node->num_objs) && !found; i++) {
> + this = node->objs + i;
> +
> + for (j = 0; j < test->num_objs; j++) {
> + that = test->objs + j;
> +
> + if (this->obj != that->obj)
> + continue;
> +
> + /* Only need to worry about writes */
> + if (this->read_only && that->read_only)
> + continue;
> +
> + found = true;
> + break;
> + }
> + }
> +
> + if (found) {
> + node->dep_list[node->num_deps] = test;
> + node->num_deps++;
> + }
> + }
> +}
> +
> +static int i915_scheduler_queue_execbuffer_bypass(struct i915_scheduler_queue_entry *qe)
> +{
> + struct drm_i915_private *dev_priv = to_i915(qe->params.dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + int ret;
> +
> + scheduler->flags[qe->params.engine->id] |= I915_SF_SUBMITTING;
> + ret = dev_priv->gt.execbuf_final(&qe->params);
> + scheduler->flags[qe->params.engine->id] &= ~I915_SF_SUBMITTING;
> +
> + /*
> + * Don't do any clean up on failure because the caller will
> + * do it all anyway.
> + */
> + if (ret)
> + return ret;
> +
> + /* Need to release any resources held by the node: */
> + qe->status = I915_SQS_COMPLETE;
> + i915_scheduler_clean_node(qe);
> +
> + return 0;
> +}
> +
> +static inline uint32_t i915_scheduler_count_incomplete(struct i915_scheduler *scheduler)
> +{
> + int e, incomplete = 0;
> +
> + for (e = 0; e < I915_NUM_ENGINES; e++)
> + incomplete += scheduler->counts[e].queued + scheduler->counts[e].flying;
> +
> + return incomplete;
> +}
> +
> +/**
> + * i915_scheduler_queue_execbuffer - Submit a batch buffer request to the
> + * scheduler.
> + * @qe: The batch buffer request to be queued.
> + * The expectation is the qe passed in is a local stack variable. This
> + * function will copy its contents into a freshly allocated list node. The
> + * new node takes ownership of said contents so the original qe should simply
> + * be discarded and not cleaned up (i.e. don't free memory it points to or
> + * dereference objects it holds). The node is added to the scheduler's queue
> + * and the batch buffer will be submitted to the hardware at some future
> + * point in time (which may be immediately, before returning or may be quite
> + * a lot later).
> + */
> +int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe)
> +{
> + struct drm_i915_private *dev_priv = to_i915(qe->params.dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct intel_engine_cs *engine = qe->params.engine;
> + struct i915_scheduler_queue_entry *node;
> + bool not_flying;
> + int i, e;
> + int incomplete;
> +
> + /* Bypass the scheduler and send the buffer immediately? */
> + if (1/*!i915.enable_scheduler*/)
> + return i915_scheduler_queue_execbuffer_bypass(qe);
> +
> + node = kmalloc(sizeof(*node), GFP_KERNEL);
> + if (!node)
> + return -ENOMEM;
> +
> + *node = *qe;
> + INIT_LIST_HEAD(&node->link);
> + node->status = I915_SQS_QUEUED;
> + node->stamp = jiffies;
> + i915_gem_request_reference(node->params.request);
> +
> + WARN_ON(node->params.request->scheduler_qe);
> + node->params.request->scheduler_qe = node;
> +
> + /*
> + * Need to determine the number of incomplete entries in the list as
> + * that will be the maximum size of the dependency list.
> + *
> + * Note that the allocation must not be made with the spinlock acquired
> + * as kmalloc can sleep. However, the unlock/relock is safe because no
> + * new entries can be queued up during the unlock as the i915 driver
> + * mutex is still held. Entries could be removed from the list but that
> + * just means the dep_list will be over-allocated which is fine.
> + */
> + spin_lock_irq(&scheduler->lock);
> + incomplete = i915_scheduler_count_incomplete(scheduler);
> +
> + /* Temporarily unlock to allocate memory: */
> + spin_unlock_irq(&scheduler->lock);
> + if (incomplete) {
> + node->dep_list = kmalloc_array(incomplete,
> + sizeof(*node->dep_list),
> + GFP_KERNEL);
> + if (!node->dep_list) {
> + kfree(node);
> + return -ENOMEM;
> + }
> + } else
> + node->dep_list = NULL;
> +
> + spin_lock_irq(&scheduler->lock);
> + node->num_deps = 0;
> +
> + if (node->dep_list) {
> + for (e = 0; e < I915_NUM_ENGINES; e++)
> + i915_generate_dependencies(scheduler, node, e);
> +
> + WARN_ON(node->num_deps > incomplete);
> + }
> +
> + node->priority = clamp(node->priority,
> + scheduler->priority_level_min,
> + scheduler->priority_level_max);
> +
> + if ((node->priority > 0) && node->num_deps) {
> + i915_scheduler_priority_bump_clear(scheduler);
> +
> + for (i = 0; i < node->num_deps; i++)
> + i915_scheduler_priority_bump(scheduler,
> + node->dep_list[i], node->priority);
> + }
> +
> + list_add_tail(&node->link, &scheduler->node_queue[engine->id]);
> +
> + not_flying = i915_scheduler_count_flying(scheduler, engine) <
> + scheduler->min_flying;
> +
> + scheduler->counts[engine->id].queued++;
> +
> + spin_unlock_irq(&scheduler->lock);
> +
> + if (not_flying)
> + i915_scheduler_submit(engine);
> +
> + return 0;
> +}
> +
> +/**
> + * i915_scheduler_notify_request - Notify the scheduler that the given
> + * request has completed on the hardware.
> + * @req: Request structure which has completed
> + * @preempt: Did it complete pre-emptively?
> + * A sequence number has popped out of the hardware and the request handling
> + * code has mapped it back to a request and will mark that request complete.
> + * It also calls this function to notify the scheduler about the completion
> + * so the scheduler's node can be updated appropriately.
> + * Returns true if the request is scheduler managed, false if not. The return
> + * value is combined for all freshly completed requests and if any were true
> + * then i915_scheduler_wakeup() is called so the scheduler can do further
> + * processing (submit more work) at the end.
> + */
> +bool i915_scheduler_notify_request(struct drm_i915_gem_request *req)
> +{
> + struct drm_i915_private *dev_priv = to_i915(req->engine->dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *node = req->scheduler_qe;
> + unsigned long flags;
> +
> + if (!node)
> + return false;
> +
> + spin_lock_irqsave(&scheduler->lock, flags);
> +
> + WARN_ON(!I915_SQS_IS_FLYING(node));
> +
> + /* Node was in flight so mark it as complete. */
> + if (req->cancelled)
> + node->status = I915_SQS_DEAD;
> + else
> + node->status = I915_SQS_COMPLETE;
> +
> + scheduler->counts[req->engine->id].flying--;
> +
> + spin_unlock_irqrestore(&scheduler->lock, flags);
> +
> + return true;
> +}
> +
> +static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
> + struct i915_scheduler_queue_entry *remove)
> +{
> + struct i915_scheduler_queue_entry *node;
> + int i, r;
> + int count = 0;
> +
> + /*
> + * Ensure that a node is not being removed which is still dependent
> + * upon other (not completed) work. If that happens, it implies
> + * something has gone very wrong with the dependency tracking! Note
> + * that there is no need to worry if this node has been explicitly
> + * killed for some reason - it might be being killed before it got
> + * sent to the hardware.
> + */
> + if (remove->status != I915_SQS_DEAD) {
> + for (i = 0; i < remove->num_deps; i++)
> + if ((remove->dep_list[i]) &&
> + (!I915_SQS_IS_COMPLETE(remove->dep_list[i])))
> + count++;
> + WARN_ON(count);
> + }
> +
> + /*
> + * Remove this node from the dependency lists of any other node which
> + * might be waiting on it.
> + */
> + for (r = 0; r < I915_NUM_ENGINES; r++) {
> + for_each_scheduler_node(node, r) {
> + for (i = 0; i < node->num_deps; i++) {
> + if (node->dep_list[i] != remove)
> + continue;
> +
> + node->dep_list[i] = NULL;
Can the same node be listed in other's node dependency list multiple
times? If not you could break after clearing it and not iterate the rest
of the list.
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> +/**
> + * i915_scheduler_wakeup - wake the scheduler's worker thread
> + * @dev: DRM device
> + * Called at the end of seqno interrupt processing if any request has
> + * completed that corresponds to a scheduler node.
> + */
> +void i915_scheduler_wakeup(struct drm_device *dev)
> +{
> + /* XXX: Need to call i915_scheduler_remove() via work handler. */
> +}
> +
> +/**
> + * i915_scheduler_clean_node - free up any allocations/references
> + * associated with the given scheduler queue entry.
> + * @node: Queue entry structure which is complete
> + * After a give batch buffer completes on the hardware, all the information
> + * required to resubmit it is no longer required. However, the node entry
> + * itself might still be required for tracking purposes for a while longer.
> + * This function should be called as soon as the node is known to be complete
> + * so that these resources may be freed even though the node itself might
> + * hang around.
> + */
> +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node)
> +{
> + if (!I915_SQS_IS_COMPLETE(node)) {
> + WARN(!node->params.request->cancelled,
> + "Cleaning active node: %d!\n", node->status);
> + return;
> + }
> +
> + if (node->params.batch_obj) {
> + /*
> + * The batch buffer must be unpinned before it is unreferenced
> + * otherwise the unpin fails with a missing vma!?
> + */
> + if (node->params.dispatch_flags & I915_DISPATCH_SECURE)
> + i915_gem_execbuf_release_batch_obj(node->params.batch_obj);
> +
> + node->params.batch_obj = NULL;
> + }
> +
> + /* And anything else owned by the node: */
> + if (node->params.cliprects) {
> + kfree(node->params.cliprects);
> + node->params.cliprects = NULL;
> + }
> +}
> +
> +static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
> + struct intel_engine_cs *engine,
> + struct list_head *remove)
> +{
> + struct i915_scheduler_queue_entry *node, *node_next;
> + bool do_submit;
> +
> + spin_lock_irq(&scheduler->lock);
> +
> + INIT_LIST_HEAD(remove);
> + list_for_each_entry_safe(node, node_next, &scheduler->node_queue[engine->id], link) {
> + if (!I915_SQS_IS_COMPLETE(node))
> + break;
> +
> + list_del(&node->link);
> + list_add(&node->link, remove);
list_move
> +
> + /* Strip the dependency info while the mutex is still locked */
> + i915_scheduler_remove_dependent(scheduler, node);
> +
> + continue;
Could kill the continue. :)
> + }
> +
> + /*
> + * Release the interrupt reference count if there are no longer any
> + * nodes to worry about.
> + */
> + if (list_empty(&scheduler->node_queue[engine->id]) &&
> + (scheduler->flags[engine->id] & I915_SF_INTERRUPTS_ENABLED)) {
> + engine->irq_put(engine);
> + scheduler->flags[engine->id] &= ~I915_SF_INTERRUPTS_ENABLED;
> + }
> +
> + /* Launch more packets now? */
> + do_submit = (scheduler->counts[engine->id].queued > 0) &&
> + (scheduler->counts[engine->id].flying < scheduler->min_flying);
> +
> + spin_unlock_irq(&scheduler->lock);
> +
> + return do_submit;
> +}
> +
> +void i915_scheduler_process_work(struct intel_engine_cs *engine)
> +{
> + struct drm_i915_private *dev_priv = to_i915(engine->dev);
> + struct i915_scheduler *scheduler = dev_priv->scheduler;
> + struct i915_scheduler_queue_entry *node;
> + bool do_submit;
> + struct list_head remove;
Could do LIST_HEAD(remove); to declare an empty list and then wouldn't
have to do INIT_LIST_HEAD in i915_scheduler_remove. Would probably be
better to group the logic together like that.
> +
> + if (list_empty(&scheduler->node_queue[engine->id]))
> + return;
> +
> + /* Remove completed nodes. */
> + do_submit = i915_scheduler_remove(scheduler, engine, &remove);
> +
> + if (!do_submit && list_empty(&remove))
> + return;
> +
> + /* Need to grab the pm lock outside of the mutex lock */
> + if (do_submit)
> + intel_runtime_pm_get(dev_priv);
> +
> + mutex_lock(&engine->dev->struct_mutex);
> +
> + if (do_submit)
> + i915_scheduler_submit(engine);
> +
> + while (!list_empty(&remove)) {
> + node = list_first_entry(&remove, typeof(*node), link);
> + list_del(&node->link);
> +
> + /* Free up all the DRM references */
> + i915_scheduler_clean_node(node);
> +
> + /* And anything else owned by the node: */
> + node->params.request->scheduler_qe = NULL;
> + i915_gem_request_unreference(node->params.request);
> + kfree(node->dep_list);
> + kfree(node);
> + }
> +
> + mutex_unlock(&engine->dev->struct_mutex);
> +
> + if (do_submit)
> + intel_runtime_pm_put(dev_priv);
> +}
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> new file mode 100644
> index 0000000..c895c4c
> --- /dev/null
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -0,0 +1,113 @@
> +/*
> + * Copyright (c) 2014 Intel Corporation
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
> + * IN THE SOFTWARE.
> + *
> + */
> +
> +#ifndef _I915_SCHEDULER_H_
> +#define _I915_SCHEDULER_H_
> +
> +enum i915_scheduler_queue_status {
> + /* Limbo: */
> + I915_SQS_NONE = 0,
> + /* Not yet submitted to hardware: */
> + I915_SQS_QUEUED,
> + /* Popped from queue, ready to fly: */
> + I915_SQS_POPPED,
> + /* Sent to hardware for processing: */
> + I915_SQS_FLYING,
> + /* Finished processing on the hardware: */
> + I915_SQS_COMPLETE,
> + /* Killed by watchdog or catastrophic submission failure: */
> + I915_SQS_DEAD,
> + /* Limit value for use with arrays/loops */
> + I915_SQS_MAX
> +};
> +
> +#define I915_SQS_IS_QUEUED(node) (((node)->status == I915_SQS_QUEUED))
> +#define I915_SQS_IS_FLYING(node) (((node)->status == I915_SQS_FLYING))
> +#define I915_SQS_IS_COMPLETE(node) (((node)->status == I915_SQS_COMPLETE) || \
> + ((node)->status == I915_SQS_DEAD))
> +
> +struct i915_scheduler_obj_entry {
> + struct drm_i915_gem_object *obj;
> + bool read_only;
> +};
> +
> +struct i915_scheduler_queue_entry {
> + /* Any information required to submit this batch buffer to the hardware */
> + struct i915_execbuffer_params params;
> +
> + /* -1023 = lowest priority, 0 = default, 1023 = highest */
> + int32_t priority;
> + bool bumped;
> +
> + /* Objects referenced by this batch buffer */
> + struct i915_scheduler_obj_entry *objs;
> + int num_objs;
> +
> + /* Batch buffers this one is dependent upon */
> + struct i915_scheduler_queue_entry **dep_list;
> + int num_deps;
> +
> + enum i915_scheduler_queue_status status;
> + unsigned long stamp;
> +
> + /* List of all scheduler queue entry nodes */
> + struct list_head link;
> +};
> +
> +struct i915_scheduler_node_states {
> + uint32_t flying;
> + uint32_t queued;
> +};
> +
> +struct i915_scheduler {
> + struct list_head node_queue[I915_NUM_ENGINES];
> + uint32_t flags[I915_NUM_ENGINES];
> + spinlock_t lock;
> +
> + /* Node counts: */
> + struct i915_scheduler_node_states counts[I915_NUM_ENGINES];
> +
> + /* Tuning parameters: */
> + int32_t priority_level_min;
> + int32_t priority_level_max;
> + int32_t priority_level_bump;
> + int32_t priority_level_preempt;
> + uint32_t min_flying;
> +};
> +
> +/* Flag bits for i915_scheduler::flags */
> +enum {
> + I915_SF_INTERRUPTS_ENABLED = (1 << 0),
> + I915_SF_SUBMITTING = (1 << 1),
> +};
> +
> +bool i915_scheduler_is_enabled(struct drm_device *dev);
> +int i915_scheduler_init(struct drm_device *dev);
> +void i915_scheduler_destroy(struct drm_i915_private *dev_priv);
> +void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
> +int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
> +bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
> +void i915_scheduler_wakeup(struct drm_device *dev);
> +
> +#endif /* _I915_SCHEDULER_H_ */
>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: [PATCH v6 11/34] drm/i915: Added deferred work handler for scheduler
2016-04-20 17:13 ` [PATCH v6 11/34] drm/i915: Added deferred work handler for scheduler John.C.Harrison
@ 2016-06-10 16:29 ` Tvrtko Ursulin
0 siblings, 0 replies; 50+ messages in thread
From: Tvrtko Ursulin @ 2016-06-10 16:29 UTC (permalink / raw)
To: John.C.Harrison, Intel-GFX
Hi,
More random comments.
On 20/04/16 18:13, John.C.Harrison@Intel.com wrote:
> From: John Harrison <John.C.Harrison@Intel.com>
>
> The scheduler needs to do interrupt triggered work that is too complex
> to do in the interrupt handler. Thus it requires a deferred work
> handler to process such tasks asynchronously.
>
> v2: Updated to reduce mutex lock usage. The lock is now only held for
> the minimum time within the remove function rather than for the whole
> of the worker thread's operation.
>
> v5: Removed objectionable white space and added some documentation.
> [Joonas Lahtinen]
>
> v6: Updated to newer nightly (lots of ring -> engine renaming).
>
> Added an i915_scheduler_destroy() function instead of doing explicit
> clean up of scheduler internals from i915_driver_unload().
> [review feedback from Joonas Lahtinen]
>
> For: VIZ-1587
> Signed-off-by: John Harrison <John.C.Harrison@Intel.com>
> Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
> ---
> drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++
> drivers/gpu/drm/i915/i915_gem.c | 2 ++
> drivers/gpu/drm/i915/i915_scheduler.c | 28 ++++++++++++++++++++++++++--
> drivers/gpu/drm/i915/i915_scheduler.h | 1 +
> 4 files changed, 39 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 7b62e2c..ed9d829 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -1296,6 +1296,16 @@ struct i915_gem_mm {
> struct delayed_work retire_work;
>
> /**
> + * New scheme is to get an interrupt after every work packet
> + * in order to allow the low latency scheduling of pending
> + * packets. The idea behind adding new packets to a pending
> + * queue rather than directly into the hardware ring buffer
> + * is to allow high priority packets to over take low priority
> + * ones.
> + */
> + struct work_struct scheduler_work;
> +
> + /**
> * When we detect an idle GPU, we want to turn on
> * powersaving features. So once we see that there
> * are no more requests outstanding and no more
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 14dc641..50c45f3 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -5546,6 +5546,8 @@ i915_gem_load_init(struct drm_device *dev)
> i915_gem_retire_work_handler);
> INIT_DELAYED_WORK(&dev_priv->mm.idle_work,
> i915_gem_idle_work_handler);
> + INIT_WORK(&dev_priv->mm.scheduler_work,
> + i915_scheduler_work_handler);
> init_waitqueue_head(&dev_priv->gpu_error.reset_queue);
>
> dev_priv->relative_constants_mode = I915_EXEC_CONSTANTS_REL_GENERAL;
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.c b/drivers/gpu/drm/i915/i915_scheduler.c
> index 6dd9838..2dc5597 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.c
> +++ b/drivers/gpu/drm/i915/i915_scheduler.c
> @@ -95,6 +95,8 @@ void i915_scheduler_destroy(struct drm_i915_private *dev_priv)
> if (!scheduler)
> return;
>
> + cancel_work_sync(&dev_priv->mm.scheduler_work);
> +
> for (e = 0; e < I915_NUM_ENGINES; e++)
> WARN(!list_empty(&scheduler->node_queue[e]), "Destroying with list entries on engine %d!", e);
>
> @@ -738,7 +740,9 @@ static int i915_scheduler_remove_dependent(struct i915_scheduler *scheduler,
> */
> void i915_scheduler_wakeup(struct drm_device *dev)
> {
> - /* XXX: Need to call i915_scheduler_remove() via work handler. */
> + struct drm_i915_private *dev_priv = to_i915(dev);
> +
> + queue_work(dev_priv->wq, &dev_priv->mm.scheduler_work);
As I commented in person, sharing this wq with the rest of the driver
could introduce scheduling latency since it is an ordered (one work item
at a time) queue.
It would probably be good to create a dedicated wq for the scheduler,
maybe even WQ_HIGHPRI one.
> }
>
> /**
> @@ -820,7 +824,7 @@ static bool i915_scheduler_remove(struct i915_scheduler *scheduler,
> return do_submit;
> }
>
> -void i915_scheduler_process_work(struct intel_engine_cs *engine)
> +static void i915_scheduler_process_work(struct intel_engine_cs *engine)
> {
> struct drm_i915_private *dev_priv = to_i915(engine->dev);
> struct i915_scheduler *scheduler = dev_priv->scheduler;
> @@ -867,6 +871,26 @@ void i915_scheduler_process_work(struct intel_engine_cs *engine)
> }
>
> /**
> + * i915_scheduler_work_handler - scheduler's work handler callback.
> + * @work: Work structure
> + * A lot of the scheduler's work must be done asynchronously in response to
> + * an interrupt or other event. However, that work cannot be done at
> + * interrupt time or in the context of the event signaller (which might in
> + * fact be an interrupt). Thus a worker thread is required. This function
> + * will cause the thread to wake up and do its processing.
> + */
> +void i915_scheduler_work_handler(struct work_struct *work)
> +{
> + struct intel_engine_cs *engine;
> + struct drm_i915_private *dev_priv;
> +
> + dev_priv = container_of(work, struct drm_i915_private, mm.scheduler_work);
> +
> + for_each_engine(engine, dev_priv)
> + i915_scheduler_process_work(engine);
I wonder how easy or hard it would be to refactor things a bit so that
the i915_scheduler_process_work does not have to grab and drop the
global scheduler->lock multiple times. Maybe splitting
i915_scheduler_process_work in some stages could help, like the remove
bit and submit or something.
> +}
> +
> +/**
> * i915_scheduler_closefile - notify the scheduler that a DRM file handle
> * has been closed.
> * @dev: DRM device
> diff --git a/drivers/gpu/drm/i915/i915_scheduler.h b/drivers/gpu/drm/i915/i915_scheduler.h
> index 40398bb..b8d4a343 100644
> --- a/drivers/gpu/drm/i915/i915_scheduler.h
> +++ b/drivers/gpu/drm/i915/i915_scheduler.h
> @@ -110,5 +110,6 @@ void i915_scheduler_clean_node(struct i915_scheduler_queue_entry *node);
> int i915_scheduler_queue_execbuffer(struct i915_scheduler_queue_entry *qe);
> bool i915_scheduler_notify_request(struct drm_i915_gem_request *req);
> void i915_scheduler_wakeup(struct drm_device *dev);
> +void i915_scheduler_work_handler(struct work_struct *work);
>
> #endif /* _I915_SCHEDULER_H_ */
>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 50+ messages in thread
end of thread, other threads:[~2016-06-10 16:29 UTC | newest]
Thread overview: 50+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-20 17:13 [PATCH v6 00/34] GPU scheduler for i915 driver John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 01/34] drm/i915: Add total count to context status debugfs output John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 02/34] drm/i915: Prelude to splitting i915_gem_do_execbuffer in two John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 03/34] drm/i915: Split i915_dem_do_execbuffer() in half John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 04/34] drm/i915: Cache request pointer in *_submission_final() John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 05/34] drm/i915: Re-instate request->uniq because it is extremely useful John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 06/34] drm/i915: Start of GPU scheduler John.C.Harrison
2016-06-10 16:24 ` Tvrtko Ursulin
2016-04-20 17:13 ` [PATCH v6 07/34] drm/i915: Disable hardware semaphores when GPU scheduler is enabled John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 08/34] drm/i915: Force MMIO flips when scheduler enabled John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 09/34] drm/i915: Added scheduler hook when closing DRM file handles John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 10/34] drm/i915: Added scheduler hook into i915_gem_request_notify() John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 11/34] drm/i915: Added deferred work handler for scheduler John.C.Harrison
2016-06-10 16:29 ` Tvrtko Ursulin
2016-04-20 17:13 ` [PATCH v6 12/34] drm/i915: Redirect execbuffer_final() via scheduler John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 13/34] drm/i915: Keep the reserved space mechanism happy John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 14/34] drm/i915: Added tracking/locking of batch buffer objects John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 15/34] drm/i915: Hook scheduler node clean up into retire requests John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 16/34] drm/i915: Added scheduler support to __wait_request() calls John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 17/34] drm/i915: Added scheduler support to page fault handler John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 18/34] drm/i915: Added scheduler flush calls to ring throttle and idle functions John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 19/34] drm/i915: Add scheduler hook to GPU reset John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 20/34] drm/i915: Added a module parameter to allow the scheduler to be disabled John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 21/34] drm/i915: Support for 'unflushed' ring idle John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 22/34] drm/i915: Defer seqno allocation until actual hardware submission time John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 23/34] drm/i915: Added trace points to scheduler John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 24/34] drm/i915: Added scheduler queue throttling by DRM file handle John.C.Harrison
2016-05-06 13:19 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 25/34] drm/i915: Added debugfs interface to scheduler tuning parameters John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 26/34] drm/i915: Add early exit to execbuff_final() if insufficient ring space John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 27/34] drm/i915: Added scheduler statistic reporting to debugfs John.C.Harrison
2016-05-06 13:21 ` John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 28/34] drm/i915: Add scheduler support functions for TDR John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 29/34] drm/i915: Enable GPU scheduler by default John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 30/34] drm/i915: Add scheduling priority to per-context parameters John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 31/34] drm/i915: Add support for retro-actively banning batch buffers John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 32/34] drm/i915: Allow scheduler to manage inter-ring object synchronisation John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 33/34] drm/i915: Added debug state dump facilities to scheduler John.C.Harrison
2016-04-20 17:13 ` [PATCH v6 34/34] drm/i915: Scheduler state dump via debugfs John.C.Harrison
2016-04-20 17:13 ` [PATCH 1/1] drm/i915: Add wrapper for context priority interface John.C.Harrison
2016-04-20 17:13 ` [PATCH 1/2] igt/gem_ctx_param_basic: Updated to support scheduler " John.C.Harrison
2016-04-20 17:13 ` [PATCH 2/2] igt/gem_scheduler: Add gem_scheduler test John.C.Harrison
2016-04-21 9:43 ` ✓ Fi.CI.BAT: success for GPU scheduler for i915 driver (rev2) Patchwork
2016-04-22 15:37 ` [PATCH v6 00/34] GPU scheduler for i915 driver John Harrison
2016-04-23 9:57 ` ✗ Fi.CI.BAT: failure for GPU scheduler for i915 driver (rev2) Patchwork
2016-04-25 9:54 ` [PATCH v6 00/34] GPU scheduler for i915 driver Chris Wilson
2016-04-25 11:55 ` John Harrison
2016-04-26 13:20 ` Daniel Vetter
2016-05-05 11:54 ` John Harrison
2016-05-09 9:49 ` ✗ Fi.CI.BAT: warning for GPU scheduler for i915 driver (rev4) Patchwork
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.