* [Intel-gfx] [PATCH 01/19] drm/i915: Drop inspection of execbuf flags during evict
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 02/19] drm/i915/gem: Extract transient execbuf flags from i915_vma Maarten Lankhorst
` (19 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
From: Chris Wilson <chris@chris-wilson.co.uk>
With the goal of removing the serialisation from around execbuf, we will
no longer have the privilege of there being a single execbuf in flight
at any time and so will only be able to inspect the user's flags within
the carefully controlled execbuf context. i915_gem_evict_for_node() is
the only user outside of execbuf that currently peeks at the flag to
convert an overlapping softpinned request from ENOSPC to EINVAL. Retract
this nicety and only report ENOSPC if the location is in current use,
either due to this execbuf or another.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20191203121316.2972257-1-chris@chris-wilson.co.uk
---
drivers/gpu/drm/i915/i915_gem_evict.c | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c
index 0697bedebeef..b75fd3ccd63b 100644
--- a/drivers/gpu/drm/i915/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/i915_gem_evict.c
@@ -292,7 +292,8 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
GEM_BUG_ON(!drm_mm_node_allocated(node));
vma = container_of(node, typeof(*vma), node);
- /* If we are using coloring to insert guard pages between
+ /*
+ * If we are using coloring to insert guard pages between
* different cache domains within the address space, we have
* to check whether the objects on either side of our range
* abutt and conflict. If they are in conflict, then we evict
@@ -309,22 +310,18 @@ int i915_gem_evict_for_node(struct i915_address_space *vm,
}
}
- if (flags & PIN_NONBLOCK &&
- (i915_vma_is_pinned(vma) || i915_vma_is_active(vma))) {
+ if (i915_vma_is_pinned(vma)) {
ret = -ENOSPC;
break;
}
- /* Overlap of objects in the same batch? */
- if (i915_vma_is_pinned(vma)) {
+ if (flags & PIN_NONBLOCK && i915_vma_is_active(vma)) {
ret = -ENOSPC;
- if (vma->exec_flags &&
- *vma->exec_flags & EXEC_OBJECT_PINNED)
- ret = -EINVAL;
break;
}
- /* Never show fear in the face of dragons!
+ /*
+ * Never show fear in the face of dragons!
*
* We cannot directly remove this node from within this
* iterator and as with i915_gem_evict_something() we employ
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 02/19] drm/i915/gem: Extract transient execbuf flags from i915_vma
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 01/19] drm/i915: Drop inspection of execbuf flags during evict Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 03/19] drm/i915: Separate lookup and pinning in execbuf Maarten Lankhorst
` (18 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
From: Chris Wilson <chris@chris-wilson.co.uk>
For our convenience, and to avoid frequent allocations, we placed some
lists we use for execbuf inside the common i915_vma struct. As we look
to parallelise execbuf, such fields guarded by the struct_mutex BKL must
be pulled under local control. Instead of using the i915_vma as our
primary means of tracking the user's list of objects and their virtual
mappings, we use a local eb_vma with the same lists as before (just now
local not global).
This should allow us to only perform the lookup of vma used for
execution once during the execbuf ioctl, as currently we need to remove
our secrets from inside i915_vma everytime we drop the struct_mutex as
another execbuf may use the shared locations.
Once user visible consequence is that we can remove the requirement that
the execobj[] be unique, and only require that they do not conflict
(i.e. you cannot softpin the same object into two locations... hmm, we
could allow that if truly required, but atm we only allow one vma per
ppGTT per object.)
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20191203121316.2972257-2-chris@chris-wilson.co.uk
---
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 279 +++++++++---------
drivers/gpu/drm/i915/i915_vma_types.h | 11 -
2 files changed, 132 insertions(+), 158 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 87fa5f42c39a..efec64e92a74 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -28,6 +28,19 @@
#include "i915_sw_fence_work.h"
#include "i915_trace.h"
+struct eb_vma {
+ struct i915_vma *vma;
+ unsigned int flags;
+
+ /** This vma's place in the execbuf reservation list */
+ struct drm_i915_gem_exec_object2 *exec;
+ struct list_head bind_link;
+ struct list_head reloc_link;
+
+ struct hlist_node node;
+ u32 handle;
+};
+
enum {
FORCE_CPU_RELOC = 1,
FORCE_GTT_RELOC,
@@ -220,15 +233,14 @@ struct i915_execbuffer {
struct drm_file *file; /** per-file lookup tables and limits */
struct drm_i915_gem_execbuffer2 *args; /** ioctl parameters */
struct drm_i915_gem_exec_object2 *exec; /** ioctl execobj[] */
- struct i915_vma **vma;
- unsigned int *flags;
+ struct eb_vma *vma;
struct intel_engine_cs *engine; /** engine to queue the request to */
struct intel_context *context; /* logical state for the request */
struct i915_gem_context *gem_context; /** caller's context */
struct i915_request *request; /** our request to build */
- struct i915_vma *batch; /** identity of the batch obj/vma */
+ struct eb_vma *batch; /** identity of the batch obj/vma */
struct i915_vma *trampoline; /** trampoline used for chaining */
/** actual size of execobj[] as we may extend it for the cmdparser */
@@ -276,8 +288,6 @@ struct i915_execbuffer {
struct hlist_head *buckets; /** ht for relocation handles */
};
-#define exec_entry(EB, VMA) (&(EB)->exec[(VMA)->exec_flags - (EB)->flags])
-
static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
{
return intel_engine_requires_cmd_parser(eb->engine) ||
@@ -364,9 +374,9 @@ eb_vma_misplaced(const struct drm_i915_gem_exec_object2 *entry,
static inline bool
eb_pin_vma(struct i915_execbuffer *eb,
const struct drm_i915_gem_exec_object2 *entry,
- struct i915_vma *vma)
+ struct eb_vma *ev)
{
- unsigned int exec_flags = *vma->exec_flags;
+ struct i915_vma *vma = ev->vma;
u64 pin_flags;
if (vma->node.size)
@@ -375,24 +385,24 @@ eb_pin_vma(struct i915_execbuffer *eb,
pin_flags = entry->offset & PIN_OFFSET_MASK;
pin_flags |= PIN_USER | PIN_NOEVICT | PIN_OFFSET_FIXED;
- if (unlikely(exec_flags & EXEC_OBJECT_NEEDS_GTT))
+ if (unlikely(ev->flags & EXEC_OBJECT_NEEDS_GTT))
pin_flags |= PIN_GLOBAL;
if (unlikely(i915_vma_pin(vma, 0, 0, pin_flags)))
return false;
- if (unlikely(exec_flags & EXEC_OBJECT_NEEDS_FENCE)) {
+ if (unlikely(ev->flags & EXEC_OBJECT_NEEDS_FENCE)) {
if (unlikely(i915_vma_pin_fence(vma))) {
i915_vma_unpin(vma);
return false;
}
if (vma->fence)
- exec_flags |= __EXEC_OBJECT_HAS_FENCE;
+ ev->flags |= __EXEC_OBJECT_HAS_FENCE;
}
- *vma->exec_flags = exec_flags | __EXEC_OBJECT_HAS_PIN;
- return !eb_vma_misplaced(entry, vma, exec_flags);
+ ev->flags |= __EXEC_OBJECT_HAS_PIN;
+ return !eb_vma_misplaced(entry, vma, ev->flags);
}
static inline void __eb_unreserve_vma(struct i915_vma *vma, unsigned int flags)
@@ -406,13 +416,13 @@ static inline void __eb_unreserve_vma(struct i915_vma *vma, unsigned int flags)
}
static inline void
-eb_unreserve_vma(struct i915_vma *vma, unsigned int *flags)
+eb_unreserve_vma(struct eb_vma *ev)
{
- if (!(*flags & __EXEC_OBJECT_HAS_PIN))
+ if (!(ev->flags & __EXEC_OBJECT_HAS_PIN))
return;
- __eb_unreserve_vma(vma, *flags);
- *flags &= ~__EXEC_OBJECT_RESERVED;
+ __eb_unreserve_vma(ev->vma, ev->flags);
+ ev->flags &= ~__EXEC_OBJECT_RESERVED;
}
static int
@@ -420,7 +430,6 @@ eb_validate_vma(struct i915_execbuffer *eb,
struct drm_i915_gem_exec_object2 *entry,
struct i915_vma *vma)
{
- struct drm_i915_private *i915 = eb->i915;
if (unlikely(entry->flags & eb->invalid_flags))
return -EINVAL;
@@ -442,14 +451,6 @@ eb_validate_vma(struct i915_execbuffer *eb,
} else {
entry->pad_to_size = 0;
}
-
- if (unlikely(vma->exec_flags)) {
- drm_dbg(&i915->drm,
- "Object [handle %d, index %d] appears more than once in object list\n",
- entry->handle, (int)(entry - eb->exec));
- return -EINVAL;
- }
-
/*
* From drm_mm perspective address space is continuous,
* so from this point we're always using non-canonical
@@ -478,6 +479,7 @@ eb_add_vma(struct i915_execbuffer *eb,
struct i915_vma *vma)
{
struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
+ struct eb_vma *ev = &eb->vma[i];
int err;
GEM_BUG_ON(i915_vma_is_closed(vma));
@@ -488,25 +490,19 @@ eb_add_vma(struct i915_execbuffer *eb,
return err;
}
+ ev->vma = vma;
+ ev->exec = entry;
+ ev->flags = entry->flags;
+
if (eb->lut_size > 0) {
- vma->exec_handle = entry->handle;
- hlist_add_head(&vma->exec_node,
+ ev->handle = entry->handle;
+ hlist_add_head(&ev->node,
&eb->buckets[hash_32(entry->handle,
eb->lut_size)]);
}
if (entry->relocation_count)
- list_add_tail(&vma->reloc_link, &eb->relocs);
-
- /*
- * Stash a pointer from the vma to execobj, so we can query its flags,
- * size, alignment etc as provided by the user. Also we stash a pointer
- * to the vma inside the execobj so that we can use a direct lookup
- * to find the right target VMA when doing relocations.
- */
- eb->vma[i] = vma;
- eb->flags[i] = entry->flags;
- vma->exec_flags = &eb->flags[i];
+ list_add_tail(&ev->reloc_link, &eb->relocs);
/*
* SNA is doing fancy tricks with compressing batch buffers, which leads
@@ -519,28 +515,26 @@ eb_add_vma(struct i915_execbuffer *eb,
*/
if (i == batch_idx) {
if (entry->relocation_count &&
- !(eb->flags[i] & EXEC_OBJECT_PINNED))
- eb->flags[i] |= __EXEC_OBJECT_NEEDS_BIAS;
+ !(ev->flags & EXEC_OBJECT_PINNED))
+ ev->flags |= __EXEC_OBJECT_NEEDS_BIAS;
if (eb->reloc_cache.has_fence)
- eb->flags[i] |= EXEC_OBJECT_NEEDS_FENCE;
+ ev->flags |= EXEC_OBJECT_NEEDS_FENCE;
- eb->batch = vma;
+ eb->batch = ev;
}
err = 0;
- if (eb_pin_vma(eb, entry, vma)) {
+ if (eb_pin_vma(eb, entry, ev)) {
if (entry->offset != vma->node.start) {
entry->offset = vma->node.start | UPDATE;
eb->args->flags |= __EXEC_HAS_RELOC;
}
} else {
- eb_unreserve_vma(vma, vma->exec_flags);
+ eb_unreserve_vma(ev);
- list_add_tail(&vma->exec_link, &eb->unbound);
+ list_add_tail(&ev->bind_link, &eb->unbound);
if (drm_mm_node_allocated(&vma->node))
err = i915_vma_unbind(vma);
- if (unlikely(err))
- vma->exec_flags = NULL;
}
return err;
}
@@ -562,11 +556,11 @@ static inline int use_cpu_reloc(const struct reloc_cache *cache,
obj->cache_level != I915_CACHE_NONE);
}
-static int eb_reserve_vma(const struct i915_execbuffer *eb,
- struct i915_vma *vma)
+static int eb_reserve_vma(const struct i915_execbuffer *eb, struct eb_vma *ev)
{
- struct drm_i915_gem_exec_object2 *entry = exec_entry(eb, vma);
- unsigned int exec_flags = *vma->exec_flags;
+ struct drm_i915_gem_exec_object2 *entry = ev->exec;
+ unsigned int exec_flags = ev->flags;
+ struct i915_vma *vma = ev->vma;
u64 pin_flags;
int err;
@@ -613,8 +607,8 @@ static int eb_reserve_vma(const struct i915_execbuffer *eb,
exec_flags |= __EXEC_OBJECT_HAS_FENCE;
}
- *vma->exec_flags = exec_flags | __EXEC_OBJECT_HAS_PIN;
- GEM_BUG_ON(eb_vma_misplaced(entry, vma, exec_flags));
+ ev->flags = exec_flags | __EXEC_OBJECT_HAS_PIN;
+ GEM_BUG_ON(eb_vma_misplaced(entry, vma, ev->flags));
return 0;
}
@@ -623,7 +617,7 @@ static int eb_reserve(struct i915_execbuffer *eb)
{
const unsigned int count = eb->buffer_count;
struct list_head last;
- struct i915_vma *vma;
+ struct eb_vma *ev;
unsigned int i, pass;
int err;
@@ -644,8 +638,8 @@ static int eb_reserve(struct i915_execbuffer *eb)
pass = 0;
err = 0;
do {
- list_for_each_entry(vma, &eb->unbound, exec_link) {
- err = eb_reserve_vma(eb, vma);
+ list_for_each_entry(ev, &eb->unbound, bind_link) {
+ err = eb_reserve_vma(eb, ev);
if (err)
break;
}
@@ -656,26 +650,26 @@ static int eb_reserve(struct i915_execbuffer *eb)
INIT_LIST_HEAD(&eb->unbound);
INIT_LIST_HEAD(&last);
for (i = 0; i < count; i++) {
- unsigned int flags = eb->flags[i];
- struct i915_vma *vma = eb->vma[i];
+ struct eb_vma *ev = &eb->vma[i];
+ unsigned int flags = ev->flags;
if (flags & EXEC_OBJECT_PINNED &&
flags & __EXEC_OBJECT_HAS_PIN)
continue;
- eb_unreserve_vma(vma, &eb->flags[i]);
+ eb_unreserve_vma(ev);
if (flags & EXEC_OBJECT_PINNED)
/* Pinned must have their slot */
- list_add(&vma->exec_link, &eb->unbound);
+ list_add(&ev->bind_link, &eb->unbound);
else if (flags & __EXEC_OBJECT_NEEDS_MAP)
/* Map require the lowest 256MiB (aperture) */
- list_add_tail(&vma->exec_link, &eb->unbound);
+ list_add_tail(&ev->bind_link, &eb->unbound);
else if (!(flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS))
/* Prioritise 4GiB region for restricted bo */
- list_add(&vma->exec_link, &last);
+ list_add(&ev->bind_link, &last);
else
- list_add_tail(&vma->exec_link, &last);
+ list_add_tail(&ev->bind_link, &last);
}
list_splice_tail(&last, &eb->unbound);
@@ -791,10 +785,8 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
if (unlikely(err))
goto err_vma;
- GEM_BUG_ON(vma != eb->vma[i]);
- GEM_BUG_ON(vma->exec_flags != &eb->flags[i]);
GEM_BUG_ON(drm_mm_node_allocated(&vma->node) &&
- eb_vma_misplaced(&eb->exec[i], vma, eb->flags[i]));
+ eb_vma_misplaced(&eb->exec[i], vma, eb->vma[i].flags));
}
mutex_unlock(&eb->gem_context->mutex);
@@ -805,27 +797,27 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
err_obj:
i915_gem_object_put(obj);
err_vma:
- eb->vma[i] = NULL;
+ eb->vma[i].vma = NULL;
err_ctx:
mutex_unlock(&eb->gem_context->mutex);
return err;
}
-static struct i915_vma *
+static struct eb_vma *
eb_get_vma(const struct i915_execbuffer *eb, unsigned long handle)
{
if (eb->lut_size < 0) {
if (handle >= -eb->lut_size)
return NULL;
- return eb->vma[handle];
+ return &eb->vma[handle];
} else {
struct hlist_head *head;
- struct i915_vma *vma;
+ struct eb_vma *ev;
head = &eb->buckets[hash_32(handle, eb->lut_size)];
- hlist_for_each_entry(vma, head, exec_node) {
- if (vma->exec_handle == handle)
- return vma;
+ hlist_for_each_entry(ev, head, node) {
+ if (ev->handle == handle)
+ return ev;
}
return NULL;
}
@@ -837,20 +829,18 @@ static void eb_release_vmas(const struct i915_execbuffer *eb)
unsigned int i;
for (i = 0; i < count; i++) {
- struct i915_vma *vma = eb->vma[i];
- unsigned int flags = eb->flags[i];
+ struct eb_vma *ev = &eb->vma[i];
+ struct i915_vma *vma = ev->vma;
if (!vma)
break;
- GEM_BUG_ON(vma->exec_flags != &eb->flags[i]);
- vma->exec_flags = NULL;
- eb->vma[i] = NULL;
+ eb->vma[i].vma = NULL;
- if (flags & __EXEC_OBJECT_HAS_PIN)
- __eb_unreserve_vma(vma, flags);
+ if (ev->flags & __EXEC_OBJECT_HAS_PIN)
+ __eb_unreserve_vma(vma, ev->flags);
- if (flags & __EXEC_OBJECT_HAS_REF)
+ if (ev->flags & __EXEC_OBJECT_HAS_REF)
i915_vma_put(vma);
}
}
@@ -1329,11 +1319,11 @@ relocate_entry(struct i915_vma *vma,
static u64
eb_relocate_entry(struct i915_execbuffer *eb,
- struct i915_vma *vma,
+ struct eb_vma *ev,
const struct drm_i915_gem_relocation_entry *reloc)
{
struct drm_i915_private *i915 = eb->i915;
- struct i915_vma *target;
+ struct eb_vma *target;
int err;
/* we've already hold a reference to all valid objects */
@@ -1365,7 +1355,7 @@ eb_relocate_entry(struct i915_execbuffer *eb,
}
if (reloc->write_domain) {
- *target->exec_flags |= EXEC_OBJECT_WRITE;
+ target->flags |= EXEC_OBJECT_WRITE;
/*
* Sandybridge PPGTT errata: We need a global gtt mapping
@@ -1375,7 +1365,8 @@ eb_relocate_entry(struct i915_execbuffer *eb,
*/
if (reloc->write_domain == I915_GEM_DOMAIN_INSTRUCTION &&
IS_GEN(eb->i915, 6)) {
- err = i915_vma_bind(target, target->obj->cache_level,
+ err = i915_vma_bind(target->vma,
+ target->vma->obj->cache_level,
PIN_GLOBAL, NULL);
if (WARN_ONCE(err,
"Unexpected failure to bind target VMA!"))
@@ -1388,17 +1379,17 @@ eb_relocate_entry(struct i915_execbuffer *eb,
* more work needs to be done.
*/
if (!DBG_FORCE_RELOC &&
- gen8_canonical_addr(target->node.start) == reloc->presumed_offset)
+ gen8_canonical_addr(target->vma->node.start) == reloc->presumed_offset)
return 0;
/* Check that the relocation address is valid... */
if (unlikely(reloc->offset >
- vma->size - (eb->reloc_cache.use_64bit_reloc ? 8 : 4))) {
+ ev->vma->size - (eb->reloc_cache.use_64bit_reloc ? 8 : 4))) {
drm_dbg(&i915->drm, "Relocation beyond object bounds: "
"target %d offset %d size %d.\n",
reloc->target_handle,
(int)reloc->offset,
- (int)vma->size);
+ (int)ev->vma->size);
return -EINVAL;
}
if (unlikely(reloc->offset & 3)) {
@@ -1417,18 +1408,18 @@ eb_relocate_entry(struct i915_execbuffer *eb,
* do relocations we are already stalling, disable the user's opt
* out of our synchronisation.
*/
- *vma->exec_flags &= ~EXEC_OBJECT_ASYNC;
+ ev->flags &= ~EXEC_OBJECT_ASYNC;
/* and update the user's relocation entry */
- return relocate_entry(vma, reloc, eb, target);
+ return relocate_entry(ev->vma, reloc, eb, target->vma);
}
-static int eb_relocate_vma(struct i915_execbuffer *eb, struct i915_vma *vma)
+static int eb_relocate_vma(struct i915_execbuffer *eb, struct eb_vma *ev)
{
#define N_RELOC(x) ((x) / sizeof(struct drm_i915_gem_relocation_entry))
struct drm_i915_gem_relocation_entry stack[N_RELOC(512)];
struct drm_i915_gem_relocation_entry __user *urelocs;
- const struct drm_i915_gem_exec_object2 *entry = exec_entry(eb, vma);
+ const struct drm_i915_gem_exec_object2 *entry = ev->exec;
unsigned int remain;
urelocs = u64_to_user_ptr(entry->relocs_ptr);
@@ -1468,7 +1459,7 @@ static int eb_relocate_vma(struct i915_execbuffer *eb, struct i915_vma *vma)
remain -= count;
do {
- u64 offset = eb_relocate_entry(eb, vma, r);
+ u64 offset = eb_relocate_entry(eb, ev, r);
if (likely(offset == 0)) {
} else if ((s64)offset < 0) {
@@ -1511,16 +1502,16 @@ static int eb_relocate_vma(struct i915_execbuffer *eb, struct i915_vma *vma)
}
static int
-eb_relocate_vma_slow(struct i915_execbuffer *eb, struct i915_vma *vma)
+eb_relocate_vma_slow(struct i915_execbuffer *eb, struct eb_vma *ev)
{
- const struct drm_i915_gem_exec_object2 *entry = exec_entry(eb, vma);
+ const struct drm_i915_gem_exec_object2 *entry = ev->exec;
struct drm_i915_gem_relocation_entry *relocs =
u64_to_ptr(typeof(*relocs), entry->relocs_ptr);
unsigned int i;
int err;
for (i = 0; i < entry->relocation_count; i++) {
- u64 offset = eb_relocate_entry(eb, vma, &relocs[i]);
+ u64 offset = eb_relocate_entry(eb, ev, &relocs[i]);
if ((s64)offset < 0) {
err = (int)offset;
@@ -1661,7 +1652,7 @@ static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
{
struct drm_device *dev = &eb->i915->drm;
bool have_copy = false;
- struct i915_vma *vma;
+ struct eb_vma *ev;
int err = 0;
repeat:
@@ -1717,15 +1708,15 @@ static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
GEM_BUG_ON(!eb->batch);
- list_for_each_entry(vma, &eb->relocs, reloc_link) {
+ list_for_each_entry(ev, &eb->relocs, reloc_link) {
if (!have_copy) {
pagefault_disable();
- err = eb_relocate_vma(eb, vma);
+ err = eb_relocate_vma(eb, ev);
pagefault_enable();
if (err)
goto repeat;
} else {
- err = eb_relocate_vma_slow(eb, vma);
+ err = eb_relocate_vma_slow(eb, ev);
if (err)
goto err;
}
@@ -1770,10 +1761,10 @@ static int eb_relocate(struct i915_execbuffer *eb)
/* The objects are in their final locations, apply the relocations. */
if (eb->args->flags & __EXEC_HAS_RELOC) {
- struct i915_vma *vma;
+ struct eb_vma *ev;
- list_for_each_entry(vma, &eb->relocs, reloc_link) {
- if (eb_relocate_vma(eb, vma))
+ list_for_each_entry(ev, &eb->relocs, reloc_link) {
+ if (eb_relocate_vma(eb, ev))
goto slow;
}
}
@@ -1794,39 +1785,34 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
ww_acquire_init(&acquire, &reservation_ww_class);
for (i = 0; i < count; i++) {
- struct i915_vma *vma = eb->vma[i];
+ struct eb_vma *ev = &eb->vma[i];
+ struct i915_vma *vma = ev->vma;
err = ww_mutex_lock_interruptible(&vma->resv->lock, &acquire);
- if (!err)
- continue;
-
- GEM_BUG_ON(err == -EALREADY); /* No duplicate vma */
-
if (err == -EDEADLK) {
GEM_BUG_ON(i == 0);
do {
int j = i - 1;
- ww_mutex_unlock(&eb->vma[j]->resv->lock);
+ ww_mutex_unlock(&eb->vma[j].vma->resv->lock);
- swap(eb->flags[i], eb->flags[j]);
swap(eb->vma[i], eb->vma[j]);
- eb->vma[i]->exec_flags = &eb->flags[i];
} while (--i);
- GEM_BUG_ON(vma != eb->vma[0]);
- vma->exec_flags = &eb->flags[0];
err = ww_mutex_lock_slow_interruptible(&vma->resv->lock,
&acquire);
}
+ if (err == -EALREADY)
+ err = 0;
if (err)
break;
}
ww_acquire_done(&acquire);
while (i--) {
- unsigned int flags = eb->flags[i];
- struct i915_vma *vma = eb->vma[i];
+ struct eb_vma *ev = &eb->vma[i];
+ struct i915_vma *vma = ev->vma;
+ unsigned int flags = ev->flags;
struct drm_i915_gem_object *obj = vma->obj;
assert_vma_held(vma);
@@ -1870,10 +1856,10 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
i915_vma_unlock(vma);
__eb_unreserve_vma(vma, flags);
- vma->exec_flags = NULL;
-
if (unlikely(flags & __EXEC_OBJECT_HAS_REF))
i915_vma_put(vma);
+
+ ev->vma = NULL;
}
ww_acquire_fini(&acquire);
@@ -2008,7 +1994,7 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
if (!pw)
return -ENOMEM;
- err = i915_active_acquire(&eb->batch->active);
+ err = i915_active_acquire(&eb->batch->vma->active);
if (err)
goto err_free;
@@ -2025,7 +2011,7 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
dma_fence_work_init(&pw->base, &eb_parse_ops);
pw->engine = eb->engine;
- pw->batch = eb->batch;
+ pw->batch = eb->batch->vma;
pw->batch_offset = eb->batch_start_offset;
pw->batch_length = eb->batch_len;
pw->shadow = shadow;
@@ -2067,7 +2053,7 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
err_shadow:
i915_active_release(&shadow->active);
err_batch:
- i915_active_release(&eb->batch->active);
+ i915_active_release(&eb->batch->vma->active);
err_free:
kfree(pw);
return err;
@@ -2130,15 +2116,13 @@ static int eb_parse(struct i915_execbuffer *eb)
if (err)
goto err_trampoline;
- eb->vma[eb->buffer_count] = i915_vma_get(shadow);
- eb->flags[eb->buffer_count] =
+ eb->vma[eb->buffer_count].vma = i915_vma_get(shadow);
+ eb->vma[eb->buffer_count].flags =
__EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_REF;
- shadow->exec_flags = &eb->flags[eb->buffer_count];
- eb->buffer_count++;
+ eb->batch = &eb->vma[eb->buffer_count++];
eb->trampoline = trampoline;
eb->batch_start_offset = 0;
- eb->batch = shadow;
shadow->private = pool;
return 0;
@@ -2165,7 +2149,7 @@ add_to_client(struct i915_request *rq, struct drm_file *file)
spin_unlock(&file_priv->mm.lock);
}
-static int eb_submit(struct i915_execbuffer *eb)
+static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
{
int err;
@@ -2192,7 +2176,7 @@ static int eb_submit(struct i915_execbuffer *eb)
}
err = eb->engine->emit_bb_start(eb->request,
- eb->batch->node.start +
+ batch->node.start +
eb->batch_start_offset,
eb->batch_len,
eb->batch_flags);
@@ -2572,6 +2556,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
struct dma_fence *in_fence = NULL;
struct dma_fence *exec_fence = NULL;
struct sync_file *out_fence = NULL;
+ struct i915_vma *batch;
int out_fence_fd = -1;
int err;
@@ -2586,9 +2571,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,
args->flags |= __EXEC_HAS_RELOC;
eb.exec = exec;
- eb.vma = (struct i915_vma **)(exec + args->buffer_count + 1);
- eb.vma[0] = NULL;
- eb.flags = (unsigned int *)(eb.vma + args->buffer_count + 1);
+ eb.vma = (struct eb_vma *)(exec + args->buffer_count + 1);
+ eb.vma[0].vma = NULL;
eb.invalid_flags = __EXEC_OBJECT_UNKNOWN_FLAGS;
reloc_cache_init(&eb.reloc_cache, eb.i915);
@@ -2673,21 +2657,23 @@ i915_gem_do_execbuffer(struct drm_device *dev,
goto err_vma;
}
- if (unlikely(*eb.batch->exec_flags & EXEC_OBJECT_WRITE)) {
+ if (unlikely(eb.batch->flags & EXEC_OBJECT_WRITE)) {
drm_dbg(&i915->drm,
"Attempting to use self-modifying batch buffer\n");
err = -EINVAL;
goto err_vma;
}
- if (eb.batch_start_offset > eb.batch->size ||
- eb.batch_len > eb.batch->size - eb.batch_start_offset) {
+
+ if (range_overflows_t(u64,
+ eb.batch_start_offset, eb.batch_len,
+ eb.batch->vma->size)) {
drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
err = -EINVAL;
goto err_vma;
}
if (eb.batch_len == 0)
- eb.batch_len = eb.batch->size - eb.batch_start_offset;
+ eb.batch_len = eb.batch->vma->size - eb.batch_start_offset;
err = eb_parse(&eb);
if (err)
@@ -2697,6 +2683,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
* snb/ivb/vlv conflate the "batch in ppgtt" bit with the "non-secure
* batch" bit. Hence we need to pin secure batches into the global gtt.
* hsw should have this fixed, but bdw mucks it up again. */
+ batch = eb.batch->vma;
if (eb.batch_flags & I915_DISPATCH_SECURE) {
struct i915_vma *vma;
@@ -2710,13 +2697,13 @@ i915_gem_do_execbuffer(struct drm_device *dev,
* fitting due to fragmentation.
* So this is actually safe.
*/
- vma = i915_gem_object_ggtt_pin(eb.batch->obj, NULL, 0, 0, 0);
+ vma = i915_gem_object_ggtt_pin(batch->obj, NULL, 0, 0, 0);
if (IS_ERR(vma)) {
err = PTR_ERR(vma);
goto err_vma;
}
- eb.batch = vma;
+ batch = vma;
}
/* All GPU relocation batches must be submitted prior to the user rq */
@@ -2763,12 +2750,12 @@ i915_gem_do_execbuffer(struct drm_device *dev,
* inactive_list and lose its active reference. Hence we do not need
* to explicitly hold another reference here.
*/
- eb.request->batch = eb.batch;
- if (eb.batch->private)
- intel_engine_pool_mark_active(eb.batch->private, eb.request);
+ eb.request->batch = batch;
+ if (batch->private)
+ intel_engine_pool_mark_active(batch->private, eb.request);
trace_i915_request_queue(eb.request, eb.batch_flags);
- err = eb_submit(&eb);
+ err = eb_submit(&eb, batch);
err_request:
add_to_client(eb.request, file);
i915_request_get(eb.request);
@@ -2791,9 +2778,9 @@ i915_gem_do_execbuffer(struct drm_device *dev,
err_batch_unpin:
if (eb.batch_flags & I915_DISPATCH_SECURE)
- i915_vma_unpin(eb.batch);
- if (eb.batch->private)
- intel_engine_pool_put(eb.batch->private);
+ i915_vma_unpin(batch);
+ if (batch->private)
+ intel_engine_pool_put(batch->private);
err_vma:
if (eb.exec)
eb_release_vmas(&eb);
@@ -2818,9 +2805,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
static size_t eb_element_size(void)
{
- return (sizeof(struct drm_i915_gem_exec_object2) +
- sizeof(struct i915_vma *) +
- sizeof(unsigned int));
+ return sizeof(struct drm_i915_gem_exec_object2) + sizeof(struct eb_vma);
}
static bool check_buffer_count(size_t count)
diff --git a/drivers/gpu/drm/i915/i915_vma_types.h b/drivers/gpu/drm/i915/i915_vma_types.h
index e0942efd5236..63831cdb7402 100644
--- a/drivers/gpu/drm/i915/i915_vma_types.h
+++ b/drivers/gpu/drm/i915/i915_vma_types.h
@@ -273,21 +273,10 @@ struct i915_vma {
struct rb_node obj_node;
struct hlist_node obj_hash;
- /** This vma's place in the execbuf reservation list */
- struct list_head exec_link;
- struct list_head reloc_link;
-
/** This vma's place in the eviction list */
struct list_head evict_link;
struct list_head closed_link;
-
- /**
- * Used for performing relocations during execbuffer insertion.
- */
- unsigned int *exec_flags;
- struct hlist_node exec_node;
- u32 exec_handle;
};
#endif
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 03/19] drm/i915: Separate lookup and pinning in execbuf.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 01/19] drm/i915: Drop inspection of execbuf flags during evict Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 02/19] drm/i915/gem: Extract transient execbuf flags from i915_vma Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 04/19] drm/i915: Add an implementation for i915_gem_ww_ctx locking, v2 Maarten Lankhorst
` (17 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
We want to get rid of struct mutex, but vma reservation was entangled
with vma lookup. Make vma lookup and validation a preparation step,
and vma pinning a repeatable step. This allows us to keep all the looked
up information when dropping all locks, which saves an extra lookup during
relocation slowpath.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 98 ++++++++++---------
drivers/gpu/drm/i915/i915_drv.h | 6 --
2 files changed, 54 insertions(+), 50 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index efec64e92a74..9970f5111daa 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -57,8 +57,7 @@ enum {
#define __EXEC_OBJECT_RESERVED (__EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_FENCE)
#define __EXEC_HAS_RELOC BIT(31)
-#define __EXEC_VALIDATED BIT(30)
-#define __EXEC_INTERNAL_FLAGS (~0u << 30)
+#define __EXEC_INTERNAL_FLAGS (~0u << 31)
#define UPDATE PIN_OFFSET_FIXED
#define BATCH_OFFSET_BIAS (256*1024)
@@ -484,11 +483,9 @@ eb_add_vma(struct i915_execbuffer *eb,
GEM_BUG_ON(i915_vma_is_closed(vma));
- if (!(eb->args->flags & __EXEC_VALIDATED)) {
- err = eb_validate_vma(eb, entry, vma);
- if (unlikely(err))
- return err;
- }
+ err = eb_validate_vma(eb, entry, vma);
+ if (unlikely(err))
+ return err;
ev->vma = vma;
ev->exec = entry;
@@ -523,20 +520,7 @@ eb_add_vma(struct i915_execbuffer *eb,
eb->batch = ev;
}
- err = 0;
- if (eb_pin_vma(eb, entry, ev)) {
- if (entry->offset != vma->node.start) {
- entry->offset = vma->node.start | UPDATE;
- eb->args->flags |= __EXEC_HAS_RELOC;
- }
- } else {
- eb_unreserve_vma(ev);
-
- list_add_tail(&ev->bind_link, &eb->unbound);
- if (drm_mm_node_allocated(&vma->node))
- err = i915_vma_unbind(vma);
- }
- return err;
+ return 0;
}
static inline int use_cpu_reloc(const struct reloc_cache *cache,
@@ -727,7 +711,6 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
int err;
INIT_LIST_HEAD(&eb->relocs);
- INIT_LIST_HEAD(&eb->unbound);
batch = eb_batch_index(eb);
@@ -784,15 +767,11 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
err = eb_add_vma(eb, i, batch, vma);
if (unlikely(err))
goto err_vma;
-
- GEM_BUG_ON(drm_mm_node_allocated(&vma->node) &&
- eb_vma_misplaced(&eb->exec[i], vma, eb->vma[i].flags));
}
mutex_unlock(&eb->gem_context->mutex);
- eb->args->flags |= __EXEC_VALIDATED;
- return eb_reserve(eb);
+ return 0;
err_obj:
i915_gem_object_put(obj);
@@ -803,6 +782,41 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
return err;
}
+static int eb_validate_vmas(struct i915_execbuffer *eb)
+{
+ unsigned int i;
+ int err;
+
+ INIT_LIST_HEAD(&eb->unbound);
+
+ for (i = 0; i < eb->buffer_count; i++) {
+ struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
+ struct eb_vma *ev = &eb->vma[i];
+ struct i915_vma *vma = ev->vma;
+
+ if (eb_pin_vma(eb, entry, ev)) {
+ if (entry->offset != vma->node.start) {
+ entry->offset = vma->node.start | UPDATE;
+ eb->args->flags |= __EXEC_HAS_RELOC;
+ }
+ } else {
+ eb_unreserve_vma(ev);
+
+ list_add_tail(&ev->bind_link, &eb->unbound);
+ if (drm_mm_node_allocated(&vma->node)) {
+ err = i915_vma_unbind(vma);
+ if (err)
+ return err;
+ }
+ }
+
+ GEM_BUG_ON(drm_mm_node_allocated(&vma->node) &&
+ eb_vma_misplaced(&eb->exec[i], vma, ev->flags));
+ }
+
+ return eb_reserve(eb);
+}
+
static struct eb_vma *
eb_get_vma(const struct i915_execbuffer *eb, unsigned long handle)
{
@@ -835,22 +849,12 @@ static void eb_release_vmas(const struct i915_execbuffer *eb)
if (!vma)
break;
- eb->vma[i].vma = NULL;
-
- if (ev->flags & __EXEC_OBJECT_HAS_PIN)
- __eb_unreserve_vma(vma, ev->flags);
+ eb_unreserve_vma(ev);
if (ev->flags & __EXEC_OBJECT_HAS_REF)
i915_vma_put(vma);
- }
-}
-static void eb_reset_vmas(const struct i915_execbuffer *eb)
-{
- eb_release_vmas(eb);
- if (eb->lut_size > 0)
- memset(eb->buckets, 0,
- sizeof(struct hlist_head) << eb->lut_size);
+ }
}
static void eb_destroy(const struct i915_execbuffer *eb)
@@ -1662,7 +1666,7 @@ static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
}
/* We may process another execbuffer during the unlock... */
- eb_reset_vmas(eb);
+ eb_release_vmas(eb);
mutex_unlock(&dev->struct_mutex);
/*
@@ -1695,14 +1699,14 @@ static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
/* A frequent cause for EAGAIN are currently unavailable client pages */
flush_workqueue(eb->i915->mm.userptr_wq);
- err = i915_mutex_lock_interruptible(dev);
+ err = mutex_lock_interruptible(&dev->struct_mutex);
if (err) {
mutex_lock(&dev->struct_mutex);
goto out;
}
/* reacquire the objects */
- err = eb_lookup_vmas(eb);
+ err = eb_validate_vmas(eb);
if (err)
goto err;
@@ -1756,7 +1760,13 @@ static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
static int eb_relocate(struct i915_execbuffer *eb)
{
- if (eb_lookup_vmas(eb))
+ int err;
+
+ err = eb_lookup_vmas(eb);
+ if (err)
+ return err;
+
+ if (eb_validate_vmas(eb))
goto slow;
/* The objects are in their final locations, apply the relocations. */
@@ -2640,7 +2650,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
if (unlikely(err))
goto err_context;
- err = i915_mutex_lock_interruptible(dev);
+ err = mutex_lock_interruptible(&dev->struct_mutex);
if (err)
goto err_engine;
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index da509d9b8895..6f351d85b226 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1831,12 +1831,6 @@ int i915_gem_object_unbind(struct drm_i915_gem_object *obj,
void i915_gem_runtime_suspend(struct drm_i915_private *dev_priv);
-static inline int __must_check
-i915_mutex_lock_interruptible(struct drm_device *dev)
-{
- return mutex_lock_interruptible(&dev->struct_mutex);
-}
-
int i915_gem_dumb_create(struct drm_file *file_priv,
struct drm_device *dev,
struct drm_mode_create_dumb *args);
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 04/19] drm/i915: Add an implementation for i915_gem_ww_ctx locking, v2.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (2 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 03/19] drm/i915: Separate lookup and pinning in execbuf Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 05/19] drm/i915: Remove locking from i915_gem_object_prepare_read/write Maarten Lankhorst
` (16 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
i915_gem_ww_ctx is used to lock all gem bo's for pinning and memory
eviction. We don't use it yet, but lets start adding the definition
first.
To use it, we have to pass a non-NULL ww to gem_object_lock, and don't
unlock directly. It is done in i915_gem_ww_ctx_fini.
Changes since v1:
- Change ww_ctx and obj order in locking functions (Jonas Lahtinen)
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/display/intel_display.c | 4 +-
.../gpu/drm/i915/gem/i915_gem_client_blt.c | 2 +-
drivers/gpu/drm/i915/gem/i915_gem_context.c | 2 +-
drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c | 4 +-
drivers/gpu/drm/i915/gem/i915_gem_domain.c | 10 ++--
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 4 +-
drivers/gpu/drm/i915/gem/i915_gem_object.c | 2 +-
drivers/gpu/drm/i915/gem/i915_gem_object.h | 38 +++++++++++---
.../gpu/drm/i915/gem/i915_gem_object_blt.c | 2 +-
.../gpu/drm/i915/gem/i915_gem_object_types.h | 9 ++++
drivers/gpu/drm/i915/gem/i915_gem_pm.c | 2 +-
drivers/gpu/drm/i915/gem/i915_gem_tiling.c | 2 +-
.../gpu/drm/i915/gem/selftests/huge_pages.c | 2 +-
.../i915/gem/selftests/i915_gem_client_blt.c | 2 +-
.../i915/gem/selftests/i915_gem_coherency.c | 10 ++--
.../drm/i915/gem/selftests/i915_gem_context.c | 4 +-
.../drm/i915/gem/selftests/i915_gem_mman.c | 4 +-
.../drm/i915/gem/selftests/i915_gem_phys.c | 2 +-
drivers/gpu/drm/i915/gt/intel_gt.c | 2 +-
.../gpu/drm/i915/gt/selftest_workarounds.c | 2 +-
drivers/gpu/drm/i915/gvt/cmd_parser.c | 2 +-
drivers/gpu/drm/i915/i915_gem.c | 52 +++++++++++++++++--
drivers/gpu/drm/i915/i915_gem.h | 11 ++++
drivers/gpu/drm/i915/selftests/i915_gem.c | 41 +++++++++++++++
drivers/gpu/drm/i915/selftests/i915_vma.c | 2 +-
.../drm/i915/selftests/intel_memory_region.c | 2 +-
26 files changed, 175 insertions(+), 44 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c
index e09d3c93c52b..71d6daa273c0 100644
--- a/drivers/gpu/drm/i915/display/intel_display.c
+++ b/drivers/gpu/drm/i915/display/intel_display.c
@@ -2301,7 +2301,7 @@ intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
void intel_unpin_fb_vma(struct i915_vma *vma, unsigned long flags)
{
- i915_gem_object_lock(vma->obj);
+ i915_gem_object_lock(vma->obj, NULL);
if (flags & PLANE_HAS_FENCE)
i915_vma_unpin_fence(vma);
i915_gem_object_unpin_from_display_plane(vma);
@@ -17284,7 +17284,7 @@ static int intel_framebuffer_init(struct intel_framebuffer *intel_fb,
if (!intel_fb->frontbuffer)
return -ENOMEM;
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
tiling = i915_gem_object_get_tiling(obj);
stride = i915_gem_object_get_stride(obj);
i915_gem_object_unlock(obj);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c b/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c
index 81366aa4812b..d92a35163330 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c
@@ -287,7 +287,7 @@ int i915_gem_schedule_fill_pages_blt(struct drm_i915_gem_object *obj,
dma_fence_init(&work->dma, &clear_pages_work_ops, &fence_lock, 0, 0);
i915_sw_fence_init(&work->wait, clear_pages_work_notify);
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_sw_fence_await_reservation(&work->wait,
obj->base.resv, NULL,
true, I915_FENCE_TIMEOUT,
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index 3e82739bdbc0..e946b395ddf6 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -114,7 +114,7 @@ static void lut_close(struct i915_gem_context *ctx)
continue;
rcu_read_unlock();
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
list_for_each_entry(lut, &obj->lut_list, obj_link) {
if (lut->ctx != ctx)
continue;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
index 7db5a793739d..cfadccfc2990 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c
@@ -128,7 +128,7 @@ static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_dire
if (err)
return err;
- err = i915_gem_object_lock_interruptible(obj);
+ err = i915_gem_object_lock_interruptible(obj, NULL);
if (err)
goto out;
@@ -149,7 +149,7 @@ static int i915_gem_end_cpu_access(struct dma_buf *dma_buf, enum dma_data_direct
if (err)
return err;
- err = i915_gem_object_lock_interruptible(obj);
+ err = i915_gem_object_lock_interruptible(obj, NULL);
if (err)
goto out;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
index 0cc40e77bbd2..f4602faa8db9 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
@@ -32,7 +32,7 @@ void i915_gem_object_flush_if_display(struct drm_i915_gem_object *obj)
if (!i915_gem_object_is_framebuffer(obj))
return;
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
__i915_gem_object_flush_for_display(obj);
i915_gem_object_unlock(obj);
}
@@ -197,7 +197,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
if (ret)
return ret;
- ret = i915_gem_object_lock_interruptible(obj);
+ ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
return ret;
@@ -541,7 +541,7 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
if (err)
goto out;
- err = i915_gem_object_lock_interruptible(obj);
+ err = i915_gem_object_lock_interruptible(obj, NULL);
if (err)
goto out_unpin;
@@ -581,7 +581,7 @@ int i915_gem_object_prepare_read(struct drm_i915_gem_object *obj,
if (!i915_gem_object_has_struct_page(obj))
return -ENODEV;
- ret = i915_gem_object_lock_interruptible(obj);
+ ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
return ret;
@@ -635,7 +635,7 @@ int i915_gem_object_prepare_write(struct drm_i915_gem_object *obj,
if (!i915_gem_object_has_struct_page(obj))
return -ENODEV;
- ret = i915_gem_object_lock_interruptible(obj);
+ ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 9970f5111daa..f567e4c99e29 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -759,7 +759,7 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
lut->handle = handle;
lut->ctx = eb->gem_context;
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
list_add(&lut->obj_link, &obj->lut_list);
i915_gem_object_unlock(obj);
@@ -1013,7 +1013,7 @@ static void *reloc_iomap(struct drm_i915_gem_object *obj,
if (use_cpu_reloc(cache, obj))
return NULL;
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_gtt_domain(obj, true);
i915_gem_object_unlock(obj);
if (err)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.c b/drivers/gpu/drm/i915/gem/i915_gem_object.c
index 35985218bd85..7c56be94354d 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.c
@@ -104,7 +104,7 @@ void i915_gem_close_object(struct drm_gem_object *gem, struct drm_file *file)
struct i915_lut_handle *lut, *ln;
LIST_HEAD(close);
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
list_for_each_entry_safe(lut, ln, &obj->lut_list, obj_link) {
struct i915_gem_context *ctx = lut->ctx;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h
index e44a2f40b520..95e732f83d06 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h
@@ -112,20 +112,44 @@ i915_gem_object_put(struct drm_i915_gem_object *obj)
#define assert_object_held(obj) dma_resv_assert_held((obj)->base.resv)
-static inline void i915_gem_object_lock(struct drm_i915_gem_object *obj)
+static inline int __i915_gem_object_lock(struct drm_i915_gem_object *obj,
+ struct i915_gem_ww_ctx *ww,
+ bool intr)
{
- dma_resv_lock(obj->base.resv, NULL);
+ int ret;
+
+ if (intr)
+ ret = dma_resv_lock_interruptible(obj->base.resv, ww ? &ww->ctx : NULL);
+ else
+ ret = dma_resv_lock(obj->base.resv, ww ? &ww->ctx : NULL);
+
+ if (!ret && ww)
+ list_add_tail(&obj->obj_link, &ww->obj_list);
+ if (ret == -EALREADY)
+ ret = 0;
+
+ if (ret == -EDEADLK)
+ ww->contended = obj;
+
+ return ret;
}
-static inline bool i915_gem_object_trylock(struct drm_i915_gem_object *obj)
+static inline int i915_gem_object_lock(struct drm_i915_gem_object *obj,
+ struct i915_gem_ww_ctx *ww)
{
- return dma_resv_trylock(obj->base.resv);
+ return __i915_gem_object_lock(obj, ww, ww && ww->intr);
}
-static inline int
-i915_gem_object_lock_interruptible(struct drm_i915_gem_object *obj)
+static inline int i915_gem_object_lock_interruptible(struct drm_i915_gem_object *obj,
+ struct i915_gem_ww_ctx *ww)
{
- return dma_resv_lock_interruptible(obj->base.resv, NULL);
+ WARN_ON(ww && !ww->intr);
+ return __i915_gem_object_lock(obj, ww, true);
+}
+
+static inline bool i915_gem_object_trylock(struct drm_i915_gem_object *obj)
+{
+ return dma_resv_trylock(obj->base.resv);
}
static inline void i915_gem_object_unlock(struct drm_i915_gem_object *obj)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c b/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c
index 70809d8897cd..2b23e000bb57 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c
@@ -142,7 +142,7 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
return err;
if (obj->cache_dirty & ~obj->cache_coherent) {
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
i915_gem_clflush_object(obj, 0);
i915_gem_object_unlock(obj);
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h
index a0b10bcd8d8a..26350defc44b 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h
@@ -120,6 +120,15 @@ struct drm_i915_gem_object {
*/
struct list_head lut_list;
+ /**
+ * @obj_link: Link into @i915_gem_ww_ctx.obj_list
+ *
+ * When we lock this object through i915_gem_object_lock() with a
+ * context, we add it to the list to ensure we can unlock everything
+ * when i915_gem_ww_ctx_backoff() or i915_gem_ww_ctx_fini() are called.
+ */
+ struct list_head obj_link;
+
/** Stolen memory for this object, instead of being backed by shmem. */
struct drm_mm_node *stolen;
union {
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_pm.c b/drivers/gpu/drm/i915/gem/i915_gem_pm.c
index 3d215164dd5a..40d3e40500fa 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_pm.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_pm.c
@@ -84,7 +84,7 @@ void i915_gem_suspend_late(struct drm_i915_private *i915)
spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
drm_WARN_ON(&i915->drm,
i915_gem_object_set_to_gtt_domain(obj, false));
i915_gem_object_unlock(obj);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_tiling.c b/drivers/gpu/drm/i915/gem/i915_gem_tiling.c
index 6c7825a2dc2a..b63da25f3497 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_tiling.c
@@ -236,7 +236,7 @@ i915_gem_object_set_tiling(struct drm_i915_gem_object *obj,
* whilst executing a fenced command for an untiled object.
*/
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
if (i915_gem_object_is_framebuffer(obj)) {
i915_gem_object_unlock(obj);
return -EBUSY;
diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
index 2d0fd50c5312..2a2f33228b3d 100644
--- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
+++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
@@ -974,7 +974,7 @@ static int gpu_write(struct intel_context *ce,
{
int err;
- i915_gem_object_lock(vma->obj);
+ i915_gem_object_lock(vma->obj, NULL);
err = i915_gem_object_set_to_gtt_domain(vma->obj, true);
i915_gem_object_unlock(vma->obj);
if (err)
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
index b972be165e85..a4bf026b2988 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c
@@ -72,7 +72,7 @@ static int __igt_client_fill(struct intel_engine_cs *engine)
if (err)
goto err_unpin;
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_cpu_domain(obj, false);
i915_gem_object_unlock(obj);
if (err)
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
index 3f6079e1dfb6..f5a0c7b1e113 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
@@ -82,7 +82,7 @@ static int gtt_set(struct context *ctx, unsigned long offset, u32 v)
u32 __iomem *map;
int err = 0;
- i915_gem_object_lock(ctx->obj);
+ i915_gem_object_lock(ctx->obj, NULL);
err = i915_gem_object_set_to_gtt_domain(ctx->obj, true);
i915_gem_object_unlock(ctx->obj);
if (err)
@@ -115,7 +115,7 @@ static int gtt_get(struct context *ctx, unsigned long offset, u32 *v)
u32 __iomem *map;
int err = 0;
- i915_gem_object_lock(ctx->obj);
+ i915_gem_object_lock(ctx->obj, NULL);
err = i915_gem_object_set_to_gtt_domain(ctx->obj, false);
i915_gem_object_unlock(ctx->obj);
if (err)
@@ -147,7 +147,7 @@ static int wc_set(struct context *ctx, unsigned long offset, u32 v)
u32 *map;
int err;
- i915_gem_object_lock(ctx->obj);
+ i915_gem_object_lock(ctx->obj, NULL);
err = i915_gem_object_set_to_wc_domain(ctx->obj, true);
i915_gem_object_unlock(ctx->obj);
if (err)
@@ -168,7 +168,7 @@ static int wc_get(struct context *ctx, unsigned long offset, u32 *v)
u32 *map;
int err;
- i915_gem_object_lock(ctx->obj);
+ i915_gem_object_lock(ctx->obj, NULL);
err = i915_gem_object_set_to_wc_domain(ctx->obj, false);
i915_gem_object_unlock(ctx->obj);
if (err)
@@ -191,7 +191,7 @@ static int gpu_set(struct context *ctx, unsigned long offset, u32 v)
u32 *cs;
int err;
- i915_gem_object_lock(ctx->obj);
+ i915_gem_object_lock(ctx->obj, NULL);
err = i915_gem_object_set_to_gtt_domain(ctx->obj, true);
i915_gem_object_unlock(ctx->obj);
if (err)
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
index 375d864736f3..3eb58a28fa6e 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
@@ -950,7 +950,7 @@ emit_rpcs_query(struct drm_i915_gem_object *obj,
if (IS_ERR(vma))
return PTR_ERR(vma);
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_gtt_domain(obj, false);
i915_gem_object_unlock(obj);
if (err)
@@ -1691,7 +1691,7 @@ static int read_from_scratch(struct i915_gem_context *ctx,
i915_request_add(rq);
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_cpu_domain(obj, false);
i915_gem_object_unlock(obj);
if (err)
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
index ef7c74cff28a..533d65b9b9e8 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c
@@ -103,7 +103,7 @@ static int check_partial_mapping(struct drm_i915_gem_object *obj,
GEM_BUG_ON(i915_gem_object_get_tiling(obj) != tile->tiling);
GEM_BUG_ON(i915_gem_object_get_stride(obj) != tile->stride);
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_gtt_domain(obj, true);
i915_gem_object_unlock(obj);
if (err) {
@@ -188,7 +188,7 @@ static int check_partial_mappings(struct drm_i915_gem_object *obj,
GEM_BUG_ON(i915_gem_object_get_tiling(obj) != tile->tiling);
GEM_BUG_ON(i915_gem_object_get_stride(obj) != tile->stride);
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_gtt_domain(obj, true);
i915_gem_object_unlock(obj);
if (err) {
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c
index 34932871b3a5..a94243dc4c5c 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c
@@ -44,7 +44,7 @@ static int mock_phys_object(void *arg)
}
/* Make the object dirty so that put_pages must do copy back the data */
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_gtt_domain(obj, true);
i915_gem_object_unlock(obj);
if (err) {
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
index f1f1b306e0af..fb4a487e0341 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt.c
@@ -485,7 +485,7 @@ static int __engines_record_defaults(struct intel_gt *gt)
if (err)
goto out;
- i915_gem_object_lock(state->obj);
+ i915_gem_object_lock(state->obj, NULL);
err = i915_gem_object_set_to_cpu_domain(state->obj, false);
i915_gem_object_unlock(state->obj);
if (err)
diff --git a/drivers/gpu/drm/i915/gt/selftest_workarounds.c b/drivers/gpu/drm/i915/gt/selftest_workarounds.c
index 5ed323254ee1..282f51feacd5 100644
--- a/drivers/gpu/drm/i915/gt/selftest_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/selftest_workarounds.c
@@ -214,7 +214,7 @@ static int check_whitelist(struct i915_gem_context *ctx,
return PTR_ERR(results);
err = 0;
- i915_gem_object_lock(results);
+ i915_gem_object_lock(results, NULL);
intel_wedge_on_timeout(&wedge, engine->gt, HZ / 5) /* safety net! */
err = i915_gem_object_set_to_cpu_domain(results, false);
i915_gem_object_unlock(results);
diff --git a/drivers/gpu/drm/i915/gvt/cmd_parser.c b/drivers/gpu/drm/i915/gvt/cmd_parser.c
index 21a176cd8acc..a54d96a144be 100644
--- a/drivers/gpu/drm/i915/gvt/cmd_parser.c
+++ b/drivers/gpu/drm/i915/gvt/cmd_parser.c
@@ -2954,7 +2954,7 @@ static int shadow_indirect_ctx(struct intel_shadow_wa_ctx *wa_ctx)
goto put_obj;
}
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
ret = i915_gem_object_set_to_cpu_domain(obj, false);
i915_gem_object_unlock(obj);
if (ret) {
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index d92cf966fa3f..7ea845be76b3 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -416,7 +416,7 @@ i915_gem_gtt_pread(struct drm_i915_gem_object *obj,
GEM_BUG_ON(!drm_mm_node_allocated(&node));
}
- ret = i915_gem_object_lock_interruptible(obj);
+ ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
goto out_unpin;
@@ -615,7 +615,7 @@ i915_gem_gtt_pwrite_fast(struct drm_i915_gem_object *obj,
GEM_BUG_ON(!drm_mm_node_allocated(&node));
}
- ret = i915_gem_object_lock_interruptible(obj);
+ ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
goto out_unpin;
@@ -1271,7 +1271,7 @@ int i915_gem_freeze_late(struct drm_i915_private *i915)
i915_gem_drain_freed_objects(i915);
list_for_each_entry(obj, &i915->mm.shrink_list, mm.link) {
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
drm_WARN_ON(&i915->drm,
i915_gem_object_set_to_cpu_domain(obj, true));
i915_gem_object_unlock(obj);
@@ -1325,6 +1325,52 @@ int i915_gem_open(struct drm_i915_private *i915, struct drm_file *file)
return ret;
}
+void i915_gem_ww_ctx_init(struct i915_gem_ww_ctx *ww, bool intr)
+{
+ ww_acquire_init(&ww->ctx, &reservation_ww_class);
+ INIT_LIST_HEAD(&ww->obj_list);
+ ww->intr = intr;
+ ww->contended = NULL;
+}
+
+static void i915_gem_ww_ctx_unlock_all(struct i915_gem_ww_ctx *ww)
+{
+ struct drm_i915_gem_object *obj;
+
+ while ((obj = list_first_entry_or_null(&ww->obj_list, struct drm_i915_gem_object, obj_link))) {
+ list_del(&obj->obj_link);
+ i915_gem_object_unlock(obj);
+ }
+}
+
+void i915_gem_ww_ctx_fini(struct i915_gem_ww_ctx *ww)
+{
+ i915_gem_ww_ctx_unlock_all(ww);
+ WARN_ON(ww->contended);
+ ww_acquire_fini(&ww->ctx);
+}
+
+int __must_check i915_gem_ww_ctx_backoff(struct i915_gem_ww_ctx *ww)
+{
+ int ret = 0;
+
+ if (WARN_ON(!ww->contended))
+ return -EINVAL;
+
+ i915_gem_ww_ctx_unlock_all(ww);
+ if (ww->intr)
+ ret = dma_resv_lock_slow_interruptible(ww->contended->base.resv, &ww->ctx);
+ else
+ dma_resv_lock_slow(ww->contended->base.resv, &ww->ctx);
+
+ if (!ret)
+ list_add_tail(&ww->contended->obj_link, &ww->obj_list);
+
+ ww->contended = NULL;
+
+ return ret;
+}
+
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftests/mock_gem_device.c"
#include "selftests/i915_gem.c"
diff --git a/drivers/gpu/drm/i915/i915_gem.h b/drivers/gpu/drm/i915/i915_gem.h
index 1753c84d6c0d..988755dbf4be 100644
--- a/drivers/gpu/drm/i915/i915_gem.h
+++ b/drivers/gpu/drm/i915/i915_gem.h
@@ -116,4 +116,15 @@ static inline bool __tasklet_is_scheduled(struct tasklet_struct *t)
return test_bit(TASKLET_STATE_SCHED, &t->state);
}
+struct i915_gem_ww_ctx {
+ struct ww_acquire_ctx ctx;
+ struct list_head obj_list;
+ bool intr;
+ struct drm_i915_gem_object *contended;
+};
+
+void i915_gem_ww_ctx_init(struct i915_gem_ww_ctx *ctx, bool intr);
+void i915_gem_ww_ctx_fini(struct i915_gem_ww_ctx *ctx);
+int __must_check i915_gem_ww_ctx_backoff(struct i915_gem_ww_ctx *ctx);
+
#endif /* __I915_GEM_H__ */
diff --git a/drivers/gpu/drm/i915/selftests/i915_gem.c b/drivers/gpu/drm/i915/selftests/i915_gem.c
index 623759b73bb4..6444b7558cc0 100644
--- a/drivers/gpu/drm/i915/selftests/i915_gem.c
+++ b/drivers/gpu/drm/i915/selftests/i915_gem.c
@@ -201,11 +201,52 @@ static int igt_gem_hibernate(void *arg)
return err;
}
+static int igt_gem_ww_ctx(void *arg)
+{
+ struct drm_i915_private *i915 = arg;
+ struct drm_i915_gem_object *obj, *obj2;
+ struct i915_gem_ww_ctx ww;
+ int err = 0;
+
+ obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ obj2 = i915_gem_object_create_internal(i915, PAGE_SIZE);
+ if (IS_ERR(obj)) {
+ err = PTR_ERR(obj);
+ goto put1;
+ }
+
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ /* Lock the objects, twice for good measure (-EALREADY handling) */
+ err = i915_gem_object_lock(obj, &ww);
+ if (!err)
+ err = i915_gem_object_lock_interruptible(obj, &ww);
+ if (!err)
+ err = i915_gem_object_lock_interruptible(obj2, &ww);
+ if (!err)
+ err = i915_gem_object_lock(obj2, &ww);
+
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
+ i915_gem_object_put(obj2);
+put1:
+ i915_gem_object_put(obj);
+ return err;
+}
+
int i915_gem_live_selftests(struct drm_i915_private *i915)
{
static const struct i915_subtest tests[] = {
SUBTEST(igt_gem_suspend),
SUBTEST(igt_gem_hibernate),
+ SUBTEST(igt_gem_ww_ctx),
};
if (intel_gt_is_wedged(&i915->gt))
diff --git a/drivers/gpu/drm/i915/selftests/i915_vma.c b/drivers/gpu/drm/i915/selftests/i915_vma.c
index 58b5f40a07dd..9bc00c724b0e 100644
--- a/drivers/gpu/drm/i915/selftests/i915_vma.c
+++ b/drivers/gpu/drm/i915/selftests/i915_vma.c
@@ -892,7 +892,7 @@ static int igt_vma_remapped_gtt(void *arg)
unsigned int x, y;
int err;
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_gtt_domain(obj, true);
i915_gem_object_unlock(obj);
if (err)
diff --git a/drivers/gpu/drm/i915/selftests/intel_memory_region.c b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
index 2a1d4ba1f9f3..feab1852e2e3 100644
--- a/drivers/gpu/drm/i915/selftests/intel_memory_region.c
+++ b/drivers/gpu/drm/i915/selftests/intel_memory_region.c
@@ -509,7 +509,7 @@ static int igt_lmem_write_cpu(void *arg)
if (err)
goto out_unpin;
- i915_gem_object_lock(obj);
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_wc_domain(obj, true);
i915_gem_object_unlock(obj);
if (err)
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 05/19] drm/i915: Remove locking from i915_gem_object_prepare_read/write
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (3 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 04/19] drm/i915: Add an implementation for i915_gem_ww_ctx locking, v2 Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 06/19] drm/i915: Parse command buffer earlier in eb_relocate(slow) Maarten Lankhorst
` (15 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
Execbuffer submission will perform its own WW locking, and we
cannot rely on the implicit lock there.
This also makes it clear that the GVT code will get a lockdep splat when
multiple batchbuffer shadows need to be performed in the same instance,
fix that up.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_domain.c | 20 ++++++-------------
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 13 ++++++++++--
drivers/gpu/drm/i915/gem/i915_gem_object.h | 1 -
.../gpu/drm/i915/gem/selftests/huge_pages.c | 5 ++++-
.../i915/gem/selftests/i915_gem_coherency.c | 14 +++++++++----
.../drm/i915/gem/selftests/i915_gem_context.c | 12 ++++++++---
drivers/gpu/drm/i915/gt/intel_renderstate.c | 5 ++++-
drivers/gpu/drm/i915/gvt/cmd_parser.c | 9 ++++++++-
drivers/gpu/drm/i915/i915_gem.c | 20 +++++++++++++++++--
9 files changed, 70 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_domain.c b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
index f4602faa8db9..e9d3b587f562 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_domain.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_domain.c
@@ -581,19 +581,17 @@ int i915_gem_object_prepare_read(struct drm_i915_gem_object *obj,
if (!i915_gem_object_has_struct_page(obj))
return -ENODEV;
- ret = i915_gem_object_lock_interruptible(obj, NULL);
- if (ret)
- return ret;
+ assert_object_held(obj);
ret = i915_gem_object_wait(obj,
I915_WAIT_INTERRUPTIBLE,
MAX_SCHEDULE_TIMEOUT);
if (ret)
- goto err_unlock;
+ return ret;
ret = i915_gem_object_pin_pages(obj);
if (ret)
- goto err_unlock;
+ return ret;
if (obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_READ ||
!static_cpu_has(X86_FEATURE_CLFLUSH)) {
@@ -621,8 +619,6 @@ int i915_gem_object_prepare_read(struct drm_i915_gem_object *obj,
err_unpin:
i915_gem_object_unpin_pages(obj);
-err_unlock:
- i915_gem_object_unlock(obj);
return ret;
}
@@ -635,20 +631,18 @@ int i915_gem_object_prepare_write(struct drm_i915_gem_object *obj,
if (!i915_gem_object_has_struct_page(obj))
return -ENODEV;
- ret = i915_gem_object_lock_interruptible(obj, NULL);
- if (ret)
- return ret;
+ assert_object_held(obj);
ret = i915_gem_object_wait(obj,
I915_WAIT_INTERRUPTIBLE |
I915_WAIT_ALL,
MAX_SCHEDULE_TIMEOUT);
if (ret)
- goto err_unlock;
+ return ret;
ret = i915_gem_object_pin_pages(obj);
if (ret)
- goto err_unlock;
+ return ret;
if (obj->cache_coherent & I915_BO_CACHE_COHERENT_FOR_WRITE ||
!static_cpu_has(X86_FEATURE_CLFLUSH)) {
@@ -685,7 +679,5 @@ int i915_gem_object_prepare_write(struct drm_i915_gem_object *obj,
err_unpin:
i915_gem_object_unpin_pages(obj);
-err_unlock:
- i915_gem_object_unlock(obj);
return ret;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index f567e4c99e29..e7dd29541727 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -933,11 +933,14 @@ static void reloc_cache_reset(struct reloc_cache *cache)
vaddr = unmask_page(cache->vaddr);
if (cache->vaddr & KMAP) {
+ struct drm_i915_gem_object *obj =
+ (struct drm_i915_gem_object *)cache->node.mm;
if (cache->vaddr & CLFLUSH_AFTER)
mb();
kunmap_atomic(vaddr);
- i915_gem_object_finish_access((struct drm_i915_gem_object *)cache->node.mm);
+ i915_gem_object_finish_access(obj);
+ i915_gem_object_unlock(obj);
} else {
struct i915_ggtt *ggtt = cache_to_ggtt(cache);
@@ -972,10 +975,16 @@ static void *reloc_kmap(struct drm_i915_gem_object *obj,
unsigned int flushes;
int err;
- err = i915_gem_object_prepare_write(obj, &flushes);
+ err = i915_gem_object_lock_interruptible(obj, NULL);
if (err)
return ERR_PTR(err);
+ err = i915_gem_object_prepare_write(obj, &flushes);
+ if (err) {
+ i915_gem_object_unlock(obj);
+ return ERR_PTR(err);
+ }
+
BUILD_BUG_ON(KMAP & CLFLUSH_FLAGS);
BUILD_BUG_ON((KMAP | CLFLUSH_FLAGS) & PAGE_MASK);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object.h b/drivers/gpu/drm/i915/gem/i915_gem_object.h
index 95e732f83d06..9f381490b7c6 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object.h
@@ -436,7 +436,6 @@ static inline void
i915_gem_object_finish_access(struct drm_i915_gem_object *obj)
{
i915_gem_object_unpin_pages(obj);
- i915_gem_object_unlock(obj);
}
static inline struct intel_engine_cs *
diff --git a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
index 2a2f33228b3d..36db96f902cc 100644
--- a/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
+++ b/drivers/gpu/drm/i915/gem/selftests/huge_pages.c
@@ -991,9 +991,10 @@ __cpu_check_shmem(struct drm_i915_gem_object *obj, u32 dword, u32 val)
unsigned long n;
int err;
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_prepare_read(obj, &needs_flush);
if (err)
- return err;
+ goto err_unlock;
for (n = 0; n < obj->base.size >> PAGE_SHIFT; ++n) {
u32 *ptr = kmap_atomic(i915_gem_object_get_page(obj, n));
@@ -1013,6 +1014,8 @@ __cpu_check_shmem(struct drm_i915_gem_object *obj, u32 dword, u32 val)
}
i915_gem_object_finish_access(obj);
+err_unlock:
+ i915_gem_object_unlock(obj);
return err;
}
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
index f5a0c7b1e113..99f8466a108a 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c
@@ -27,9 +27,10 @@ static int cpu_set(struct context *ctx, unsigned long offset, u32 v)
u32 *cpu;
int err;
+ i915_gem_object_lock(ctx->obj, NULL);
err = i915_gem_object_prepare_write(ctx->obj, &needs_clflush);
if (err)
- return err;
+ goto out;
page = i915_gem_object_get_page(ctx->obj, offset >> PAGE_SHIFT);
map = kmap_atomic(page);
@@ -46,7 +47,9 @@ static int cpu_set(struct context *ctx, unsigned long offset, u32 v)
kunmap_atomic(map);
i915_gem_object_finish_access(ctx->obj);
- return 0;
+out:
+ i915_gem_object_unlock(ctx->obj);
+ return err;
}
static int cpu_get(struct context *ctx, unsigned long offset, u32 *v)
@@ -57,9 +60,10 @@ static int cpu_get(struct context *ctx, unsigned long offset, u32 *v)
u32 *cpu;
int err;
+ i915_gem_object_lock(ctx->obj, NULL);
err = i915_gem_object_prepare_read(ctx->obj, &needs_clflush);
if (err)
- return err;
+ goto out;
page = i915_gem_object_get_page(ctx->obj, offset >> PAGE_SHIFT);
map = kmap_atomic(page);
@@ -73,7 +77,9 @@ static int cpu_get(struct context *ctx, unsigned long offset, u32 *v)
kunmap_atomic(map);
i915_gem_object_finish_access(ctx->obj);
- return 0;
+out:
+ i915_gem_object_unlock(ctx->obj);
+ return err;
}
static int gtt_set(struct context *ctx, unsigned long offset, u32 v)
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
index 3eb58a28fa6e..d3d8981b268f 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
@@ -461,9 +461,10 @@ static int cpu_fill(struct drm_i915_gem_object *obj, u32 value)
unsigned int n, m, need_flush;
int err;
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_prepare_write(obj, &need_flush);
if (err)
- return err;
+ goto out;
for (n = 0; n < real_page_count(obj); n++) {
u32 *map;
@@ -479,7 +480,9 @@ static int cpu_fill(struct drm_i915_gem_object *obj, u32 value)
i915_gem_object_finish_access(obj);
obj->read_domains = I915_GEM_DOMAIN_GTT | I915_GEM_DOMAIN_CPU;
obj->write_domain = 0;
- return 0;
+out:
+ i915_gem_object_unlock(obj);
+ return err;
}
static noinline int cpu_check(struct drm_i915_gem_object *obj,
@@ -488,9 +491,10 @@ static noinline int cpu_check(struct drm_i915_gem_object *obj,
unsigned int n, m, needs_flush;
int err;
+ i915_gem_object_lock(obj, NULL);
err = i915_gem_object_prepare_read(obj, &needs_flush);
if (err)
- return err;
+ goto out_unlock;
for (n = 0; n < real_page_count(obj); n++) {
u32 *map;
@@ -527,6 +531,8 @@ static noinline int cpu_check(struct drm_i915_gem_object *obj,
}
i915_gem_object_finish_access(obj);
+out_unlock:
+ i915_gem_object_unlock(obj);
return err;
}
diff --git a/drivers/gpu/drm/i915/gt/intel_renderstate.c b/drivers/gpu/drm/i915/gt/intel_renderstate.c
index 5954ecc3207f..fff61e86cfbf 100644
--- a/drivers/gpu/drm/i915/gt/intel_renderstate.c
+++ b/drivers/gpu/drm/i915/gt/intel_renderstate.c
@@ -74,9 +74,10 @@ static int render_state_setup(struct intel_renderstate *so,
u32 *d;
int ret;
+ i915_gem_object_lock(so->vma->obj, NULL);
ret = i915_gem_object_prepare_write(so->vma->obj, &needs_clflush);
if (ret)
- return ret;
+ goto out_unlock;
d = kmap_atomic(i915_gem_object_get_dirty_page(so->vma->obj, 0));
@@ -157,6 +158,8 @@ static int render_state_setup(struct intel_renderstate *so,
ret = 0;
out:
i915_gem_object_finish_access(so->vma->obj);
+out_unlock:
+ i915_gem_object_unlock(so->vma->obj);
return ret;
err:
diff --git a/drivers/gpu/drm/i915/gvt/cmd_parser.c b/drivers/gpu/drm/i915/gvt/cmd_parser.c
index a54d96a144be..5d17ee2bdb9b 100644
--- a/drivers/gpu/drm/i915/gvt/cmd_parser.c
+++ b/drivers/gpu/drm/i915/gvt/cmd_parser.c
@@ -1865,10 +1865,14 @@ static int perform_bb_shadow(struct parser_exec_state *s)
goto err_free_bb;
}
- ret = i915_gem_object_prepare_write(bb->obj, &bb->clflush);
+ ret = i915_gem_object_lock_interruptible(bb->obj, NULL);
if (ret)
goto err_free_obj;
+ ret = i915_gem_object_prepare_write(bb->obj, &bb->clflush);
+ if (ret)
+ goto err_unlock;
+
bb->va = i915_gem_object_pin_map(bb->obj, I915_MAP_WB);
if (IS_ERR(bb->va)) {
ret = PTR_ERR(bb->va);
@@ -1893,6 +1897,7 @@ static int perform_bb_shadow(struct parser_exec_state *s)
if (ret)
goto err_unmap;
+ i915_gem_object_unlock(bb->obj);
INIT_LIST_HEAD(&bb->list);
list_add(&bb->list, &s->workload->shadow_bb);
@@ -1919,6 +1924,8 @@ static int perform_bb_shadow(struct parser_exec_state *s)
i915_gem_object_unpin_map(bb->obj);
err_finish_shmem_access:
i915_gem_object_finish_access(bb->obj);
+err_unlock:
+ i915_gem_object_unlock(bb->obj);
err_free_obj:
i915_gem_object_put(bb->obj);
err_free_bb:
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 7ea845be76b3..84b95fbd0b11 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -331,12 +331,20 @@ i915_gem_shmem_pread(struct drm_i915_gem_object *obj,
u64 remain;
int ret;
- ret = i915_gem_object_prepare_read(obj, &needs_clflush);
+ ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
return ret;
+ ret = i915_gem_object_prepare_read(obj, &needs_clflush);
+ if (ret) {
+ i915_gem_object_unlock(obj);
+ return ret;
+ }
+
fence = i915_gem_object_lock_fence(obj);
i915_gem_object_finish_access(obj);
+ i915_gem_object_unlock(obj);
+
if (!fence)
return -ENOMEM;
@@ -730,12 +738,20 @@ i915_gem_shmem_pwrite(struct drm_i915_gem_object *obj,
u64 remain;
int ret;
- ret = i915_gem_object_prepare_write(obj, &needs_clflush);
+ ret = i915_gem_object_lock_interruptible(obj, NULL);
if (ret)
return ret;
+ ret = i915_gem_object_prepare_write(obj, &needs_clflush);
+ if (ret) {
+ i915_gem_object_unlock(obj);
+ return ret;
+ }
+
fence = i915_gem_object_lock_fence(obj);
i915_gem_object_finish_access(obj);
+ i915_gem_object_unlock(obj);
+
if (!fence)
return -ENOMEM;
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 06/19] drm/i915: Parse command buffer earlier in eb_relocate(slow)
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (4 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 05/19] drm/i915: Remove locking from i915_gem_object_prepare_read/write Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 07/19] drm/i915: Use per object locking in execbuf on top of struct_mutex, v3 Maarten Lankhorst
` (14 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
We want to introduce backoff logic, but we need to lock the
pool object as well for command parsing. Because of this, we
will need backoff logic for the engine pool obj, move the batch
validation up slightly to eb_lookup_vmas, and the actual command
parsing in a separate function which can get called from execbuf
relocation fast and slowpath.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 66 +++++++++++--------
1 file changed, 37 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index e7dd29541727..bcbcd0ea8da6 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -287,6 +287,8 @@ struct i915_execbuffer {
struct hlist_head *buckets; /** ht for relocation handles */
};
+static int eb_parse(struct i915_execbuffer *eb);
+
static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
{
return intel_engine_requires_cmd_parser(eb->engine) ||
@@ -706,6 +708,7 @@ static int eb_select_context(struct i915_execbuffer *eb)
static int eb_lookup_vmas(struct i915_execbuffer *eb)
{
struct radix_tree_root *handles_vma = &eb->gem_context->handles_vma;
+ struct drm_i915_private *i915 = eb->i915;
struct drm_i915_gem_object *obj;
unsigned int i, batch;
int err;
@@ -771,6 +774,22 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
mutex_unlock(&eb->gem_context->mutex);
+ if (unlikely(eb->batch->flags & EXEC_OBJECT_WRITE)) {
+ drm_dbg(&i915->drm,
+ "Attempting to use self-modifying batch buffer\n");
+ return -EINVAL;
+ }
+
+ if (range_overflows_t(u64,
+ eb->batch_start_offset, eb->batch_len,
+ eb->batch->vma->size)) {
+ drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
+ return -EINVAL;
+ }
+
+ if (eb->batch_len == 0)
+ eb->batch_len = eb->batch->vma->size - eb->batch_start_offset;
+
return 0;
err_obj:
@@ -1661,7 +1680,7 @@ static int eb_prefault_relocations(const struct i915_execbuffer *eb)
return 0;
}
-static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
+static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
{
struct drm_device *dev = &eb->i915->drm;
bool have_copy = false;
@@ -1735,6 +1754,11 @@ static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
}
}
+ /* as last step, parse the command buffer */
+ err = eb_parse(eb);
+ if (err)
+ goto err;
+
/*
* Leave the user relocations as are, this is the painfully slow path,
* and we want to avoid the complication of dropping the lock whilst
@@ -1767,7 +1791,7 @@ static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
return err;
}
-static int eb_relocate(struct i915_execbuffer *eb)
+static int eb_relocate_parse(struct i915_execbuffer *eb)
{
int err;
@@ -1775,7 +1799,8 @@ static int eb_relocate(struct i915_execbuffer *eb)
if (err)
return err;
- if (eb_validate_vmas(eb))
+ err = eb_validate_vmas(eb);
+ if (err)
goto slow;
/* The objects are in their final locations, apply the relocations. */
@@ -1788,10 +1813,14 @@ static int eb_relocate(struct i915_execbuffer *eb)
}
}
+ err = eb_parse(eb);
+ if (err)
+ goto slow;
+
return 0;
slow:
- return eb_relocate_slow(eb);
+ return eb_relocate_parse_slow(eb);
}
static int eb_move_to_gpu(struct i915_execbuffer *eb)
@@ -2663,7 +2692,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
if (err)
goto err_engine;
- err = eb_relocate(&eb);
+ err = eb_relocate_parse(&eb);
if (err) {
/*
* If the user expects the execobject.offset and
@@ -2676,33 +2705,10 @@ i915_gem_do_execbuffer(struct drm_device *dev,
goto err_vma;
}
- if (unlikely(eb.batch->flags & EXEC_OBJECT_WRITE)) {
- drm_dbg(&i915->drm,
- "Attempting to use self-modifying batch buffer\n");
- err = -EINVAL;
- goto err_vma;
- }
-
- if (range_overflows_t(u64,
- eb.batch_start_offset, eb.batch_len,
- eb.batch->vma->size)) {
- drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
- err = -EINVAL;
- goto err_vma;
- }
-
- if (eb.batch_len == 0)
- eb.batch_len = eb.batch->vma->size - eb.batch_start_offset;
-
- err = eb_parse(&eb);
- if (err)
- goto err_vma;
-
/*
* snb/ivb/vlv conflate the "batch in ppgtt" bit with the "non-secure
* batch" bit. Hence we need to pin secure batches into the global gtt.
* hsw should have this fixed, but bdw mucks it up again. */
- batch = eb.batch->vma;
if (eb.batch_flags & I915_DISPATCH_SECURE) {
struct i915_vma *vma;
@@ -2716,13 +2722,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
* fitting due to fragmentation.
* So this is actually safe.
*/
- vma = i915_gem_object_ggtt_pin(batch->obj, NULL, 0, 0, 0);
+ vma = i915_gem_object_ggtt_pin(eb.batch->vma->obj, NULL, 0, 0, 0);
if (IS_ERR(vma)) {
err = PTR_ERR(vma);
goto err_vma;
}
batch = vma;
+ } else {
+ batch = eb.batch->vma;
}
/* All GPU relocation batches must be submitted prior to the user rq */
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 07/19] drm/i915: Use per object locking in execbuf on top of struct_mutex, v3.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (5 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 06/19] drm/i915: Parse command buffer earlier in eb_relocate(slow) Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 08/19] drm/i915: Use ww locking in intel_renderstate Maarten Lankhorst
` (13 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
Now that we changed execbuf submission slightly to allow us to do all
pinning in one place, we can now simply add ww versions on top of
struct_mutex. All we have to do is a separate path for -EDEADLK
handling, which needs to unpin all gem bo's before dropping the lock,
then starting over.
This finally allows us to do parallel submission, but because vma
pinning does not use the ww ctx yet, we cannot drop struct_mutex yet.
Changes since v1:
- Keep struct_mutex for now. :(
Changes since v2:
- Make sure we always lock the ww context in slowpath.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 142 +++++++++---------
1 file changed, 72 insertions(+), 70 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index bcbcd0ea8da6..a27aa85985bd 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -251,6 +251,8 @@ struct i915_execbuffer {
/** list of vma that have execobj.relocation_count */
struct list_head relocs;
+ struct i915_gem_ww_ctx ww;
+
/**
* Track the most recently used object for relocations, as we
* frequently have to perform multiple relocations within the same
@@ -813,6 +815,10 @@ static int eb_validate_vmas(struct i915_execbuffer *eb)
struct eb_vma *ev = &eb->vma[i];
struct i915_vma *vma = ev->vma;
+ err = i915_gem_object_lock(vma->obj, &eb->ww);
+ if (err)
+ return err;
+
if (eb_pin_vma(eb, entry, ev)) {
if (entry->offset != vma->node.start) {
entry->offset = vma->node.start | UPDATE;
@@ -959,7 +965,6 @@ static void reloc_cache_reset(struct reloc_cache *cache)
kunmap_atomic(vaddr);
i915_gem_object_finish_access(obj);
- i915_gem_object_unlock(obj);
} else {
struct i915_ggtt *ggtt = cache_to_ggtt(cache);
@@ -994,15 +999,9 @@ static void *reloc_kmap(struct drm_i915_gem_object *obj,
unsigned int flushes;
int err;
- err = i915_gem_object_lock_interruptible(obj, NULL);
- if (err)
- return ERR_PTR(err);
-
err = i915_gem_object_prepare_write(obj, &flushes);
- if (err) {
- i915_gem_object_unlock(obj);
+ if (err)
return ERR_PTR(err);
- }
BUILD_BUG_ON(KMAP & CLFLUSH_FLAGS);
BUILD_BUG_ON((KMAP | CLFLUSH_FLAGS) & PAGE_MASK);
@@ -1041,9 +1040,7 @@ static void *reloc_iomap(struct drm_i915_gem_object *obj,
if (use_cpu_reloc(cache, obj))
return NULL;
- i915_gem_object_lock(obj, NULL);
err = i915_gem_object_set_to_gtt_domain(obj, true);
- i915_gem_object_unlock(obj);
if (err)
return ERR_PTR(err);
@@ -1132,7 +1129,7 @@ static int reloc_move_to_gpu(struct i915_request *rq, struct i915_vma *vma)
struct drm_i915_gem_object *obj = vma->obj;
int err;
- i915_vma_lock(vma);
+ assert_vma_held(vma);
if (obj->cache_dirty & ~obj->cache_coherent)
i915_gem_clflush_object(obj, 0);
@@ -1142,8 +1139,6 @@ static int reloc_move_to_gpu(struct i915_request *rq, struct i915_vma *vma)
if (err == 0)
err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
- i915_vma_unlock(vma);
-
return err;
}
@@ -1162,6 +1157,10 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb,
if (IS_ERR(pool))
return PTR_ERR(pool);
+ err = i915_gem_object_lock(pool->obj, &eb->ww);
+ if (err)
+ goto out_pool;
+
cmd = i915_gem_object_pin_map(pool->obj,
cache->has_llc ?
I915_MAP_FORCE_WB :
@@ -1201,11 +1200,10 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb,
if (err)
goto skip_request;
- i915_vma_lock(batch);
+ assert_vma_held(batch);
err = i915_request_await_object(rq, batch->obj, false);
if (err == 0)
err = i915_vma_move_to_active(batch, rq, 0);
- i915_vma_unlock(batch);
if (err)
goto skip_request;
@@ -1286,7 +1284,9 @@ relocate_entry(struct i915_vma *vma,
len = 3;
batch = reloc_gpu(eb, vma, len);
- if (IS_ERR(batch))
+ if (batch == ERR_PTR(-EDEADLK))
+ return (s32)-EDEADLK;
+ else if (IS_ERR(batch))
goto repeat;
addr = gen8_canonical_addr(vma->node.start + offset);
@@ -1695,6 +1695,7 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
/* We may process another execbuffer during the unlock... */
eb_release_vmas(eb);
+ i915_gem_ww_ctx_fini(&eb->ww);
mutex_unlock(&dev->struct_mutex);
/*
@@ -1719,21 +1720,22 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
cond_resched();
err = 0;
}
- if (err) {
- mutex_lock(&dev->struct_mutex);
- goto out;
- }
- /* A frequent cause for EAGAIN are currently unavailable client pages */
- flush_workqueue(eb->i915->mm.userptr_wq);
+ if (!err) {
+ /* A frequent cause for EAGAIN are currently unavailable client pages */
+ flush_workqueue(eb->i915->mm.userptr_wq);
- err = mutex_lock_interruptible(&dev->struct_mutex);
+ err = mutex_lock_interruptible(&dev->struct_mutex);
+ }
if (err) {
mutex_lock(&dev->struct_mutex);
+ i915_gem_ww_ctx_init(&eb->ww, true);
goto out;
}
+ i915_gem_ww_ctx_init(&eb->ww, true);
/* reacquire the objects */
+repeat_validate:
err = eb_validate_vmas(eb);
if (err)
goto err;
@@ -1745,7 +1747,9 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
pagefault_disable();
err = eb_relocate_vma(eb, ev);
pagefault_enable();
- if (err)
+ if (err == -EDEADLK)
+ goto err;
+ else if (err)
goto repeat;
} else {
err = eb_relocate_vma_slow(eb, ev);
@@ -1767,6 +1771,13 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
*/
err:
+ if (err == -EDEADLK) {
+ eb_release_vmas(eb);
+ err = i915_gem_ww_ctx_backoff(&eb->ww);
+ if (!err)
+ goto repeat_validate;
+ }
+
if (err == -EAGAIN)
goto repeat;
@@ -1799,64 +1810,58 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
if (err)
return err;
+retry:
err = eb_validate_vmas(eb);
if (err)
- goto slow;
+ goto err;
/* The objects are in their final locations, apply the relocations. */
if (eb->args->flags & __EXEC_HAS_RELOC) {
struct eb_vma *ev;
list_for_each_entry(ev, &eb->relocs, reloc_link) {
- if (eb_relocate_vma(eb, ev))
- goto slow;
+ err = eb_relocate_vma(eb, ev);
+ if (err)
+ goto err;
}
}
err = eb_parse(eb);
- if (err)
+err:
+ if (err == -EDEADLK) {
+ eb_release_vmas(eb);
+ err = i915_gem_ww_ctx_backoff(&eb->ww);
+ if (err)
+ return err;
+
+ goto retry;
+ }
+ else if (err)
goto slow;
return 0;
slow:
- return eb_relocate_parse_slow(eb);
+ err = eb_relocate_parse_slow(eb);
+ if (err)
+ /*
+ * If the user expects the execobject.offset and
+ * reloc.presumed_offset to be an exact match,
+ * as for using NO_RELOC, then we cannot update
+ * the execobject.offset until we have completed
+ * relocation.
+ */
+ eb->args->flags &= ~__EXEC_HAS_RELOC;
+
+ return err;
}
static int eb_move_to_gpu(struct i915_execbuffer *eb)
{
const unsigned int count = eb->buffer_count;
- struct ww_acquire_ctx acquire;
- unsigned int i;
+ unsigned int i = count;
int err = 0;
- ww_acquire_init(&acquire, &reservation_ww_class);
-
- for (i = 0; i < count; i++) {
- struct eb_vma *ev = &eb->vma[i];
- struct i915_vma *vma = ev->vma;
-
- err = ww_mutex_lock_interruptible(&vma->resv->lock, &acquire);
- if (err == -EDEADLK) {
- GEM_BUG_ON(i == 0);
- do {
- int j = i - 1;
-
- ww_mutex_unlock(&eb->vma[j].vma->resv->lock);
-
- swap(eb->vma[i], eb->vma[j]);
- } while (--i);
-
- err = ww_mutex_lock_slow_interruptible(&vma->resv->lock,
- &acquire);
- }
- if (err == -EALREADY)
- err = 0;
- if (err)
- break;
- }
- ww_acquire_done(&acquire);
-
while (i--) {
struct eb_vma *ev = &eb->vma[i];
struct i915_vma *vma = ev->vma;
@@ -1901,15 +1906,12 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
if (err == 0)
err = i915_vma_move_to_active(vma, eb->request, flags);
- i915_vma_unlock(vma);
-
__eb_unreserve_vma(vma, flags);
if (unlikely(flags & __EXEC_OBJECT_HAS_REF))
i915_vma_put(vma);
ev->vma = NULL;
}
- ww_acquire_fini(&acquire);
if (unlikely(err))
goto err_skip;
@@ -2065,10 +2067,6 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
pw->shadow = shadow;
pw->trampoline = trampoline;
- err = dma_resv_lock_interruptible(pw->batch->resv, NULL);
- if (err)
- goto err_trampoline;
-
err = dma_resv_reserve_shared(pw->batch->resv, 1);
if (err)
goto err_batch_unlock;
@@ -2083,19 +2081,14 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
/* Keep the batch alive and unwritten as we parse */
dma_resv_add_shared_fence(pw->batch->resv, &pw->base.dma);
- dma_resv_unlock(pw->batch->resv);
-
/* Force execution to wait for completion of the parser */
- dma_resv_lock(shadow->resv, NULL);
dma_resv_add_excl_fence(shadow->resv, &pw->base.dma);
- dma_resv_unlock(shadow->resv);
dma_fence_work_commit(&pw->base);
return 0;
err_batch_unlock:
dma_resv_unlock(pw->batch->resv);
-err_trampoline:
if (trampoline)
i915_active_release(&trampoline->active);
err_shadow:
@@ -2137,6 +2130,10 @@ static int eb_parse(struct i915_execbuffer *eb)
if (IS_ERR(pool))
return PTR_ERR(pool);
+ err = i915_gem_object_lock(pool->obj, &eb->ww);
+ if (err)
+ goto err;
+
shadow = shadow_batch_pin(pool->obj, eb->context->vm, PIN_USER);
if (IS_ERR(shadow)) {
err = PTR_ERR(shadow);
@@ -2691,6 +2688,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
err = mutex_lock_interruptible(&dev->struct_mutex);
if (err)
goto err_engine;
+ i915_gem_ww_ctx_init(&eb.ww, true);
err = eb_relocate_parse(&eb);
if (err) {
@@ -2705,6 +2703,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,
goto err_vma;
}
+ ww_acquire_done(&eb.ww.ctx);
+
/*
* snb/ivb/vlv conflate the "batch in ppgtt" bit with the "non-secure
* batch" bit. Hence we need to pin secure batches into the global gtt.
@@ -2813,6 +2813,8 @@ i915_gem_do_execbuffer(struct drm_device *dev,
eb_release_vmas(&eb);
if (eb.trampoline)
i915_vma_unpin(eb.trampoline);
+ WARN_ON(err == -EDEADLK);
+ i915_gem_ww_ctx_fini(&eb.ww);
mutex_unlock(&dev->struct_mutex);
err_engine:
eb_unpin_engine(&eb);
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 08/19] drm/i915: Use ww locking in intel_renderstate.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (6 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 07/19] drm/i915: Use per object locking in execbuf on top of struct_mutex, v3 Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 09/19] drm/i915: Add ww context handling to context_barrier_task Maarten Lankhorst
` (12 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
We want to start using ww locking in intel_context_pin, for this
we need to lock multiple objects, and the single i915_gem_object_lock
is not enough.
Convert to using ww-waiting, and make sure we always pin intel_context_state,
even if we don't have a renderstate object.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gt/intel_gt.c | 21 +++---
drivers/gpu/drm/i915/gt/intel_renderstate.c | 71 ++++++++++++++-------
drivers/gpu/drm/i915/gt/intel_renderstate.h | 9 ++-
3 files changed, 65 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
index fb4a487e0341..bcae951392ce 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt.c
@@ -406,21 +406,20 @@ static int __engines_record_defaults(struct intel_gt *gt)
/* We must be able to switch to something! */
GEM_BUG_ON(!engine->kernel_context);
- err = intel_renderstate_init(&so, engine);
- if (err)
- goto out;
-
ce = intel_context_create(engine);
if (IS_ERR(ce)) {
err = PTR_ERR(ce);
goto out;
}
- rq = intel_context_create_request(ce);
+ err = intel_renderstate_init(&so, ce);
+ if (err)
+ goto err;
+
+ rq = i915_request_create(ce);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
- intel_context_put(ce);
- goto out;
+ goto err_fini;
}
err = intel_engine_emit_ctx_wa(rq);
@@ -434,9 +433,13 @@ static int __engines_record_defaults(struct intel_gt *gt)
err_rq:
requests[id] = i915_request_get(rq);
i915_request_add(rq);
- intel_renderstate_fini(&so);
- if (err)
+err_fini:
+ intel_renderstate_fini(&so, ce);
+err:
+ if (err) {
+ intel_context_put(ce);
goto out;
+ }
}
/* Flush the default context image to memory, and enable powersaving. */
diff --git a/drivers/gpu/drm/i915/gt/intel_renderstate.c b/drivers/gpu/drm/i915/gt/intel_renderstate.c
index fff61e86cfbf..09ea4d0a14f9 100644
--- a/drivers/gpu/drm/i915/gt/intel_renderstate.c
+++ b/drivers/gpu/drm/i915/gt/intel_renderstate.c
@@ -27,6 +27,7 @@
#include "i915_drv.h"
#include "intel_renderstate.h"
+#include "gt/intel_context.h"
#include "intel_ring.h"
static const struct intel_renderstate_rodata *
@@ -74,10 +75,9 @@ static int render_state_setup(struct intel_renderstate *so,
u32 *d;
int ret;
- i915_gem_object_lock(so->vma->obj, NULL);
ret = i915_gem_object_prepare_write(so->vma->obj, &needs_clflush);
if (ret)
- goto out_unlock;
+ return ret;
d = kmap_atomic(i915_gem_object_get_dirty_page(so->vma->obj, 0));
@@ -158,8 +158,6 @@ static int render_state_setup(struct intel_renderstate *so,
ret = 0;
out:
i915_gem_object_finish_access(so->vma->obj);
-out_unlock:
- i915_gem_object_unlock(so->vma->obj);
return ret;
err:
@@ -171,33 +169,47 @@ static int render_state_setup(struct intel_renderstate *so,
#undef OUT_BATCH
int intel_renderstate_init(struct intel_renderstate *so,
- struct intel_engine_cs *engine)
+ struct intel_context *ce)
{
- struct drm_i915_gem_object *obj;
+ struct intel_engine_cs *engine = ce->engine;
+ struct drm_i915_gem_object *obj = NULL;
int err;
memset(so, 0, sizeof(*so));
so->rodata = render_state_get_rodata(engine);
- if (!so->rodata)
- return 0;
+ if (so->rodata) {
+ if (so->rodata->batch_items * 4 > PAGE_SIZE)
+ return -EINVAL;
+
+ obj = i915_gem_object_create_internal(engine->i915, PAGE_SIZE);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ so->vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);
+ if (IS_ERR(so->vma)) {
+ err = PTR_ERR(so->vma);
+ goto err_obj;
+ }
+ }
- if (so->rodata->batch_items * 4 > PAGE_SIZE)
- return -EINVAL;
+ i915_gem_ww_ctx_init(&so->ww, true);
+retry:
+ err = intel_context_pin(ce);
+ if (err)
+ goto err_fini;
- obj = i915_gem_object_create_internal(engine->i915, PAGE_SIZE);
- if (IS_ERR(obj))
- return PTR_ERR(obj);
+ /* return early if there's nothing to setup */
+ if (!err && !so->rodata)
+ return 0;
- so->vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);
- if (IS_ERR(so->vma)) {
- err = PTR_ERR(so->vma);
- goto err_obj;
- }
+ err = i915_gem_object_lock(so->vma->obj, &so->ww);
+ if (err)
+ goto err_context;
err = i915_vma_pin(so->vma, 0, 0, PIN_GLOBAL | PIN_HIGH);
if (err)
- goto err_vma;
+ goto err_context;
err = render_state_setup(so, engine->i915);
if (err)
@@ -207,10 +219,19 @@ int intel_renderstate_init(struct intel_renderstate *so,
err_unpin:
i915_vma_unpin(so->vma);
-err_vma:
+err_context:
+ intel_context_unpin(ce);
+err_fini:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&so->ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&so->ww);
i915_vma_close(so->vma);
err_obj:
- i915_gem_object_put(obj);
+ if (obj)
+ i915_gem_object_put(obj);
so->vma = NULL;
return err;
}
@@ -238,16 +259,18 @@ int intel_renderstate_emit(struct intel_renderstate *so,
return err;
}
- i915_vma_lock(so->vma);
err = i915_request_await_object(rq, so->vma->obj, false);
if (err == 0)
err = i915_vma_move_to_active(so->vma, rq, 0);
- i915_vma_unlock(so->vma);
return err;
}
-void intel_renderstate_fini(struct intel_renderstate *so)
+void intel_renderstate_fini(struct intel_renderstate *so,
+ struct intel_context *ce)
{
i915_vma_unpin_and_release(&so->vma, 0);
+
+ intel_context_unpin(ce);
+ i915_gem_ww_ctx_fini(&so->ww);
}
diff --git a/drivers/gpu/drm/i915/gt/intel_renderstate.h b/drivers/gpu/drm/i915/gt/intel_renderstate.h
index 5700be69a05a..713aa1e86c80 100644
--- a/drivers/gpu/drm/i915/gt/intel_renderstate.h
+++ b/drivers/gpu/drm/i915/gt/intel_renderstate.h
@@ -25,9 +25,10 @@
#define _INTEL_RENDERSTATE_H_
#include <linux/types.h>
+#include "i915_gem.h"
struct i915_request;
-struct intel_engine_cs;
+struct intel_context;
struct i915_vma;
struct intel_renderstate_rodata {
@@ -49,6 +50,7 @@ extern const struct intel_renderstate_rodata gen8_null_state;
extern const struct intel_renderstate_rodata gen9_null_state;
struct intel_renderstate {
+ struct i915_gem_ww_ctx ww;
const struct intel_renderstate_rodata *rodata;
struct i915_vma *vma;
u32 batch_offset;
@@ -58,9 +60,10 @@ struct intel_renderstate {
};
int intel_renderstate_init(struct intel_renderstate *so,
- struct intel_engine_cs *engine);
+ struct intel_context *ce);
int intel_renderstate_emit(struct intel_renderstate *so,
struct i915_request *rq);
-void intel_renderstate_fini(struct intel_renderstate *so);
+void intel_renderstate_fini(struct intel_renderstate *so,
+ struct intel_context *ce);
#endif /* _INTEL_RENDERSTATE_H_ */
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 09/19] drm/i915: Add ww context handling to context_barrier_task
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (7 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 08/19] drm/i915: Use ww locking in intel_renderstate Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 10/19] drm/i915: Nuke arguments to eb_pin_engine Maarten Lankhorst
` (11 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
This is required if we want to pass a ww context in intel_context_pin
and gen6_ppgtt_pin().
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_context.c | 55 ++++++++++++++-----
.../drm/i915/gem/selftests/i915_gem_context.c | 22 +++-----
2 files changed, 48 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index e946b395ddf6..2152ac209450 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -975,12 +975,14 @@ I915_SELFTEST_DECLARE(static intel_engine_mask_t context_barrier_inject_fault);
static int context_barrier_task(struct i915_gem_context *ctx,
intel_engine_mask_t engines,
bool (*skip)(struct intel_context *ce, void *data),
+ int (*pin)(struct intel_context *ce, struct i915_gem_ww_ctx *ww, void *data),
int (*emit)(struct i915_request *rq, void *data),
void (*task)(void *data),
void *data)
{
struct context_barrier_task *cb;
struct i915_gem_engines_iter it;
+ struct i915_gem_ww_ctx ww;
struct intel_context *ce;
int err = 0;
@@ -1012,10 +1014,21 @@ static int context_barrier_task(struct i915_gem_context *ctx,
if (skip && skip(ce, data))
continue;
- rq = intel_context_create_request(ce);
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ err = intel_context_pin(ce);
+ if (err)
+ goto err;
+
+ if (pin)
+ err = pin(ce, &ww, data);
+ if (err)
+ goto err_unpin;
+
+ rq = i915_request_create(ce);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
- break;
+ goto err_unpin;
}
err = 0;
@@ -1025,6 +1038,16 @@ static int context_barrier_task(struct i915_gem_context *ctx,
err = i915_active_add_request(&cb->base, rq);
i915_request_add(rq);
+err_unpin:
+ intel_context_unpin(ce);
+err:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
+
if (err)
break;
}
@@ -1080,6 +1103,17 @@ static void set_ppgtt_barrier(void *data)
i915_vm_close(old);
}
+static int pin_ppgtt_update(struct intel_context *ce, struct i915_gem_ww_ctx *ww, void *data)
+{
+ struct i915_address_space *vm = ce->vm;
+
+ if (!HAS_LOGICAL_RING_CONTEXTS(vm->i915))
+ /* ppGTT is not part of the legacy context image */
+ return gen6_ppgtt_pin(i915_vm_to_ppgtt(vm));
+
+ return 0;
+}
+
static int emit_ppgtt_update(struct i915_request *rq, void *data)
{
struct i915_address_space *vm = rq->context->vm;
@@ -1136,20 +1170,10 @@ static int emit_ppgtt_update(struct i915_request *rq, void *data)
static bool skip_ppgtt_update(struct intel_context *ce, void *data)
{
- if (!test_bit(CONTEXT_ALLOC_BIT, &ce->flags))
- return true;
-
if (HAS_LOGICAL_RING_CONTEXTS(ce->engine->i915))
- return false;
-
- if (!atomic_read(&ce->pin_count))
- return true;
-
- /* ppGTT is not part of the legacy context image */
- if (gen6_ppgtt_pin(i915_vm_to_ppgtt(ce->vm)))
- return true;
-
- return false;
+ return !ce->state;
+ else
+ return !atomic_read(&ce->pin_count);
}
static int set_ppgtt(struct drm_i915_file_private *file_priv,
@@ -1200,6 +1224,7 @@ static int set_ppgtt(struct drm_i915_file_private *file_priv,
*/
err = context_barrier_task(ctx, ALL_ENGINES,
skip_ppgtt_update,
+ pin_ppgtt_update,
emit_ppgtt_update,
set_ppgtt_barrier,
old);
diff --git a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
index d3d8981b268f..50b465b62d88 100644
--- a/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c
@@ -1904,8 +1904,8 @@ static int mock_context_barrier(void *arg)
return -ENOMEM;
counter = 0;
- err = context_barrier_task(ctx, 0,
- NULL, NULL, mock_barrier_task, &counter);
+ err = context_barrier_task(ctx, 0, NULL, NULL, NULL,
+ mock_barrier_task, &counter);
if (err) {
pr_err("Failed at line %d, err=%d\n", __LINE__, err);
goto out;
@@ -1917,11 +1917,8 @@ static int mock_context_barrier(void *arg)
}
counter = 0;
- err = context_barrier_task(ctx, ALL_ENGINES,
- skip_unused_engines,
- NULL,
- mock_barrier_task,
- &counter);
+ err = context_barrier_task(ctx, ALL_ENGINES, skip_unused_engines,
+ NULL, NULL, mock_barrier_task, &counter);
if (err) {
pr_err("Failed at line %d, err=%d\n", __LINE__, err);
goto out;
@@ -1941,8 +1938,8 @@ static int mock_context_barrier(void *arg)
counter = 0;
context_barrier_inject_fault = BIT(RCS0);
- err = context_barrier_task(ctx, ALL_ENGINES,
- NULL, NULL, mock_barrier_task, &counter);
+ err = context_barrier_task(ctx, ALL_ENGINES, NULL, NULL, NULL,
+ mock_barrier_task, &counter);
context_barrier_inject_fault = 0;
if (err == -ENXIO)
err = 0;
@@ -1956,11 +1953,8 @@ static int mock_context_barrier(void *arg)
goto out;
counter = 0;
- err = context_barrier_task(ctx, ALL_ENGINES,
- skip_unused_engines,
- NULL,
- mock_barrier_task,
- &counter);
+ err = context_barrier_task(ctx, ALL_ENGINES, skip_unused_engines,
+ NULL, NULL, mock_barrier_task, &counter);
if (err) {
pr_err("Failed at line %d, err=%d\n", __LINE__, err);
goto out;
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 10/19] drm/i915: Nuke arguments to eb_pin_engine
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (8 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 09/19] drm/i915: Add ww context handling to context_barrier_task Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 20:07 ` Ruhl, Michael J
2020-02-14 10:30 ` [Intel-gfx] [PATCH 11/19] drm/i915: Pin engine before pinning all objects, v2 Maarten Lankhorst
` (10 subsequent siblings)
20 siblings, 1 reply; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
Those arguments are already set as eb.file and eb.args, so kill off
the extra arguments. This will allow us to move eb_pin_engine() to
after we reserved all BO's.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 17 +++++++----------
1 file changed, 7 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index a27aa85985bd..d96e7649314c 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -2393,11 +2393,10 @@ static void eb_unpin_engine(struct i915_execbuffer *eb)
}
static unsigned int
-eb_select_legacy_ring(struct i915_execbuffer *eb,
- struct drm_file *file,
- struct drm_i915_gem_execbuffer2 *args)
+eb_select_legacy_ring(struct i915_execbuffer *eb)
{
struct drm_i915_private *i915 = eb->i915;
+ struct drm_i915_gem_execbuffer2 *args = eb->args;
unsigned int user_ring_id = args->flags & I915_EXEC_RING_MASK;
if (user_ring_id != I915_EXEC_BSD &&
@@ -2412,7 +2411,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb,
unsigned int bsd_idx = args->flags & I915_EXEC_BSD_MASK;
if (bsd_idx == I915_EXEC_BSD_DEFAULT) {
- bsd_idx = gen8_dispatch_bsd_engine(i915, file);
+ bsd_idx = gen8_dispatch_bsd_engine(i915, eb->file);
} else if (bsd_idx >= I915_EXEC_BSD_RING1 &&
bsd_idx <= I915_EXEC_BSD_RING2) {
bsd_idx >>= I915_EXEC_BSD_SHIFT;
@@ -2437,18 +2436,16 @@ eb_select_legacy_ring(struct i915_execbuffer *eb,
}
static int
-eb_pin_engine(struct i915_execbuffer *eb,
- struct drm_file *file,
- struct drm_i915_gem_execbuffer2 *args)
+eb_pin_engine(struct i915_execbuffer *eb)
{
struct intel_context *ce;
unsigned int idx;
int err;
if (i915_gem_context_user_engines(eb->gem_context))
- idx = args->flags & I915_EXEC_RING_MASK;
+ idx = eb->args->flags & I915_EXEC_RING_MASK;
else
- idx = eb_select_legacy_ring(eb, file, args);
+ idx = eb_select_legacy_ring(eb);
ce = i915_gem_context_get_engine(eb->gem_context, idx);
if (IS_ERR(ce))
@@ -2681,7 +2678,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
if (unlikely(err))
goto err_destroy;
- err = eb_pin_engine(&eb, file, args);
+ err = eb_pin_engine(&eb);
if (unlikely(err))
goto err_context;
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [Intel-gfx] [PATCH 10/19] drm/i915: Nuke arguments to eb_pin_engine
2020-02-14 10:30 ` [Intel-gfx] [PATCH 10/19] drm/i915: Nuke arguments to eb_pin_engine Maarten Lankhorst
@ 2020-02-14 20:07 ` Ruhl, Michael J
0 siblings, 0 replies; 24+ messages in thread
From: Ruhl, Michael J @ 2020-02-14 20:07 UTC (permalink / raw)
To: Maarten Lankhorst, intel-gfx
>-----Original Message-----
>From: Intel-gfx <intel-gfx-bounces@lists.freedesktop.org> On Behalf Of
>Maarten Lankhorst
>Sent: Friday, February 14, 2020 5:31 AM
>To: intel-gfx@lists.freedesktop.org
>Subject: [Intel-gfx] [PATCH 10/19] drm/i915: Nuke arguments to
>eb_pin_engine
>
>Those arguments are already set as eb.file and eb.args, so kill off
>the extra arguments. This will allow us to move eb_pin_engine() to
>after we reserved all BO's.
"move eb_pin_engine() to"
I think you mean "too"? (as in "move eb_pin_engine also"
Or do you mean "move_eb_pin_engine to <somewhere else>, after we"?
Other than that,
Acked-by: Michael J. Ruhl <michael.j.ruhl@intel.com>
Mike
>
>Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
>---
> drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 17 +++++++----------
> 1 file changed, 7 insertions(+), 10 deletions(-)
>
>diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
>b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
>index a27aa85985bd..d96e7649314c 100644
>--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
>+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
>@@ -2393,11 +2393,10 @@ static void eb_unpin_engine(struct
>i915_execbuffer *eb)
> }
>
> static unsigned int
>-eb_select_legacy_ring(struct i915_execbuffer *eb,
>- struct drm_file *file,
>- struct drm_i915_gem_execbuffer2 *args)
>+eb_select_legacy_ring(struct i915_execbuffer *eb)
> {
> struct drm_i915_private *i915 = eb->i915;
>+ struct drm_i915_gem_execbuffer2 *args = eb->args;
> unsigned int user_ring_id = args->flags & I915_EXEC_RING_MASK;
>
> if (user_ring_id != I915_EXEC_BSD &&
>@@ -2412,7 +2411,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb,
> unsigned int bsd_idx = args->flags & I915_EXEC_BSD_MASK;
>
> if (bsd_idx == I915_EXEC_BSD_DEFAULT) {
>- bsd_idx = gen8_dispatch_bsd_engine(i915, file);
>+ bsd_idx = gen8_dispatch_bsd_engine(i915, eb->file);
> } else if (bsd_idx >= I915_EXEC_BSD_RING1 &&
> bsd_idx <= I915_EXEC_BSD_RING2) {
> bsd_idx >>= I915_EXEC_BSD_SHIFT;
>@@ -2437,18 +2436,16 @@ eb_select_legacy_ring(struct i915_execbuffer
>*eb,
> }
>
> static int
>-eb_pin_engine(struct i915_execbuffer *eb,
>- struct drm_file *file,
>- struct drm_i915_gem_execbuffer2 *args)
>+eb_pin_engine(struct i915_execbuffer *eb)
> {
> struct intel_context *ce;
> unsigned int idx;
> int err;
>
> if (i915_gem_context_user_engines(eb->gem_context))
>- idx = args->flags & I915_EXEC_RING_MASK;
>+ idx = eb->args->flags & I915_EXEC_RING_MASK;
> else
>- idx = eb_select_legacy_ring(eb, file, args);
>+ idx = eb_select_legacy_ring(eb);
>
> ce = i915_gem_context_get_engine(eb->gem_context, idx);
> if (IS_ERR(ce))
>@@ -2681,7 +2678,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
> if (unlikely(err))
> goto err_destroy;
>
>- err = eb_pin_engine(&eb, file, args);
>+ err = eb_pin_engine(&eb);
> if (unlikely(err))
> goto err_context;
>
>--
>2.25.0.24.g3f081b084b0
>
>_______________________________________________
>Intel-gfx mailing list
>Intel-gfx@lists.freedesktop.org
>https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Intel-gfx] [PATCH 11/19] drm/i915: Pin engine before pinning all objects, v2.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (9 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 10/19] drm/i915: Nuke arguments to eb_pin_engine Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 12/19] drm/i915: Rework intel_context pinning to do everything outside of pin_mutex Maarten Lankhorst
` (9 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
We want to lock all gem objects, including the engine context objects,
rework the throttling to ensure that we can do this. Now we only throttle
once, but can take eb_pin_engine while acquiring objects. This means we
will have to drop the lock to wait. If we don't have to throttle we can
still take the fastpath, if not we will take the slowpath and wait for
the throttle request while unlocked.
The engine has to be pinned as first step, otherwise gpu relocations
won't work.
Changes since v1:
- Only need to get a throttled request in the fastpath, no need for
a global flag any more.
- Always free the waited request correctly.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 154 ++++++++++++------
1 file changed, 103 insertions(+), 51 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index d96e7649314c..72bad053c6b5 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -57,7 +57,8 @@ enum {
#define __EXEC_OBJECT_RESERVED (__EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_FENCE)
#define __EXEC_HAS_RELOC BIT(31)
-#define __EXEC_INTERNAL_FLAGS (~0u << 31)
+#define __EXEC_ENGINE_PINNED BIT(30)
+#define __EXEC_INTERNAL_FLAGS (~0u << 30)
#define UPDATE PIN_OFFSET_FIXED
#define BATCH_OFFSET_BIAS (256*1024)
@@ -290,6 +291,9 @@ struct i915_execbuffer {
};
static int eb_parse(struct i915_execbuffer *eb);
+static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb,
+ bool throttle);
+static void eb_unpin_engine(struct i915_execbuffer *eb);
static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
{
@@ -862,7 +866,7 @@ eb_get_vma(const struct i915_execbuffer *eb, unsigned long handle)
}
}
-static void eb_release_vmas(const struct i915_execbuffer *eb)
+static void eb_release_vmas(struct i915_execbuffer *eb)
{
const unsigned int count = eb->buffer_count;
unsigned int i;
@@ -878,8 +882,9 @@ static void eb_release_vmas(const struct i915_execbuffer *eb)
if (ev->flags & __EXEC_OBJECT_HAS_REF)
i915_vma_put(vma);
-
}
+
+ eb_unpin_engine(eb);
}
static void eb_destroy(const struct i915_execbuffer *eb)
@@ -1680,7 +1685,8 @@ static int eb_prefault_relocations(const struct i915_execbuffer *eb)
return 0;
}
-static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
+static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
+ struct i915_request *rq)
{
struct drm_device *dev = &eb->i915->drm;
bool have_copy = false;
@@ -1698,6 +1704,20 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
i915_gem_ww_ctx_fini(&eb->ww);
mutex_unlock(&dev->struct_mutex);
+ if (rq) {
+ if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
+ MAX_SCHEDULE_TIMEOUT) < 0) {
+ i915_request_put(rq);
+ rq = NULL;
+
+ err = -EINTR;
+ goto err_relock;
+ }
+
+ i915_request_put(rq);
+ rq = NULL;
+ }
+
/*
* We take 3 passes through the slowpatch.
*
@@ -1727,6 +1747,8 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
err = mutex_lock_interruptible(&dev->struct_mutex);
}
+
+err_relock:
if (err) {
mutex_lock(&dev->struct_mutex);
i915_gem_ww_ctx_init(&eb->ww, true);
@@ -1736,6 +1758,15 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
/* reacquire the objects */
repeat_validate:
+ rq = eb_pin_engine(eb, false);
+ if (IS_ERR(rq)) {
+ err = PTR_ERR(rq);
+ goto err;
+ }
+
+ /* We didn't throttle, should be NULL */
+ GEM_WARN_ON(rq);
+
err = eb_validate_vmas(eb);
if (err)
goto err;
@@ -1799,18 +1830,41 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
}
}
+ if (rq)
+ i915_request_put(rq);
+
return err;
}
static int eb_relocate_parse(struct i915_execbuffer *eb)
{
int err;
+ struct i915_request *rq = NULL;
+ bool throttle = true;
err = eb_lookup_vmas(eb);
if (err)
return err;
retry:
+ rq = eb_pin_engine(eb, throttle);
+ if (IS_ERR(rq)) {
+ err = PTR_ERR(rq);
+ rq = NULL;
+ if (err != -EDEADLK)
+ return err;
+
+ goto err;
+ }
+
+ if (rq) {
+ /* Need to drop all locks now for throttling, take slowpath */
+ goto slow;
+ }
+
+ /* only throttle once, even if we didn't need to throttle */
+ throttle = false;
+
err = eb_validate_vmas(eb);
if (err)
goto err;
@@ -1835,14 +1889,14 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
return err;
goto retry;
- }
- else if (err)
+ } else if (err) {
goto slow;
+ }
return 0;
slow:
- err = eb_relocate_parse_slow(eb);
+ err = eb_relocate_parse_slow(eb, rq);
if (err)
/*
* If the user expects the execobject.offset and
@@ -2276,7 +2330,7 @@ static const enum intel_engine_id user_ring_map[] = {
[I915_EXEC_VEBOX] = VECS0
};
-static struct i915_request *eb_throttle(struct intel_context *ce)
+static struct i915_request *eb_throttle(struct i915_execbuffer *eb, struct intel_context *ce)
{
struct intel_ring *ring = ce->ring;
struct intel_timeline *tl = ce->timeline;
@@ -2310,22 +2364,17 @@ static struct i915_request *eb_throttle(struct intel_context *ce)
return i915_request_get(rq);
}
-static int __eb_pin_engine(struct i915_execbuffer *eb, struct intel_context *ce)
+static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
{
+ struct intel_context *ce = eb->context;
struct intel_timeline *tl;
- struct i915_request *rq;
+ struct i915_request *rq = NULL;
int err;
- /*
- * ABI: Before userspace accesses the GPU (e.g. execbuffer), report
- * EIO if the GPU is already wedged.
- */
- err = intel_gt_terminally_wedged(ce->engine->gt);
- if (err)
- return err;
+ GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
if (unlikely(intel_context_is_banned(ce)))
- return -EIO;
+ return ERR_PTR(-EIO);
/*
* Pinning the contexts may generate requests in order to acquire
@@ -2334,7 +2383,7 @@ static int __eb_pin_engine(struct i915_execbuffer *eb, struct intel_context *ce)
*/
err = intel_context_pin(ce);
if (err)
- return err;
+ return ERR_PTR(err);
/*
* Take a local wakeref for preparing to dispatch the execbuf as
@@ -2346,38 +2395,17 @@ static int __eb_pin_engine(struct i915_execbuffer *eb, struct intel_context *ce)
*/
tl = intel_context_timeline_lock(ce);
if (IS_ERR(tl)) {
- err = PTR_ERR(tl);
- goto err_unpin;
+ intel_context_unpin(ce);
+ return ERR_CAST(tl);
}
intel_context_enter(ce);
- rq = eb_throttle(ce);
-
+ if (throttle)
+ rq = eb_throttle(eb, ce);
intel_context_timeline_unlock(tl);
- if (rq) {
- if (i915_request_wait(rq,
- I915_WAIT_INTERRUPTIBLE,
- MAX_SCHEDULE_TIMEOUT) < 0) {
- i915_request_put(rq);
- err = -EINTR;
- goto err_exit;
- }
-
- i915_request_put(rq);
- }
-
- eb->engine = ce->engine;
- eb->context = ce;
- return 0;
-
-err_exit:
- mutex_lock(&tl->mutex);
- intel_context_exit(ce);
- intel_context_timeline_unlock(tl);
-err_unpin:
- intel_context_unpin(ce);
- return err;
+ eb->args->flags |= __EXEC_ENGINE_PINNED;
+ return rq;
}
static void eb_unpin_engine(struct i915_execbuffer *eb)
@@ -2385,6 +2413,11 @@ static void eb_unpin_engine(struct i915_execbuffer *eb)
struct intel_context *ce = eb->context;
struct intel_timeline *tl = ce->timeline;
+ if (!(eb->args->flags & __EXEC_ENGINE_PINNED))
+ return;
+
+ eb->args->flags &= ~__EXEC_ENGINE_PINNED;
+
mutex_lock(&tl->mutex);
intel_context_exit(ce);
mutex_unlock(&tl->mutex);
@@ -2436,7 +2469,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb)
}
static int
-eb_pin_engine(struct i915_execbuffer *eb)
+eb_select_engine(struct i915_execbuffer *eb)
{
struct intel_context *ce;
unsigned int idx;
@@ -2451,12 +2484,28 @@ eb_pin_engine(struct i915_execbuffer *eb)
if (IS_ERR(ce))
return PTR_ERR(ce);
- err = __eb_pin_engine(eb, ce);
- intel_context_put(ce);
+ /*
+ * ABI: Before userspace accesses the GPU (e.g. execbuffer), report
+ * EIO if the GPU is already wedged.
+ */
+ err = intel_gt_terminally_wedged(ce->engine->gt);
+ if (err) {
+ intel_context_put(ce);
+ return err;
+ }
+
+ eb->context = ce;
+ eb->engine = ce->engine;
return err;
}
+static void
+eb_put_engine(struct i915_execbuffer *eb)
+{
+ intel_context_put(eb->context);
+}
+
static void
__free_fence_array(struct drm_syncobj **fences, unsigned int n)
{
@@ -2678,7 +2727,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
if (unlikely(err))
goto err_destroy;
- err = eb_pin_engine(&eb);
+ err = eb_select_engine(&eb);
if (unlikely(err))
goto err_context;
@@ -2808,13 +2857,16 @@ i915_gem_do_execbuffer(struct drm_device *dev,
err_vma:
if (eb.exec)
eb_release_vmas(&eb);
+ else
+ eb_unpin_engine(&eb);
+
if (eb.trampoline)
i915_vma_unpin(eb.trampoline);
WARN_ON(err == -EDEADLK);
i915_gem_ww_ctx_fini(&eb.ww);
mutex_unlock(&dev->struct_mutex);
err_engine:
- eb_unpin_engine(&eb);
+ eb_put_engine(&eb);
err_context:
i915_gem_context_put(eb.gem_context);
err_destroy:
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 12/19] drm/i915: Rework intel_context pinning to do everything outside of pin_mutex
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (10 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 11/19] drm/i915: Pin engine before pinning all objects, v2 Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 13/19] drm/i915: Make sure execbuffer always passes ww state to i915_vma_pin Maarten Lankhorst
` (8 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
Instead of doing everything inside of pin_mutex, we move all pinning
outside. Because i915_active has its own reference counting and
pinning is also having the same issues vs mutexes, we make sure
everything is pinned first, so the pinning in i915_active only needs
to bump refcounts. This allows us to take pin refcounts correctly
all the time.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 221 +++++++++++-------
drivers/gpu/drm/i915/gt/intel_context_types.h | 4 +-
drivers/gpu/drm/i915/gt/intel_lrc.c | 34 ++-
drivers/gpu/drm/i915/gt/intel_renderstate.c | 1 -
.../gpu/drm/i915/gt/intel_ring_submission.c | 13 +-
drivers/gpu/drm/i915/gt/mock_engine.c | 13 +-
6 files changed, 185 insertions(+), 101 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index e4f89341d17c..44dafa5177c3 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -88,74 +88,6 @@ static void intel_context_active_release(struct intel_context *ce)
i915_active_release(&ce->active);
}
-int __intel_context_do_pin(struct intel_context *ce)
-{
- int err;
-
- if (unlikely(!test_bit(CONTEXT_ALLOC_BIT, &ce->flags))) {
- err = intel_context_alloc_state(ce);
- if (err)
- return err;
- }
-
- err = i915_active_acquire(&ce->active);
- if (err)
- return err;
-
- if (mutex_lock_interruptible(&ce->pin_mutex)) {
- err = -EINTR;
- goto out_release;
- }
-
- if (likely(!atomic_add_unless(&ce->pin_count, 1, 0))) {
- err = intel_context_active_acquire(ce);
- if (unlikely(err))
- goto out_unlock;
-
- err = ce->ops->pin(ce);
- if (unlikely(err))
- goto err_active;
-
- CE_TRACE(ce, "pin ring:{start:%08x, head:%04x, tail:%04x}\n",
- i915_ggtt_offset(ce->ring->vma),
- ce->ring->head, ce->ring->tail);
-
- smp_mb__before_atomic(); /* flush pin before it is visible */
- atomic_inc(&ce->pin_count);
- }
-
- GEM_BUG_ON(!intel_context_is_pinned(ce)); /* no overflow! */
- GEM_BUG_ON(i915_active_is_idle(&ce->active));
- goto out_unlock;
-
-err_active:
- intel_context_active_release(ce);
-out_unlock:
- mutex_unlock(&ce->pin_mutex);
-out_release:
- i915_active_release(&ce->active);
- return err;
-}
-
-void intel_context_unpin(struct intel_context *ce)
-{
- if (!atomic_dec_and_test(&ce->pin_count))
- return;
-
- CE_TRACE(ce, "unpin\n");
- ce->ops->unpin(ce);
-
- /*
- * Once released, we may asynchronously drop the active reference.
- * As that may be the only reference keeping the context alive,
- * take an extra now so that it is not freed before we finish
- * dereferencing it.
- */
- intel_context_get(ce);
- intel_context_active_release(ce);
- intel_context_put(ce);
-}
-
static int __context_pin_state(struct i915_vma *vma)
{
unsigned int bias = i915_ggtt_pin_bias(vma) | PIN_OFFSET_BIAS;
@@ -215,51 +147,170 @@ static void __ring_retire(struct intel_ring *ring)
i915_active_release(&ring->vma->active);
}
-__i915_active_call
-static void __intel_context_retire(struct i915_active *active)
+static int intel_context_pre_pin(struct intel_context *ce)
{
- struct intel_context *ce = container_of(active, typeof(*ce), active);
+ int err;
- CE_TRACE(ce, "retire\n");
+ CE_TRACE(ce, "active\n");
- set_bit(CONTEXT_VALID_BIT, &ce->flags);
+ err = __ring_active(ce->ring);
+ if (err)
+ return err;
+
+ err = intel_timeline_pin(ce->timeline);
+ if (err)
+ goto err_ring;
+
+ if (!ce->state)
+ return 0;
+
+ err = __context_pin_state(ce->state);
+ if (err)
+ goto err_timeline;
+
+
+ return 0;
+
+err_timeline:
+ intel_timeline_unpin(ce->timeline);
+err_ring:
+ __ring_retire(ce->ring);
+ return err;
+}
+
+static void intel_context_post_unpin(struct intel_context *ce)
+{
if (ce->state)
__context_unpin_state(ce->state);
intel_timeline_unpin(ce->timeline);
__ring_retire(ce->ring);
+}
+
+int __intel_context_do_pin(struct intel_context *ce)
+{
+ bool handoff = false;
+ void *vaddr;
+ int err = 0;
+
+ if (unlikely(!test_bit(CONTEXT_ALLOC_BIT, &ce->flags))) {
+ err = intel_context_alloc_state(ce);
+ if (err)
+ return err;
+ }
+
+ /*
+ * We always pin the context/ring/timeline here, to ensure a pin
+ * refcount for __intel_context_active(), which prevent a lock
+ * inversion of ce->pin_mutex vs dma_resv_lock().
+ */
+ err = intel_context_pre_pin(ce);
+ if (err)
+ return err;
+
+ err = i915_active_acquire(&ce->active);
+ if (err)
+ goto err_ctx_unpin;
+
+ err = ce->ops->pre_pin(ce, &vaddr);
+ if (err)
+ goto err_release;
+
+ err = mutex_lock_interruptible(&ce->pin_mutex);
+ if (err)
+ goto err_post_unpin;
+
+ if (likely(!atomic_add_unless(&ce->pin_count, 1, 0))) {
+ err = intel_context_active_acquire(ce);
+ if (unlikely(err))
+ goto err_unlock;
+
+ err = ce->ops->pin(ce, vaddr);
+ if (err) {
+ intel_context_active_release(ce);
+ goto err_unlock;
+ }
+
+ CE_TRACE(ce, "pin ring:{start:%08x, head:%04x, tail:%04x}\n",
+ i915_ggtt_offset(ce->ring->vma),
+ ce->ring->head, ce->ring->tail);
+
+ handoff = true;
+ smp_mb__before_atomic(); /* flush pin before it is visible */
+ atomic_inc(&ce->pin_count);
+ }
+
+ GEM_BUG_ON(!intel_context_is_pinned(ce)); /* no overflow! */
+
+err_unlock:
+ mutex_unlock(&ce->pin_mutex);
+err_post_unpin:
+ if (!handoff)
+ ce->ops->post_unpin(ce);
+err_release:
+ i915_active_release(&ce->active);
+err_ctx_unpin:
+ intel_context_post_unpin(ce);
+ return err;
+}
+
+void intel_context_unpin(struct intel_context *ce)
+{
+ if (!atomic_dec_and_test(&ce->pin_count))
+ return;
+
+ CE_TRACE(ce, "unpin\n");
+ ce->ops->unpin(ce);
+ ce->ops->post_unpin(ce);
+
+ /*
+ * Once released, we may asynchronously drop the active reference.
+ * As that may be the only reference keeping the context alive,
+ * take an extra now so that it is not freed before we finish
+ * dereferencing it.
+ */
+ intel_context_get(ce);
+ intel_context_active_release(ce);
+ intel_context_put(ce);
+}
+__i915_active_call
+static void __intel_context_retire(struct i915_active *active)
+{
+ struct intel_context *ce = container_of(active, typeof(*ce), active);
+
+ CE_TRACE(ce, "retire\n");
+
+ set_bit(CONTEXT_VALID_BIT, &ce->flags);
+ intel_context_post_unpin(ce);
intel_context_put(ce);
}
+__i915_active_call
static int __intel_context_active(struct i915_active *active)
{
struct intel_context *ce = container_of(active, typeof(*ce), active);
int err;
- CE_TRACE(ce, "active\n");
-
intel_context_get(ce);
+ /* everything should already be activated by intel_context_pre_pin() */
err = __ring_active(ce->ring);
- if (err)
+ if (GEM_WARN_ON(err))
goto err_put;
err = intel_timeline_pin(ce->timeline);
- if (err)
+ if (GEM_WARN_ON(err))
goto err_ring;
- if (!ce->state)
- return 0;
-
- err = __context_pin_state(ce->state);
- if (err)
- goto err_timeline;
+ if (ce->state) {
+ GEM_WARN_ON(!i915_active_acquire_if_busy(&ce->state->active));
+ __i915_vma_pin(ce->state);
+ i915_vma_make_unshrinkable(ce->state);
+ }
return 0;
-err_timeline:
- intel_timeline_unpin(ce->timeline);
err_ring:
__ring_retire(ce->ring);
err_put:
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index ca1420fb8b53..65457354eb39 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -27,8 +27,10 @@ struct intel_ring;
struct intel_context_ops {
int (*alloc)(struct intel_context *ce);
- int (*pin)(struct intel_context *ce);
+ int (*pre_pin)(struct intel_context *ce, void **vaddr);
+ int (*pin)(struct intel_context *ce, void *vaddr);
void (*unpin)(struct intel_context *ce);
+ void (*post_unpin)(struct intel_context *ce);
void (*enter)(struct intel_context *ce);
void (*exit)(struct intel_context *ce);
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index c3d7727021db..d9ae07411e62 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -2901,7 +2901,10 @@ static void execlists_context_unpin(struct intel_context *ce)
{
check_redzone((void *)ce->lrc_reg_state - LRC_STATE_PN * PAGE_SIZE,
ce->engine);
+}
+static void execlists_context_post_unpin(struct intel_context *ce)
+{
i915_gem_object_unpin_map(ce->state->obj);
}
@@ -2930,20 +2933,23 @@ __execlists_update_reg_state(const struct intel_context *ce,
}
static int
-__execlists_context_pin(struct intel_context *ce,
- struct intel_engine_cs *engine)
+execlists_context_pre_pin(struct intel_context *ce, void **vaddr)
{
- void *vaddr;
-
GEM_BUG_ON(!ce->state);
GEM_BUG_ON(!i915_vma_is_pinned(ce->state));
- vaddr = i915_gem_object_pin_map(ce->state->obj,
- i915_coherent_map_type(engine->i915) |
+ *vaddr = i915_gem_object_pin_map(ce->state->obj,
+ i915_coherent_map_type(ce->engine->i915) |
I915_MAP_OVERRIDE);
- if (IS_ERR(vaddr))
- return PTR_ERR(vaddr);
+ return PTR_ERR_OR_ZERO(*vaddr);
+}
+
+static int
+__execlists_context_pin(struct intel_context *ce,
+ struct intel_engine_cs *engine,
+ void *vaddr)
+{
ce->lrc_desc = lrc_descriptor(ce, engine) | CTX_DESC_FORCE_RESTORE;
ce->lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
__execlists_update_reg_state(ce, engine, ce->ring->tail);
@@ -2951,9 +2957,9 @@ __execlists_context_pin(struct intel_context *ce,
return 0;
}
-static int execlists_context_pin(struct intel_context *ce)
+static int execlists_context_pin(struct intel_context *ce, void *vaddr)
{
- return __execlists_context_pin(ce, ce->engine);
+ return __execlists_context_pin(ce, ce->engine, vaddr);
}
static int execlists_context_alloc(struct intel_context *ce)
@@ -2979,8 +2985,10 @@ static void execlists_context_reset(struct intel_context *ce)
static const struct intel_context_ops execlists_context_ops = {
.alloc = execlists_context_alloc,
+ .pre_pin = execlists_context_pre_pin,
.pin = execlists_context_pin,
.unpin = execlists_context_unpin,
+ .post_unpin = execlists_context_post_unpin,
.enter = intel_context_enter_engine,
.exit = intel_context_exit_engine,
@@ -4764,13 +4772,13 @@ static int virtual_context_alloc(struct intel_context *ce)
return __execlists_context_alloc(ce, ve->siblings[0]);
}
-static int virtual_context_pin(struct intel_context *ce)
+static int virtual_context_pin(struct intel_context *ce, void *vaddr)
{
struct virtual_engine *ve = container_of(ce, typeof(*ve), context);
int err;
/* Note: we must use a real engine class for setting up reg state */
- err = __execlists_context_pin(ce, ve->siblings[0]);
+ err = __execlists_context_pin(ce, ve->siblings[0], vaddr);
if (err)
return err;
@@ -4803,8 +4811,10 @@ static void virtual_context_exit(struct intel_context *ce)
static const struct intel_context_ops virtual_context_ops = {
.alloc = virtual_context_alloc,
+ .pre_pin = execlists_context_pre_pin,
.pin = virtual_context_pin,
.unpin = execlists_context_unpin,
+ .post_unpin = execlists_context_post_unpin,
.enter = virtual_context_enter,
.exit = virtual_context_exit,
diff --git a/drivers/gpu/drm/i915/gt/intel_renderstate.c b/drivers/gpu/drm/i915/gt/intel_renderstate.c
index 09ea4d0a14f9..37c4c253942b 100644
--- a/drivers/gpu/drm/i915/gt/intel_renderstate.c
+++ b/drivers/gpu/drm/i915/gt/intel_renderstate.c
@@ -270,7 +270,6 @@ void intel_renderstate_fini(struct intel_renderstate *so,
struct intel_context *ce)
{
i915_vma_unpin_and_release(&so->vma, 0);
-
intel_context_unpin(ce);
i915_gem_ww_ctx_fini(&so->ww);
}
diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
index f70b903a98bc..2a27dbcec00e 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
@@ -1206,6 +1206,10 @@ static void __context_unpin_ppgtt(struct intel_context *ce)
}
static void ring_context_unpin(struct intel_context *ce)
+{
+}
+
+static void ring_context_post_unpin(struct intel_context *ce)
{
__context_unpin_ppgtt(ce);
}
@@ -1303,11 +1307,16 @@ static int ring_context_alloc(struct intel_context *ce)
return 0;
}
-static int ring_context_pin(struct intel_context *ce)
+static int ring_context_pre_pin(struct intel_context *ce, void **unused)
{
return __context_pin_ppgtt(ce);
}
+static int ring_context_pin(struct intel_context *ce, void *unused)
+{
+ return 0;
+}
+
static void ring_context_reset(struct intel_context *ce)
{
intel_ring_reset(ce->ring, ce->ring->emit);
@@ -1316,8 +1325,10 @@ static void ring_context_reset(struct intel_context *ce)
static const struct intel_context_ops ring_context_ops = {
.alloc = ring_context_alloc,
+ .pre_pin = ring_context_pre_pin,
.pin = ring_context_pin,
.unpin = ring_context_unpin,
+ .post_unpin = ring_context_post_unpin,
.enter = intel_context_enter_engine,
.exit = intel_context_exit_engine,
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index f2806381733f..3b15e749a0ed 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -129,6 +129,10 @@ static void mock_context_unpin(struct intel_context *ce)
{
}
+static void mock_context_post_unpin(struct intel_context *ce)
+{
+}
+
static void mock_context_destroy(struct kref *ref)
{
struct intel_context *ce = container_of(ref, typeof(*ce), ref);
@@ -162,7 +166,12 @@ static int mock_context_alloc(struct intel_context *ce)
return 0;
}
-static int mock_context_pin(struct intel_context *ce)
+static int mock_context_pre_pin(struct intel_context *ce, void **unused)
+{
+ return 0;
+}
+
+static int mock_context_pin(struct intel_context *ce, void *unused)
{
return 0;
}
@@ -174,8 +183,10 @@ static void mock_context_reset(struct intel_context *ce)
static const struct intel_context_ops mock_context_ops = {
.alloc = mock_context_alloc,
+ .pre_pin = mock_context_pre_pin,
.pin = mock_context_pin,
.unpin = mock_context_unpin,
+ .post_unpin = mock_context_post_unpin,
.enter = intel_context_enter_engine,
.exit = intel_context_exit_engine,
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 13/19] drm/i915: Make sure execbuffer always passes ww state to i915_vma_pin.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (11 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 12/19] drm/i915: Rework intel_context pinning to do everything outside of pin_mutex Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 14/19] drm/i915: Convert i915_gem_object/client_blt.c to use ww locking as well Maarten Lankhorst
` (7 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
As a preparation step for full object locking and wait/wound handling
during pin and object mapping, ensure that we always pass the ww context
in i915_gem_execbuffer.c to i915_vma_pin, use lockdep to ensure this
happens.
This also requires changing the order of eb_parse slightly, to ensure
we pass ww at a point where we could still handle -EDEADLK safely.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/display/intel_display.c | 2 +-
drivers/gpu/drm/i915/gem/i915_gem_context.c | 4 +-
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 124 ++++++++++--------
drivers/gpu/drm/i915/gt/gen6_ppgtt.c | 4 +-
drivers/gpu/drm/i915/gt/gen6_ppgtt.h | 4 +-
drivers/gpu/drm/i915/gt/intel_context.c | 65 +++++----
drivers/gpu/drm/i915/gt/intel_context.h | 13 ++
drivers/gpu/drm/i915/gt/intel_context_types.h | 3 +-
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +-
drivers/gpu/drm/i915/gt/intel_gt.c | 2 +-
drivers/gpu/drm/i915/gt/intel_lrc.c | 5 +-
drivers/gpu/drm/i915/gt/intel_renderstate.c | 2 +-
drivers/gpu/drm/i915/gt/intel_ring.c | 10 +-
drivers/gpu/drm/i915/gt/intel_ring.h | 3 +-
.../gpu/drm/i915/gt/intel_ring_submission.c | 15 +--
drivers/gpu/drm/i915/gt/intel_timeline.c | 12 +-
drivers/gpu/drm/i915/gt/intel_timeline.h | 3 +-
drivers/gpu/drm/i915/gt/mock_engine.c | 3 +-
drivers/gpu/drm/i915/gt/selftest_timeline.c | 4 +-
drivers/gpu/drm/i915/gt/uc/intel_guc.c | 2 +-
drivers/gpu/drm/i915/i915_drv.h | 13 +-
drivers/gpu/drm/i915/i915_gem.c | 11 +-
drivers/gpu/drm/i915/i915_vma.c | 13 +-
drivers/gpu/drm/i915/i915_vma.h | 13 +-
24 files changed, 205 insertions(+), 127 deletions(-)
diff --git a/drivers/gpu/drm/i915/display/intel_display.c b/drivers/gpu/drm/i915/display/intel_display.c
index 71d6daa273c0..3ad114d3896f 100644
--- a/drivers/gpu/drm/i915/display/intel_display.c
+++ b/drivers/gpu/drm/i915/display/intel_display.c
@@ -3436,7 +3436,7 @@ initial_plane_vma(struct drm_i915_private *i915,
if (IS_ERR(vma))
goto err_obj;
- if (i915_ggtt_pin(vma, 0, PIN_MAPPABLE | PIN_OFFSET_FIXED | base))
+ if (i915_ggtt_pin(vma, NULL, 0, PIN_MAPPABLE | PIN_OFFSET_FIXED | base))
goto err_obj;
if (i915_gem_object_is_tiled(obj) &&
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index 2152ac209450..7cfccd91c6d8 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -1016,7 +1016,7 @@ static int context_barrier_task(struct i915_gem_context *ctx,
i915_gem_ww_ctx_init(&ww, true);
retry:
- err = intel_context_pin(ce);
+ err = intel_context_pin_ww(ce, &ww);
if (err)
goto err;
@@ -1109,7 +1109,7 @@ static int pin_ppgtt_update(struct intel_context *ce, struct i915_gem_ww_ctx *ww
if (!HAS_LOGICAL_RING_CONTEXTS(vm->i915))
/* ppGTT is not part of the legacy context image */
- return gen6_ppgtt_pin(i915_vm_to_ppgtt(vm));
+ return gen6_ppgtt_pin(i915_vm_to_ppgtt(vm), ww);
return 0;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index 72bad053c6b5..4d3ec33cf893 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -395,7 +395,7 @@ eb_pin_vma(struct i915_execbuffer *eb,
if (unlikely(ev->flags & EXEC_OBJECT_NEEDS_GTT))
pin_flags |= PIN_GLOBAL;
- if (unlikely(i915_vma_pin(vma, 0, 0, pin_flags)))
+ if (unlikely(i915_vma_pin_ww(vma, &eb->ww, 0, 0, pin_flags)))
return false;
if (unlikely(ev->flags & EXEC_OBJECT_NEEDS_FENCE)) {
@@ -548,7 +548,7 @@ static inline int use_cpu_reloc(const struct reloc_cache *cache,
obj->cache_level != I915_CACHE_NONE);
}
-static int eb_reserve_vma(const struct i915_execbuffer *eb, struct eb_vma *ev)
+static int eb_reserve_vma(struct i915_execbuffer *eb, struct eb_vma *ev)
{
struct drm_i915_gem_exec_object2 *entry = ev->exec;
unsigned int exec_flags = ev->flags;
@@ -577,7 +577,7 @@ static int eb_reserve_vma(const struct i915_execbuffer *eb, struct eb_vma *ev)
pin_flags |= BATCH_OFFSET_BIAS | PIN_OFFSET_BIAS;
}
- err = i915_vma_pin(vma,
+ err = i915_vma_pin_ww(vma, &eb->ww,
entry->pad_to_size, entry->alignment,
pin_flags);
if (err)
@@ -1025,9 +1025,10 @@ static void *reloc_kmap(struct drm_i915_gem_object *obj,
}
static void *reloc_iomap(struct drm_i915_gem_object *obj,
- struct reloc_cache *cache,
+ struct i915_execbuffer *eb,
unsigned long page)
{
+ struct reloc_cache *cache = &eb->reloc_cache;
struct i915_ggtt *ggtt = cache_to_ggtt(cache);
unsigned long offset;
void *vaddr;
@@ -1049,10 +1050,10 @@ static void *reloc_iomap(struct drm_i915_gem_object *obj,
if (err)
return ERR_PTR(err);
- vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0,
- PIN_MAPPABLE |
- PIN_NONBLOCK /* NOWARN */ |
- PIN_NOEVICT);
+ vma = i915_gem_object_ggtt_pin_ww(obj, &eb->ww, NULL, 0, 0,
+ PIN_MAPPABLE |
+ PIN_NONBLOCK /* NOWARN */ |
+ PIN_NOEVICT);
if (IS_ERR(vma)) {
memset(&cache->node, 0, sizeof(cache->node));
mutex_lock(&ggtt->vm.mutex);
@@ -1088,9 +1089,10 @@ static void *reloc_iomap(struct drm_i915_gem_object *obj,
}
static void *reloc_vaddr(struct drm_i915_gem_object *obj,
- struct reloc_cache *cache,
+ struct i915_execbuffer *eb,
unsigned long page)
{
+ struct reloc_cache *cache = &eb->reloc_cache;
void *vaddr;
if (cache->page == page) {
@@ -1098,7 +1100,7 @@ static void *reloc_vaddr(struct drm_i915_gem_object *obj,
} else {
vaddr = NULL;
if ((cache->vaddr & KMAP) == 0)
- vaddr = reloc_iomap(obj, cache, page);
+ vaddr = reloc_iomap(obj, eb, page);
if (!vaddr)
vaddr = reloc_kmap(obj, cache, page);
}
@@ -1181,7 +1183,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb,
goto err_unmap;
}
- err = i915_vma_pin(batch, 0, 0, PIN_USER | PIN_NONBLOCK);
+ err = i915_vma_pin_ww(batch, &eb->ww, 0, 0, PIN_USER | PIN_NONBLOCK);
if (err)
goto err_unmap;
@@ -1335,7 +1337,7 @@ relocate_entry(struct i915_vma *vma,
}
repeat:
- vaddr = reloc_vaddr(vma->obj, &eb->reloc_cache, offset >> PAGE_SHIFT);
+ vaddr = reloc_vaddr(vma->obj, eb, offset >> PAGE_SHIFT);
if (IS_ERR(vaddr))
return PTR_ERR(vaddr);
@@ -1761,6 +1763,7 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
rq = eb_pin_engine(eb, false);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
+ rq = NULL;
goto err;
}
@@ -2031,7 +2034,8 @@ static int i915_reset_gen7_sol_offsets(struct i915_request *rq)
}
static struct i915_vma *
-shadow_batch_pin(struct drm_i915_gem_object *obj,
+shadow_batch_pin(struct i915_execbuffer *eb,
+ struct drm_i915_gem_object *obj,
struct i915_address_space *vm,
unsigned int flags)
{
@@ -2042,7 +2046,7 @@ shadow_batch_pin(struct drm_i915_gem_object *obj,
if (IS_ERR(vma))
return vma;
- err = i915_vma_pin(vma, 0, 0, flags);
+ err = i915_vma_pin_ww(vma, &eb->ww, 0, 0, flags);
if (err)
return ERR_PTR(err);
@@ -2154,16 +2158,33 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
return err;
}
+static struct i915_vma *eb_dispatch_secure(struct i915_execbuffer *eb, struct i915_vma *vma)
+{
+ /*
+ * snb/ivb/vlv conflate the "batch in ppgtt" bit with the "non-secure
+ * batch" bit. Hence we need to pin secure batches into the global gtt.
+ * hsw should have this fixed, but bdw mucks it up again. */
+ if (eb->batch_flags & I915_DISPATCH_SECURE)
+ return i915_gem_object_ggtt_pin_ww(vma->obj, &eb->ww, NULL, 0, 0, 0);
+
+ return NULL;
+}
+
static int eb_parse(struct i915_execbuffer *eb)
{
struct drm_i915_private *i915 = eb->i915;
struct intel_engine_pool_node *pool;
- struct i915_vma *shadow, *trampoline;
+ struct i915_vma *shadow, *trampoline, *batch;
unsigned int len;
int err;
- if (!eb_use_cmdparser(eb))
- return 0;
+ if (!eb_use_cmdparser(eb)) {
+ batch = eb_dispatch_secure(eb, eb->batch->vma);
+ if (IS_ERR(batch))
+ return PTR_ERR(batch);
+
+ goto secure_batch;
+ }
len = eb->batch_len;
if (!CMDPARSER_USES_GGTT(eb->i915)) {
@@ -2188,7 +2209,7 @@ static int eb_parse(struct i915_execbuffer *eb)
if (err)
goto err;
- shadow = shadow_batch_pin(pool->obj, eb->context->vm, PIN_USER);
+ shadow = shadow_batch_pin(eb, pool->obj, eb->context->vm, PIN_USER);
if (IS_ERR(shadow)) {
err = PTR_ERR(shadow);
goto err;
@@ -2199,7 +2220,7 @@ static int eb_parse(struct i915_execbuffer *eb)
if (CMDPARSER_USES_GGTT(eb->i915)) {
trampoline = shadow;
- shadow = shadow_batch_pin(pool->obj,
+ shadow = shadow_batch_pin(eb, pool->obj,
&eb->engine->gt->ggtt->vm,
PIN_GLOBAL);
if (IS_ERR(shadow)) {
@@ -2211,21 +2232,37 @@ static int eb_parse(struct i915_execbuffer *eb)
eb->batch_flags |= I915_DISPATCH_SECURE;
}
+ batch = eb_dispatch_secure(eb, shadow);
+ if (IS_ERR(batch)) {
+ err = PTR_ERR(batch);
+ goto err_trampoline;
+ }
+
err = eb_parse_pipeline(eb, shadow, trampoline);
if (err)
- goto err_trampoline;
+ goto err_unpin_batch;
- eb->vma[eb->buffer_count].vma = i915_vma_get(shadow);
- eb->vma[eb->buffer_count].flags =
- __EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_REF;
eb->batch = &eb->vma[eb->buffer_count++];
+ eb->batch->vma = i915_vma_get(shadow);
+ eb->batch->flags =
+ __EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_REF;
eb->trampoline = trampoline;
eb->batch_start_offset = 0;
shadow->private = pool;
+
+secure_batch:
+ if (batch) {
+ eb->batch = &eb->vma[eb->buffer_count++];
+ eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
+ eb->batch->vma = batch;
+ }
return 0;
+err_unpin_batch:
+ if (batch)
+ i915_vma_unpin(batch);
err_trampoline:
if (trampoline)
i915_vma_unpin(trampoline);
@@ -2381,7 +2418,7 @@ static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throt
* GGTT space, so do this first before we reserve a seqno for
* ourselves.
*/
- err = intel_context_pin(ce);
+ err = intel_context_pin_ww(ce, &eb->ww);
if (err)
return ERR_PTR(err);
@@ -2750,34 +2787,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
}
ww_acquire_done(&eb.ww.ctx);
-
- /*
- * snb/ivb/vlv conflate the "batch in ppgtt" bit with the "non-secure
- * batch" bit. Hence we need to pin secure batches into the global gtt.
- * hsw should have this fixed, but bdw mucks it up again. */
- if (eb.batch_flags & I915_DISPATCH_SECURE) {
- struct i915_vma *vma;
-
- /*
- * So on first glance it looks freaky that we pin the batch here
- * outside of the reservation loop. But:
- * - The batch is already pinned into the relevant ppgtt, so we
- * already have the backing storage fully allocated.
- * - No other BO uses the global gtt (well contexts, but meh),
- * so we don't really have issues with multiple objects not
- * fitting due to fragmentation.
- * So this is actually safe.
- */
- vma = i915_gem_object_ggtt_pin(eb.batch->vma->obj, NULL, 0, 0, 0);
- if (IS_ERR(vma)) {
- err = PTR_ERR(vma);
- goto err_vma;
- }
-
- batch = vma;
- } else {
- batch = eb.batch->vma;
- }
+ batch = eb.batch->vma;
/* All GPU relocation batches must be submitted prior to the user rq */
GEM_BUG_ON(eb.reloc_cache.rq);
@@ -2850,8 +2860,6 @@ i915_gem_do_execbuffer(struct drm_device *dev,
i915_request_put(eb.request);
err_batch_unpin:
- if (eb.batch_flags & I915_DISPATCH_SECURE)
- i915_vma_unpin(batch);
if (batch->private)
intel_engine_pool_put(batch->private);
err_vma:
@@ -2939,7 +2947,9 @@ i915_gem_execbuffer_ioctl(struct drm_device *dev, void *data,
/* Copy in the exec list from userland */
exec_list = kvmalloc_array(count, sizeof(*exec_list),
__GFP_NOWARN | GFP_KERNEL);
- exec2_list = kvmalloc_array(count + 1, eb_element_size(),
+
+ /* Allocate extra slots for use by the command parser */
+ exec2_list = kvmalloc_array(count + 2, eb_element_size(),
__GFP_NOWARN | GFP_KERNEL);
if (exec_list == NULL || exec2_list == NULL) {
drm_dbg(&i915->drm,
@@ -3017,8 +3027,8 @@ i915_gem_execbuffer2_ioctl(struct drm_device *dev, void *data,
if (err)
return err;
- /* Allocate an extra slot for use by the command parser */
- exec2_list = kvmalloc_array(count + 1, eb_element_size(),
+ /* Allocate extra slots for use by the command parser */
+ exec2_list = kvmalloc_array(count + 2, eb_element_size(),
__GFP_NOWARN | GFP_KERNEL);
if (exec2_list == NULL) {
drm_dbg(&i915->drm, "Failed to allocate exec list for %zd buffers\n",
diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c
index f4fec7eb4064..8248efa9229f 100644
--- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.c
+++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.c
@@ -376,7 +376,7 @@ static struct i915_vma *pd_vma_create(struct gen6_ppgtt *ppgtt, int size)
return vma;
}
-int gen6_ppgtt_pin(struct i915_ppgtt *base)
+int gen6_ppgtt_pin(struct i915_ppgtt *base, struct i915_gem_ww_ctx *ww)
{
struct gen6_ppgtt *ppgtt = to_gen6_ppgtt(base);
int err;
@@ -402,7 +402,7 @@ int gen6_ppgtt_pin(struct i915_ppgtt *base)
*/
err = 0;
if (!atomic_read(&ppgtt->pin_count))
- err = i915_ggtt_pin(ppgtt->vma, GEN6_PD_ALIGN, PIN_HIGH);
+ err = i915_ggtt_pin(ppgtt->vma, ww, GEN6_PD_ALIGN, PIN_HIGH);
if (!err)
atomic_inc(&ppgtt->pin_count);
mutex_unlock(&ppgtt->pin_mutex);
diff --git a/drivers/gpu/drm/i915/gt/gen6_ppgtt.h b/drivers/gpu/drm/i915/gt/gen6_ppgtt.h
index 72e481806c96..00032a931bae 100644
--- a/drivers/gpu/drm/i915/gt/gen6_ppgtt.h
+++ b/drivers/gpu/drm/i915/gt/gen6_ppgtt.h
@@ -8,6 +8,8 @@
#include "intel_gtt.h"
+struct i915_gem_ww_ctx;
+
struct gen6_ppgtt {
struct i915_ppgtt base;
@@ -66,7 +68,7 @@ static inline struct gen6_ppgtt *to_gen6_ppgtt(struct i915_ppgtt *base)
(pt = i915_pt_entry(pd, iter), true); \
++iter)
-int gen6_ppgtt_pin(struct i915_ppgtt *base);
+int gen6_ppgtt_pin(struct i915_ppgtt *base, struct i915_gem_ww_ctx *ww);
void gen6_ppgtt_unpin(struct i915_ppgtt *base);
void gen6_ppgtt_unpin_all(struct i915_ppgtt *base);
void gen6_ppgtt_enable(struct intel_gt *gt);
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 44dafa5177c3..f64b1020f686 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -88,12 +88,12 @@ static void intel_context_active_release(struct intel_context *ce)
i915_active_release(&ce->active);
}
-static int __context_pin_state(struct i915_vma *vma)
+static int __context_pin_state(struct i915_vma *vma, struct i915_gem_ww_ctx *ww)
{
unsigned int bias = i915_ggtt_pin_bias(vma) | PIN_OFFSET_BIAS;
int err;
- err = i915_ggtt_pin(vma, 0, bias | PIN_HIGH);
+ err = i915_ggtt_pin(vma, ww, 0, bias | PIN_HIGH);
if (err)
return err;
@@ -122,7 +122,8 @@ static void __context_unpin_state(struct i915_vma *vma)
__i915_vma_unpin(vma);
}
-static int __ring_active(struct intel_ring *ring)
+static int __ring_active(struct intel_ring *ring,
+ struct i915_gem_ww_ctx *ww)
{
int err;
@@ -130,7 +131,7 @@ static int __ring_active(struct intel_ring *ring)
if (err)
return err;
- err = intel_ring_pin(ring);
+ err = intel_ring_pin(ring, ww);
if (err)
goto err_active;
@@ -147,24 +148,25 @@ static void __ring_retire(struct intel_ring *ring)
i915_active_release(&ring->vma->active);
}
-static int intel_context_pre_pin(struct intel_context *ce)
+static int intel_context_pre_pin(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww)
{
int err;
CE_TRACE(ce, "active\n");
- err = __ring_active(ce->ring);
+ err = __ring_active(ce->ring, ww);
if (err)
return err;
- err = intel_timeline_pin(ce->timeline);
+ err = intel_timeline_pin(ce->timeline, ww);
if (err)
goto err_ring;
if (!ce->state)
return 0;
- err = __context_pin_state(ce->state);
+ err = __context_pin_state(ce->state, ww);
if (err)
goto err_timeline;
@@ -187,7 +189,8 @@ static void intel_context_post_unpin(struct intel_context *ce)
__ring_retire(ce->ring);
}
-int __intel_context_do_pin(struct intel_context *ce)
+int __intel_context_do_pin_ww(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww)
{
bool handoff = false;
void *vaddr;
@@ -204,7 +207,14 @@ int __intel_context_do_pin(struct intel_context *ce)
* refcount for __intel_context_active(), which prevent a lock
* inversion of ce->pin_mutex vs dma_resv_lock().
*/
- err = intel_context_pre_pin(ce);
+
+ err = i915_gem_object_lock(ce->timeline->hwsp_ggtt->obj, ww);
+ if (!err && ce->ring->vma->obj)
+ err = i915_gem_object_lock(ce->ring->vma->obj, ww);
+ if (!err && ce->state)
+ err = i915_gem_object_lock(ce->state->obj, ww);
+ if (!err)
+ err = intel_context_pre_pin(ce, ww);
if (err)
return err;
@@ -212,7 +222,7 @@ int __intel_context_do_pin(struct intel_context *ce)
if (err)
goto err_ctx_unpin;
- err = ce->ops->pre_pin(ce, &vaddr);
+ err = ce->ops->pre_pin(ce, ww, &vaddr);
if (err)
goto err_release;
@@ -254,6 +264,23 @@ int __intel_context_do_pin(struct intel_context *ce)
return err;
}
+int __intel_context_do_pin(struct intel_context *ce)
+{
+ struct i915_gem_ww_ctx ww;
+ int err;
+
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ err = __intel_context_do_pin_ww(ce, &ww);
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
+ return err;
+}
+
void intel_context_unpin(struct intel_context *ce)
{
if (!atomic_dec_and_test(&ce->pin_count))
@@ -290,18 +317,14 @@ __i915_active_call
static int __intel_context_active(struct i915_active *active)
{
struct intel_context *ce = container_of(active, typeof(*ce), active);
- int err;
intel_context_get(ce);
/* everything should already be activated by intel_context_pre_pin() */
- err = __ring_active(ce->ring);
- if (GEM_WARN_ON(err))
- goto err_put;
+ GEM_WARN_ON(!i915_active_acquire_if_busy(&ce->ring->vma->active));
+ __intel_ring_pin(ce->ring);
- err = intel_timeline_pin(ce->timeline);
- if (GEM_WARN_ON(err))
- goto err_ring;
+ __intel_timeline_pin(ce->timeline);
if (ce->state) {
GEM_WARN_ON(!i915_active_acquire_if_busy(&ce->state->active));
@@ -310,12 +333,6 @@ static int __intel_context_active(struct i915_active *active)
}
return 0;
-
-err_ring:
- __ring_retire(ce->ring);
-err_put:
- intel_context_put(ce);
- return err;
}
void
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index 604d5cfc46ba..563e3d6d9a19 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -24,6 +24,8 @@
##__VA_ARGS__); \
} while (0)
+struct i915_gem_ww_ctx;
+
void intel_context_init(struct intel_context *ce,
struct intel_engine_cs *engine);
void intel_context_fini(struct intel_context *ce);
@@ -80,6 +82,8 @@ static inline void intel_context_unlock_pinned(struct intel_context *ce)
}
int __intel_context_do_pin(struct intel_context *ce);
+int __intel_context_do_pin_ww(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww);
static inline bool intel_context_pin_if_active(struct intel_context *ce)
{
@@ -94,6 +98,15 @@ static inline int intel_context_pin(struct intel_context *ce)
return __intel_context_do_pin(ce);
}
+static inline int intel_context_pin_ww(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww)
+{
+ if (likely(intel_context_pin_if_active(ce)))
+ return 0;
+
+ return __intel_context_do_pin_ww(ce, ww);
+}
+
static inline void __intel_context_pin(struct intel_context *ce)
{
GEM_BUG_ON(!intel_context_is_pinned(ce));
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 65457354eb39..c5766baa40db 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -20,6 +20,7 @@
#define CONTEXT_REDZONE POISON_INUSE
struct i915_gem_context;
+struct i915_gem_ww_ctx;
struct i915_vma;
struct intel_context;
struct intel_ring;
@@ -27,7 +28,7 @@ struct intel_ring;
struct intel_context_ops {
int (*alloc)(struct intel_context *ce);
- int (*pre_pin)(struct intel_context *ce, void **vaddr);
+ int (*pre_pin)(struct intel_context *ce, struct i915_gem_ww_ctx *ww, void **vaddr);
int (*pin)(struct intel_context *ce, void *vaddr);
void (*unpin)(struct intel_context *ce);
void (*post_unpin)(struct intel_context *ce);
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index f6f5e1ec48fc..9265bb3f1f46 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -543,7 +543,7 @@ static int pin_ggtt_status_page(struct intel_engine_cs *engine,
else
flags = PIN_HIGH;
- return i915_ggtt_pin(vma, 0, flags);
+ return i915_ggtt_pin(vma, NULL, 0, flags);
}
static int init_status_page(struct intel_engine_cs *engine)
diff --git a/drivers/gpu/drm/i915/gt/intel_gt.c b/drivers/gpu/drm/i915/gt/intel_gt.c
index bcae951392ce..c02ea8012a4e 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt.c
+++ b/drivers/gpu/drm/i915/gt/intel_gt.c
@@ -344,7 +344,7 @@ static int intel_gt_init_scratch(struct intel_gt *gt, unsigned int size)
goto err_unref;
}
- ret = i915_ggtt_pin(vma, 0, PIN_HIGH);
+ ret = i915_ggtt_pin(vma, NULL, 0, PIN_HIGH);
if (ret)
goto err_unref;
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index d9ae07411e62..e9ffbc5334d4 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -2933,7 +2933,8 @@ __execlists_update_reg_state(const struct intel_context *ce,
}
static int
-execlists_context_pre_pin(struct intel_context *ce, void **vaddr)
+execlists_context_pre_pin(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww, void **vaddr)
{
GEM_BUG_ON(!ce->state);
GEM_BUG_ON(!i915_vma_is_pinned(ce->state));
@@ -3295,7 +3296,7 @@ static int lrc_setup_wa_ctx(struct intel_engine_cs *engine)
goto err;
}
- err = i915_ggtt_pin(vma, 0, PIN_HIGH);
+ err = i915_ggtt_pin(vma, NULL, 0, PIN_HIGH);
if (err)
goto err;
diff --git a/drivers/gpu/drm/i915/gt/intel_renderstate.c b/drivers/gpu/drm/i915/gt/intel_renderstate.c
index 37c4c253942b..6fd946016628 100644
--- a/drivers/gpu/drm/i915/gt/intel_renderstate.c
+++ b/drivers/gpu/drm/i915/gt/intel_renderstate.c
@@ -195,7 +195,7 @@ int intel_renderstate_init(struct intel_renderstate *so,
i915_gem_ww_ctx_init(&so->ww, true);
retry:
- err = intel_context_pin(ce);
+ err = intel_context_pin_ww(ce, &so->ww);
if (err)
goto err_fini;
diff --git a/drivers/gpu/drm/i915/gt/intel_ring.c b/drivers/gpu/drm/i915/gt/intel_ring.c
index 8cda1b7e17ba..71b404973ce1 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring.c
+++ b/drivers/gpu/drm/i915/gt/intel_ring.c
@@ -21,7 +21,13 @@ unsigned int intel_ring_update_space(struct intel_ring *ring)
return space;
}
-int intel_ring_pin(struct intel_ring *ring)
+void __intel_ring_pin(struct intel_ring *ring)
+{
+ GEM_BUG_ON(!atomic_read(&ring->pin_count));
+ atomic_inc(&ring->pin_count);
+}
+
+int intel_ring_pin(struct intel_ring *ring, struct i915_gem_ww_ctx *ww)
{
struct i915_vma *vma = ring->vma;
unsigned int flags;
@@ -39,7 +45,7 @@ int intel_ring_pin(struct intel_ring *ring)
else
flags |= PIN_HIGH;
- ret = i915_ggtt_pin(vma, 0, flags);
+ ret = i915_ggtt_pin(vma, ww, 0, flags);
if (unlikely(ret))
goto err_unpin;
diff --git a/drivers/gpu/drm/i915/gt/intel_ring.h b/drivers/gpu/drm/i915/gt/intel_ring.h
index 5bdce24994aa..48a2c327efb3 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring.h
+++ b/drivers/gpu/drm/i915/gt/intel_ring.h
@@ -21,7 +21,8 @@ int intel_ring_cacheline_align(struct i915_request *rq);
unsigned int intel_ring_update_space(struct intel_ring *ring);
-int intel_ring_pin(struct intel_ring *ring);
+void __intel_ring_pin(struct intel_ring *ring);
+int intel_ring_pin(struct intel_ring *ring, struct i915_gem_ww_ctx *ww);
void intel_ring_unpin(struct intel_ring *ring);
void intel_ring_reset(struct intel_ring *ring, u32 tail);
diff --git a/drivers/gpu/drm/i915/gt/intel_ring_submission.c b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
index 2a27dbcec00e..5c2525ec8662 100644
--- a/drivers/gpu/drm/i915/gt/intel_ring_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_ring_submission.c
@@ -1184,14 +1184,16 @@ static void ring_context_destroy(struct kref *ref)
intel_context_free(ce);
}
-static int __context_pin_ppgtt(struct intel_context *ce)
+static int ring_context_pre_pin(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww,
+ void **unused)
{
struct i915_address_space *vm;
int err = 0;
vm = vm_alias(ce->vm);
if (vm)
- err = gen6_ppgtt_pin(i915_vm_to_ppgtt((vm)));
+ err = gen6_ppgtt_pin(i915_vm_to_ppgtt((vm)), ww);
return err;
}
@@ -1307,11 +1309,6 @@ static int ring_context_alloc(struct intel_context *ce)
return 0;
}
-static int ring_context_pre_pin(struct intel_context *ce, void **unused)
-{
- return __context_pin_ppgtt(ce);
-}
-
static int ring_context_pin(struct intel_context *ce, void *unused)
{
return 0;
@@ -1960,7 +1957,7 @@ int intel_ring_submission_setup(struct intel_engine_cs *engine)
}
GEM_BUG_ON(timeline->has_initial_breadcrumb);
- err = intel_timeline_pin(timeline);
+ err = intel_timeline_pin(timeline, NULL);
if (err)
goto err_timeline;
@@ -1970,7 +1967,7 @@ int intel_ring_submission_setup(struct intel_engine_cs *engine)
goto err_timeline_unpin;
}
- err = intel_ring_pin(ring);
+ err = intel_ring_pin(ring, NULL);
if (err)
goto err_ring;
diff --git a/drivers/gpu/drm/i915/gt/intel_timeline.c b/drivers/gpu/drm/i915/gt/intel_timeline.c
index 54e1e55f3c81..a0bbbfca93b7 100644
--- a/drivers/gpu/drm/i915/gt/intel_timeline.c
+++ b/drivers/gpu/drm/i915/gt/intel_timeline.c
@@ -301,14 +301,20 @@ intel_timeline_create(struct intel_gt *gt, struct i915_vma *global_hwsp)
return timeline;
}
-int intel_timeline_pin(struct intel_timeline *tl)
+void __intel_timeline_pin(struct intel_timeline *tl)
+{
+ GEM_BUG_ON(!atomic_read(&tl->pin_count));
+ atomic_inc(&tl->pin_count);
+}
+
+int intel_timeline_pin(struct intel_timeline *tl, struct i915_gem_ww_ctx *ww)
{
int err;
if (atomic_add_unless(&tl->pin_count, 1, 0))
return 0;
- err = i915_ggtt_pin(tl->hwsp_ggtt, 0, PIN_HIGH);
+ err = i915_ggtt_pin(tl->hwsp_ggtt, ww, 0, PIN_HIGH);
if (err)
return err;
@@ -433,7 +439,7 @@ __intel_timeline_get_seqno(struct intel_timeline *tl,
goto err_rollback;
}
- err = i915_ggtt_pin(vma, 0, PIN_HIGH);
+ err = i915_ggtt_pin(vma, NULL, 0, PIN_HIGH);
if (err) {
__idle_hwsp_free(vma->private, cacheline);
goto err_rollback;
diff --git a/drivers/gpu/drm/i915/gt/intel_timeline.h b/drivers/gpu/drm/i915/gt/intel_timeline.h
index f5b7eade3809..39dd897ad057 100644
--- a/drivers/gpu/drm/i915/gt/intel_timeline.h
+++ b/drivers/gpu/drm/i915/gt/intel_timeline.h
@@ -76,7 +76,8 @@ static inline bool intel_timeline_sync_is_later(struct intel_timeline *tl,
return __intel_timeline_sync_is_later(tl, fence->context, fence->seqno);
}
-int intel_timeline_pin(struct intel_timeline *tl);
+void __intel_timeline_pin(struct intel_timeline *tl);
+int intel_timeline_pin(struct intel_timeline *tl, struct i915_gem_ww_ctx *ww);
void intel_timeline_enter(struct intel_timeline *tl);
int intel_timeline_get_seqno(struct intel_timeline *tl,
struct i915_request *rq,
diff --git a/drivers/gpu/drm/i915/gt/mock_engine.c b/drivers/gpu/drm/i915/gt/mock_engine.c
index 3b15e749a0ed..b87b019f733a 100644
--- a/drivers/gpu/drm/i915/gt/mock_engine.c
+++ b/drivers/gpu/drm/i915/gt/mock_engine.c
@@ -166,7 +166,8 @@ static int mock_context_alloc(struct intel_context *ce)
return 0;
}
-static int mock_context_pre_pin(struct intel_context *ce, void **unused)
+static int mock_context_pre_pin(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww, void **unused)
{
return 0;
}
diff --git a/drivers/gpu/drm/i915/gt/selftest_timeline.c b/drivers/gpu/drm/i915/gt/selftest_timeline.c
index c2578a0f2f14..00801549871a 100644
--- a/drivers/gpu/drm/i915/gt/selftest_timeline.c
+++ b/drivers/gpu/drm/i915/gt/selftest_timeline.c
@@ -454,7 +454,7 @@ tl_write(struct intel_timeline *tl, struct intel_engine_cs *engine, u32 value)
struct i915_request *rq;
int err;
- err = intel_timeline_pin(tl);
+ err = intel_timeline_pin(tl, NULL);
if (err) {
rq = ERR_PTR(err);
goto out;
@@ -664,7 +664,7 @@ static int live_hwsp_wrap(void *arg)
if (!tl->has_initial_breadcrumb || !tl->hwsp_cacheline)
goto out_free;
- err = intel_timeline_pin(tl);
+ err = intel_timeline_pin(tl, NULL);
if (err)
goto out_free;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
index c4c1523da7a6..6b9a947cf24d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
@@ -679,7 +679,7 @@ struct i915_vma *intel_guc_allocate_vma(struct intel_guc *guc, u32 size)
goto err;
flags = PIN_OFFSET_BIAS | i915_ggtt_pin_bias(vma);
- ret = i915_ggtt_pin(vma, 0, flags);
+ ret = i915_ggtt_pin(vma, NULL, 0, flags);
if (ret) {
vma = ERR_PTR(ret);
goto err;
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 6f351d85b226..ff206dfc9ead 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1818,11 +1818,18 @@ static inline void i915_gem_drain_workqueue(struct drm_i915_private *i915)
}
struct i915_vma * __must_check
+i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj,
+ struct i915_gem_ww_ctx *ww,
+ const struct i915_ggtt_view *view,
+ u64 size, u64 alignment, u64 flags);
+
+static inline struct i915_vma * __must_check
i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj,
const struct i915_ggtt_view *view,
- u64 size,
- u64 alignment,
- u64 flags);
+ u64 size, u64 alignment, u64 flags)
+{
+ return i915_gem_object_ggtt_pin_ww(obj, NULL, view, size, alignment, flags);
+}
int i915_gem_object_unbind(struct drm_i915_gem_object *obj,
unsigned long flags);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 84b95fbd0b11..c72c39a3a76d 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -946,11 +946,10 @@ void i915_gem_runtime_suspend(struct drm_i915_private *i915)
}
struct i915_vma *
-i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj,
- const struct i915_ggtt_view *view,
- u64 size,
- u64 alignment,
- u64 flags)
+i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj,
+ struct i915_gem_ww_ctx *ww,
+ const struct i915_ggtt_view *view,
+ u64 size, u64 alignment, u64 flags)
{
struct drm_i915_private *i915 = to_i915(obj->base.dev);
struct i915_ggtt *ggtt = &i915->ggtt;
@@ -1018,7 +1017,7 @@ i915_gem_object_ggtt_pin(struct drm_i915_gem_object *obj,
return ERR_PTR(ret);
}
- ret = i915_vma_pin(vma, size, alignment, flags | PIN_GLOBAL);
+ ret = i915_vma_pin_ww(vma, ww, size, alignment, flags | PIN_GLOBAL);
if (ret)
return ERR_PTR(ret);
diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index 74dc3ba59ce5..461a3eaad4af 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -882,13 +882,19 @@ static void vma_unbind_pages(struct i915_vma *vma)
__vma_put_pages(vma, count | count << I915_VMA_PAGES_BIAS);
}
-int i915_vma_pin(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
+int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
+ u64 size, u64 alignment, u64 flags)
{
struct i915_vma_work *work = NULL;
intel_wakeref_t wakeref = 0;
unsigned int bound;
int err;
+#ifdef CONFIG_PROVE_LOCKING
+ if (debug_locks && lockdep_is_held(&vma->vm->i915->drm.struct_mutex))
+ WARN_ON(!ww);
+#endif
+
BUILD_BUG_ON(PIN_GLOBAL != I915_VMA_GLOBAL_BIND);
BUILD_BUG_ON(PIN_USER != I915_VMA_LOCAL_BIND);
@@ -995,7 +1001,8 @@ static void flush_idle_contexts(struct intel_gt *gt)
intel_gt_wait_for_idle(gt, MAX_SCHEDULE_TIMEOUT);
}
-int i915_ggtt_pin(struct i915_vma *vma, u32 align, unsigned int flags)
+int i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
+ u32 align, unsigned int flags)
{
struct i915_address_space *vm = vma->vm;
int err;
@@ -1003,7 +1010,7 @@ int i915_ggtt_pin(struct i915_vma *vma, u32 align, unsigned int flags)
GEM_BUG_ON(!i915_vma_is_ggtt(vma));
do {
- err = i915_vma_pin(vma, 0, align, flags | PIN_GLOBAL);
+ err = i915_vma_pin_ww(vma, ww, 0, align, flags | PIN_GLOBAL);
if (err != -ENOSPC) {
if (!err) {
err = i915_vma_wait_for_bind(vma);
diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
index e1ced1df13e1..4da7987712b3 100644
--- a/drivers/gpu/drm/i915/i915_vma.h
+++ b/drivers/gpu/drm/i915/i915_vma.h
@@ -236,8 +236,17 @@ static inline void i915_vma_unlock(struct i915_vma *vma)
}
int __must_check
-i915_vma_pin(struct i915_vma *vma, u64 size, u64 alignment, u64 flags);
-int i915_ggtt_pin(struct i915_vma *vma, u32 align, unsigned int flags);
+i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
+ u64 size, u64 alignment, u64 flags);
+
+static inline int __must_check
+i915_vma_pin(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
+{
+ return i915_vma_pin_ww(vma, NULL, size, alignment, flags);
+}
+
+int i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
+ u32 align, unsigned int flags);
static inline int i915_vma_pin_count(const struct i915_vma *vma)
{
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 14/19] drm/i915: Convert i915_gem_object/client_blt.c to use ww locking as well
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (12 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 13/19] drm/i915: Make sure execbuffer always passes ww state to i915_vma_pin Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 15/19] drm/i915: Kill last user of intel_context_create_request outside of selftests Maarten Lankhorst
` (6 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
This is the last part outside of selftests that still don't use the
correct lock ordering of timeline->mutex vs resv_lock.
With gem fixed, there are a few places that still get locking wrong:
- gvt/scheduler.c
- i915_perf.c
- Most if not all selftests.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
.../gpu/drm/i915/gem/i915_gem_client_blt.c | 78 ++++++---
.../gpu/drm/i915/gem/i915_gem_object_blt.c | 152 +++++++++++-------
.../gpu/drm/i915/gem/i915_gem_object_blt.h | 3 +
3 files changed, 159 insertions(+), 74 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c b/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c
index d92a35163330..9c0078650347 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_client_blt.c
@@ -157,6 +157,7 @@ static void clear_pages_worker(struct work_struct *work)
struct clear_pages_work *w = container_of(work, typeof(*w), work);
struct drm_i915_gem_object *obj = w->sleeve->vma->obj;
struct i915_vma *vma = w->sleeve->vma;
+ struct i915_gem_ww_ctx ww;
struct i915_request *rq;
struct i915_vma *batch;
int err = w->dma.error;
@@ -172,17 +173,19 @@ static void clear_pages_worker(struct work_struct *work)
obj->read_domains = I915_GEM_GPU_DOMAINS;
obj->write_domain = 0;
- err = i915_vma_pin(vma, 0, 0, PIN_USER);
- if (unlikely(err))
+ i915_gem_ww_ctx_init(&ww, false);
+retry:
+ err = intel_context_pin_ww(w->ce, &ww);
+ if (err)
goto out_signal;
- batch = intel_emit_vma_fill_blt(w->ce, vma, w->value);
+ batch = intel_emit_vma_fill_blt(w->ce, vma, &ww, w->value);
if (IS_ERR(batch)) {
err = PTR_ERR(batch);
- goto out_unpin;
+ goto out_ctx;
}
- rq = intel_context_create_request(w->ce);
+ rq = i915_request_create(w->ce);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
goto out_batch;
@@ -224,9 +227,18 @@ static void clear_pages_worker(struct work_struct *work)
i915_request_add(rq);
out_batch:
intel_emit_vma_release(w->ce, batch);
-out_unpin:
- i915_vma_unpin(vma);
+out_ctx:
+ intel_context_unpin(w->ce);
out_signal:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
+
+ i915_vma_unpin(w->sleeve->vma);
+
if (unlikely(err)) {
dma_fence_set_error(&w->dma, err);
dma_fence_signal(&w->dma);
@@ -234,6 +246,45 @@ static void clear_pages_worker(struct work_struct *work)
}
}
+static int pin_wait_clear_pages_work(struct clear_pages_work *w,
+ struct intel_context *ce)
+{
+ struct i915_vma *vma = w->sleeve->vma;
+ struct i915_gem_ww_ctx ww;
+ int err;
+
+ i915_gem_ww_ctx_init(&ww, false);
+retry:
+ err = i915_gem_object_lock(vma->obj, &ww);
+ if (err)
+ goto out;
+
+ err = i915_vma_pin_ww(vma, &ww, 0, 0, PIN_USER);
+ if (unlikely(err))
+ goto out;
+
+ err = i915_sw_fence_await_reservation(&w->wait,
+ vma->obj->base.resv, NULL,
+ true, I915_FENCE_TIMEOUT,
+ I915_FENCE_GFP);
+ if (err)
+ goto err_unpin_vma;
+
+ dma_resv_add_excl_fence(vma->obj->base.resv, &w->dma);
+
+err_unpin_vma:
+ if (err)
+ i915_vma_unpin(vma);
+out:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
+ return err;
+}
+
static int __i915_sw_fence_call
clear_pages_work_notify(struct i915_sw_fence *fence,
enum i915_sw_fence_notify state)
@@ -287,18 +338,9 @@ int i915_gem_schedule_fill_pages_blt(struct drm_i915_gem_object *obj,
dma_fence_init(&work->dma, &clear_pages_work_ops, &fence_lock, 0, 0);
i915_sw_fence_init(&work->wait, clear_pages_work_notify);
- i915_gem_object_lock(obj, NULL);
- err = i915_sw_fence_await_reservation(&work->wait,
- obj->base.resv, NULL,
- true, I915_FENCE_TIMEOUT,
- I915_FENCE_GFP);
- if (err < 0) {
+ err = pin_wait_clear_pages_work(work, ce);
+ if (err < 0)
dma_fence_set_error(&work->dma, err);
- } else {
- dma_resv_add_excl_fence(obj->base.resv, &work->dma);
- err = 0;
- }
- i915_gem_object_unlock(obj);
dma_fence_get(&work->dma);
i915_sw_fence_commit(&work->wait);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c b/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c
index 2b23e000bb57..ec9bd5c45e94 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object_blt.c
@@ -14,6 +14,7 @@
struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce,
struct i915_vma *vma,
+ struct i915_gem_ww_ctx *ww,
u32 value)
{
struct drm_i915_private *i915 = ce->vm->i915;
@@ -39,10 +40,24 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce,
goto out_pm;
}
+ err = i915_gem_object_lock(pool->obj, ww);
+ if (err)
+ goto out_put;
+
+ batch = i915_vma_instance(pool->obj, ce->vm, NULL);
+ if (IS_ERR(batch)) {
+ err = PTR_ERR(batch);
+ goto out_put;
+ }
+
+ err = i915_vma_pin_ww(batch, ww, 0, 0, PIN_USER);
+ if (unlikely(err))
+ goto out_put;
+
cmd = i915_gem_object_pin_map(pool->obj, I915_MAP_WC);
if (IS_ERR(cmd)) {
err = PTR_ERR(cmd);
- goto out_put;
+ goto out_unpin;
}
rem = vma->size;
@@ -82,19 +97,11 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce,
i915_gem_object_unpin_map(pool->obj);
- batch = i915_vma_instance(pool->obj, ce->vm, NULL);
- if (IS_ERR(batch)) {
- err = PTR_ERR(batch);
- goto out_put;
- }
-
- err = i915_vma_pin(batch, 0, 0, PIN_USER);
- if (unlikely(err))
- goto out_put;
-
batch->private = pool;
return batch;
+out_unpin:
+ i915_vma_unpin(batch);
out_put:
intel_engine_pool_put(pool);
out_pm:
@@ -106,11 +113,9 @@ int intel_emit_vma_mark_active(struct i915_vma *vma, struct i915_request *rq)
{
int err;
- i915_vma_lock(vma);
err = i915_request_await_object(rq, vma->obj, false);
if (err == 0)
err = i915_vma_move_to_active(vma, rq, 0);
- i915_vma_unlock(vma);
if (unlikely(err))
return err;
@@ -128,6 +133,7 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
struct intel_context *ce,
u32 value)
{
+ struct i915_gem_ww_ctx ww;
struct i915_request *rq;
struct i915_vma *batch;
struct i915_vma *vma;
@@ -137,23 +143,30 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
if (IS_ERR(vma))
return PTR_ERR(vma);
- err = i915_vma_pin(vma, 0, 0, PIN_USER);
- if (unlikely(err))
- return err;
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ err = i915_gem_object_lock(obj, &ww);
+ if (err)
+ goto out;
+
+ err = intel_context_pin_ww(ce, &ww);
+ if (err)
+ goto out;
- if (obj->cache_dirty & ~obj->cache_coherent) {
- i915_gem_object_lock(obj, NULL);
+ err = i915_vma_pin_ww(vma, &ww, 0, 0, PIN_USER);
+ if (err)
+ goto out_ctx;
+
+ if (obj->cache_dirty & ~obj->cache_coherent)
i915_gem_clflush_object(obj, 0);
- i915_gem_object_unlock(obj);
- }
- batch = intel_emit_vma_fill_blt(ce, vma, value);
+ batch = intel_emit_vma_fill_blt(ce, vma, &ww, value);
if (IS_ERR(batch)) {
err = PTR_ERR(batch);
- goto out_unpin;
+ goto out_vma;
}
- rq = intel_context_create_request(ce);
+ rq = i915_request_create(ce);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
goto out_batch;
@@ -173,11 +186,9 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
goto out_request;
}
- i915_vma_lock(vma);
err = i915_request_await_object(rq, vma->obj, true);
if (err == 0)
err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
- i915_vma_unlock(vma);
if (unlikely(err))
goto out_request;
@@ -191,12 +202,22 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
i915_request_add(rq);
out_batch:
intel_emit_vma_release(ce, batch);
-out_unpin:
+out_vma:
i915_vma_unpin(vma);
+out_ctx:
+ intel_context_unpin(ce);
+out:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
return err;
}
struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww,
struct i915_vma *src,
struct i915_vma *dst)
{
@@ -223,10 +244,24 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce,
goto out_pm;
}
+ err = i915_gem_object_lock(pool->obj, ww);
+ if (err)
+ goto out_put;
+
+ batch = i915_vma_instance(pool->obj, ce->vm, NULL);
+ if (IS_ERR(batch)) {
+ err = PTR_ERR(batch);
+ goto out_put;
+ }
+
+ err = i915_vma_pin_ww(batch, ww, 0, 0, PIN_USER);
+ if (unlikely(err))
+ goto out_put;
+
cmd = i915_gem_object_pin_map(pool->obj, I915_MAP_WC);
if (IS_ERR(cmd)) {
err = PTR_ERR(cmd);
- goto out_put;
+ goto out_unpin;
}
rem = src->size;
@@ -281,19 +316,11 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce,
i915_gem_object_unpin_map(pool->obj);
- batch = i915_vma_instance(pool->obj, ce->vm, NULL);
- if (IS_ERR(batch)) {
- err = PTR_ERR(batch);
- goto out_put;
- }
-
- err = i915_vma_pin(batch, 0, 0, PIN_USER);
- if (unlikely(err))
- goto out_put;
-
batch->private = pool;
return batch;
+out_unpin:
+ i915_vma_unpin(batch);
out_put:
intel_engine_pool_put(pool);
out_pm:
@@ -315,10 +342,9 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
struct drm_i915_gem_object *dst,
struct intel_context *ce)
{
- struct drm_gem_object *objs[] = { &src->base, &dst->base };
struct i915_address_space *vm = ce->vm;
struct i915_vma *vma[2], *batch;
- struct ww_acquire_ctx acquire;
+ struct i915_gem_ww_ctx ww;
struct i915_request *rq;
int err, i;
@@ -326,25 +352,35 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
if (IS_ERR(vma[0]))
return PTR_ERR(vma[0]);
- err = i915_vma_pin(vma[0], 0, 0, PIN_USER);
- if (unlikely(err))
- return err;
-
vma[1] = i915_vma_instance(dst, vm, NULL);
if (IS_ERR(vma[1]))
- goto out_unpin_src;
+ return PTR_ERR(vma);
- err = i915_vma_pin(vma[1], 0, 0, PIN_USER);
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ err = i915_gem_object_lock(src, &ww);
+ if (!err)
+ err = i915_gem_object_lock(dst, &ww);
+ if (!err)
+ err = intel_context_pin_ww(ce, &ww);
+ if (err)
+ goto out;
+
+ err = i915_vma_pin_ww(vma[0], &ww, 0, 0, PIN_USER);
+ if (err)
+ goto out_ctx;
+
+ err = i915_vma_pin_ww(vma[1], &ww, 0, 0, PIN_USER);
if (unlikely(err))
goto out_unpin_src;
- batch = intel_emit_vma_copy_blt(ce, vma[0], vma[1]);
+ batch = intel_emit_vma_copy_blt(ce, &ww, vma[0], vma[1]);
if (IS_ERR(batch)) {
err = PTR_ERR(batch);
goto out_unpin_dst;
}
- rq = intel_context_create_request(ce);
+ rq = i915_request_create(ce);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
goto out_batch;
@@ -354,14 +390,10 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
if (unlikely(err))
goto out_request;
- err = drm_gem_lock_reservations(objs, ARRAY_SIZE(objs), &acquire);
- if (unlikely(err))
- goto out_request;
-
for (i = 0; i < ARRAY_SIZE(vma); i++) {
err = move_to_gpu(vma[i], rq, i);
if (unlikely(err))
- goto out_unlock;
+ goto out_request;
}
for (i = 0; i < ARRAY_SIZE(vma); i++) {
@@ -369,20 +401,19 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
err = i915_vma_move_to_active(vma[i], rq, flags);
if (unlikely(err))
- goto out_unlock;
+ goto out_request;
}
if (rq->engine->emit_init_breadcrumb) {
err = rq->engine->emit_init_breadcrumb(rq);
if (unlikely(err))
- goto out_unlock;
+ goto out_request;
}
err = rq->engine->emit_bb_start(rq,
batch->node.start, batch->node.size,
0);
-out_unlock:
- drm_gem_unlock_reservations(objs, ARRAY_SIZE(objs), &acquire);
+
out_request:
if (unlikely(err))
i915_request_skip(rq, err);
@@ -394,6 +425,15 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
i915_vma_unpin(vma[1]);
out_unpin_src:
i915_vma_unpin(vma[0]);
+out_ctx:
+ intel_context_unpin(ce);
+out:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
return err;
}
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_blt.h b/drivers/gpu/drm/i915/gem/i915_gem_object_blt.h
index 243a43a87824..cdb537076752 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_object_blt.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_object_blt.h
@@ -14,12 +14,15 @@
#include "i915_vma.h"
struct drm_i915_gem_object;
+struct i915_gem_ww_ctx;
struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce,
struct i915_vma *vma,
+ struct i915_gem_ww_ctx *ww,
u32 value);
struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce,
+ struct i915_gem_ww_ctx *ww,
struct i915_vma *src,
struct i915_vma *dst);
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 15/19] drm/i915: Kill last user of intel_context_create_request outside of selftests
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (13 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 14/19] drm/i915: Convert i915_gem_object/client_blt.c to use ww locking as well Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 16/19] drm/i915: Convert i915_perf to ww locking as well Maarten Lankhorst
` (5 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
Instead of using intel_context_create_request(), use intel_context_pin()
and i915_create_request directly.
Now all those calls are gone outside of selftests. :)
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gt/intel_workarounds.c | 43 ++++++++++++++-------
1 file changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_workarounds.c b/drivers/gpu/drm/i915/gt/intel_workarounds.c
index ba86511f1ef9..5044b315cd45 100644
--- a/drivers/gpu/drm/i915/gt/intel_workarounds.c
+++ b/drivers/gpu/drm/i915/gt/intel_workarounds.c
@@ -1656,6 +1656,7 @@ static int engine_wa_list_verify(struct intel_context *ce,
const struct i915_wa *wa;
struct i915_request *rq;
struct i915_vma *vma;
+ struct i915_gem_ww_ctx ww;
unsigned int i;
u32 *results;
int err;
@@ -1668,29 +1669,34 @@ static int engine_wa_list_verify(struct intel_context *ce,
return PTR_ERR(vma);
intel_engine_pm_get(ce->engine);
- rq = intel_context_create_request(ce);
- intel_engine_pm_put(ce->engine);
+ i915_gem_ww_ctx_init(&ww, false);
+retry:
+ err = i915_gem_object_lock(vma->obj, &ww);
+ if (err == 0)
+ err = intel_context_pin_ww(ce, &ww);
+ if (err)
+ goto err_pm;
+
+ rq = i915_request_create(ce);
if (IS_ERR(rq)) {
err = PTR_ERR(rq);
- goto err_vma;
+ goto err_unpin;
}
- i915_vma_lock(vma);
err = i915_request_await_object(rq, vma->obj, true);
if (err == 0)
err = i915_vma_move_to_active(vma, rq, EXEC_OBJECT_WRITE);
- i915_vma_unlock(vma);
- if (err) {
- i915_request_add(rq);
- goto err_vma;
- }
-
- err = wa_list_srm(rq, wal, vma);
- if (err)
- goto err_vma;
+ if (err == 0)
+ err = wa_list_srm(rq, wal, vma);
i915_request_get(rq);
+ if (err)
+ i915_request_skip(rq, err);
i915_request_add(rq);
+
+ if (err)
+ goto err_rq;
+
if (i915_request_wait(rq, 0, HZ / 5) < 0) {
err = -ETIME;
goto err_rq;
@@ -1715,7 +1721,16 @@ static int engine_wa_list_verify(struct intel_context *ce,
err_rq:
i915_request_put(rq);
-err_vma:
+err_unpin:
+ intel_context_unpin(ce);
+err_pm:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
+ intel_engine_pm_put(ce->engine);
i915_vma_unpin(vma);
i915_vma_put(vma);
return err;
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 16/19] drm/i915: Convert i915_perf to ww locking as well
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (14 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 15/19] drm/i915: Kill last user of intel_context_create_request outside of selftests Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 17/19] drm/i915: Dirty hack to fix selftests locking inversion Maarten Lankhorst
` (4 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
We have the ordering of timeline->mutex vs resv_lock wrong,
convert the i915_pin_vma and intel_context_pin as well to
future-proof this.
We may need to do future changes to do this more transaction-like,
and only get down to a single i915_gem_ww_ctx, but for now this
should work.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/i915_perf.c | 57 +++++++++++++++++++++++---------
1 file changed, 42 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_perf.c b/drivers/gpu/drm/i915/i915_perf.c
index b5249ee5bda6..f3b1a8b9fc38 100644
--- a/drivers/gpu/drm/i915/i915_perf.c
+++ b/drivers/gpu/drm/i915/i915_perf.c
@@ -1239,24 +1239,39 @@ static struct intel_context *oa_pin_context(struct i915_perf_stream *stream)
struct i915_gem_engines_iter it;
struct i915_gem_context *ctx = stream->ctx;
struct intel_context *ce;
- int err;
+ struct i915_gem_ww_ctx ww;
+ int err = -ENODEV;
for_each_gem_engine(ce, i915_gem_context_lock_engines(ctx), it) {
if (ce->engine != stream->engine) /* first match! */
continue;
- /*
- * As the ID is the gtt offset of the context's vma we
- * pin the vma to ensure the ID remains fixed.
- */
- err = intel_context_pin(ce);
- if (err == 0) {
- stream->pinned_ctx = ce;
- break;
- }
+ err = 0;
+ break;
}
i915_gem_context_unlock_engines(ctx);
+ if (err)
+ return ERR_PTR(err);
+
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ /*
+ * As the ID is the gtt offset of the context's vma we
+ * pin the vma to ensure the ID remains fixed.
+ */
+ err = intel_context_pin_ww(ce, &ww);
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
+
+ if (err)
+ return ERR_PTR(err);
+
+ stream->pinned_ctx = ce;
return stream->pinned_ctx;
}
@@ -1972,15 +1987,22 @@ static int emit_oa_config(struct i915_perf_stream *stream,
{
struct i915_request *rq;
struct i915_vma *vma;
+ struct i915_gem_ww_ctx ww;
int err;
vma = get_oa_vma(stream, oa_config);
if (IS_ERR(vma))
return PTR_ERR(vma);
- err = i915_vma_pin(vma, 0, 0, PIN_GLOBAL | PIN_HIGH);
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ err = i915_gem_object_lock(vma->obj, &ww);
if (err)
- goto err_vma_put;
+ goto err;
+
+ err = i915_vma_pin_ww(vma, &ww, 0, 0, PIN_GLOBAL | PIN_HIGH);
+ if (err)
+ goto err;
intel_engine_pm_get(ce->engine);
rq = i915_request_create(ce);
@@ -1990,11 +2012,9 @@ static int emit_oa_config(struct i915_perf_stream *stream,
goto err_vma_unpin;
}
- i915_vma_lock(vma);
err = i915_request_await_object(rq, vma->obj, 0);
if (!err)
err = i915_vma_move_to_active(vma, rq, 0);
- i915_vma_unlock(vma);
if (err)
goto err_add_request;
@@ -2005,7 +2025,14 @@ static int emit_oa_config(struct i915_perf_stream *stream,
i915_request_add(rq);
err_vma_unpin:
i915_vma_unpin(vma);
-err_vma_put:
+err:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+
+ i915_gem_ww_ctx_fini(&ww);
i915_vma_put(vma);
return err;
}
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 17/19] drm/i915: Dirty hack to fix selftests locking inversion
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (15 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 16/19] drm/i915: Convert i915_perf to ww locking as well Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 18/19] drm/i915/selftests: Fix locking inversion in lrc selftest Maarten Lankhorst
` (3 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
Some i915 selftests still use i915_vma_lock() as inner lock, and
intel_context_create_request() intel_timeline->mutex as outer lock.
Fortunately for selftests this is not an issue, they should be fixed
but we can move ahead and cleanify lockdep now.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index f64b1020f686..87f9f9e61916 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -446,6 +446,18 @@ struct i915_request *intel_context_create_request(struct intel_context *ce)
rq = i915_request_create(ce);
intel_context_unpin(ce);
+ if (IS_ERR(rq))
+ return rq;
+
+ /*
+ * timeline->mutex should be the inner lock, but is used as outer lock.
+ * Hack around this to shut up lockdep in selftests..
+ */
+ lockdep_unpin_lock(&ce->timeline->mutex, rq->cookie);
+ mutex_release(&ce->timeline->mutex.dep_map, _RET_IP_);
+ mutex_acquire(&ce->timeline->mutex.dep_map, SINGLE_DEPTH_NESTING, 0, _RET_IP_);
+ rq->cookie = lockdep_pin_lock(&ce->timeline->mutex);
+
return rq;
}
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 18/19] drm/i915/selftests: Fix locking inversion in lrc selftest.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (16 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 17/19] drm/i915: Dirty hack to fix selftests locking inversion Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 10:30 ` [Intel-gfx] [PATCH 19/19] drm/i915: Use ww pinning for intel_context_create_request() Maarten Lankhorst
` (2 subsequent siblings)
20 siblings, 0 replies; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
This function does not use intel_context_create_request, so it has
to use the same locking order as normal code. This is required to
shut up lockdep in selftests.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gt/selftest_lrc.c | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/selftest_lrc.c b/drivers/gpu/drm/i915/gt/selftest_lrc.c
index 3a74a54739a8..d05fcbd2c337 100644
--- a/drivers/gpu/drm/i915/gt/selftest_lrc.c
+++ b/drivers/gpu/drm/i915/gt/selftest_lrc.c
@@ -4187,6 +4187,7 @@ static int __live_lrc_state(struct intel_engine_cs *engine,
{
struct intel_context *ce;
struct i915_request *rq;
+ struct i915_gem_ww_ctx ww;
enum {
RING_START_IDX = 0,
RING_TAIL_IDX,
@@ -4201,7 +4202,11 @@ static int __live_lrc_state(struct intel_engine_cs *engine,
if (IS_ERR(ce))
return PTR_ERR(ce);
- err = intel_context_pin(ce);
+ i915_gem_ww_ctx_init(&ww, false);
+retry:
+ err = i915_gem_object_lock(scratch->obj, &ww);
+ if (!err)
+ err = intel_context_pin_ww(ce, &ww);
if (err)
goto err_put;
@@ -4230,11 +4235,9 @@ static int __live_lrc_state(struct intel_engine_cs *engine,
*cs++ = i915_ggtt_offset(scratch) + RING_TAIL_IDX * sizeof(u32);
*cs++ = 0;
- i915_vma_lock(scratch);
err = i915_request_await_object(rq, scratch->obj, true);
if (!err)
err = i915_vma_move_to_active(scratch, rq, EXEC_OBJECT_WRITE);
- i915_vma_unlock(scratch);
i915_request_get(rq);
i915_request_add(rq);
@@ -4271,6 +4274,12 @@ static int __live_lrc_state(struct intel_engine_cs *engine,
err_unpin:
intel_context_unpin(ce);
err_put:
+ if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ }
+ i915_gem_ww_ctx_fini(&ww);
intel_context_put(ce);
return err;
}
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* [Intel-gfx] [PATCH 19/19] drm/i915: Use ww pinning for intel_context_create_request()
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (17 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 18/19] drm/i915/selftests: Fix locking inversion in lrc selftest Maarten Lankhorst
@ 2020-02-14 10:30 ` Maarten Lankhorst
2020-02-14 20:01 ` Ruhl, Michael J
2020-02-14 11:16 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915/gem: Implement parallel execbuf submission Patchwork
2020-02-14 11:46 ` [Intel-gfx] ✗ Fi.CI.BAT: failure " Patchwork
20 siblings, 1 reply; 24+ messages in thread
From: Maarten Lankhorst @ 2020-02-14 10:30 UTC (permalink / raw)
To: intel-gfx
We want to get rid of intel_context_pin(), convert
intel_context_create_request() first. :)
Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 87f9f9e61916..44868b10be0a 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -436,15 +436,25 @@ int intel_context_prepare_remote_request(struct intel_context *ce,
struct i915_request *intel_context_create_request(struct intel_context *ce)
{
+ struct i915_gem_ww_ctx ww;
struct i915_request *rq;
int err;
- err = intel_context_pin(ce);
- if (unlikely(err))
- return ERR_PTR(err);
+ i915_gem_ww_ctx_init(&ww, true);
+retry:
+ err = intel_context_pin_ww(ce, &ww);
+ if (!err) {
+ rq = i915_request_create(ce);
+ intel_context_unpin(ce);
+ } else if (err == -EDEADLK) {
+ err = i915_gem_ww_ctx_backoff(&ww);
+ if (!err)
+ goto retry;
+ } else {
+ rq = ERR_PTR(err);
+ }
- rq = i915_request_create(ce);
- intel_context_unpin(ce);
+ i915_gem_ww_ctx_fini(&ww);
if (IS_ERR(rq))
return rq;
--
2.25.0.24.g3f081b084b0
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [Intel-gfx] [PATCH 19/19] drm/i915: Use ww pinning for intel_context_create_request()
2020-02-14 10:30 ` [Intel-gfx] [PATCH 19/19] drm/i915: Use ww pinning for intel_context_create_request() Maarten Lankhorst
@ 2020-02-14 20:01 ` Ruhl, Michael J
0 siblings, 0 replies; 24+ messages in thread
From: Ruhl, Michael J @ 2020-02-14 20:01 UTC (permalink / raw)
To: Maarten Lankhorst, intel-gfx
>-----Original Message-----
>From: Intel-gfx <intel-gfx-bounces@lists.freedesktop.org> On Behalf Of
>Maarten Lankhorst
>Sent: Friday, February 14, 2020 5:31 AM
>To: intel-gfx@lists.freedesktop.org
>Subject: [Intel-gfx] [PATCH 19/19] drm/i915: Use ww pinning for
>intel_context_create_request()
>
>We want to get rid of intel_context_pin(), convert
>intel_context_create_request() first. :)
>
>Signed-off-by: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
>---
> drivers/gpu/drm/i915/gt/intel_context.c | 20 +++++++++++++++-----
> 1 file changed, 15 insertions(+), 5 deletions(-)
>
>diff --git a/drivers/gpu/drm/i915/gt/intel_context.c
>b/drivers/gpu/drm/i915/gt/intel_context.c
>index 87f9f9e61916..44868b10be0a 100644
>--- a/drivers/gpu/drm/i915/gt/intel_context.c
>+++ b/drivers/gpu/drm/i915/gt/intel_context.c
>@@ -436,15 +436,25 @@ int intel_context_prepare_remote_request(struct
>intel_context *ce,
>
> struct i915_request *intel_context_create_request(struct intel_context *ce)
> {
>+ struct i915_gem_ww_ctx ww;
> struct i915_request *rq;
> int err;
>
>- err = intel_context_pin(ce);
>- if (unlikely(err))
>- return ERR_PTR(err);
>+ i915_gem_ww_ctx_init(&ww, true);
>+retry:
>+ err = intel_context_pin_ww(ce, &ww);
>+ if (!err) {
>+ rq = i915_request_create(ce);
>+ intel_context_unpin(ce);
>+ } else if (err == -EDEADLK) {
>+ err = i915_gem_ww_ctx_backoff(&ww);
>+ if (!err)
>+ goto retry;
>+ } else {
>+ rq = ERR_PTR(err);
>+ }
If you have the pathological path:
err = intel_context_pin_ww(cd, &&))
else if (err == -EDEADLK)
err = i915_gem_ww_ctx_backoff(&ww) ; (where err != 0)
It appears that you can get to IS_ERR(rq) with rq being garbage from the
stack.
Do you need to preset rq, or set it on:
if (!err)
goto retry;
else
rq = ERR_PTR(err);
?
Thanks,
Mike
>
>- rq = i915_request_create(ce);
>- intel_context_unpin(ce);
>+ i915_gem_ww_ctx_fini(&ww);
>
> if (IS_ERR(rq))
> return rq;
>--
>2.25.0.24.g3f081b084b0
>
>_______________________________________________
>Intel-gfx mailing list
>Intel-gfx@lists.freedesktop.org
>https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915/gem: Implement parallel execbuf submission.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (18 preceding siblings ...)
2020-02-14 10:30 ` [Intel-gfx] [PATCH 19/19] drm/i915: Use ww pinning for intel_context_create_request() Maarten Lankhorst
@ 2020-02-14 11:16 ` Patchwork
2020-02-14 11:46 ` [Intel-gfx] ✗ Fi.CI.BAT: failure " Patchwork
20 siblings, 0 replies; 24+ messages in thread
From: Patchwork @ 2020-02-14 11:16 UTC (permalink / raw)
To: Maarten Lankhorst; +Cc: intel-gfx
== Series Details ==
Series: drm/i915/gem: Implement parallel execbuf submission.
URL : https://patchwork.freedesktop.org/series/73456/
State : warning
== Summary ==
$ dim checkpatch origin/drm-tip
45e26eb55b49 drm/i915: Drop inspection of execbuf flags during evict
77137cc44da0 drm/i915/gem: Extract transient execbuf flags from i915_vma
1c9c54ea3f9f drm/i915: Separate lookup and pinning in execbuf.
-:153: CHECK:BRACES: Blank lines aren't necessary before a close brace '}'
#153: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:857:
+ }
total: 0 errors, 0 warnings, 1 checks, 190 lines checked
74411de36fec drm/i915: Add an implementation for i915_gem_ww_ctx locking, v2.
-:506: WARNING:LONG_LINE: line over 100 characters
#506: FILE: drivers/gpu/drm/i915/i915_gem.c:1340:
+ while ((obj = list_first_entry_or_null(&ww->obj_list, struct drm_i915_gem_object, obj_link))) {
total: 0 errors, 1 warnings, 0 checks, 481 lines checked
4fb224307417 drm/i915: Remove locking from i915_gem_object_prepare_read/write
6d289dfa8e94 drm/i915: Parse command buffer earlier in eb_relocate(slow)
94e03631a69b drm/i915: Use per object locking in execbuf on top of struct_mutex, v3.
-:224: CHECK:BRACES: braces {} should be used on all arms of this statement
#224: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:1831:
+ if (err == -EDEADLK) {
[...]
+ else if (err)
[...]
-:232: ERROR:ELSE_AFTER_BRACE: else should follow close brace '}'
#232: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:1839:
+ }
+ else if (err)
total: 1 errors, 0 warnings, 1 checks, 326 lines checked
15b80a481667 drm/i915: Use ww locking in intel_renderstate.
-:10: WARNING:COMMIT_LOG_LONG_LINE: Possible unwrapped commit description (prefer a maximum 75 chars per line)
#10:
Convert to using ww-waiting, and make sure we always pin intel_context_state,
total: 0 errors, 1 warnings, 0 checks, 202 lines checked
656138053d87 drm/i915: Add ww context handling to context_barrier_task
-:19: WARNING:LONG_LINE: line over 100 characters
#19: FILE: drivers/gpu/drm/i915/gem/i915_gem_context.c:978:
+ int (*pin)(struct intel_context *ce, struct i915_gem_ww_ctx *ww, void *data),
total: 0 errors, 1 warnings, 0 checks, 146 lines checked
75b5a2f7f6c3 drm/i915: Nuke arguments to eb_pin_engine
39baf3b54346 drm/i915: Pin engine before pinning all objects, v2.
b3a660cd040f drm/i915: Rework intel_context pinning to do everything outside of pin_mutex
-:125: CHECK:LINE_SPACING: Please don't use multiple blank lines
#125: FILE: drivers/gpu/drm/i915/gt/intel_context.c:171:
+
+
-:330: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#330: FILE: drivers/gpu/drm/i915/gt/intel_lrc.c:2942:
+ *vaddr = i915_gem_object_pin_map(ce->state->obj,
+ i915_coherent_map_type(ce->engine->i915) |
total: 0 errors, 0 warnings, 2 checks, 436 lines checked
4782909f0467 drm/i915: Make sure execbuffer always passes ww state to i915_vma_pin.
-:80: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis
#80: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:581:
+ err = i915_vma_pin_ww(vma, &eb->ww,
entry->pad_to_size, entry->alignment,
-:185: WARNING:BLOCK_COMMENT_STYLE: Block comments use a trailing */ on a separate line
#185: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:2166:
+ * hsw should have this fixed, but bdw mucks it up again. */
total: 0 errors, 1 warnings, 1 checks, 811 lines checked
5819fba41820 drm/i915: Convert i915_gem_object/client_blt.c to use ww locking as well
4f5b737716e8 drm/i915: Kill last user of intel_context_create_request outside of selftests
e1a5ab78d885 drm/i915: Convert i915_perf to ww locking as well
80cc1c0dea35 drm/i915: Dirty hack to fix selftests locking inversion
e091c73e4158 drm/i915/selftests: Fix locking inversion in lrc selftest.
6e5d190d8121 drm/i915: Use ww pinning for intel_context_create_request()
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 24+ messages in thread* [Intel-gfx] ✗ Fi.CI.BAT: failure for drm/i915/gem: Implement parallel execbuf submission.
2020-02-14 10:30 [Intel-gfx] [PATCH 00/19] drm/i915/gem: Implement parallel execbuf submission Maarten Lankhorst
` (19 preceding siblings ...)
2020-02-14 11:16 ` [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for drm/i915/gem: Implement parallel execbuf submission Patchwork
@ 2020-02-14 11:46 ` Patchwork
20 siblings, 0 replies; 24+ messages in thread
From: Patchwork @ 2020-02-14 11:46 UTC (permalink / raw)
To: Maarten Lankhorst; +Cc: intel-gfx
== Series Details ==
Series: drm/i915/gem: Implement parallel execbuf submission.
URL : https://patchwork.freedesktop.org/series/73456/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_7938 -> Patchwork_16567
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with Patchwork_16567 absolutely need to be
verified manually.
If you think the reported changes have nothing to do with the changes
introduced in Patchwork_16567, please notify your bug team to allow them
to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/index.html
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_16567:
### IGT changes ###
#### Possible regressions ####
* igt@gem_close_race@basic-threads:
- fi-icl-guc: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-icl-guc/igt@gem_close_race@basic-threads.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-icl-guc/igt@gem_close_race@basic-threads.html
- fi-bdw-5557u: [PASS][3] -> [INCOMPLETE][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-bdw-5557u/igt@gem_close_race@basic-threads.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-bdw-5557u/igt@gem_close_race@basic-threads.html
- fi-kbl-x1275: [PASS][5] -> [INCOMPLETE][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-kbl-x1275/igt@gem_close_race@basic-threads.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-kbl-x1275/igt@gem_close_race@basic-threads.html
* igt@gem_exec_parallel@fds:
- fi-cfl-8109u: [PASS][7] -> [INCOMPLETE][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-cfl-8109u/igt@gem_exec_parallel@fds.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-cfl-8109u/igt@gem_exec_parallel@fds.html
* igt@gem_sync@basic-each:
- fi-kbl-7500u: [PASS][9] -> [INCOMPLETE][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-kbl-7500u/igt@gem_sync@basic-each.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-kbl-7500u/igt@gem_sync@basic-each.html
* igt@i915_selftest@live_gt_engines:
- fi-snb-2520m: [PASS][11] -> [FAIL][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-snb-2520m/igt@i915_selftest@live_gt_engines.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-snb-2520m/igt@i915_selftest@live_gt_engines.html
* igt@i915_selftest@live_requests:
- fi-hsw-peppy: [PASS][13] -> [DMESG-WARN][14]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-hsw-peppy/igt@i915_selftest@live_requests.html
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-hsw-peppy/igt@i915_selftest@live_requests.html
- fi-skl-6700k2: [PASS][15] -> [DMESG-WARN][16]
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-skl-6700k2/igt@i915_selftest@live_requests.html
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-skl-6700k2/igt@i915_selftest@live_requests.html
- fi-hsw-4770r: [PASS][17] -> [DMESG-WARN][18]
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-hsw-4770r/igt@i915_selftest@live_requests.html
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-hsw-4770r/igt@i915_selftest@live_requests.html
- fi-skl-lmem: [PASS][19] -> [DMESG-WARN][20]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-skl-lmem/igt@i915_selftest@live_requests.html
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-skl-lmem/igt@i915_selftest@live_requests.html
- fi-ivb-3770: [PASS][21] -> [DMESG-WARN][22]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-ivb-3770/igt@i915_selftest@live_requests.html
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-ivb-3770/igt@i915_selftest@live_requests.html
- fi-skl-6600u: [PASS][23] -> [DMESG-WARN][24]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-skl-6600u/igt@i915_selftest@live_requests.html
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-skl-6600u/igt@i915_selftest@live_requests.html
- fi-cml-s: [PASS][25] -> [DMESG-WARN][26]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-cml-s/igt@i915_selftest@live_requests.html
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-cml-s/igt@i915_selftest@live_requests.html
- fi-hsw-4770: [PASS][27] -> [DMESG-WARN][28]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-hsw-4770/igt@i915_selftest@live_requests.html
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-hsw-4770/igt@i915_selftest@live_requests.html
- fi-kbl-guc: [PASS][29] -> [DMESG-WARN][30]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-kbl-guc/igt@i915_selftest@live_requests.html
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-kbl-guc/igt@i915_selftest@live_requests.html
- fi-glk-dsi: [PASS][31] -> [DMESG-WARN][32]
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-glk-dsi/igt@i915_selftest@live_requests.html
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-glk-dsi/igt@i915_selftest@live_requests.html
- fi-cfl-guc: [PASS][33] -> [DMESG-WARN][34]
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-cfl-guc/igt@i915_selftest@live_requests.html
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-cfl-guc/igt@i915_selftest@live_requests.html
- fi-cfl-8700k: [PASS][35] -> [DMESG-WARN][36]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-cfl-8700k/igt@i915_selftest@live_requests.html
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-cfl-8700k/igt@i915_selftest@live_requests.html
- fi-kbl-r: [PASS][37] -> [DMESG-WARN][38]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-kbl-r/igt@i915_selftest@live_requests.html
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-kbl-r/igt@i915_selftest@live_requests.html
- fi-skl-guc: [PASS][39] -> [DMESG-WARN][40]
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-skl-guc/igt@i915_selftest@live_requests.html
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-skl-guc/igt@i915_selftest@live_requests.html
- fi-bwr-2160: [PASS][41] -> [DMESG-WARN][42]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-bwr-2160/igt@i915_selftest@live_requests.html
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-bwr-2160/igt@i915_selftest@live_requests.html
* igt@runner@aborted:
- fi-ilk-650: NOTRUN -> [FAIL][43]
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-ilk-650/igt@runner@aborted.html
- fi-pnv-d510: NOTRUN -> [FAIL][44]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-pnv-d510/igt@runner@aborted.html
- fi-cfl-8109u: NOTRUN -> [FAIL][45]
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-cfl-8109u/igt@runner@aborted.html
- fi-hsw-peppy: NOTRUN -> [FAIL][46]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-hsw-peppy/igt@runner@aborted.html
- fi-gdg-551: NOTRUN -> [FAIL][47]
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-gdg-551/igt@runner@aborted.html
- fi-snb-2520m: NOTRUN -> [FAIL][48]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-snb-2520m/igt@runner@aborted.html
- fi-hsw-4770: NOTRUN -> [FAIL][49]
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-hsw-4770/igt@runner@aborted.html
- fi-kbl-7500u: NOTRUN -> [FAIL][50]
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-kbl-7500u/igt@runner@aborted.html
- fi-ivb-3770: NOTRUN -> [FAIL][51]
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-ivb-3770/igt@runner@aborted.html
- fi-bxt-dsi: NOTRUN -> [FAIL][52]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-bxt-dsi/igt@runner@aborted.html
- fi-blb-e6850: NOTRUN -> [FAIL][53]
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-blb-e6850/igt@runner@aborted.html
- fi-cfl-8700k: NOTRUN -> [FAIL][54]
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-cfl-8700k/igt@runner@aborted.html
- fi-hsw-4770r: NOTRUN -> [FAIL][55]
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-hsw-4770r/igt@runner@aborted.html
- fi-apl-guc: NOTRUN -> [FAIL][56]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-apl-guc/igt@runner@aborted.html
- fi-kbl-r: NOTRUN -> [FAIL][57]
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-kbl-r/igt@runner@aborted.html
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses.
They do not affect the overall result.
* igt@i915_selftest@live_requests:
- {fi-tgl-dsi}: [PASS][58] -> [DMESG-WARN][59]
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-tgl-dsi/igt@i915_selftest@live_requests.html
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-tgl-dsi/igt@i915_selftest@live_requests.html
- {fi-tgl-u}: [PASS][60] -> [DMESG-WARN][61]
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-tgl-u/igt@i915_selftest@live_requests.html
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-tgl-u/igt@i915_selftest@live_requests.html
* {igt@kms_busy@basic@flip}:
- fi-bwr-2160: [PASS][62] -> [FAIL][63]
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-bwr-2160/igt@kms_busy@basic@flip.html
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-bwr-2160/igt@kms_busy@basic@flip.html
- fi-pnv-d510: [PASS][64] -> [FAIL][65]
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-pnv-d510/igt@kms_busy@basic@flip.html
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-pnv-d510/igt@kms_busy@basic@flip.html
* igt@runner@aborted:
- {fi-tgl-dsi}: NOTRUN -> [FAIL][66]
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-tgl-dsi/igt@runner@aborted.html
- {fi-tgl-u}: NOTRUN -> [FAIL][67]
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-tgl-u/igt@runner@aborted.html
Known issues
------------
Here are the changes found in Patchwork_16567 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_close_race@basic-threads:
- fi-hsw-peppy: [PASS][68] -> [TIMEOUT][69] ([fdo#112271] / [i915#1084])
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-hsw-peppy/igt@gem_close_race@basic-threads.html
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-hsw-peppy/igt@gem_close_race@basic-threads.html
* igt@gem_exec_parallel@basic:
- fi-bxt-dsi: [PASS][70] -> [INCOMPLETE][71] ([fdo#103927])
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-bxt-dsi/igt@gem_exec_parallel@basic.html
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-bxt-dsi/igt@gem_exec_parallel@basic.html
* igt@gem_exec_parallel@fds:
- fi-cml-u2: [PASS][72] -> [INCOMPLETE][73] ([i915#283])
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-cml-u2/igt@gem_exec_parallel@fds.html
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-cml-u2/igt@gem_exec_parallel@fds.html
- fi-apl-guc: [PASS][74] -> [INCOMPLETE][75] ([fdo#103927])
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-apl-guc/igt@gem_exec_parallel@fds.html
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-apl-guc/igt@gem_exec_parallel@fds.html
* igt@gem_exec_suspend@basic-s3:
- fi-skl-6770hq: [PASS][76] -> [INCOMPLETE][77] ([i915#69])
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-skl-6770hq/igt@gem_exec_suspend@basic-s3.html
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-skl-6770hq/igt@gem_exec_suspend@basic-s3.html
* igt@i915_selftest@live_blt:
- fi-bsw-nick: [PASS][78] -> [INCOMPLETE][79] ([i915#392])
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-bsw-nick/igt@i915_selftest@live_blt.html
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-bsw-nick/igt@i915_selftest@live_blt.html
* igt@i915_selftest@live_gt_lrc:
- fi-snb-2600: [PASS][80] -> [INCOMPLETE][81] ([i915#82])
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-snb-2600/igt@i915_selftest@live_gt_lrc.html
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-snb-2600/igt@i915_selftest@live_gt_lrc.html
* igt@i915_selftest@live_requests:
- fi-icl-dsi: [PASS][82] -> [DMESG-WARN][83] ([fdo#109644] / [fdo#110464])
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-icl-dsi/igt@i915_selftest@live_requests.html
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-icl-dsi/igt@i915_selftest@live_requests.html
- fi-blb-e6850: [PASS][84] -> [DMESG-WARN][85] ([i915#478])
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-blb-e6850/igt@i915_selftest@live_requests.html
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-blb-e6850/igt@i915_selftest@live_requests.html
- fi-icl-y: [PASS][86] -> [DMESG-WARN][87] ([fdo#109644] / [fdo#110464])
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-icl-y/igt@i915_selftest@live_requests.html
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-icl-y/igt@i915_selftest@live_requests.html
- fi-gdg-551: [PASS][88] -> [DMESG-WARN][89] ([i915#478])
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-gdg-551/igt@i915_selftest@live_requests.html
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-gdg-551/igt@i915_selftest@live_requests.html
- fi-icl-u3: [PASS][90] -> [DMESG-WARN][91] ([fdo#109644] / [fdo#110464])
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-icl-u3/igt@i915_selftest@live_requests.html
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-icl-u3/igt@i915_selftest@live_requests.html
- fi-ilk-650: [PASS][92] -> [DMESG-WARN][93] ([i915#478])
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-ilk-650/igt@i915_selftest@live_requests.html
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-ilk-650/igt@i915_selftest@live_requests.html
- fi-pnv-d510: [PASS][94] -> [DMESG-WARN][95] ([i915#478])
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-pnv-d510/igt@i915_selftest@live_requests.html
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-pnv-d510/igt@i915_selftest@live_requests.html
- fi-snb-2520m: [PASS][96] -> [DMESG-WARN][97] ([i915#478])
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-snb-2520m/igt@i915_selftest@live_requests.html
[97]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-snb-2520m/igt@i915_selftest@live_requests.html
- fi-icl-u2: [PASS][98] -> [DMESG-WARN][99] ([fdo#109644] / [fdo#110464])
[98]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7938/fi-icl-u2/igt@i915_selftest@live_requests.html
[99]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/fi-icl-u2/igt@i915_selftest@live_requests.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#103927]: https://bugs.freedesktop.org/show_bug.cgi?id=103927
[fdo#109644]: https://bugs.freedesktop.org/show_bug.cgi?id=109644
[fdo#110464]: https://bugs.freedesktop.org/show_bug.cgi?id=110464
[fdo#112271]: https://bugs.freedesktop.org/show_bug.cgi?id=112271
[i915#1084]: https://gitlab.freedesktop.org/drm/intel/issues/1084
[i915#283]: https://gitlab.freedesktop.org/drm/intel/issues/283
[i915#392]: https://gitlab.freedesktop.org/drm/intel/issues/392
[i915#478]: https://gitlab.freedesktop.org/drm/intel/issues/478
[i915#69]: https://gitlab.freedesktop.org/drm/intel/issues/69
[i915#82]: https://gitlab.freedesktop.org/drm/intel/issues/82
Participating hosts (52 -> 46)
------------------------------
Missing (6): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-byt-clapper fi-bdw-samus
Build changes
-------------
* CI: CI-20190529 -> None
* Linux: CI_DRM_7938 -> Patchwork_16567
CI-20190529: 20190529
CI_DRM_7938: 9fda6807b50ceb40ef01b055f6428db8965e3d06 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_5441: 534ca091fe4ffed916752165bc5becd7ff56cd84 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
Patchwork_16567: 6e5d190d8121d03590842d2f155906c675789e0c @ git://anongit.freedesktop.org/gfx-ci/linux
== Linux commits ==
6e5d190d8121 drm/i915: Use ww pinning for intel_context_create_request()
e091c73e4158 drm/i915/selftests: Fix locking inversion in lrc selftest.
80cc1c0dea35 drm/i915: Dirty hack to fix selftests locking inversion
e1a5ab78d885 drm/i915: Convert i915_perf to ww locking as well
4f5b737716e8 drm/i915: Kill last user of intel_context_create_request outside of selftests
5819fba41820 drm/i915: Convert i915_gem_object/client_blt.c to use ww locking as well
4782909f0467 drm/i915: Make sure execbuffer always passes ww state to i915_vma_pin.
b3a660cd040f drm/i915: Rework intel_context pinning to do everything outside of pin_mutex
39baf3b54346 drm/i915: Pin engine before pinning all objects, v2.
75b5a2f7f6c3 drm/i915: Nuke arguments to eb_pin_engine
656138053d87 drm/i915: Add ww context handling to context_barrier_task
15b80a481667 drm/i915: Use ww locking in intel_renderstate.
94e03631a69b drm/i915: Use per object locking in execbuf on top of struct_mutex, v3.
6d289dfa8e94 drm/i915: Parse command buffer earlier in eb_relocate(slow)
4fb224307417 drm/i915: Remove locking from i915_gem_object_prepare_read/write
74411de36fec drm/i915: Add an implementation for i915_gem_ww_ctx locking, v2.
1c9c54ea3f9f drm/i915: Separate lookup and pinning in execbuf.
77137cc44da0 drm/i915/gem: Extract transient execbuf flags from i915_vma
45e26eb55b49 drm/i915: Drop inspection of execbuf flags during evict
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_16567/index.html
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 24+ messages in thread