[PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[Chris Wilson]

Avoid requiring struct_mutex for exclusive access to the temporary
dfs_link inside the i915_dependency as not all callers may want to touch
struct_mutex. So rather than force them to take a highly contended
lock, introduce a local lock for the execlists schedule operation.

Reported-by: David Weinehall <david.weinehall@linux.intel.com>
Fixes: 9a151987d709 ("drm/i915: Add execution priority boosting for mmioflips")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: David Weinehall <david.weinehall@linux.intel.com>
---
 drivers/gpu/drm/i915/intel_lrc.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index e23b6a2600fb..10e59ff0d8f1 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -694,6 +694,7 @@ pt_lock_engine(struct i915_priotree *pt, struct intel_engine_cs *locked)
 
 static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
 {
+	static DEFINE_MUTEX(lock);
 	struct intel_engine_cs *engine = NULL;
 	struct i915_dependency *dep, *p;
 	struct i915_dependency stack;
@@ -702,8 +703,8 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
 	if (prio <= READ_ONCE(request->priotree.priority))
 		return;
 
-	/* Need BKL in order to use the temporary link inside i915_dependency */
-	lockdep_assert_held(&request->i915->drm.struct_mutex);
+	/* Need global lock to use the temporary link inside i915_dependency */
+	mutex_lock(&lock);
 
 	stack.signaler = &request->priotree;
 	list_add(&stack.dfs_link, &dfs);
@@ -770,6 +771,8 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
 	if (engine)
 		spin_unlock_irq(&engine->timeline->lock);
 
+	mutex_unlock(&lock);
+
 	/* XXX Do we need to preempt to make room for us and our deps? */
 }
 
-- 
2.10.2

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[Tvrtko Ursulin]

On 16/11/2016 15:27, Chris Wilson wrote:
> Avoid requiring struct_mutex for exclusive access to the temporary
> dfs_link inside the i915_dependency as not all callers may want to touch
> struct_mutex. So rather than force them to take a highly contended
> lock, introduce a local lock for the execlists schedule operation.
>
> Reported-by: David Weinehall <david.weinehall@linux.intel.com>
> Fixes: 9a151987d709 ("drm/i915: Add execution priority boosting for mmioflips")

Grumble grumble, sloppy review. :I

> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: David Weinehall <david.weinehall@linux.intel.com>
> ---
>  drivers/gpu/drm/i915/intel_lrc.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> index e23b6a2600fb..10e59ff0d8f1 100644
> --- a/drivers/gpu/drm/i915/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/intel_lrc.c
> @@ -694,6 +694,7 @@ pt_lock_engine(struct i915_priotree *pt, struct intel_engine_cs *locked)
>
>  static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
>  {
> +	static DEFINE_MUTEX(lock);

Good enough for one GPU. :) Consider improving in the future as it is 
not in the spirit of the driver.

>  	struct intel_engine_cs *engine = NULL;
>  	struct i915_dependency *dep, *p;
>  	struct i915_dependency stack;
> @@ -702,8 +703,8 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
>  	if (prio <= READ_ONCE(request->priotree.priority))
>  		return;
>
> -	/* Need BKL in order to use the temporary link inside i915_dependency */
> -	lockdep_assert_held(&request->i915->drm.struct_mutex);
> +	/* Need global lock to use the temporary link inside i915_dependency */
> +	mutex_lock(&lock);
>
>  	stack.signaler = &request->priotree;
>  	list_add(&stack.dfs_link, &dfs);
> @@ -770,6 +771,8 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
>  	if (engine)
>  		spin_unlock_irq(&engine->timeline->lock);
>
> +	mutex_unlock(&lock);
> +
>  	/* XXX Do we need to preempt to make room for us and our deps? */
>  }
>
>

Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[Chris Wilson]

On Wed, Nov 16, 2016 at 03:54:23PM +0000, Tvrtko Ursulin wrote:
> 
> On 16/11/2016 15:27, Chris Wilson wrote:
> >Avoid requiring struct_mutex for exclusive access to the temporary
> >dfs_link inside the i915_dependency as not all callers may want to touch
> >struct_mutex. So rather than force them to take a highly contended
> >lock, introduce a local lock for the execlists schedule operation.
> >
> >Reported-by: David Weinehall <david.weinehall@linux.intel.com>
> >Fixes: 9a151987d709 ("drm/i915: Add execution priority boosting for mmioflips")
> 
> Grumble grumble, sloppy review. :I

Too busy living the good life with working atomic flips.
 
> >Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> >Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> >Cc: David Weinehall <david.weinehall@linux.intel.com>
> >---
> > drivers/gpu/drm/i915/intel_lrc.c | 7 +++++--
> > 1 file changed, 5 insertions(+), 2 deletions(-)
> >
> >diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> >index e23b6a2600fb..10e59ff0d8f1 100644
> >--- a/drivers/gpu/drm/i915/intel_lrc.c
> >+++ b/drivers/gpu/drm/i915/intel_lrc.c
> >@@ -694,6 +694,7 @@ pt_lock_engine(struct i915_priotree *pt, struct intel_engine_cs *locked)
> >
> > static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
> > {
> >+	static DEFINE_MUTEX(lock);
> 
> Good enough for one GPU. :) Consider improving in the future as it
> is not in the spirit of the driver.

Actually... Being able to PI across multiple GPUs is part of the dream.
In that case, it does need to be a global lock - just a bit iffy on
getting dependency tracking into a common layer.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✗ Fi.CI.BAT: warning for drm/i915/execlists: Use a local lock for dfs_link access

[Patchwork]

== Series Details ==

Series: drm/i915/execlists: Use a local lock for dfs_link access
URL   : https://patchwork.freedesktop.org/series/15425/
State : warning

== Summary ==

Series 15425v1 drm/i915/execlists: Use a local lock for dfs_link access
https://patchwork.freedesktop.org/api/1.0/series/15425/revisions/1/mbox/

Test drv_module_reload_basic:
                pass       -> DMESG-WARN (fi-skl-6770hq)

fi-bdw-5557u     total:244  pass:229  dwarn:0   dfail:0   fail:0   skip:15 
fi-bsw-n3050     total:244  pass:204  dwarn:0   dfail:0   fail:0   skip:40 
fi-bxt-t5700     total:244  pass:216  dwarn:0   dfail:0   fail:0   skip:28 
fi-byt-j1900     total:244  pass:216  dwarn:0   dfail:0   fail:0   skip:28 
fi-byt-n2820     total:244  pass:212  dwarn:0   dfail:0   fail:0   skip:32 
fi-hsw-4770      total:244  pass:224  dwarn:0   dfail:0   fail:0   skip:20 
fi-hsw-4770r     total:244  pass:224  dwarn:0   dfail:0   fail:0   skip:20 
fi-ilk-650       total:244  pass:191  dwarn:0   dfail:0   fail:0   skip:53 
fi-ivb-3520m     total:244  pass:222  dwarn:0   dfail:0   fail:0   skip:22 
fi-ivb-3770      total:244  pass:222  dwarn:0   dfail:0   fail:0   skip:22 
fi-kbl-7200u     total:244  pass:222  dwarn:0   dfail:0   fail:0   skip:22 
fi-skl-6260u     total:244  pass:230  dwarn:0   dfail:0   fail:0   skip:14 
fi-skl-6700hq    total:244  pass:223  dwarn:0   dfail:0   fail:0   skip:21 
fi-skl-6700k     total:244  pass:222  dwarn:1   dfail:0   fail:0   skip:21 
fi-skl-6770hq    total:244  pass:229  dwarn:1   dfail:0   fail:0   skip:14 
fi-snb-2520m     total:244  pass:212  dwarn:0   dfail:0   fail:0   skip:32 
fi-snb-2600      total:244  pass:211  dwarn:0   dfail:0   fail:0   skip:33 

6294f67611ebe69006c0e85c372efadcac8e9d66 drm-intel-nightly: 2016y-11m-16d-09h-57m-25s UTC integration manifest
cf75a49 drm/i915/execlists: Use a local lock for dfs_link access

== Logs ==

For more details see: https://intel-gfx-ci.01.org/CI/Patchwork_3025/
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[Daniel Vetter]

On Wed, Nov 16, 2016 at 04:03:27PM +0000, Chris Wilson wrote:
> On Wed, Nov 16, 2016 at 03:54:23PM +0000, Tvrtko Ursulin wrote:
> > 
> > On 16/11/2016 15:27, Chris Wilson wrote:
> > >Avoid requiring struct_mutex for exclusive access to the temporary
> > >dfs_link inside the i915_dependency as not all callers may want to touch
> > >struct_mutex. So rather than force them to take a highly contended
> > >lock, introduce a local lock for the execlists schedule operation.
> > >
> > >Reported-by: David Weinehall <david.weinehall@linux.intel.com>
> > >Fixes: 9a151987d709 ("drm/i915: Add execution priority boosting for mmioflips")
> > 
> > Grumble grumble, sloppy review. :I
> 
> Too busy living the good life with working atomic flips.
>  
> > >Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > >Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> > >Cc: David Weinehall <david.weinehall@linux.intel.com>
> > >---
> > > drivers/gpu/drm/i915/intel_lrc.c | 7 +++++--
> > > 1 file changed, 5 insertions(+), 2 deletions(-)
> > >
> > >diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> > >index e23b6a2600fb..10e59ff0d8f1 100644
> > >--- a/drivers/gpu/drm/i915/intel_lrc.c
> > >+++ b/drivers/gpu/drm/i915/intel_lrc.c
> > >@@ -694,6 +694,7 @@ pt_lock_engine(struct i915_priotree *pt, struct intel_engine_cs *locked)
> > >
> > > static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
> > > {
> > >+	static DEFINE_MUTEX(lock);
> > 
> > Good enough for one GPU. :) Consider improving in the future as it
> > is not in the spirit of the driver.
> 
> Actually... Being able to PI across multiple GPUs is part of the dream.
> In that case, it does need to be a global lock - just a bit iffy on
> getting dependency tracking into a common layer.

Why exactly does this need a global lock? And yes static mutex in a
function looks evil, pls move right next to the data.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[Chris Wilson]

On Thu, Nov 17, 2016 at 09:45:17AM +0100, Daniel Vetter wrote:
> On Wed, Nov 16, 2016 at 04:03:27PM +0000, Chris Wilson wrote:
> > On Wed, Nov 16, 2016 at 03:54:23PM +0000, Tvrtko Ursulin wrote:
> > > 
> > > On 16/11/2016 15:27, Chris Wilson wrote:
> > > >Avoid requiring struct_mutex for exclusive access to the temporary
> > > >dfs_link inside the i915_dependency as not all callers may want to touch
> > > >struct_mutex. So rather than force them to take a highly contended
> > > >lock, introduce a local lock for the execlists schedule operation.
> > > >
> > > >Reported-by: David Weinehall <david.weinehall@linux.intel.com>
> > > >Fixes: 9a151987d709 ("drm/i915: Add execution priority boosting for mmioflips")
> > > 
> > > Grumble grumble, sloppy review. :I
> > 
> > Too busy living the good life with working atomic flips.
> >  
> > > >Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > > >Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> > > >Cc: David Weinehall <david.weinehall@linux.intel.com>
> > > >---
> > > > drivers/gpu/drm/i915/intel_lrc.c | 7 +++++--
> > > > 1 file changed, 5 insertions(+), 2 deletions(-)
> > > >
> > > >diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> > > >index e23b6a2600fb..10e59ff0d8f1 100644
> > > >--- a/drivers/gpu/drm/i915/intel_lrc.c
> > > >+++ b/drivers/gpu/drm/i915/intel_lrc.c
> > > >@@ -694,6 +694,7 @@ pt_lock_engine(struct i915_priotree *pt, struct intel_engine_cs *locked)
> > > >
> > > > static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
> > > > {
> > > >+	static DEFINE_MUTEX(lock);
> > > 
> > > Good enough for one GPU. :) Consider improving in the future as it
> > > is not in the spirit of the driver.
> > 
> > Actually... Being able to PI across multiple GPUs is part of the dream.
> > In that case, it does need to be a global lock - just a bit iffy on
> > getting dependency tracking into a common layer.
> 
> Why exactly does this need a global lock? And yes static mutex in a
> function looks evil, pls move right next to the data.

It is next to the data.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[Daniel Vetter]

On Thu, Nov 17, 2016 at 08:52:59AM +0000, Chris Wilson wrote:
> On Thu, Nov 17, 2016 at 09:45:17AM +0100, Daniel Vetter wrote:
> > On Wed, Nov 16, 2016 at 04:03:27PM +0000, Chris Wilson wrote:
> > > On Wed, Nov 16, 2016 at 03:54:23PM +0000, Tvrtko Ursulin wrote:
> > > > 
> > > > On 16/11/2016 15:27, Chris Wilson wrote:
> > > > >Avoid requiring struct_mutex for exclusive access to the temporary
> > > > >dfs_link inside the i915_dependency as not all callers may want to touch
> > > > >struct_mutex. So rather than force them to take a highly contended
> > > > >lock, introduce a local lock for the execlists schedule operation.
> > > > >
> > > > >Reported-by: David Weinehall <david.weinehall@linux.intel.com>
> > > > >Fixes: 9a151987d709 ("drm/i915: Add execution priority boosting for mmioflips")
> > > > 
> > > > Grumble grumble, sloppy review. :I
> > > 
> > > Too busy living the good life with working atomic flips.
> > >  
> > > > >Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > > > >Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> > > > >Cc: David Weinehall <david.weinehall@linux.intel.com>
> > > > >---
> > > > > drivers/gpu/drm/i915/intel_lrc.c | 7 +++++--
> > > > > 1 file changed, 5 insertions(+), 2 deletions(-)
> > > > >
> > > > >diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
> > > > >index e23b6a2600fb..10e59ff0d8f1 100644
> > > > >--- a/drivers/gpu/drm/i915/intel_lrc.c
> > > > >+++ b/drivers/gpu/drm/i915/intel_lrc.c
> > > > >@@ -694,6 +694,7 @@ pt_lock_engine(struct i915_priotree *pt, struct intel_engine_cs *locked)
> > > > >
> > > > > static void execlists_schedule(struct drm_i915_gem_request *request, int prio)
> > > > > {
> > > > >+	static DEFINE_MUTEX(lock);
> > > > 
> > > > Good enough for one GPU. :) Consider improving in the future as it
> > > > is not in the spirit of the driver.
> > > 
> > > Actually... Being able to PI across multiple GPUs is part of the dream.
> > > In that case, it does need to be a global lock - just a bit iffy on
> > > getting dependency tracking into a common layer.
> > 
> > Why exactly does this need a global lock? And yes static mutex in a
> > function looks evil, pls move right next to the data.
> 
> It is next to the data.

diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index e2b077df2da0..c7c4c465adce 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -44,10 +44,13 @@ struct intel_signal_node {
 	struct intel_wait wait;
 };
 
+extern struct mutex dfs_lock;
+
 struct i915_dependency {
 	struct i915_priotree *signaler;
 	struct list_head signal_link;
 	struct list_head wait_link;
+	/* protected by dfs_lock */
 	struct list_head dfs_link;
 	unsigned long flags;
 #define I915_DEPENDENCY_ALLOC BIT(0)

Or something like that. Or at least a note about this stuff.

Oh and broken record again: Any plans to kernel-doc all the fancy new
stuff (datastructures and interface functions), or is that not on the
plan?
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH] drm/i915/execlists: Use a local lock for dfs_link access

[David Weinehall]

On Wed, Nov 16, 2016 at 03:27:21PM +0000, Chris Wilson wrote:
> Avoid requiring struct_mutex for exclusive access to the temporary
> dfs_link inside the i915_dependency as not all callers may want to touch
> struct_mutex. So rather than force them to take a highly contended
> lock, introduce a local lock for the execlists schedule operation.
> 
> Reported-by: David Weinehall <david.weinehall@linux.intel.com>
> Fixes: 9a151987d709 ("drm/i915: Add execution priority boosting for mmioflips")
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Cc: David Weinehall <david.weinehall@linux.intel.com>

I'll provide a retroactive:

Tested-by: David Weinehall <david.weinehall@linux.intel.com>

Seems to work fine; no flooded logs on my test machine with this patch.


Kind regards, David
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx