* [PATCH 1/5] i915: avoid kernel hang caused by synchronize rcu struct_mutex deadlock
2017-04-06 23:23 ` [PATCH 0/5] " Andrea Arcangeli
@ 2017-04-06 23:23 ` Andrea Arcangeli
2017-04-07 9:05 ` [Intel-gfx] " Joonas Lahtinen
2017-04-06 23:23 ` [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker Andrea Arcangeli
` (4 subsequent siblings)
5 siblings, 1 reply; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-06 23:23 UTC (permalink / raw)
To: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
Chris Wilson
Cc: intel-gfx, linux-kernel, dri-devel
synchronize_rcu/synchronize_sched/synchronize_rcu_expedited() will
hang until its own workqueues are run. The i915 gem workqueues will
wait on the struct_mutex to be released. So we cannot wait for a
quiescent state using those rcu primitives while holding the
struct_mutex or it creates a circular lock dependency resulting in
kernel hangs (which is reproducible but goes undetected by lockdep).
This started in commit 3d3d18f086cdda72ee18a454db70ca72c6e3246c and
lockdep didn't detect it apparently.
kswapd0 D 0 700 2 0x00000000
Call Trace:
? __schedule+0x1a5/0x660
? schedule+0x36/0x80
? _synchronize_rcu_expedited.constprop.65+0x2ef/0x300
? wake_up_bit+0x20/0x20
? rcu_stall_kick_kthreads.part.54+0xc0/0xc0
? rcu_exp_wait_wake+0x530/0x530
? i915_gem_shrink+0x34b/0x4b0
? i915_gem_shrinker_scan+0x7c/0x90
? i915_gem_shrinker_scan+0x7c/0x90
? shrink_slab.part.61.constprop.72+0x1c1/0x3a0
? shrink_zone+0x154/0x160
? kswapd+0x40a/0x720
? kthread+0xf4/0x130
? try_to_free_pages+0x450/0x450
? kthread_create_on_node+0x40/0x40
? ret_from_fork+0x23/0x30
plasmashell D 0 4657 4614 0x00000000
Call Trace:
? __schedule+0x1a5/0x660
? schedule+0x36/0x80
? schedule_preempt_disabled+0xe/0x10
? __mutex_lock.isra.4+0x1c9/0x790
? i915_gem_close_object+0x26/0xc0
? i915_gem_close_object+0x26/0xc0
? drm_gem_object_release_handle+0x48/0x90
? drm_gem_handle_delete+0x50/0x80
? drm_ioctl+0x1fa/0x420
? drm_gem_handle_create+0x40/0x40
? pipe_write+0x391/0x410
? __vfs_write+0xc6/0x120
? do_vfs_ioctl+0x8b/0x5d0
? SyS_ioctl+0x3b/0x70
? entry_SYSCALL_64_fastpath+0x13/0x94
kworker/0:0 D 0 29186 2 0x00000000
Workqueue: events __i915_gem_free_work
Call Trace:
? __schedule+0x1a5/0x660
? schedule+0x36/0x80
? schedule_preempt_disabled+0xe/0x10
? __mutex_lock.isra.4+0x1c9/0x790
? del_timer_sync+0x44/0x50
? update_curr+0x57/0x110
? __i915_gem_free_objects+0x31/0x300
? __i915_gem_free_objects+0x31/0x300
? __i915_gem_free_work+0x2d/0x40
? process_one_work+0x13a/0x3b0
? worker_thread+0x4a/0x460
? kthread+0xf4/0x130
? process_one_work+0x3b0/0x3b0
? kthread_create_on_node+0x40/0x40
? ret_from_fork+0x23/0x30
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
drivers/gpu/drm/i915/i915_gem.c | 9 +++++++++
drivers/gpu/drm/i915/i915_gem_shrinker.c | 14 ++++++++++----
2 files changed, 19 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 67b1fc5..3982489 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -4742,6 +4742,13 @@ int i915_gem_freeze(struct drm_i915_private *dev_priv)
i915_gem_shrink_all(dev_priv);
mutex_unlock(&dev_priv->drm.struct_mutex);
+ /*
+ * Cannot call synchronize_rcu() inside the struct_mutex
+ * because it may block until workqueues complete, and the
+ * running workqueue may wait on the struct_mutex.
+ */
+ synchronize_rcu(); /* wait for our earlier RCU delayed slab frees */
+
intel_runtime_pm_put(dev_priv);
return 0;
@@ -4781,6 +4788,8 @@ int i915_gem_freeze_late(struct drm_i915_private *dev_priv)
}
mutex_unlock(&dev_priv->drm.struct_mutex);
+ synchronize_rcu_expedited();
+
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
index d5d2b4c..fea1454 100644
--- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
@@ -235,9 +235,6 @@ i915_gem_shrink(struct drm_i915_private *dev_priv,
if (unlock)
mutex_unlock(&dev_priv->drm.struct_mutex);
- /* expedite the RCU grace period to free some request slabs */
- synchronize_rcu_expedited();
-
return count;
}
@@ -263,7 +260,6 @@ unsigned long i915_gem_shrink_all(struct drm_i915_private *dev_priv)
I915_SHRINK_BOUND |
I915_SHRINK_UNBOUND |
I915_SHRINK_ACTIVE);
- synchronize_rcu(); /* wait for our earlier RCU delayed slab frees */
return freed;
}
@@ -324,6 +320,16 @@ i915_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
if (unlock)
mutex_unlock(&dev->struct_mutex);
+ if (likely(__mutex_owner(&dev->struct_mutex) != current))
+ /*
+ * If reclaim was invoked by an allocation done while
+ * holding the struct mutex, we cannot call
+ * synchronize_rcu_expedited() as it depends on
+ * workqueues to run but the running workqueue may be
+ * blocked waiting on us to release struct_mutex.
+ */
+ synchronize_rcu_expedited();
+
return freed;
}
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [Intel-gfx] [PATCH 1/5] i915: avoid kernel hang caused by synchronize rcu struct_mutex deadlock
2017-04-06 23:23 ` [PATCH 1/5] i915: avoid kernel hang caused by synchronize rcu struct_mutex deadlock Andrea Arcangeli
@ 2017-04-07 9:05 ` Joonas Lahtinen
0 siblings, 0 replies; 21+ messages in thread
From: Joonas Lahtinen @ 2017-04-07 9:05 UTC (permalink / raw)
To: Andrea Arcangeli, Martin Kepplinger, Thorsten Leemhuis,
daniel.vetter, Dave Airlie, Chris Wilson
Cc: intel-gfx, linux-kernel, dri-devel
On pe, 2017-04-07 at 01:23 +0200, Andrea Arcangeli wrote:
> synchronize_rcu/synchronize_sched/synchronize_rcu_expedited() will
> hang until its own workqueues are run. The i915 gem workqueues will
> wait on the struct_mutex to be released. So we cannot wait for a
> quiescent state using those rcu primitives while holding the
> struct_mutex or it creates a circular lock dependency resulting in
> kernel hangs (which is reproducible but goes undetected by lockdep).
>
> This started in commit 3d3d18f086cdda72ee18a454db70ca72c6e3246c and
> lockdep didn't detect it apparently.
The right format is;
Fixes: 3d3d18f086cd ("drm/i915: Avoid rcu_barrier() from reclaim paths (shrinker)")
> @@ -324,6 +320,16 @@ i915_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
> if (unlock)
> mutex_unlock(&dev->struct_mutex);
>
> + if (likely(__mutex_owner(&dev->struct_mutex) != current))
This check can be dropped and synchronize_rcu_expedited() should be
embedded directly to the if (unlock) branch as it's functionally
equivalent. This can be applied to all the unlock cases, not just this
one. That should be the correct action to avoid the deadlock. I've sent
a patch to do this (Cc'd you), can you verify that it gets rid of the
problem for you?
> + /*
> + * If reclaim was invoked by an allocation done while
> + * holding the struct mutex, we cannot call
> + * synchronize_rcu_expedited() as it depends on
> + * workqueues to run but the running workqueue may be
> + * blocked waiting on us to release struct_mutex.
> + */
> + synchronize_rcu_expedited();
> +
> return freed;
> }
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker
2017-04-06 23:23 ` [PATCH 0/5] " Andrea Arcangeli
2017-04-06 23:23 ` [PATCH 1/5] i915: avoid kernel hang caused by synchronize rcu struct_mutex deadlock Andrea Arcangeli
@ 2017-04-06 23:23 ` Andrea Arcangeli
2017-04-07 10:02 ` Chris Wilson
2017-04-06 23:23 ` [PATCH 3/5] i915: initialize the free_list of the fencing atomic_helper Andrea Arcangeli
` (3 subsequent siblings)
5 siblings, 1 reply; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-06 23:23 UTC (permalink / raw)
To: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
Chris Wilson
Cc: intel-gfx, linux-kernel, dri-devel
Waiting a RCU grace period only guarantees the work gets queued, but
until after the queued workqueue returns, there's no guarantee the
memory was actually freed. So flush the work to provide better
guarantees to the reclaim code in addition of waiting a RCU grace
period to pass.
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
drivers/gpu/drm/i915/i915_gem.c | 2 ++
drivers/gpu/drm/i915/i915_gem_shrinker.c | 1 +
2 files changed, 3 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 3982489..612fde3 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -4748,6 +4748,7 @@ int i915_gem_freeze(struct drm_i915_private *dev_priv)
* running workqueue may wait on the struct_mutex.
*/
synchronize_rcu(); /* wait for our earlier RCU delayed slab frees */
+ flush_work(&dev_priv->mm.free_work);
intel_runtime_pm_put(dev_priv);
@@ -4789,6 +4790,7 @@ int i915_gem_freeze_late(struct drm_i915_private *dev_priv)
mutex_unlock(&dev_priv->drm.struct_mutex);
synchronize_rcu_expedited();
+ flush_work(&dev_priv->mm.free_work);
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
index fea1454..30f79af 100644
--- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
@@ -329,6 +329,7 @@ i915_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
* blocked waiting on us to release struct_mutex.
*/
synchronize_rcu_expedited();
+ flush_work(&dev_priv->mm.free_work);
return freed;
}
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker
2017-04-06 23:23 ` [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker Andrea Arcangeli
@ 2017-04-07 10:02 ` Chris Wilson
2017-04-07 13:06 ` Andrea Arcangeli
0 siblings, 1 reply; 21+ messages in thread
From: Chris Wilson @ 2017-04-07 10:02 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 01:23:44AM +0200, Andrea Arcangeli wrote:
> Waiting a RCU grace period only guarantees the work gets queued, but
> until after the queued workqueue returns, there's no guarantee the
> memory was actually freed. So flush the work to provide better
> guarantees to the reclaim code in addition of waiting a RCU grace
> period to pass.
We are not allowed to call flush_work() from the shrinker, the workqueue
doesn't have and can't have the right reclaim flags.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker
2017-04-07 10:02 ` Chris Wilson
@ 2017-04-07 13:06 ` Andrea Arcangeli
2017-04-07 15:30 ` Chris Wilson
0 siblings, 1 reply; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-07 13:06 UTC (permalink / raw)
To: Chris Wilson, Martin Kepplinger, Thorsten Leemhuis,
daniel.vetter, Dave Airlie, intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 11:02:11AM +0100, Chris Wilson wrote:
> On Fri, Apr 07, 2017 at 01:23:44AM +0200, Andrea Arcangeli wrote:
> > Waiting a RCU grace period only guarantees the work gets queued, but
> > until after the queued workqueue returns, there's no guarantee the
> > memory was actually freed. So flush the work to provide better
> > guarantees to the reclaim code in addition of waiting a RCU grace
> > period to pass.
>
> We are not allowed to call flush_work() from the shrinker, the workqueue
> doesn't have and can't have the right reclaim flags.
I figured the flush_work had to be conditional to "unlock" being true
too in the i915 shrinker (not only synchronize_rcu_expedited()), and I
already fixed that bit, but I didn't think it would be a problem to
wait for the workqueue as long as reclaim didn't recurse on the
struct_mutex (it is a problem if unlock is false of course as we would
be back to square one). I didn't get further hangs and I assume I've
been running a couple of synchronize_rcu_expedited() and flush_work (I
should add dynamic tracing to be sure).
Also note, I didn't get any lockdep warning when I reproduced the
workqueue hang in 4.11-rc5 so at least as far as lockdep is concerned
there's no problem to call synchronize_rcu_expedited and it couldn't
notice we were holding the struct_mutex while waiting for the new
workqueue to run.
Also note recursing on the lock (unlock false case) is something
nothing else does, I'm not sure if it's worth the risk and if you
shouldn't just call mutex_trylock in the shrinker instead of
mutex_trylock_recursive. One thing was to recurse on the lock
internally in the same context, but recursing through the whole
reclaim is more dubious as safe.
You could start dropping objects and wiping vmas and stuff in the
middle of some kmalloc/alloc_pages that doesn't expect it and then
crash for other reasons. So this reclaim recursion model of the
shinker is quite unique and quite challenging to proof as safe if you
keep using mutex_trylock_recursive in i915_gem_shrinker_scan.
Lock recursion in all other places could be dropped without runtime
downsides, the only place mutex_trylock_recursive makes a design
difference and makes sense to be used is in i915_gem_shrinker_scan,
the rest are implementation issues not fundamental shrinker design and
it'd be nice if those other mutex_trylock_recursive would all be
removed and the only one that is left is in i915_gem_shrinker_scan and
nowhere else (or to drop it also from i915_gem_shrinker_scan).
mutex_trylock_recursive() should also be patched to use
READ_ONCE(__mutex_owner(lock)) because currently it breaks C.
In the whole kernel i915 and msm drm are the only two users of such
function in fact.
Another thing is what value return from i915_gem_shrinker_scan when
unlock is false, and we can't possibly wait for the memory to be freed
let alone for a rcu grace period. For various reasons I think it's
safer to return the current "free" even if we could also return "0" in
such case. There are different tradeoffs, returning "free" is less
likely to trigger an early OOM as the VM thinks it's still making
progress and in fact it will get more free memory shortly, while
returning SHRINK_STOP would also be an option and it would insist more
on the other slabs so it would be more reliable at freeing memory
timely, but it would be more at risk of early OOM. I think returning
"free" is the better tradeoff of the two, but I suggest to add a
comment as it's not exactly obvious what is better.
Thanks,
Andrea
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker
2017-04-07 13:06 ` Andrea Arcangeli
@ 2017-04-07 15:30 ` Chris Wilson
2017-04-07 16:48 ` Andrea Arcangeli
0 siblings, 1 reply; 21+ messages in thread
From: Chris Wilson @ 2017-04-07 15:30 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 03:06:00PM +0200, Andrea Arcangeli wrote:
> On Fri, Apr 07, 2017 at 11:02:11AM +0100, Chris Wilson wrote:
> > On Fri, Apr 07, 2017 at 01:23:44AM +0200, Andrea Arcangeli wrote:
> > > Waiting a RCU grace period only guarantees the work gets queued, but
> > > until after the queued workqueue returns, there's no guarantee the
> > > memory was actually freed. So flush the work to provide better
> > > guarantees to the reclaim code in addition of waiting a RCU grace
> > > period to pass.
> >
> > We are not allowed to call flush_work() from the shrinker, the workqueue
> > doesn't have and can't have the right reclaim flags.
>
> I figured the flush_work had to be conditional to "unlock" being true
> too in the i915 shrinker (not only synchronize_rcu_expedited()), and I
> already fixed that bit, but I didn't think it would be a problem to
> wait for the workqueue as long as reclaim didn't recurse on the
> struct_mutex (it is a problem if unlock is false of course as we would
> be back to square one). I didn't get further hangs and I assume I've
> been running a couple of synchronize_rcu_expedited() and flush_work (I
> should add dynamic tracing to be sure).
Not getting hangs is a good sign, but lockdep doesn't like it:
[ 460.684901] WARNING: CPU: 1 PID: 172 at kernel/workqueue.c:2418 check_flush_dependency+0x92/0x130
[ 460.684924] workqueue: PF_MEMALLOC task 172(kworker/1:1H) is flushing !WQ_MEM_RECLAIM events:__i915_gem_free_work [i915]
If I allocated the workqueue with WQ_MEM_RELCAIM, it complains bitterly
as well.
> Also note, I didn't get any lockdep warning when I reproduced the
> workqueue hang in 4.11-rc5 so at least as far as lockdep is concerned
> there's no problem to call synchronize_rcu_expedited and it couldn't
> notice we were holding the struct_mutex while waiting for the new
> workqueue to run.
Yes, that is concerning. I think it's due to the coupling via
the struct completion that is not being picked up lockdep, and I hope
the "crossrelease" patches would fix the lack of warnings.
> Also note recursing on the lock (unlock false case) is something
> nothing else does, I'm not sure if it's worth the risk and if you
> shouldn't just call mutex_trylock in the shrinker instead of
> mutex_trylock_recursive. One thing was to recurse on the lock
> internally in the same context, but recursing through the whole
> reclaim is more dubious as safe.
We know. We don't use trylock in order to reduce the frequency of users'
oom. Peter added mutex_trylock_recursive() because we already were doing
recursive locking in the shrinker and although we know we shouldn't,
getting rid of the recursion is something we are doing, but slowly.
> You could start dropping objects and wiping vmas and stuff in the
> middle of some kmalloc/alloc_pages that doesn't expect it and then
> crash for other reasons. So this reclaim recursion model of the
> shinker is quite unique and quite challenging to proof as safe if you
> keep using mutex_trylock_recursive in i915_gem_shrinker_scan.
I know. Trying to stay on top of all the kmallocs under the struct_mutex
and being aware that the shrinker can and will undo your objects as you
work is a continual battle. And catches everyone working on i915.ko by
surprise. Our policy to avoid surprises is based around pin before alloc.
> Lock recursion in all other places could be dropped without runtime
> downsides, the only place mutex_trylock_recursive makes a design
> difference and makes sense to be used is in i915_gem_shrinker_scan,
> the rest are implementation issues not fundamental shrinker design and
> it'd be nice if those other mutex_trylock_recursive would all be
> removed and the only one that is left is in i915_gem_shrinker_scan and
> nowhere else (or to drop it also from i915_gem_shrinker_scan).
We do need it for shrinker_count as well. If we just report 0 objects,
the shrinker_scan callback will be skipped, iirc. All we do need it for
direct calls to i915_gem_shrink() which themselves may or may not be
underneath the struct_mutex at the time.
> mutex_trylock_recursive() should also be patched to use
> READ_ONCE(__mutex_owner(lock)) because currently it breaks C.
I don't follow,
static inline struct task_struct *__mutex_owner(struct mutex *lock)
{
return (struct task_struct *)(atomic_long_read(&lock->owner) & ~0x07);
}
The atomic read is equivalent to READ_ONCE(). What's broken here? (I
guess strict aliasing and pointer cast?)
> In the whole kernel i915 and msm drm are the only two users of such
> function in fact.
Yes, Peter will continue to remind us to fix our code and complain until
it is.
> Another thing is what value return from i915_gem_shrinker_scan when
> unlock is false, and we can't possibly wait for the memory to be freed
> let alone for a rcu grace period. For various reasons I think it's
> safer to return the current "free" even if we could also return "0" in
> such case. There are different tradeoffs, returning "free" is less
> likely to trigger an early OOM as the VM thinks it's still making
> progress and in fact it will get more free memory shortly, while
> returning SHRINK_STOP would also be an option and it would insist more
> on the other slabs so it would be more reliable at freeing memory
> timely, but it would be more at risk of early OOM. I think returning
> "free" is the better tradeoff of the two, but I suggest to add a
> comment as it's not exactly obvious what is better.
Ah. The RCU freeing is only for the small fry, the slabs from which
requests and objects are allocated. It's the gigabytes of pages we have
pinned that can be released by i915_gem_shrink() are what we count as
freed, even though we only return them to the system and hope the lru
anon page scanner will make available for reuse (if they are dirty, they
will need to be swapped out, but some will be returned to the system
directly by truncating the shmemfs filp backing an object.)
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker
2017-04-07 15:30 ` Chris Wilson
@ 2017-04-07 16:48 ` Andrea Arcangeli
2017-04-10 9:39 ` Chris Wilson
0 siblings, 1 reply; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-07 16:48 UTC (permalink / raw)
To: Chris Wilson, Martin Kepplinger, Thorsten Leemhuis,
daniel.vetter, Dave Airlie, intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 04:30:11PM +0100, Chris Wilson wrote:
> Not getting hangs is a good sign, but lockdep doesn't like it:
>
> [ 460.684901] WARNING: CPU: 1 PID: 172 at kernel/workqueue.c:2418 check_flush_dependency+0x92/0x130
> [ 460.684924] workqueue: PF_MEMALLOC task 172(kworker/1:1H) is flushing !WQ_MEM_RECLAIM events:__i915_gem_free_work [i915]
>
> If I allocated the workqueue with WQ_MEM_RELCAIM, it complains bitterly
> as well.
So in PF_MEMALLOC context we can't flush a workqueue with
!WQ_MEM_RECLAIM.
system_wq = alloc_workqueue("events", 0, 0);
My point is synchronize_rcu_expedited will still push its work in
the same system_wq workqueue...
/* Marshall arguments & schedule the expedited grace period. */
rew.rew_func = func;
rew.rew_rsp = rsp;
rew.rew_s = s;
INIT_WORK_ONSTACK(&rew.rew_work, wait_rcu_exp_gp);
schedule_work(&rew.rew_work);
It's also using schedule_work, so either the above is a false
positive, or we've still a problem with synchronize_rcu_expedited,
just a reproducible issue anymore after we stop running it under the
struct_mutex.
Even synchronize_sched will wait on the system_wq if
synchronize_rcu_expedited has been issued in parallel by some other
code, it's just there's no check for it because it's not invoking
flush_work to wait.
The deadlock happens if we flush_work() while holding any lock that
may be taken by any of the workqueues that could be queued there. i915
makes sure to flush_work only if the struct_mutex was released (not
my initial version) but we're not checking if any of the other
system_wq workqueues could possibly taking a lock that may have been
hold during the allocation that invoked reclaim, I suppose that is the
problem left, but I don't see how flush_work is special about it,
synchronize_rcu_expedited would still have the same issue no? (despite
no lockdep warning)
I suspect this means synchronize_rcu_expedited() is not usable in
reclaim context and lockdep should warn if PF_MEMALLOC is set when
synchronize_rcu_expedited/synchronize_sched/synchronize_rcu are
called.
Probably to fix this we should create a private workqueue for both RCU
and i915 and stop sharing the system_wq with the rest of the system
(and of course set WQ_MEM_RECLAIM in both workqueues). This makes sure
when we call synchronize_rcu_expedited; flush_work from the shrinker,
we won't risk waiting on other random work that may be taking locks
that are hold by the code that invoked reclaim during an allocation.
The macro bug of waiting on system_wq 100% of the time while always
holding the struct_mutex is gone, but we need to perfect this further
and stop using the system_wq for RCU and i915 shrinker work.
> We do need it for shrinker_count as well. If we just report 0 objects,
Yes the shrinker_count too.
> the shrinker_scan callback will be skipped, iirc. All we do need it for
> direct calls to i915_gem_shrink() which themselves may or may not be
> underneath the struct_mutex at the time.
Yes.
> I don't follow,
>
> static inline struct task_struct *__mutex_owner(struct mutex *lock)
> {
> return (struct task_struct *)(atomic_long_read(&lock->owner) & ~0x07);
> }
>
> The atomic read is equivalent to READ_ONCE(). What's broken here? (I
> guess strict aliasing and pointer cast?)
That was an oversight, atomic64_read does READ_ONCE internally (as it
should of course being an atomic read). I didn't recall it uses
atomic_long_read.
Thanks,
Andrea
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker
2017-04-07 16:48 ` Andrea Arcangeli
@ 2017-04-10 9:39 ` Chris Wilson
0 siblings, 0 replies; 21+ messages in thread
From: Chris Wilson @ 2017-04-10 9:39 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Martin Kepplinger, Joonas Lahtinen, Thorsten Leemhuis,
daniel.vetter, Dave Airlie, intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 06:48:58PM +0200, Andrea Arcangeli wrote:
> On Fri, Apr 07, 2017 at 04:30:11PM +0100, Chris Wilson wrote:
> > Not getting hangs is a good sign, but lockdep doesn't like it:
> >
> > [ 460.684901] WARNING: CPU: 1 PID: 172 at kernel/workqueue.c:2418 check_flush_dependency+0x92/0x130
> > [ 460.684924] workqueue: PF_MEMALLOC task 172(kworker/1:1H) is flushing !WQ_MEM_RECLAIM events:__i915_gem_free_work [i915]
> >
> > If I allocated the workqueue with WQ_MEM_RELCAIM, it complains bitterly
> > as well.
>
> So in PF_MEMALLOC context we can't flush a workqueue with
> !WQ_MEM_RECLAIM.
>
> system_wq = alloc_workqueue("events", 0, 0);
>
> My point is synchronize_rcu_expedited will still push its work in
> the same system_wq workqueue...
>
> /* Marshall arguments & schedule the expedited grace period. */
> rew.rew_func = func;
> rew.rew_rsp = rsp;
> rew.rew_s = s;
> INIT_WORK_ONSTACK(&rew.rew_work, wait_rcu_exp_gp);
> schedule_work(&rew.rew_work);
>
> It's also using schedule_work, so either the above is a false
> positive, or we've still a problem with synchronize_rcu_expedited,
> just a reproducible issue anymore after we stop running it under the
> struct_mutex.
We still do have a problem with using synchronize_rcu_expedited() from
the shrinker as we maybe under someone else's mutex is that involved in
its own RCU dance.
> Even synchronize_sched will wait on the system_wq if
> synchronize_rcu_expedited has been issued in parallel by some other
> code, it's just there's no check for it because it's not invoking
> flush_work to wait.
Right.
> The deadlock happens if we flush_work() while holding any lock that
> may be taken by any of the workqueues that could be queued there. i915
> makes sure to flush_work only if the struct_mutex was released (not
> my initial version) but we're not checking if any of the other
> system_wq workqueues could possibly taking a lock that may have been
> hold during the allocation that invoked reclaim, I suppose that is the
> problem left, but I don't see how flush_work is special about it,
> synchronize_rcu_expedited would still have the same issue no? (despite
> no lockdep warning)
>
> I suspect this means synchronize_rcu_expedited() is not usable in
> reclaim context and lockdep should warn if PF_MEMALLOC is set when
> synchronize_rcu_expedited/synchronize_sched/synchronize_rcu are
> called.
Yes.
> Probably to fix this we should create a private workqueue for both RCU
> and i915 and stop sharing the system_wq with the rest of the system
> (and of course set WQ_MEM_RECLAIM in both workqueues). This makes sure
> when we call synchronize_rcu_expedited; flush_work from the shrinker,
> we won't risk waiting on other random work that may be taking locks
> that are hold by the code that invoked reclaim during an allocation.
We simply do not need to do our own synchronize_rcu* -- it's only used
to flush our slab frees on the off chance that (a) we have any and (b)
we do manage to free a whole slab. It is not the bulk of the memory that
we return to the system from the shrinker.
In the other thread, I stated that we should simply remove it. The
kswapd reclaim path should try to reclaim RCU slabs (by doing a
synhronize_sched or equivalent).
> The macro bug of waiting on system_wq 100% of the time while always
> holding the struct_mutex is gone, but we need to perfect this further
> and stop using the system_wq for RCU and i915 shrinker work.
Agreed. My preference is to simply not do it and leave the dangling RCU
to the core reclaim paths.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 3/5] i915: initialize the free_list of the fencing atomic_helper
2017-04-06 23:23 ` [PATCH 0/5] " Andrea Arcangeli
2017-04-06 23:23 ` [PATCH 1/5] i915: avoid kernel hang caused by synchronize rcu struct_mutex deadlock Andrea Arcangeli
2017-04-06 23:23 ` [PATCH 2/5] i915: flush gem obj freeing workqueues to add accuracy to the i915 shrinker Andrea Arcangeli
@ 2017-04-06 23:23 ` Andrea Arcangeli
2017-04-07 10:35 ` Chris Wilson
2017-04-06 23:23 ` [PATCH 4/5] i915: schedule while freeing the lists of gem objects Andrea Arcangeli
` (2 subsequent siblings)
5 siblings, 1 reply; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-06 23:23 UTC (permalink / raw)
To: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
Chris Wilson
Cc: intel-gfx, linux-kernel, dri-devel
Just in case the llist model changes and NULL isn't valid
initialization.
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
drivers/gpu/drm/i915/intel_display.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index ed1f4f2..24f303e 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -16630,6 +16630,7 @@ int intel_modeset_init(struct drm_device *dev)
dev->mode_config.funcs = &intel_mode_funcs;
+ init_llist_head(&dev_priv->atomic_helper.free_list);
INIT_WORK(&dev_priv->atomic_helper.free_work,
intel_atomic_helper_free_state_worker);
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [PATCH 3/5] i915: initialize the free_list of the fencing atomic_helper
2017-04-06 23:23 ` [PATCH 3/5] i915: initialize the free_list of the fencing atomic_helper Andrea Arcangeli
@ 2017-04-07 10:35 ` Chris Wilson
0 siblings, 0 replies; 21+ messages in thread
From: Chris Wilson @ 2017-04-07 10:35 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 01:23:45AM +0200, Andrea Arcangeli wrote:
> Just in case the llist model changes and NULL isn't valid
> initialization.
>
> Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
Applied, thanks.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 4/5] i915: schedule while freeing the lists of gem objects
2017-04-06 23:23 ` [PATCH 0/5] " Andrea Arcangeli
` (2 preceding siblings ...)
2017-04-06 23:23 ` [PATCH 3/5] i915: initialize the free_list of the fencing atomic_helper Andrea Arcangeli
@ 2017-04-06 23:23 ` Andrea Arcangeli
2017-04-06 23:23 ` [PATCH 5/5] i915: fence workqueue optimization Andrea Arcangeli
2017-04-10 10:15 ` [PATCH 0/5] Re: [Intel-gfx] [BUG][REGRESSION] i915 gpu hangs under load Martin Kepplinger
5 siblings, 0 replies; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-06 23:23 UTC (permalink / raw)
To: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
Chris Wilson
Cc: intel-gfx, linux-kernel, dri-devel
Add cond_resched().
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
drivers/gpu/drm/i915/i915_gem.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 612fde3..c81baeb 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -4205,6 +4205,8 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915,
GEM_BUG_ON(!RB_EMPTY_ROOT(&obj->vma_tree));
list_del(&obj->global_link);
+
+ cond_resched();
}
intel_runtime_pm_put(i915);
mutex_unlock(&i915->drm.struct_mutex);
@@ -4230,6 +4232,8 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915,
kfree(obj->bit_17);
i915_gem_object_free(obj);
+
+ cond_resched();
}
}
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH 5/5] i915: fence workqueue optimization
2017-04-06 23:23 ` [PATCH 0/5] " Andrea Arcangeli
` (3 preceding siblings ...)
2017-04-06 23:23 ` [PATCH 4/5] i915: schedule while freeing the lists of gem objects Andrea Arcangeli
@ 2017-04-06 23:23 ` Andrea Arcangeli
2017-04-07 9:58 ` Chris Wilson
2017-04-10 10:15 ` [PATCH 0/5] Re: [Intel-gfx] [BUG][REGRESSION] i915 gpu hangs under load Martin Kepplinger
5 siblings, 1 reply; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-06 23:23 UTC (permalink / raw)
To: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
Chris Wilson
Cc: intel-gfx, linux-kernel, dri-devel
Insist to run llist_del_all() until the free_list is found empty, this
may avoid having to schedule more workqueues.
Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
---
drivers/gpu/drm/i915/intel_display.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 24f303e..931f0c7 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -14374,9 +14374,9 @@ static void intel_atomic_helper_free_state(struct drm_i915_private *dev_priv)
struct intel_atomic_state *state, *next;
struct llist_node *freed;
- freed = llist_del_all(&dev_priv->atomic_helper.free_list);
- llist_for_each_entry_safe(state, next, freed, freed)
- drm_atomic_state_put(&state->base);
+ while ((freed = llist_del_all(&dev_priv->atomic_helper.free_list)))
+ llist_for_each_entry_safe(state, next, freed, freed)
+ drm_atomic_state_put(&state->base);
}
static void intel_atomic_helper_free_state_worker(struct work_struct *work)
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [PATCH 5/5] i915: fence workqueue optimization
2017-04-06 23:23 ` [PATCH 5/5] i915: fence workqueue optimization Andrea Arcangeli
@ 2017-04-07 9:58 ` Chris Wilson
2017-04-07 13:13 ` Andrea Arcangeli
0 siblings, 1 reply; 21+ messages in thread
From: Chris Wilson @ 2017-04-07 9:58 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Martin Kepplinger, Thorsten Leemhuis, daniel.vetter, Dave Airlie,
intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 01:23:47AM +0200, Andrea Arcangeli wrote:
> Insist to run llist_del_all() until the free_list is found empty, this
> may avoid having to schedule more workqueues.
The work will already be scheduled (everytime we add the first element,
the work is scheduled, and the scheduled bit is cleared before the work
is executed). So we aren't saving the kworker from having to process
another work, but we may make that having nothing to do. The question is
whether we want to trap the kworker here, and presumably you will also want
to add a cond_resched() between passes.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5/5] i915: fence workqueue optimization
2017-04-07 9:58 ` Chris Wilson
@ 2017-04-07 13:13 ` Andrea Arcangeli
0 siblings, 0 replies; 21+ messages in thread
From: Andrea Arcangeli @ 2017-04-07 13:13 UTC (permalink / raw)
To: Chris Wilson, Martin Kepplinger, Thorsten Leemhuis,
daniel.vetter, Dave Airlie, intel-gfx, linux-kernel, dri-devel
On Fri, Apr 07, 2017 at 10:58:38AM +0100, Chris Wilson wrote:
> On Fri, Apr 07, 2017 at 01:23:47AM +0200, Andrea Arcangeli wrote:
> > Insist to run llist_del_all() until the free_list is found empty, this
> > may avoid having to schedule more workqueues.
>
> The work will already be scheduled (everytime we add the first element,
> the work is scheduled, and the scheduled bit is cleared before the work
> is executed). So we aren't saving the kworker from having to process
> another work, but we may make that having nothing to do. The question is
> whether we want to trap the kworker here, and presumably you will also want
> to add a cond_resched() between passes.
Yes it is somewhat dubious in the two event only case, but it will
save kworker in case of more events if there is a flood of
llist_add. It just looked fast enough but it's up to you, it's a
cmpxchg more for each intel_atomic_helper_free_state. If it's unlikely
more work is added, it's better to drop it. Agree about
cond_resched() if we keep it.
The same issue exists in __i915_gem_free_work, but I guess it's more
likely there that by the time __i915_gem_free_objects returns the
free_list isn't empty anymore because __i915_gem_free_objects has a
longer runtime but then you may want to re-evaluate that too as it's
slower for the two llist_add in a row case and only pays off from the
third.
while ((freed = llist_del_all(&i915->mm.free_list)))
__i915_gem_free_objects(i915, freed);
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 0/5] Re: [Intel-gfx] [BUG][REGRESSION] i915 gpu hangs under load
2017-04-06 23:23 ` [PATCH 0/5] " Andrea Arcangeli
` (4 preceding siblings ...)
2017-04-06 23:23 ` [PATCH 5/5] i915: fence workqueue optimization Andrea Arcangeli
@ 2017-04-10 10:15 ` Martin Kepplinger
5 siblings, 0 replies; 21+ messages in thread
From: Martin Kepplinger @ 2017-04-10 10:15 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Thorsten Leemhuis, daniel.vetter, Dave Airlie, Chris Wilson,
intel-gfx, linux-kernel, dri-devel
Am 07.04.2017 01:23 schrieb Andrea Arcangeli:
> I'm also getting kernel hangs every couple of days. For me it's still
> not fixed here in 4.11-rc5. It's hard to reproduce, the best
> reproducer is to build lineageos 14.1 on host while running LTP in a
> guest to stress the guest VM.
>
> Initially I thought it was related to the fact I upgraded the xf86
> intel driver just a few weeks ago (I deferred any upgrade of the
> userland intel driver since last July because of a regression that
> never got fixed and broke xterm for me). After I found a workaround
> for the userland regression (appended at the end for reference) I
> started getting kernel hangs but they are separate issues as far as I
> can tell.
>
> It's not well tested so beware... (it survived a couple of builds and
> some VM reclaim but that's it).
>
> The first patch 1/5 is the potential fix for the i915 kernel hang. The
> rest are incremental improvements.
>
> And I've no great solution for when the shrinker was invoked with the
> struct_mutex held and and recurse on the lock. I don't think we can
> possibly wait in such case (other than flush work that the second
> patch does) but then practically it shouldn't be a big deal, the big
> RAM eater is unlikely to be i915 when the system is low on memory.
>
FWIW without having insight here, -rc6 seems to be good.
No disturbing gpu hangs under load so far.
^ permalink raw reply [flat|nested] 21+ messages in thread