* [PATCH] mm,compaction: serialize waitqueue_active() checks
@ 2017-01-09 15:25 ` Davidlohr Bueso
0 siblings, 0 replies; 2+ messages in thread
From: Davidlohr Bueso @ 2017-01-09 15:25 UTC (permalink / raw)
To: akpm; +Cc: vbabka, dave, linux-mm, linux-kernel, Davidlohr Bueso
Without a memory barrier, the following race can occur with a high-order
allocation:
wakeup_kcompactd(order == 1) kcompactd()
[L] waitqueue_active(kcompactd_wait)
[S] prepare_to_wait_event(kcompactd_wait)
[L] (kcompactd_max_order == 0)
[S] kcompactd_max_order = order; schedule()
Where the waitqueue_active() check is speculatively re-ordered to before
setting the actual condition (max_order), not seeing the threads that's
going to block; making us miss a wakeup. There are a couple of options to
fix this, including calling wq_has_sleepers() which adds a full barrier,
or unconditionally doing the wake_up_interruptible() and serialize on the
q->lock. However, to make use of the control dependency, we just need to
add L->L guarantees.
While this bug is theoretical, there have been other offenders of the lockless
waitqueue_active() in the past -- this is also documented in the call itself.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
mm/compaction.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/mm/compaction.c b/mm/compaction.c
index 949198d01260..fb0f87554eb9 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1950,6 +1950,13 @@ void wakeup_kcompactd(pg_data_t *pgdat, int order, int classzone_idx)
if (pgdat->kcompactd_max_order < order)
pgdat->kcompactd_max_order = order;
+ /*
+ * Pairs with implicit barrier in wait_event_freezable()
+ * such that wakeups are not missed in the lockless
+ * waitqueue_active() call.
+ */
+ smp_acquire__after_ctrl_dep();
+
if (pgdat->kcompactd_classzone_idx > classzone_idx)
pgdat->kcompactd_classzone_idx = classzone_idx;
--
2.6.6
^ permalink raw reply related [flat|nested] 2+ messages in thread
* [PATCH] mm,compaction: serialize waitqueue_active() checks
@ 2017-01-09 15:25 ` Davidlohr Bueso
0 siblings, 0 replies; 2+ messages in thread
From: Davidlohr Bueso @ 2017-01-09 15:25 UTC (permalink / raw)
To: akpm; +Cc: vbabka, dave, linux-mm, linux-kernel, Davidlohr Bueso
Without a memory barrier, the following race can occur with a high-order
allocation:
wakeup_kcompactd(order == 1) kcompactd()
[L] waitqueue_active(kcompactd_wait)
[S] prepare_to_wait_event(kcompactd_wait)
[L] (kcompactd_max_order == 0)
[S] kcompactd_max_order = order; schedule()
Where the waitqueue_active() check is speculatively re-ordered to before
setting the actual condition (max_order), not seeing the threads that's
going to block; making us miss a wakeup. There are a couple of options to
fix this, including calling wq_has_sleepers() which adds a full barrier,
or unconditionally doing the wake_up_interruptible() and serialize on the
q->lock. However, to make use of the control dependency, we just need to
add L->L guarantees.
While this bug is theoretical, there have been other offenders of the lockless
waitqueue_active() in the past -- this is also documented in the call itself.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
mm/compaction.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/mm/compaction.c b/mm/compaction.c
index 949198d01260..fb0f87554eb9 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1950,6 +1950,13 @@ void wakeup_kcompactd(pg_data_t *pgdat, int order, int classzone_idx)
if (pgdat->kcompactd_max_order < order)
pgdat->kcompactd_max_order = order;
+ /*
+ * Pairs with implicit barrier in wait_event_freezable()
+ * such that wakeups are not missed in the lockless
+ * waitqueue_active() call.
+ */
+ smp_acquire__after_ctrl_dep();
+
if (pgdat->kcompactd_classzone_idx > classzone_idx)
pgdat->kcompactd_classzone_idx = classzone_idx;
--
2.6.6
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 2+ messages in thread
end of thread, other threads:[~2017-01-09 15:25 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-09 15:25 [PATCH] mm,compaction: serialize waitqueue_active() checks Davidlohr Bueso
2017-01-09 15:25 ` Davidlohr Bueso
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.