* [PATCH RFC v2 0/2] bugfix for sbitmap
@ 2022-06-19 8:03 Yu Kuai
2022-06-19 8:03 ` [PATCH RFC v2 1/2] sbitmap: fix that same waitqueue can be woken up continuously Yu Kuai
2022-06-19 8:03 ` [PATCH RFC v2 2/2] sbitmap: fix possible io hung due to lost wakeups Yu Kuai
0 siblings, 2 replies; 3+ messages in thread
From: Yu Kuai @ 2022-06-19 8:03 UTC (permalink / raw)
To: axboe, osandov, asml.silence, ming.lei
Cc: linux-block, linux-kernel, yukuai3, yi.zhang
Changes in v2:
- split to spearate patches for different problem.
- add fix tag
Yu Kuai (2):
sbitmap: fix that same waitqueue can be woken up continuously
sbitmap: fix possible io hung due to lost wakeups
lib/sbitmap.c | 60 +++++++++++++++++++++++++++------------------------
1 file changed, 32 insertions(+), 28 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH RFC v2 1/2] sbitmap: fix that same waitqueue can be woken up continuously
2022-06-19 8:03 [PATCH RFC v2 0/2] bugfix for sbitmap Yu Kuai
@ 2022-06-19 8:03 ` Yu Kuai
2022-06-19 8:03 ` [PATCH RFC v2 2/2] sbitmap: fix possible io hung due to lost wakeups Yu Kuai
1 sibling, 0 replies; 3+ messages in thread
From: Yu Kuai @ 2022-06-19 8:03 UTC (permalink / raw)
To: axboe, osandov, asml.silence, ming.lei
Cc: linux-block, linux-kernel, yukuai3, yi.zhang
__sbq_wake_up __sbq_wake_up
sbq_wake_ptr -> assume 0
sbq_wake_ptr -> 0
atomic_dec_return
atomic_dec_return
atomic_cmpxchg -> succeed
atomic_cmpxchg -> failed
return true
__sbq_wake_up
sbq_wake_ptr
atomic_read(&sbq->wake_index) -> still 0
sbq_index_atomic_inc -> inc to 1
if (waitqueue_active(&ws->wait))
if (wake_index != atomic_read(&sbq->wake_index))
atomic_set -> reset from 1 to 0
wake_up_nr -> wake up first waitqueue
// continue to wake up in first waitqueue
Fix the problem by using atomic_cmpxchg() instead of atomic_set()
to update 'wake_index'.
Fixes: 417232880c8a ("sbitmap: Replace cmpxchg with xchg")
Signed-off-by: Yu Kuai <yukuai3@huawei.com>
---
lib/sbitmap.c | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/lib/sbitmap.c b/lib/sbitmap.c
index ae4fd4de9ebe..4a230e5baacf 100644
--- a/lib/sbitmap.c
+++ b/lib/sbitmap.c
@@ -576,19 +576,26 @@ EXPORT_SYMBOL_GPL(sbitmap_queue_min_shallow_depth);
static struct sbq_wait_state *sbq_wake_ptr(struct sbitmap_queue *sbq)
{
- int i, wake_index;
+ int i, wake_index, old_wake_index;
+again:
if (!atomic_read(&sbq->ws_active))
return NULL;
- wake_index = atomic_read(&sbq->wake_index);
+ old_wake_index = wake_index = atomic_read(&sbq->wake_index);
for (i = 0; i < SBQ_WAIT_QUEUES; i++) {
struct sbq_wait_state *ws = &sbq->ws[wake_index];
if (waitqueue_active(&ws->wait)) {
- if (wake_index != atomic_read(&sbq->wake_index))
- atomic_set(&sbq->wake_index, wake_index);
- return ws;
+ if (wake_index != old_wake_index) {
+ if (atomic_cmpxchg(&sbq->wake_index,
+ old_wake_index, wake_index) ==
+ old_wake_index)
+ return ws;
+ goto again;
+ } else {
+ return ws;
+ }
}
wake_index = sbq_index_inc(wake_index);
--
2.31.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH RFC v2 2/2] sbitmap: fix possible io hung due to lost wakeups
2022-06-19 8:03 [PATCH RFC v2 0/2] bugfix for sbitmap Yu Kuai
2022-06-19 8:03 ` [PATCH RFC v2 1/2] sbitmap: fix that same waitqueue can be woken up continuously Yu Kuai
@ 2022-06-19 8:03 ` Yu Kuai
1 sibling, 0 replies; 3+ messages in thread
From: Yu Kuai @ 2022-06-19 8:03 UTC (permalink / raw)
To: axboe, osandov, asml.silence, ming.lei
Cc: linux-block, linux-kernel, yukuai3, yi.zhang
For example, 2 * wake_batch tags are put, while only wake_batch threads
are woken:
__sbq_wake_up
atomic_cmpxchg -> reset wait_cnt
__sbq_wake_up -> decrease wait_cnt
...
__sbq_wake_up -> wait_cnt is decreased to 0 again
atomic_cmpxchg
sbq_index_atomic_inc -> increase wake_index
wake_up_nr -> wake up and waitqueue might be empty
sbq_index_atomic_inc -> increase again, one waitqueue is skipped
wake_up_nr -> invalid wake up because old wakequeue might be empty
To fix the problem, increasing 'wake_index' before resetting 'wait_cnt'.
Fixes: 88459642cba4 ("blk-mq: abstract tag allocation out into sbitmap library")
Signed-off-by: Yu Kuai <yukuai3@huawei.com>
---
lib/sbitmap.c | 43 ++++++++++++++++++++-----------------------
1 file changed, 20 insertions(+), 23 deletions(-)
diff --git a/lib/sbitmap.c b/lib/sbitmap.c
index 4a230e5baacf..00ddb6593eff 100644
--- a/lib/sbitmap.c
+++ b/lib/sbitmap.c
@@ -615,34 +615,31 @@ static bool __sbq_wake_up(struct sbitmap_queue *sbq)
return false;
wait_cnt = atomic_dec_return(&ws->wait_cnt);
- if (wait_cnt <= 0) {
- int ret;
+ if (wait_cnt > 0)
+ return false;
- wake_batch = READ_ONCE(sbq->wake_batch);
+ /*
+ * For concurrent callers of this, callers should call this function
+ * again to wakeup a new batch on a different 'ws'.
+ */
+ if (wait_cnt < 0)
+ return true;
- /*
- * Pairs with the memory barrier in sbitmap_queue_resize() to
- * ensure that we see the batch size update before the wait
- * count is reset.
- */
- smp_mb__before_atomic();
+ wake_batch = READ_ONCE(sbq->wake_batch);
- /*
- * For concurrent callers of this, the one that failed the
- * atomic_cmpxhcg() race should call this function again
- * to wakeup a new batch on a different 'ws'.
- */
- ret = atomic_cmpxchg(&ws->wait_cnt, wait_cnt, wake_batch);
- if (ret == wait_cnt) {
- sbq_index_atomic_inc(&sbq->wake_index);
- wake_up_nr(&ws->wait, wake_batch);
- return false;
- }
+ /*
+ * Pairs with the memory barrier in sbitmap_queue_resize() to
+ * ensure that we see the batch size update before the wait
+ * count is reset.
+ */
+ smp_mb__before_atomic();
- return true;
- }
+ /* increase wake_index first to prevent possible lost wakeups */
+ sbq_index_atomic_inc(&sbq->wake_index);
+ atomic_set(&ws->wait_cnt, wake_batch);
+ wake_up_nr(&ws->wait, wake_batch);
- return false;
+ return true;
}
void sbitmap_queue_wake_up(struct sbitmap_queue *sbq)
--
2.31.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-06-19 7:50 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-19 8:03 [PATCH RFC v2 0/2] bugfix for sbitmap Yu Kuai
2022-06-19 8:03 ` [PATCH RFC v2 1/2] sbitmap: fix that same waitqueue can be woken up continuously Yu Kuai
2022-06-19 8:03 ` [PATCH RFC v2 2/2] sbitmap: fix possible io hung due to lost wakeups Yu Kuai
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).