From: Chengming Zhou <zhouchengming@bytedance.com>
To: axboe@kernel.dk
Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
zhouchengming@bytedance.com, stable@vger.kernel.org
Subject: [PATCH] blk-wbt: fix scale logic when disable wbt
Date: Fri, 23 Apr 2021 17:45:03 +0800 [thread overview]
Message-ID: <20210423094503.25733-1-zhouchengming@bytedance.com> (raw)
We encountered kernel crash when disable wbt through min_lat_nsec
setting to zero, found the problem is the reset of wb_max to zero in
calc_wb_limits() would break the normal scale logic, caused the
scale_step value overflow and kernel crash. Below is the crash backtrace:
[43061417.487135] task: ffff9250828d6540 task.stack: ffffbc8b839f0000
[43061417.487331] RIP: 0010:rwb_arm_timer+0x52/0x60
[43061417.487472] RSP: 0000:ffff9250bfec3ea8 EFLAGS: 00010206
[43061417.487646] RAX: 000000005f5e1000 RBX: ffff9250ab6113c0 RCX: 0000000000000000
[43061417.487877] RDX: 0000000000000000 RSI: ffffffff9fe4a484 RDI: 000000005f5e1000
[43061417.488109] RBP: 0000000000000100 R08: ffffffff00000000 R09: 00000000ffffffff
[43061417.488343] R10: 0000000000000000 R11: ffffdc8b3fdcf938 R12: ffff9250a9324d90
[43061417.488575] R13: ffffffff9f3583a0 R14: ffff9250a9324d80 R15: 0000000000000000
[43061417.488808] FS: 00007f7aadbee700(0000) GS:ffff9250bfec0000(0000) knlGS:0000000000000000
[43061417.489069] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[43061417.489258] CR2: 00007f43b7c809b8 CR3: 0000007e42994006 CR4: 00000000007606e0
[43061417.489490] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[43061417.489722] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[43061417.489952] PKRU: 55555554
[43061417.490046] Call Trace:
[43061417.490136] <IRQ>
[43061417.490206] call_timer_fn+0x2e/0x130
[43061417.490328] run_timer_softirq+0x1d4/0x420
[43061417.490466] ? timerqueue_add+0x54/0x80
[43061417.490593] ? enqueue_hrtimer+0x38/0x80
[43061417.490722] __do_softirq+0x108/0x2a9
[43061417.490846] irq_exit+0xc2/0xd0
[43061417.490953] smp_apic_timer_interrupt+0x6c/0x120
[43061417.491106] apic_timer_interrupt+0x7d/0x90
[43061417.491245] </IRQ>
Seen from the crash dump, the scale_step became a very big value and
overflow to zero divisor in div_u64, so kernel crash happened.
Since wbt use wb_max == 1 and scaled_max flag as the scale min/max
point, we only reset wb_normal and wb_background when set min_lat_nsec
to zero, leave wb_max and scaled_max to be driven by the scale timer.
Higher version kernels than v4.18 include a code refactor patchset that
split the scale up/down logic and calc_wb_limits(), so disable wbt by
setting min_lat_nsec to zero will NOT affect the normal scale logic.
But we don't want to backport that patchset because of very big code
changes, may introduce other problems. So just fix the crash bug in
this patch.
Fixes: e34cbd307477 ("blk-wbt: add general throttling mechanism")
Cc: <stable@vger.kernel.org> # 4.9.x
Signed-off-by: Chengming Zhou <zhouchengming@bytedance.com>
---
block/blk-wbt.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/block/blk-wbt.c b/block/blk-wbt.c
index 5c105514bca7..24c84ee39029 100644
--- a/block/blk-wbt.c
+++ b/block/blk-wbt.c
@@ -194,11 +194,6 @@ static bool calc_wb_limits(struct rq_wb *rwb)
unsigned int depth;
bool ret = false;
- if (!rwb->min_lat_nsec) {
- rwb->wb_max = rwb->wb_normal = rwb->wb_background = 0;
- return false;
- }
-
/*
* For QD=1 devices, this is a special case. It's important for those
* to have one request ready when one completes, so force a depth of
@@ -244,6 +239,9 @@ static bool calc_wb_limits(struct rq_wb *rwb)
rwb->wb_background = (rwb->wb_max + 3) / 4;
}
+ if (!rwb->min_lat_nsec)
+ rwb->wb_normal = rwb->wb_background = 0;
+
return ret;
}
--
2.11.0
reply other threads:[~2021-04-23 9:45 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210423094503.25733-1-zhouchengming@bytedance.com \
--to=zhouchengming@bytedance.com \
--cc=axboe@kernel.dk \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).