From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8383C433F5 for ; Mon, 16 May 2022 01:30:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235072AbiEPBay (ORCPT ); Sun, 15 May 2022 21:30:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238900AbiEPBay (ORCPT ); Sun, 15 May 2022 21:30:54 -0400 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 322456421; Sun, 15 May 2022 18:30:50 -0700 (PDT) Received: from kwepemi500016.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4L1hVy2RBQz1JCFc; Mon, 16 May 2022 09:29:30 +0800 (CST) Received: from localhost.localdomain (10.175.127.227) by kwepemi500016.china.huawei.com (7.221.188.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 16 May 2022 09:30:48 +0800 From: Zhang Wensheng To: , CC: , , , Subject: [PATCH -next] block: fix io hung of setting throttle limit frequently Date: Mon, 16 May 2022 09:44:29 +0800 Message-ID: <20220516014429.33723-1-zhangwensheng5@huawei.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Content-Transfer-Encoding: 7BIT Content-Type: text/plain; charset=US-ASCII X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemi500016.china.huawei.com (7.221.188.220) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Our test find a io hung problem which could be simplified: setting throttle iops/bps limit to small, and to issue a big bio. if the io is limited to 10s, just wait 1s, continue to set same throttle iops/bps limit again, now, we could see that the new throttle time become 10s again, like this, if we distribute limit repeatedly within 10s, this io will always in throttle queue. when the throttle limit iops/bps is set to io. tg_conf_updated will be called, it will start a new slice and update a new dispatch time to pending timer which lead to wait again. Because of commit 9f5ede3c01f9 ("block: throttle split bio in case of iops limit"), the io will work fine if limited by bps. which could fix part of the problem, not the root cause. To fix this problem, adding the judge before update dispatch time. if the pending timer is alive, we should not to update time. Signed-off-by: Zhang Wensheng --- block/blk-throttle.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 469c483719be..8acb205dfa85 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -1321,12 +1321,14 @@ static void tg_conf_updated(struct throtl_grp *tg, bool global) * that a group's limit are dropped suddenly and we don't want to * account recently dispatched IO with new low rate. */ - throtl_start_new_slice(tg, READ); - throtl_start_new_slice(tg, WRITE); + if (!timer_pending(&sq->parent_sq->pending_timer)) { + throtl_start_new_slice(tg, READ); + throtl_start_new_slice(tg, WRITE); - if (tg->flags & THROTL_TG_PENDING) { - tg_update_disptime(tg); - throtl_schedule_next_dispatch(sq->parent_sq, true); + if (tg->flags & THROTL_TG_PENDING) { + tg_update_disptime(tg); + throtl_schedule_next_dispatch(sq->parent_sq, true); + } } } -- 2.31.1 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Zhang Wensheng Subject: [PATCH -next] block: fix io hung of setting throttle limit frequently Date: Mon, 16 May 2022 09:44:29 +0800 Message-ID: <20220516014429.33723-1-zhangwensheng5@huawei.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7BIT Return-path: List-ID: Content-Type: text/plain; charset="us-ascii" To: tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org Cc: linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, yukuai3-hv44wF8Li93QT0dZR+AlfA@public.gmane.org Our test find a io hung problem which could be simplified: setting throttle iops/bps limit to small, and to issue a big bio. if the io is limited to 10s, just wait 1s, continue to set same throttle iops/bps limit again, now, we could see that the new throttle time become 10s again, like this, if we distribute limit repeatedly within 10s, this io will always in throttle queue. when the throttle limit iops/bps is set to io. tg_conf_updated will be called, it will start a new slice and update a new dispatch time to pending timer which lead to wait again. Because of commit 9f5ede3c01f9 ("block: throttle split bio in case of iops limit"), the io will work fine if limited by bps. which could fix part of the problem, not the root cause. To fix this problem, adding the judge before update dispatch time. if the pending timer is alive, we should not to update time. Signed-off-by: Zhang Wensheng --- block/blk-throttle.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 469c483719be..8acb205dfa85 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -1321,12 +1321,14 @@ static void tg_conf_updated(struct throtl_grp *tg, bool global) * that a group's limit are dropped suddenly and we don't want to * account recently dispatched IO with new low rate. */ - throtl_start_new_slice(tg, READ); - throtl_start_new_slice(tg, WRITE); + if (!timer_pending(&sq->parent_sq->pending_timer)) { + throtl_start_new_slice(tg, READ); + throtl_start_new_slice(tg, WRITE); - if (tg->flags & THROTL_TG_PENDING) { - tg_update_disptime(tg); - throtl_schedule_next_dispatch(sq->parent_sq, true); + if (tg->flags & THROTL_TG_PENDING) { + tg_update_disptime(tg); + throtl_schedule_next_dispatch(sq->parent_sq, true); + } } } -- 2.31.1