All of lore.kernel.org
 help / color / mirror / Atom feed
From: Kemeng Shi <shikemeng@huaweicloud.com>
To: tj@kernel.org, josef@toxicpanda.com, axboe@kernel.dk
Cc: cgroups@vger.kernel.org, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org, shikemeng@huawei.com,
	linfeilong@huawei.com, liuzhiqiang26@huawei.com
Subject: [PATCH v3 2/9] blk-throttle: Fix that bps of child could exceed bps limited in parent
Date: Mon,  5 Dec 2022 19:57:02 +0800	[thread overview]
Message-ID: <20221205115709.251489-3-shikemeng@huaweicloud.com> (raw)
In-Reply-To: <20221205115709.251489-1-shikemeng@huaweicloud.com>

From: Kemeng Shi <shikemeng@huawei.com>

Consider situation as following (on the default hierarchy):
 HDD
  |
root (bps limit: 4k)
  |
child (bps limit :8k)
  |
fio bs=8k
Rate of fio is supposed to be 4k, but result is 8k. Reason is as
following:
Size of single IO from fio is larger than bytes allowed in one
throtl_slice in child, so IOs are always queued in child group first.
When queued IOs in child are dispatched to parent group, BIO_BPS_THROTTLED
is set and these IOs will not be limited by tg_within_bps_limit anymore.
Fix this by only set BIO_BPS_THROTTLED when the bio traversed the entire
tree.

There patch has no influence on situation which is not on the default
hierarchy as each group is a single root group without parent.

Signed-off-by: Kemeng Shi <shikemeng@huawei.com>
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Kemeng Shi <shikemeng@huaweicloud.com>
---
 block/blk-throttle.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 8e2349b17936..2444ebf5f11d 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -1067,7 +1067,6 @@ static void tg_dispatch_one_bio(struct throtl_grp *tg, bool rw)
 	sq->nr_queued[rw]--;
 
 	throtl_charge_bio(tg, bio);
-	bio_set_flag(bio, BIO_BPS_THROTTLED);
 
 	/*
 	 * If our parent is another tg, we just need to transfer @bio to
@@ -1080,6 +1079,7 @@ static void tg_dispatch_one_bio(struct throtl_grp *tg, bool rw)
 		throtl_add_bio_tg(bio, &tg->qnode_on_parent[rw], parent_tg);
 		start_parent_slice_with_credit(tg, parent_tg, rw);
 	} else {
+		bio_set_flag(bio, BIO_BPS_THROTTLED);
 		throtl_qnode_add_bio(bio, &tg->qnode_on_parent[rw],
 				     &parent_sq->queued[rw]);
 		BUG_ON(tg->td->nr_queued[rw] <= 0);
-- 
2.30.0


WARNING: multiple messages have this Message-ID (diff)
From: Kemeng Shi <shikemeng-XF6JlduFytWkHkcT6e4Xnw@public.gmane.org>
To: tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org,
	josef-DigfWCa+lFGyeJad7bwFQA@public.gmane.org,
	axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org
Cc: cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	shikemeng-hv44wF8Li93QT0dZR+AlfA@public.gmane.org,
	linfeilong-hv44wF8Li93QT0dZR+AlfA@public.gmane.org,
	liuzhiqiang26-hv44wF8Li93QT0dZR+AlfA@public.gmane.org
Subject: [PATCH v3 2/9] blk-throttle: Fix that bps of child could exceed bps limited in parent
Date: Mon,  5 Dec 2022 19:57:02 +0800	[thread overview]
Message-ID: <20221205115709.251489-3-shikemeng@huaweicloud.com> (raw)
In-Reply-To: <20221205115709.251489-1-shikemeng-XF6JlduFytWkHkcT6e4Xnw@public.gmane.org>

From: Kemeng Shi <shikemeng-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>

Consider situation as following (on the default hierarchy):
 HDD
  |
root (bps limit: 4k)
  |
child (bps limit :8k)
  |
fio bs=8k
Rate of fio is supposed to be 4k, but result is 8k. Reason is as
following:
Size of single IO from fio is larger than bytes allowed in one
throtl_slice in child, so IOs are always queued in child group first.
When queued IOs in child are dispatched to parent group, BIO_BPS_THROTTLED
is set and these IOs will not be limited by tg_within_bps_limit anymore.
Fix this by only set BIO_BPS_THROTTLED when the bio traversed the entire
tree.

There patch has no influence on situation which is not on the default
hierarchy as each group is a single root group without parent.

Signed-off-by: Kemeng Shi <shikemeng-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Acked-by: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Signed-off-by: Kemeng Shi <shikemeng-XF6JlduFytWkHkcT6e4Xnw@public.gmane.org>
---
 block/blk-throttle.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 8e2349b17936..2444ebf5f11d 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -1067,7 +1067,6 @@ static void tg_dispatch_one_bio(struct throtl_grp *tg, bool rw)
 	sq->nr_queued[rw]--;
 
 	throtl_charge_bio(tg, bio);
-	bio_set_flag(bio, BIO_BPS_THROTTLED);
 
 	/*
 	 * If our parent is another tg, we just need to transfer @bio to
@@ -1080,6 +1079,7 @@ static void tg_dispatch_one_bio(struct throtl_grp *tg, bool rw)
 		throtl_add_bio_tg(bio, &tg->qnode_on_parent[rw], parent_tg);
 		start_parent_slice_with_credit(tg, parent_tg, rw);
 	} else {
+		bio_set_flag(bio, BIO_BPS_THROTTLED);
 		throtl_qnode_add_bio(bio, &tg->qnode_on_parent[rw],
 				     &parent_sq->queued[rw]);
 		BUG_ON(tg->td->nr_queued[rw] <= 0);
-- 
2.30.0


  parent reply	other threads:[~2022-12-05  3:58 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-12-05 11:57 [PATCH v3 0/9] A few bugfix and cleanup patches for blk-throttle Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 1/9] blk-throttle: correct stale comment in throtl_pd_init Kemeng Shi
2022-12-05 20:43   ` Jens Axboe
2022-12-05 20:43     ` Jens Axboe
2022-12-06  1:17     ` Kemeng Shi
2022-12-05 11:57 ` Kemeng Shi [this message]
2022-12-05 11:57   ` [PATCH v3 2/9] blk-throttle: Fix that bps of child could exceed bps limited in parent Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 3/9] blk-throttle: ignore cgroup without io queued in blk_throtl_cancel_bios Kemeng Shi
2022-12-05 11:57   ` Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 4/9] blk-throttle: correct calculation of wait time in tg_may_dispatch Kemeng Shi
2022-12-05 11:57   ` Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 5/9] blk-throttle: simpfy low limit reached check in throtl_tg_can_upgrade Kemeng Shi
2022-12-05 11:57   ` Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 6/9] blk-throttle: fix typo in comment of throtl_adjusted_limit Kemeng Shi
2022-12-05 11:57   ` Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 7/9] blk-throttle: remove incorrect comment for tg_last_low_overflow_time Kemeng Shi
2022-12-05 11:57   ` Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 8/9] blk-throttle: remove repeat check of elapsed time from last upgrade in throtl_hierarchy_can_downgrade Kemeng Shi
2022-12-05 11:57   ` Kemeng Shi
2022-12-05 11:57 ` [PATCH v3 9/9] blk-throttle: Use more siutable time_after check for update of slice_start Kemeng Shi
2022-12-05 20:46 ` [PATCH v3 0/9] A few bugfix and cleanup patches for blk-throttle Jens Axboe
2022-12-05 20:46   ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221205115709.251489-3-shikemeng@huaweicloud.com \
    --to=shikemeng@huaweicloud.com \
    --cc=axboe@kernel.dk \
    --cc=cgroups@vger.kernel.org \
    --cc=josef@toxicpanda.com \
    --cc=linfeilong@huawei.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=liuzhiqiang26@huawei.com \
    --cc=shikemeng@huawei.com \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.