From: Yu Kuai <yukuai1@huaweicloud.com>
To: bvanassche@acm.org, hch@lst.de, kbusch@kernel.org,
ming.lei@redhat.com, axboe@kernel.dk
Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
yukuai3@huawei.com, yukuai1@huaweicloud.com, yi.zhang@huawei.com,
yangerkun@huawei.com
Subject: [PATCH RFC v2 0/8] blk-mq: improve tag fair sharing
Date: Sat, 21 Oct 2023 23:47:58 +0800 [thread overview]
Message-ID: <20231021154806.4019417-1-yukuai1@huaweicloud.com> (raw)
From: Yu Kuai <yukuai3@huawei.com>
Current implementation:
- a counter active_queues record how many queue/hctx is sharing tags,
and it's updated while issue new IO, and cleared in
blk_mq_timeout_work().
- if active_queues is more than 1, then tags is fair shared to each
node;
New implementation:
- a new field 'available_tags' is added to each node, and it's
calculate in slow path, hence fast path won't be affected, patch 5;
- a new counter 'busy_queues' is added to blk_mq_tags, and it's updated
while fail to get driver tag, and it's also cleared in
blk_mq_timeout_work(), and tag sharing will based on 'busy_queues'
instead of 'active_queues', patch 6,7;
- a new counter 'busy_count' is added to each node to record how many
times a node failed to get driver tag, and it's used to judge if a node
is busy and need more tags, patch 8;
- a new timer is added to blk_mq_tags, it will start if any node failed
to get driver tag, and timer function will be used to borrow tags and
return borrowed tags, patch 8;
A simple test, 32 tags with two shared node:
[global]
ioengine=libaio
iodepth=2
bs=4k
direct=1
rw=randrw
group_reporting
[sda]
numjobs=32
filename=/dev/sda
[sdb]
numjobs=1
filename=/dev/sdb
Test result(monitor new debugfs entry):
time active available
sda sdb sda sdb
0 0 0 32 32
1 16 2 16 16 -> start fair sharing
2 19 2 20 16
3 24 2 24 16
4 26 2 28 16 -> borrow 32/8=4 tags each round
5 28 2 28 16 -> save at lease 4 tags for sdb
Yu Kuai (8):
blk-mq: factor out a structure from blk_mq_tags
blk-mq: factor out a structure to store information for tag sharing
blk-mq: add a helper to initialize shared_tag_info
blk-mq: support to track active queues from blk_mq_tags
blk-mq: precalculate available tags for hctx_may_queue()
blk-mq: add new helpers blk_mq_driver_tag_busy/idle()
blk-mq-tag: delay tag sharing until fail to get driver tag
blk-mq-tag: allow shared queue/hctx to get more driver tags
block/blk-core.c | 2 -
block/blk-mq-debugfs.c | 30 +++++-
block/blk-mq-tag.c | 226 +++++++++++++++++++++++++++++++++++++++--
block/blk-mq.c | 12 ++-
block/blk-mq.h | 64 +++++++-----
include/linux/blk-mq.h | 36 +++++--
include/linux/blkdev.h | 11 +-
7 files changed, 328 insertions(+), 53 deletions(-)
--
2.39.2
next reply other threads:[~2023-10-21 7:52 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-21 15:47 Yu Kuai [this message]
2023-10-21 15:47 ` [PATCH RFC v2 1/8] blk-mq: factor out a structure from blk_mq_tags Yu Kuai
2023-10-21 15:48 ` [PATCH RFC v2 2/8] blk-mq: factor out a structure to store information for tag sharing Yu Kuai
2023-10-21 15:48 ` [PATCH RFC v2 3/8] blk-mq: add a helper to initialize shared_tag_info Yu Kuai
2023-10-21 15:48 ` [PATCH RFC v2 4/8] blk-mq: support to track active queues from blk_mq_tags Yu Kuai
2023-10-21 15:48 ` [PATCH RFC v2 5/8] blk-mq: precalculate available tags for hctx_may_queue() Yu Kuai
2023-10-21 15:48 ` [PATCH RFC v2 6/8] blk-mq: add new helpers blk_mq_driver_tag_busy/idle() Yu Kuai
2023-10-21 15:48 ` [PATCH RFC v2 7/8] blk-mq-tag: delay tag sharing until fail to get driver tag Yu Kuai
2023-10-21 15:48 ` [PATCH RFC v2 8/8] blk-mq-tag: allow shared queue/hctx to get more driver tags Yu Kuai
2023-10-23 3:24 ` kernel test robot
2023-10-23 20:46 ` Bart Van Assche
2023-10-24 1:07 ` Yu Kuai
2023-10-23 4:38 ` [PATCH RFC v2 0/8] blk-mq: improve tag fair sharing Ming Lei
2023-10-23 7:26 ` Yu Kuai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231021154806.4019417-1-yukuai1@huaweicloud.com \
--to=yukuai1@huaweicloud.com \
--cc=axboe@kernel.dk \
--cc=bvanassche@acm.org \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=yangerkun@huawei.com \
--cc=yi.zhang@huawei.com \
--cc=yukuai3@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.