From: "yukuai (C)" <yukuai3@huawei.com>
To: <axboe@kernel.dk>, <ming.lei@redhat.com>,
<andriy.shevchenko@linux.intel.com>, <john.garry@huawei.com>,
<bvanassche@acm.org>
Cc: <linux-block@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<yi.zhang@huawei.com>
Subject: Re: [PATCH RFC -next 0/3] improve fairness for sbitmap waitqueues
Date: Fri, 25 Mar 2022 15:30:02 +0800 [thread overview]
Message-ID: <d91e88e0-cd40-1670-2542-390dd4fb5ddf@huawei.com> (raw)
In-Reply-To: <20220318082505.3025427-1-yukuai3@huawei.com>
friendly ping ...
在 2022/03/18 16:25, Yu Kuai 写道:
> During some io test, I found that waitqueues can be extremly unbalanced,
> especially when tags are little.
>
> For example:
> test cmd: nr_requests is set to 64, and queue_depth is set to 32
> [global]
> filename=/dev/sdh
> ioengine=libaio
> direct=1
> allow_mounted_write=0
> group_reporting
>
> [test]
> rw=randwrite
> bs=4k
> numjobs=512
> iodepth=2
>
> With patch 1 applied, I observe the following status:
> ws_active=484
> ws={
> {.wait_cnt=8, .waiters_cnt=117},
> {.wait_cnt=8, .waiters_cnt=59},
> {.wait_cnt=8, .waiters_cnt=76},
> {.wait_cnt=8, .waiters_cnt=0},
> {.wait_cnt=5, .waiters_cnt=24},
> {.wait_cnt=8, .waiters_cnt=12},
> {.wait_cnt=8, .waiters_cnt=21},
> {.wait_cnt=8, .waiters_cnt=175},
> }
>
> 'waiters_cnt' means how many threads are waitng for tags in the 'ws',
> and such extremely unbalanced status is very frequent. After reading the
> sbitmap code, I found there are two situations that might cause the
> problem:
>
> 1) blk_mq_get_tag() can call 'bt_wait_ptr()' while the threads might get
> tag successfully before going to wait. - patch 2
>
> 2) After a 'ws' is woken up, following blk_mq_put_tag() might wake up
> the same 'ws' again instead of the next one. - patch 3
>
> I'm not sure if the unbalanced status is really a *problem* and need to
> be fixed, this patchset is just to improve fairness and not a thorough
> fix. Any comments and suggestions are welcome.
>
> Yu Kuai (3):
> sbitmap: record the number of waiters for each waitqueue
> blk-mq: call 'bt_wait_ptr()' later in blk_mq_get_tag()
> sbitmap: improve the fairness of waitqueues' wake up
>
> block/blk-mq-tag.c | 6 ++---
> include/linux/sbitmap.h | 5 ++++
> lib/sbitmap.c | 57 ++++++++++++++++++++++-------------------
> 3 files changed, 39 insertions(+), 29 deletions(-)
>
next prev parent reply other threads:[~2022-03-25 7:30 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-18 8:25 [PATCH RFC -next 0/3] improve fairness for sbitmap waitqueues Yu Kuai
2022-03-18 8:25 ` [PATCH RFC -next 1/3] sbitmap: record the number of waiters for each waitqueue Yu Kuai
2022-03-18 8:25 ` [PATCH RFC -next 2/3] blk-mq: call 'bt_wait_ptr()' later in blk_mq_get_tag() Yu Kuai
2022-03-18 8:25 ` [PATCH RFC -next 3/3] sbitmap: improve the fairness of waitqueues' wake up Yu Kuai
2022-03-25 7:30 ` yukuai (C) [this message]
2022-04-01 3:43 ` [PATCH RFC -next 0/3] improve fairness for sbitmap waitqueues yukuai (C)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=d91e88e0-cd40-1670-2542-390dd4fb5ddf@huawei.com \
--to=yukuai3@huawei.com \
--cc=andriy.shevchenko@linux.intel.com \
--cc=axboe@kernel.dk \
--cc=bvanassche@acm.org \
--cc=john.garry@huawei.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=yi.zhang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).