From: Ming Lei <ming.lei@redhat.com>
To: Jens Axboe <axboe@fb.com>,
linux-block@vger.kernel.org,
Christoph Hellwig <hch@infradead.org>,
Mike Snitzer <snitzer@redhat.com>,
dm-devel@redhat.com
Cc: Bart Van Assche <bart.vanassche@sandisk.com>,
Laurence Oberman <loberman@redhat.com>,
Paolo Valente <paolo.valente@linaro.org>,
Oleksandr Natalenko <oleksandr@natalenko.name>,
Tom Nguyen <tom81094@gmail.com>,
linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org,
Omar Sandoval <osandov@fb.com>, Ming Lei <ming.lei@redhat.com>
Subject: [PATCH V5 00/14] blk-mq-sched: improve sequential I/O performance(part 1)
Date: Sat, 30 Sep 2017 18:27:13 +0800 [thread overview]
Message-ID: <20170930102720.30219-1-ming.lei@redhat.com> (raw)
Hi Jens,
In Red Hat internal storage test wrt. blk-mq scheduler, we
found that I/O performance is much bad with mq-deadline, especially
about sequential I/O on some multi-queue SCSI devcies(lpfc, qla2xxx,
SRP...)
Turns out one big issue causes the performance regression: requests
are still dequeued from sw queue/scheduler queue even when ldd's
queue is busy, so I/O merge becomes quite difficult to make, then
sequential IO degrades a lot.
This issue becomes one of mains reasons for reverting default SCSI_MQ
in V4.13.
The 1st patch takes direct issue in blk_mq_request_bypass_insert(),
then we can improve dm-mpath's performance in part 2, which will
be posted out soon.
The 2nd six patches improve this situation, and brings back
some performance loss.
With this change, SCSI-MQ sequential I/O performance is
improved much, Paolo reported that mq-deadline performance
improved much[2] in his dbench test wrt V2. Also performanc
improvement on lpfc/qla2xx was observed with V1.[1]
Please consider it for V4.15.
[1] http://marc.info/?l=linux-block&m=150151989915776&w=2
[2] https://marc.info/?l=linux-block&m=150217980602843&w=2
V5:
- address some comments from Omar
- add Tested-by & Reveiewed-by tag
- use direct issue for blk_mq_request_bypass_insert(), and
start to consider to improve sequential I/O for dm-mpath
- only include part 1(the original patch 1 ~ 6), as suggested
by Omar
V4:
- add Reviewed-by tag
- some trival change: typo fix in commit log or comment,
variable name, no actual functional change
V3:
- totally round robin for picking req from ctx, as suggested
by Bart
- remove one local variable in __sbitmap_for_each_set()
- drop patches of single dispatch list, which can improve
performance on mq-deadline, but cause a bit degrade on
none because all hctxs need to be checked after ->dispatch
is flushed. Will post it again once it is mature.
- rebase on v4.13-rc6 with block for-next
V2:
- dequeue request from sw queues in round roubin's style
as suggested by Bart, and introduces one helper in sbitmap
for this purpose
- improve bio merge via hash table from sw queue
- add comments about using DISPATCH_BUSY state in lockless way,
simplifying handling on busy state,
- hold ctx->lock when clearing ctx busy bit as suggested
by Bart
Ming Lei (7):
blk-mq: issue rq directly in blk_mq_request_bypass_insert()
blk-mq-sched: fix scheduler bad performance
sbitmap: introduce __sbitmap_for_each_set()
blk-mq: introduce blk_mq_dequeue_from_ctx()
blk-mq-sched: move actual dispatching into one helper
blk-mq-sched: improve dispatching from sw queue
blk-mq-sched: don't dequeue request until all in ->dispatch are
flushed
block/blk-core.c | 3 +-
block/blk-mq-debugfs.c | 1 +
block/blk-mq-sched.c | 104 ++++++++++++++++++++++++++++++++++++-------
block/blk-mq.c | 114 +++++++++++++++++++++++++++++++++++++++---------
block/blk-mq.h | 4 +-
drivers/md/dm-rq.c | 2 +-
include/linux/blk-mq.h | 3 ++
include/linux/sbitmap.h | 64 +++++++++++++++++++--------
8 files changed, 238 insertions(+), 57 deletions(-)
--
2.9.5
next reply other threads:[~2017-09-30 10:27 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-30 10:27 Ming Lei [this message]
2017-09-30 10:27 ` [PATCH V5 1/7] blk-mq: issue rq directly in blk_mq_request_bypass_insert() Ming Lei
2017-10-03 8:58 ` Christoph Hellwig
2017-10-03 13:39 ` Ming Lei
2017-09-30 10:27 ` [PATCH V5 2/7] blk-mq-sched: fix scheduler bad performance Ming Lei
2017-10-02 14:19 ` Christoph Hellwig
2017-09-30 10:27 ` [PATCH V5 3/7] sbitmap: introduce __sbitmap_for_each_set() Ming Lei
2017-09-30 10:27 ` [PATCH V5 4/7] blk-mq: introduce blk_mq_dequeue_from_ctx() Ming Lei
2017-10-03 9:01 ` Christoph Hellwig
2017-10-09 4:36 ` Ming Lei
2017-09-30 10:27 ` [PATCH V5 5/7] blk-mq-sched: move actual dispatching into one helper Ming Lei
2017-10-02 14:19 ` Christoph Hellwig
2017-10-09 9:07 ` Ming Lei
2017-09-30 10:27 ` [PATCH V5 6/7] blk-mq-sched: improve dispatching from sw queue Ming Lei
2017-10-03 9:05 ` Christoph Hellwig
2017-10-09 10:15 ` Ming Lei
2017-09-30 10:27 ` [PATCH V5 7/7] blk-mq-sched: don't dequeue request until all in ->dispatch are flushed Ming Lei
2017-10-03 9:11 ` Christoph Hellwig
2017-10-09 10:40 ` Ming Lei
2017-09-30 10:32 ` [PATCH V5 00/14] blk-mq-sched: improve sequential I/O performance(part 1) Ming Lei
2017-10-09 12:09 ` John Garry
2017-10-09 12:09 ` John Garry
2017-10-09 15:04 ` Ming Lei
2017-10-10 1:46 ` Ming Lei
2017-10-10 12:24 ` John Garry
2017-10-10 12:24 ` John Garry
2017-10-10 12:34 ` Johannes Thumshirn
2017-10-10 12:34 ` Johannes Thumshirn
2017-10-10 12:37 ` Paolo Valente
2017-10-10 12:37 ` Paolo Valente
2017-10-10 13:45 ` Ming Lei
2017-10-10 15:10 ` John Garry
2017-10-10 15:10 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170930102720.30219-1-ming.lei@redhat.com \
--to=ming.lei@redhat.com \
--cc=axboe@fb.com \
--cc=bart.vanassche@sandisk.com \
--cc=dm-devel@redhat.com \
--cc=hch@infradead.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=loberman@redhat.com \
--cc=oleksandr@natalenko.name \
--cc=osandov@fb.com \
--cc=paolo.valente@linaro.org \
--cc=snitzer@redhat.com \
--cc=tom81094@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.