From: Jens Axboe <axboe@fb.com>
To: Hannes Reinecke <hare@suse.de>
Cc: <linux-kernel@vger.kernel.org>, <linux-block@vger.kernel.org>,
<osandov@osandov.com>, <bart.vanassche@sandisk.com>
Subject: Re: [PATCHSET v6] blk-mq scheduling framework
Date: Fri, 13 Jan 2017 08:05:11 -0700 [thread overview]
Message-ID: <20170113150511.GD23338@kernel.dk> (raw)
In-Reply-To: <cab72b22-d672-3c68-ec4a-6e3cb65f93ab@suse.de>
On Fri, Jan 13 2017, Hannes Reinecke wrote:
> On 01/13/2017 12:04 PM, Hannes Reinecke wrote:
> > On 01/13/2017 09:15 AM, Hannes Reinecke wrote:
> >> On 01/11/2017 10:39 PM, Jens Axboe wrote:
> >>> Another year, another posting of this patchset. The previous posting
> >>> was here:
> >>>
> >>> https://www.spinics.net/lists/kernel/msg2406106.html
> >>>
> >>> (yes, I've skipped v5, it was fixes on top of v4, not the rework).
> >>>
> >>> I've reworked bits of this to get rid of the shadow requests, thanks
> >>> to Bart for the inspiration. The missing piece, for me, was the fact
> >>> that we have the tags->rqs[] indirection array already. I've done this
> >>> somewhat differently, though, by having the internal scheduler tag
> >>> map be allocated/torn down when an IO scheduler is attached or
> >>> detached. This also means that when we run without a scheduler, we
> >>> don't have to do double tag allocations, it'll work like before.
> >>>
> >>> The patchset applies on top of 4.10-rc3, or can be pulled here:
> >>>
> >>> git://git.kernel.dk/linux-block blk-mq-sched.6
> >>>
> >> Well ... something's wrong here on my machine:
> >>
> [ .. ]
>
> Turns out that selecting CONFIG_DEFAULT_MQ_DEADLINE is the culprit;
> switching to CONFIG_DEFAULT_MQ_NONE and selecting mq-deadline after
> booting manually makes the problem go away.
>
> So there is a race condition during device init and switching the I/O
> scheduler.
>
> But the results from using mq-deadline are promising; the performance
> drop I've seen on older hardware seems to be resolved:
>
> mq iosched:
> seq read : io=13383MB, bw=228349KB/s, iops=57087
> rand read : io=12876MB, bw=219709KB/s, iops=54927
> seq write: io=14532MB, bw=247987KB/s, iops=61996
> rand write: io=13779MB, bw=235127KB/s, iops=58781
> mq default:
> seq read : io=13056MB, bw=222588KB/s, iops=55647
> rand read : io=12908MB, bw=220069KB/s, iops=55017
> seq write: io=13986MB, bw=238444KB/s, iops=59611
> rand write: io=13733MB, bw=234128KB/s, iops=58532
> sq default:
> seq read : io=10240MB, bw=194787KB/s, iops=48696
> rand read : io=10240MB, bw=191374KB/s, iops=47843
> seq write: io=10240MB, bw=245333KB/s, iops=61333
> rand write: io=10240MB, bw=228239KB/s, iops=57059
>
> measured on mpt2sas with SSD devices.
Perfect! Straight on the path of kill of non scsi-mq, then.
I'll fix up the async scan issue. The new mq schedulers don't really
behave differently in this regard, so I'm a bit puzzled. Hopefully it
reproduces here.
--
Jens Axboe
next prev parent reply other threads:[~2017-01-13 15:05 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-11 21:39 [PATCHSET v6] blk-mq scheduling framework Jens Axboe
2017-01-11 21:39 ` [PATCH 01/10] block: move existing elevator ops to union Jens Axboe
2017-01-12 10:15 ` Johannes Thumshirn
2017-01-12 21:17 ` Bart Van Assche
2017-01-13 8:34 ` Christoph Hellwig
2017-01-13 15:00 ` Jens Axboe
2017-01-11 21:39 ` [PATCH 02/10] blk-mq: make mq_ops a const pointer Jens Axboe
2017-01-12 10:14 ` Johannes Thumshirn
2017-01-13 8:16 ` Christoph Hellwig
2017-01-11 21:39 ` [PATCH 03/10] block: move rq_ioc() to blk.h Jens Axboe
2017-01-12 10:14 ` Johannes Thumshirn
2017-01-12 21:18 ` Bart Van Assche
2017-01-13 8:33 ` Christoph Hellwig
2017-01-11 21:39 ` [PATCH 04/10] blk-mq: un-export blk_mq_free_hctx_request() Jens Axboe
2017-01-12 10:13 ` Johannes Thumshirn
2017-01-12 21:18 ` Bart Van Assche
2017-01-13 8:16 ` Christoph Hellwig
2017-01-11 21:39 ` [PATCH 05/10] blk-mq: export some helpers we need to the scheduling framework Jens Axboe
2017-01-12 10:17 ` Johannes Thumshirn
2017-01-12 21:20 ` Bart Van Assche
2017-01-13 8:17 ` Christoph Hellwig
2017-01-13 15:01 ` Jens Axboe
2017-01-11 21:39 ` [PATCH 06/10] blk-mq-tag: cleanup the normal/reserved tag allocation Jens Axboe
2017-01-12 21:22 ` Bart Van Assche
2017-01-12 22:07 ` Jens Axboe
2017-01-13 8:30 ` Christoph Hellwig
2017-01-13 15:06 ` Jens Axboe
2017-01-11 21:40 ` [PATCH 07/10] blk-mq: abstract out helpers for allocating/freeing tag maps Jens Axboe
2017-01-12 21:29 ` Bart Van Assche
2017-01-12 21:54 ` Jens Axboe
2017-01-13 8:25 ` Johannes Thumshirn
2017-01-11 21:40 ` [PATCH 08/10] blk-mq-sched: add framework for MQ capable IO schedulers Jens Axboe
2017-01-12 21:45 ` Bart Van Assche
2017-01-12 21:59 ` Jens Axboe
2017-01-13 11:15 ` Hannes Reinecke
2017-01-13 16:39 ` Bart Van Assche
2017-01-13 16:41 ` Omar Sandoval
2017-01-13 17:43 ` Hannes Reinecke
2017-01-11 21:40 ` [PATCH 09/10] mq-deadline: add blk-mq adaptation of the deadline IO scheduler Jens Axboe
2017-01-12 21:53 ` Bart Van Assche
2017-01-11 21:40 ` [PATCH 10/10] blk-mq-sched: allow setting of default " Jens Axboe
2017-01-12 21:54 ` Bart Van Assche
2017-01-12 21:16 ` [PATCHSET v6] blk-mq scheduling framework Bart Van Assche
2017-01-13 8:15 ` Hannes Reinecke
2017-01-13 11:04 ` Hannes Reinecke
2017-01-13 12:10 ` Hannes Reinecke
2017-01-13 15:05 ` Jens Axboe [this message]
2017-01-13 15:03 ` Jens Axboe
2017-01-13 15:23 ` Jens Axboe
2017-01-13 15:33 ` Hannes Reinecke
2017-01-13 15:34 ` Jens Axboe
2017-01-13 15:59 ` Hannes Reinecke
2017-01-13 16:00 ` Jens Axboe
2017-01-13 16:02 ` Jens Axboe
2017-01-13 21:45 ` Jens Axboe
2017-01-16 8:11 ` Hannes Reinecke
2017-01-16 15:12 ` Jens Axboe
2017-01-16 15:16 ` Jens Axboe
2017-01-16 15:47 ` Jens Axboe
2017-01-13 10:09 ` Hannes Reinecke
2017-01-15 10:12 ` Paolo Valente
2017-01-15 15:55 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170113150511.GD23338@kernel.dk \
--to=axboe@fb.com \
--cc=bart.vanassche@sandisk.com \
--cc=hare@suse.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=osandov@osandov.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).