From: Keith Busch <kbusch@kernel.org>
To: "Ober, Frank" <frank.ober@intel.com>
Cc: "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"Rajendiran, Swetha" <swetha.rajendiran@intel.com>,
"Liang, Mark" <mark.liang@intel.com>,
"Derrick, Jonathan" <jonathan.derrick@intel.com>,
"linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>
Subject: Re: Polled io for Linux kernel 5.x
Date: Fri, 20 Dec 2019 05:52:10 +0900 [thread overview]
Message-ID: <20191219205210.GA26490@redsun51.ssa.fujisawa.hgst.com> (raw)
In-Reply-To: <SN6PR11MB2669E7A65DD0AD9DC65A67C58B520@SN6PR11MB2669.namprd11.prod.outlook.com>
On Thu, Dec 19, 2019 at 07:25:51PM +0000, Ober, Frank wrote:
> Hi block/nvme communities,
> On 4.x kernels we used to be able to do:
> # echo 1 > /sys/block/nvme0n1/queue/io_poll
> And then run a polled_io job in fio with pvsync2 as our ioengine, with the hipri flag set. This is actually how we test the very best SSDs that depend on 3D xpoint media.
>
> On 5.x kernels we see the following error trying to write the device settings>>>
> -bash: echo: write error: Invalid argument
>
> We can reload the entire nvme module with nvme poll_queues but this is not well explained or written up anywhere? Or sorry "not found"?
>
> This is verifiable on 5.3, 5.4 kernels with fio 3.16 builds.
>
> What is the background on what has changed because Jens wrote this note back in 2015, which did work in the 4.x kernel era.
The original polling implementation shared resources that generate
interrupts. This prevents it from running as fast as it can, so
dedicated polling queues are used now.
> How come we cannot have a device/controller level setup of polled io today in 5.x kernels, all that exists is module based?
Polled queues are a dedicated resource that we have to reserve up front.
They're optional, so you don't need to use the hipri flag if you have a
device you don't want polled. But we need to know how many queues to
reserve before we've even discovered the controllers, so we don't have a
good way to define it per-controller.
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
next prev parent reply other threads:[~2019-12-19 20:52 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-19 19:25 Polled io for Linux kernel 5.x Ober, Frank
2019-12-19 20:52 ` Keith Busch [this message]
2019-12-19 21:59 ` Ober, Frank
2019-12-20 21:20 ` Keith Busch
2019-12-31 19:06 ` Ober, Frank
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191219205210.GA26490@redsun51.ssa.fujisawa.hgst.com \
--to=kbusch@kernel.org \
--cc=frank.ober@intel.com \
--cc=jonathan.derrick@intel.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=mark.liang@intel.com \
--cc=swetha.rajendiran@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).