Re: [PATCH] nvme: align io queue count with allocted nvme_queue in nvme_probe

From: weiping zhang <zhangweiping@didichuxing.com>
To: Christoph Hellwig <hch@infradead.org>
Cc: Jens Axboe <axboe@kernel.dk>, Weiping Zhang <zwp10758@gmail.com>,
	Weiping Zhang <zhangweiping@didiglobal.com>,
	linux-nvme@lists.infradead.org, Keith Busch <kbusch@kernel.org>,
	Max Gurtovoy <maxg@mellanox.com>,
	sagi@grimberg.me
Subject: Re: [PATCH] nvme: align io queue count with allocted nvme_queue in nvme_probe
Date: Wed, 22 Apr 2020 17:24:27 +0800	[thread overview]
Message-ID: <20200422092416.GA12930@192.168.3.9> (raw)
In-Reply-To: <20200422083747.GA26915@infradead.org>

On Wed, Apr 22, 2020 at 01:37:47AM -0700, Christoph Hellwig wrote:
> On Tue, Apr 14, 2020 at 03:59:12PM +0300, Max Gurtovoy wrote:
> > > > write_queues and poll queues shouldn't be writable IMO.
> > > > 
> > > I think we can keep it writeable, the user case is that setup as many io
> > > queues as possible when load nvme module, then change queue count
> > > for each tag set map dynamically.
> > 
> > We can keep it writable but I prefer not change the controller initial queue
> > count after reset controller operation.
> > 
> > So we can keep dev->write_queues and dev->poll_queues count for that.
> > 
> > You can use the writable param in case you aim to hotplug a new device and
> > you want it to probe with less/more queues.
> > 
> > IMO this feature should've somehow configured using nvme-cli as we do with
> > fabrics controllers that we never change this values after initial
> > connection.
> > 
> > Keith/Christoph,
> > 
> > what is the right approach in your opinion ?
> 
> The problem with PCIe is that we only have a per-controller interface
> once the controller is probed.  So a global paramter that can be
> changed, but only is sampled once at probe time seems the easiest to
> me.  We could also allow a per-controller sysfs file that only takes
> effect after a reset, which seems a little nicer, but adds a lot of
> boilerplate for just being a little nicer, so I'm not entirely sure
> if it is worth the effort.
Hi Christoph,

Because in the real user case, the number of each type queue may
not very suitable, it needs a ability to adjust them without hotplug.

If so, the nvme_dev needs record how many write/poll
queues saved in nvme_probe, and then use them in reset flow.

struct nvme_dev {
...
	unsigned int write_queues;
	unsigned int poll_queues;
}

How about add sysfs file
/sys/block/nvme0n1/device/io_queues_reload

and then check it in nvme_setup_io_queues, if it's true,
use module paramter, otherwise use parameters saved in nvme_probe.

Thanks

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme