All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ming Lei <ming.lei@redhat.com>
To: Keith Busch <keith.busch@intel.com>
Cc: Christoph Hellwig <hch@lst.de>,
	sagi@grimberg.me, linux-kernel@vger.kernel.org,
	linux-nvme@lists.infradead.org, axboe@fb.com,
	Jianchao Wang <jianchao.w.wang@oracle.com>,
	linux-block@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH V2] nvme-pci: assign separate irq vectors for adminq and ioq0
Date: Mon, 12 Mar 2018 17:09:13 +0800	[thread overview]
Message-ID: <20180312090827.GB23903@ming.t460p> (raw)
In-Reply-To: <20180309172445.GC14765@localhost.localdomain>

On Fri, Mar 09, 2018 at 10:24:45AM -0700, Keith Busch wrote:
> On Thu, Mar 08, 2018 at 08:42:20AM +0100, Christoph Hellwig wrote:
> > 
> > So I suspect we'll need to go with a patch like this, just with a way
> > better changelog.
> 
> I have to agree this is required for that use case. I'll run some
> quick tests and propose an alternate changelog.
> 
> Longer term, the current way we're including offline present cpus either
> (a) has the driver allocate resources it can't use or (b) spreads the
> ones it can use thinner than they need to be. Why don't we rerun the
> irq spread under a hot cpu notifier for only online CPUs?

4b855ad371 ("blk-mq: Create hctx for each present CPU") removes handling
mapping change via hot cpu notifier. Not only code is cleaned up, but
also fixes very complicated queue dependency issue:

- loop/dm-rq queue depends on underlying queue
- for NVMe, IO queue depends on admin queue

If freezing queue can be avoided in CPU notifier, it should be fine to
do that, otherwise it need to be avoided.

Thanks,
Ming

WARNING: multiple messages have this Message-ID (diff)
From: ming.lei@redhat.com (Ming Lei)
Subject: [PATCH V2] nvme-pci: assign separate irq vectors for adminq and ioq0
Date: Mon, 12 Mar 2018 17:09:13 +0800	[thread overview]
Message-ID: <20180312090827.GB23903@ming.t460p> (raw)
In-Reply-To: <20180309172445.GC14765@localhost.localdomain>

On Fri, Mar 09, 2018@10:24:45AM -0700, Keith Busch wrote:
> On Thu, Mar 08, 2018@08:42:20AM +0100, Christoph Hellwig wrote:
> > 
> > So I suspect we'll need to go with a patch like this, just with a way
> > better changelog.
> 
> I have to agree this is required for that use case. I'll run some
> quick tests and propose an alternate changelog.
> 
> Longer term, the current way we're including offline present cpus either
> (a) has the driver allocate resources it can't use or (b) spreads the
> ones it can use thinner than they need to be. Why don't we rerun the
> irq spread under a hot cpu notifier for only online CPUs?

4b855ad371 ("blk-mq: Create hctx for each present CPU") removes handling
mapping change via hot cpu notifier. Not only code is cleaned up, but
also fixes very complicated queue dependency issue:

- loop/dm-rq queue depends on underlying queue
- for NVMe, IO queue depends on admin queue

If freezing queue can be avoided in CPU notifier, it should be fine to
do that, otherwise it need to be avoided.

Thanks,
Ming

  reply	other threads:[~2018-03-12  9:09 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-28 15:48 [PATCH V2] nvme-pci: assign separate irq vectors for adminq and ioq0 Jianchao Wang
2018-02-28 15:48 ` Jianchao Wang
2018-02-28 15:59 ` Andy Shevchenko
2018-02-28 15:59   ` Andy Shevchenko
2018-03-02  3:06   ` jianchao.wang
2018-03-02  3:06     ` jianchao.wang
2018-03-12 18:59     ` Keith Busch
2018-03-12 18:59       ` Keith Busch
2018-03-13  1:47       ` jianchao.wang
2018-03-13  1:47         ` jianchao.wang
2018-02-28 16:47 ` Christoph Hellwig
2018-02-28 16:47   ` Christoph Hellwig
2018-03-01  9:28   ` Sagi Grimberg
2018-03-01  9:28     ` Sagi Grimberg
2018-03-01 10:05     ` jianchao.wang
2018-03-01 10:05       ` jianchao.wang
2018-03-01 15:15       ` Keith Busch
2018-03-01 15:15         ` Keith Busch
2018-03-02  3:11         ` jianchao.wang
2018-03-02  3:11           ` jianchao.wang
2018-03-01 15:03   ` Ming Lei
2018-03-01 15:03     ` Ming Lei
2018-03-01 16:10     ` Keith Busch
2018-03-01 16:10       ` Keith Busch
2018-03-08  7:42       ` Christoph Hellwig
2018-03-08  7:42         ` Christoph Hellwig
2018-03-09 17:24         ` Keith Busch
2018-03-09 17:24           ` Keith Busch
2018-03-12  9:09           ` Ming Lei [this message]
2018-03-12  9:09             ` Ming Lei
2018-10-08  5:05             ` nvme-pci: number of queues off by one Prasun Ratn
2018-10-08  5:59               ` Dongli Zhang
2018-10-08  5:59                 ` Dongli Zhang
2018-10-08  6:58                 ` Dongli Zhang
2018-10-08  6:58                   ` Dongli Zhang
2018-10-08 14:54                   ` Keith Busch
2018-10-08 14:54                     ` Keith Busch
2018-10-08 10:19                 ` Ming Lei
2018-10-08 10:19                   ` Ming Lei
2018-03-02  3:18   ` [PATCH V2] nvme-pci: assign separate irq vectors for adminq and ioq0 jianchao.wang
2018-03-02  3:18     ` jianchao.wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180312090827.GB23903@ming.t460p \
    --to=ming.lei@redhat.com \
    --cc=axboe@fb.com \
    --cc=hch@lst.de \
    --cc=jianchao.w.wang@oracle.com \
    --cc=keith.busch@intel.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=sagi@grimberg.me \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.