From: Andy Shevchenko <andy.shevchenko@gmail.com>
To: Jianchao Wang <jianchao.w.wang@oracle.com>
Cc: Keith Busch <keith.busch@intel.com>, Jens Axboe <axboe@fb.com>,
Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>,
Linux NVMe Mailinglist <linux-nvme@lists.infradead.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH V2] nvme-pci: assign separate irq vectors for adminq and ioq0
Date: Wed, 28 Feb 2018 17:59:54 +0200 [thread overview]
Message-ID: <CAHp75Vet0gt0RW4eu+TCE8qBSk2ymMKkG5=SuM9ytzO3-6Y+Mg@mail.gmail.com> (raw)
In-Reply-To: <1519832921-13915-1-git-send-email-jianchao.w.wang@oracle.com>
On Wed, Feb 28, 2018 at 5:48 PM, Jianchao Wang
<jianchao.w.wang@oracle.com> wrote:
> Currently, adminq and ioq0 share the same irq vector. This is
> unfair for both amdinq and ioq0.
> - For adminq, its completion irq has to be bound on cpu0. It
> just has only one hw queue, it is unreasonable to do this.
> - For ioq0, when the irq fires for io completion, the adminq irq
> action on this irq vector will introduce an uncached access on
> adminq cqe at least, even worse when adminq is busy.
>
> To improve this, allocate separate irq vectors for adminq and
> ioq0, and not set irq affinity for adminq one. If just one irq
> vector, setup adminq + 1 ioq and let them share it. In addition
> add new helper interface nvme_ioq_vector to get ioq vector.
> +static inline unsigned int nvme_ioq_vector(struct nvme_dev *dev,
> + unsigned int qid)
> +{
> + /*
> + * If controller has only legacy or single-message MSI, there will
> + * be only 1 irq vector. At the moment, we setup adminq + 1 ioq
> + * and let them share irq vector.
> + */
> + return (dev->num_vecs == 1) ? 0 : qid;
Redundant parens.
> +}
>
> for (i = dev->ctrl.queue_count; i <= dev->max_qid; i++) {
> - /* vector == qid - 1, match nvme_create_queue */
> if (nvme_alloc_queue(dev, i, dev->q_depth,
> - pci_irq_get_node(to_pci_dev(dev->dev), i - 1))) {
> + pci_irq_get_node(to_pci_dev(dev->dev),
> + nvme_ioq_vector(dev, i)))) {
Perhaps better to introduce a temporary variable to make it readable?
> ret = -ENOMEM;
> break;
> }
> + ret = pci_alloc_irq_vectors_affinity(pdev, 1, (nr_io_queues + 1),
> + PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY, &affd);
> + if (ret <= 0)
> return -EIO;
> - dev->max_qid = nr_io_queues;
> -
> + dev->num_vecs = ret;
> + dev->max_qid = (ret > 1) ? (ret - 1) : 1;
I don not see how ret can possible be < 1 here.
Thus, the logic looks like this:
if ret >= 2 => return ret - 1; // Possible variants [1..ret - 1]
if ret == 1 => return 1;
So, for ret == 1 or ret == 2 we still use 1.
Is it by design?
Can it be written like
dev->max_qid = max(ret - 1, 1);
--
With Best Regards,
Andy Shevchenko
next prev parent reply other threads:[~2018-02-28 16:00 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-28 15:48 [PATCH V2] nvme-pci: assign separate irq vectors for adminq and ioq0 Jianchao Wang
2018-02-28 15:59 ` Andy Shevchenko [this message]
2018-03-02 3:06 ` jianchao.wang
2018-03-12 18:59 ` Keith Busch
2018-03-13 1:47 ` jianchao.wang
2018-02-28 16:47 ` Christoph Hellwig
2018-03-01 9:28 ` Sagi Grimberg
2018-03-01 10:05 ` jianchao.wang
2018-03-01 15:15 ` Keith Busch
2018-03-02 3:11 ` jianchao.wang
2018-03-01 15:03 ` Ming Lei
2018-03-01 16:10 ` Keith Busch
2018-03-08 7:42 ` Christoph Hellwig
2018-03-09 17:24 ` Keith Busch
2018-03-12 9:09 ` Ming Lei
2018-03-02 3:18 ` jianchao.wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAHp75Vet0gt0RW4eu+TCE8qBSk2ymMKkG5=SuM9ytzO3-6Y+Mg@mail.gmail.com' \
--to=andy.shevchenko@gmail.com \
--cc=axboe@fb.com \
--cc=hch@lst.de \
--cc=jianchao.w.wang@oracle.com \
--cc=keith.busch@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).