From: Ming Lei <ming.lei@redhat.com>
To: Keith Busch <kbusch@kernel.org>
Cc: sagi@grimberg.me, bigeasy@linutronix.de,
linux-nvme@lists.infradead.org, helgaas@kernel.org,
Thomas Gleixner <tglx@linutronix.de>,
hch@lst.de
Subject: Re: [PATCH 2/4] nvme/pci: Mask legacy and MSI in threaded handler
Date: Thu, 28 Nov 2019 11:39:56 +0800 [thread overview]
Message-ID: <20191128033956.GD3277@ming.t460p> (raw)
In-Reply-To: <20191127175824.1929-3-kbusch@kernel.org>
On Thu, Nov 28, 2019 at 02:58:22AM +0900, Keith Busch wrote:
> Local interrupts are re-enabled when the nvme irq thread is
> woken. Subsequent MSI or a level triggered legacy interrupts may restart
> the nvme irq check while the thread handler is running. This unnecessarily
> spends CPU cycles and potentially triggers spurious interrupt detection,
> disabling our NVMe irq.
>
> Use the NVMe interrupt mask/clear registers to disable controller
> interrupts while the nvme bottom half processes completions.
>
> Signed-off-by: Keith Busch <kbusch@kernel.org>
> ---
> drivers/nvme/host/pci.c | 27 +++++++++++++++++++++++++++
> 1 file changed, 27 insertions(+)
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 9d307593b94f..c5b837cba730 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1048,6 +1048,28 @@ static irqreturn_t nvme_irq_check(int irq, void *data)
> return IRQ_NONE;
> }
>
> +static irqreturn_t nvme_irq_thread_msi(int irq, void *data)
> +{
> + struct nvme_queue *nvmeq = data;
> + struct nvme_dev *dev = nvmeq->dev;
> +
> + nvme_irq(irq, data);
> + writel(1 << nvmeq->cq_vector, dev->bar + NVME_REG_INTMC);
> + return IRQ_HANDLED;
> +}
> +
> +static irqreturn_t nvme_irq_check_msi(int irq, void *data)
> +{
> + struct nvme_queue *nvmeq = data;
> + struct nvme_dev *dev = nvmeq->dev;
> +
> + if (nvme_cqe_pending(nvmeq)) {
> + writel(1 << nvmeq->cq_vector, dev->bar + NVME_REG_INTMS);
> + return IRQ_WAKE_THREAD;
> + }
> + return IRQ_NONE;
> +}
> +
> /*
> * Poll for completions any queue, including those not dedicated to polling.
> * Can be called from any context.
> @@ -1502,6 +1524,11 @@ static int queue_request_irq(struct nvme_queue *nvmeq)
> int nr = nvmeq->dev->ctrl.instance;
>
> if (use_threaded_interrupts) {
> + /* MSI and Legacy use the same NVMe IRQ masking */
> + if (!pdev->msix_enabled)
> + return pci_request_irq(pdev, nvmeq->cq_vector,
> + nvme_irq_check_msi, nvme_irq_thread_msi,
> + nvmeq, "nvme%dq%d", nr, nvmeq->qid);
> return pci_request_irq(pdev, nvmeq->cq_vector, nvme_irq_check,
> nvme_irq, nvmeq, "nvme%dq%d", nr, nvmeq->qid);
Just wondering why don't do that for misx_enabled, and according to
document of request_threaded_irq(), the handler is supposed to
disable the device's interrupt:
* If you want to set up a threaded irq handler for your device
* then you need to supply @handler and @thread_fn. @handler is
* still called in hard interrupt context and has to check
* whether the interrupt originates from the device. If yes it
* needs to disable the interrupt on the device and return
* IRQ_WAKE_THREAD which will wake up the handler thread and run
* @thread_fn. This split handler design is necessary to support
* shared interrupts.
However, MSI irq controller is said to be one shot safe, see
923aa4c378f9("PCI/MSI: Set IRQCHIP_ONESHOT_SAFE for PCI-MSI irqchips"),
then the question is that if interrupt mask is needed.
Thanks,
Ming
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
next prev parent reply other threads:[~2019-11-28 3:40 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-27 17:58 [PATCH 0/4] nvme: Threaded interrupt handling improvements Keith Busch
2019-11-27 17:58 ` [PATCH 1/4] PCI/MSI: Export __pci_msix_desc_mask_irq Keith Busch
2019-11-28 2:42 ` Sagi Grimberg
2019-11-28 3:41 ` Keith Busch
2019-11-28 7:17 ` Christoph Hellwig
2019-11-27 17:58 ` [PATCH 2/4] nvme/pci: Mask legacy and MSI in threaded handler Keith Busch
2019-11-28 3:39 ` Ming Lei [this message]
2019-11-28 3:48 ` Keith Busch
2019-11-28 3:58 ` Ming Lei
2019-11-28 4:14 ` Keith Busch
2019-11-28 8:41 ` Ming Lei
2019-11-27 17:58 ` [PATCH 3/4] nvme/pci: Mask MSIx interrupts for threaded handling Keith Busch
2019-11-28 7:19 ` Christoph Hellwig
2019-11-27 17:58 ` [PATCH 4/4] nvme/pci: Spin threaded interrupt completions Keith Busch
2019-11-28 2:46 ` Sagi Grimberg
2019-11-28 3:28 ` Keith Busch
2019-11-28 3:51 ` Ming Lei
2019-11-28 3:58 ` Keith Busch
2019-11-28 7:22 ` Christoph Hellwig
2019-11-29 9:13 ` Sebastian Andrzej Siewior
2019-11-30 18:10 ` Keith Busch
2019-12-02 1:10 ` Ming Lei
2019-12-02 1:30 ` Keith Busch
2019-12-02 16:51 ` Sebastian Andrzej Siewior
2019-11-28 7:50 ` [PATCH 0/4] nvme: Threaded interrupt handling improvements Christoph Hellwig
2019-11-28 17:59 ` Keith Busch
2019-11-29 8:30 ` Christoph Hellwig
2019-11-29 9:46 ` Sebastian Andrzej Siewior
2019-11-29 16:27 ` Keith Busch
2019-11-29 17:05 ` Sebastian Andrzej Siewior
2019-11-30 17:02 ` Keith Busch
2019-12-02 17:05 ` Sebastian Andrzej Siewior
2019-12-02 17:12 ` Christoph Hellwig
2019-12-02 18:06 ` Keith Busch
2019-12-03 7:40 ` Christoph Hellwig
2019-12-02 19:57 ` Sebastian Andrzej Siewior
2019-12-03 7:42 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191128033956.GD3277@ming.t460p \
--to=ming.lei@redhat.com \
--cc=bigeasy@linutronix.de \
--cc=hch@lst.de \
--cc=helgaas@kernel.org \
--cc=kbusch@kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).