From: Ming Lei <ming.lei@redhat.com>
To: Keith Busch <kbusch@kernel.org>
Cc: sagi@grimberg.me, bigeasy@linutronix.de,
linux-nvme@lists.infradead.org, helgaas@kernel.org,
Thomas Gleixner <tglx@linutronix.de>,
hch@lst.de
Subject: Re: [PATCH 2/4] nvme/pci: Mask legacy and MSI in threaded handler
Date: Thu, 28 Nov 2019 16:41:26 +0800 [thread overview]
Message-ID: <20191128084126.GB15549@ming.t460p> (raw)
In-Reply-To: <20191128041404.GB1947@redsun51.ssa.fujisawa.hgst.com>
On Thu, Nov 28, 2019 at 01:14:04PM +0900, Keith Busch wrote:
> On Thu, Nov 28, 2019 at 11:58:53AM +0800, Ming Lei wrote:
> > On Thu, Nov 28, 2019 at 12:48:17PM +0900, Keith Busch wrote:
> > > On Thu, Nov 28, 2019 at 11:39:56AM +0800, Ming Lei wrote:
> > > > 923aa4c378f9("PCI/MSI: Set IRQCHIP_ONESHOT_SAFE for PCI-MSI irqchips"),
> > > > then the question is that if interrupt mask is needed.
> > >
> > > We don't want to use IRQF_ONESHOT for our MSI interrupts because that
> > > will write to the MSI mask config register, which is a costly non-posted
> > > transaction. The NVMe specific way uses much faster posted writes.
> >
> > What I meant is that IRQF_ONESHOT isn't needed in case of IRQCHIP_ONESHOT_SAFE.
> >
> > So it is reasonable to understand that interrupt mask isn't needed in the
> > hard interrupt handler in case of IRQCHIP_ONESHOT_SAFE. That is
> > basically what commit dc9b229a58dc("genirq: Allow irq chips to mark themself
> > oneshot safe") does.
>
> Hmm, it doesn't look like it's always safe. We have to stop the device
> from generating MSIs for new completions somehow while the threaded
> handler is running, otherwise those MSIs will be considered spurious
> when the thread never gets a chance to increment desc->threads_handled.
>
I just observe hard interrupts triggered between start of nvme_irq_check()
and end of nvme_irq(). Yeah, there could be at most 36 interrupts comes during
the period on one machine, and most are <= 5.
So looks this patchset make sense, and also means that IRQCHIP_ONESHOT_SAFE
might be broken.
@interrupts_during_threaded:
[0, 1) 2074375 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ |
[1, 2) 3668018 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[2, 3) 1508944 |@@@@@@@@@@@@@@@@@@@@@ |
[3, 4) 554496 |@@@@@@@ |
[4, 5) 225761 |@@@ |
[5, 6) 99354 |@ |
[6, 7) 45127 | |
[7, 8) 20912 | |
[8, 9) 9940 | |
[9, 10) 4765 | |
[10, 11) 2458 | |
[11, 12) 1365 | |
[12, 13) 719 | |
[13, 14) 451 | |
[14, 15) 265 | |
[15, 16) 168 | |
[16, 17) 103 | |
[17, 18) 67 | |
[18, 19) 60 | |
[19, 20) 41 | |
[20, 21) 27 | |
[21, 22) 18 | |
[22, 23) 17 | |
[23, 24) 8 | |
[24, 25) 2 | |
[25, 26) 9 | |
[26, 27) 6 | |
[27, 28) 3 | |
[28, 29) 1 | |
[29, 30) 0 | |
[30, 31) 0 | |
[31, 32) 0 | |
[32, 33) 0 | |
[33, 34) 0 | |
[34, 35) 0 | |
[35, 36) 0 | |
[36, 37) 1 | |
Thanks,
Ming
_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
next prev parent reply other threads:[~2019-11-28 8:41 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-27 17:58 [PATCH 0/4] nvme: Threaded interrupt handling improvements Keith Busch
2019-11-27 17:58 ` [PATCH 1/4] PCI/MSI: Export __pci_msix_desc_mask_irq Keith Busch
2019-11-28 2:42 ` Sagi Grimberg
2019-11-28 3:41 ` Keith Busch
2019-11-28 7:17 ` Christoph Hellwig
2019-11-27 17:58 ` [PATCH 2/4] nvme/pci: Mask legacy and MSI in threaded handler Keith Busch
2019-11-28 3:39 ` Ming Lei
2019-11-28 3:48 ` Keith Busch
2019-11-28 3:58 ` Ming Lei
2019-11-28 4:14 ` Keith Busch
2019-11-28 8:41 ` Ming Lei [this message]
2019-11-27 17:58 ` [PATCH 3/4] nvme/pci: Mask MSIx interrupts for threaded handling Keith Busch
2019-11-28 7:19 ` Christoph Hellwig
2019-11-27 17:58 ` [PATCH 4/4] nvme/pci: Spin threaded interrupt completions Keith Busch
2019-11-28 2:46 ` Sagi Grimberg
2019-11-28 3:28 ` Keith Busch
2019-11-28 3:51 ` Ming Lei
2019-11-28 3:58 ` Keith Busch
2019-11-28 7:22 ` Christoph Hellwig
2019-11-29 9:13 ` Sebastian Andrzej Siewior
2019-11-30 18:10 ` Keith Busch
2019-12-02 1:10 ` Ming Lei
2019-12-02 1:30 ` Keith Busch
2019-12-02 16:51 ` Sebastian Andrzej Siewior
2019-11-28 7:50 ` [PATCH 0/4] nvme: Threaded interrupt handling improvements Christoph Hellwig
2019-11-28 17:59 ` Keith Busch
2019-11-29 8:30 ` Christoph Hellwig
2019-11-29 9:46 ` Sebastian Andrzej Siewior
2019-11-29 16:27 ` Keith Busch
2019-11-29 17:05 ` Sebastian Andrzej Siewior
2019-11-30 17:02 ` Keith Busch
2019-12-02 17:05 ` Sebastian Andrzej Siewior
2019-12-02 17:12 ` Christoph Hellwig
2019-12-02 18:06 ` Keith Busch
2019-12-03 7:40 ` Christoph Hellwig
2019-12-02 19:57 ` Sebastian Andrzej Siewior
2019-12-03 7:42 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191128084126.GB15549@ming.t460p \
--to=ming.lei@redhat.com \
--cc=bigeasy@linutronix.de \
--cc=hch@lst.de \
--cc=helgaas@kernel.org \
--cc=kbusch@kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).