From: Keith Busch <email@example.com> To: Christoph Hellwig <firstname.lastname@example.org> Cc: email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com Subject: Re: [RFC PATCH 1/3] nvme/pci: Poll the cq in the primary irq handler Date: Tue, 3 Dec 2019 04:50:06 -0700 [thread overview] Message-ID: <20191203115006.GB86476@C02WT3WMHTD6.lpcnextlight.net> (raw) In-Reply-To: <20191203075046.GF23881@lst.de> On Tue, Dec 03, 2019 at 08:50:46AM +0100, Christoph Hellwig wrote: > On Tue, Dec 03, 2019 at 07:22:04AM +0900, Keith Busch wrote: > > The nvme threaded interrupt handler reduces CPU time spent in hard irq > > context, but waking it increases latency for low queue depth workloads. > > > > Poll the completion queue once from the primary handler and wake the > > thread only if more completions remain after. > > How is this going to work with -rt, which wants to run all actual > interrupt work in the irq thread? That just has the primary and the bottom half run as threads. This patch just avoids a context switch for shallow workloads whether use -rt or not. > > Since there is a window > > of time where the threaded and primary handlers may run simultaneously, > > add a new nvmeq flag so that the two can synchronize which owns processing > > the queue. > > If the above scheme is something that the irq subsystem maintainers are > fine with I think we need to lift that synchronization into the core > irq code instead of open coding it in drivers. It's not generally applicable to all drivers, so it'd have to be an opt-in. I think IRQF_ONESHOT pretty much captures the same idea, though. > But I'd love to understand what kind of performance this split scheme > gives us to start with. Do you have any numbers? I do have a little bit of data. On a queue-depth 1 test, returning IRQ_WAKE_THREAD adds just under 1usec latency compared to completing in the primary handler. That's about 10% for some of the faster devices out there. _______________________________________________ linux-nvme mailing list firstname.lastname@example.org http://lists.infradead.org/mailman/listinfo/linux-nvme
next prev parent reply other threads:[~2019-12-03 11:50 UTC|newest] Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-02 22:22 [RFC PATCH 0/3] nvme threaded interrupt improvements Keith Busch 2019-12-02 22:22 ` [RFC PATCH 1/3] nvme/pci: Poll the cq in the primary irq handler Keith Busch 2019-12-03 7:50 ` Christoph Hellwig 2019-12-03 9:39 ` Sebastian Andrzej Siewior 2019-12-03 11:50 ` Keith Busch [this message] 2019-12-03 10:09 ` Sebastian Andrzej Siewior 2019-12-03 11:16 ` Keith Busch 2019-12-04 10:21 ` Ming Lei 2019-12-02 22:22 ` [RFC PATCH 2/3] nvme/pci: Remove use_threaded_interrupts parameter Keith Busch 2019-12-02 22:22 ` [RFC PATCH 3/3] nvme/pci: Poll for new completions in irq thread Keith Busch 2019-12-03 10:17 ` Sebastian Andrzej Siewior
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191203115006.GB86476@C02WT3WMHTD6.lpcnextlight.net \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [RFC PATCH 1/3] nvme/pci: Poll the cq in the primary irq handler' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).