[PATCHv3 0/4] nvme pci interrupt handling improvements

[Keith Busch <kbusch@kernel.org>]

Here's the next revision to nvme pci interrupt handling. The series is
attempting to address two issues that have been recently raised:

  1. The nvme driver makes it possible to soft lockup a CPU due to high
     utilization in irq context. This is most prevalent when multiple
     CPUs are driving a single hardware queue on one or more controllers.

  2. The threaded handler left interrupts unmasked, which breaks when
     used with level triggered interrupts, or unnecessarily runs
     in interrupt context when edge interrupts occur frequently.

Both issues are addressed by always configuring nvme interrupts to
run the threaded handler with interrupts disabled. A hybrid approch
to handling nvme completions in hard irq context and thread context is
introduced so that there should not be a performance impact from removing
the nvme.use_threaded_interrupts option.

The series appears to be a win or no impact on performance from what I
can test. I would be greatful to hear if others can confirm this with
other hardware.

I've dropped the fast MSIx masking. While I believe it's safe to skip
flushing the memory write, I think this series mitigates the impact of
the read back by ensuring the ratio of memory reads to IO is low enough
to be negligable (AFAICT on hardware available to me).

I've changed the exit condition for the polling nvme irq thread to
break out of the loop if we've wrapped the completion queue. Other irq
threads may be affinitized to the same CPU, so we need to schedule out
at some point, but I've been told multiple times that need_resched()
or cond_resched() won't work as desired from the thread's fifo priority.

Keith Busch (4):
  nvme/pci: Disable interrupts for threaded handler
  nvme/pci: Complete commands from primary handler
  nvme/pci: Remove use_threaded_interrupts
  nvme/pci: Poll threaded completions

 drivers/nvme/host/pci.c | 56 ++++++++++++++++++++++++++---------------
 1 file changed, 36 insertions(+), 20 deletions(-)

-- 
2.21.0


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme