linux-block.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 0/3] softirq/blk-mq: implement interrupt flood detection for avoiding cpu lockup
@ 2019-12-18  7:19 Ming Lei
  2019-12-18  7:19 ` [RFC PATCH 1/3] sched/core: add API for exporting runqueue clock Ming Lei
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Ming Lei @ 2019-12-18  7:19 UTC (permalink / raw)
  To: Thomas Gleixner, Jens Axboe
  Cc: linux-kernel, linux-block, Ming Lei, Long Li, Ingo Molnar,
	Peter Zijlstra, Keith Busch, Christoph Hellwig, Sagi Grimberg,
	John Garry, Hannes Reinecke

Hi Guys,

For minimizing IO latency, blk-mq often completes IO in the hardware
interrupt context. However, in case of multiple HBAs, or multiple
storage devices attached to same HBA, if one CPU core has to handle
interrupts of IOs submitted from multile CPU , there is risk to lock
up CPUs[1][2].

Follows the idea when there is mutliple IO submitter and single
CPU for completing these IOs:

1) in case of multiple storages attached to one single HBA, these
storages may handle IO more quickly than single CPU core

2) in case of multiple HBAs in one system, one single effective CPU
can be selected for handling interrupts from several queues, then
multiple storages still may handle IO more quickly than single CPU
core.

When handling IO completion in interrupt context, IO latency is good,
but there is risk to lock up CPU. When moving IO completion to process
context via NAPI or threaded interrupt handler, CPU lockup can be
avoided. So people try to move part of IO completion into process
context for avoiding CPU lockup, meantime trying to not hurt IO
performance, such as Keith's work[3].

However, it is hard to partition IO completion in the two contexts, if
less work is moved to process context, risk of locking up CPU can't be
eliminated; if more work is moved to process context, extra IO latency is
introduced, then IO performance is hurt.

Interrupt flood information can be one useful hint for partitioning IO
completion work into the two contexts effectively.

The 1st two patches implement interrpupt flood detection, and the 3rd
patch applies the hint to complet IO in process context when interrupt
flood happens. This way avoids CPU lockup, meantime IO performance isn't
hurt obviously.

[1] https://lore.kernel.org/lkml/1566281669-48212-1-git-send-email-longli@linuxonhyperv.com/
[2] https://lore.kernel.org/lkml/a7ef3810-31af-013a-6d18-ceb6154aa2ef@huawei.com/
[3] https://lore.kernel.org/linux-nvme/20191209175622.1964-1-kbusch@kernel.org/T/#t

Ming Lei (3):
  sched/core: add API for exporting runqueue clock
  softirq: implement interrupt flood detection
  blk-mq: complete request in rescuer process context in case of irq
    flood

 block/blk-mq.c          |  68 ++++++++++++++++-
 drivers/base/cpu.c      |  23 ++++++
 include/linux/hardirq.h |   2 +
 include/linux/sched.h   |   2 +
 kernel/sched/core.c     |   5 ++
 kernel/softirq.c        | 161 ++++++++++++++++++++++++++++++++++++++++
 6 files changed, 260 insertions(+), 1 deletion(-)

Cc: Long Li <longli@microsoft.com>
Cc: Ingo Molnar <mingo@redhat.com>,
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Keith Busch <keith.busch@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: John Garry <john.garry@huawei.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Hannes Reinecke <hare@suse.com>
-- 
2.20.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2020-01-02 10:28 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-18  7:19 [RFC PATCH 0/3] softirq/blk-mq: implement interrupt flood detection for avoiding cpu lockup Ming Lei
2019-12-18  7:19 ` [RFC PATCH 1/3] sched/core: add API for exporting runqueue clock Ming Lei
2019-12-18  9:51   ` Peter Zijlstra
2019-12-19  1:29     ` Ming Lei
2019-12-19  9:20       ` Peter Zijlstra
2019-12-18  7:19 ` [RFC PATCH 2/3] softirq: implement interrupt flood detection Ming Lei
2019-12-18 10:49   ` Peter Zijlstra
2019-12-18 12:29     ` Peter Zijlstra
2019-12-19  1:59     ` Ming Lei
2019-12-19  9:23       ` Peter Zijlstra
2019-12-19  9:52         ` Ming Lei
2019-12-19 10:43         ` Daniel Wagner
2019-12-31  3:48           ` Ming Lei
2020-01-02 10:28             ` Daniel Wagner
2019-12-18  7:19 ` [RFC PATCH 3/3] blk-mq: complete request in rescuer process context in case of irq flood Ming Lei

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).