linux-block.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH V2 RESEND 0/5] blk-mq: improvement on handling IO during CPU hotplug
@ 2019-10-06  2:45 Ming Lei
  2019-10-06  2:45 ` [PATCH V2 RESEND 1/5] blk-mq: add new state of BLK_MQ_S_INTERNAL_STOPPED Ming Lei
                   ` (4 more replies)
  0 siblings, 5 replies; 16+ messages in thread
From: Ming Lei @ 2019-10-06  2:45 UTC (permalink / raw)
  To: Jens Axboe
  Cc: linux-block, John Garry, Ming Lei, Bart Van Assche,
	Hannes Reinecke, Christoph Hellwig, Thomas Gleixner, Keith Busch

Hi,

Thomas mentioned:
    "
     That was the constraint of managed interrupts from the very beginning:
    
      The driver/subsystem has to quiesce the interrupt line and the associated
      queue _before_ it gets shutdown in CPU unplug and not fiddle with it
      until it's restarted by the core when the CPU is plugged in again.
    "

But no drivers or blk-mq do that before one hctx becomes dead(all
CPUs for one hctx are offline), and even it is worse, blk-mq stills tries
to run hw queue after hctx is dead, see blk_mq_hctx_notify_dead().

This patchset tries to address the issue by two stages:

1) add one new cpuhp state of CPUHP_AP_BLK_MQ_ONLINE

- mark the hctx as internal stopped, and drain all in-flight requests
if the hctx is going to be dead.

2) re-submit IO in the state of CPUHP_BLK_MQ_DEAD after the hctx becomes dead

- steal bios from the request, and resubmit them via generic_make_request(),
then these IO will be mapped to other live hctx for dispatch

Please comment & review, thanks!

V2:
	- patch4 & patch 5 in V1 have been merged to block tree, so remove
	  them
	- address comments from John Garry and Minwoo

Ming Lei (5):
  blk-mq: add new state of BLK_MQ_S_INTERNAL_STOPPED
  blk-mq: add blk-mq flag of BLK_MQ_F_NO_MANAGED_IRQ
  blk-mq: stop to handle IO before hctx's all CPUs become offline
  blk-mq: re-submit IO in case that hctx is dead
  blk-mq: handle requests dispatched from IO scheduler in case that hctx
    is dead

 block/blk-mq-debugfs.c     |   2 +
 block/blk-mq-tag.c         |   2 +-
 block/blk-mq-tag.h         |   2 +
 block/blk-mq.c             | 143 +++++++++++++++++++++++++++++++++----
 block/blk-mq.h             |   3 +-
 drivers/block/loop.c       |   2 +-
 drivers/md/dm-rq.c         |   2 +-
 include/linux/blk-mq.h     |   5 ++
 include/linux/cpuhotplug.h |   1 +
 9 files changed, 146 insertions(+), 16 deletions(-)

Cc: Bart Van Assche <bvanassche@acm.org>
Cc: Hannes Reinecke <hare@suse.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Keith Busch <keith.busch@intel.com>
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2019-10-07 16:49 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-06  2:45 [PATCH V2 RESEND 0/5] blk-mq: improvement on handling IO during CPU hotplug Ming Lei
2019-10-06  2:45 ` [PATCH V2 RESEND 1/5] blk-mq: add new state of BLK_MQ_S_INTERNAL_STOPPED Ming Lei
2019-10-07  6:02   ` Hannes Reinecke
2019-10-06  2:45 ` [PATCH V2 RESEND 2/5] blk-mq: add blk-mq flag of BLK_MQ_F_NO_MANAGED_IRQ Ming Lei
2019-10-07  6:06   ` Hannes Reinecke
2019-10-06  2:45 ` [PATCH V2 RESEND 3/5] blk-mq: stop to handle IO before hctx's all CPUs become offline Ming Lei
2019-10-07  6:23   ` Hannes Reinecke
2019-10-07 15:01     ` Ming Lei
2019-10-07 10:23   ` John Garry
2019-10-07 15:04     ` Ming Lei
2019-10-07 16:49       ` John Garry
2019-10-06  2:45 ` [PATCH V2 RESEND 4/5] blk-mq: re-submit IO in case that hctx is dead Ming Lei
2019-10-07  6:27   ` Hannes Reinecke
2019-10-07 15:05     ` Ming Lei
2019-10-06  2:45 ` [PATCH V2 RESEND 5/5] blk-mq: handle requests dispatched from IO scheduler " Ming Lei
2019-10-07  6:28   ` Hannes Reinecke

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).