From: Ming Lei <email@example.com> To: John Garry <firstname.lastname@example.org> Cc: Jens Axboe <email@example.com>, "firstname.lastname@example.org" <email@example.com>, Bart Van Assche <firstname.lastname@example.org>, Hannes Reinecke <email@example.com>, Christoph Hellwig <firstname.lastname@example.org>, Thomas Gleixner <email@example.com>, Keith Busch <firstname.lastname@example.org> Subject: Re: [PATCH V4 0/5] blk-mq: improvement on handling IO during CPU hotplug Date: Tue, 29 Oct 2019 09:50:09 +0800 [thread overview] Message-ID: <20191029015009.GD22088@ming.t460p> (raw) In-Reply-To: <email@example.com> On Mon, Oct 28, 2019 at 11:55:42AM +0000, John Garry wrote: > > > > > > For the SCSI commands which timeout, I notice that > > > scsi_set_blocked(reason=SCSI_MLQUEUE_EH_RETRY) was called 30 seconds > > > earlier. > > > > > > scsi_set_blocked+0x20/0xb8 > > > __scsi_queue_insert+0x40/0x90 > > > scsi_softirq_done+0x164/0x1c8 > > > __blk_mq_complete_request_remote+0x18/0x20 > > > flush_smp_call_function_queue+0xa8/0x150 > > > generic_smp_call_function_single_interrupt+0x10/0x18 > > > handle_IPI+0xec/0x1a8 > > > arch_cpu_idle+0x10/0x18 > > > do_idle+0x1d0/0x2b0 > > > cpu_startup_entry+0x24/0x40 > > > secondary_start_kernel+0x1b4/0x208 > > > > Could you investigate a bit the reason why timeout is triggered? > > Yeah, it does seem a strange coincidence that the SCSI command even failed > and we have to retry, since these should be uncommon events. I'll check on > this LLDD error. > > > > > Especially we suppose to drain all in-flight requests before the > > last CPU of this hctx becomes offline, and it shouldn't be caused by > > the hctx becoming dead, so still need you to confirm that all > > in-flight requests are really drained in your test. > > ok > > Or is it still > > possible to dispatch to LDD after BLK_MQ_S_INTERNAL_STOPPED is set? > > It shouldn't be. However it would seem that this IO had been dispatched to > the LLDD, the hctx dies, and then we attempt to requeue on that hctx. But this patch does wait for completion of in-flight request before shutdown the last CPU of this hctx. Thanks, Ming
next prev parent reply other threads:[~2019-10-29 1:50 UTC|newest] Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-10-14 1:50 Ming Lei 2019-10-14 1:50 ` [PATCH V4 1/5] blk-mq: add new state of BLK_MQ_S_INTERNAL_STOPPED Ming Lei 2019-10-14 1:50 ` [PATCH V4 2/5] blk-mq: prepare for draining IO when hctx's all CPUs are offline Ming Lei 2019-10-14 1:50 ` [PATCH V4 3/5] blk-mq: stop to handle IO and drain IO before hctx becomes dead Ming Lei 2019-11-28 9:29 ` John Garry 2019-10-14 1:50 ` [PATCH V4 4/5] blk-mq: re-submit IO in case that hctx is dead Ming Lei 2019-10-14 1:50 ` [PATCH V4 5/5] blk-mq: handle requests dispatched from IO scheduler " Ming Lei 2019-10-16 8:58 ` [PATCH V4 0/5] blk-mq: improvement on handling IO during CPU hotplug John Garry 2019-10-16 12:07 ` Ming Lei 2019-10-16 16:19 ` John Garry [not found] ` <firstname.lastname@example.org> 2019-10-20 10:14 ` Ming Lei 2019-10-21 9:19 ` John Garry 2019-10-21 9:34 ` Ming Lei 2019-10-21 9:47 ` John Garry 2019-10-21 10:24 ` Ming Lei 2019-10-21 11:49 ` John Garry 2019-10-21 12:53 ` Ming Lei 2019-10-21 14:02 ` John Garry 2019-10-22 0:16 ` Ming Lei 2019-10-22 11:19 ` John Garry 2019-10-22 13:45 ` Ming Lei 2019-10-25 16:33 ` John Garry 2019-10-28 10:42 ` Ming Lei 2019-10-28 11:55 ` John Garry 2019-10-29 1:50 ` Ming Lei [this message] 2019-10-29 9:22 ` John Garry 2019-10-29 10:05 ` Ming Lei 2019-10-29 17:54 ` John Garry 2019-10-31 16:28 ` John Garry 2019-11-28 1:09 ` chenxiang (M) 2019-11-28 2:02 ` Ming Lei 2019-11-28 10:45 ` John Garry
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191029015009.GD22088@ming.t460p \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [PATCH V4 0/5] blk-mq: improvement on handling IO during CPU hotplug' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).