From: Hannes Reinecke <hare@suse.com>
To: John Garry <john.garry@huawei.com>,
tglx@linutronix.de, Christoph Hellwig <hch@lst.de>
Cc: Marc Zyngier <marc.zyngier@arm.com>,
"axboe@kernel.dk" <axboe@kernel.dk>,
Keith Busch <keith.busch@intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Michael Ellerman <mpe@ellerman.id.au>,
Linuxarm <linuxarm@huawei.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
SCSI Mailing List <linux-scsi@vger.kernel.org>
Subject: Re: Question on handling managed IRQs when hotplugging CPUs
Date: Tue, 29 Jan 2019 12:54:44 +0100 [thread overview]
Message-ID: <5bff8227-16fd-6bca-c16e-3992ef6bec5a@suse.com> (raw)
In-Reply-To: <f6f6e031-8b79-439d-c2af-8d3e76f30710@huawei.com>
On 1/29/19 12:25 PM, John Garry wrote:
> Hi,
>
> I have a question on $subject which I hope you can shed some light on.
>
> According to commit c5cb83bb337c25 ("genirq/cpuhotplug: Handle managed
> IRQs on CPU hotplug"), if we offline the last CPU in a managed IRQ
> affinity mask, the IRQ is shutdown.
>
> The reasoning is that this IRQ is thought to be associated with a
> specific queue on a MQ device, and the CPUs in the IRQ affinity mask are
> the same CPUs associated with the queue. So, if no CPU is using the
> queue, then no need for the IRQ.
>
> However how does this handle scenario of last CPU in IRQ affinity mask
> being offlined while IO associated with queue is still in flight?
>
> Or if we make the decision to use queue associated with the current CPU,
> and then that CPU (being the last CPU online in the queue's IRQ
> afffinity mask) goes offline and we finish the delivery with another CPU?
>
> In these cases, when the IO completes, it would not be serviced and
> timeout.
>
> I have actually tried this on my arm64 system and I see IO timeouts.
>
That actually is a very good question, and I have been wondering about
this for quite some time.
I find it a bit hard to envision a scenario where the IRQ affinity is
automatically (and, more importantly, atomically!) re-routed to one of
the other CPUs.
And even it it were, chances are that there are checks in the driver
_preventing_ them from handling those requests, seeing that they should
have been handled by another CPU ...
I guess the safest bet is to implement a 'cleanup' worker queue which is
responsible of looking through all the outstanding commands (on all
hardware queues), and then complete those for which no corresponding CPU
/ irqhandler can be found.
But I defer to the higher authorities here; maybe I'm totally wrong and
it's already been taken care of.
But if there is no generic mechanism this really is a fit topic for
LSF/MM, as most other drivers would be affected, too.
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.com +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
next prev parent reply other threads:[~2019-01-29 11:54 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-29 11:25 Question on handling managed IRQs when hotplugging CPUs John Garry
2019-01-29 11:54 ` Hannes Reinecke [this message]
2019-01-29 12:01 ` Thomas Gleixner
2019-01-29 15:27 ` John Garry
2019-01-29 16:27 ` Thomas Gleixner
2019-01-29 17:23 ` John Garry
2019-01-29 15:44 ` Keith Busch
2019-01-29 17:12 ` John Garry
2019-01-29 17:20 ` Keith Busch
2019-01-30 10:38 ` John Garry
2019-01-30 12:43 ` Thomas Gleixner
2019-01-31 17:48 ` John Garry
2019-02-01 15:56 ` Hannes Reinecke
2019-02-01 21:57 ` Thomas Gleixner
2019-02-04 7:12 ` Hannes Reinecke
2019-02-05 13:24 ` John Garry
2019-02-05 14:52 ` Keith Busch
2019-02-05 15:09 ` John Garry
2019-02-05 15:11 ` Keith Busch
2019-02-05 15:15 ` Hannes Reinecke
2019-02-05 15:27 ` John Garry
2019-02-05 18:23 ` Christoph Hellwig
2019-02-06 9:21 ` John Garry
2019-02-06 13:34 ` Benjamin Block
2019-02-05 15:10 ` Hannes Reinecke
2019-02-05 15:16 ` Keith Busch
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5bff8227-16fd-6bca-c16e-3992ef6bec5a@suse.com \
--to=hare@suse.com \
--cc=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=john.garry@huawei.com \
--cc=keith.busch@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=marc.zyngier@arm.com \
--cc=mpe@ellerman.id.au \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.