From: Marc Zyngier <maz@kernel.org>
To: John Garry <john.garry@huawei.com>
Cc: linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
chenxiang <chenxiang66@hisilicon.com>,
Zhou Wang <wangzhou1@hisilicon.com>,
Ming Lei <ming.lei@redhat.com>,
Jason Cooper <jason@lakedaemon.net>,
Thomas Gleixner <tglx@linutronix.de>,
luojiaxing@huawei.com
Subject: Re: [PATCH v3 2/2] irqchip/gic-v3-its: Balance initial LPI affinity across CPUs
Date: Wed, 18 Mar 2020 14:16:37 +0000 [thread overview]
Message-ID: <7b97c24ceced7560b5acb03edaf2cd70@kernel.org> (raw)
In-Reply-To: <8b141d09-ac11-34ec-0922-c21c22f94f36@huawei.com>
On 2020-03-17 18:43, John Garry wrote:
>>>
>>>> + int this_count = its_read_lpi_count(d, tmp);
>>>
>>> Not sure if it's intentional, but now there seems to be a subtle
>>> difference to what Thomas described for non-managed interrupts - for
>>> non-managed interrupts, x86 selects the CPU based on the total
>>> interrupt load per CPU (or, more specifically, lowest vector
>>> allocation count), and not just the non-managed load. Or maybe I
>>> misread it.
>>
>> So far, I'm trying to keep the two allocation paths separate, as the
>> two systems I have access to have very different behaviours: D05 has
>> no managed interrupts to speak of, and my top-secret work machine
>> has almost no unmanaged interrupts, so the two sets are almost
>> completely disjoint.
>
> Sure, but I'd say that it would be a more common scenario to have a
> mixture of both.
>
>>
>> Also, it all depends on the interrupt allocation order, and whether
>> something will rebalance the non-managed interrupts at a later time.
>> At least, these two patches make it easy to alter the placement policy
>> (the behaviour you describe above is a 2 line change).
>>
>>> Anyway, we can test this now for NVMe with its managed interrupts.
>>
>> Looking forward to hearing from you!
>>
>
> On my D06CS board (128 core), there seems to be something wrong, as
> the q0 affinity mask looks incorrect:
>
> PCI name is 81:00.0: nvme0n1
>
>
> irq 322, cpu list 69, effective list 69
>
>
> irq 325, cpu list 32-38, effective list 32
>
>
> irq 326, cpu list 39-45, effective list 40
>
>
> irq 327, cpu list 46-51, effective list 47
>
>
> irq 328, cpu list 52-57, effective list 53
>
>
> irq 329, cpu list 58-63, effective list 59
Sorry, can you explain in more detail what you find wrong in this log?
Is it that interrupt 322 has a single CPU affinity instead of a list?
> And something stranger for my colleague Luo Jiaxing, specifically the
> effective affinity:
>
> PCI name is 85:00.0: nvme2n1
> irq 196, cpu list 0-31, effective list 82
Right, this one we have seen in your other email. Being a non-managed
interrupt, it lands on the closest socket.
> irq 377, cpu list 32-38, effective list 32
> irq 378, cpu list 39-45, effective list 39
> irq 379, cpu list 46-51, effective list 46
>
> But then v5.6-rc5 vanilla also looks to have this issue when I tested
> on my board:
>
> john@ubuntu:~$ more /proc/irq/322/smp_affinity_list
>
>
> 69
>
> My D06ES (96 core) board looks sensible for the affinity in this
> regard (I did not try vanilla v5.6-rc5, but only with your patches on
> top). I'll need to debug this.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
next prev parent reply other threads:[~2020-03-18 14:16 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-16 11:54 [PATCH v3 0/2] irqchip/gic-v3-its: Balance LPI affinity across CPUs Marc Zyngier
2020-03-16 11:54 ` [PATCH v3 1/2] irqchip/gic-v3-its: Track LPI distribution on a per CPU basis Marc Zyngier
2020-03-16 11:54 ` [PATCH v3 2/2] irqchip/gic-v3-its: Balance initial LPI affinity across CPUs Marc Zyngier
2020-03-16 13:02 ` John Garry
2020-03-16 13:14 ` Marc Zyngier
2020-03-17 18:43 ` John Garry
2020-03-18 14:16 ` Marc Zyngier [this message]
2020-03-18 14:25 ` John Garry
2020-03-18 12:22 ` John Garry
2020-03-18 14:04 ` Marc Zyngier
2020-03-18 15:34 ` John Garry
2020-03-18 17:30 ` Marc Zyngier
2020-03-18 19:00 ` John Garry
2020-03-27 17:52 ` John Garry
2020-03-19 12:31 ` [PATCH v3 0/2] irqchip/gic-v3-its: Balance " John Garry
2020-03-27 17:47 ` John Garry
2020-04-01 11:33 ` John Garry
2020-05-14 12:05 ` John Garry
2020-05-15 10:14 ` Marc Zyngier
2020-05-15 11:50 ` John Garry
2020-05-15 15:37 ` Marc Zyngier
2020-05-15 16:15 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7b97c24ceced7560b5acb03edaf2cd70@kernel.org \
--to=maz@kernel.org \
--cc=chenxiang66@hisilicon.com \
--cc=jason@lakedaemon.net \
--cc=john.garry@huawei.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luojiaxing@huawei.com \
--cc=ming.lei@redhat.com \
--cc=tglx@linutronix.de \
--cc=wangzhou1@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).