Re: [PATCH v4 0/2] irqchip/gic-v3-its: Balance LPI affinity across CPUs

From: John Garry <john.garry@huawei.com>
To: Marc Zyngier <maz@kernel.org>, <linux-kernel@vger.kernel.org>,
	<linux-arm-kernel@lists.infradead.org>
Cc: chenxiang <chenxiang66@hisilicon.com>,
	Zhou Wang <wangzhou1@hisilicon.com>,
	Ming Lei <ming.lei@redhat.com>,
	Jason Cooper <jason@lakedaemon.net>,
	Thomas Gleixner <tglx@linutronix.de>, <kernel-team@android.com>
Subject: Re: [PATCH v4 0/2] irqchip/gic-v3-its: Balance LPI affinity across CPUs
Date: Fri, 15 May 2020 19:09:16 +0100	[thread overview]
Message-ID: <f87f34fa-a512-7f73-2f53-85333257dd8e@huawei.com> (raw)
In-Reply-To: <20200515165752.121296-1-maz@kernel.org>

On 15/05/2020 17:57, Marc Zyngier wrote:
> When mapping a LPI, the ITS driver picks the first possible
> affinity, which is in most cases CPU0, assuming that if
> that's not suitable, someone will come and set the affinity
> to something more interesting.
> 
> It apparently isn't the case, and people complain of poor
> performance when many interrupts are glued to the same CPU.
> So let's place the interrupts by finding the "least loaded"
> CPU (that is, the one that has the fewer LPIs mapped to it).
> So called 'managed' interrupts are an interesting case where
> the affinity is actually dictated by the kernel itself, and
> we should honor this.
> 

Cheers Marc, I ran these again and the figures look good:
NVMe with nvme.use_threaded_interrupts=1/0
Before: 1000K/1100K IOPS
After: 1100K/1400K IOPS

For some reason v5.7-rc5 looks more stable than rc4 (which I tested 
previously).

Tested-by: John Garry <john.garry@huawei.com>

> * From v3:
>    - Always pre-decrement/post-increment affinity to avoid useless
>      changes of affinity (John)
>    - Don't use the node mask as a superset of the proposed affinity
>      as the ACPI tables can't really describe this (John)
>    - Rebased on v5.7-rc5
> 
> * From v2:
>    - Split accounting from CPU selection
>    - Track managed and unmanaged interrupts separately
> 
> Marc Zyngier (2):
>    irqchip/gic-v3-its: Track LPI distribution on a per CPU basis
>    irqchip/gic-v3-its: Balance initial LPI affinity across CPUs
> 
>   drivers/irqchip/irq-gic-v3-its.c | 170 ++++++++++++++++++++++++++-----
>   1 file changed, 143 insertions(+), 27 deletions(-)
> 