Re: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS

From: Marc Zyngier <maz@kernel.org>
To: Robin Murphy <robin.murphy@arm.com>,
	Sunil Muthuswamy <sunilmut@microsoft.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	"catalin.marinas@arm.com" <catalin.marinas@arm.com>,
	"will@kernel.org" <will@kernel.org>,
	Michael Kelley <mikelley@microsoft.com>,
	Boqun Feng <Boqun.Feng@microsoft.com>,
	KY Srinivasan <kys@microsoft.com>, Arnd Bergmann <arnd@arndb.de>
Subject: Re: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS
Date: Wed, 04 Aug 2021 10:21:23 +0100	[thread overview]
Message-ID: <87r1f9wooc.wl-maz@kernel.org> (raw)
In-Reply-To: <b2dea108-9166-dc9d-abd0-d22491f78568@arm.com>

On Tue, 03 Aug 2021 09:35:12 +0100,
Robin Murphy <robin.murphy@arm.com> wrote:
> 
> On 2021-08-03 03:11, Sunil Muthuswamy wrote:
> >   On Saturday, July 31, 2021 2:52 AM,
> >   Marc Zyngier <maz@kernel.org> wrote:
> >> 
> >> [...]
> >> 
> >>>> I also want to understand *how* you are going to plumb this into both
> >>>> ACPI and DT, given that neither understand how to link a PCI endpoint
> >>>> to a set of RDs.
> >>>> 
> >>>> 	M.
> >>> 
> >>> One way to do this for NUMA-aware systems would be to use the NUMA
> >>> related information that is available with PCI endpoints or root complex, to
> >>> pick a Redistributor/CPU that is in the NUMA node, as specified by the PCI
> >>> endpoint/root complex. In DT PCI devices can specify this using
> >>> 'numa-node-id' and in ACPI using the '_PXM (Proximity)'. For systems that
> >>> are not NUMA-aware, we can go with *any* Redistributor/CPU.
> >> 
> >> This makes zero sense. From the point of view of a device, all the RDs
> >> should be reachable, and firmware has no say in it. Dealing with
> >> interrupt affinity is the responsibility of the endpoint driver, and
> >> NUMA affinity is only a performance optimisation.
> >> 
> >>> Is there any additional information we would be able to gather from ACPI
> >>> or DT that's not there currently, that would be useful here?
> >> 
> >> You will need some firmware information describing that a given set of
> >> devices must use the RDs for their MSIs. Just like we currently
> >> describe it in IORT for the ITS. You cannot /assume/ things. At the
> >> moment, there is nothing at all, because no-one (including Microsoft)
> >> thought it would be a good idea not to have an ITS, which is also why
> >> ACPI doesn't describe MBIs as a potential MSI provider.
> >> 
> > I am a little bit confused by your above comment. Maybe you can help me
> > understand the ask. You indicate that from the point of the view of the
> > device, all the RDs should be reachable. But, then if we define a mapping
> > between PCI endpoint and RD in the firmware, we would be doing exactly
> > the opposite. i.e. restricting the RDs that are reachable by the device. Can
> > you please clarify?

Not at all. What I am saying is that you need to describe that the
MSIs have to be routed to the RDs, and there is no way to express this
at the moment.

> > 
> > Is your concern that the device should be able to only DMA to a subset of
> > GIC Redistributor, for the MSIs? If so, in the IORT, there is "memory address
> > size limit" for both device and root complex nodes. In the implementation,
> > we can enforce that the GICR is within that range. And, if a device deviates
> > further than that (ex: by having accessibility gaps within the GICR range),
> > then that is out of scope for support.
> 
> No, please don't try to abuse the Memory Address Size Limit - that has
> far more chance of adversely affecting normal DMA operation than of
> being any use here.
> 
> I believe the point Marc was trying to make is that firmware should
> not associate a device with any one *specific* redistributor, however
> ACPI currently has no way to describe that MSIs can target
> redistributors *at all*, only ITS groups - there is no such concept as
> a "redistributor group".

Thanks Robin.

That is exactly my point. There is no linkage whatsoever between a
device generating MSIs and the redistributors. In that respect, this
is the same issue we have with DT, as none of the firmware interfaces
can currently describe MSIs directly targeting the RDs. The only way
to describe MSIs with GICv3 in ACPI is to describe an ITS, and you
cannot use the *absence* of an ITS to decide and use DirectLPIs.

Given the state of things, using DirectLPI means that you need to
extend the firmware interfaces. This means both an extension to the DT
binding, and an update to the ACPI spec. There is no way around this.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel