From: Sunil Muthuswamy <firstname.lastname@example.org>
To: Marc Zyngier <email@example.com>, Robin Murphy <firstname.lastname@example.org>
Cc: Thomas Gleixner <email@example.com>,
Michael Kelley <firstname.lastname@example.org>,
Boqun Feng <Boqun.Feng@microsoft.com>,
KY Srinivasan <email@example.com>, Arnd Bergmann <firstname.lastname@example.org>
Subject: RE: [EXTERNAL] Re: [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS
Date: Wed, 4 Aug 2021 20:10:43 +0000 [thread overview]
Message-ID: <MW4PR21MB2002860BAED850B2B52D6E2BC0F19@MW4PR21MB2002.namprd21.prod.outlook.com> (raw)
On Wednesday, August 4, 2021 2:21 AM,
Marc Zyngier <email@example.com> wrote:
> On Tue, 03 Aug 2021 09:35:12 +0100,
> Robin Murphy <firstname.lastname@example.org> wrote:
> > On 2021-08-03 03:11, Sunil Muthuswamy wrote:
> > > On Saturday, July 31, 2021 2:52 AM,
> > > Marc Zyngier <email@example.com> wrote:
> > >>
> > >> [...]
> > >>
> > >>>> I also want to understand *how* you are going to plumb this into both
> > >>>> ACPI and DT, given that neither understand how to link a PCI endpoint
> > >>>> to a set of RDs.
> > >>>>
> > >>>> M.
> > >>>
> > >>> One way to do this for NUMA-aware systems would be to use the NUMA
> > >>> related information that is available with PCI endpoints or root complex, to
> > >>> pick a Redistributor/CPU that is in the NUMA node, as specified by the PCI
> > >>> endpoint/root complex. In DT PCI devices can specify this using
> > >>> 'numa-node-id' and in ACPI using the '_PXM (Proximity)'. For systems that
> > >>> are not NUMA-aware, we can go with *any* Redistributor/CPU.
> > >>
> > >> This makes zero sense. From the point of view of a device, all the RDs
> > >> should be reachable, and firmware has no say in it. Dealing with
> > >> interrupt affinity is the responsibility of the endpoint driver, and
> > >> NUMA affinity is only a performance optimisation.
> > >>
> > >>> Is there any additional information we would be able to gather from ACPI
> > >>> or DT that's not there currently, that would be useful here?
> > >>
> > >> You will need some firmware information describing that a given set of
> > >> devices must use the RDs for their MSIs. Just like we currently
> > >> describe it in IORT for the ITS. You cannot /assume/ things. At the
> > >> moment, there is nothing at all, because no-one (including Microsoft)
> > >> thought it would be a good idea not to have an ITS, which is also why
> > >> ACPI doesn't describe MBIs as a potential MSI provider.
> > >>
> > > I am a little bit confused by your above comment. Maybe you can help me
> > > understand the ask. You indicate that from the point of the view of the
> > > device, all the RDs should be reachable. But, then if we define a mapping
> > > between PCI endpoint and RD in the firmware, we would be doing exactly
> > > the opposite. i.e. restricting the RDs that are reachable by the device. Can
> > > you please clarify?
> Not at all. What I am saying is that you need to describe that the
> MSIs have to be routed to the RDs, and there is no way to express this
> at the moment.
> > >
> > > Is your concern that the device should be able to only DMA to a subset of
> > > GIC Redistributor, for the MSIs? If so, in the IORT, there is "memory address
> > > size limit" for both device and root complex nodes. In the implementation,
> > > we can enforce that the GICR is within that range. And, if a device deviates
> > > further than that (ex: by having accessibility gaps within the GICR range),
> > > then that is out of scope for support.
> > No, please don't try to abuse the Memory Address Size Limit - that has
> > far more chance of adversely affecting normal DMA operation than of
> > being any use here.
> > I believe the point Marc was trying to make is that firmware should
> > not associate a device with any one *specific* redistributor, however
> > ACPI currently has no way to describe that MSIs can target
> > redistributors *at all*, only ITS groups - there is no such concept as
> > a "redistributor group".
> Thanks Robin.
> That is exactly my point. There is no linkage whatsoever between a
> device generating MSIs and the redistributors. In that respect, this
> is the same issue we have with DT, as none of the firmware interfaces
> can currently describe MSIs directly targeting the RDs. The only way
> to describe MSIs with GICv3 in ACPI is to describe an ITS, and you
> cannot use the *absence* of an ITS to decide and use DirectLPIs.
> Given the state of things, using DirectLPI means that you need to
> extend the firmware interfaces. This means both an extension to the DT
> binding, and an update to the ACPI spec. There is no way around this.
Thanks Marc and Robin for clarifying. I see and understand the point
about having explicit MSI mappings in the firmware specification for
Direct LPIs for generic hardware support.
Hey Mark, would you be willing to consider a scoped down implementation of
GIC Direct LPI with just an IRQ chip implementation and no
Direct LPI PCI-MSI IRQ chip. This will allow a MSI provider (such as Hyper-V vPCI) to
provide a PCI-MSI IRQ chip on top of the Direct LPI IRQ chip and enable
PCI-MSI scenarios, and avoid building in assumptions in other cases (like PCI) where
firmware specification is not available.
- This scoped down implementation would allow Microsoft to build virtual
PCI on top, to enable our business needs.
- If there's a need for a generic support for a Direct LPI PCI MSI IRQ, that could
drive firmware revision efforts, and we are happy to assist there.
Looking forward to hearing back.
next prev parent reply other threads:[~2021-08-04 20:10 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-08 19:36 [RFC 1/1] irqchip/gic-v3-its: Add irq domain and chip for Direct LPI without ITS Sunil Muthuswamy
2021-07-11 11:09 ` Marc Zyngier
2021-07-26 15:33 ` [EXTERNAL] " Sunil Muthuswamy
2021-07-31 9:52 ` Marc Zyngier
2021-08-03 2:11 ` Sunil Muthuswamy
2021-08-03 8:35 ` Robin Murphy
2021-08-04 9:21 ` Marc Zyngier
2021-08-04 20:10 ` Sunil Muthuswamy [this message]
2021-08-05 8:35 ` Marc Zyngier
2021-08-06 19:14 ` Sunil Muthuswamy
2021-08-08 10:19 ` Marc Zyngier
2021-08-09 2:35 ` Sunil Muthuswamy
2021-08-09 9:15 ` Marc Zyngier
2021-08-10 1:10 ` Sunil Muthuswamy
2021-08-10 13:57 ` Marc Zyngier
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).