RE: Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

RE: Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

[Pankaj Bansal (OSS)]

> -----Original Message-----
> From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Sent: Monday, February 17, 2020 8:55 PM
> To: Pankaj Bansal <pankaj.bansal@nxp.com>
> Cc: Marc Zyngier <maz@kernel.org>; Ard Biesheuvel
> <ard.biesheuvel@linaro.org>; Makarand Pawagi <makarand.pawagi@nxp.com>;
> Calvin Johnson <calvin.johnson@nxp.com>; stuyoder@gmail.com;
> nleeder@codeaurora.org; Ioana Ciornei <ioana.ciornei@nxp.com>; Cristi
> Sovaiala <cristian.sovaiala@nxp.com>; Hanjun Guo <guohanjun@huawei.com>;
> Will Deacon <will@kernel.org>; jon@solid-run.com; Russell King
> <linux@armlinux.org.uk>; ACPI Devel Maling List <linux-acpi@vger.kernel.org>;
> Len Brown <lenb@kernel.org>; Jason Cooper <jason@lakedaemon.net>; Andy
> Wang <Andy.Wang@arm.com>; Varun Sethi <V.Sethi@nxp.com>; Thomas
> Gleixner <tglx@linutronix.de>; linux-arm-kernel <linux-arm-
> kernel@lists.infradead.org>; Laurentiu Tudor <laurentiu.tudor@nxp.com>; Paul
> Yang <Paul.Yang@arm.com>; netdev@vger.kernel.org; Rafael J. Wysocki
> <rjw@rjwysocki.net>; Linux Kernel Mailing List <linux-kernel@vger.kernel.org>;
> Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> Sudeep Holla <sudeep.holla@arm.com>; Robin Murphy
> <robin.murphy@arm.com>
> Subject: Re: [EXT] Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc
> 
> On Mon, Feb 17, 2020 at 12:35:12PM +0000, Pankaj Bansal wrote:
> >
> >
> > > -----Original Message-----
> > > From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > > Sent: Friday, February 14, 2020 11:20 PM
> > > To: Pankaj Bansal <pankaj.bansal@nxp.com>
> > > Cc: Marc Zyngier <maz@kernel.org>; Ard Biesheuvel
> > > <ard.biesheuvel@linaro.org>; Makarand Pawagi
> <makarand.pawagi@nxp.com>;
> > > Calvin Johnson <calvin.johnson@nxp.com>; stuyoder@gmail.com;
> > > nleeder@codeaurora.org; Ioana Ciornei <ioana.ciornei@nxp.com>; Cristi
> > > Sovaiala <cristian.sovaiala@nxp.com>; Hanjun Guo
> <guohanjun@huawei.com>;
> > > Will Deacon <will@kernel.org>; jon@solid-run.com; Russell King
> > > <linux@armlinux.org.uk>; ACPI Devel Maling List <linux-
> acpi@vger.kernel.org>;
> > > Len Brown <lenb@kernel.org>; Jason Cooper <jason@lakedaemon.net>;
> Andy
> > > Wang <Andy.Wang@arm.com>; Varun Sethi <V.Sethi@nxp.com>; Thomas
> > > Gleixner <tglx@linutronix.de>; linux-arm-kernel <linux-arm-
> > > kernel@lists.infradead.org>; Laurentiu Tudor <laurentiu.tudor@nxp.com>;
> Paul
> > > Yang <Paul.Yang@arm.com>; netdev@vger.kernel.org; Rafael J. Wysocki
> > > <rjw@rjwysocki.net>; Linux Kernel Mailing List <linux-
> kernel@vger.kernel.org>;
> > > Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> > > Sudeep Holla <sudeep.holla@arm.com>; Robin Murphy
> > > <robin.murphy@arm.com>
> > > Subject: Re: [EXT] Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc
> > >
> > > On Fri, Feb 14, 2020 at 04:35:10PM +0000, Pankaj Bansal wrote:
> > >
> > > [...]
> > >
> > > > > -----Original Message-----
> > > > > From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > > > > Sent: Friday, February 14, 2020 9:50 PM
> > > > > To: Pankaj Bansal <pankaj.bansal@nxp.com>
> > > > > Cc: Marc Zyngier <maz@kernel.org>; Ard Biesheuvel
> > > > > <ard.biesheuvel@linaro.org>; Makarand Pawagi
> > > <makarand.pawagi@nxp.com>;
> > > > > Calvin Johnson <calvin.johnson@nxp.com>; stuyoder@gmail.com;
> > > > > nleeder@codeaurora.org; Ioana Ciornei <ioana.ciornei@nxp.com>; Cristi
> > > > > Sovaiala <cristian.sovaiala@nxp.com>; Hanjun Guo
> > > <guohanjun@huawei.com>;
> > > > > Will Deacon <will@kernel.org>; jon@solid-run.com; Russell King
> > > > > <linux@armlinux.org.uk>; ACPI Devel Maling List <linux-
> > > acpi@vger.kernel.org>;
> > > > > Len Brown <lenb@kernel.org>; Jason Cooper <jason@lakedaemon.net>;
> > > Andy
> > > > > Wang <Andy.Wang@arm.com>; Varun Sethi <V.Sethi@nxp.com>;
> Thomas
> > > > > Gleixner <tglx@linutronix.de>; linux-arm-kernel <linux-arm-
> > > > > kernel@lists.infradead.org>; Laurentiu Tudor
> <laurentiu.tudor@nxp.com>;
> > > Paul
> > > > > Yang <Paul.Yang@arm.com>; netdev@vger.kernel.org; Rafael J. Wysocki
> > > > > <rjw@rjwysocki.net>; Linux Kernel Mailing List <linux-
> > > kernel@vger.kernel.org>;
> > > > > Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> > > > > Sudeep Holla <sudeep.holla@arm.com>; Robin Murphy
> > > > > <robin.murphy@arm.com>
> > > > > Subject: Re: [EXT] Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc
> > > Side note: would you mind removing the email headers (as above) in your
> > > replies please ?
> 
> Read the question above please.
> 
> [...]
> 
> > > > As stated above, in Linux MC is a bus (just like PCI bus, AMBA bus etc)
> > > > There can be multiple devices attached to this bus. Moreover, we can
> > > dynamically create/destroy these devices.
> > > > Now, we want to represent this BUS (not individual devices connected to
> bus)
> > > in IORT table.
> > > > The only possible way right now we see is that we describe it as Named
> > > components having a pool of ID mappings.
> > > > As and when devices are created and attached to bus, we sift through this
> pool
> > > to correctly determine the output ID for the device.
> > > > Now the input ID that we provide, can come from device itself.
> > > > Then we can use the Platform MSI framework for MC bus devices.
> > >
> > > So are you asking me if that's OK ? Or there is something you can't
> > > describe with IORT ?
> >
> > I am asking if that would be acceptable?
> > i.e. we represent MC bus as Named component is IORT table with a pool of IDs
> (without single ID mapping flag)
> > and then we use the Platform MSI framework for all children devices of MC
> bus.
> > Note that it would require the Platform MSI layer to correctly pass an input id
> for a platform device to IORT layer.
> 
> How is this solved in DT ? You don't seem to need any DT binding on top
> of the msi-parent property, which is equivalent to IORT single mappings
> AFAICS so I would like to understand the whole DT flow (so that I
> understand how this FSL bus works) before commenting any further.

In DT case, we create the domain DOMAIN_BUS_FSL_MC_MSI for MC bus and it's children.
And then when MC child device is created, we search the "msi-parent" property from the MC
DT node and get the ITS associated with MC bus. Then we search DOMAIN_BUS_FSL_MC_MSI
on that ITS. Once we find the domain, we can call msi_domain_alloc_irqs for that domain.

This is exactly what we tried to do initially with ACPI. But the searching DOMAIN_BUS_FSL_MC_MSI
associated to an ITS, is something that is part of drivers/acpi/arm64/iort.c.
(similar to DOMAIN_BUS_PLATFORM_MSI and DOMAIN_BUS_PCI_MSI)

> 
> > And IORT layer ought to retrieve the output id based on single ID mapping flag
> as well as input id.
> >
> > >
> > > Side note: can you explain to me please how the MSI allocation flow
> > > and kernel data structures/drivers are modeled in DT ? I had a quick
> > > look at:
> > >
> > > drivers/irqchip/irq-gic-v3-its-fsl-mc-msi.c
> > >
> > > and to start with, does that code imply that we create a
> > > DOMAIN_BUS_FSL_MC_MSI on ALL DT systems with an ITS device node ?
> >
> > Yes. It's being done for all DT systems having ITS node.
> 
> This does not seem correct to me, I will let Marc comment on
> the matter.
> 
> > The domain creation is handled in drivers/bus/fsl-mc/fsl-mc-msi.c
> 
> Thanks,
> Lorenzo

Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

[Hanjun Guo]

Hi Pankaj,

On 2020/2/18 16:00, Pankaj Bansal (OSS) wrote:
[...]
>>>> Side note: would you mind removing the email headers (as above) in your
>>>> replies please ?
>>
>> Read the question above please.
>>
>> [...]
>>
>>>>> As stated above, in Linux MC is a bus (just like PCI bus, AMBA bus etc)
>>>>> There can be multiple devices attached to this bus. Moreover, we can
>>>> dynamically create/destroy these devices.
>>>>> Now, we want to represent this BUS (not individual devices connected to
>> bus)
>>>> in IORT table.
>>>>> The only possible way right now we see is that we describe it as Named
>>>> components having a pool of ID mappings.
>>>>> As and when devices are created and attached to bus, we sift through this
>> pool
>>>> to correctly determine the output ID for the device.
>>>>> Now the input ID that we provide, can come from device itself.
>>>>> Then we can use the Platform MSI framework for MC bus devices.
>>>>
>>>> So are you asking me if that's OK ? Or there is something you can't
>>>> describe with IORT ?
>>>
>>> I am asking if that would be acceptable?
>>> i.e. we represent MC bus as Named component is IORT table with a pool of IDs
>> (without single ID mapping flag)
>>> and then we use the Platform MSI framework for all children devices of MC
>> bus.
>>> Note that it would require the Platform MSI layer to correctly pass an input id
>> for a platform device to IORT layer.
>>
>> How is this solved in DT ? You don't seem to need any DT binding on top
>> of the msi-parent property, which is equivalent to IORT single mappings
>> AFAICS so I would like to understand the whole DT flow (so that I
>> understand how this FSL bus works) before commenting any further.
> 
> In DT case, we create the domain DOMAIN_BUS_FSL_MC_MSI for MC bus and it's children.
> And then when MC child device is created, we search the "msi-parent" property from the MC
> DT node and get the ITS associated with MC bus. Then we search DOMAIN_BUS_FSL_MC_MSI
> on that ITS. Once we find the domain, we can call msi_domain_alloc_irqs for that domain.
> 
> This is exactly what we tried to do initially with ACPI. But the searching DOMAIN_BUS_FSL_MC_MSI
> associated to an ITS, is something that is part of drivers/acpi/arm64/iort.c.
> (similar to DOMAIN_BUS_PLATFORM_MSI and DOMAIN_BUS_PCI_MSI)

Can you have a look at mbigen driver (drivers/irqchip/irq-mbigen.c) to see if
it helps you?

mbigen is an irq converter to convert device's wired interrupts into MSI
(connecting to ITS), which will alloc a bunch of MSIs from ITS platform MSI
domain at the setup.

Thanks
Hanjun

RE: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

[Pankaj Bansal (OSS)]

> >>>>> As stated above, in Linux MC is a bus (just like PCI bus, AMBA bus etc)
> >>>>> There can be multiple devices attached to this bus. Moreover, we can
> >>>> dynamically create/destroy these devices.
> >>>>> Now, we want to represent this BUS (not individual devices connected to
> >> bus)
> >>>> in IORT table.
> >>>>> The only possible way right now we see is that we describe it as Named
> >>>> components having a pool of ID mappings.
> >>>>> As and when devices are created and attached to bus, we sift through this
> >> pool
> >>>> to correctly determine the output ID for the device.
> >>>>> Now the input ID that we provide, can come from device itself.
> >>>>> Then we can use the Platform MSI framework for MC bus devices.
> >>>>
> >>>> So are you asking me if that's OK ? Or there is something you can't
> >>>> describe with IORT ?
> >>>
> >>> I am asking if that would be acceptable?
> >>> i.e. we represent MC bus as Named component is IORT table with a pool of
> IDs
> >> (without single ID mapping flag)
> >>> and then we use the Platform MSI framework for all children devices of MC
> >> bus.
> >>> Note that it would require the Platform MSI layer to correctly pass an input
> id
> >> for a platform device to IORT layer.
> >>
> >> How is this solved in DT ? You don't seem to need any DT binding on top
> >> of the msi-parent property, which is equivalent to IORT single mappings
> >> AFAICS so I would like to understand the whole DT flow (so that I
> >> understand how this FSL bus works) before commenting any further.
> >
> > In DT case, we create the domain DOMAIN_BUS_FSL_MC_MSI for MC bus and
> it's children.
> > And then when MC child device is created, we search the "msi-parent"
> property from the MC
> > DT node and get the ITS associated with MC bus. Then we search
> DOMAIN_BUS_FSL_MC_MSI
> > on that ITS. Once we find the domain, we can call msi_domain_alloc_irqs for
> that domain.
> >
> > This is exactly what we tried to do initially with ACPI. But the searching
> DOMAIN_BUS_FSL_MC_MSI
> > associated to an ITS, is something that is part of drivers/acpi/arm64/iort.c.
> > (similar to DOMAIN_BUS_PLATFORM_MSI and DOMAIN_BUS_PCI_MSI)
> 
> Can you have a look at mbigen driver (drivers/irqchip/irq-mbigen.c) to see if
> it helps you?
> 
> mbigen is an irq converter to convert device's wired interrupts into MSI
> (connecting to ITS), which will alloc a bunch of MSIs from ITS platform MSI
> domain at the setup.

Unfortunately this is not the same case as ours. As I see Hisilicon IORT table
Is using single id mapping with named components.

https://github.com/tianocore/edk2-platforms/blob/master/Silicon/Hisilicon/Hi1616/D05AcpiTables/D05Iort.asl#L300

while we are not:

https://source.codeaurora.org/external/qoriq/qoriq-components/edk2-platforms/tree/Platform/NXP/LX2160aRdbPkg/AcpiTables/Iort.aslc?h=LX2160_UEFI_ACPI_EAR1#n290

This is because as I said, we are trying to represent a bus in IORT via named components and
not individual devices connected to that bus.

> 
> Thanks
> Hanjun

Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

[Lorenzo Pieralisi]

On Tue, Feb 18, 2020 at 12:48:39PM +0000, Pankaj Bansal (OSS) wrote:

[...]

> > > In DT case, we create the domain DOMAIN_BUS_FSL_MC_MSI for MC bus and
> > it's children.
> > > And then when MC child device is created, we search the "msi-parent"
> > property from the MC
> > > DT node and get the ITS associated with MC bus. Then we search
> > DOMAIN_BUS_FSL_MC_MSI
> > > on that ITS. Once we find the domain, we can call msi_domain_alloc_irqs for
> > that domain.
> > >
> > > This is exactly what we tried to do initially with ACPI. But the searching
> > DOMAIN_BUS_FSL_MC_MSI
> > > associated to an ITS, is something that is part of drivers/acpi/arm64/iort.c.
> > > (similar to DOMAIN_BUS_PLATFORM_MSI and DOMAIN_BUS_PCI_MSI)
> > 
> > Can you have a look at mbigen driver (drivers/irqchip/irq-mbigen.c) to see if
> > it helps you?
> > 
> > mbigen is an irq converter to convert device's wired interrupts into MSI
> > (connecting to ITS), which will alloc a bunch of MSIs from ITS platform MSI
> > domain at the setup.
> 
> Unfortunately this is not the same case as ours. As I see Hisilicon IORT table
> Is using single id mapping with named components.
> 
> https://github.com/tianocore/edk2-platforms/blob/master/Silicon/Hisilicon/Hi1616/D05AcpiTables/D05Iort.asl#L300
> 
> while we are not:
> 
> https://source.codeaurora.org/external/qoriq/qoriq-components/edk2-platforms/tree/Platform/NXP/LX2160aRdbPkg/AcpiTables/Iort.aslc?h=LX2160_UEFI_ACPI_EAR1#n290
> 
> This is because as I said, we are trying to represent a bus in IORT
> via named components and not individual devices connected to that bus.

I had a thorough look into this and strictly speaking there is no
*mapping* requirement at all, all you need to know is what ITS the FSL
MC bus is mapping MSIs to. Which brings me to the next question (which
is orthogonal to how to model FSL MC in IORT, that has to be discussed
but I want to have a full picture in mind first).

When you probe the FSL MC as a platform device, the ACPI core,
through IORT (if you add the 1:1 mapping as an array of single
mappings) already link the platform device to ITS platform
device MSI domain (acpi_configure_pmsi_domain()).

The associated fwnode is the *same* (IIUC) as for the
DOMAIN_BUS_FSL_MC_MSI and ITS DOMAIN_BUS_NEXUS, so in practice
you don't need IORT code to retrieve the DOMAIN_BUS_FSL_MC_MSI
domain, the fwnode is the same as the one in the FSL MC platform
device IRQ domain->fwnode pointer and you can use it to
retrieve the DOMAIN_BUS_FSL_MC_MSI domain through it.

Is my reading correct ?

Overall, DOMAIN_BUS_FSL_MC_MSI is just an MSI layer to override the
provide the MSI domain ->prepare hook (ie to stash the MC device id), no
more (ie its_fsl_mc_msi_prepare()).

That's it for the MSI layer - I need to figure out whether we *want* to
extend IORT (and/or ACPI) to defined bindings for "additional busses",
what I write above is a summary of my understanding, I have not made my
mind up yet.

As for the IOMMU code, it seems like the only thing needed is
extending named components configuration to child devices,
hierarchically.

As Marc already mentioned, IOMMU and IRQ code must be separate for
future postings but first we need to find a suitable answer to
the problem at hand.

Lorenzo

Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

[Robin Murphy]

On 18/02/2020 2:46 pm, Lorenzo Pieralisi wrote:
> On Tue, Feb 18, 2020 at 12:48:39PM +0000, Pankaj Bansal (OSS) wrote:
> 
> [...]
> 
>>>> In DT case, we create the domain DOMAIN_BUS_FSL_MC_MSI for MC bus and
>>> it's children.
>>>> And then when MC child device is created, we search the "msi-parent"
>>> property from the MC
>>>> DT node and get the ITS associated with MC bus. Then we search
>>> DOMAIN_BUS_FSL_MC_MSI
>>>> on that ITS. Once we find the domain, we can call msi_domain_alloc_irqs for
>>> that domain.
>>>>
>>>> This is exactly what we tried to do initially with ACPI. But the searching
>>> DOMAIN_BUS_FSL_MC_MSI
>>>> associated to an ITS, is something that is part of drivers/acpi/arm64/iort.c.
>>>> (similar to DOMAIN_BUS_PLATFORM_MSI and DOMAIN_BUS_PCI_MSI)
>>>
>>> Can you have a look at mbigen driver (drivers/irqchip/irq-mbigen.c) to see if
>>> it helps you?
>>>
>>> mbigen is an irq converter to convert device's wired interrupts into MSI
>>> (connecting to ITS), which will alloc a bunch of MSIs from ITS platform MSI
>>> domain at the setup.
>>
>> Unfortunately this is not the same case as ours. As I see Hisilicon IORT table
>> Is using single id mapping with named components.
>>
>> https://github.com/tianocore/edk2-platforms/blob/master/Silicon/Hisilicon/Hi1616/D05AcpiTables/D05Iort.asl#L300
>>
>> while we are not:
>>
>> https://source.codeaurora.org/external/qoriq/qoriq-components/edk2-platforms/tree/Platform/NXP/LX2160aRdbPkg/AcpiTables/Iort.aslc?h=LX2160_UEFI_ACPI_EAR1#n290
>>
>> This is because as I said, we are trying to represent a bus in IORT
>> via named components and not individual devices connected to that bus.
> 
> I had a thorough look into this and strictly speaking there is no
> *mapping* requirement at all, all you need to know is what ITS the FSL
> MC bus is mapping MSIs to. Which brings me to the next question (which
> is orthogonal to how to model FSL MC in IORT, that has to be discussed
> but I want to have a full picture in mind first).
> 
> When you probe the FSL MC as a platform device, the ACPI core,
> through IORT (if you add the 1:1 mapping as an array of single
> mappings) already link the platform device to ITS platform
> device MSI domain (acpi_configure_pmsi_domain()).
> 
> The associated fwnode is the *same* (IIUC) as for the
> DOMAIN_BUS_FSL_MC_MSI and ITS DOMAIN_BUS_NEXUS, so in practice
> you don't need IORT code to retrieve the DOMAIN_BUS_FSL_MC_MSI
> domain, the fwnode is the same as the one in the FSL MC platform
> device IRQ domain->fwnode pointer and you can use it to
> retrieve the DOMAIN_BUS_FSL_MC_MSI domain through it.
> 
> Is my reading correct ?
> 
> Overall, DOMAIN_BUS_FSL_MC_MSI is just an MSI layer to override the
> provide the MSI domain ->prepare hook (ie to stash the MC device id), no
> more (ie its_fsl_mc_msi_prepare()).
> 
> That's it for the MSI layer - I need to figure out whether we *want* to
> extend IORT (and/or ACPI) to defined bindings for "additional busses",
> what I write above is a summary of my understanding, I have not made my
> mind up yet.

I'm really not sure we'd need to go near any bindings - the IORT spec 
*can* reasonably describe "giant black box of DPAA2 stuff" as a single 
named component, and that's arguably the most accurate abstraction 
already, even when it comes to the namespace device. This isn't a bus in 
any traditional sense, it's a set of accelerator components with an 
interface to dynamically configure them into custom pipelines, and the 
expected use-case seems to be for userspace to freely reconfigure 
whatever virtual network adapters it wants at any given time. Thus I 
don't see that it's logical or even practical for firmware itself to be 
involved beyond describing "here's your toolbox", and in particular, 
basing any decisions on the particular way that DPAA2 has been 
shoehorned into the Linux driver model would almost certainly be a step 
in the wrong direction.

IMO the scope of this issue belongs entirely within the 
implementation(s) of Linux's own abstraction layers.

Robin.

> As for the IOMMU code, it seems like the only thing needed i
> extending named components configuration to child devices,
> hierarchically.
> 
> As Marc already mentioned, IOMMU and IRQ code must be separate for
> future postings but first we need to find a suitable answer to
> the problem at hand.
> 
> Lorenzo
> 

Re: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

[Diana Craciun OSS]

Hi Lorenzo,

On 2/18/2020 4:46 PM, Lorenzo Pieralisi wrote:
> On Tue, Feb 18, 2020 at 12:48:39PM +0000, Pankaj Bansal (OSS) wrote:
>
> [...]
>
>>>> In DT case, we create the domain DOMAIN_BUS_FSL_MC_MSI for MC bus and
>>> it's children.
>>>> And then when MC child device is created, we search the "msi-parent"
>>> property from the MC
>>>> DT node and get the ITS associated with MC bus. Then we search
>>> DOMAIN_BUS_FSL_MC_MSI
>>>> on that ITS. Once we find the domain, we can call msi_domain_alloc_irqs for
>>> that domain.
>>>> This is exactly what we tried to do initially with ACPI. But the searching
>>> DOMAIN_BUS_FSL_MC_MSI
>>>> associated to an ITS, is something that is part of drivers/acpi/arm64/iort.c.
>>>> (similar to DOMAIN_BUS_PLATFORM_MSI and DOMAIN_BUS_PCI_MSI)
>>> Can you have a look at mbigen driver (drivers/irqchip/irq-mbigen.c) to see if
>>> it helps you?
>>>
>>> mbigen is an irq converter to convert device's wired interrupts into MSI
>>> (connecting to ITS), which will alloc a bunch of MSIs from ITS platform MSI
>>> domain at the setup.
>> Unfortunately this is not the same case as ours. As I see Hisilicon IORT table
>> Is using single id mapping with named components.
>>
>> https://github.com/tianocore/edk2-platforms/blob/master/Silicon/Hisilicon/Hi1616/D05AcpiTables/D05Iort.asl#L300
>>
>> while we are not:
>>
>> https://source.codeaurora.org/external/qoriq/qoriq-components/edk2-platforms/tree/Platform/NXP/LX2160aRdbPkg/AcpiTables/Iort.aslc?h=LX2160_UEFI_ACPI_EAR1#n290
>>
>> This is because as I said, we are trying to represent a bus in IORT
>> via named components and not individual devices connected to that bus.
> I had a thorough look into this and strictly speaking there is no
> *mapping* requirement at all, all you need to know is what ITS the FSL
> MC bus is mapping MSIs to. Which brings me to the next question (which
> is orthogonal to how to model FSL MC in IORT, that has to be discussed
> but I want to have a full picture in mind first).
>
> When you probe the FSL MC as a platform device, the ACPI core,
> through IORT (if you add the 1:1 mapping as an array of single
> mappings) already link the platform device to ITS platform
> device MSI domain (acpi_configure_pmsi_domain()).
>
> The associated fwnode is the *same* (IIUC) as for the
> DOMAIN_BUS_FSL_MC_MSI and ITS DOMAIN_BUS_NEXUS, so in practice
> you don't need IORT code to retrieve the DOMAIN_BUS_FSL_MC_MSI
> domain, the fwnode is the same as the one in the FSL MC platform
> device IRQ domain->fwnode pointer and you can use it to
> retrieve the DOMAIN_BUS_FSL_MC_MSI domain through it.
>
> Is my reading correct ?

Thank you very much for your effort! Really appreciated! Yes, the 
understanding is correct. I have prototyped this idea for DT, see below [1].
So, I get the fwnode from the platform device domain (because they are 
the same with the devices underneath the MC-BUS bridge) and use the 
fwnode to retrieve the MC-BUS domain.

>
> Overall, DOMAIN_BUS_FSL_MC_MSI is just an MSI layer to override the
> provide the MSI domain ->prepare hook (ie to stash the MC device id), no
> more (ie its_fsl_mc_msi_prepare()).
>
> That's it for the MSI layer - I need to figure out whether we *want* to
> extend IORT (and/or ACPI) to defined bindings for "additional busses",
> what I write above is a summary of my understanding, I have not made my
> mind up yet.
>
> As for the IOMMU code, it seems like the only thing needed is
> extending named components configuration to child devices,
> hierarchically.

Laurentiu used a similar approach for DMA configuration (again 
prototyped for DT). [2]
It involves wiring up a custom .dma_configure for our devices as anyway, 
it made little sense to pretend that these devices are platform devices 
and trick the DT or ACPI layers into that. As a nice side effect, this 
will allow to get rid of our existing hooks in the DT generic code.

>
> As Marc already mentioned, IOMMU and IRQ code must be separate for
> future postings but first we need to find a suitable answer to
> the problem at hand.
>
> Lorenzo
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[1] MSI configuration

  drivers/bus/fsl-mc/fsl-mc-msi.c | 11 +++++++++--
  1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/bus/fsl-mc/fsl-mc-msi.c 
b/drivers/bus/fsl-mc/fsl-mc-msi.c
index 8b9c66d7c4ff..674f5a60109b 100644
--- a/drivers/bus/fsl-mc/fsl-mc-msi.c
+++ b/drivers/bus/fsl-mc/fsl-mc-msi.c
@@ -182,16 +182,23 @@ int fsl_mc_find_msi_domain(struct device 
*mc_platform_dev,
  {
      struct irq_domain *msi_domain;
      struct device_node *mc_of_node = mc_platform_dev->of_node;
+    struct fwnode_handle *fwnode;

-    msi_domain = of_msi_get_domain(mc_platform_dev, mc_of_node,
-                       DOMAIN_BUS_FSL_MC_MSI);
+    msi_domain = dev_get_msi_domain(mc_platform_dev);
      if (!msi_domain) {
          pr_err("Unable to find fsl-mc MSI domain for %pOF\n",
                 mc_of_node);

          return -ENOENT;
      }
+    fwnode = msi_domain->fwnode;
+    msi_domain = irq_find_matching_fwnode(fwnode, DOMAIN_BUS_FSL_MC_MSI);
+    if (!msi_domain) {
+        pr_err("Unable to find fsl-mc MSI domain for %pOF\n",
+              mc_of_node);

+        return -ENOENT;
+    }
      *mc_msi_domain = msi_domain;
      return 0;
  }
-- 
2.17.1



[2] DMA configuration

  drivers/bus/fsl-mc/fsl-mc-bus.c | 42 ++++++++++++++++++++++++++++++++-
  1 file changed, 41 insertions(+), 1 deletion(-)

diff --git a/drivers/bus/fsl-mc/fsl-mc-bus.c 
b/drivers/bus/fsl-mc/fsl-mc-bus.c
index f9bc9c384ab5..5c6021a13612 100644
--- a/drivers/bus/fsl-mc/fsl-mc-bus.c
+++ b/drivers/bus/fsl-mc/fsl-mc-bus.c
@@ -132,11 +132,51 @@ static int fsl_mc_bus_uevent(struct device *dev, 
struct kobj_uevent_env *env)
  static int fsl_mc_dma_configure(struct device *dev)
  {
      struct device *dma_dev = dev;
+    struct iommu_fwspec *fwspec;
+    const struct iommu_ops *iommu_ops;
+    struct fsl_mc_device *mc_dev = to_fsl_mc_device(dev);
+    int ret;
+    u32 icid;

      while (dev_is_fsl_mc(dma_dev))
          dma_dev = dma_dev->parent;

-    return of_dma_configure(dev, dma_dev->of_node, 0);
+    fwspec = dev_iommu_fwspec_get(dma_dev);
+    if (!fwspec) {
+        dev_err(dev, "%s: null fwspec\n", __func__);
+        return -ENODEV;
+    }
+    iommu_ops = iommu_ops_from_fwnode(fwspec->iommu_fwnode);
+    if (!iommu_ops) {
+        dev_err(dev, "%s: null iommu ops\n", __func__);
+        return -ENODEV;
+    }
+
+    ret = iommu_fwspec_init(dev, fwspec->iommu_fwnode, iommu_ops);
+    if (ret) {
+        dev_err(dev, "%s: iommu_fwspec_init failed with %d\n", 
__func__, ret);
+        return ret;
+    }
+
+    icid = mc_dev->icid;
+    ret = iommu_fwspec_add_ids(dev, &icid, 1);
+    if (ret) {
+        dev_err(dev, "%s: iommu_fwspec_add_ids failed with %d\n", 
__func__, ret);
+        return ret;
+    }
+
+    if (!device_iommu_mapped(dev)) {
+        ret = iommu_probe_device(dev);
+        if (ret) {
+            dev_err(dev, "%s: iommu_fwspec_add_ids failed with %d\n", 
__func__, ret);
+            return ret;
+        }
+    }
+
+    arch_setup_dma_ops(dev, 0, *dma_dev->dma_mask + 1,
+                iommu_ops, true);
+
+    return 0;
  }

  static ssize_t modalias_show(struct device *dev, struct 
device_attribute *attr,
-- 
2.17.1

Regards,
Diana

RE: [PATCH] bus: fsl-mc: Add ACPI support for fsl-mc

[Pankaj Bansal (OSS)]

> 
> On 18/02/2020 2:46 pm, Lorenzo Pieralisi wrote:
> > On Tue, Feb 18, 2020 at 12:48:39PM +0000, Pankaj Bansal (OSS) wrote:
> >
> > [...]
> >
> >>>> In DT case, we create the domain DOMAIN_BUS_FSL_MC_MSI for MC bus
> and
> >>> it's children.
> >>>> And then when MC child device is created, we search the "msi-parent"
> >>> property from the MC
> >>>> DT node and get the ITS associated with MC bus. Then we search
> >>> DOMAIN_BUS_FSL_MC_MSI
> >>>> on that ITS. Once we find the domain, we can call msi_domain_alloc_irqs
> for
> >>> that domain.
> >>>>
> >>>> This is exactly what we tried to do initially with ACPI. But the searching
> >>> DOMAIN_BUS_FSL_MC_MSI
> >>>> associated to an ITS, is something that is part of drivers/acpi/arm64/iort.c.
> >>>> (similar to DOMAIN_BUS_PLATFORM_MSI and DOMAIN_BUS_PCI_MSI)
> >>>
> >>> Can you have a look at mbigen driver (drivers/irqchip/irq-mbigen.c) to see if
> >>> it helps you?
> >>>
> >>> mbigen is an irq converter to convert device's wired interrupts into MSI
> >>> (connecting to ITS), which will alloc a bunch of MSIs from ITS platform MSI
> >>> domain at the setup.
> >>
> >> Unfortunately this is not the same case as ours. As I see Hisilicon IORT table
> >> Is using single id mapping with named components.
> >>
> >> https://github.com/tianocore/edk2-
> platforms/blob/master/Silicon/Hisilicon/Hi1616/D05AcpiTables/D05Iort.asl#L30
> 0
> >>
> >> while we are not:
> >>
> >> https://source.codeaurora.org/external/qoriq/qoriq-components/edk2-
> platforms/tree/Platform/NXP/LX2160aRdbPkg/AcpiTables/Iort.aslc?h=LX2160_
> UEFI_ACPI_EAR1#n290
> >>
> >> This is because as I said, we are trying to represent a bus in IORT
> >> via named components and not individual devices connected to that bus.
> >
> > I had a thorough look into this and strictly speaking there is no
> > *mapping* requirement at all, all you need to know is what ITS the FSL
> > MC bus is mapping MSIs to. Which brings me to the next question (which
> > is orthogonal to how to model FSL MC in IORT, that has to be discussed
> > but I want to have a full picture in mind first).
> >
> > When you probe the FSL MC as a platform device, the ACPI core,
> > through IORT (if you add the 1:1 mapping as an array of single
> > mappings) already link the platform device to ITS platform
> > device MSI domain (acpi_configure_pmsi_domain()).
> >
> > The associated fwnode is the *same* (IIUC) as for the
> > DOMAIN_BUS_FSL_MC_MSI and ITS DOMAIN_BUS_NEXUS, so in practice
> > you don't need IORT code to retrieve the DOMAIN_BUS_FSL_MC_MSI
> > domain, the fwnode is the same as the one in the FSL MC platform
> > device IRQ domain->fwnode pointer and you can use it to
> > retrieve the DOMAIN_BUS_FSL_MC_MSI domain through it.
> >
> > Is my reading correct ?
> >
> > Overall, DOMAIN_BUS_FSL_MC_MSI is just an MSI layer to override the
> > provide the MSI domain ->prepare hook (ie to stash the MC device id), no
> > more (ie its_fsl_mc_msi_prepare()).
> >
> > That's it for the MSI layer - I need to figure out whether we *want* to
> > extend IORT (and/or ACPI) to defined bindings for "additional busses",
> > what I write above is a summary of my understanding, I have not made my
> > mind up yet.
> 
> I'm really not sure we'd need to go near any bindings - the IORT spec
> *can* reasonably describe "giant black box of DPAA2 stuff" as a single
> named component, and that's arguably the most accurate abstraction
> already, even when it comes to the namespace device. This isn't a bus in
> any traditional sense, it's a set of accelerator components with an
> interface to dynamically configure them into custom pipelines, and the
> expected use-case seems to be for userspace to freely reconfigure
> whatever virtual network adapters it wants at any given time. Thus I
> don't see that it's logical or even practical for firmware itself to be
> involved beyond describing "here's your toolbox", and in particular,
> basing any decisions on the particular way that DPAA2 has been
> shoehorned into the Linux driver model would almost certainly be a step
> in the wrong direction.
> 
> IMO the scope of this issue belongs entirely within the
> implementation(s) of Linux's own abstraction layers.

I agree. I think first we ought to get the consensus on how to represent the MC
bus in IORT table. And it should not be based on the fact that "that's how we have
handled IORT in linux". Once this is done, then we can move forward on how to
handle that in linux.

> 
> Robin.
> 
> > As for the IOMMU code, it seems like the only thing needed i
> > extending named components configuration to child devices,
> > hierarchically.
> >
> > As Marc already mentioned, IOMMU and IRQ code must be separate for
> > future postings but first we need to find a suitable answer to
> > the problem at hand.
> >
> > Lorenzo
> >