Question: multiple SMMUv3 devices probe

Question: multiple SMMUv3 devices probe

[richard clark]

Hi guys,
There're 3 SMMUv3 devices in my arm64 box. I am confused by the below
probe process:

arm_smmu_device_probe(...) --> arm_smmu_set_bus_ops(&arm_smmu_ops):
if (platform_bus_type.iommu_ops != ops) {
        err = bus_set_iommu(&platform_bus_type, ops);
        if (err)
}

so for the 1st smmu device on the platform_bus, the
arm_smmu_device_probe(...) will set the platform_bus_type.iommu_bus =
arm_smmu_ops, so for the arm_smmu_device_probe(...) when probing the
2nd smmu device, the bus_set_iommu() will have no chance be called
since the platform_bus_type.iommu_bus == arm_smmu_ops is true, my
question is how the followed bus_set_iommu(...) being called?

Thanks,
RC

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Question: multiple SMMUv3 devices probe

[richard clark]

> Hi guys,
> There're 3 SMMUv3 devices in my arm64 box. I am confused by the below
> probe process:
>
> arm_smmu_device_probe(...) --> arm_smmu_set_bus_ops(&arm_smmu_ops):
> if (platform_bus_type.iommu_ops != ops) {
>         err = bus_set_iommu(&platform_bus_type, ops);
>         if (err)
> }
>
> so for the 1st smmu device on the platform_bus, the
> arm_smmu_device_probe(...) will set the platform_bus_type.iommu_bus =
> arm_smmu_ops, so for the arm_smmu_device_probe(...) when probing the
> 2nd smmu device, the bus_set_iommu() will have no chance be called
> since the platform_bus_type.iommu_bus == arm_smmu_ops is true, my
> question is how the followed bus_set_iommu(...) being called?
>
> Thanks,
> RC

For this question,  any comments about that? Will, the owner of the
device driver, do you have some any? How about others too?
Thanks,
RC

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Question: multiple SMMUv3 devices probe

[Jean-Philippe Brucker]

Hi,

On Wed, Apr 07, 2021 at 05:37:52PM +0800, richard clark wrote:
> Hi guys,
> There're 3 SMMUv3 devices in my arm64 box. I am confused by the below
> probe process:
> 
> arm_smmu_device_probe(...) --> arm_smmu_set_bus_ops(&arm_smmu_ops):
> if (platform_bus_type.iommu_ops != ops) {
>         err = bus_set_iommu(&platform_bus_type, ops);
>         if (err)
> }
> 
> so for the 1st smmu device on the platform_bus, the
> arm_smmu_device_probe(...) will set the platform_bus_type.iommu_bus =
> arm_smmu_ops, so for the arm_smmu_device_probe(...) when probing the
> 2nd smmu device, the bus_set_iommu() will have no chance be called
> since the platform_bus_type.iommu_bus == arm_smmu_ops is true, my
> question is how the followed bus_set_iommu(...) being called?

arm_smmu_ops is the same for all SMMU devices, so bus_set_iommu() only
needs to be called once.

Thanks,
Jean

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Question: multiple SMMUv3 devices probe

[richard clark]

Jean-Philippe Brucker <jean-philippe@linaro.org> 于2021年4月13日周二 下午8:58写道：
>
> On Tue, Apr 13, 2021 at 08:40:47PM +0800, richard clark wrote:
> > >
> > > Hi,
> > >
> > > On Wed, Apr 07, 2021 at 05:37:52PM +0800, richard clark wrote:
> > > > Hi guys,
> > > > There're 3 SMMUv3 devices in my arm64 box. I am confused by the below
> > > > probe process:
> > > >
> > > > arm_smmu_device_probe(...) --> arm_smmu_set_bus_ops(&arm_smmu_ops):
> > > > if (platform_bus_type.iommu_ops != ops) {
> > > >         err = bus_set_iommu(&platform_bus_type, ops);
> > > >         if (err)
> > > > }
> > > >
> > > > so for the 1st smmu device on the platform_bus, the
> > > > arm_smmu_device_probe(...) will set the platform_bus_type.iommu_bus =
> > > > arm_smmu_ops, so for the arm_smmu_device_probe(...) when probing the
> > > > 2nd smmu device, the bus_set_iommu() will have no chance be called
> > > > since the platform_bus_type.iommu_bus == arm_smmu_ops is true, my
> > > > question is how the followed bus_set_iommu(...) being called?
> > >
> > > arm_smmu_ops is the same for all SMMU devices, so bus_set_iommu() only
> > > needs to be called once.
> > >
> > Actually, my question is a if pci device driver probes its device
> > before the SMMUv3 driver probing,
> > how does that pci device using the DMA with SMMU translation?
>
> It doesn't - by looking at the dependencies described in device-tree
(forget to reply all, so add the CC'ing...)
I understand 'it doesn't' here means the pci driver will not probe its
device before SMMU driver is on
> (iommu-map property) or in ACPI IORT, we defer probe of the PCI device
> until the SMMU is operational.
>
If that is correct, Jean, would you point me to the code snippet that
a pci driver will defer
its probe until the SMMU is functional? Some key points are enough...

> Thanks,
> Jean

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Question: multiple SMMUv3 devices probe

[Zenghui Yu]

On 2021/4/14 9:45, richard clark wrote:
> Jean-Philippe Brucker <jean-philippe@linaro.org> 于2021年4月13日周二 下午8:58写道：
>>
>> On Tue, Apr 13, 2021 at 08:40:47PM +0800, richard clark wrote:

[...]

>>> Actually, my question is a if pci device driver probes its device
>>> before the SMMUv3 driver probing,
>>> how does that pci device using the DMA with SMMU translation?
>>
>> It doesn't - by looking at the dependencies described in device-tree
> (forget to reply all, so add the CC'ing...)
> I understand 'it doesn't' here means the pci driver will not probe its
> device before SMMU driver is on
>> (iommu-map property) or in ACPI IORT, we defer probe of the PCI device
>> until the SMMU is operational.
>>
> If that is correct, Jean, would you point me to the code snippet that
> a pci driver will defer
> its probe until the SMMU is functional? Some key points are enough...

I think it's correct. Take ACPI-based system as an example, we setup PCI
device's DMA configuration by pci_dma_configure()/acpi_dma_configure().
If as you said, the PCI device is probed before the SMMUv3 probing, we
will get *-EPROBE_DEFER* during the DMA configuration. Please have a
look at iort_pci_iommu_init()/iort_iommu_xlate(), which should have
provided the whole details for us.

The device we *are* probing is then put onto the "deferred list" (grep
it in the driver core code) and will be retried from the list by
deferred_probe_work.

Hope it helps.


Zenghui

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Question: multiple SMMUv3 devices probe

[richard clark]

Zenghui Yu <yuzenghui@huawei.com> 于2021年4月14日周三 上午10:40写道：
>
> On 2021/4/14 9:45, richard clark wrote:
> > Jean-Philippe Brucker <jean-philippe@linaro.org> 于2021年4月13日周二 下午8:58写道：
> >>
> >> On Tue, Apr 13, 2021 at 08:40:47PM +0800, richard clark wrote:
>
> [...]
>
> >>> Actually, my question is a if pci device driver probes its device
> >>> before the SMMUv3 driver probing,
> >>> how does that pci device using the DMA with SMMU translation?
> >>
> >> It doesn't - by looking at the dependencies described in device-tree
> > (forget to reply all, so add the CC'ing...)
> > I understand 'it doesn't' here means the pci driver will not probe its
> > device before SMMU driver is on
> >> (iommu-map property) or in ACPI IORT, we defer probe of the PCI device
> >> until the SMMU is operational.
> >>
> > If that is correct, Jean, would you point me to the code snippet that
> > a pci driver will defer
> > its probe until the SMMU is functional? Some key points are enough...
>
> I think it's correct. Take ACPI-based system as an example, we setup PCI
> device's DMA configuration by pci_dma_configure()/acpi_dma_configure().
> If as you said, the PCI device is probed before the SMMUv3 probing, we
> will get *-EPROBE_DEFER* during the DMA configuration. Please have a
> look at iort_pci_iommu_init()/iort_iommu_xlate(), which should have
> provided the whole details for us.
>
> The device we *are* probing is then put onto the "deferred list" (grep
> it in the driver core code) and will be retried from the list by
> deferred_probe_work.
>
> Hope it helps.
>
Ah, very clear workflow to clarify my confusion. It really does help,
thanks guy, good luck!
>
> Zenghui

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel