* Re: [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
2020-04-21 15:57 [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver Jean-Philippe Brucker
@ 2020-04-21 17:25 ` Robin Murphy
2020-04-28 18:10 ` Tuan Phan
2020-05-20 12:24 ` Will Deacon
2 siblings, 0 replies; 8+ messages in thread
From: Robin Murphy @ 2020-04-21 17:25 UTC (permalink / raw)
To: Jean-Philippe Brucker, will, mark.rutland
Cc: iommu, lorenzo.pieralisi, linux-arm-kernel
On 2020-04-21 4:57 pm, Jean-Philippe Brucker wrote:
> Some Arm SMMUv3 implementations, for example Arm CoreLink MMU-600, embed
> the PMCG registers into the SMMU MMIO regions. It currently causes probe
> failure because the PMU and SMMU drivers request overlapping resources.
>
> Avoid the conflict by calling devm_ioremap() directly from the PMU
> driver. We loose some sanity-checking of the memory map provided by
> firmware, which doesn't seem catastrophic.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>
> So this is the simplest solution, and I don't think we're missing much
> by skipping the resource reservation. I've also been exploring a more
> complex approach [1] which has the SMMU driver perform resource
> reservation on behalf of the PMU driver, but I'm not sure it's
> necessary.
Now try it for potential future PMCGs on DTI masters in any old device
or root complex ;)
If we did want to go down the resource-claiming route, rather than new
inter-driver APIs it would probably be more sensible to just resolve the
associated device and rifle through its resource list directly within
the PMCG driver. Of course that probably leads to a whole bunch of probe
ordering and dependency issues, and if the end result is just to make
/proc/iomem look slightly nicer then I'd agree it's not worth the bother.
> Please test, I've only tried the RevC FastModel using devicetree so far.
For ACPI there's the additional fun that all the resources may already
have been claimed at least once more, by companion devices, but I guess
SMMU and PMCG at least escape that by virtue of not being namespace objects.
Robin.
> [1] https://jpbrucker.net/git/linux/log/?h=smmu/pmu
> ---
> drivers/perf/arm_smmuv3_pmu.c | 28 +++++++++++++++++++++-------
> 1 file changed, 21 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/perf/arm_smmuv3_pmu.c b/drivers/perf/arm_smmuv3_pmu.c
> index ca183a53a7f10..ad63d1e73333f 100644
> --- a/drivers/perf/arm_smmuv3_pmu.c
> +++ b/drivers/perf/arm_smmuv3_pmu.c
> @@ -730,8 +730,8 @@ static void smmu_pmu_get_acpi_options(struct smmu_pmu *smmu_pmu)
>
> static int smmu_pmu_probe(struct platform_device *pdev)
> {
> + struct resource *res_0, *res_1;
> struct smmu_pmu *smmu_pmu;
> - struct resource *res_0;
> u32 cfgr, reg_size;
> u64 ceid_64[2];
> int irq, err;
> @@ -759,18 +759,32 @@ static int smmu_pmu_probe(struct platform_device *pdev)
> .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
> };
>
> + /*
> + * If the PMCG registers are embedded into the SMMU regions, the
> + * resources have to be shared with the SMMU driver. Use ioremap()
> + * rather than ioremap_resource() to avoid conflicts.
> + */
> res_0 = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> - smmu_pmu->reg_base = devm_ioremap_resource(dev, res_0);
> - if (IS_ERR(smmu_pmu->reg_base))
> - return PTR_ERR(smmu_pmu->reg_base);
> + if (!res_0)
> + return -ENXIO;
> +
> + smmu_pmu->reg_base = devm_ioremap(dev, res_0->start,
> + resource_size(res_0));
> + if (!smmu_pmu->reg_base)
> + return -ENOMEM;
>
> cfgr = readl_relaxed(smmu_pmu->reg_base + SMMU_PMCG_CFGR);
>
> /* Determine if page 1 is present */
> if (cfgr & SMMU_PMCG_CFGR_RELOC_CTRS) {
> - smmu_pmu->reloc_base = devm_platform_ioremap_resource(pdev, 1);
> - if (IS_ERR(smmu_pmu->reloc_base))
> - return PTR_ERR(smmu_pmu->reloc_base);
> + res_1 = platform_get_resource(pdev, IORESOURCE_MEM, 1);
> + if (!res_1)
> + return -ENXIO;
> +
> + smmu_pmu->reloc_base = devm_ioremap(dev, res_1->start,
> + resource_size(res_1));
> + if (!smmu_pmu->reloc_base)
> + return -ENOMEM;
> } else {
> smmu_pmu->reloc_base = smmu_pmu->reg_base;
> }
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
2020-04-21 15:57 [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver Jean-Philippe Brucker
2020-04-21 17:25 ` Robin Murphy
@ 2020-04-28 18:10 ` Tuan Phan
2020-04-29 7:21 ` Jean-Philippe Brucker
2020-05-20 12:24 ` Will Deacon
2 siblings, 1 reply; 8+ messages in thread
From: Tuan Phan @ 2020-04-28 18:10 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Mark Rutland, lorenzo.pieralisi, Will Deacon, iommu,
robin.murphy, linux-arm-kernel
> On Apr 21, 2020, at 8:57 AM, Jean-Philippe Brucker <jean-philippe@linaro.org> wrote:
>
> Some Arm SMMUv3 implementations, for example Arm CoreLink MMU-600, embed
> the PMCG registers into the SMMU MMIO regions. It currently causes probe
> failure because the PMU and SMMU drivers request overlapping resources.
>
> Avoid the conflict by calling devm_ioremap() directly from the PMU
> driver. We loose some sanity-checking of the memory map provided by
> firmware, which doesn't seem catastrophic.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>
> So this is the simplest solution, and I don't think we're missing much
> by skipping the resource reservation. I've also been exploring a more
> complex approach [1] which has the SMMU driver perform resource
> reservation on behalf of the PMU driver, but I'm not sure it's
> necessary.
>
> Please test, I've only tried the RevC FastModel using devicetree so far.
>
> [1] https://jpbrucker.net/git/linux/log/?h=smmu/pmu
> ---
> drivers/perf/arm_smmuv3_pmu.c | 28 +++++++++++++++++++++-------
> 1 file changed, 21 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/perf/arm_smmuv3_pmu.c b/drivers/perf/arm_smmuv3_pmu.c
> index ca183a53a7f10..ad63d1e73333f 100644
> --- a/drivers/perf/arm_smmuv3_pmu.c
> +++ b/drivers/perf/arm_smmuv3_pmu.c
> @@ -730,8 +730,8 @@ static void smmu_pmu_get_acpi_options(struct smmu_pmu *smmu_pmu)
>
> static int smmu_pmu_probe(struct platform_device *pdev)
> {
> + struct resource *res_0, *res_1;
> struct smmu_pmu *smmu_pmu;
> - struct resource *res_0;
> u32 cfgr, reg_size;
> u64 ceid_64[2];
> int irq, err;
> @@ -759,18 +759,32 @@ static int smmu_pmu_probe(struct platform_device *pdev)
> .capabilities = PERF_PMU_CAP_NO_EXCLUDE,
> };
>
> + /*
> + * If the PMCG registers are embedded into the SMMU regions, the
> + * resources have to be shared with the SMMU driver. Use ioremap()
> + * rather than ioremap_resource() to avoid conflicts.
> + */
> res_0 = platform_get_resource(pdev, IORESOURCE_MEM, 0);
> - smmu_pmu->reg_base = devm_ioremap_resource(dev, res_0);
> - if (IS_ERR(smmu_pmu->reg_base))
> - return PTR_ERR(smmu_pmu->reg_base);
> + if (!res_0)
> + return -ENXIO;
> +
> + smmu_pmu->reg_base = devm_ioremap(dev, res_0->start,
> + resource_size(res_0));
> + if (!smmu_pmu->reg_base)
> + return -ENOMEM;
>
> cfgr = readl_relaxed(smmu_pmu->reg_base + SMMU_PMCG_CFGR);
>
> /* Determine if page 1 is present */
> if (cfgr & SMMU_PMCG_CFGR_RELOC_CTRS) {
> - smmu_pmu->reloc_base = devm_platform_ioremap_resource(pdev, 1);
> - if (IS_ERR(smmu_pmu->reloc_base))
> - return PTR_ERR(smmu_pmu->reloc_base);
> + res_1 = platform_get_resource(pdev, IORESOURCE_MEM, 1);
> + if (!res_1)
> + return -ENXIO;
> +
> + smmu_pmu->reloc_base = devm_ioremap(dev, res_1->start,
> + resource_size(res_1));
> + if (!smmu_pmu->reloc_base)
> + return -ENOMEM;
I tested this patch on HW, however I need to add one more following change to make it works
@@ -2854,7 +2854,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
}
ioaddr = res->start;
- smmu->base = devm_ioremap_resource(dev, res);
+ smmu->base = devm_ioremap(dev, res->start, resource_size(res));
if (IS_ERR(smmu->base))
return PTR_ERR(smmu->base);
> } else {
> smmu_pmu->reloc_base = smmu_pmu->reg_base;
> }
> --
> 2.26.0
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
2020-04-28 18:10 ` Tuan Phan
@ 2020-04-29 7:21 ` Jean-Philippe Brucker
2020-04-29 18:01 ` Tuan Phan
0 siblings, 1 reply; 8+ messages in thread
From: Jean-Philippe Brucker @ 2020-04-29 7:21 UTC (permalink / raw)
To: Tuan Phan
Cc: Mark Rutland, lorenzo.pieralisi, Will Deacon, iommu,
robin.murphy, linux-arm-kernel
On Tue, Apr 28, 2020 at 11:10:09AM -0700, Tuan Phan wrote:
> I tested this patch on HW, however I need to add one more following change to make it works
Thanks for testing. I don't understand why you need the change below
though, do you know which other region is conflicting with the SMMU?
It should be displayed in the error message and /proc/iomem.
Thanks,
Jean
> @@ -2854,7 +2854,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
> }
> ioaddr = res->start;
>
> - smmu->base = devm_ioremap_resource(dev, res);
> + smmu->base = devm_ioremap(dev, res->start, resource_size(res));
> if (IS_ERR(smmu->base))
> return PTR_ERR(smmu->base);
>
>
> > } else {
> > smmu_pmu->reloc_base = smmu_pmu->reg_base;
> > }
> > --
> > 2.26.0
> >
> >
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> >
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
2020-04-29 7:21 ` Jean-Philippe Brucker
@ 2020-04-29 18:01 ` Tuan Phan
2020-05-06 16:11 ` Jean-Philippe Brucker
0 siblings, 1 reply; 8+ messages in thread
From: Tuan Phan @ 2020-04-29 18:01 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Mark Rutland, lorenzo.pieralisi, Will Deacon, iommu,
robin.murphy, linux-arm-kernel
> On Apr 29, 2020, at 12:21 AM, Jean-Philippe Brucker <jean-philippe@linaro.org> wrote:
>
> On Tue, Apr 28, 2020 at 11:10:09AM -0700, Tuan Phan wrote:
>> I tested this patch on HW, however I need to add one more following change to make it works
>
> Thanks for testing. I don't understand why you need the change below
> though, do you know which other region is conflicting with the SMMU?
> It should be displayed in the error message and /proc/iomem.
>
> Thanks,
> Jean
The error if I don’t apply that patch:
[ 4.943655] arm-smmu-v3 arm-smmu-v3.0.auto: can't request region for resource [mem 0x3bffe0000000-0x3bffe001ffff]
The output of /proc/iomem for that region:
3bffe0000000-3bffe001ffff : arm-smmu-v3.0.auto
3bffe0002000-3bffe0002fff : arm-smmu-v3-pmcg.17.auto
3bffe0012000-3bffe0012fff : arm-smmu-v3-pmcg.17.auto
3bffe0042000-3bffe0042fff : arm-smmu-v3-pmcg.11.auto
3bffe0052000-3bffe0052fff : arm-smmu-v3-pmcg.11.auto
3bffe0062000-3bffe0062fff : arm-smmu-v3-pmcg.12.auto
3bffe0072000-3bffe0072fff : arm-smmu-v3-pmcg.12.auto
3bffe00a2000-3bffe00a2fff : arm-smmu-v3-pmcg.13.auto
3bffe00b2000-3bffe00b2fff : arm-smmu-v3-pmcg.13.auto
3bffe00e2000-3bffe00e2fff : arm-smmu-v3-pmcg.14.auto
3bffe00f2000-3bffe00f2fff : arm-smmu-v3-pmcg.14.auto
3bffe0102000-3bffe0102fff : arm-smmu-v3-pmcg.15.auto
3bffe0112000-3bffe0112fff : arm-smmu-v3-pmcg.15.auto
3bffe0142000-3bffe0142fff : arm-smmu-v3-pmcg.16.auto
3bffe0152000-3bffe0152fff : arm-smmu-v3-pmcg.16.auto
>
>> @@ -2854,7 +2854,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
>> }
>> ioaddr = res->start;
>>
>> - smmu->base = devm_ioremap_resource(dev, res);
>> + smmu->base = devm_ioremap(dev, res->start, resource_size(res));
>> if (IS_ERR(smmu->base))
>> return PTR_ERR(smmu->base);
>>
>>
>>> } else {
>>> smmu_pmu->reloc_base = smmu_pmu->reg_base;
>>> }
>>> --
>>> 2.26.0
>>>
>>>
>>> _______________________________________________
>>> linux-arm-kernel mailing list
>>> linux-arm-kernel@lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>>
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
2020-04-29 18:01 ` Tuan Phan
@ 2020-05-06 16:11 ` Jean-Philippe Brucker
0 siblings, 0 replies; 8+ messages in thread
From: Jean-Philippe Brucker @ 2020-05-06 16:11 UTC (permalink / raw)
To: Tuan Phan
Cc: Mark Rutland, lorenzo.pieralisi, Will Deacon, iommu,
robin.murphy, linux-arm-kernel
On Wed, Apr 29, 2020 at 11:01:05AM -0700, Tuan Phan wrote:
>
>
> > On Apr 29, 2020, at 12:21 AM, Jean-Philippe Brucker <jean-philippe@linaro.org> wrote:
> >
> > On Tue, Apr 28, 2020 at 11:10:09AM -0700, Tuan Phan wrote:
> >> I tested this patch on HW, however I need to add one more following change to make it works
> >
> > Thanks for testing. I don't understand why you need the change below
> > though, do you know which other region is conflicting with the SMMU?
> > It should be displayed in the error message and /proc/iomem.
> >
> > Thanks,
> > Jean
>
> The error if I don’t apply that patch:
> [ 4.943655] arm-smmu-v3 arm-smmu-v3.0.auto: can't request region for resource [mem 0x3bffe0000000-0x3bffe001ffff]
>
> The output of /proc/iomem for that region:
> 3bffe0000000-3bffe001ffff : arm-smmu-v3.0.auto
> 3bffe0002000-3bffe0002fff : arm-smmu-v3-pmcg.17.auto
> 3bffe0012000-3bffe0012fff : arm-smmu-v3-pmcg.17.auto
Thanks for this. It looks like the regions are added to the resource tree
twice, when booting with ACPI:
When the IORT creates the platform devices, platform_device_add() inserts
these resources but doesn't mark them busy. As I was testing with
devicetree I was missing this step.
Then the SMMU probe calls devm_ioremap_resource(), which tries to reserve
the resources and mark them busy this time. If there only was the non-busy
SMMU region inserted above, then __request_region() would adds the
reserved region as a child. However since there are smaller PMCG regions
as children of the SMMU region, __request_region() fails.
Another idea is to avoid mapping the IMPDEF registers in the SMMU driver.
I think it's nicer, and I'll post that patch shortly.
Thanks,
Jean
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
2020-04-21 15:57 [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver Jean-Philippe Brucker
2020-04-21 17:25 ` Robin Murphy
2020-04-28 18:10 ` Tuan Phan
@ 2020-05-20 12:24 ` Will Deacon
2020-05-20 12:44 ` Jean-Philippe Brucker
2 siblings, 1 reply; 8+ messages in thread
From: Will Deacon @ 2020-05-20 12:24 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: mark.rutland, iommu, lorenzo.pieralisi, robin.murphy, linux-arm-kernel
On Tue, Apr 21, 2020 at 05:57:46PM +0200, Jean-Philippe Brucker wrote:
> Some Arm SMMUv3 implementations, for example Arm CoreLink MMU-600, embed
> the PMCG registers into the SMMU MMIO regions. It currently causes probe
> failure because the PMU and SMMU drivers request overlapping resources.
>
> Avoid the conflict by calling devm_ioremap() directly from the PMU
> driver. We loose some sanity-checking of the memory map provided by
> firmware, which doesn't seem catastrophic.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>
> So this is the simplest solution, and I don't think we're missing much
> by skipping the resource reservation. I've also been exploring a more
> complex approach [1] which has the SMMU driver perform resource
> reservation on behalf of the PMU driver, but I'm not sure it's
> necessary.
>
> Please test, I've only tried the RevC FastModel using devicetree so far.
>
> [1] https://jpbrucker.net/git/linux/log/?h=smmu/pmu
> ---
> drivers/perf/arm_smmuv3_pmu.c | 28 +++++++++++++++++++++-------
> 1 file changed, 21 insertions(+), 7 deletions(-)
Is this patch still needed? I can't quite follow from the discussion.
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] perf/smmuv3: Allow sharing MMIO registers with the SMMU driver
2020-05-20 12:24 ` Will Deacon
@ 2020-05-20 12:44 ` Jean-Philippe Brucker
0 siblings, 0 replies; 8+ messages in thread
From: Jean-Philippe Brucker @ 2020-05-20 12:44 UTC (permalink / raw)
To: Will Deacon
Cc: mark.rutland, iommu, lorenzo.pieralisi, robin.murphy, linux-arm-kernel
On Wed, May 20, 2020 at 01:24:53PM +0100, Will Deacon wrote:
> On Tue, Apr 21, 2020 at 05:57:46PM +0200, Jean-Philippe Brucker wrote:
> > Some Arm SMMUv3 implementations, for example Arm CoreLink MMU-600, embed
> > the PMCG registers into the SMMU MMIO regions. It currently causes probe
> > failure because the PMU and SMMU drivers request overlapping resources.
> >
> > Avoid the conflict by calling devm_ioremap() directly from the PMU
> > driver. We loose some sanity-checking of the memory map provided by
> > firmware, which doesn't seem catastrophic.
> >
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> >
> > So this is the simplest solution, and I don't think we're missing much
> > by skipping the resource reservation. I've also been exploring a more
> > complex approach [1] which has the SMMU driver perform resource
> > reservation on behalf of the PMU driver, but I'm not sure it's
> > necessary.
> >
> > Please test, I've only tried the RevC FastModel using devicetree so far.
> >
> > [1] https://jpbrucker.net/git/linux/log/?h=smmu/pmu
> > ---
> > drivers/perf/arm_smmuv3_pmu.c | 28 +++++++++++++++++++++-------
> > 1 file changed, 21 insertions(+), 7 deletions(-)
>
> Is this patch still needed? I can't quite follow from the discussion.
No, it is superseded by "[PATCH v2] iommu/arm-smmu-v3: Don't reserve
implementation defined register space" that you applied yesterday
(Thanks!)
Jean
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 8+ messages in thread