Re: [PATCH 2/2] arm64: dts: juno: Enable more SMMUs

From: Anders Roxell <anders.roxell@linaro.org>
To: Robin Murphy <robin.murphy@arm.com>
Cc: sudeep.holla@arm.com, lorenzo.pieralisi@arm.com,
	liviu.dudau@arm.com, linux-arm-kernel@lists.infradead.org,
	stable@vger.kernel.org, linux-kernel@vger.kernel.org,
	gregkh@linuxfoundation.org, sashal@kernel.org, arnd@arndb.de,
	lkft-triage@lists.linaro.org,
	Linux Kernel Functional Testing <lkft@linaro.org>
Subject: Re: [PATCH 2/2] arm64: dts: juno: Enable more SMMUs
Date: Fri, 30 Jul 2021 16:34:31 +0200	[thread overview]
Message-ID: <20210730143431.GB1517404@mutt> (raw)
In-Reply-To: <0a1d437d-9ea0-de83-3c19-e07f560ad37c@arm.com>

On 2021-07-30 13:17, Robin Murphy wrote:
> On 2021-07-30 12:35, Anders Roxell wrote:
> > From: Robin Murphy <robin.murphy@arm.com>
> > 
> > > Now that PCI inbound window restrictions are handled generically between
> > > the of_pci resource parsing and the IOMMU layer, and described in the
> > > Juno DT, we can finally enable the PCIe SMMU without the risk of DMA
> > > mappings inadvertently allocating unusable addresses.
> > > 
> > > Similarly, the relevant support for IOMMU mappings for peripheral
> > > transfers has been hooked up in the pl330 driver for ages, so we can
> > > happily enable the DMA SMMU without that breaking anything either.
> > > 
> > > Signed-off-by: Robin Murphy <robin.murphy@arm.com>
> > 
> > When we build a kernel with 64k page size and run the ltp syscalls we
> > sporadically see a kernel crash while doing a mkfs on a connected SATA
> > drive.  This is happening every third test run on any juno-r2 device in
> > the lab with the same kernel image (stable-rc 5.13.y, mainline and next)
> > with gcc-11.
> 
> Hmm, I guess 64K pages might make a difference in that we'll chew through
> IOVA space a lot faster with small mappings...
> 
> I'll have to try to reproduce this locally, since the interesting thing
> would be knowing what DMA address it was trying to use that went wrong, but
> IOMMU tracepoints and/or dma-debug are going to generate an crazy amount of
> data to sift through and try to correlate - having done it before it's not
> something I'd readily ask someone else to do for me :)
> 
> On a hunch, though, does it make any difference if you remove the first
> entry from the PCIe "dma-ranges" (the 0x2c1c0000 one)?

I did this change, and run the job 7 times and could not reproduce the
issue.

diff --git a/arch/arm64/boot/dts/arm/juno-base.dtsi b/arch/arm64/boot/dts/arm/juno-base.dtsi
index 8e7a66943b01..d3148730e951 100644
--- a/arch/arm64/boot/dts/arm/juno-base.dtsi
+++ b/arch/arm64/boot/dts/arm/juno-base.dtsi
@@ -545,8 +545,7 @@ pcie_ctlr: pcie@40000000 {
                         <0x02000000 0x00 0x50000000 0x00 0x50000000 0x0 0x08000000>,
                         <0x42000000 0x40 0x00000000 0x40 0x00000000 0x1 0x00000000>;
                /* Standard AXI Translation entries as programmed by EDK2 */
-               dma-ranges = <0x02000000 0x0 0x2c1c0000 0x0 0x2c1c0000 0x0 0x00040000>,
-                            <0x02000000 0x0 0x80000000 0x0 0x80000000 0x0 0x80000000>,
+               dma-ranges = <0x02000000 0x0 0x80000000 0x0 0x80000000 0x0 0x80000000>,
                             <0x43000000 0x8 0x00000000 0x8 0x00000000 0x2 0x00000000>;
                #interrupt-cells = <1>;
                interrupt-map-mask = <0 0 0 7>;


Cheers,
Anders
