Re: [RFC 09/13] iommu/arm-smmu: Make use of dev_64bit_mmio_supported()

From: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
To: Robin Murphy <robin.murphy@arm.com>,
	linux-arm-kernel@lists.infradead.org,
	 devicetree@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: f.fainelli@gmail.com, robh+dt@kernel.org, ardb@kernel.org,
	hch@infradead.org,  narmstrong@baylibre.com, dwmw2@infradead.org,
	linux@armlinux.org.uk,  catalin.marinas@arm.com, arnd@arndb.de,
	will@kernel.org
Subject: Re: [RFC 09/13] iommu/arm-smmu: Make use of dev_64bit_mmio_supported()
Date: Tue, 02 Mar 2021 14:38:14 +0100	[thread overview]
Message-ID: <ad6361433104d703338d2005cd8f3714508bccbb.camel@suse.de> (raw)
In-Reply-To: <3a4cf13f-c098-9ff3-6c0e-2c94daae452b@arm.com>

[-- Attachment #1.1: Type: text/plain, Size: 3610 bytes --]

Hi Robin, thanks for taking the time to look at this.

On Tue, 2021-03-02 at 11:07 +0000, Robin Murphy wrote:
> On 2021-02-26 14:03, Nicolas Saenz Julienne wrote:
> > Some arm SMMU implementations might sit on a bus that doesn't support
> > 64bit memory accesses. In that case default to using hi_lo_{readq,
> > writeq}() and BUG if such platform tries to use AArch64 formats as they
> > rely on writeq()'s atomicity.
> > 
> > Signed-off-by: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
> > ---
> >   drivers/iommu/arm/arm-smmu/arm-smmu.c | 9 +++++++++
> >   drivers/iommu/arm/arm-smmu/arm-smmu.h | 9 +++++++--
> >   2 files changed, 16 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c
> > index d8c6bfde6a61..239ff42b20c3 100644
> > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c
> > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c
> > @@ -1889,6 +1889,15 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
> >   			smmu->features |= ARM_SMMU_FEAT_FMT_AARCH64_64K;
> >   	}
> >   
> > 
> > +	/*
> > +	 * 64bit accesses not possible through the interconnect, AArch64
> > +	 * formats depend on it.
> > +	 */
> > +	BUG_ON(!dev_64bit_mmio_supported(smmu->dev) &&
> > +	       smmu->features & (ARM_SMMU_FEAT_FMT_AARCH64_4K |
> > +				ARM_SMMU_FEAT_FMT_AARCH64_16K |
> > +				ARM_SMMU_FEAT_FMT_AARCH64_64K));
> 
> No. Crashing the kernel in a probe routine which is free to fail is 
> unacceptable either way, but guaranteeing failure in the case that the 
> workaround *would* be required is doubly so.
> 
> Basically, this logic is backwards - if you really wanted to handle it 
> generically, this would be the point at which you'd need to actively 
> suppress all the detected hardware features which depend on 64-bit 
> atomicity, not complain about them.

Understood.

> > +
> >   	if (smmu->impl && smmu->impl->cfg_probe) {
> >   		ret = smmu->impl->cfg_probe(smmu);
> >   		if (ret)
> > diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.h b/drivers/iommu/arm/arm-smmu/arm-smmu.h
> > index d2a2d1bc58ba..997d13a21717 100644
> > --- a/drivers/iommu/arm/arm-smmu/arm-smmu.h
> > +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.h
> > @@ -477,15 +477,20 @@ static inline void arm_smmu_writel(struct arm_smmu_device *smmu, int page,
> >   {
> >   	if (smmu->impl && unlikely(smmu->impl->write_reg))
> >   		smmu->impl->write_reg(smmu, page, offset, val);
> > -	else
> > +	else if (dev_64bit_mmio_supported(smmu->dev))
> >   		writel_relaxed(val, arm_smmu_page(smmu, page) + offset);
> > +	else
> > +		hi_lo_writeq_relaxed(val, arm_smmu_page(smmu, page) + offset);
> 
> As Arnd pointed out, this is in completely the wrong place. Also, in 

Yes, sorry for that, not too proud of it.

> general it doesn't work if the implementation already needs a hook to 
> filter or override register accesses for any other reason. TBH I'm not 

I'm not sure I get your point here, 'smmu->impl' has precedence over the MMIO
capability check. Custom implementations would still get their callbacks.

> convinced that this isn't *more* of a mess than handling it on a 
> SoC-specific basis...

I see your point.

Just to explain why I went to these lengths: my understanding is that the
specifics of how to perform 32bit accesses to SMMU's 64bit registers is defined
in spec. So it made sense to move it into the non implementation dependent side
of the driver.

All in all, I'll think of something simpler.

Regards,
Nicolas

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel