Re: [PATCH 2/3] mmc: sdhci-of-esdhc: set DMA snooping based on DMA coherence

[Russell King - ARM Linux admin <linux@armlinux.org.uk>]

On Mon, Jan 20, 2020 at 10:09:00AM +0000, Y.b. Lu wrote:
> Hi Russell,
> 
> Recently I got eSDHC ADMA issue on PowerPC T2080 platform. After checking, the issue is related to this patch.
> This patch was to make eSDHC DMA SNOOP bit set per dma-coherent. That resolved issue on LX2160A ARM64 platform.
> However on T2080, we are facing similar issue again. It didn't have dma-coherent in dts.
> Adding dma-coherent in dts, or reverting the patch could resolve the problem.
> 
> Would you please help to have a look at it too? Thanks:)

All aspects of this was discussed extensively by many parties, and as
far as I remember, no conclusion was reached - the discussion became
rather unproductive, so I walked away from it.

I've more or less washed my hands of this in disgust that no way
forward can be found, not even reverting the patch (and then I'll
just carry the patch locally, so at least my machines work - shame
for other ARM64 folk trying to use the LX2160A.)

> 
> mmc0: ADMA error: 0x02000000
> mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
> mmc0: sdhci: Sys addr:  0x00000000 | Version:  0x00002002
> mmc0: sdhci: Blk size:  0x00000008 | Blk cnt:  0x00000001
> mmc0: sdhci: Argument:  0x00000000 | Trn mode: 0x00000013
> NET: Registered protocol family 10
> mmc0: sdhci: Present:   0x01fd020a | Host ctl: 0x00000038
> mmc0: sdhci: Power:     0x00000003 | Blk gap:  0x00000000
> mmc0: sdhci: Wake-up:   0x00000000 | Clock:    0x000020b8
> mmc0: sdhci: Timeout:   0x00000003 | Int stat: 0x00000001
> mmc0: sdhci: Int enab:  0x037f100f | Sig enab: 0x037f100b
> mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00002002
> mmc0: sdhci: Caps:      0x34fa0000 | Caps_1:   0x0000af00
> mmc0: sdhci: Cmd:       0x0000333a | Max curr: 0x00000000
> mmc0: sdhci: Resp[0]:   0x00000920 | Resp[1]:  0x003b3733
> mmc0: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x3f400e00
> mmc0: sdhci: Host ctl2: 0x00000000
> mmc0: sdhci: ADMA Err:  0x0000000d | ADMA Ptr: 0x00000000f35ad20c
> mmc0: sdhci: ============================================
> mmc0: sdhci: f35ad200: DMA 0x00000000f3587710, LEN 0x0008, Attr=0x23
> mmc0: error -5 whilst initialising SD card
> 
> Best regards,
> Yangbo Lu
> 
> 
> > -----Original Message-----
> > From: Russell King <rmk@armlinux.org.uk> On Behalf Of Russell King
> > Sent: Sunday, September 22, 2019 6:27 PM
> > To: Robin Murphy <robin.murphy@arm.com>; dann frazier
> > <dann.frazier@canonical.com>; Will Deacon <will.deacon@arm.com>; Nicolin
> > Chen <nicoleotsuka@gmail.com>; Y.b. Lu <yangbo.lu@nxp.com>; Christoph
> > Hellwig <hch@lst.de>
> > Cc: Adrian Hunter <adrian.hunter@intel.com>; Ulf Hansson
> > <ulf.hansson@linaro.org>; linux-mmc@vger.kernel.org
> > Subject: [PATCH 2/3] mmc: sdhci-of-esdhc: set DMA snooping based on DMA
> > coherence
> > 
> > We must not unconditionally set the DMA snoop bit; if the DMA API is
> > assuming that the device is not DMA coherent, and the device snoops the
> > CPU caches, the device can see stale cache lines brought in by
> > speculative prefetch.
> > 
> > This leads to the device seeing stale data, potentially resulting in
> > corrupted data transfers.  Commonly, this results in a descriptor fetch
> > error such as:
> > 
> > mmc0: ADMA error
> > mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
> > mmc0: sdhci: Sys addr:  0x00000000 | Version:  0x00002202
> > mmc0: sdhci: Blk size:  0x00000008 | Blk cnt:  0x00000001
> > mmc0: sdhci: Argument:  0x00000000 | Trn mode: 0x00000013
> > mmc0: sdhci: Present:   0x01f50008 | Host ctl: 0x00000038
> > mmc0: sdhci: Power:     0x00000003 | Blk gap:  0x00000000
> > mmc0: sdhci: Wake-up:   0x00000000 | Clock:    0x000040d8
> > mmc0: sdhci: Timeout:   0x00000003 | Int stat: 0x00000001
> > mmc0: sdhci: Int enab:  0x037f108f | Sig enab: 0x037f108b
> > mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00002202
> > mmc0: sdhci: Caps:      0x35fa0000 | Caps_1:   0x0000af00
> > mmc0: sdhci: Cmd:       0x0000333a | Max curr: 0x00000000
> > mmc0: sdhci: Resp[0]:   0x00000920 | Resp[1]:  0x001d8a33
> > mmc0: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x3f400e00
> > mmc0: sdhci: Host ctl2: 0x00000000
> > mmc0: sdhci: ADMA Err:  0x00000009 | ADMA Ptr: 0x000000236d43820c
> > mmc0: sdhci: ============================================
> > mmc0: error -5 whilst initialising SD card
> > 
> > but can lead to other errors, and potentially direct the SDHCI
> > controller to read/write data to other memory locations (e.g. if a valid
> > descriptor is visible to the device in a stale cache line.)
> > 
> > Fix this by ensuring that the DMA snoop bit corresponds with the
> > behaviour of the DMA API.  Since the driver currently only supports DT,
> > use of_dma_is_coherent().  Note that device_get_dma_attr() can not be
> > used as that risks re-introducing this bug if/when the driver is
> > converted to ACPI.
> > 
> > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> > ---
> >  drivers/mmc/host/sdhci-of-esdhc.c | 7 ++++++-
> >  1 file changed, 6 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/mmc/host/sdhci-of-esdhc.c
> > b/drivers/mmc/host/sdhci-of-esdhc.c
> > index 4dd43b1adf2c..74de5e8c45c8 100644
> > --- a/drivers/mmc/host/sdhci-of-esdhc.c
> > +++ b/drivers/mmc/host/sdhci-of-esdhc.c
> > @@ -495,7 +495,12 @@ static int esdhc_of_enable_dma(struct sdhci_host
> > *host)
> >  		dma_set_mask_and_coherent(dev, DMA_BIT_MASK(40));
> > 
> >  	value = sdhci_readl(host, ESDHC_DMA_SYSCTL);
> > -	value |= ESDHC_DMA_SNOOP;
> > +
> > +	if (of_dma_is_coherent(dev->of_node))
> > +		value |= ESDHC_DMA_SNOOP;
> > +	else
> > +		value &= ~ESDHC_DMA_SNOOP;
> > +
> >  	sdhci_writel(host, value, ESDHC_DMA_SYSCTL);
> >  	return 0;
> >  }
> > --
> > 2.7.4
> 

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up