IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[Tim Harvey]

Greetings,

I've been seeing mmc failures with Linux 4.10 on some IMX6 boards with
UHS-I support when used with DDR50 cards (I've got a couple from
different manufacturers). I do not see the same issues when using
SDR104 cards. I can also see the issue on the latest linux-mmc repo.

The issue is easily replicated by ejecting and inserting the DDR50 card

A successful init looks like this:
[  150.514018] mmc0: host does not support reading read-only switch,
assuming write-enable
[  150.578204] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
[  150.592140] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
[  150.611512]  mmcblk0: p1

A failing init (about 1 of every 5 trys) looks like this:
[   49.514082] mmc0: host does not support reading read-only switch,
assuming write-enable
[   49.676112] mmc0: tuning execution failed: -5
[   49.680510] mmc0: ddr50 tuning failed
[   49.684206] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
[   49.696456] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
^^^^ failed to read the partition table and at this point all access fails:
# dd if=/dev/mmcblk0 of=/dev/null bs=1k count=1
[   89.417210] mmcblk0: error -84 transferring data, sector 0, nr 32,
cmd response 0x900, card status 0xb00
1+0 records in
1+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.1227 s, 8.3 kB/s

I know the issue started occurring in Linux v4.4 but I also have heard
there are some tuning patches in the queue so before I bisected
between v4.4 and v4.3. Note that v4.3 is missing DDR50 tuning but when
I backport that I found that when ddr tuning failed there it did not
cause the card to not be accessible.

Regards,

Tim

Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[David Haworth]

[-- Attachment #1: Type: text/plain, Size: 2688 bytes --]

Hi Tim,

This looks uncomfortably familiar.
I've been having problems with "some" micro-SDHC cards, but in
different circumstances: mmc_spi on a raspberry pi.

"Good" cards sign on as "mmcblk1: mmc1:0000 00000 7.35 GiB"
"Bad" cards sign on as  "mmcblk1: mmc1:0000 SU08G 7.40 GiB"
and eventually return a bit shifted response.

The "bad" cards are all SanDisk cards. The "good" cards are cheap
cards with one exception that is marked as SanDisk but looks like
a fake.

More details here:
https://www.raspberrypi.org/forums/viewtopic.php?f=44&t=172562&p=1103943#p1103943
if you can read it.
Unfortunately I don't have a solution.

All the best
Dave

On 2017-04-04 14:26:40 -0700, Tim Harvey wrote:
> Greetings,
> 
> I've been seeing mmc failures with Linux 4.10 on some IMX6 boards with
> UHS-I support when used with DDR50 cards (I've got a couple from
> different manufacturers). I do not see the same issues when using
> SDR104 cards. I can also see the issue on the latest linux-mmc repo.
> 
> The issue is easily replicated by ejecting and inserting the DDR50 card
> 
> A successful init looks like this:
> [  150.514018] mmc0: host does not support reading read-only switch,
> assuming write-enable
> [  150.578204] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
> [  150.592140] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> [  150.611512]  mmcblk0: p1
> 
> A failing init (about 1 of every 5 trys) looks like this:
> [   49.514082] mmc0: host does not support reading read-only switch,
> assuming write-enable
> [   49.676112] mmc0: tuning execution failed: -5
> [   49.680510] mmc0: ddr50 tuning failed
> [   49.684206] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
> [   49.696456] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> ^^^^ failed to read the partition table and at this point all access fails:
> # dd if=/dev/mmcblk0 of=/dev/null bs=1k count=1
> [   89.417210] mmcblk0: error -84 transferring data, sector 0, nr 32,
> cmd response 0x900, card status 0xb00
> 1+0 records in
> 1+0 records out
> 1024 bytes (1.0 kB, 1.0 KiB) copied, 0.1227 s, 8.3 kB/s
> 
> I know the issue started occurring in Linux v4.4 but I also have heard
> there are some tuning patches in the queue so before I bisected
> between v4.4 and v4.3. Note that v4.3 is missing DDR50 tuning but when
> I backport that I found that when ddr tuning failed there it did not
> cause the card to not be accessible.
> 
> Regards,
> 
> Tim
> --
> To unsubscribe from this list: send the line "unsubscribe linux-mmc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[Tim Harvey]

On Tue, Apr 4, 2017 at 3:00 PM, David Haworth <dave@fen-net.de> wrote:
> Hi Tim,
>
> This looks uncomfortably familiar.
> I've been having problems with "some" micro-SDHC cards, but in
> different circumstances: mmc_spi on a raspberry pi.
>
> "Good" cards sign on as "mmcblk1: mmc1:0000 00000 7.35 GiB"
> "Bad" cards sign on as  "mmcblk1: mmc1:0000 SU08G 7.40 GiB"
> and eventually return a bit shifted response.
>
> The "bad" cards are all SanDisk cards. The "good" cards are cheap
> cards with one exception that is marked as SanDisk but looks like
> a fake.
>
> More details here:
> https://www.raspberrypi.org/forums/viewtopic.php?f=44&t=172562&p=1103943#p1103943
> if you can read it.
> Unfortunately I don't have a solution.
>
> All the best
> Dave
>

Dave,

Thanks for the response, but I don't think this is the same thing
because in your case I don't see any tuning or transfer errors
reported.

I've moved on to 4.11-rc5 and notice that I'm always seeing that on a
failure the imx sdhc not able to determine a min/max tuning delay:

4.11.0-rc5:
[   48.812381] mmc0: card aaaa removed
[   49.793392] mmc0: host does not support reading read-only switch,
assuming write-enable
[   49.810294] sdhci_execute_tuning
[   49.813612] esdhc_executing_tuning
[   49.968154] sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f
ret -5 (min=127/max=128)
^^^^ added debug shows -EIO returned from mmc_send_tuning although
I've also seen -84 get returned
[   49.980540] mmc0: tuning execution failed: -5
[   49.984974] mmc0: ddr50 tuning failed
[   49.989068] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
[   49.999844] mmcblk0: mmc0:aaaa SL08G 7.40 GiB

I've also found that if I revert '9faac7b mmc: sdhci: enable tuning
for DDR50' thus skipping the tuning the issue goes away however note
that occasionally I do see a transfer error that perhaps is retried
[  361.067763] mmc0: card aaaa removed
[  361.412732] mmc0: host does not support reading read-only switch,
assuming write-enable
[  361.429719] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
[  361.443060] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
[  361.465658]  mmcblk0: p1
[  362.422335] mmc0: card aaaa removed
[  362.752719] mmc0: host does not support reading read-only switch,
assuming write-enable
[  362.769675] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
[  362.782900] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
[  362.805996] mmcblk0: error -84 transferring data, sector 0, nr 8,
cmd response 0x900, card status 0xb00
^^^^ occasionally see this error but because retries occur at the
block level it still works
[  362.931689]  mmcblk0: p1

Again with 9faac7b reverted and some extra debugging:
[   33.823170] mmc0: host does not support reading read-only switch,
assuming write-enable
[   33.847185] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
[   33.856063] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
[   33.874227] mmcblk0: error -84 transferring data, sector 0, nr 8,
cmd response 0x900, card status 0xb00
[   33.896829] mmcblk0: error -84 transferring data, sector 0, nr 8,
cmd response 0x900, card status 0xb00
[   33.906312] mmcblk0: error -84 transferring data, sector 0, nr 8,
cmd response 0x900, card status 0xb00
[   34.022117] mmcblk0: error 0 transferring data, sector 0, nr 8, cmd
response 0x900, card status 0xb00
^^^^ added debugging shows retries at block level and eventually all is fine
[   34.031500]  mmcblk0: p1

So far every DDR50 card I have (several Kingston and Sandisk card
models) behaves like this.

I tried adding a settling delay when VSELECT was changed in the
sdhci-esdhc-imx driver and that didn't help.

Tim

RE: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[Bough Chen]

> -----Original Message-----
> From: linux-mmc-owner@vger.kernel.org [mailto:linux-mmc-
> owner@vger.kernel.org] On Behalf Of Tim Harvey
> Sent: Saturday, April 08, 2017 3:55 AM
> To: David Haworth <dave@fen-net.de>
> Cc: Linux MMC List <linux-mmc@vger.kernel.org>; Fabio Estevam
> <fabio.estevam@nxp.com>; Ulf Hansson <ulf.hansson@linaro.org>; Weijun
> Yang <york.yang@csr.com>
> Subject: Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards
> 
> On Tue, Apr 4, 2017 at 3:00 PM, David Haworth <dave@fen-net.de> wrote:
> > Hi Tim,
> >
> > This looks uncomfortably familiar.
> > I've been having problems with "some" micro-SDHC cards, but in
> > different circumstances: mmc_spi on a raspberry pi.
> >
> > "Good" cards sign on as "mmcblk1: mmc1:0000 00000 7.35 GiB"
> > "Bad" cards sign on as  "mmcblk1: mmc1:0000 SU08G 7.40 GiB"
> > and eventually return a bit shifted response.
> >
> > The "bad" cards are all SanDisk cards. The "good" cards are cheap
> > cards with one exception that is marked as SanDisk but looks like a
> > fake.
> >
> > More details here:
> >
> https://www.raspberrypi.org/forums/viewtopic.php?f=44&t=172562&p=11039
> > 43#p1103943
> > if you can read it.
> > Unfortunately I don't have a solution.
> >
> > All the best
> > Dave
> >
> 
> Dave,
> 
> Thanks for the response, but I don't think this is the same thing because in your
> case I don't see any tuning or transfer errors reported.
> 
> I've moved on to 4.11-rc5 and notice that I'm always seeing that on a failure the
> imx sdhc not able to determine a min/max tuning delay:
> 
> 4.11.0-rc5:
> [   48.812381] mmc0: card aaaa removed
> [   49.793392] mmc0: host does not support reading read-only switch,
> assuming write-enable
> [   49.810294] sdhci_execute_tuning
> [   49.813612] esdhc_executing_tuning
> [   49.968154] sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f
> ret -5 (min=127/max=128)
> ^^^^ added debug shows -EIO returned from mmc_send_tuning although I've
> also seen -84 get returned
> [   49.980540] mmc0: tuning execution failed: -5
> [   49.984974] mmc0: ddr50 tuning failed
> [   49.989068] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
> [   49.999844] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> 
> I've also found that if I revert '9faac7b mmc: sdhci: enable tuning for DDR50'
> thus skipping the tuning the issue goes away however note that occasionally I
> do see a transfer error that perhaps is retried [  361.067763] mmc0: card aaaa
> removed [  361.412732] mmc0: host does not support reading read-only switch,
> assuming write-enable [  361.429719] mmc0: new ultra high speed DDR50 SDHC
> card at address aaaa [  361.443060] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> [  361.465658]  mmcblk0: p1 [  362.422335] mmc0: card aaaa removed
> [  362.752719] mmc0: host does not support reading read-only switch, assuming
> write-enable [  362.769675] mmc0: new ultra high speed DDR50 SDHC card at
> address aaaa [  362.782900] mmcblk0: mmc0:aaaa SL08G 7.40 GiB [  362.805996]
> mmcblk0: error -84 transferring data, sector 0, nr 8, cmd response 0x900, card
> status 0xb00 ^^^^ occasionally see this error but because retries occur at the
> block level it still works [  362.931689]  mmcblk0: p1
> 
> Again with 9faac7b reverted and some extra debugging:
> [   33.823170] mmc0: host does not support reading read-only switch,
> assuming write-enable
> [   33.847185] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
> [   33.856063] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> [   33.874227] mmcblk0: error -84 transferring data, sector 0, nr 8,
> cmd response 0x900, card status 0xb00
> [   33.896829] mmcblk0: error -84 transferring data, sector 0, nr 8,
> cmd response 0x900, card status 0xb00
> [   33.906312] mmcblk0: error -84 transferring data, sector 0, nr 8,
> cmd response 0x900, card status 0xb00
> [   34.022117] mmcblk0: error 0 transferring data, sector 0, nr 8, cmd
> response 0x900, card status 0xb00
> ^^^^ added debugging shows retries at block level and eventually all is fine
> [   34.031500]  mmcblk0: p1
> 
> So far every DDR50 card I have (several Kingston and Sandisk card
> models) behaves like this.
> 
> I tried adding a settling delay when VSELECT was changed in the sdhci-esdhc-
> imx driver and that didn't help.
> 
Hi Tim

Which imx6 soc do you use?  From your log, I notice you use the MAN tuning method, so I guess you use imx6qdl.

imx usdhc has an IC bug. For the tuning procedure, the first pass tuning delay value must bigger than 10. 
Due to DDR50 timing is not that rigorous, so maybe even when we just add 1 delay cell, tuning can pass, but this trigger the internal IC bug, then you meet the following issue just as you meet:
       sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f        ( min=127/max=128)

So you can change the ESDHC_TUNE_CTRL_MIN from 0 to 15, then you can see the DDR50 tuning return to normal.

For the STD tuning method, you can add the property 'fsl,tuning-start-tap = <20>' in dts.

All this make the first trying tuning delay value larger than 10, to avoid  the internal IMX USDHC IC bug.

SDR104 card do not has this issue, because for SDR104 card, the first pass tuning value normally larger than 10. 

Best Regards,
Haibo

> Tim
> --
> To unsubscribe from this list: send the line "unsubscribe linux-mmc" in the
> body of a message to majordomo@vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html

Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[Tim Harvey]

On Tue, Apr 11, 2017 at 2:28 AM, Bough Chen <haibo.chen@nxp.com> wrote:
>> -----Original Message-----
<snip>
>>
>> I've moved on to 4.11-rc5 and notice that I'm always seeing that on a failure the
>> imx sdhc not able to determine a min/max tuning delay:
>>
>> 4.11.0-rc5:
>> [   48.812381] mmc0: card aaaa removed
>> [   49.793392] mmc0: host does not support reading read-only switch,
>> assuming write-enable
>> [   49.810294] sdhci_execute_tuning
>> [   49.813612] esdhc_executing_tuning
>> [   49.968154] sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f
>> ret -5 (min=127/max=128)
>> ^^^^ added debug shows -EIO returned from mmc_send_tuning although I've
>> also seen -84 get returned
>> [   49.980540] mmc0: tuning execution failed: -5
>> [   49.984974] mmc0: ddr50 tuning failed
>> [   49.989068] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
>> [   49.999844] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
>>
>> I've also found that if I revert '9faac7b mmc: sdhci: enable tuning for DDR50'
>> thus skipping the tuning the issue goes away however note that occasionally I
>> do see a transfer error that perhaps is retried [  361.067763] mmc0: card aaaa
>> removed [  361.412732] mmc0: host does not support reading read-only switch,
>> assuming write-enable [  361.429719] mmc0: new ultra high speed DDR50 SDHC
>> card at address aaaa [  361.443060] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
>> [  361.465658]  mmcblk0: p1 [  362.422335] mmc0: card aaaa removed
>> [  362.752719] mmc0: host does not support reading read-only switch, assuming
>> write-enable [  362.769675] mmc0: new ultra high speed DDR50 SDHC card at
>> address aaaa [  362.782900] mmcblk0: mmc0:aaaa SL08G 7.40 GiB [  362.805996]
>> mmcblk0: error -84 transferring data, sector 0, nr 8, cmd response 0x900, card
>> status 0xb00 ^^^^ occasionally see this error but because retries occur at the
>> block level it still works [  362.931689]  mmcblk0: p1
>>
>> Again with 9faac7b reverted and some extra debugging:
>> [   33.823170] mmc0: host does not support reading read-only switch,
>> assuming write-enable
>> [   33.847185] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
>> [   33.856063] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
>> [   33.874227] mmcblk0: error -84 transferring data, sector 0, nr 8,
>> cmd response 0x900, card status 0xb00
>> [   33.896829] mmcblk0: error -84 transferring data, sector 0, nr 8,
>> cmd response 0x900, card status 0xb00
>> [   33.906312] mmcblk0: error -84 transferring data, sector 0, nr 8,
>> cmd response 0x900, card status 0xb00
>> [   34.022117] mmcblk0: error 0 transferring data, sector 0, nr 8, cmd
>> response 0x900, card status 0xb00
>> ^^^^ added debugging shows retries at block level and eventually all is fine
>> [   34.031500]  mmcblk0: p1
>>
>> So far every DDR50 card I have (several Kingston and Sandisk card
>> models) behaves like this.
>>
>> I tried adding a settling delay when VSELECT was changed in the sdhci-esdhc-
>> imx driver and that didn't help.
>>
> Hi Tim
>
> Which imx6 soc do you use?  From your log, I notice you use the MAN tuning method, so I guess you use imx6qdl.
>
> imx usdhc has an IC bug. For the tuning procedure, the first pass tuning delay value must bigger than 10.
> Due to DDR50 timing is not that rigorous, so maybe even when we just add 1 delay cell, tuning can pass, but this trigger the internal IC bug, then you meet the following issue just as you meet:
>        sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f        ( min=127/max=128)
>

Haibo,

Thanks for the response.

Yes, this issue is seen on the IMX6DQ and IMX6SDL. Can you point me to
information on this bug? I don't see it in the latest IMX6DQCE.pdf
errata document.

> So you can change the ESDHC_TUNE_CTRL_MIN from 0 to 15, then you can see the DDR50 tuning return to normal.
>
> For the STD tuning method, you can add the property 'fsl,tuning-start-tap = <20>' in dts.
>
> All this make the first trying tuning delay value larger than 10, to avoid  the internal IMX USDHC IC bug.
>
> SDR104 card do not has this issue, because for SDR104 card, the first pass tuning value normally larger than 10.
>

unfortunately, this does not resolve the issue. I tried
ESDHC_TUNE_CTRL_MIN of 15, 20, and 30 and they all produce the same
issue around 20% of insertions:
sdhci-esdhc-imx 2198000.usdhc: tunning failed at 0x7f ret -84

Here are some min/max tuning values from some various cards I have:
- Samsung SL08G DDR50: min=8 max=37
- Kingston SD8GB DDR50: min=8 max=36
- Sandisk SS16G DDR50: min=10 max=37
- Sandisk SU08G DDR50: min=9 max=36
- Samsung Pro 00000 SDR104: min=14 max=26
- Kingston SDR104 SDCIT: min=25 max=36

All of the DDR50 cards above fail in the same way appx 20% of insertions.

Has anyone else been able to confirm they see the same issue on IMX6
boards with UHS-I support and DDR50?

Regards,

Tim

Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[David Haworth]

Hi Tim,

On 2017-04-07 12:55:28 -0700, Tim Harvey wrote:
> 
> Thanks for the response, but I don't think this is the same thing
> because in your case I don't see any tuning or transfer errors
> reported.

I think the "shifted response code" might be the result of a transfer
error, but you're right - no tuning.

I see you had another reply though - I hope you manage to find a solution.

Cheers,
Dave

RE: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[Bough Chen]

> -----Original Message-----
> From: Tim Harvey [mailto:tharvey@gateworks.com]
> Sent: Tuesday, April 11, 2017 10:22 PM
> To: Bough Chen <haibo.chen@nxp.com>
> Cc: David Haworth <dave@fen-net.de>; Linux MMC List <linux-
> mmc@vger.kernel.org>; Fabio Estevam <fabio.estevam@nxp.com>; Ulf
> Hansson <ulf.hansson@linaro.org>; Weijun Yang <york.yang@csr.com>; Adrian
> Hunter <adrian.hunter@intel.com>
> Subject: Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards
> 
> On Tue, Apr 11, 2017 at 2:28 AM, Bough Chen <haibo.chen@nxp.com> wrote:
> >> -----Original Message-----
> <snip>
> >>
> >> I've moved on to 4.11-rc5 and notice that I'm always seeing that on a
> >> failure the imx sdhc not able to determine a min/max tuning delay:
> >>
> >> 4.11.0-rc5:
> >> [   48.812381] mmc0: card aaaa removed
> >> [   49.793392] mmc0: host does not support reading read-only switch,
> >> assuming write-enable
> >> [   49.810294] sdhci_execute_tuning
> >> [   49.813612] esdhc_executing_tuning
> >> [   49.968154] sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f
> >> ret -5 (min=127/max=128)
> >> ^^^^ added debug shows -EIO returned from mmc_send_tuning although
> >> I've also seen -84 get returned
> >> [   49.980540] mmc0: tuning execution failed: -5
> >> [   49.984974] mmc0: ddr50 tuning failed
> >> [   49.989068] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
> >> [   49.999844] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> >>
> >> I've also found that if I revert '9faac7b mmc: sdhci: enable tuning for DDR50'
> >> thus skipping the tuning the issue goes away however note that
> >> occasionally I do see a transfer error that perhaps is retried [
> >> 361.067763] mmc0: card aaaa removed [  361.412732] mmc0: host does
> >> not support reading read-only switch, assuming write-enable [
> >> 361.429719] mmc0: new ultra high speed DDR50 SDHC card at address
> >> aaaa [  361.443060] mmcblk0: mmc0:aaaa SL08G 7.40 GiB [  361.465658]
> >> mmcblk0: p1 [  362.422335] mmc0: card aaaa removed [  362.752719]
> >> mmc0: host does not support reading read-only switch, assuming
> >> write-enable [  362.769675] mmc0: new ultra high speed DDR50 SDHC
> >> card at address aaaa [  362.782900] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> >> [  362.805996]
> >> mmcblk0: error -84 transferring data, sector 0, nr 8, cmd response
> >> 0x900, card status 0xb00 ^^^^ occasionally see this error but because
> >> retries occur at the block level it still works [  362.931689]
> >> mmcblk0: p1
> >>
> >> Again with 9faac7b reverted and some extra debugging:
> >> [   33.823170] mmc0: host does not support reading read-only switch,
> >> assuming write-enable
> >> [   33.847185] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
> >> [   33.856063] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
> >> [   33.874227] mmcblk0: error -84 transferring data, sector 0, nr 8,
> >> cmd response 0x900, card status 0xb00
> >> [   33.896829] mmcblk0: error -84 transferring data, sector 0, nr 8,
> >> cmd response 0x900, card status 0xb00
> >> [   33.906312] mmcblk0: error -84 transferring data, sector 0, nr 8,
> >> cmd response 0x900, card status 0xb00
> >> [   34.022117] mmcblk0: error 0 transferring data, sector 0, nr 8, cmd
> >> response 0x900, card status 0xb00
> >> ^^^^ added debugging shows retries at block level and eventually all is fine
> >> [   34.031500]  mmcblk0: p1
> >>
> >> So far every DDR50 card I have (several Kingston and Sandisk card
> >> models) behaves like this.
> >>
> >> I tried adding a settling delay when VSELECT was changed in the
> >> sdhci-esdhc- imx driver and that didn't help.
> >>
> > Hi Tim
> >
> > Which imx6 soc do you use?  From your log, I notice you use the MAN tuning
> method, so I guess you use imx6qdl.
> >
> > imx usdhc has an IC bug. For the tuning procedure, the first pass tuning delay
> value must bigger than 10.
> > Due to DDR50 timing is not that rigorous, so maybe even when we just add 1
> delay cell, tuning can pass, but this trigger the internal IC bug, then you meet
> the following issue just as you meet:
> >        sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f
> ( min=127/max=128)
> >
> 
> Haibo,
> 
> Thanks for the response.
> 
> Yes, this issue is seen on the IMX6DQ and IMX6SDL. Can you point me to
> information on this bug? I don't see it in the latest IMX6DQCE.pdf errata
> document.
> 
> > So you can change the ESDHC_TUNE_CTRL_MIN from 0 to 15, then you can
> see the DDR50 tuning return to normal.
> >
> > For the STD tuning method, you can add the property 'fsl,tuning-start-tap =
> <20>' in dts.
> >
> > All this make the first trying tuning delay value larger than 10, to avoid  the
> internal IMX USDHC IC bug.
> >
> > SDR104 card do not has this issue, because for SDR104 card, the first pass
> tuning value normally larger than 10.
> >
> 
> unfortunately, this does not resolve the issue. I tried ESDHC_TUNE_CTRL_MIN
> of 15, 20, and 30 and they all produce the same issue around 20% of insertions:
> sdhci-esdhc-imx 2198000.usdhc: tunning failed at 0x7f ret -84
> 
> Here are some min/max tuning values from some various cards I have:
> - Samsung SL08G DDR50: min=8 max=37
> - Kingston SD8GB DDR50: min=8 max=36
> - Sandisk SS16G DDR50: min=10 max=37
> - Sandisk SU08G DDR50: min=9 max=36
> - Samsung Pro 00000 SDR104: min=14 max=26
> - Kingston SDR104 SDCIT: min=25 max=36
> 
> All of the DDR50 cards above fail in the same way appx 20% of insertions.
> 
> Has anyone else been able to confirm they see the same issue on IMX6 boards
> with UHS-I support and DDR50?

Hi Tim,

I debug this issue these days, and find this is the I/O timing issue. 

Can you try to improve the pad I/O drive strength for DDR50 card? You can apply the following patch:

diff --git a/drivers/mmc/host/sdhci-esdhc-imx.c b/drivers/mmc/host/sdhci-esdhc-imx.c
index 6a1328e..4c3aeac 100644
--- a/drivers/mmc/host/sdhci-esdhc-imx.c
+++ b/drivers/mmc/host/sdhci-esdhc-imx.c
@@ -848,6 +848,7 @@ static int esdhc_change_pinstate(struct sdhci_host *host,

        switch (uhs) {
        case MMC_TIMING_UHS_SDR50:
+       case MMC_TIMING_UHS_DDR50:
                pinctrl = imx_data->pins_100mhz;
                break;
        case MMC_TIMING_UHS_SDR104:

If this patch also test pass on your side, I will upstream it.

By the way, I just confirm with our IC guys, the IC bug I mentioned in the before mail do not impact the software tuning method (MAN_TUNING), this IC bug just impact the STD_TUNING, sorry for the mislead.

Haibo Chen

> 
> Regards,
> 
> Tim

Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards

[Tim Harvey]

On Fri, Apr 14, 2017 at 12:41 AM, Bough Chen <haibo.chen@nxp.com> wrote:
>> -----Original Message-----
>> From: Tim Harvey [mailto:tharvey@gateworks.com]
>> Sent: Tuesday, April 11, 2017 10:22 PM
>> To: Bough Chen <haibo.chen@nxp.com>
>> Cc: David Haworth <dave@fen-net.de>; Linux MMC List <linux-
>> mmc@vger.kernel.org>; Fabio Estevam <fabio.estevam@nxp.com>; Ulf
>> Hansson <ulf.hansson@linaro.org>; Weijun Yang <york.yang@csr.com>; Adrian
>> Hunter <adrian.hunter@intel.com>
>> Subject: Re: IMX6 ESDHC mmc with UHS-I occasional failure with DDR50 cards
>>
>> On Tue, Apr 11, 2017 at 2:28 AM, Bough Chen <haibo.chen@nxp.com> wrote:
>> >> -----Original Message-----
>> <snip>
>> >>
>> >> I've moved on to 4.11-rc5 and notice that I'm always seeing that on a
>> >> failure the imx sdhc not able to determine a min/max tuning delay:
>> >>
>> >> 4.11.0-rc5:
>> >> [   48.812381] mmc0: card aaaa removed
>> >> [   49.793392] mmc0: host does not support reading read-only switch,
>> >> assuming write-enable
>> >> [   49.810294] sdhci_execute_tuning
>> >> [   49.813612] esdhc_executing_tuning
>> >> [   49.968154] sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f
>> >> ret -5 (min=127/max=128)
>> >> ^^^^ added debug shows -EIO returned from mmc_send_tuning although
>> >> I've also seen -84 get returned
>> >> [   49.980540] mmc0: tuning execution failed: -5
>> >> [   49.984974] mmc0: ddr50 tuning failed
>> >> [   49.989068] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
>> >> [   49.999844] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
>> >>
>> >> I've also found that if I revert '9faac7b mmc: sdhci: enable tuning for DDR50'
>> >> thus skipping the tuning the issue goes away however note that
>> >> occasionally I do see a transfer error that perhaps is retried [
>> >> 361.067763] mmc0: card aaaa removed [  361.412732] mmc0: host does
>> >> not support reading read-only switch, assuming write-enable [
>> >> 361.429719] mmc0: new ultra high speed DDR50 SDHC card at address
>> >> aaaa [  361.443060] mmcblk0: mmc0:aaaa SL08G 7.40 GiB [  361.465658]
>> >> mmcblk0: p1 [  362.422335] mmc0: card aaaa removed [  362.752719]
>> >> mmc0: host does not support reading read-only switch, assuming
>> >> write-enable [  362.769675] mmc0: new ultra high speed DDR50 SDHC
>> >> card at address aaaa [  362.782900] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
>> >> [  362.805996]
>> >> mmcblk0: error -84 transferring data, sector 0, nr 8, cmd response
>> >> 0x900, card status 0xb00 ^^^^ occasionally see this error but because
>> >> retries occur at the block level it still works [  362.931689]
>> >> mmcblk0: p1
>> >>
>> >> Again with 9faac7b reverted and some extra debugging:
>> >> [   33.823170] mmc0: host does not support reading read-only switch,
>> >> assuming write-enable
>> >> [   33.847185] mmc0: new ultra high speed DDR50 SDHC card at address aaaa
>> >> [   33.856063] mmcblk0: mmc0:aaaa SL08G 7.40 GiB
>> >> [   33.874227] mmcblk0: error -84 transferring data, sector 0, nr 8,
>> >> cmd response 0x900, card status 0xb00
>> >> [   33.896829] mmcblk0: error -84 transferring data, sector 0, nr 8,
>> >> cmd response 0x900, card status 0xb00
>> >> [   33.906312] mmcblk0: error -84 transferring data, sector 0, nr 8,
>> >> cmd response 0x900, card status 0xb00
>> >> [   34.022117] mmcblk0: error 0 transferring data, sector 0, nr 8, cmd
>> >> response 0x900, card status 0xb00
>> >> ^^^^ added debugging shows retries at block level and eventually all is fine
>> >> [   34.031500]  mmcblk0: p1
>> >>
>> >> So far every DDR50 card I have (several Kingston and Sandisk card
>> >> models) behaves like this.
>> >>
>> >> I tried adding a settling delay when VSELECT was changed in the
>> >> sdhci-esdhc- imx driver and that didn't help.
>> >>
>> > Hi Tim
>> >
>> > Which imx6 soc do you use?  From your log, I notice you use the MAN tuning
>> method, so I guess you use imx6qdl.
>> >
>> > imx usdhc has an IC bug. For the tuning procedure, the first pass tuning delay
>> value must bigger than 10.
>> > Due to DDR50 timing is not that rigorous, so maybe even when we just add 1
>> delay cell, tuning can pass, but this trigger the internal IC bug, then you meet
>> the following issue just as you meet:
>> >        sdhci-esdhc-imx 2198000.usdhc: tuning failed at 0x7f
>> ( min=127/max=128)
>> >
>>
>> Haibo,
>>
>> Thanks for the response.
>>
>> Yes, this issue is seen on the IMX6DQ and IMX6SDL. Can you point me to
>> information on this bug? I don't see it in the latest IMX6DQCE.pdf errata
>> document.
>>
>> > So you can change the ESDHC_TUNE_CTRL_MIN from 0 to 15, then you can
>> see the DDR50 tuning return to normal.
>> >
>> > For the STD tuning method, you can add the property 'fsl,tuning-start-tap =
>> <20>' in dts.
>> >
>> > All this make the first trying tuning delay value larger than 10, to avoid  the
>> internal IMX USDHC IC bug.
>> >
>> > SDR104 card do not has this issue, because for SDR104 card, the first pass
>> tuning value normally larger than 10.
>> >
>>
>> unfortunately, this does not resolve the issue. I tried ESDHC_TUNE_CTRL_MIN
>> of 15, 20, and 30 and they all produce the same issue around 20% of insertions:
>> sdhci-esdhc-imx 2198000.usdhc: tunning failed at 0x7f ret -84
>>
>> Here are some min/max tuning values from some various cards I have:
>> - Samsung SL08G DDR50: min=8 max=37
>> - Kingston SD8GB DDR50: min=8 max=36
>> - Sandisk SS16G DDR50: min=10 max=37
>> - Sandisk SU08G DDR50: min=9 max=36
>> - Samsung Pro 00000 SDR104: min=14 max=26
>> - Kingston SDR104 SDCIT: min=25 max=36
>>
>> All of the DDR50 cards above fail in the same way appx 20% of insertions.
>>
>> Has anyone else been able to confirm they see the same issue on IMX6 boards
>> with UHS-I support and DDR50?
>
> Hi Tim,
>
> I debug this issue these days, and find this is the I/O timing issue.
>
> Can you try to improve the pad I/O drive strength for DDR50 card? You can apply the following patch:
>
> diff --git a/drivers/mmc/host/sdhci-esdhc-imx.c b/drivers/mmc/host/sdhci-esdhc-imx.c
> index 6a1328e..4c3aeac 100644
> --- a/drivers/mmc/host/sdhci-esdhc-imx.c
> +++ b/drivers/mmc/host/sdhci-esdhc-imx.c
> @@ -848,6 +848,7 @@ static int esdhc_change_pinstate(struct sdhci_host *host,
>
>         switch (uhs) {
>         case MMC_TIMING_UHS_SDR50:
> +       case MMC_TIMING_UHS_DDR50:
>                 pinctrl = imx_data->pins_100mhz;
>                 break;
>         case MMC_TIMING_UHS_SDR104:
>
> If this patch also test pass on your side, I will upstream it.
>

Haibo,

Yes, this does resolve the issue where DDR50 cards fail to tune 20% of
the time. Using the 100mhz pinctrl's for DDR50 passes tuning every
time with a min=1 and max=29 (as opposed to a min=8 max=36 I was
seeing on previous successes).

Please submit a patch if you don't mind. Also, stable@vger.kernel.org
should be cc'd on the patch and the patch should be applied all the
way back to Linux v4.4 where DDR50 tuning first appeared with 9faac7b
mmc: sdhci: enable tuning for DDR50.

> By the way, I just confirm with our IC guys, the IC bug I mentioned in the before mail do not impact the software tuning method (MAN_TUNING), this IC bug just impact the STD_TUNING, sorry for the mislead.
>

Ok, thanks for the info. Note still that I don't see anything in the
IMX6 errata regarding thisb.

Regards,

Tim