From: Ulf Hansson <ulf.hansson@linaro.org>
To: Faiz Abbas <faiz_abbas@ti.com>
Cc: Jon Hunter <jonathanh@nvidia.com>,
Bitan Biswas <bbiswas@nvidia.com>,
Sowjanya Komatineni <skomatineni@nvidia.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Naresh Kamboju <naresh.kamboju@linaro.org>,
Jens Axboe <axboe@kernel.dk>, Alexei Starovoitov <ast@kernel.org>,
linux-block <linux-block@vger.kernel.org>,
lkft-triage@lists.linaro.org,
open list <linux-kernel@vger.kernel.org>,
"linux-mmc@vger.kernel.org" <linux-mmc@vger.kernel.org>,
Arnd Bergmann <arnd@arndb.de>,
John Stultz <john.stultz@linaro.org>,
Thierry Reding <treding@nvidia.com>,
Anders Roxell <anders.roxell@linaro.org>, Kishon <kishon@ti.com>
Subject: Re: LKFT: arm x15: mmc1: cache flush error -110
Date: Tue, 3 Mar 2020 22:35:50 +0100 [thread overview]
Message-ID: <CAPDyKFr-ntyXv8C0FB5oq4wWe-SL_YVjpvJVykK2+7jgqY82-Q@mail.gmail.com> (raw)
In-Reply-To: <CAPDyKFrrO4noYqdxWL9Y8Nx75LopbDudKGMotkGbGcAF1oq==w@mail.gmail.com>
On Mon, 2 Mar 2020 at 17:50, Ulf Hansson <ulf.hansson@linaro.org> wrote:
>
> On Mon, 2 Mar 2020 at 14:11, Faiz Abbas <faiz_abbas@ti.com> wrote:
> >
> > Uffe,
> >
> > On 26/02/20 8:51 pm, Ulf Hansson wrote:
> > > + Anders, Kishon
> > >
> > > On Tue, 25 Feb 2020 at 17:24, Jon Hunter <jonathanh@nvidia.com> wrote:
> > >>
> > >>
> > >> On 25/02/2020 14:26, Ulf Hansson wrote:
> > >>
> > >> ...
> > >>
> > >>> However, from the core point of view, the response is still requested,
> > >>> only that we don't want the driver to wait for the card to stop
> > >>> signaling busy. Instead we want to deal with that via "polling" from
> > >>> the core.
> > >>>
> > >>> This is a rather worrying behaviour, as it seems like the host driver
> > >>> doesn't really follow this expectations from the core point of view.
> > >>> And mmc_flush_cache() is not the only case, as we have erase, bkops,
> > >>> sanitize, etc. Are all these working or not really well tested?
> > >>
> > >> I don't believe that they are well tested. We have a simple test to
> > >> mount an eMMC partition, create a file, check the contents, remove the
> > >> file and unmount. The timeouts always occur during unmounting.
> > >>
> > >>> Earlier, before my three patches, if the provided timeout_ms parameter
> > >>> to __mmc_switch() was zero, which was the case for
> > >>> mmc_mmc_flush_cache() - this lead to that __mmc_switch() simply
> > >>> ignored validating host->max_busy_timeout, which was wrong. In any
> > >>> case, this also meant that an R1B response was always used for
> > >>> mmc_flush_cache(), as you also indicated above. Perhaps this is the
> > >>> critical part where things can go wrong.
> > >>>
> > >>> BTW, have you tried erase commands for sdhci tegra driver? If those
> > >>> are working fine, do you have any special treatments for these?
> > >>
> > >> That I am not sure, but I will check.
> > >
> > > Great, thanks. Looking forward to your report.
> > >
> > > So, from my side, me and Anders Roxell, have been collaborating on
> > > testing the behaviour on a TI Beagleboard x15 (remotely with limited
> > > debug options), which is using the sdhci-omap variant. I am trying to
> > > get hold of an Nvidia jetson-TX2, but not found one yet. These are the
> > > conclusions from the observed behaviour on the Beagleboard for the
> > > CMD6 cache flush command.
> > >
> > > First, the reported host->max_busy_timeout is 2581 (ms) for the
> > > sdhci-omap driver in this configuration.
> > >
> > > 1. As we all know by now, the cache flush command (CMD6) fails with
> > > -110 currently. This is when MMC_CACHE_FLUSH_TIMEOUT_MS is set to 30 *
> > > 1000 (30s), which means __mmc_switch() drops the MMC_RSP_BUSY flag
> > > from the command.
> > >
> > > 2. Changing the MMC_CACHE_FLUSH_TIMEOUT_MS to 2000 (2s), means that
> > > the MMC_RSP_BUSY flag becomes set by __mmc_switch, because of the
> > > timeout_ms parameter is less than max_busy_timeout (2000 < 2581).
> > > Then everything works fine.
> > >
> > > 3. Updating the code to again use 30s as the
> > > MMC_CACHE_FLUSH_TIMEOUT_MS, but instead forcing the MMC_RSP_BUSY to be
> > > set, even when the timeout_ms becomes greater than max_busy_timeout.
> > > This also works fine.
> > >
> > > Clearly this indicates a problem that I think needs to be addressed in
> > > the sdhci driver. However, of course I can revert the three discussed
> > > patches to fix the problem, but that would only hide the issues and I
> > > am sure we would then get back to this issue, sooner or later.
> > >
> > > To fix the problem in the sdhci driver, I would appreciate if someone
> > > from TI and Nvidia can step in to help, as I don't have the HW on my
> > > desk.
> > >
> > > Comments or other ideas of how to move forward?
> > >
> >
> > Sorry I missed this earlier.
> >
> > I don't have an X15 with me here but I'm trying to set one up in our
> > remote farm. In the meantime, I tried to reproduce this issue on two
> > platforms (dra72-evm and am57xx-evm) and wasn't able to see the issue
> > because those eMMC's don't even have a cache. I will keep you updated
> > when I do get a board with a eMMC that has a cache.
> >
> > Is there a way to reproduce this CMD6 issue with another operation?
>
> Yes, most definitely.
>
> Let me cook a debug patch for you that should trigger the problem for
> another CMD6 operation. I will post something later this evening or in
> the mornings (Swedish timezone).
A bit later than promised, I am clearly an optimist. In any case
here's the patch I had in mind to trigger the problem for other CMD6
operations. Please give at shot and see what happens.
-------
From: Ulf Hansson <ulf.hansson@linaro.org>
Date: Tue, 3 Mar 2020 22:11:05 +0100
Subject: [PATCH] mmc: core: DEBUG: Force a long timeout for all CMD6
This is to test sdhci-omap, for example, to see what happens when using a
longer timeout. My guess is that it triggers __mmc_switch() to disable the
MMC_RSP_BUSY flag for the command. If so, it likely to make the host driver
to fail, in some way or the other.
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
---
drivers/mmc/core/mmc_ops.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/mmc/core/mmc_ops.c b/drivers/mmc/core/mmc_ops.c
index da425ee2d9bf..f0d2563961f6 100644
--- a/drivers/mmc/core/mmc_ops.c
+++ b/drivers/mmc/core/mmc_ops.c
@@ -532,6 +532,9 @@ int __mmc_switch(struct mmc_card *card, u8 set, u8
index, u8 value,
mmc_retune_hold(host);
+ /* Force a long timeout to likely make use_r1b_resp to become false. */
+ timeout_ms = MMC_CACHE_FLUSH_TIMEOUT_MS;
+
if (!timeout_ms) {
pr_warn("%s: unspecified timeout for CMD6 - use generic\n",
mmc_hostname(host));
@@ -544,8 +547,11 @@ int __mmc_switch(struct mmc_card *card, u8 set,
u8 index, u8 value,
* the host to avoid HW busy detection, by converting to a R1 response
* instead of a R1B.
*/
- if (host->max_busy_timeout && (timeout_ms > host->max_busy_timeout))
+ if (host->max_busy_timeout && (timeout_ms > host->max_busy_timeout)) {
+ pr_warn("%s:Disable MMC_RSP_BUSY. timeout_ms(%u) >
max_busy_timeout(%u)\n",
+ mmc_hostname(host), timeout_ms, host->max_busy_timeout);
use_r1b_resp = false;
+ }
cmd.opcode = MMC_SWITCH;
cmd.arg = (MMC_SWITCH_MODE_WRITE_BYTE << 24) |
--
Kind regards
Uffe
next prev parent reply other threads:[~2020-03-03 21:36 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-13 15:42 LKFT: arm x15: mmc1: cache flush error -110 Naresh Kamboju
2020-02-14 9:09 ` Arnd Bergmann
2020-02-14 12:09 ` Mark Brown
2020-02-19 16:23 ` Ulf Hansson
2020-02-20 17:54 ` Naresh Kamboju
2020-02-21 9:48 ` Ulf Hansson
2020-02-21 19:44 ` Bitan Biswas
2020-02-24 11:16 ` Ulf Hansson
2020-02-24 12:59 ` Adrian Hunter
2020-02-25 10:04 ` Jon Hunter
2020-02-25 11:35 ` Ulf Hansson
2020-02-25 11:41 ` Jon Hunter
2020-02-25 14:26 ` Ulf Hansson
2020-02-25 16:24 ` Jon Hunter
2020-02-26 15:21 ` Ulf Hansson
2020-02-26 17:04 ` Jon Hunter
2020-03-02 13:12 ` Faiz Abbas
2020-03-02 16:50 ` Ulf Hansson
2020-03-03 21:35 ` Ulf Hansson [this message]
[not found] ` <5e9b5646-bd48-e55b-54ee-1c2c41fc9218@nvidia.com>
2020-03-04 10:18 ` Ulf Hansson
2020-03-04 10:32 ` Ulf Hansson
2020-03-04 16:56 ` Sowjanya Komatineni
2020-03-04 17:21 ` Sowjanya Komatineni
2020-03-04 17:26 ` Sowjanya Komatineni
2020-03-04 17:51 ` Sowjanya Komatineni
2020-03-04 22:35 ` Sowjanya Komatineni
2020-03-05 0:20 ` Sowjanya Komatineni
2020-03-05 3:06 ` Sowjanya Komatineni
2020-03-05 13:05 ` Ulf Hansson
2020-03-06 2:44 ` Sowjanya Komatineni
2020-03-06 11:14 ` Ulf Hansson
2020-03-09 14:07 ` Faiz Abbas
2020-03-09 15:57 ` Ulf Hansson
2020-03-09 17:35 ` Sowjanya Komatineni
2020-03-10 9:46 ` Ulf Hansson
2020-03-10 16:59 ` Sowjanya Komatineni
2020-03-10 17:09 ` Ulf Hansson
2020-03-10 17:27 ` Sowjanya Komatineni
2020-03-10 21:59 ` Sowjanya Komatineni
2020-03-10 23:10 ` Sowjanya Komatineni
2020-03-11 0:22 ` Sowjanya Komatineni
2020-03-11 8:34 ` Ulf Hansson
2020-03-19 19:12 ` Naresh Kamboju
2020-03-20 9:20 ` Ulf Hansson
2020-03-20 9:49 ` Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAPDyKFr-ntyXv8C0FB5oq4wWe-SL_YVjpvJVykK2+7jgqY82-Q@mail.gmail.com \
--to=ulf.hansson@linaro.org \
--cc=adrian.hunter@intel.com \
--cc=anders.roxell@linaro.org \
--cc=arnd@arndb.de \
--cc=ast@kernel.org \
--cc=axboe@kernel.dk \
--cc=bbiswas@nvidia.com \
--cc=faiz_abbas@ti.com \
--cc=john.stultz@linaro.org \
--cc=jonathanh@nvidia.com \
--cc=kishon@ti.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mmc@vger.kernel.org \
--cc=lkft-triage@lists.linaro.org \
--cc=naresh.kamboju@linaro.org \
--cc=skomatineni@nvidia.com \
--cc=treding@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).