Re: LKFT: arm x15: mmc1: cache flush error -110

From: Bitan Biswas <bbiswas@nvidia.com>
To: Ulf Hansson <ulf.hansson@linaro.org>,
	Naresh Kamboju <naresh.kamboju@linaro.org>
Cc: Jens Axboe <axboe@kernel.dk>, Alexei Starovoitov <ast@kernel.org>,
	linux-block <linux-block@vger.kernel.org>,
	<lkft-triage@lists.linaro.org>,
	open list <linux-kernel@vger.kernel.org>,
	"linux-mmc@vger.kernel.org" <linux-mmc@vger.kernel.org>,
	Arnd Bergmann <arnd@arndb.de>,
	John Stultz <john.stultz@linaro.org>,
	Thierry Reding <treding@nvidia.com>,
	Jon Hunter <jonathanh@nvidia.com>
Subject: Re: LKFT: arm x15: mmc1: cache flush error -110
Date: Fri, 21 Feb 2020 11:44:44 -0800	[thread overview]
Message-ID: <a602a27a-b960-ce56-c541-3b4b95f5dce2@nvidia.com> (raw)
In-Reply-To: <CAPDyKFo-vEO7zN_F+NqcKtnKmAo_deOZx3gYNiks3yTAQAjv-Q@mail.gmail.com>

On 2/21/20 1:48 AM, Ulf Hansson wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Thu, 20 Feb 2020 at 18:54, Naresh Kamboju <naresh.kamboju@linaro.org> wrote:
>>
>> On Wed, 19 Feb 2020 at 21:54, Ulf Hansson <ulf.hansson@linaro.org> wrote:
>>>
>>> On Thu, 13 Feb 2020 at 16:43, Naresh Kamboju <naresh.kamboju@linaro.org> wrote:
>>>>
>>>
>>> Try to restore the value for the cache flush timeout, by updating the
>>> define MMC_CACHE_FLUSH_TIMEOUT_MS to 10 * 60 * 1000".
>>
>> I have increased the timeout to 10 minutes but it did not help.
>> Same error found.
>> [  608.679353] mmc1: Card stuck being busy! mmc_poll_for_busy
>> [  608.684964] mmc1: cache flush error -110
>> [  608.689005] blk_update_request: I/O error, dev mmcblk1, sector
>> 4302400 op 0x1:(WRITE) flags 0x20800 phys_seg 1 prio class 0
>>
>> OTOH, What best i could do for my own experiment to revert all three patches and
>> now the reported error gone and device mount successfully [1].
>>
>> List of patches reverted,
>>    mmc: core: Specify timeouts for BKOPS and CACHE_FLUSH for eMMC
>>    mmc: block: Use generic_cmd6_time when modifying
>>      INAND_CMD38_ARG_EXT_CSD
>>    mmc: core: Default to generic_cmd6_time as timeout in __mmc_switch()
>>
>> [1] https://lkft.validation.linaro.org/scheduler/job/1238275#L4346
>>
>> - Naresh
> 
> Thanks for testing!
> 
> This sounds a bit weird, I must say. Also, while looking into the
> logs, it seems like you are comparing a v5.5 kernel with v5.6-rc2, but
> maybe I didn't read the logs carefully enough.
> 
>   In any case, I am looking into creating a debug patch so we can
> narrow down the problem a bit further.
> 
> Kind regards
> Uffe
> 

Hi Ulf,

  I see that Jetson-TX2 / Jetson-Xavier suspend test is aborted and 
below error is seen due to the commit 
24ed3bd01d6a844fd5e8a75f48d0a3d10ed71bf9  ("mmc: core: Specify timeouts 
for BKOPS and CACHE_FLUSH for eMMC"):

##
[  268.976197] Freezing user space processes ... (elapsed 0.001 seconds) 
done.
[  268.984414] OOM killer disabled.
[  268.987635] Freezing remaining freezable tasks ... (elapsed 0.000 
seconds) done.
[  269.217471] PM: dpm_run_callback(): mmc_bus_suspend+0x0/0x58 returns -110
[  269.224255] PM: Device mmc1:0001 failed to suspend: error -110
[  269.230080] PM: Some devices failed to suspend, or early wake event 
detected
##

  I find that from the commit the changes in mmc_flush_cache below is 
the cause.

##
@@ -961,7 +963,8 @@ int mmc_flush_cache(struct mmc_card *card)
                         (card->ext_csd.cache_size > 0) &&
                         (card->ext_csd.cache_ctrl & 1)) {
                 err = mmc_switch(card, EXT_CSD_CMD_SET_NORMAL,
-                               EXT_CSD_FLUSH_CACHE, 1, 0);
+                                EXT_CSD_FLUSH_CACHE, 1,
+                                MMC_CACHE_FLUSH_TIMEOUT_MS);

##

  Do you have suggestion to try for the suspend errors ?

-regards,
  Bitan

