All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Asutosh Das (asd)" <asutoshd@codeaurora.org>
To: Adrian Hunter <adrian.hunter@intel.com>,
	cang@codeaurora.org, martin.petersen@oracle.com,
	linux-scsi@vger.kernel.org
Cc: linux-arm-msm@vger.kernel.org,
	Alim Akhtar <alim.akhtar@samsung.com>,
	Avri Altman <avri.altman@wdc.com>,
	"James E.J. Bottomley" <jejb@linux.ibm.com>,
	Krzysztof Kozlowski <krzk@kernel.org>,
	Stanley Chu <stanley.chu@mediatek.com>,
	Andy Gross <agross@kernel.org>,
	Bjorn Andersson <bjorn.andersson@linaro.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ingo Molnar <mingo@redhat.com>,
	Matthias Brugger <matthias.bgg@gmail.com>,
	Lee Jones <lee.jones@linaro.org>, Bean Huo <beanhuo@micron.com>,
	Kiwoong Kim <kwmad.kim@samsung.com>,
	Colin Ian King <colin.king@canonical.com>,
	Wei Yongjun <weiyongjun1@huawei.com>, Yue Hu <huyue2@yulong.com>,
	Bart van Assche <bvanassche@acm.org>,
	"Gustavo A. R. Silva" <gustavoars@kernel.org>,
	Dinghao Liu <dinghao.liu@zju.edu.cn>,
	Jaegeuk Kim <jaegeuk@kernel.org>,
	Satya Tangirala <satyat@google.com>,
	open list <linux-kernel@vger.kernel.org>,
	"moderated list:ARM/SAMSUNG S3C,
	S5P AND EXYNOS ARM ARCHITECTURES" 
	<linux-arm-kernel@lists.infradead.org>,
	"open list:ARM/SAMSUNG S3C,
	S5P AND EXYNOS ARM ARCHITECTURES" 
	<linux-samsung-soc@vger.kernel.org>,
	"moderated list:UNIVERSAL FLASH STORAGE HOST CONTROLLER
	DRIVER..."  <linux-mediatek@lists.infradead.org>
Subject: Re: [PATCH v20 1/2] scsi: ufs: Enable power management for wlun
Date: Thu, 22 Apr 2021 09:38:58 -0700	[thread overview]
Message-ID: <651f5d8a-5ab7-77dd-3fed-05feb3fd3e1a@codeaurora.org> (raw)
In-Reply-To: <1bc4a73e-b22a-6bad-2583-3a0ffa979414@intel.com>

On 4/20/2021 12:42 AM, Adrian Hunter wrote:
> On 20/04/21 7:15 am, Adrian Hunter wrote:
>> On 20/04/21 12:53 am, Asutosh Das (asd) wrote:
>>> On 4/19/2021 11:37 AM, Adrian Hunter wrote:
>>>> On 16/04/21 10:49 pm, Asutosh Das wrote:
>>>>>
>>>>> Co-developed-by: Can Guo <cang@codeaurora.org>
>>>>> Signed-off-by: Can Guo <cang@codeaurora.org>
>>>>> Signed-off-by: Asutosh Das <asutoshd@codeaurora.org>
>>>>> ---
>>>>
>>>> I came across 3 issues while testing.  See comments below.
>>>>
>>> Hi Adrian
>>> Thanks for the comments.
>>>> <SNIP>
>>>>
>>>>> @@ -5794,7 +5839,7 @@ static void ufshcd_err_handling_unprepare(struct ufs_hba *hba)
>>>>>        if (ufshcd_is_clkscaling_supported(hba))
>>>>>            ufshcd_clk_scaling_suspend(hba, false);
>>>>>        ufshcd_clear_ua_wluns(hba);
>>>>
>>>> ufshcd_clear_ua_wluns() deadlocks trying to clear UFS_UPIU_RPMB_WLUN
>>>> if sdev_rpmb is suspended and sdev_ufs_device is suspending.
>>>> e.g. ufshcd_wl_suspend() is waiting on host_sem while ufshcd_err_handler()
>>>> is running, at which point sdev_rpmb has already suspended.
>>>>
>>> Umm, I didn't understand this deadlock.
>>> When you say, sdev_rpmb is suspended, does it mean runtime_suspended?
>>> sdev_ufs_device is suspending - this can't be runtime_suspending, while ufshcd_err_handling_unprepare is running.
>>>
>>> If you've a call-stack of this deadlock, please can you share it with me. I'll also try to reproduce this.
>>
>> Yes it is system suspend. sdev_rpmb has suspended, sdev_ufs_device is waiting on host_sem.
>> ufshcd_err_handler() holds host_sem. ufshcd_clear_ua_wlun(UFS_UPIU_RPMB_WLUN) gets stuck.
>> I will get some call-stacks.
> 
Hi Adrian,

Thanks for the call stacks.
 From the current information, I can't say for sure why it'd get stuck 
in blk_queue_enter().

I tried reproducing this issue on my setup yesterday but couldn't.
Here's what I did:
1. sdev_rpmb is RPM_SUSPENDED, checked before initiating system suspend
2. sdev_ufs_device is RPM_RESUMED
3. I triggered system suspend (echo mem > /sys/power/state) and 
scheduled the error handler from ufshcd_wl_suspend().
4. Waited until error handler ran and then ufshcd_wl_suspend() blocks on 
host_sem.
5. The ufshcd_clear_wa_wlun(UFS_UPIU_RPMB_WLUN) went through fine.

Do you've some specific steps to reproduce this or a script, perhaps? If 
so, please can you share it with me. I will try again.
My test environment is in 5.10 kernel with Android, I suppose that 
should be ok though.

Thanks
-asd

> Here are the call stacks
> 
> [   34.094321] Workqueue: ufs_eh_wq_0 ufshcd_err_handler
> [   34.094788] Call Trace:
> [   34.095281]  __schedule+0x275/0x6c0
> [   34.095743]  schedule+0x41/0xa0
> [   34.096240]  blk_queue_enter+0x10d/0x230
> [   34.096693]  ? wait_woken+0x70/0x70
> [   34.097167]  blk_mq_alloc_request+0x53/0xc0
> [   34.097610]  blk_get_request+0x1e/0x60
> [   34.098053]  __scsi_execute+0x3c/0x260
> [   34.098529]  ufshcd_clear_ua_wlun.cold+0xa6/0x14b
> [   34.098977]  ufshcd_clear_ua_wluns.part.0+0x4d/0x92
> [   34.099456]  ufshcd_err_handler+0x97a/0x9ff
> [   34.099902]  process_one_work+0x1cc/0x360
> [   34.100384]  worker_thread+0x45/0x3b0
> [   34.100851]  ? process_one_work+0x360/0x360
> [   34.101308]  kthread+0xf6/0x130
> [   34.101728]  ? kthread_park+0x80/0x80
> [   34.102186]  ret_from_fork+0x1f/0x30
> 
> [   34.640751] task:kworker/u10:9   state:D stack:14528 pid:  255 ppid:     2 flags:0x00004000
> [   34.641253] Workqueue: events_unbound async_run_entry_fn
> [   34.641722] Call Trace:
> [   34.642217]  __schedule+0x275/0x6c0
> [   34.642683]  schedule+0x41/0xa0
> [   34.643179]  schedule_timeout+0x18b/0x290
> [   34.643645]  ? del_timer_sync+0x30/0x30
> [   34.644131]  __down_timeout+0x6b/0xc0
> [   34.644568]  ? ufshcd_clkscale_enable_show+0x20/0x20
> [   34.645014]  ? async_schedule_node_domain+0x17d/0x190
> [   34.645496]  down_timeout+0x42/0x50
> [   34.645947]  ufshcd_wl_suspend+0x79/0xa0
> [   34.646432]  ? scmd_printk+0x100/0x100
> [   34.646917]  scsi_bus_suspend_common+0x56/0xc0
> [   34.647405]  ? scsi_bus_freeze+0x10/0x10
> [   34.647858]  dpm_run_callback+0x45/0x110
> [   34.648347]  __device_suspend+0x117/0x460
> [   34.648788]  async_suspend+0x16/0x90
> [   34.649251]  async_run_entry_fn+0x26/0x110
> [   34.649676]  process_one_work+0x1cc/0x360
> [   34.650137]  worker_thread+0x45/0x3b0
> [   34.650563]  ? process_one_work+0x360/0x360
> [   34.650994]  kthread+0xf6/0x130
> [   34.651455]  ? kthread_park+0x80/0x80
> [   34.651882]  ret_from_fork+0x1f/0x30
> 
> 
> 
>>
>>>
>>> I'll address the other comments in the next version.
>>>
>>>
>>> Thank you!
>>>
>>>>> -    pm_runtime_put(hba->dev);
>>>>> +    ufshcd_rpm_put(hba);
>>>>>    }
>>>>
>>>> <SNIP>
>>>>
>>>>> +void ufshcd_resume_complete(struct device *dev)
>>>>> +{
>>>
>>
> 


-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
Linux Foundation Collaborative Project

WARNING: multiple messages have this Message-ID
From: "Asutosh Das (asd)" <asutoshd@codeaurora.org>
To: Adrian Hunter <adrian.hunter@intel.com>,
	cang@codeaurora.org, martin.petersen@oracle.com,
	linux-scsi@vger.kernel.org
Cc: linux-arm-msm@vger.kernel.org,
	Alim Akhtar <alim.akhtar@samsung.com>,
	Avri Altman <avri.altman@wdc.com>,
	"James E.J. Bottomley" <jejb@linux.ibm.com>,
	Krzysztof Kozlowski <krzk@kernel.org>,
	Stanley Chu <stanley.chu@mediatek.com>,
	Andy Gross <agross@kernel.org>,
	Bjorn Andersson <bjorn.andersson@linaro.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ingo Molnar <mingo@redhat.com>,
	Matthias Brugger <matthias.bgg@gmail.com>,
	Lee Jones <lee.jones@linaro.org>, Bean Huo <beanhuo@micron.com>,
	Kiwoong Kim <kwmad.kim@samsung.com>,
	Colin Ian King <colin.king@canonical.com>,
	Wei Yongjun <weiyongjun1@huawei.com>, Yue Hu <huyue2@yulong.com>,
	Bart van Assche <bvanassche@acm.org>,
	"Gustavo A. R. Silva" <gustavoars@kernel.org>,
	Dinghao Liu <dinghao.liu@zju.edu.cn>,
	Jaegeuk Kim <jaegeuk@kernel.org>,
	Satya Tangirala <satyat@google.com>,
	open list <linux-kernel@vger.kernel.org>,
	"moderated list:ARM/SAMSUNG S3C,
	S5P AND EXYNOS ARM ARCHITECTURES"
	<linux-arm-kernel@lists.infradead.org>,
	"open list:ARM/SAMSUNG S3C,
	S5P AND EXYNOS ARM ARCHITECTURES"
	<linux-samsung-soc@vger.kernel.org>,
	"moderated list:UNIVERSAL FLASH STORAGE HOST CONTROLLER
	DRIVER..." <linux-mediatek@lists.infradead.org>
Subject: Re: [PATCH v20 1/2] scsi: ufs: Enable power management for wlun
Date: Thu, 22 Apr 2021 09:38:58 -0700	[thread overview]
Message-ID: <651f5d8a-5ab7-77dd-3fed-05feb3fd3e1a@codeaurora.org> (raw)
In-Reply-To: <1bc4a73e-b22a-6bad-2583-3a0ffa979414@intel.com>

On 4/20/2021 12:42 AM, Adrian Hunter wrote:
> On 20/04/21 7:15 am, Adrian Hunter wrote:
>> On 20/04/21 12:53 am, Asutosh Das (asd) wrote:
>>> On 4/19/2021 11:37 AM, Adrian Hunter wrote:
>>>> On 16/04/21 10:49 pm, Asutosh Das wrote:
>>>>>
>>>>> Co-developed-by: Can Guo <cang@codeaurora.org>
>>>>> Signed-off-by: Can Guo <cang@codeaurora.org>
>>>>> Signed-off-by: Asutosh Das <asutoshd@codeaurora.org>
>>>>> ---
>>>>
>>>> I came across 3 issues while testing.  See comments below.
>>>>
>>> Hi Adrian
>>> Thanks for the comments.
>>>> <SNIP>
>>>>
>>>>> @@ -5794,7 +5839,7 @@ static void ufshcd_err_handling_unprepare(struct ufs_hba *hba)
>>>>>        if (ufshcd_is_clkscaling_supported(hba))
>>>>>            ufshcd_clk_scaling_suspend(hba, false);
>>>>>        ufshcd_clear_ua_wluns(hba);
>>>>
>>>> ufshcd_clear_ua_wluns() deadlocks trying to clear UFS_UPIU_RPMB_WLUN
>>>> if sdev_rpmb is suspended and sdev_ufs_device is suspending.
>>>> e.g. ufshcd_wl_suspend() is waiting on host_sem while ufshcd_err_handler()
>>>> is running, at which point sdev_rpmb has already suspended.
>>>>
>>> Umm, I didn't understand this deadlock.
>>> When you say, sdev_rpmb is suspended, does it mean runtime_suspended?
>>> sdev_ufs_device is suspending - this can't be runtime_suspending, while ufshcd_err_handling_unprepare is running.
>>>
>>> If you've a call-stack of this deadlock, please can you share it with me. I'll also try to reproduce this.
>>
>> Yes it is system suspend. sdev_rpmb has suspended, sdev_ufs_device is waiting on host_sem.
>> ufshcd_err_handler() holds host_sem. ufshcd_clear_ua_wlun(UFS_UPIU_RPMB_WLUN) gets stuck.
>> I will get some call-stacks.
> 
Hi Adrian,

Thanks for the call stacks.
 From the current information, I can't say for sure why it'd get stuck 
in blk_queue_enter().

I tried reproducing this issue on my setup yesterday but couldn't.
Here's what I did:
1. sdev_rpmb is RPM_SUSPENDED, checked before initiating system suspend
2. sdev_ufs_device is RPM_RESUMED
3. I triggered system suspend (echo mem > /sys/power/state) and 
scheduled the error handler from ufshcd_wl_suspend().
4. Waited until error handler ran and then ufshcd_wl_suspend() blocks on 
host_sem.
5. The ufshcd_clear_wa_wlun(UFS_UPIU_RPMB_WLUN) went through fine.

Do you've some specific steps to reproduce this or a script, perhaps? If 
so, please can you share it with me. I will try again.
My test environment is in 5.10 kernel with Android, I suppose that 
should be ok though.

Thanks
-asd

> Here are the call stacks
> 
> [   34.094321] Workqueue: ufs_eh_wq_0 ufshcd_err_handler
> [   34.094788] Call Trace:
> [   34.095281]  __schedule+0x275/0x6c0
> [   34.095743]  schedule+0x41/0xa0
> [   34.096240]  blk_queue_enter+0x10d/0x230
> [   34.096693]  ? wait_woken+0x70/0x70
> [   34.097167]  blk_mq_alloc_request+0x53/0xc0
> [   34.097610]  blk_get_request+0x1e/0x60
> [   34.098053]  __scsi_execute+0x3c/0x260
> [   34.098529]  ufshcd_clear_ua_wlun.cold+0xa6/0x14b
> [   34.098977]  ufshcd_clear_ua_wluns.part.0+0x4d/0x92
> [   34.099456]  ufshcd_err_handler+0x97a/0x9ff
> [   34.099902]  process_one_work+0x1cc/0x360
> [   34.100384]  worker_thread+0x45/0x3b0
> [   34.100851]  ? process_one_work+0x360/0x360
> [   34.101308]  kthread+0xf6/0x130
> [   34.101728]  ? kthread_park+0x80/0x80
> [   34.102186]  ret_from_fork+0x1f/0x30
> 
> [   34.640751] task:kworker/u10:9   state:D stack:14528 pid:  255 ppid:     2 flags:0x00004000
> [   34.641253] Workqueue: events_unbound async_run_entry_fn
> [   34.641722] Call Trace:
> [   34.642217]  __schedule+0x275/0x6c0
> [   34.642683]  schedule+0x41/0xa0
> [   34.643179]  schedule_timeout+0x18b/0x290
> [   34.643645]  ? del_timer_sync+0x30/0x30
> [   34.644131]  __down_timeout+0x6b/0xc0
> [   34.644568]  ? ufshcd_clkscale_enable_show+0x20/0x20
> [   34.645014]  ? async_schedule_node_domain+0x17d/0x190
> [   34.645496]  down_timeout+0x42/0x50
> [   34.645947]  ufshcd_wl_suspend+0x79/0xa0
> [   34.646432]  ? scmd_printk+0x100/0x100
> [   34.646917]  scsi_bus_suspend_common+0x56/0xc0
> [   34.647405]  ? scsi_bus_freeze+0x10/0x10
> [   34.647858]  dpm_run_callback+0x45/0x110
> [   34.648347]  __device_suspend+0x117/0x460
> [   34.648788]  async_suspend+0x16/0x90
> [   34.649251]  async_run_entry_fn+0x26/0x110
> [   34.649676]  process_one_work+0x1cc/0x360
> [   34.650137]  worker_thread+0x45/0x3b0
> [   34.650563]  ? process_one_work+0x360/0x360
> [   34.650994]  kthread+0xf6/0x130
> [   34.651455]  ? kthread_park+0x80/0x80
> [   34.651882]  ret_from_fork+0x1f/0x30
> 
> 
> 
>>
>>>
>>> I'll address the other comments in the next version.
>>>
>>>
>>> Thank you!
>>>
>>>>> -    pm_runtime_put(hba->dev);
>>>>> +    ufshcd_rpm_put(hba);
>>>>>    }
>>>>
>>>> <SNIP>
>>>>
>>>>> +void ufshcd_resume_complete(struct device *dev)
>>>>> +{
>>>
>>
> 


-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
Linux Foundation Collaborative Project

_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek

  reply	other threads:[~2021-04-22 16:39 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-16 19:49 [PATCH v20 0/2] Enable power management for ufs wlun Asutosh Das
2021-04-16 19:49 ` Asutosh Das
2021-04-16 19:49 ` Asutosh Das
2021-04-16 19:49 ` [PATCH v20 1/2] scsi: ufs: Enable power management for wlun Asutosh Das
2021-04-16 19:49   ` Asutosh Das
2021-04-16 19:49   ` Asutosh Das
2021-04-19 18:37   ` Adrian Hunter
2021-04-19 18:37     ` Adrian Hunter
2021-04-19 18:37     ` Adrian Hunter
2021-04-19 21:53     ` Asutosh Das (asd)
2021-04-19 21:53       ` Asutosh Das (asd)
2021-04-20  4:15       ` Adrian Hunter
2021-04-20  4:15         ` Adrian Hunter
2021-04-20  4:15         ` Adrian Hunter
2021-04-20  7:42         ` Adrian Hunter
2021-04-20  7:42           ` Adrian Hunter
2021-04-20  7:42           ` Adrian Hunter
2021-04-22 16:38           ` Asutosh Das (asd) [this message]
2021-04-22 16:38             ` Asutosh Das (asd)
2021-04-23  4:23             ` Adrian Hunter
2021-04-23  4:23               ` Adrian Hunter
2021-04-23  4:23               ` Adrian Hunter
2021-04-23  6:18               ` Adrian Hunter
2021-04-23  6:18                 ` Adrian Hunter
2021-04-23  6:18                 ` Adrian Hunter
2021-04-23  8:01                 ` Adrian Hunter
2021-04-23  8:01                   ` Adrian Hunter
2021-04-23  8:01                   ` Adrian Hunter
2021-04-23 23:44                   ` Asutosh Das (asd)
2021-04-23 23:44                     ` Asutosh Das (asd)
2021-04-16 19:49 ` [PATCH v20 2/2] ufs: sysfs: Resume the proper scsi device Asutosh Das

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=651f5d8a-5ab7-77dd-3fed-05feb3fd3e1a@codeaurora.org \
    --to=asutoshd@codeaurora.org \
    --cc=adrian.hunter@intel.com \
    --cc=agross@kernel.org \
    --cc=alim.akhtar@samsung.com \
    --cc=avri.altman@wdc.com \
    --cc=beanhuo@micron.com \
    --cc=bjorn.andersson@linaro.org \
    --cc=bvanassche@acm.org \
    --cc=cang@codeaurora.org \
    --cc=colin.king@canonical.com \
    --cc=dinghao.liu@zju.edu.cn \
    --cc=gustavoars@kernel.org \
    --cc=huyue2@yulong.com \
    --cc=jaegeuk@kernel.org \
    --cc=jejb@linux.ibm.com \
    --cc=krzk@kernel.org \
    --cc=kwmad.kim@samsung.com \
    --cc=lee.jones@linaro.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-arm-msm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mediatek@lists.infradead.org \
    --cc=linux-samsung-soc@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=matthias.bgg@gmail.com \
    --cc=mingo@redhat.com \
    --cc=rostedt@goodmis.org \
    --cc=satyat@google.com \
    --cc=stanley.chu@mediatek.com \
    --cc=weiyongjun1@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.