Re: Fix drm suspend and resume issue

Re: Fix drm suspend and resume issue

[Tzung-Bi Shih]

On Wed, Nov 17, 2021 at 02:41:55PM +0800, jason-jh.lin wrote:
> Subject: [PATCH 0/3] Fix drm suspend and resume issue

You have 2 Subjects.  The first one takes precedence.

Re: [PATCH 1/3] mialbox: move cmdq suspend,resume and remove after cmdq_mbox_flush

[Tzung-Bi Shih]

On Wed, Nov 17, 2021 at 02:41:56PM +0800, jason-jh.lin wrote:

Typo in the commit title "mialbox: move cmdq suspend,resume and remove after cmdq_mbox_flush".

s/mialbox/mailbox/

Re: [PATCH 3/3] drm/mediatek: add devlink to cmdq dev

[Tzung-Bi Shih]

On Wed, Nov 17, 2021 at 02:41:58PM +0800, jason-jh.lin wrote:
> @@ -158,6 +159,7 @@ static void mtk_drm_crtc_destroy(struct drm_crtc *crtc)
>  	mtk_drm_cmdq_pkt_destroy(&mtk_crtc->cmdq_handle);
>  
>  	if (mtk_crtc->cmdq_client.chan) {
> +		device_link_remove(mtk_crtc->drm_dev, mtk_crtc->cmdq_client.chan->mbox->dev);
>  		mbox_free_channel(mtk_crtc->cmdq_client.chan);
>  		mtk_crtc->cmdq_client.chan = NULL;
>  	}
[...]
> @@ -956,6 +959,16 @@ int mtk_drm_crtc_create(struct drm_device *drm_dev,
>  	}
>  
>  	if (mtk_crtc->cmdq_client.chan) {
> +		struct device_link *link;
> +
> +		/* add devlink to cmdq dev to make sure suspend/resume order is correct */
> +		link = device_link_add(dev, mtk_crtc->cmdq_client.chan->mbox->dev,
> +				       DL_FLAG_PM_RUNTIME | DL_FLAG_STATELESS);
> +		if (!link) {
> +			dev_err(dev, "Unable to link dev=%s\n",
> +				dev_name(mtk_crtc->cmdq_client.chan->mbox->dev));
> +		}
> +

If device_link_add() failed, doesn't mtk_drm_crtc_create() need to return an error and exit?

OTOH, if device_link_add() failed, won't it bring any side effects to call device_link_remove()?

Re: [PATCH 2/3] mailbox: add cmdq_mbox_flush to clear all task before suspend

[Chun-Kuang Hu]

Hi, Jason:

jason-jh.lin <jason-jh.lin@mediatek.com> 於 2021年11月17日 週三 下午2:42寫道：
>
> CMDQ driver will occupy GCE clock to execute the task in GCE thread.
>
> So call cmdq_mbox_flush to clear all task in GCE thread before
> CMDQ suspend.
>
> Signed-off-by: jason-jh.lin <jason-jh.lin@mediatek.com>
> ---
>  drivers/mailbox/mtk-cmdq-mailbox.c | 9 +++------
>  1 file changed, 3 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/mailbox/mtk-cmdq-mailbox.c b/drivers/mailbox/mtk-cmdq-mailbox.c
> index 03f9ed4c5131..28cadfc0091b 100644
> --- a/drivers/mailbox/mtk-cmdq-mailbox.c
> +++ b/drivers/mailbox/mtk-cmdq-mailbox.c
> @@ -484,21 +484,18 @@ static int cmdq_suspend(struct device *dev)
>         struct cmdq *cmdq = dev_get_drvdata(dev);
>         struct cmdq_thread *thread;
>         int i;
> -       bool task_running = false;
>
>         cmdq->suspended = true;
>
>         for (i = 0; i < cmdq->thread_nr; i++) {
>                 thread = &cmdq->thread[i];
>                 if (!list_empty(&thread->task_busy_list)) {
> -                       task_running = true;
> -                       break;
> +                       /* try to clear all task in this thread */
> +                       cmdq_mbox_flush(thread->chan, 2000);

I would like the normal control flow rather than error handling. So
the normal control flow is:

1. Client driver suspend: Flush command.
2. CMDQ driver suspend: There is no command to flush. If there are
command, show error message and debug the client driver.

The error handling flow:

1. Client driver suspend: Does not flush command.
2. CMDQ driver suspend: Flush command and callback to client driver.
Client driver process these callback as error handling.

The client driver may integrate multiple driver. In the suspend flow,
it may need to stop these driver in a sequence such as.

1. Stop driver 1
2. Stop driver 2
3. Stop driver 3 (cmdq)
4. Stop driver 4
5. Stop driver 5.

In the normal flow, client driver could control the stop flow. In the
error handling flow, it does not match the stop flow.

Regards,
Chun-Kuang.

> +                       dev_warn(dev, "thread[%d] exist running task(s) in suspend\n", i);
>                 }
>         }
>
> -       if (task_running)
> -               dev_warn(dev, "exist running task(s) in suspend\n");
> -
>         clk_bulk_unprepare(cmdq->gce_num, cmdq->clocks);
>
>         return 0;
> --
> 2.18.0
>

Re: [PATCH 1/3] mialbox: move cmdq suspend,resume and remove after cmdq_mbox_flush

[Jason-JH Lin]

Hi Tzung-Bi,

Thanks for the reviews.

On Thu, 2021-11-18 at 13:55 +0800, Tzung-Bi Shih wrote:
> On Wed, Nov 17, 2021 at 02:41:56PM +0800, jason-jh.lin wrote:
> 
> Typo in the commit title "mialbox: move cmdq suspend,resume and
> remove after cmdq_mbox_flush".
> 
> s/mialbox/mailbox/
-- 
I'll fix this typo at the next version.

Regards,
Jason-JH Lin <jason-jh.lin@mediatek.com>

Re: Fix drm suspend and resume issue

[Jason-JH Lin]

Hi Tzung-Bi,

Thanks, I'll fix it.

On Thu, 2021-11-18 at 13:54 +0800, Tzung-Bi Shih wrote:
> On Wed, Nov 17, 2021 at 02:41:55PM +0800, jason-jh.lin wrote:
> > Subject: [PATCH 0/3] Fix drm suspend and resume issue
> 
> You have 2 Subjects.  The first one takes precedence.
-- 
Regards,
Jason-JH Lin <jason-jh.lin@mediatek.com>

Re: [PATCH 2/3] mailbox: add cmdq_mbox_flush to clear all task before suspend

[Jason-JH Lin]

Hi Chun-Kuang,

Thanks for the reviews.

On Fri, 2021-11-19 at 08:01 +0800, Chun-Kuang Hu wrote:
> Hi, Jason:
> 
> jason-jh.lin <jason-jh.lin@mediatek.com> 於 2021年11月17日 週三 下午2:42寫道：
> > 
> > CMDQ driver will occupy GCE clock to execute the task in GCE
> > thread.
> > 
> > So call cmdq_mbox_flush to clear all task in GCE thread before
> > CMDQ suspend.
> > 
> > Signed-off-by: jason-jh.lin <jason-jh.lin@mediatek.com>
> > ---
> >  drivers/mailbox/mtk-cmdq-mailbox.c | 9 +++------
> >  1 file changed, 3 insertions(+), 6 deletions(-)
> > 
> > diff --git a/drivers/mailbox/mtk-cmdq-mailbox.c
> > b/drivers/mailbox/mtk-cmdq-mailbox.c
> > index 03f9ed4c5131..28cadfc0091b 100644
> > --- a/drivers/mailbox/mtk-cmdq-mailbox.c
> > +++ b/drivers/mailbox/mtk-cmdq-mailbox.c
> > @@ -484,21 +484,18 @@ static int cmdq_suspend(struct device *dev)
> >         struct cmdq *cmdq = dev_get_drvdata(dev);
> >         struct cmdq_thread *thread;
> >         int i;
> > -       bool task_running = false;
> > 
> >         cmdq->suspended = true;
> > 
> >         for (i = 0; i < cmdq->thread_nr; i++) {
> >                 thread = &cmdq->thread[i];
> >                 if (!list_empty(&thread->task_busy_list)) {
> > -                       task_running = true;
> > -                       break;
> > +                       /* try to clear all task in this thread */
> > +                       cmdq_mbox_flush(thread->chan, 2000);
> 
> I would like the normal control flow rather than error handling. So
> the normal control flow is:
> 
> 1. Client driver suspend: Flush command.
> 2. CMDQ driver suspend: There is no command to flush. If there are
> command, show error message and debug the client driver.
> 
> The error handling flow:
> 
> 1. Client driver suspend: Does not flush command.
> 2. CMDQ driver suspend: Flush command and callback to client driver.
> Client driver process these callback as error handling.
> 
> The client driver may integrate multiple driver. In the suspend flow,
> it may need to stop these driver in a sequence such as.
> 
> 1. Stop driver 1
> 2. Stop driver 2
> 3. Stop driver 3 (cmdq)
> 4. Stop driver 4
> 5. Stop driver 5.
> 
> In the normal flow, client driver could control the stop flow. In the
> error handling flow, it does not match the stop flow.
> 
> Regards,
> Chun-Kuang.
> 

I have tried the normal flow:
1. Client driver suspend: Flush command.
2. CMDQ driver suspend: There is no command to flush. If there are
command, show error message and debug the client driver.

Then I found the cmdq task of crtc_1 is still executing when
cmdq_suspend sometimes. I think it is the latest async cmd sent in
mtk_drm_crtc_disable.
So I'll try to change it to blocking cmd and see the issue is still
happened or not.

If it works, then I'll revert this patch and send add blocking cmd in
mtk_drm_crtc_disable at the next version.

Regards,
Jason-JH.Lin

> > +                       dev_warn(dev, "thread[%d] exist running
> > task(s) in suspend\n", i);
> >                 }
> >         }
> > 
> > -       if (task_running)
> > -               dev_warn(dev, "exist running task(s) in
> > suspend\n");
> > -
> >         clk_bulk_unprepare(cmdq->gce_num, cmdq->clocks);
> > 
> >         return 0;
> > --
> > 2.18.0
> > 
-- 
Jason-JH Lin <jason-jh.lin@mediatek.com>

Re: FW: [PATCH 3/3] drm/mediatek: add devlink to cmdq dev

[Jason-JH Lin]

Hi Tzung-Bi,

Thanks for the reviews.

> From: Linux-mediatek <linux-mediatek-bounces@lists.infradead.org> On
> Behalf Of Tzung-Bi Shih
> Sent: Thursday, November 18, 2021 1:55 PM
> To: Jason-JH Lin (林睿祥) <Jason-JH.Lin@mediatek.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>; Philipp Zabel <
> p.zabel@pengutronix.de>; Matthias Brugger <matthias.bgg@gmail.com>;
> Jassi Brar <jassisinghbrar@gmail.com>; David Airlie <airlied@linux.ie
> >; Daniel Vetter <daniel@ffwll.ch>; dri-devel@lists.freedesktop.org; 
> linux-mediatek@lists.infradead.org; 
> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; 
> hsinyi@chromium.org; fshao@chromium.org; Nancy Lin (林欣螢) <
> Nancy.Lin@mediatek.com>; Singo Chang (張興國) <Singo.Chang@mediatek.com>
> Subject: Re: [PATCH 3/3] drm/mediatek: add devlink to cmdq dev
> 
> On Wed, Nov 17, 2021 at 02:41:58PM +0800, jason-jh.lin wrote:
> > @@ -158,6 +159,7 @@ static void mtk_drm_crtc_destroy(struct
> > drm_crtc *crtc)
> >  	mtk_drm_cmdq_pkt_destroy(&mtk_crtc->cmdq_handle);
> >  
> >  	if (mtk_crtc->cmdq_client.chan) {
> > +		device_link_remove(mtk_crtc->drm_dev, mtk_crtc-
> > >cmdq_client.chan->mbox->dev);
> >  		mbox_free_channel(mtk_crtc->cmdq_client.chan);
> >  		mtk_crtc->cmdq_client.chan = NULL;
> >  	}
> 
> [...]
> > @@ -956,6 +959,16 @@ int mtk_drm_crtc_create(struct drm_device
> > *drm_dev,
> >  	}
> >  
> >  	if (mtk_crtc->cmdq_client.chan) {
> > +		struct device_link *link;
> > +
> > +		/* add devlink to cmdq dev to make sure suspend/resume
> > order is correct */
> > +		link = device_link_add(dev, mtk_crtc->cmdq_client.chan-
> > >mbox->dev,
> > +				       DL_FLAG_PM_RUNTIME |
> > DL_FLAG_STATELESS);
> > +		if (!link) {
> > +			dev_err(dev, "Unable to link dev=%s\n",
> > +				dev_name(mtk_crtc->cmdq_client.chan-
> > >mbox->dev));
> > +		}
> > +
> 
> If device_link_add() failed, doesn't mtk_drm_crtc_create() need to
> return an error and exit?

OK, I'll add the return error at the next verion.
> 
> OTOH, if device_link_add() failed, won't it bring any side effects to
> call device_link_remove()?
> 

Because device_link_remove() will find the device_link of supplier and
consumer, then delete the device_link between them.
If device_link_add() failed, supplier and consumer won't create the
device_link.
So calling device_link_remove() won't do anything without deive_link
and won't bring any side effects.

Regards,
Jason-JH.Lin
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> 
https://urldefense.com/v3/__http://lists.infradead.org/mailman/listinfo/linux-mediatek__;!!CTRNKA9wMg0ARbw!125dmOZ7xCLkwbIrzZf1VqVLQBbdT4RFHoMjR38U1GeCxaTM6xvvB-mSr7sTc6BuY5IH$
>  
-- 
Jason-JH Lin <jason-jh.lin@mediatek.com>

Re: FW: [PATCH 3/3] drm/mediatek: add devlink to cmdq dev

[Jason-JH Lin]

Hi Tzung-Bi,

Thanks for the reviews.

> From: Linux-mediatek <linux-mediatek-bounces@lists.infradead.org> On
> Behalf Of Tzung-Bi Shih
> Sent: Thursday, November 18, 2021 1:55 PM
> To: Jason-JH Lin (林睿祥) <Jason-JH.Lin@mediatek.com>
> Cc: Chun-Kuang Hu <chunkuang.hu@kernel.org>; Philipp Zabel <
> p.zabel@pengutronix.de>; Matthias Brugger <matthias.bgg@gmail.com>;
> Jassi Brar <jassisinghbrar@gmail.com>; David Airlie <airlied@linux.ie
> >; Daniel Vetter <daniel@ffwll.ch>; dri-devel@lists.freedesktop.org; 
> linux-mediatek@lists.infradead.org; 
> linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; 
> hsinyi@chromium.org; fshao@chromium.org; Nancy Lin (林欣螢) <
> Nancy.Lin@mediatek.com>; Singo Chang (張興國) <Singo.Chang@mediatek.com>
> Subject: Re: [PATCH 3/3] drm/mediatek: add devlink to cmdq dev
> 
> On Wed, Nov 17, 2021 at 02:41:58PM +0800, jason-jh.lin wrote:
> > @@ -158,6 +159,7 @@ static void mtk_drm_crtc_destroy(struct
> > drm_crtc *crtc)
> >  	mtk_drm_cmdq_pkt_destroy(&mtk_crtc->cmdq_handle);
> >  
> >  	if (mtk_crtc->cmdq_client.chan) {
> > +		device_link_remove(mtk_crtc->drm_dev, mtk_crtc-
> > >cmdq_client.chan->mbox->dev);
> >  		mbox_free_channel(mtk_crtc->cmdq_client.chan);
> >  		mtk_crtc->cmdq_client.chan = NULL;
> >  	}
> 
> [...]
> > @@ -956,6 +959,16 @@ int mtk_drm_crtc_create(struct drm_device
> > *drm_dev,
> >  	}
> >  
> >  	if (mtk_crtc->cmdq_client.chan) {
> > +		struct device_link *link;
> > +
> > +		/* add devlink to cmdq dev to make sure suspend/resume
> > order is correct */
> > +		link = device_link_add(dev, mtk_crtc->cmdq_client.chan-
> > >mbox->dev,
> > +				       DL_FLAG_PM_RUNTIME |
> > DL_FLAG_STATELESS);
> > +		if (!link) {
> > +			dev_err(dev, "Unable to link dev=%s\n",
> > +				dev_name(mtk_crtc->cmdq_client.chan-
> > >mbox->dev));
> > +		}
> > +
> 
> If device_link_add() failed, doesn't mtk_drm_crtc_create() need to
> return an error and exit?

OK, I'll add the return error at the next verion.
> 
> OTOH, if device_link_add() failed, won't it bring any side effects to
> call device_link_remove()?
> 

Because device_link_remove() will find the device_link of supplier and
consumer, then delete the device_link between them.
If device_link_add() failed, supplier and consumer won't create the
device_link.
So calling device_link_remove() won't do anything without deive_link
and won't bring any side effects.

Regards,
Jason-JH.Lin
> _______________________________________________
> Linux-mediatek mailing list
> Linux-mediatek@lists.infradead.org
> 
https://urldefense.com/v3/__http://lists.infradead.org/mailman/listinfo/linux-mediatek__;!!CTRNKA9wMg0ARbw!125dmOZ7xCLkwbIrzZf1VqVLQBbdT4RFHoMjR38U1GeCxaTM6xvvB-mSr7sTc6BuY5IH$
>  
-- 
Jason-JH Lin <jason-jh.lin@mediatek.com>