linux-media.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
@ 2021-09-20 17:04 Dafna Hirschfeld
  2021-10-04  9:37 ` Hans Verkuil
  2022-02-15  9:41 ` AngeloGioacchino Del Regno
  0 siblings, 2 replies; 13+ messages in thread
From: Dafna Hirschfeld @ 2021-09-20 17:04 UTC (permalink / raw)
  To: linux-media, linux-mediatek, linux-kernel
  Cc: Alexandre Courbot, dafna.hirschfeld, kernel,
	Enric Balletbo i Serra, dafna3, tiffany.lin, andrew-ct.chen,
	minghsiu.tsai, houlong.wei, mchehab, matthias.bgg, hverkuil

From: Alexandre Courbot <acourbot@chromium.org>

When running memcpy_toio:
memcpy_toio(send_obj->share_buf, buf, len);
it was found that errors appear if len is not a multiple of 8:

[58.350841] mtk-mdp 14001000.rdma: processing failed: -22

This patch ensures the copy of a multiple of 8 size by calling
round_up(len, 8) when copying

Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to DTCM buffer.")
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
---
changes since v3:
1. multile -> multiple
2. add inline doc

changes since v2:
1. do the extra copy only if len is not multiple of 8

changes since v1:
1. change sign-off-by tags
2. change values to memset

 drivers/media/platform/mtk-vpu/mtk_vpu.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
index ec290dde59cf..1df031716c8f 100644
--- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
+++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
@@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device *pdev,
 		}
 	} while (vpu_cfg_readl(vpu, HOST_TO_VPU));
 
-	memcpy_toio(send_obj->share_buf, buf, len);
+	/*
+	 * when copying data to the vpu hardware, the memcpy_toio operation must copy
+	 * a multiple of 8. Otherwise the processing fails
+	 */
+	if (len % 8 != 0) {
+		unsigned char data[SHARE_BUF_SIZE];
+
+		memset(data + len, 0, sizeof(data) - len);
+		memcpy(data, buf, len);
+		memcpy_toio(send_obj->share_buf, data, round_up(len, 8));
+	} else {
+		memcpy_toio(send_obj->share_buf, buf, len);
+	}
+
 	writel(len, &send_obj->len);
 	writel(id, &send_obj->id);
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-09-20 17:04 [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer Dafna Hirschfeld
@ 2021-10-04  9:37 ` Hans Verkuil
  2021-10-18  1:16   ` Alexandre Courbot
  2022-02-15  9:41 ` AngeloGioacchino Del Regno
  1 sibling, 1 reply; 13+ messages in thread
From: Hans Verkuil @ 2021-10-04  9:37 UTC (permalink / raw)
  To: Dafna Hirschfeld, linux-media, linux-mediatek, linux-kernel
  Cc: Alexandre Courbot, kernel, Enric Balletbo i Serra, dafna3,
	tiffany.lin, andrew-ct.chen, minghsiu.tsai, houlong.wei, mchehab,
	matthias.bgg

On 20/09/2021 19:04, Dafna Hirschfeld wrote:
> From: Alexandre Courbot <acourbot@chromium.org>
> 
> When running memcpy_toio:
> memcpy_toio(send_obj->share_buf, buf, len);
> it was found that errors appear if len is not a multiple of 8:
> 
> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22

Why do errors appear? Is that due to a HW bug? Some other reason?

> 
> This patch ensures the copy of a multiple of 8 size by calling
> round_up(len, 8) when copying
> 
> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to DTCM buffer.")
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
> Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
> ---
> changes since v3:
> 1. multile -> multiple
> 2. add inline doc
> 
> changes since v2:
> 1. do the extra copy only if len is not multiple of 8
> 
> changes since v1:
> 1. change sign-off-by tags
> 2. change values to memset
> 
>  drivers/media/platform/mtk-vpu/mtk_vpu.c | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> index ec290dde59cf..1df031716c8f 100644
> --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device *pdev,
>  		}
>  	} while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>  
> -	memcpy_toio(send_obj->share_buf, buf, len);
> +	/*
> +	 * when copying data to the vpu hardware, the memcpy_toio operation must copy
> +	 * a multiple of 8. Otherwise the processing fails

Same here: it needs to explain why the processing fails.

> +	 */
> +	if (len % 8 != 0) {
> +		unsigned char data[SHARE_BUF_SIZE];

Wouldn't it be more robust if you say:

		unsigned char data[sizeof(send_obj->share_buf)];

I also think that the SHARE_BUF_SIZE define needs a comment stating that it must be a
multiple of 8, otherwise unexpected things can happen.

You also noticed that the current SHARE_BUF_SIZE define is too low, but I saw
no patch correcting this. Shouldn't that be fixed as well?

Regards,

	Hans

> +
> +		memset(data + len, 0, sizeof(data) - len);
> +		memcpy(data, buf, len);
> +		memcpy_toio(send_obj->share_buf, data, round_up(len, 8));
> +	} else {
> +		memcpy_toio(send_obj->share_buf, buf, len);
> +	}
> +
>  	writel(len, &send_obj->len);
>  	writel(id, &send_obj->id);
>  
> 


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-10-04  9:37 ` Hans Verkuil
@ 2021-10-18  1:16   ` Alexandre Courbot
  2021-10-18  7:07     ` Dafna Hirschfeld
  2021-11-29 14:39     ` Dafna Hirschfeld
  0 siblings, 2 replies; 13+ messages in thread
From: Alexandre Courbot @ 2021-10-18  1:16 UTC (permalink / raw)
  To: Hans Verkuil
  Cc: Dafna Hirschfeld, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Enric Balletbo i Serra, Dafna Hirschfeld, Tiffany Lin,
	Andrew-CT Chen, minghsiu.tsai, houlong.wei,
	Mauro Carvalho Chehab, Matthias Brugger

Hi Hans!

On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 20/09/2021 19:04, Dafna Hirschfeld wrote:
> > From: Alexandre Courbot <acourbot@chromium.org>
> >
> > When running memcpy_toio:
> > memcpy_toio(send_obj->share_buf, buf, len);
> > it was found that errors appear if len is not a multiple of 8:
> >
> > [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
>
> Why do errors appear? Is that due to a HW bug? Some other reason?

MTK folks would be the best placed to answer this, but since the
failure is reported by the firmware I'd suspect either a firmware or
hardware limitation.

>
> >
> > This patch ensures the copy of a multiple of 8 size by calling
> > round_up(len, 8) when copying
> >
> > Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to DTCM buffer.")
> > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
> > Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
> > Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
> > ---
> > changes since v3:
> > 1. multile -> multiple
> > 2. add inline doc
> >
> > changes since v2:
> > 1. do the extra copy only if len is not multiple of 8
> >
> > changes since v1:
> > 1. change sign-off-by tags
> > 2. change values to memset
> >
> >  drivers/media/platform/mtk-vpu/mtk_vpu.c | 15 ++++++++++++++-
> >  1 file changed, 14 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > index ec290dde59cf..1df031716c8f 100644
> > --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device *pdev,
> >               }
> >       } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >
> > -     memcpy_toio(send_obj->share_buf, buf, len);
> > +     /*
> > +      * when copying data to the vpu hardware, the memcpy_toio operation must copy
> > +      * a multiple of 8. Otherwise the processing fails
>
> Same here: it needs to explain why the processing fails.
>
> > +      */
> > +     if (len % 8 != 0) {
> > +             unsigned char data[SHARE_BUF_SIZE];
>
> Wouldn't it be more robust if you say:
>
>                 unsigned char data[sizeof(send_obj->share_buf)];

Definitely yes.

>
> I also think that the SHARE_BUF_SIZE define needs a comment stating that it must be a
> multiple of 8, otherwise unexpected things can happen.
>
> You also noticed that the current SHARE_BUF_SIZE define is too low, but I saw
> no patch correcting this. Shouldn't that be fixed as well?

AFAICT the firmware expects this exact size on its end, so I don't
believe it can be changed that easily. But maybe someone from MTK can
prove me wrong.

Cheers,
Alex.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-10-18  1:16   ` Alexandre Courbot
@ 2021-10-18  7:07     ` Dafna Hirschfeld
  2021-11-03  8:03       ` houlong wei
  2021-11-29 14:39     ` Dafna Hirschfeld
  1 sibling, 1 reply; 13+ messages in thread
From: Dafna Hirschfeld @ 2021-10-18  7:07 UTC (permalink / raw)
  To: Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Enric Balletbo i Serra, Dafna Hirschfeld, Tiffany Lin,
	Andrew-CT Chen, minghsiu.tsai, houlong.wei,
	Mauro Carvalho Chehab, Matthias Brugger



On 18.10.21 03:16, Alexandre Courbot wrote:
> Hi Hans!
> 
> On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 20/09/2021 19:04, Dafna Hirschfeld wrote:
>>> From: Alexandre Courbot <acourbot@chromium.org>
>>>
>>> When running memcpy_toio:
>>> memcpy_toio(send_obj->share_buf, buf, len);
>>> it was found that errors appear if len is not a multiple of 8:
>>>
>>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
>>
>> Why do errors appear? Is that due to a HW bug? Some other reason?
> 
> MTK folks would be the best placed to answer this, but since the
> failure is reported by the firmware I'd suspect either a firmware or
> hardware limitation.
> 
>>
>>>
>>> This patch ensures the copy of a multiple of 8 size by calling
>>> round_up(len, 8) when copying
>>>
>>> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to DTCM buffer.")
>>> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
>>> Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
>>> Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
>>> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
>>> ---
>>> changes since v3:
>>> 1. multile -> multiple
>>> 2. add inline doc
>>>
>>> changes since v2:
>>> 1. do the extra copy only if len is not multiple of 8
>>>
>>> changes since v1:
>>> 1. change sign-off-by tags
>>> 2. change values to memset
>>>
>>>   drivers/media/platform/mtk-vpu/mtk_vpu.c | 15 ++++++++++++++-
>>>   1 file changed, 14 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>> index ec290dde59cf..1df031716c8f 100644
>>> --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>> @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device *pdev,
>>>                }
>>>        } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>>>
>>> -     memcpy_toio(send_obj->share_buf, buf, len);
>>> +     /*
>>> +      * when copying data to the vpu hardware, the memcpy_toio operation must copy
>>> +      * a multiple of 8. Otherwise the processing fails
>>
>> Same here: it needs to explain why the processing fails.

Is writing 'due to hardware or firmware limitation' enough?
If not, then we should wait for mediatek people's response to explain if they know more

>>
>>> +      */
>>> +     if (len % 8 != 0) {
>>> +             unsigned char data[SHARE_BUF_SIZE];
>>
>> Wouldn't it be more robust if you say:
>>
>>                  unsigned char data[sizeof(send_obj->share_buf)];
> 
> Definitely yes.

I'll send v5 fixing this

> 
>>
>> I also think that the SHARE_BUF_SIZE define needs a comment stating that it must be a
>> multiple of 8, otherwise unexpected things can happen.
>>
>> You also noticed that the current SHARE_BUF_SIZE define is too low, but I saw
>> no patch correcting this. Shouldn't that be fixed as well?
> 
> AFAICT the firmware expects this exact size on its end, so I don't
> believe it can be changed that easily. But maybe someone from MTK can
> prove me wrong.
> 

I looked further and noted that the structs that are larger than 'SHARE_BUF_SIZE'
(venc_ap_ipi_msg_enc_ext venc_ap_ipi_msg_set_param_ext)
are used by drivers that don't use this vpu api, so actually SHARE_BUF_SIZE is
not too low and as Corurbot worte probably not changeable.


Thanks,
Dafna

> Cheers,
> Alex.
> 

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-10-18  7:07     ` Dafna Hirschfeld
@ 2021-11-03  8:03       ` houlong wei
  2021-11-03  9:19         ` Irui Wang
  0 siblings, 1 reply; 13+ messages in thread
From: houlong wei @ 2021-11-03  8:03 UTC (permalink / raw)
  To: Irui.wang, Dafna Hirschfeld, Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Enric Balletbo i Serra, Dafna Hirschfeld,
	Tiffany Lin (林慧珊),
	Andrew-CT Chen (陳智迪),
	Minghsiu Tsai (蔡明修),
	Mauro Carvalho Chehab, Matthias Brugger, houlong.wei

Add mtk-vpu driver expert irui.wang in the loop.

On Mon, 2021-10-18 at 15:07 +0800, Dafna Hirschfeld wrote:
> 
> On 18.10.21 03:16, Alexandre Courbot wrote:
> > Hi Hans!
> > 
> > On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl>
> > wrote:
> > > 
> > > On 20/09/2021 19:04, Dafna Hirschfeld wrote:
> > > > From: Alexandre Courbot <acourbot@chromium.org>
> > > > 
> > > > When running memcpy_toio:
> > > > memcpy_toio(send_obj->share_buf, buf, len);
> > > > it was found that errors appear if len is not a multiple of 8:
> > > > 
> > > > [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
> > > 
> > > Why do errors appear? Is that due to a HW bug? Some other reason?
> > 
> > MTK folks would be the best placed to answer this, but since the
> > failure is reported by the firmware I'd suspect either a firmware
> > or
> > hardware limitation.
> > 
> > > 
> > > > 
> > > > This patch ensures the copy of a multiple of 8 size by calling
> > > > round_up(len, 8) when copying
> > > > 
> > > > Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to
> > > > DTCM buffer.")
> > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > > Signed-off-by: Enric Balletbo i Serra <
> > > > enric.balletbo@collabora.com>
> > > > Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com
> > > > >
> > > > Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
> > > > ---
> > > > changes since v3:
> > > > 1. multile -> multiple
> > > > 2. add inline doc
> > > > 
> > > > changes since v2:
> > > > 1. do the extra copy only if len is not multiple of 8
> > > > 
> > > > changes since v1:
> > > > 1. change sign-off-by tags
> > > > 2. change values to memset
> > > > 
> > > >   drivers/media/platform/mtk-vpu/mtk_vpu.c | 15 ++++++++++++++-
> > > >   1 file changed, 14 insertions(+), 1 deletion(-)
> > > > 
> > > > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > index ec290dde59cf..1df031716c8f 100644
> > > > --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device
> > > > *pdev,
> > > >                }
> > > >        } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> > > > 
> > > > -     memcpy_toio(send_obj->share_buf, buf, len);
> > > > +     /*
> > > > +      * when copying data to the vpu hardware, the memcpy_toio
> > > > operation must copy
> > > > +      * a multiple of 8. Otherwise the processing fails
> > > 
> > > Same here: it needs to explain why the processing fails.
> 
> Is writing 'due to hardware or firmware limitation' enough?
> If not, then we should wait for mediatek people's response to explain
> if they know more
> 
> > > 
> > > > +      */
> > > > +     if (len % 8 != 0) {
> > > > +             unsigned char data[SHARE_BUF_SIZE];
> > > 
> > > Wouldn't it be more robust if you say:
> > > 
> > >                  unsigned char data[sizeof(send_obj->share_buf)];
> > 
> > Definitely yes.
> 
> I'll send v5 fixing this
> 
> > 
> > > 
> > > I also think that the SHARE_BUF_SIZE define needs a comment
> > > stating that it must be a
> > > multiple of 8, otherwise unexpected things can happen.
> > > 
> > > You also noticed that the current SHARE_BUF_SIZE define is too
> > > low, but I saw
> > > no patch correcting this. Shouldn't that be fixed as well?
> > 
> > AFAICT the firmware expects this exact size on its end, so I don't
> > believe it can be changed that easily. But maybe someone from MTK
> > can
> > prove me wrong.
> > 
> 
> I looked further and noted that the structs that are larger than
> 'SHARE_BUF_SIZE'
> (venc_ap_ipi_msg_enc_ext venc_ap_ipi_msg_set_param_ext)
> are used by drivers that don't use this vpu api, so actually
> SHARE_BUF_SIZE is
> not too low and as Corurbot worte probably not changeable.
> 
> 
> Thanks,
> Dafna
> 
> > Cheers,
> > Alex.
> > 

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-11-03  8:03       ` houlong wei
@ 2021-11-03  9:19         ` Irui Wang
  2021-11-03 11:04           ` Dafna Hirschfeld
  0 siblings, 1 reply; 13+ messages in thread
From: Irui Wang @ 2021-11-03  9:19 UTC (permalink / raw)
  To: houlong wei, Dafna Hirschfeld, Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Enric Balletbo i Serra, Dafna Hirschfeld,
	Tiffany Lin (林慧珊),
	Andrew-CT Chen (陳智迪),
	Minghsiu Tsai (蔡明修),
	Mauro Carvalho Chehab, Matthias Brugger

Hi,

The "len" of share_buf copied should be always 8 alignment;
do you have other logs to prove the len is not 8 alignment when errors
appear? 
>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22

On Wed, 2021-11-03 at 16:03 +0800, houlong wei wrote:
> Add mtk-vpu driver expert irui.wang in the loop.
> 
> On Mon, 2021-10-18 at 15:07 +0800, Dafna Hirschfeld wrote:
> > 
> > On 18.10.21 03:16, Alexandre Courbot wrote:
> > > Hi Hans!
> > > 
> > > On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl>
> > > wrote:
> > > > 
> > > > On 20/09/2021 19:04, Dafna Hirschfeld wrote:
> > > > > From: Alexandre Courbot <acourbot@chromium.org>
> > > > > 
> > > > > When running memcpy_toio:
> > > > > memcpy_toio(send_obj->share_buf, buf, len);
> > > > > it was found that errors appear if len is not a multiple of
> > > > > 8:
> > > > > 
> > > > > [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
> > > > 
> > > > Why do errors appear? Is that due to a HW bug? Some other
> > > > reason?
> > > 
> > > MTK folks would be the best placed to answer this, but since the
> > > failure is reported by the firmware I'd suspect either a firmware
> > > or
> > > hardware limitation.
> > > 
> > > > 
> > > > > 
> > > > > This patch ensures the copy of a multiple of 8 size by
> > > > > calling
> > > > > round_up(len, 8) when copying
> > > > > 
> > > > > Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access
> > > > > to
> > > > > DTCM buffer.")
> > > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> > > > > Signed-off-by: Enric Balletbo i Serra <
> > > > > enric.balletbo@collabora.com>
> > > > > Signed-off-by: Dafna Hirschfeld <
> > > > > dafna.hirschfeld@collabora.com
> > > > > > 
> > > > > 
> > > > > Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
> > > > > ---
> > > > > changes since v3:
> > > > > 1. multile -> multiple
> > > > > 2. add inline doc
> > > > > 
> > > > > changes since v2:
> > > > > 1. do the extra copy only if len is not multiple of 8
> > > > > 
> > > > > changes since v1:
> > > > > 1. change sign-off-by tags
> > > > > 2. change values to memset
> > > > > 
> > > > >   drivers/media/platform/mtk-vpu/mtk_vpu.c | 15
> > > > > ++++++++++++++-
> > > > >   1 file changed, 14 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > index ec290dde59cf..1df031716c8f 100644
> > > > > --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device
> > > > > *pdev,
> > > > >                }
> > > > >        } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> > > > > 
> > > > > -     memcpy_toio(send_obj->share_buf, buf, len);
> > > > > +     /*
> > > > > +      * when copying data to the vpu hardware, the
> > > > > memcpy_toio
> > > > > operation must copy
> > > > > +      * a multiple of 8. Otherwise the processing fails
> > > > 
> > > > Same here: it needs to explain why the processing fails.
> > 
> > Is writing 'due to hardware or firmware limitation' enough?
> > If not, then we should wait for mediatek people's response to
> > explain
> > if they know more
> > 
> > > > 
> > > > > +      */
> > > > > +     if (len % 8 != 0) {
> > > > > +             unsigned char data[SHARE_BUF_SIZE];
> > > > 
> > > > Wouldn't it be more robust if you say:
> > > > 
> > > >                  unsigned char data[sizeof(send_obj-
> > > > >share_buf)];
> > > 
> > > Definitely yes.
> > 
> > I'll send v5 fixing this
> > 
> > > 
> > > > 
> > > > I also think that the SHARE_BUF_SIZE define needs a comment
> > > > stating that it must be a
> > > > multiple of 8, otherwise unexpected things can happen.
> > > > 
> > > > You also noticed that the current SHARE_BUF_SIZE define is too
> > > > low, but I saw
> > > > no patch correcting this. Shouldn't that be fixed as well?
> > > 
> > > AFAICT the firmware expects this exact size on its end, so I
> > > don't
> > > believe it can be changed that easily. But maybe someone from MTK
> > > can
> > > prove me wrong.
> > > 
> > 
> > I looked further and noted that the structs that are larger than
> > 'SHARE_BUF_SIZE'
> > (venc_ap_ipi_msg_enc_ext venc_ap_ipi_msg_set_param_ext)
> > are used by drivers that don't use this vpu api, so actually
> > SHARE_BUF_SIZE is
> > not too low and as Corurbot worte probably not changeable.
> > 
> > 
> > Thanks,
> > Dafna
> > 
> > > Cheers,
> > > Alex.
> > > 
> 
> 

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-11-03  9:19         ` Irui Wang
@ 2021-11-03 11:04           ` Dafna Hirschfeld
  2021-11-09  8:46             ` Dafna Hirschfeld
  0 siblings, 1 reply; 13+ messages in thread
From: Dafna Hirschfeld @ 2021-11-03 11:04 UTC (permalink / raw)
  To: Irui Wang, houlong wei, Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Dafna Hirschfeld, Tiffany Lin (林慧珊),
	Andrew-CT Chen (陳智迪),
	Minghsiu Tsai (蔡明修),
	Mauro Carvalho Chehab, Matthias Brugger



On 03.11.21 10:19, Irui Wang wrote:
> Hi,
> 
> The "len" of share_buf copied should be always 8 alignment;
> do you have other logs to prove the len is not 8 alignment when errors
> appear?

Hi, I found out that "sizeof(mdp_ipi_comm) = 20"
this is due to the macro #pragma pack(push, 4) in mtk_mdp_ipi.h

see [1]

[1] http://lkml.iu.edu/hypermail/linux/kernel/2109.2/04978.html

Thanks,
Dafna

>>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
> 
> On Wed, 2021-11-03 at 16:03 +0800, houlong wei wrote:
>> Add mtk-vpu driver expert irui.wang in the loop.
>>
>> On Mon, 2021-10-18 at 15:07 +0800, Dafna Hirschfeld wrote:
>>>
>>> On 18.10.21 03:16, Alexandre Courbot wrote:
>>>> Hi Hans!
>>>>
>>>> On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl>
>>>> wrote:
>>>>>
>>>>> On 20/09/2021 19:04, Dafna Hirschfeld wrote:
>>>>>> From: Alexandre Courbot <acourbot@chromium.org>
>>>>>>
>>>>>> When running memcpy_toio:
>>>>>> memcpy_toio(send_obj->share_buf, buf, len);
>>>>>> it was found that errors appear if len is not a multiple of
>>>>>> 8:
>>>>>>
>>>>>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
>>>>>
>>>>> Why do errors appear? Is that due to a HW bug? Some other
>>>>> reason?
>>>>
>>>> MTK folks would be the best placed to answer this, but since the
>>>> failure is reported by the firmware I'd suspect either a firmware
>>>> or
>>>> hardware limitation.
>>>>
>>>>>
>>>>>>
>>>>>> This patch ensures the copy of a multiple of 8 size by
>>>>>> calling
>>>>>> round_up(len, 8) when copying
>>>>>>
>>>>>> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access
>>>>>> to
>>>>>> DTCM buffer.")
>>>>>> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
>>>>>> Signed-off-by: Enric Balletbo i Serra <
>>>>>> enric.balletbo@collabora.com>
>>>>>> Signed-off-by: Dafna Hirschfeld <
>>>>>> dafna.hirschfeld@collabora.com
>>>>>>>
>>>>>>
>>>>>> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
>>>>>> ---
>>>>>> changes since v3:
>>>>>> 1. multile -> multiple
>>>>>> 2. add inline doc
>>>>>>
>>>>>> changes since v2:
>>>>>> 1. do the extra copy only if len is not multiple of 8
>>>>>>
>>>>>> changes since v1:
>>>>>> 1. change sign-off-by tags
>>>>>> 2. change values to memset
>>>>>>
>>>>>>    drivers/media/platform/mtk-vpu/mtk_vpu.c | 15
>>>>>> ++++++++++++++-
>>>>>>    1 file changed, 14 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>> b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>> index ec290dde59cf..1df031716c8f 100644
>>>>>> --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>> @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device
>>>>>> *pdev,
>>>>>>                 }
>>>>>>         } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>>>>>>
>>>>>> -     memcpy_toio(send_obj->share_buf, buf, len);
>>>>>> +     /*
>>>>>> +      * when copying data to the vpu hardware, the
>>>>>> memcpy_toio
>>>>>> operation must copy
>>>>>> +      * a multiple of 8. Otherwise the processing fails
>>>>>
>>>>> Same here: it needs to explain why the processing fails.
>>>
>>> Is writing 'due to hardware or firmware limitation' enough?
>>> If not, then we should wait for mediatek people's response to
>>> explain
>>> if they know more
>>>
>>>>>
>>>>>> +      */
>>>>>> +     if (len % 8 != 0) {
>>>>>> +             unsigned char data[SHARE_BUF_SIZE];
>>>>>
>>>>> Wouldn't it be more robust if you say:
>>>>>
>>>>>                   unsigned char data[sizeof(send_obj-
>>>>>> share_buf)];
>>>>
>>>> Definitely yes.
>>>
>>> I'll send v5 fixing this
>>>
>>>>
>>>>>
>>>>> I also think that the SHARE_BUF_SIZE define needs a comment
>>>>> stating that it must be a
>>>>> multiple of 8, otherwise unexpected things can happen.
>>>>>
>>>>> You also noticed that the current SHARE_BUF_SIZE define is too
>>>>> low, but I saw
>>>>> no patch correcting this. Shouldn't that be fixed as well?
>>>>
>>>> AFAICT the firmware expects this exact size on its end, so I
>>>> don't
>>>> believe it can be changed that easily. But maybe someone from MTK
>>>> can
>>>> prove me wrong.
>>>>
>>>
>>> I looked further and noted that the structs that are larger than
>>> 'SHARE_BUF_SIZE'
>>> (venc_ap_ipi_msg_enc_ext venc_ap_ipi_msg_set_param_ext)
>>> are used by drivers that don't use this vpu api, so actually
>>> SHARE_BUF_SIZE is
>>> not too low and as Corurbot worte probably not changeable.
>>>
>>>
>>> Thanks,
>>> Dafna
>>>
>>>> Cheers,
>>>> Alex.
>>>>
>>
>>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-11-03 11:04           ` Dafna Hirschfeld
@ 2021-11-09  8:46             ` Dafna Hirschfeld
  2021-11-10  3:11               ` Irui Wang
  0 siblings, 1 reply; 13+ messages in thread
From: Dafna Hirschfeld @ 2021-11-09  8:46 UTC (permalink / raw)
  To: Irui Wang, houlong wei, Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Dafna Hirschfeld, Tiffany Lin (林慧珊),
	Andrew-CT Chen (陳智迪),
	Minghsiu Tsai (蔡明修),
	Mauro Carvalho Chehab, Matthias Brugger



On 03.11.21 13:04, Dafna Hirschfeld wrote:
> 
> 
> On 03.11.21 10:19, Irui Wang wrote:
>> Hi,
>>
>> The "len" of share_buf copied should be always 8 alignment;
>> do you have other logs to prove the len is not 8 alignment when errors
>> appear?
> 
> Hi, I found out that "sizeof(mdp_ipi_comm) = 20"
> this is due to the macro #pragma pack(push, 4) in mtk_mdp_ipi.h
> 
> see [1]
> 
> [1] http://lkml.iu.edu/hypermail/linux/kernel/2109.2/04978.html
> 

Hi Irui Wang,
Any update regarding that patch?
Can you give more explanation for that errors that we see
when the buffer size is not a multiple of 8?

Thanks,
Dafna

> Thanks,
> Dafna
> 
>>>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
>>
>> On Wed, 2021-11-03 at 16:03 +0800, houlong wei wrote:
>>> Add mtk-vpu driver expert irui.wang in the loop.
>>>
>>> On Mon, 2021-10-18 at 15:07 +0800, Dafna Hirschfeld wrote:
>>>>
>>>> On 18.10.21 03:16, Alexandre Courbot wrote:
>>>>> Hi Hans!
>>>>>
>>>>> On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl>
>>>>> wrote:
>>>>>>
>>>>>> On 20/09/2021 19:04, Dafna Hirschfeld wrote:
>>>>>>> From: Alexandre Courbot <acourbot@chromium.org>
>>>>>>>
>>>>>>> When running memcpy_toio:
>>>>>>> memcpy_toio(send_obj->share_buf, buf, len);
>>>>>>> it was found that errors appear if len is not a multiple of
>>>>>>> 8:
>>>>>>>
>>>>>>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
>>>>>>
>>>>>> Why do errors appear? Is that due to a HW bug? Some other
>>>>>> reason?
>>>>>
>>>>> MTK folks would be the best placed to answer this, but since the
>>>>> failure is reported by the firmware I'd suspect either a firmware
>>>>> or
>>>>> hardware limitation.
>>>>>
>>>>>>
>>>>>>>
>>>>>>> This patch ensures the copy of a multiple of 8 size by
>>>>>>> calling
>>>>>>> round_up(len, 8) when copying
>>>>>>>
>>>>>>> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access
>>>>>>> to
>>>>>>> DTCM buffer.")
>>>>>>> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
>>>>>>> Signed-off-by: Enric Balletbo i Serra <
>>>>>>> enric.balletbo@collabora.com>
>>>>>>> Signed-off-by: Dafna Hirschfeld <
>>>>>>> dafna.hirschfeld@collabora.com
>>>>>>>>
>>>>>>>
>>>>>>> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
>>>>>>> ---
>>>>>>> changes since v3:
>>>>>>> 1. multile -> multiple
>>>>>>> 2. add inline doc
>>>>>>>
>>>>>>> changes since v2:
>>>>>>> 1. do the extra copy only if len is not multiple of 8
>>>>>>>
>>>>>>> changes since v1:
>>>>>>> 1. change sign-off-by tags
>>>>>>> 2. change values to memset
>>>>>>>
>>>>>>>    drivers/media/platform/mtk-vpu/mtk_vpu.c | 15
>>>>>>> ++++++++++++++-
>>>>>>>    1 file changed, 14 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>> b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>> index ec290dde59cf..1df031716c8f 100644
>>>>>>> --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>> @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device
>>>>>>> *pdev,
>>>>>>>                 }
>>>>>>>         } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>>>>>>>
>>>>>>> -     memcpy_toio(send_obj->share_buf, buf, len);
>>>>>>> +     /*
>>>>>>> +      * when copying data to the vpu hardware, the
>>>>>>> memcpy_toio
>>>>>>> operation must copy
>>>>>>> +      * a multiple of 8. Otherwise the processing fails
>>>>>>
>>>>>> Same here: it needs to explain why the processing fails.
>>>>
>>>> Is writing 'due to hardware or firmware limitation' enough?
>>>> If not, then we should wait for mediatek people's response to
>>>> explain
>>>> if they know more
>>>>
>>>>>>
>>>>>>> +      */
>>>>>>> +     if (len % 8 != 0) {
>>>>>>> +             unsigned char data[SHARE_BUF_SIZE];
>>>>>>
>>>>>> Wouldn't it be more robust if you say:
>>>>>>
>>>>>>                   unsigned char data[sizeof(send_obj-
>>>>>>> share_buf)];
>>>>>
>>>>> Definitely yes.
>>>>
>>>> I'll send v5 fixing this
>>>>
>>>>>
>>>>>>
>>>>>> I also think that the SHARE_BUF_SIZE define needs a comment
>>>>>> stating that it must be a
>>>>>> multiple of 8, otherwise unexpected things can happen.
>>>>>>
>>>>>> You also noticed that the current SHARE_BUF_SIZE define is too
>>>>>> low, but I saw
>>>>>> no patch correcting this. Shouldn't that be fixed as well?
>>>>>
>>>>> AFAICT the firmware expects this exact size on its end, so I
>>>>> don't
>>>>> believe it can be changed that easily. But maybe someone from MTK
>>>>> can
>>>>> prove me wrong.
>>>>>
>>>>
>>>> I looked further and noted that the structs that are larger than
>>>> 'SHARE_BUF_SIZE'
>>>> (venc_ap_ipi_msg_enc_ext venc_ap_ipi_msg_set_param_ext)
>>>> are used by drivers that don't use this vpu api, so actually
>>>> SHARE_BUF_SIZE is
>>>> not too low and as Corurbot worte probably not changeable.
>>>>
>>>>
>>>> Thanks,
>>>> Dafna
>>>>
>>>>> Cheers,
>>>>> Alex.
>>>>>
>>>
>>>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-11-09  8:46             ` Dafna Hirschfeld
@ 2021-11-10  3:11               ` Irui Wang
  2021-11-29  8:11                 ` Dafna Hirschfeld
  0 siblings, 1 reply; 13+ messages in thread
From: Irui Wang @ 2021-11-10  3:11 UTC (permalink / raw)
  To: Dafna Hirschfeld, houlong wei, Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Dafna Hirschfeld, Tiffany Lin (林慧珊),
	Andrew-CT Chen (陳智迪),
	Minghsiu Tsai (蔡明修),
	Mauro Carvalho Chehab, Matthias Brugger

Hi, Dafna,

Thanks for the patch.
On Tue, 2021-11-09 at 10:46 +0200, Dafna Hirschfeld wrote:
> 
> On 03.11.21 13:04, Dafna Hirschfeld wrote:
> > 
> > 
> > On 03.11.21 10:19, Irui Wang wrote:
> > > Hi,
> > > 
> > > The "len" of share_buf copied should be always 8 alignment;
> > > do you have other logs to prove the len is not 8 alignment when
> > > errors
> > > appear?
> > 
> > Hi, I found out that "sizeof(mdp_ipi_comm) = 20"
> > this is due to the macro #pragma pack(push, 4) in mtk_mdp_ipi.h
> > 
> > see [1]
> > 
> > [1] http://lkml.iu.edu/hypermail/linux/kernel/2109.2/04978.html
> > 
> 
> Hi Irui Wang,
> Any update regarding that patch?
> Can you give more explanation for that errors that we see
> when the buffer size is not a multiple of 8?
> 
> Thanks,
> Dafna

share_buf is a mapped memory by ioremap, it should be better use
memcpy_to/fromio instead of memcpy because of alignment.

As for memcpy_toio, it may also have requirements for alignment, we can
also get such information from:

https://elixir.bootlin.com/linux/v5.15/source/drivers/mtd/spi-nor/controllers/aspeed-smc.c#L207
.

So, it's not VPU HW bug or limitation, it's memcpy_toio requirements,
maybe we can modify IPI message to do alignment, but it need modify
both kernel and vpu firmware, which will break upstream backward
compatible, we think it's unacceptale.

If this patch can solve the issue, we think it's OK.

Thanks
> 
> > Thanks,
> > Dafna
> > 
> > > > > [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
> > > 
> > > On Wed, 2021-11-03 at 16:03 +0800, houlong wei wrote:
> > > > Add mtk-vpu driver expert irui.wang in the loop.
> > > > 
> > > > On Mon, 2021-10-18 at 15:07 +0800, Dafna Hirschfeld wrote:
> > > > > 
> > > > > On 18.10.21 03:16, Alexandre Courbot wrote:
> > > > > > Hi Hans!
> > > > > > 
> > > > > > On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <
> > > > > > hverkuil@xs4all.nl>
> > > > > > wrote:
> > > > > > > 
> > > > > > > On 20/09/2021 19:04, Dafna Hirschfeld wrote:
> > > > > > > > From: Alexandre Courbot <acourbot@chromium.org>
> > > > > > > > 
> > > > > > > > When running memcpy_toio:
> > > > > > > > memcpy_toio(send_obj->share_buf, buf, len);
> > > > > > > > it was found that errors appear if len is not a
> > > > > > > > multiple of
> > > > > > > > 8:
> > > > > > > > 
> > > > > > > > [58.350841] mtk-mdp 14001000.rdma: processing failed:
> > > > > > > > -22
> > > > > > > 
> > > > > > > Why do errors appear? Is that due to a HW bug? Some other
> > > > > > > reason?
> > > > > > 
> > > > > > MTK folks would be the best placed to answer this, but
> > > > > > since the
> > > > > > failure is reported by the firmware I'd suspect either a
> > > > > > firmware
> > > > > > or
> > > > > > hardware limitation.
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > This patch ensures the copy of a multiple of 8 size by
> > > > > > > > calling
> > > > > > > > round_up(len, 8) when copying
> > > > > > > > 
> > > > > > > > Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned
> > > > > > > > access
> > > > > > > > to
> > > > > > > > DTCM buffer.")
> > > > > > > > Signed-off-by: Alexandre Courbot <acourbot@chromium.org
> > > > > > > > >
> > > > > > > > Signed-off-by: Enric Balletbo i Serra <
> > > > > > > > enric.balletbo@collabora.com>
> > > > > > > > Signed-off-by: Dafna Hirschfeld <
> > > > > > > > dafna.hirschfeld@collabora.com
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
> > > > > > > > ---
> > > > > > > > changes since v3:
> > > > > > > > 1. multile -> multiple
> > > > > > > > 2. add inline doc
> > > > > > > > 
> > > > > > > > changes since v2:
> > > > > > > > 1. do the extra copy only if len is not multiple of 8
> > > > > > > > 
> > > > > > > > changes since v1:
> > > > > > > > 1. change sign-off-by tags
> > > > > > > > 2. change values to memset
> > > > > > > > 
> > > > > > > >    drivers/media/platform/mtk-vpu/mtk_vpu.c | 15
> > > > > > > > ++++++++++++++-
> > > > > > > >    1 file changed, 14 insertions(+), 1 deletion(-)
> > > > > > > > 
> > > > > > > > diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > > > > b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > > > > index ec290dde59cf..1df031716c8f 100644
> > > > > > > > --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > > > > +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> > > > > > > > @@ -349,7 +349,20 @@ int vpu_ipi_send(struct
> > > > > > > > platform_device
> > > > > > > > *pdev,
> > > > > > > >                 }
> > > > > > > >         } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> > > > > > > > 
> > > > > > > > -     memcpy_toio(send_obj->share_buf, buf, len);
> > > > > > > > +     /*
> > > > > > > > +      * when copying data to the vpu hardware, the
> > > > > > > > memcpy_toio
> > > > > > > > operation must copy
> > > > > > > > +      * a multiple of 8. Otherwise the processing
> > > > > > > > fails
> > > > > > > 
> > > > > > > Same here: it needs to explain why the processing fails.
> > > > > 
> > > > > Is writing 'due to hardware or firmware limitation' enough?
> > > > > If not, then we should wait for mediatek people's response to
> > > > > explain
> > > > > if they know more
> > > > > 
> > > > > > > 
> > > > > > > > +      */
> > > > > > > > +     if (len % 8 != 0) {
> > > > > > > > +             unsigned char data[SHARE_BUF_SIZE];
> > > > > > > 
> > > > > > > Wouldn't it be more robust if you say:
> > > > > > > 
> > > > > > >                   unsigned char data[sizeof(send_obj-
> > > > > > > > share_buf)];
> > > > > > 
> > > > > > Definitely yes.
> > > > > 
> > > > > I'll send v5 fixing this
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > I also think that the SHARE_BUF_SIZE define needs a
> > > > > > > comment
> > > > > > > stating that it must be a
> > > > > > > multiple of 8, otherwise unexpected things can happen.
> > > > > > > 
> > > > > > > You also noticed that the current SHARE_BUF_SIZE define
> > > > > > > is too
> > > > > > > low, but I saw
> > > > > > > no patch correcting this. Shouldn't that be fixed as
> > > > > > > well?
> > > > > > 
> > > > > > AFAICT the firmware expects this exact size on its end, so
> > > > > > I
> > > > > > don't
> > > > > > believe it can be changed that easily. But maybe someone
> > > > > > from MTK
> > > > > > can
> > > > > > prove me wrong.
> > > > > > 
> > > > > 
> > > > > I looked further and noted that the structs that are larger
> > > > > than
> > > > > 'SHARE_BUF_SIZE'
> > > > > (venc_ap_ipi_msg_enc_ext venc_ap_ipi_msg_set_param_ext)
> > > > > are used by drivers that don't use this vpu api, so actually
> > > > > SHARE_BUF_SIZE is
> > > > > not too low and as Corurbot worte probably not changeable.
> > > > > 
> > > > > 
> > > > > Thanks,
> > > > > Dafna
> > > > > 
> > > > > > Cheers,
> > > > > > Alex.
> > > > > > 
> > > > 
> > > > 

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-11-10  3:11               ` Irui Wang
@ 2021-11-29  8:11                 ` Dafna Hirschfeld
  0 siblings, 0 replies; 13+ messages in thread
From: Dafna Hirschfeld @ 2021-11-29  8:11 UTC (permalink / raw)
  To: Irui Wang, houlong wei, Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Dafna Hirschfeld, Tiffany Lin (林慧珊),
	Andrew-CT Chen (陳智迪),
	Minghsiu Tsai (蔡明修),
	Mauro Carvalho Chehab, Matthias Brugger



On 10.11.21 05:11, Irui Wang wrote:
> Hi, Dafna,
> 
> Thanks for the patch.
> On Tue, 2021-11-09 at 10:46 +0200, Dafna Hirschfeld wrote:
>>
>> On 03.11.21 13:04, Dafna Hirschfeld wrote:
>>>
>>>
>>> On 03.11.21 10:19, Irui Wang wrote:
>>>> Hi,
>>>>
>>>> The "len" of share_buf copied should be always 8 alignment;
>>>> do you have other logs to prove the len is not 8 alignment when
>>>> errors
>>>> appear?
>>>
>>> Hi, I found out that "sizeof(mdp_ipi_comm) = 20"
>>> this is due to the macro #pragma pack(push, 4) in mtk_mdp_ipi.h
>>>
>>> see [1]
>>>
>>> [1] http://lkml.iu.edu/hypermail/linux/kernel/2109.2/04978.html
>>>
>>
>> Hi Irui Wang,
>> Any update regarding that patch?
>> Can you give more explanation for that errors that we see
>> when the buffer size is not a multiple of 8?
>>
>> Thanks,
>> Dafna
> 
> share_buf is a mapped memory by ioremap, it should be better use
> memcpy_to/fromio instead of memcpy because of alignment.
> 
> As for memcpy_toio, it may also have requirements for alignment, we can
> also get such information from:
> 
> https://elixir.bootlin.com/linux/v5.15/source/drivers/mtd/spi-nor/controllers/aspeed-smc.c#L207
> .
> 
> So, it's not VPU HW bug or limitation, it's memcpy_toio requirements,
> maybe we can modify IPI message to do alignment, but it need modify
> both kernel and vpu firmware, which will break upstream backward
> compatible, we think it's unacceptale.
> 
> If this patch can solve the issue, we think it's OK.

In such case shouldn't we also make sure that the address of
send_obj->share_buf is 8 aligend before calling memcpy_toio,
and also shouldn't we make the same checks for the memcpy_fromio ?

Thanks,
Dafna

> 
> Thanks
>>
>>> Thanks,
>>> Dafna
>>>
>>>>>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
>>>>
>>>> On Wed, 2021-11-03 at 16:03 +0800, houlong wei wrote:
>>>>> Add mtk-vpu driver expert irui.wang in the loop.
>>>>>
>>>>> On Mon, 2021-10-18 at 15:07 +0800, Dafna Hirschfeld wrote:
>>>>>>
>>>>>> On 18.10.21 03:16, Alexandre Courbot wrote:
>>>>>>> Hi Hans!
>>>>>>>
>>>>>>> On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <
>>>>>>> hverkuil@xs4all.nl>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 20/09/2021 19:04, Dafna Hirschfeld wrote:
>>>>>>>>> From: Alexandre Courbot <acourbot@chromium.org>
>>>>>>>>>
>>>>>>>>> When running memcpy_toio:
>>>>>>>>> memcpy_toio(send_obj->share_buf, buf, len);
>>>>>>>>> it was found that errors appear if len is not a
>>>>>>>>> multiple of
>>>>>>>>> 8:
>>>>>>>>>
>>>>>>>>> [58.350841] mtk-mdp 14001000.rdma: processing failed:
>>>>>>>>> -22
>>>>>>>>
>>>>>>>> Why do errors appear? Is that due to a HW bug? Some other
>>>>>>>> reason?
>>>>>>>
>>>>>>> MTK folks would be the best placed to answer this, but
>>>>>>> since the
>>>>>>> failure is reported by the firmware I'd suspect either a
>>>>>>> firmware
>>>>>>> or
>>>>>>> hardware limitation.
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> This patch ensures the copy of a multiple of 8 size by
>>>>>>>>> calling
>>>>>>>>> round_up(len, 8) when copying
>>>>>>>>>
>>>>>>>>> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned
>>>>>>>>> access
>>>>>>>>> to
>>>>>>>>> DTCM buffer.")
>>>>>>>>> Signed-off-by: Alexandre Courbot <acourbot@chromium.org
>>>>>>>>>>
>>>>>>>>> Signed-off-by: Enric Balletbo i Serra <
>>>>>>>>> enric.balletbo@collabora.com>
>>>>>>>>> Signed-off-by: Dafna Hirschfeld <
>>>>>>>>> dafna.hirschfeld@collabora.com
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
>>>>>>>>> ---
>>>>>>>>> changes since v3:
>>>>>>>>> 1. multile -> multiple
>>>>>>>>> 2. add inline doc
>>>>>>>>>
>>>>>>>>> changes since v2:
>>>>>>>>> 1. do the extra copy only if len is not multiple of 8
>>>>>>>>>
>>>>>>>>> changes since v1:
>>>>>>>>> 1. change sign-off-by tags
>>>>>>>>> 2. change values to memset
>>>>>>>>>
>>>>>>>>>     drivers/media/platform/mtk-vpu/mtk_vpu.c | 15
>>>>>>>>> ++++++++++++++-
>>>>>>>>>     1 file changed, 14 insertions(+), 1 deletion(-)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>>>> b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>>>> index ec290dde59cf..1df031716c8f 100644
>>>>>>>>> --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>>>> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>>>>>>>> @@ -349,7 +349,20 @@ int vpu_ipi_send(struct
>>>>>>>>> platform_device
>>>>>>>>> *pdev,
>>>>>>>>>                  }
>>>>>>>>>          } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>>>>>>>>>
>>>>>>>>> -     memcpy_toio(send_obj->share_buf, buf, len);
>>>>>>>>> +     /*
>>>>>>>>> +      * when copying data to the vpu hardware, the
>>>>>>>>> memcpy_toio
>>>>>>>>> operation must copy
>>>>>>>>> +      * a multiple of 8. Otherwise the processing
>>>>>>>>> fails
>>>>>>>>
>>>>>>>> Same here: it needs to explain why the processing fails.
>>>>>>
>>>>>> Is writing 'due to hardware or firmware limitation' enough?
>>>>>> If not, then we should wait for mediatek people's response to
>>>>>> explain
>>>>>> if they know more
>>>>>>
>>>>>>>>
>>>>>>>>> +      */
>>>>>>>>> +     if (len % 8 != 0) {
>>>>>>>>> +             unsigned char data[SHARE_BUF_SIZE];
>>>>>>>>
>>>>>>>> Wouldn't it be more robust if you say:
>>>>>>>>
>>>>>>>>                    unsigned char data[sizeof(send_obj-
>>>>>>>>> share_buf)];
>>>>>>>
>>>>>>> Definitely yes.
>>>>>>
>>>>>> I'll send v5 fixing this
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> I also think that the SHARE_BUF_SIZE define needs a
>>>>>>>> comment
>>>>>>>> stating that it must be a
>>>>>>>> multiple of 8, otherwise unexpected things can happen.
>>>>>>>>
>>>>>>>> You also noticed that the current SHARE_BUF_SIZE define
>>>>>>>> is too
>>>>>>>> low, but I saw
>>>>>>>> no patch correcting this. Shouldn't that be fixed as
>>>>>>>> well?
>>>>>>>
>>>>>>> AFAICT the firmware expects this exact size on its end, so
>>>>>>> I
>>>>>>> don't
>>>>>>> believe it can be changed that easily. But maybe someone
>>>>>>> from MTK
>>>>>>> can
>>>>>>> prove me wrong.
>>>>>>>
>>>>>>
>>>>>> I looked further and noted that the structs that are larger
>>>>>> than
>>>>>> 'SHARE_BUF_SIZE'
>>>>>> (venc_ap_ipi_msg_enc_ext venc_ap_ipi_msg_set_param_ext)
>>>>>> are used by drivers that don't use this vpu api, so actually
>>>>>> SHARE_BUF_SIZE is
>>>>>> not too low and as Corurbot worte probably not changeable.
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Dafna
>>>>>>
>>>>>>> Cheers,
>>>>>>> Alex.
>>>>>>>
>>>>>
>>>>>

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-10-18  1:16   ` Alexandre Courbot
  2021-10-18  7:07     ` Dafna Hirschfeld
@ 2021-11-29 14:39     ` Dafna Hirschfeld
  2021-12-06  7:23       ` Alexandre Courbot
  1 sibling, 1 reply; 13+ messages in thread
From: Dafna Hirschfeld @ 2021-11-29 14:39 UTC (permalink / raw)
  To: Alexandre Courbot, Hans Verkuil
  Cc: Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Dafna Hirschfeld, Tiffany Lin, Andrew-CT Chen, minghsiu.tsai,
	houlong.wei, Mauro Carvalho Chehab, Matthias Brugger



On 18.10.21 04:16, Alexandre Courbot wrote:
> Hi Hans!
> 
> On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 20/09/2021 19:04, Dafna Hirschfeld wrote:
>>> From: Alexandre Courbot <acourbot@chromium.org>
>>>
>>> When running memcpy_toio:
>>> memcpy_toio(send_obj->share_buf, buf, len);
>>> it was found that errors appear if len is not a multiple of 8:
>>>
>>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
>>
>> Why do errors appear? Is that due to a HW bug? Some other reason?
> 
> MTK folks would be the best placed to answer this, but since the
> failure is reported by the firmware I'd suspect either a firmware or
> hardware limitation.
> 
>>
>>>
>>> This patch ensures the copy of a multiple of 8 size by calling
>>> round_up(len, 8) when copying
>>>
>>> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to DTCM buffer.")
>>> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
>>> Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
>>> Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
>>> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
>>> ---
>>> changes since v3:
>>> 1. multile -> multiple
>>> 2. add inline doc
>>>
>>> changes since v2:
>>> 1. do the extra copy only if len is not multiple of 8
>>>
>>> changes since v1:
>>> 1. change sign-off-by tags
>>> 2. change values to memset
>>>
>>>   drivers/media/platform/mtk-vpu/mtk_vpu.c | 15 ++++++++++++++-
>>>   1 file changed, 14 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>> index ec290dde59cf..1df031716c8f 100644
>>> --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
>>> @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device *pdev,
>>>                }
>>>        } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
>>>
>>> -     memcpy_toio(send_obj->share_buf, buf, len);
>>> +     /*
>>> +      * when copying data to the vpu hardware, the memcpy_toio operation must copy
>>> +      * a multiple of 8. Otherwise the processing fails
>>
>> Same here: it needs to explain why the processing fails.
>>
>>> +      */
>>> +     if (len % 8 != 0) {
>>> +             unsigned char data[SHARE_BUF_SIZE];
>>
>> Wouldn't it be more robust if you say:
>>
>>                  unsigned char data[sizeof(send_obj->share_buf)];
> 
> Definitely yes.

won't it actually be better to implement it like this:
(assuming len is always multiply of 4 - which I think it must be since access must be 4 aligned)

	void __iomem *to = obj->share_buf;

         if (len % 8 != 0) {
                 memcpy_toio(to, buf, len - 4);
                 to += len - 4;
                 buf += len - 4;
                 writel_relaxed(*(u32 *)buf, to);
         } else {
                 memcpy_toio(obj->share_buf, buf, len);
         }
Thanks,
Dafna
> 
>>
>> I also think that the SHARE_BUF_SIZE define needs a comment stating that it must be a
>> multiple of 8, otherwise unexpected things can happen.
>>
>> You also noticed that the current SHARE_BUF_SIZE define is too low, but I saw
>> no patch correcting this. Shouldn't that be fixed as well?
> 
> AFAICT the firmware expects this exact size on its end, so I don't
> believe it can be changed that easily. But maybe someone from MTK can
> prove me wrong.
> 
> Cheers,
> Alex.
> 

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-11-29 14:39     ` Dafna Hirschfeld
@ 2021-12-06  7:23       ` Alexandre Courbot
  0 siblings, 0 replies; 13+ messages in thread
From: Alexandre Courbot @ 2021-12-06  7:23 UTC (permalink / raw)
  To: Dafna Hirschfeld
  Cc: Hans Verkuil, Linux Media Mailing List,
	moderated list:ARM/Mediatek SoC support, LKML, kernel,
	Dafna Hirschfeld, Tiffany Lin, Andrew-CT Chen, minghsiu.tsai,
	houlong.wei, Mauro Carvalho Chehab, Matthias Brugger

On Mon, Nov 29, 2021 at 11:39 PM Dafna Hirschfeld
<dafna.hirschfeld@collabora.com> wrote:
>
>
>
> On 18.10.21 04:16, Alexandre Courbot wrote:
> > Hi Hans!
> >
> > On Mon, Oct 4, 2021 at 6:37 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> >>
> >> On 20/09/2021 19:04, Dafna Hirschfeld wrote:
> >>> From: Alexandre Courbot <acourbot@chromium.org>
> >>>
> >>> When running memcpy_toio:
> >>> memcpy_toio(send_obj->share_buf, buf, len);
> >>> it was found that errors appear if len is not a multiple of 8:
> >>>
> >>> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
> >>
> >> Why do errors appear? Is that due to a HW bug? Some other reason?
> >
> > MTK folks would be the best placed to answer this, but since the
> > failure is reported by the firmware I'd suspect either a firmware or
> > hardware limitation.
> >
> >>
> >>>
> >>> This patch ensures the copy of a multiple of 8 size by calling
> >>> round_up(len, 8) when copying
> >>>
> >>> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to DTCM buffer.")
> >>> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> >>> Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
> >>> Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
> >>> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>
> >>> ---
> >>> changes since v3:
> >>> 1. multile -> multiple
> >>> 2. add inline doc
> >>>
> >>> changes since v2:
> >>> 1. do the extra copy only if len is not multiple of 8
> >>>
> >>> changes since v1:
> >>> 1. change sign-off-by tags
> >>> 2. change values to memset
> >>>
> >>>   drivers/media/platform/mtk-vpu/mtk_vpu.c | 15 ++++++++++++++-
> >>>   1 file changed, 14 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/media/platform/mtk-vpu/mtk_vpu.c b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> >>> index ec290dde59cf..1df031716c8f 100644
> >>> --- a/drivers/media/platform/mtk-vpu/mtk_vpu.c
> >>> +++ b/drivers/media/platform/mtk-vpu/mtk_vpu.c
> >>> @@ -349,7 +349,20 @@ int vpu_ipi_send(struct platform_device *pdev,
> >>>                }
> >>>        } while (vpu_cfg_readl(vpu, HOST_TO_VPU));
> >>>
> >>> -     memcpy_toio(send_obj->share_buf, buf, len);
> >>> +     /*
> >>> +      * when copying data to the vpu hardware, the memcpy_toio operation must copy
> >>> +      * a multiple of 8. Otherwise the processing fails
> >>
> >> Same here: it needs to explain why the processing fails.
> >>
> >>> +      */
> >>> +     if (len % 8 != 0) {
> >>> +             unsigned char data[SHARE_BUF_SIZE];
> >>
> >> Wouldn't it be more robust if you say:
> >>
> >>                  unsigned char data[sizeof(send_obj->share_buf)];
> >
> > Definitely yes.
>
> won't it actually be better to implement it like this:
> (assuming len is always multiply of 4 - which I think it must be since access must be 4 aligned)
>
>         void __iomem *to = obj->share_buf;
>
>          if (len % 8 != 0) {
>                  memcpy_toio(to, buf, len - 4);
>                  to += len - 4;
>                  buf += len - 4;
>                  writel_relaxed(*(u32 *)buf, to);
>          } else {
>                  memcpy_toio(obj->share_buf, buf, len);
>          }

Not sure if avoiding that stack allocation is worth the extra
complexity and requirement for len being a multiple of 4. Also I'd
like to test it on real hardware to confirm it is indeed ok.

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer
  2021-09-20 17:04 [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer Dafna Hirschfeld
  2021-10-04  9:37 ` Hans Verkuil
@ 2022-02-15  9:41 ` AngeloGioacchino Del Regno
  1 sibling, 0 replies; 13+ messages in thread
From: AngeloGioacchino Del Regno @ 2022-02-15  9:41 UTC (permalink / raw)
  To: linux-media, linux-mediatek, linux-kernel
  Cc: Alexandre Courbot, kernel, dafna3, tiffany.lin, andrew-ct.chen,
	minghsiu.tsai, houlong.wei, mchehab, matthias.bgg, hverkuil

Il 20/09/21 19:04, Dafna Hirschfeld ha scritto:
> From: Alexandre Courbot <acourbot@chromium.org>
> 
> When running memcpy_toio:
> memcpy_toio(send_obj->share_buf, buf, len);
> it was found that errors appear if len is not a multiple of 8:
> 
> [58.350841] mtk-mdp 14001000.rdma: processing failed: -22
> 
> This patch ensures the copy of a multiple of 8 size by calling
> round_up(len, 8) when copying
> 
> Fixes: e6599adfad30 ("media: mtk-vpu: avoid unaligned access to DTCM buffer.")
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> Signed-off-by: Enric Balletbo i Serra <enric.balletbo@collabora.com>
> Signed-off-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
> Reviewed-by: Houlong Wei <houlong.wei@mediatek.com>

Reviewed-by: AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com>

I acknowledge that this patch is getting very old and stale, but after some
testing on MT8173, we were able to verify that it is essential for MediaTek MDP
functionality, along with some other patches.

Without this one, it won't work.

Keeping this in mind, I would propose to get this one picked before it gets
completely stale and requiring a rewrite.

Cheers,
Angelo

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2022-02-15  9:41 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-20 17:04 [PATCH v4] media: mtk-vpu: Ensure alignment of 8 for DTCM buffer Dafna Hirschfeld
2021-10-04  9:37 ` Hans Verkuil
2021-10-18  1:16   ` Alexandre Courbot
2021-10-18  7:07     ` Dafna Hirschfeld
2021-11-03  8:03       ` houlong wei
2021-11-03  9:19         ` Irui Wang
2021-11-03 11:04           ` Dafna Hirschfeld
2021-11-09  8:46             ` Dafna Hirschfeld
2021-11-10  3:11               ` Irui Wang
2021-11-29  8:11                 ` Dafna Hirschfeld
2021-11-29 14:39     ` Dafna Hirschfeld
2021-12-06  7:23       ` Alexandre Courbot
2022-02-15  9:41 ` AngeloGioacchino Del Regno

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).