[PATCH] media: venus: dec: Fix handling of the start cmd

[PATCH] media: venus: dec: Fix handling of the start cmd

[Michał Krawczyk]

From: Michał Krawczyk <mk@semihalf.com>

The decoder driver should clear the last_buffer_dequeued flag of the
capture queue upon receiving V4L2_DEC_CMD_START.

The last_buffer_dequeued flag is set upon receiving EOS (which always
happens upon receiving V4L2_DEC_CMD_STOP).

Without this patch, after issuing the V4L2_DEC_CMD_STOP and
V4L2_DEC_CMD_START, the vb2_dqbuf() function will always fail, even if
the buffers are completed by the hardware.

Fixes: beac82904a87 ("media: venus: make decoder compliant with stateful codec API")

Signed-off-by: Michał Krawczyk <mk@semihalf.com>
---
 drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 4ceaba37e2e5..175488ea08ff 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -526,6 +526,7 @@ static int
 vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
 {
 	struct venus_inst *inst = to_inst(file);
+	struct vb2_queue *dst_vq;
 	struct hfi_frame_data fdata = {0};
 	int ret;
 
@@ -556,6 +557,13 @@ vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
 			inst->codec_state = VENUS_DEC_STATE_DRAIN;
 			inst->drain_active = true;
 		}
+	} else if (cmd->cmd == V4L2_DEC_CMD_START &&
+		   inst->codec_state == VENUS_DEC_STATE_STOPPED) {
+		dst_vq = v4l2_m2m_get_vq(inst->fh.m2m_ctx,
+					 V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+		vb2_clear_last_buffer_dequeued(&inst->fh.m2m_ctx->cap_q_ctx.q);
+
+		inst->codec_state = VENUS_DEC_STATE_DECODING;
 	}
 
 unlock:
-- 
2.39.1.456.gfc5497dd1b-goog

Re: [PATCH] media: venus: dec: Fix handling of the start cmd

[kernel test robot]

Hi Michał,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on media-tree/master]
[also build test WARNING on linus/master v6.2-rc6 next-20230130]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Micha-Krawczyk/media-venus-dec-Fix-handling-of-the-start-cmd/20230130-185626
base:   git://linuxtv.org/media_tree.git master
patch link:    https://lore.kernel.org/r/20230130105423.1338554-1-mk%40semmihalf.com
patch subject: [PATCH] media: venus: dec: Fix handling of the start cmd
config: m68k-allyesconfig (https://download.01.org/0day-ci/archive/20230130/202301302043.AAXAiTHh-lkp@intel.com/config)
compiler: m68k-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/intel-lab-lkp/linux/commit/3aa2620bc66440999bc7906165d2a5adb129402f
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Micha-Krawczyk/media-venus-dec-Fix-handling-of-the-start-cmd/20230130-185626
        git checkout 3aa2620bc66440999bc7906165d2a5adb129402f
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k olddefconfig
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=m68k SHELL=/bin/bash drivers/media/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   drivers/media/platform/qcom/venus/vdec.c: In function 'vdec_decoder_cmd':
>> drivers/media/platform/qcom/venus/vdec.c:529:27: warning: variable 'dst_vq' set but not used [-Wunused-but-set-variable]
     529 |         struct vb2_queue *dst_vq;
         |                           ^~~~~~


vim +/dst_vq +529 drivers/media/platform/qcom/venus/vdec.c

   524	
   525	static int
   526	vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
   527	{
   528		struct venus_inst *inst = to_inst(file);
 > 529		struct vb2_queue *dst_vq;
   530		struct hfi_frame_data fdata = {0};
   531		int ret;
   532	
   533		ret = v4l2_m2m_ioctl_try_decoder_cmd(file, fh, cmd);
   534		if (ret)
   535			return ret;
   536	
   537		mutex_lock(&inst->lock);
   538	
   539		if (cmd->cmd == V4L2_DEC_CMD_STOP) {
   540			/*
   541			 * Implement V4L2_DEC_CMD_STOP by enqueue an empty buffer on
   542			 * decoder input to signal EOS.
   543			 */
   544			if (!(inst->streamon_out && inst->streamon_cap))
   545				goto unlock;
   546	
   547			fdata.buffer_type = HFI_BUFFER_INPUT;
   548			fdata.flags |= HFI_BUFFERFLAG_EOS;
   549			if (IS_V6(inst->core))
   550				fdata.device_addr = 0;
   551			else
   552				fdata.device_addr = 0xdeadb000;
   553	
   554			ret = hfi_session_process_buf(inst, &fdata);
   555	
   556			if (!ret && inst->codec_state == VENUS_DEC_STATE_DECODING) {
   557				inst->codec_state = VENUS_DEC_STATE_DRAIN;
   558				inst->drain_active = true;
   559			}
   560		} else if (cmd->cmd == V4L2_DEC_CMD_START &&
   561			   inst->codec_state == VENUS_DEC_STATE_STOPPED) {
   562			dst_vq = v4l2_m2m_get_vq(inst->fh.m2m_ctx,
   563						 V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
   564			vb2_clear_last_buffer_dequeued(&inst->fh.m2m_ctx->cap_q_ctx.q);
   565	
   566			inst->codec_state = VENUS_DEC_STATE_DECODING;
   567		}
   568	
   569	unlock:
   570		mutex_unlock(&inst->lock);
   571		return ret;
   572	}
   573	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests

[PATCH v2] media: venus: dec: Fix handling of the start cmd

[Michał Krawczyk]

From: Michał Krawczyk <mk@semihalf.com>

The decoder driver should clear the last_buffer_dequeued flag of the
capture queue upon receiving V4L2_DEC_CMD_START.

The last_buffer_dequeued flag is set upon receiving EOS (which always
happens upon receiving V4L2_DEC_CMD_STOP).

Without this patch, after issuing the V4L2_DEC_CMD_STOP and
V4L2_DEC_CMD_START, the vb2_dqbuf() function will always fail, even if
the buffers are completed by the hardware.

Fixes: beac82904a87 ("media: venus: make decoder compliant with stateful codec API")

Signed-off-by: Michał Krawczyk <mk@semihalf.com>
---
V1 -> V2: Fix warning regarding unused variable

 drivers/media/platform/qcom/venus/vdec.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/venus/vdec.c
index 4ceaba37e2e5..9d26587716bf 100644
--- a/drivers/media/platform/qcom/venus/vdec.c
+++ b/drivers/media/platform/qcom/venus/vdec.c
@@ -526,6 +526,7 @@ static int
 vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
 {
 	struct venus_inst *inst = to_inst(file);
+	struct vb2_queue *dst_vq;
 	struct hfi_frame_data fdata = {0};
 	int ret;
 
@@ -556,6 +557,13 @@ vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
 			inst->codec_state = VENUS_DEC_STATE_DRAIN;
 			inst->drain_active = true;
 		}
+	} else if (cmd->cmd == V4L2_DEC_CMD_START &&
+		   inst->codec_state == VENUS_DEC_STATE_STOPPED) {
+		dst_vq = v4l2_m2m_get_vq(inst->fh.m2m_ctx,
+					 V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE);
+		vb2_clear_last_buffer_dequeued(dst_vq);
+
+		inst->codec_state = VENUS_DEC_STATE_DECODING;
 	}
 
 unlock:
-- 
2.39.1.456.gfc5497dd1b-goog

Re: [PATCH v2] media: venus: dec: Fix handling of the start cmd

[Michał Krawczyk]

pon., 30 sty 2023 o 14:55 Michał Krawczyk <mk@semihalf.com> napisał(a):
>
> From: Michał Krawczyk <mk@semihalf.com>
>
> The decoder driver should clear the last_buffer_dequeued flag of the
> capture queue upon receiving V4L2_DEC_CMD_START.
>
> The last_buffer_dequeued flag is set upon receiving EOS (which always
> happens upon receiving V4L2_DEC_CMD_STOP).
>
> Without this patch, after issuing the V4L2_DEC_CMD_STOP and
> V4L2_DEC_CMD_START, the vb2_dqbuf() function will always fail, even if
> the buffers are completed by the hardware.
>
> Fixes: beac82904a87 ("media: venus: make decoder compliant with stateful codec API")
>
> Signed-off-by: Michał Krawczyk <mk@semihalf.com>

Hello,

Did anyone have a chance to take a look at this patch? It's fairly
simple, but lack of this fix can have a big impact on the V4L2
applications which implement the flush mechanism using the stop/start
commands, especially in the middle of the video.

Thank you,
Michał

RE: [PATCH v2] media: venus: dec: Fix handling of the start cmd

[Vikash Garodia]

Hello Michal,
Thank you for raising a fix in video driver.

> -----Original Message-----
> From: Michał Krawczyk <mk@semihalf.com>
> Sent: Tuesday, February 7, 2023 2:48 PM
> To: Stanimir Varbanov <stanimir.k.varbanov@gmail.com>; Vikash Garodia
> (QUIC) <quic_vgarodia@quicinc.com>
> Cc: Andy Gross <agross@kernel.org>; Bjorn Andersson
> <andersson@kernel.org>; Konrad Dybcio <konrad.dybcio@linaro.org>; Mauro
> Carvalho Chehab <mchehab@kernel.org>; linux-media@vger.kernel.org; linux-
> arm-msm@vger.kernel.org; linux-kernel@vger.kernel.org; mw@semihalf.com
> Subject: Re: [PATCH v2] media: venus: dec: Fix handling of the start cmd
> 
> WARNING: This email originated from outside of Qualcomm. Please be wary of
> any links or attachments, and do not enable macros.
> 
> pon., 30 sty 2023 o 14:55 Michał Krawczyk <mk@semihalf.com> napisał(a):
> >
> > From: Michał Krawczyk <mk@semihalf.com>
> >
> > The decoder driver should clear the last_buffer_dequeued flag of the
> > capture queue upon receiving V4L2_DEC_CMD_START.
> >
> > The last_buffer_dequeued flag is set upon receiving EOS (which always
> > happens upon receiving V4L2_DEC_CMD_STOP).
> >
> > Without this patch, after issuing the V4L2_DEC_CMD_STOP and
> > V4L2_DEC_CMD_START, the vb2_dqbuf() function will always fail, even if
> > the buffers are completed by the hardware.
> >
> > Fixes: beac82904a87 ("media: venus: make decoder compliant with
> > stateful codec API")
> >
> > Signed-off-by: Michał Krawczyk <mk@semihalf.com>
> 
> Hello,
> 
> Did anyone have a chance to take a look at this patch? It's fairly simple, but lack
> of this fix can have a big impact on the V4L2 applications which implement the
> flush mechanism using the stop/start commands, especially in the middle of the
> video.

I have reviewed the patch, and the drain sequence handling looks good to me.
Could you share some details on the test client which you are using to catch this issue ?

> Thank you,
> Michał

Thanks,
Vikash

Re: [PATCH v2] media: venus: dec: Fix handling of the start cmd

[Michał Krawczyk]

wt., 7 lut 2023 o 10:54 Vikash Garodia <vgarodia@qti.qualcomm.com> napisał(a):
> I have reviewed the patch, and the drain sequence handling looks good to me.
> Could you share some details on the test client which you are using to catch this issue ?

Hi Vikash,

Thank you for looking at the code!

I've been testing it using the Chromium implementation of the V4L2
codec [1]. Meanwhile, we were running a test suite which changes the
encryption method in the middle of the video decoding. This triggers
the flush behavior and the Chromium sends the stop/start cmd to the
V4L2 kernel component, and the test expects the video to continue the
playback normally. Unfortunately, it was causing a stall of the video
at the same time.

[1] https://source.chromium.org/chromium/chromium/src/+/main:media/gpu/v4l2/

>
> > Thank you,
> > Michał
>
> Thanks,
> Vikash

Re: [PATCH v2] media: venus: dec: Fix handling of the start cmd

[Michał Krawczyk]

Hi,

I'm wondering if there are any more comments for this patch? I would
be happy to clarify anything that's unclear or improve the code if
needed.

I know it's pretty late, but it would be really great if this fix
could land before v6.2 is released, so I'd appreciate your help and
review.

Thank you,
Michał

wt., 7 lut 2023 o 12:15 Michał Krawczyk <mk@semihalf.com> napisał(a):
>
> wt., 7 lut 2023 o 10:54 Vikash Garodia <vgarodia@qti.qualcomm.com> napisał(a):
> > I have reviewed the patch, and the drain sequence handling looks good to me.
> > Could you share some details on the test client which you are using to catch this issue ?
>
> Hi Vikash,
>
> Thank you for looking at the code!
>
> I've been testing it using the Chromium implementation of the V4L2
> codec [1]. Meanwhile, we were running a test suite which changes the
> encryption method in the middle of the video decoding. This triggers
> the flush behavior and the Chromium sends the stop/start cmd to the
> V4L2 kernel component, and the test expects the video to continue the
> playback normally. Unfortunately, it was causing a stall of the video
> at the same time.
>
> [1] https://source.chromium.org/chromium/chromium/src/+/main:media/gpu/v4l2/
>
> >
> > > Thank you,
> > > Michał
> >
> > Thanks,
> > Vikash

Re: [PATCH v2] media: venus: dec: Fix handling of the start cmd

[Michał Krawczyk]

Hi,

Any update on this patch? It would be great if we could make some
progress there (and, hopefully, finally merge it :))

Thanks,
Michał

pt., 10 lut 2023 o 16:18 Michał Krawczyk <mk@semihalf.com> napisał(a):
>
> Hi,
>
> I'm wondering if there are any more comments for this patch? I would
> be happy to clarify anything that's unclear or improve the code if
> needed.
>
> I know it's pretty late, but it would be really great if this fix
> could land before v6.2 is released, so I'd appreciate your help and
> review.
>
> Thank you,
> Michał
>
> wt., 7 lut 2023 o 12:15 Michał Krawczyk <mk@semihalf.com> napisał(a):
> >
> > wt., 7 lut 2023 o 10:54 Vikash Garodia <vgarodia@qti.qualcomm.com> napisał(a):
> > > I have reviewed the patch, and the drain sequence handling looks good to me.
> > > Could you share some details on the test client which you are using to catch this issue ?
> >
> > Hi Vikash,
> >
> > Thank you for looking at the code!
> >
> > I've been testing it using the Chromium implementation of the V4L2
> > codec [1]. Meanwhile, we were running a test suite which changes the
> > encryption method in the middle of the video decoding. This triggers
> > the flush behavior and the Chromium sends the stop/start cmd to the
> > V4L2 kernel component, and the test expects the video to continue the
> > playback normally. Unfortunately, it was causing a stall of the video
> > at the same time.
> >
> > [1] https://source.chromium.org/chromium/chromium/src/+/main:media/gpu/v4l2/
> >
> > >
> > > > Thank you,
> > > > Michał
> > >
> > > Thanks,
> > > Vikash

Re: [PATCH v2] media: venus: dec: Fix handling of the start cmd

[Michał Krawczyk]

Hi,

just a kindly reminder about the patch.

Thanks,
Michał

pt., 10 mar 2023 o 16:05 Michał Krawczyk <mk@semihalf.com> napisał(a):
>
> Hi,
>
> Any update on this patch? It would be great if we could make some
> progress there (and, hopefully, finally merge it :))
>
> Thanks,
> Michał
>
> pt., 10 lut 2023 o 16:18 Michał Krawczyk <mk@semihalf.com> napisał(a):
> >
> > Hi,
> >
> > I'm wondering if there are any more comments for this patch? I would
> > be happy to clarify anything that's unclear or improve the code if
> > needed.
> >
> > I know it's pretty late, but it would be really great if this fix
> > could land before v6.2 is released, so I'd appreciate your help and
> > review.
> >
> > Thank you,
> > Michał
> >
> > wt., 7 lut 2023 o 12:15 Michał Krawczyk <mk@semihalf.com> napisał(a):
> > >
> > > wt., 7 lut 2023 o 10:54 Vikash Garodia <vgarodia@qti.qualcomm.com> napisał(a):
> > > > I have reviewed the patch, and the drain sequence handling looks good to me.
> > > > Could you share some details on the test client which you are using to catch this issue ?
> > >
> > > Hi Vikash,
> > >
> > > Thank you for looking at the code!
> > >
> > > I've been testing it using the Chromium implementation of the V4L2
> > > codec [1]. Meanwhile, we were running a test suite which changes the
> > > encryption method in the middle of the video decoding. This triggers
> > > the flush behavior and the Chromium sends the stop/start cmd to the
> > > V4L2 kernel component, and the test expects the video to continue the
> > > playback normally. Unfortunately, it was causing a stall of the video
> > > at the same time.
> > >
> > > [1] https://source.chromium.org/chromium/chromium/src/+/main:media/gpu/v4l2/
> > >
> > > >
> > > > > Thank you,
> > > > > Michał
> > > >
> > > > Thanks,
> > > > Vikash

Re: [PATCH v2] media: venus: dec: Fix handling of the start cmd

[Dikshita Agarwal]

On 4/5/2023 2:59 PM, Michał Krawczyk wrote:
> Hi,
>
> just a kindly reminder about the patch.
>
> Thanks,
> Michał

Hi Michal,

this patch is part of latest PR 
https://patchwork.linuxtv.org/project/linux-media/patch/20230404192722.144496-1-stanimir.k.varbanov@gmail.com/ 


Thanks,

Dikshita

> pt., 10 mar 2023 o 16:05 Michał Krawczyk <mk@semihalf.com> napisał(a):
>> Hi,
>>
>> Any update on this patch? It would be great if we could make some
>> progress there (and, hopefully, finally merge it :))
>>
>> Thanks,
>> Michał
>>
>> pt., 10 lut 2023 o 16:18 Michał Krawczyk <mk@semihalf.com> napisał(a):
>>> Hi,
>>>
>>> I'm wondering if there are any more comments for this patch? I would
>>> be happy to clarify anything that's unclear or improve the code if
>>> needed.
>>>
>>> I know it's pretty late, but it would be really great if this fix
>>> could land before v6.2 is released, so I'd appreciate your help and
>>> review.
>>>
>>> Thank you,
>>> Michał
>>>
>>> wt., 7 lut 2023 o 12:15 Michał Krawczyk <mk@semihalf.com> napisał(a):
>>>> wt., 7 lut 2023 o 10:54 Vikash Garodia <vgarodia@qti.qualcomm.com> napisał(a):
>>>>> I have reviewed the patch, and the drain sequence handling looks good to me.
>>>>> Could you share some details on the test client which you are using to catch this issue ?
>>>> Hi Vikash,
>>>>
>>>> Thank you for looking at the code!
>>>>
>>>> I've been testing it using the Chromium implementation of the V4L2
>>>> codec [1]. Meanwhile, we were running a test suite which changes the
>>>> encryption method in the middle of the video decoding. This triggers
>>>> the flush behavior and the Chromium sends the stop/start cmd to the
>>>> V4L2 kernel component, and the test expects the video to continue the
>>>> playback normally. Unfortunately, it was causing a stall of the video
>>>> at the same time.
>>>>
>>>> [1] https://source.chromium.org/chromium/chromium/src/+/main:media/gpu/v4l2/
>>>>
>>>>>> Thank you,
>>>>>> Michał
>>>>> Thanks,
>>>>> Vikash