[PATCH 0/2] media: cedrus: fix HEVC decoding

[PATCH 0/2] media: cedrus: fix HEVC decoding

[Jernej Skrabec]

After detailed comparison of register names to vendor library I noticed
that one register has completely different name. After some testing I
discovered that it was misnamed and used incorrectly. This patch series
fixes it. With that, 3 more reference bitstreams are now correctly
decoded. It is also possible that this fixes instability issue I had
after decoding such bitstreams. Running Fluster test suite very often
locked up my board, but after applying this fix, I never experienced it
again. It might still be completely coincidental, but I doubt this is
the case.

Note: Patch 2 clashes with HEVC uAPI destaging. In current form, it can
be easily backported. However, there are few users of Cedrus HEVC and
skipping this fix wouldn't be that bad.

Please let me know which way to go:
1) wait for destaging, send rebased v2 and not care about backporting
2) merge before destaging, but v9 of HEVC uAPI destaging would need to
   be rebased.
3) something else?

Best regards,
Jernej

Jernej Skrabec (2):
  media: cedrus: h265: Fix flag name
  media: cedrus: h265: Fix logic for not low delay flag

 .../staging/media/sunxi/cedrus/cedrus_h265.c  | 29 ++++++++++++++++++-
 .../staging/media/sunxi/cedrus/cedrus_regs.h  |  3 +-
 2 files changed, 29 insertions(+), 3 deletions(-)

-- 
2.36.1

[PATCH 1/2] media: cedrus: h265: Fix flag name

[Jernej Skrabec]

Bit 21 in register 0x24 (slice header info 1) actually represents
negated version of low delay flag. This can be seen in vendor Cedar
library source code. While this flag is not part of the standard, it can
be found in reference HEVC implementation.

Fix macro name and change it to flag.

Fixes: 86caab29da78 ("media: cedrus: Add HEVC/H.265 decoding support")
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
 drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 4 +++-
 drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 3 +--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
index 44f385be9f6c..2febdf7a97fe 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
@@ -559,7 +559,6 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 
 	reg = VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_TC_OFFSET_DIV2(slice_params->slice_tc_offset_div2) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_BETA_OFFSET_DIV2(slice_params->slice_beta_offset_div2) |
-	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_POC_BIGEST_IN_RPS_ST(decode_params->num_poc_st_curr_after == 0) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_CR_QP_OFFSET(slice_params->slice_cr_qp_offset) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_CB_QP_OFFSET(slice_params->slice_cb_qp_offset) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_QP_DELTA(slice_params->slice_qp_delta);
@@ -572,6 +571,9 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 				V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED,
 				slice_params->flags);
 
+	if (decode_params->num_poc_st_curr_after == 0)
+		reg |= VE_DEC_H265_DEC_SLICE_HDR_INFO1_FLAG_SLICE_NOT_LOW_DELAY;
+
 	cedrus_write(dev, VE_DEC_H265_DEC_SLICE_HDR_INFO1, reg);
 
 	chroma_log2_weight_denom = pred_weight_table->luma_log2_weight_denom +
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h
index bdb062ad8682..d81f7513ade0 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h
@@ -377,13 +377,12 @@
 
 #define VE_DEC_H265_DEC_SLICE_HDR_INFO1_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED BIT(23)
 #define VE_DEC_H265_DEC_SLICE_HDR_INFO1_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED BIT(22)
+#define VE_DEC_H265_DEC_SLICE_HDR_INFO1_FLAG_SLICE_NOT_LOW_DELAY BIT(21)
 
 #define VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_TC_OFFSET_DIV2(v) \
 	SHIFT_AND_MASK_BITS(v, 31, 28)
 #define VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_BETA_OFFSET_DIV2(v) \
 	SHIFT_AND_MASK_BITS(v, 27, 24)
-#define VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_POC_BIGEST_IN_RPS_ST(v) \
-	((v) ? BIT(21) : 0)
 #define VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_CR_QP_OFFSET(v) \
 	SHIFT_AND_MASK_BITS(v, 20, 16)
 #define VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_CB_QP_OFFSET(v) \
-- 
2.36.1

[PATCH 2/2] media: cedrus: h265: Fix logic for not low delay flag

[Jernej Skrabec]

Now that we know real purpose of "not low delay" flag, logic for
applying this flag should be fixed too. According to vendor and
reference implementation, low delay is signaled when POC of current
frame is lower than POC of at least one reference of a slice.

Implement mentioned logic and invert it to conform to flag meaning. Also
don't apply flag for I frames. They don't have any reference.

This fixes decoding of 3 reference bitstreams.

Fixes: 86caab29da78 ("media: cedrus: Add HEVC/H.265 decoding support")
Signed-off-by: Jernej Skrabec <jernej.skrabec@gmail.com>
---
 .../staging/media/sunxi/cedrus/cedrus_h265.c  | 27 ++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
index 2febdf7a97fe..3f50043093be 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
@@ -301,6 +301,31 @@ static void cedrus_h265_write_scaling_list(struct cedrus_ctx *ctx,
 		}
 }
 
+static int cedrus_h265_is_low_delay(struct cedrus_run *run)
+{
+	const struct v4l2_ctrl_hevc_slice_params *slice_params;
+	const struct v4l2_hevc_dpb_entry *dpb;
+	s32 poc;
+	int i;
+
+	slice_params = run->h265.slice_params;
+	poc = run->h265.decode_params->pic_order_cnt_val;
+	dpb = run->h265.decode_params->dpb;
+
+	for (i = 0; i < slice_params->num_ref_idx_l0_active_minus1 + 1; i++)
+		if (dpb[slice_params->ref_idx_l0[i]].pic_order_cnt[0] > poc)
+			return 1;
+
+	if (slice_params->slice_type != V4L2_HEVC_SLICE_TYPE_B)
+		return 0;
+
+	for (i = 0; i < slice_params->num_ref_idx_l1_active_minus1 + 1; i++)
+		if (dpb[slice_params->ref_idx_l1[i]].pic_order_cnt[0] > poc)
+			return 1;
+
+	return 0;
+}
+
 static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 			      struct cedrus_run *run)
 {
@@ -571,7 +596,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 				V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED,
 				slice_params->flags);
 
-	if (decode_params->num_poc_st_curr_after == 0)
+	if (slice_params->slice_type != V4L2_HEVC_SLICE_TYPE_I && !cedrus_h265_is_low_delay(run))
 		reg |= VE_DEC_H265_DEC_SLICE_HDR_INFO1_FLAG_SLICE_NOT_LOW_DELAY;
 
 	cedrus_write(dev, VE_DEC_H265_DEC_SLICE_HDR_INFO1, reg);
-- 
2.36.1

Re: [PATCH 0/2] media: cedrus: fix HEVC decoding

[Hans Verkuil]

On 6/15/22 22:44, Jernej Skrabec wrote:
> After detailed comparison of register names to vendor library I noticed
> that one register has completely different name. After some testing I
> discovered that it was misnamed and used incorrectly. This patch series
> fixes it. With that, 3 more reference bitstreams are now correctly
> decoded. It is also possible that this fixes instability issue I had
> after decoding such bitstreams. Running Fluster test suite very often
> locked up my board, but after applying this fix, I never experienced it
> again. It might still be completely coincidental, but I doubt this is
> the case.
> 
> Note: Patch 2 clashes with HEVC uAPI destaging. In current form, it can
> be easily backported. However, there are few users of Cedrus HEVC and
> skipping this fix wouldn't be that bad.
> 
> Please let me know which way to go:
> 1) wait for destaging, send rebased v2 and not care about backporting

Let's go with 1. There is not much point in backporting since destaging
the HEVC API will also change it, so userspace will need to adapt. It's
a staging driver anyway (although hopefully not for long).

If you post a v2 on top of the latest series from Benjamin, then that
should almost certainly work fine when Benjamin posts what will hopefully
be the final version of his series. When it is all OK, then I put both in
the same PR.

Regards,

	Hans

> 2) merge before destaging, but v9 of HEVC uAPI destaging would need to
>    be rebased.
> 3) something else?
> 
> Best regards,
> Jernej
> 
> Jernej Skrabec (2):
>   media: cedrus: h265: Fix flag name
>   media: cedrus: h265: Fix logic for not low delay flag
> 
>  .../staging/media/sunxi/cedrus/cedrus_h265.c  | 29 ++++++++++++++++++-
>  .../staging/media/sunxi/cedrus/cedrus_regs.h  |  3 +-
>  2 files changed, 29 insertions(+), 3 deletions(-)
> 

Re: [PATCH 0/2] media: cedrus: fix HEVC decoding

[Jernej Škrabec]

Dne četrtek, 16. junij 2022 ob 09:14:05 CEST je Hans Verkuil napisal(a):
> On 6/15/22 22:44, Jernej Skrabec wrote:
> > After detailed comparison of register names to vendor library I noticed
> > that one register has completely different name. After some testing I
> > discovered that it was misnamed and used incorrectly. This patch series
> > fixes it. With that, 3 more reference bitstreams are now correctly
> > decoded. It is also possible that this fixes instability issue I had
> > after decoding such bitstreams. Running Fluster test suite very often
> > locked up my board, but after applying this fix, I never experienced it
> > again. It might still be completely coincidental, but I doubt this is
> > the case.
> > 
> > Note: Patch 2 clashes with HEVC uAPI destaging. In current form, it can
> > be easily backported. However, there are few users of Cedrus HEVC and
> > skipping this fix wouldn't be that bad.
> > 
> > Please let me know which way to go:
> > 1) wait for destaging, send rebased v2 and not care about backporting
> 
> Let's go with 1. There is not much point in backporting since destaging
> the HEVC API will also change it, so userspace will need to adapt. It's
> a staging driver anyway (although hopefully not for long).
> 
> If you post a v2 on top of the latest series from Benjamin, then that
> should almost certainly work fine when Benjamin posts what will hopefully
> be the final version of his series. When it is all OK, then I put both in
> the same PR.

I actually have several patches for Cedrus HEVC which I plan to send soon. 
They would apply on top of series from Benjamin and would implement all 
missing decoding features, except maybe 10-bit support. I'll include these two 
patches in that series.

Best regards,
Jernej

> 
> Regards,
> 
> 	Hans
> 
> > 2) merge before destaging, but v9 of HEVC uAPI destaging would need to
> > 
> >    be rebased.
> > 
> > 3) something else?
> > 
> > Best regards,
> > Jernej
> > 
> > Jernej Skrabec (2):
> >   media: cedrus: h265: Fix flag name
> >   media: cedrus: h265: Fix logic for not low delay flag
> >  
> >  .../staging/media/sunxi/cedrus/cedrus_h265.c  | 29 ++++++++++++++++++-
> >  .../staging/media/sunxi/cedrus/cedrus_regs.h  |  3 +-
> >  2 files changed, 29 insertions(+), 3 deletions(-)