* [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders
@ 2022-04-05 20:44 Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num Nicolas Dufresne
` (25 more replies)
0 siblings, 26 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
Cc: kernel, linux-kernel
Until now, only Cedrus (a slice base decoder) supported interlaced
decoding. In order to support field decoding in our frame-based decoder,
the v4l2-h264 library needed adaptation to produce the appropriate
reference lists.
This patch extends the v4l2-h264 library to produce the larger references
list needed to represent fields separately. Hantro, MTK-VCODEC and RKVDEC
drivers have been adapted to accommodate the larger lists. Though, only
Hantro and RKVDEC actually have HW support for field decoding. So only
these two have been updated to make use of the larger lists. All this work
has been done using the H.264 specification, LibreELEC downstream kernel
patches, Rockchip MPP reference software and Hantro reference software.
For reviewers, the following is the map of all commit. Patches that could
be merge independently of this serie are marked as independent. Note that
the test results do depend on the generic fixes.
01 : Documentation fix (independent)
02-03 : Improving some generic traces (independent)
04 : Minor v4l2-h264 fix (independent)
05-11 : v4l2-h264 field decoding support
12-16 : rkvdec h.264 generic fixes (independent)
17-20 : rkvdec h.264 field decoding support
21-24 : hantro h.264 field decoding support
All this work have been tested using GStreamer mainline implementation
but also with FFMPEG LibreELEC fork using the testing tool fluster
running through the ITU-T H.264 (2016-02) AVCv2 set of bitstream. Before
this patch, the scores were:
Hantro:
FFMPEG: 88/135
GSteamer: 90/135
RKVDEC:
FFMPEG: 73/135
GSteamer: 77/135
And after these changes:
Hantro:
FFMPEG: 118/135
GSteamer: 129/135
RKVDEC:
FFMPEG: 118/135
GSteamer: 129/135
Note that a bug in FFMPEG / LibreELEC fork was noticed and fixed with the
following change:
diff --git a/libavcodec/v4l2_request_h264.c b/libavcodec/v4l2_request_h264.c
index 88da8f0a2d..394bae0550 100644
--- a/libavcodec/v4l2_request_h264.c
+++ b/libavcodec/v4l2_request_h264.c
@@ -66,7 +66,7 @@ static void fill_dpb_entry(struct v4l2_h264_dpb_entry *entry, const H264Picture
{
entry->reference_ts = ff_v4l2_request_get_capture_timestamp(pic->f);
entry->pic_num = pic->pic_id;
- entry->frame_num = pic->frame_num;
+ entry->frame_num = pic->long_ref ? pic->pic_id : pic->frame_num;
entry->fields = pic->reference & V4L2_H264_FRAME_REF;
entry->flags = V4L2_H264_DPB_ENTRY_FLAG_VALID;
if (entry->fields)
Some useful links:
Detailed Hantro Results: https://gitlab.freedesktop.org/-/snippets/5189
Detailed RKVDEC Results: https://gitlab.freedesktop.org/-/snippets/5253
ITU-T H.264 (2016-02) AVCv2: https://www.itu.int/net/itu-t/sigdb/spevideo/VideoForm-s.aspx?val=102002641
Fluster: https://github.com/fluendo/fluster
GStreamer: https://gitlab.freedesktop.org/gstreamer/gstreamer/
FFMPEG Fork: https://github.com/jernejsk/FFmpeg/tree/v4l2-request-hwaccel-4.4
Rockchip MPP: https://github.com/rockchip-linux/mpp
Changes in v3:
- Improved debug message on timestamp miss-match
- Moved H264 SPS validation into rkvdec-h264
- Added more comments around H264 SPS validation
- Also validate at streamon (rkvdec start())
- Applied more Review-by and Fixes tag
- Fixed Signed-off-by chain in Jonas patch
Changes in v2:
- Applied most of Sebastian's suggestion in comments and commit messages.
- Use a bool for dpb_valid and dpb_bottom in rkvdec
- Dropped one wrong typo fix (media: v4l2-mem2mem: Fix typo in trace message)
- Dropped Alex fix (media: rkvdec-h264: Don't hardcode SPS/PPS parameters
+ I will carry this one later, it seems cosmetic
Jonas Karlman (5):
media: rkvdec: h264: Fix bit depth wrap in pps packet
media: rkvdec: h264: Validate and use pic width and height in mbs
media: rkvdec: h264: Fix reference frame_num wrap for second field
media: rkvdec: Ensure decoded resolution fit coded resolution
media: hantro: h264: Make dpb entry management more robust
Nicolas Dufresne (18):
media: doc: Document dual use of H.264 pic_num/frame_num
media: v4l2-mem2mem: Trace on implicit un-hold
media: h264: Avoid wrapping long_term_frame_idx
media: h264: Use v4l2_h264_reference for reflist
media: h264: Increase reference lists size to 32
media: h264: Store current picture fields
media: h264: Store all fields into the unordered list
media: v4l2: Trace calculated p/b0/b1 initial reflist
media: h264: Sort p/b reflist using frame_num
media: v4l2: Reorder field reflist
media: rkvdec: Stop overclocking the decoder
media: rkvdec: h264: Fix dpb_valid implementation
media: rkvdec: Move H264 SPS validation in rkvdec-h264
media: rkvdec-h264: Add field decoding support
media: rkvdec: Enable capture buffer holding for H264
media: hantro: Stop using H.264 parameter pic_num
media: hantro: Add H.264 field decoding support
media: hantro: Enable HOLD_CAPTURE_BUF for H.264
Sebastian Fricke (1):
media: videobuf2-v4l2: Warn on holding buffers without support
.../media/v4l/ext-ctrls-codec-stateless.rst | 10 +-
.../media/common/videobuf2/videobuf2-v4l2.c | 7 +-
.../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +-
drivers/media/v4l2-core/v4l2-h264.c | 261 ++++++++++++++----
drivers/media/v4l2-core/v4l2-mem2mem.c | 1 +
.../staging/media/hantro/hantro_g1_h264_dec.c | 38 +--
drivers/staging/media/hantro/hantro_h264.c | 119 ++++++--
drivers/staging/media/hantro/hantro_hw.h | 7 +-
drivers/staging/media/hantro/hantro_v4l2.c | 25 ++
.../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++----
drivers/staging/media/rkvdec/rkvdec-h264.c | 154 ++++++++---
drivers/staging/media/rkvdec/rkvdec.c | 35 +--
drivers/staging/media/rkvdec/rkvdec.h | 2 +
include/media/v4l2-h264.h | 31 ++-
14 files changed, 580 insertions(+), 225 deletions(-)
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 6:31 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 1 reply; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
These two fields need documentation as they have dual meaning. It is also
confusing since pic_num is a derived value from frame_num, so this should
help application developers. If we ever need to make a V2 of this API, I
would suggest to remove pic_num entirely.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
.../media/v4l/ext-ctrls-codec-stateless.rst | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
index 6541e4c32b26..49f89b702068 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
@@ -649,10 +649,16 @@ Stateless Codec Control ID
:c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
* - __u32
- ``pic_num``
- -
+ - For short term references, this should match the derived value PicNum
+ (8-28) and for long term references it should match the derived value
+ LongTermPicNum (8-29). Note that pic_num is the same as FrameNumWrap
+ for frame decoding.
* - __u16
- ``frame_num``
- -
+ - For short term references, this should match the frame_num value from
+ the slice header syntax (the driver will wrap the value if neeeded). For
+ long term references, this should be set to the value of
+ long_term_frame_idx described in the dec_ref_pic_marking() syntax.
* - __u8
- ``fields``
- Specifies how the DPB entry is referenced. See :ref:`Reference Fields <h264_ref_fields>`
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 03/24] media: videobuf2-v4l2: Warn on holding buffers without support Nicolas Dufresne
` (23 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, Ezequiel Garcia, linux-media
If the timestamp of the src buffer differs from the timestamp of a held
dst buffer, the held buffer is implicitly removed and marked as done.
Add a trace to help debugging if someone hits that case.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/media/v4l2-core/v4l2-mem2mem.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 675e22895ebe..d1d29d6ab024 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -336,6 +336,7 @@ static void __v4l2_m2m_try_queue(struct v4l2_m2m_dev *m2m_dev,
if (src && dst && dst->is_held &&
dst->vb2_buf.copied_timestamp &&
dst->vb2_buf.timestamp != src->vb2_buf.timestamp) {
+ dprintk("Timestamp mismatch, returning held capture buffer\n");
dst->is_held = false;
v4l2_m2m_dst_buf_remove(m2m_ctx);
v4l2_m2m_buf_done(dst, VB2_BUF_STATE_DONE);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 03/24] media: videobuf2-v4l2: Warn on holding buffers without support
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 04/24] media: h264: Avoid wrapping long_term_frame_idx Nicolas Dufresne
` (22 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Tomasz Figa, Marek Szyprowski, Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, Ezequiel Garcia, linux-media
From: Sebastian Fricke <sebastian.fricke@collabora.com>
Using V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF flag without specifying the
subsystem flag VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, results in
silently ignoring it.
Warn the user via a debug print when the flag is requested but ignored
by the videobuf2 framework.
Signed-off-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/media/common/videobuf2/videobuf2-v4l2.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 6edf4508c636..812c8d1962e0 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -329,8 +329,13 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
*/
vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
vbuf->field = b->field;
- if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF))
+ if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF)) {
+ if (vbuf->flags & V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF)
+ dprintk(q, 1,
+ "Request holding buffer (%d), unsupported on output queue\n",
+ b->index);
vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF;
+ }
} else {
/* Zero any output buffer flags as this is a capture buffer */
vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 04/24] media: h264: Avoid wrapping long_term_frame_idx
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (2 preceding siblings ...)
2022-04-05 20:44 ` [PATCH v3 03/24] media: videobuf2-v4l2: Warn on holding buffers without support Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
` (21 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, Ezequiel Garcia, linux-media
For long term references, frame_num is set to long_term_frame_idx which
does not require wrapping. This is fixed by observation, no directly
related issue have been found yet.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/media/v4l2-core/v4l2-h264.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index 5633a242520a..ac47519a9fbe 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -57,8 +57,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
* '8.2.4.1 Decoding process for picture numbers' of the spec.
* TODO: This logic will have to be adjusted when we start
* supporting interlaced content.
+ * For long term references, frame_num is set to
+ * long_term_frame_idx which requires no wrapping.
*/
- if (dpb[i].frame_num > cur_frame_num)
+ if (!b->refs[i].longterm && dpb[i].frame_num > cur_frame_num)
b->refs[i].frame_num = (int)dpb[i].frame_num -
max_frame_num;
else
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 04/24] media: h264: Avoid wrapping long_term_frame_idx Nicolas Dufresne
` (22 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
In preparation for adding field decoding support, convert the byte arrays
for reflist into array of struct v4l2_h264_reference. That struct will
allow us to mark which field of the reference picture is being referenced.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
.../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
.../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
drivers/staging/media/hantro/hantro_hw.h | 6 +-
.../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
include/media/v4l2-h264.h | 19 ++--
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
index 43542de11e9c..72c599e05a47 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
@@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
/*
* The firmware expects unused reflist entries to have the value 0x20.
*/
-static void fixup_ref_list(u8 *ref_list, size_t num_valid)
+static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
{
- memset(&ref_list[num_valid], 0x20, 32 - num_valid);
+ u32 i;
+
+ /* FIXME mark the reference parity */
+ for (i = 0; i < b->num_valid; i++)
+ ref_list[i] = b->index;
+
+ for (; i < 32; i++)
+ ref_list[i] = 0x20;
}
static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
@@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
/* Adapt the built lists to the firmware's expectations */
- fixup_ref_list(p0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b1_reflist, reflist_builder.num_valid);
+ get_ref_list(p0_reflist, reflist_builder);
+ get_ref_list(b0_reflist, reflist_builder);
+ get_ref_list(b1_reflist, reflist_builder);
memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
sizeof(inst->vsi_ctx.h264_slice_params));
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index ac47519a9fbe..afbfcf78efe4 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
pic_order_count = dpb[i].top_field_order_cnt;
b->refs[i].pic_order_count = pic_order_count;
- b->unordered_reflist[b->num_valid] = i;
+ b->unordered_reflist[b->num_valid].index = i;
b->num_valid++;
}
for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
- b->unordered_reflist[i] = i;
+ b->unordered_reflist[i].index = i;
}
EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
@@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
const struct v4l2_h264_reflist_builder *builder = data;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist)
+ struct v4l2_h264_reference *reflist)
{
memcpy(reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
@@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist)
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist)
{
memcpy(b0_reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
index f49dbfb8a843..9de7f05eff2a 100644
--- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
@@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
int reg_num;
u32 reg;
@@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 0; i < 15; i += 3) {
- reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
+ reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
}
@@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
* of forward and backward reference picture lists and first 4 entries
* of P forward picture list.
*/
- reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
- G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
+ reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
/*
@@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
- reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
+ reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
}
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index ed018e293ba0..2bc6b8f088f5 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
* @b1: B1 reflist
*/
struct hantro_h264_dec_reflists {
- u8 p[HANTRO_H264_DPB_SIZE];
- u8 b0[HANTRO_H264_DPB_SIZE];
- u8 b1[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
};
/**
diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
index 64a6330475eb..46c1a83bcc4e 100644
--- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
+++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
@@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
u32 reg;
int i;
@@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
b1_reflist = ctx->h264_dec.reflists.b1;
p_reflist = ctx->h264_dec.reflists.p;
- reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
- VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
- VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
- VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
- VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
- VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
+ reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
+ VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
+ VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
+ VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
+ VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
+ VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
- reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
- VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
- VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
- VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
- VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
- VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
+ reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
+ VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
+ VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
+ VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
+ VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
+ VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
@@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
- reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
- VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
- VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
- VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
- VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
- VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
- reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
- VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
- VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
- VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
- VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
- VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
- reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
- VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
- VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
- VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
- reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
- VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
- VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
- VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
- VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
- VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
- reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
- VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
- VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
- VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
- VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
- VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
- reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
- VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
- VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
- VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
- reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
- VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
- VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
- VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
+ reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
+ VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
+ VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
+ VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 951e19231da2..3c7f3d87fab4 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
#define RKVDEC_H264_DPB_SIZE 16
struct rkvdec_h264_reflists {
- u8 p[RKVDEC_H264_DPB_SIZE];
- u8 b0[RKVDEC_H264_DPB_SIZE];
- u8 b1[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
u8 num_valid;
};
@@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
switch (j) {
case 0:
- idx = h264_ctx->reflists.p[i];
+ idx = h264_ctx->reflists.p[i].index;
break;
case 1:
- idx = h264_ctx->reflists.b0[i];
+ idx = h264_ctx->reflists.b0[i].index;
break;
case 2:
- idx = h264_ctx->reflists.b1[i];
+ idx = h264_ctx->reflists.b1[i].index;
break;
}
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 4b1c71c935e0..ef9a894e3c32 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
s32 cur_pic_order_count;
- u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
+ struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
u8 num_valid;
};
@@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist);
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist);
/**
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist);
+ struct v4l2_h264_reference *reflist);
#endif /* _MEDIA_V4L2_H264_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
In preparation for adding field decoding support, convert the byte arrays
for reflist into array of struct v4l2_h264_reference. That struct will
allow us to mark which field of the reference picture is being referenced.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
.../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
.../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
drivers/staging/media/hantro/hantro_hw.h | 6 +-
.../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
include/media/v4l2-h264.h | 19 ++--
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
index 43542de11e9c..72c599e05a47 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
@@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
/*
* The firmware expects unused reflist entries to have the value 0x20.
*/
-static void fixup_ref_list(u8 *ref_list, size_t num_valid)
+static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
{
- memset(&ref_list[num_valid], 0x20, 32 - num_valid);
+ u32 i;
+
+ /* FIXME mark the reference parity */
+ for (i = 0; i < b->num_valid; i++)
+ ref_list[i] = b->index;
+
+ for (; i < 32; i++)
+ ref_list[i] = 0x20;
}
static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
@@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
/* Adapt the built lists to the firmware's expectations */
- fixup_ref_list(p0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b1_reflist, reflist_builder.num_valid);
+ get_ref_list(p0_reflist, reflist_builder);
+ get_ref_list(b0_reflist, reflist_builder);
+ get_ref_list(b1_reflist, reflist_builder);
memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
sizeof(inst->vsi_ctx.h264_slice_params));
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index ac47519a9fbe..afbfcf78efe4 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
pic_order_count = dpb[i].top_field_order_cnt;
b->refs[i].pic_order_count = pic_order_count;
- b->unordered_reflist[b->num_valid] = i;
+ b->unordered_reflist[b->num_valid].index = i;
b->num_valid++;
}
for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
- b->unordered_reflist[i] = i;
+ b->unordered_reflist[i].index = i;
}
EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
@@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
const struct v4l2_h264_reflist_builder *builder = data;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist)
+ struct v4l2_h264_reference *reflist)
{
memcpy(reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
@@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist)
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist)
{
memcpy(b0_reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
index f49dbfb8a843..9de7f05eff2a 100644
--- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
@@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
int reg_num;
u32 reg;
@@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 0; i < 15; i += 3) {
- reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
+ reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
}
@@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
* of forward and backward reference picture lists and first 4 entries
* of P forward picture list.
*/
- reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
- G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
+ reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
/*
@@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
- reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
+ reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
}
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index ed018e293ba0..2bc6b8f088f5 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
* @b1: B1 reflist
*/
struct hantro_h264_dec_reflists {
- u8 p[HANTRO_H264_DPB_SIZE];
- u8 b0[HANTRO_H264_DPB_SIZE];
- u8 b1[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
};
/**
diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
index 64a6330475eb..46c1a83bcc4e 100644
--- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
+++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
@@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
u32 reg;
int i;
@@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
b1_reflist = ctx->h264_dec.reflists.b1;
p_reflist = ctx->h264_dec.reflists.p;
- reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
- VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
- VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
- VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
- VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
- VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
+ reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
+ VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
+ VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
+ VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
+ VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
+ VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
- reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
- VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
- VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
- VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
- VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
- VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
+ reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
+ VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
+ VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
+ VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
+ VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
+ VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
@@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
- reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
- VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
- VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
- VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
- VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
- VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
- reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
- VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
- VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
- VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
- VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
- VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
- reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
- VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
- VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
- VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
- reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
- VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
- VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
- VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
- VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
- VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
- reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
- VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
- VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
- VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
- VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
- VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
- reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
- VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
- VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
- VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
- reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
- VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
- VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
- VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
+ reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
+ VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
+ VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
+ VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 951e19231da2..3c7f3d87fab4 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
#define RKVDEC_H264_DPB_SIZE 16
struct rkvdec_h264_reflists {
- u8 p[RKVDEC_H264_DPB_SIZE];
- u8 b0[RKVDEC_H264_DPB_SIZE];
- u8 b1[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
u8 num_valid;
};
@@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
switch (j) {
case 0:
- idx = h264_ctx->reflists.p[i];
+ idx = h264_ctx->reflists.p[i].index;
break;
case 1:
- idx = h264_ctx->reflists.b0[i];
+ idx = h264_ctx->reflists.b0[i].index;
break;
case 2:
- idx = h264_ctx->reflists.b1[i];
+ idx = h264_ctx->reflists.b1[i].index;
break;
}
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 4b1c71c935e0..ef9a894e3c32 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
s32 cur_pic_order_count;
- u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
+ struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
u8 num_valid;
};
@@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist);
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist);
/**
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist);
+ struct v4l2_h264_reference *reflist);
#endif /* _MEDIA_V4L2_H264_H */
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
In preparation for adding field decoding support, convert the byte arrays
for reflist into array of struct v4l2_h264_reference. That struct will
allow us to mark which field of the reference picture is being referenced.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
.../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
.../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
drivers/staging/media/hantro/hantro_hw.h | 6 +-
.../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
include/media/v4l2-h264.h | 19 ++--
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
index 43542de11e9c..72c599e05a47 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
@@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
/*
* The firmware expects unused reflist entries to have the value 0x20.
*/
-static void fixup_ref_list(u8 *ref_list, size_t num_valid)
+static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
{
- memset(&ref_list[num_valid], 0x20, 32 - num_valid);
+ u32 i;
+
+ /* FIXME mark the reference parity */
+ for (i = 0; i < b->num_valid; i++)
+ ref_list[i] = b->index;
+
+ for (; i < 32; i++)
+ ref_list[i] = 0x20;
}
static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
@@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
/* Adapt the built lists to the firmware's expectations */
- fixup_ref_list(p0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b1_reflist, reflist_builder.num_valid);
+ get_ref_list(p0_reflist, reflist_builder);
+ get_ref_list(b0_reflist, reflist_builder);
+ get_ref_list(b1_reflist, reflist_builder);
memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
sizeof(inst->vsi_ctx.h264_slice_params));
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index ac47519a9fbe..afbfcf78efe4 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
pic_order_count = dpb[i].top_field_order_cnt;
b->refs[i].pic_order_count = pic_order_count;
- b->unordered_reflist[b->num_valid] = i;
+ b->unordered_reflist[b->num_valid].index = i;
b->num_valid++;
}
for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
- b->unordered_reflist[i] = i;
+ b->unordered_reflist[i].index = i;
}
EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
@@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
const struct v4l2_h264_reflist_builder *builder = data;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist)
+ struct v4l2_h264_reference *reflist)
{
memcpy(reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
@@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist)
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist)
{
memcpy(b0_reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
index f49dbfb8a843..9de7f05eff2a 100644
--- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
@@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
int reg_num;
u32 reg;
@@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 0; i < 15; i += 3) {
- reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
+ reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
}
@@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
* of forward and backward reference picture lists and first 4 entries
* of P forward picture list.
*/
- reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
- G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
+ reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
/*
@@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
- reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
+ reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
}
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index ed018e293ba0..2bc6b8f088f5 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
* @b1: B1 reflist
*/
struct hantro_h264_dec_reflists {
- u8 p[HANTRO_H264_DPB_SIZE];
- u8 b0[HANTRO_H264_DPB_SIZE];
- u8 b1[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
};
/**
diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
index 64a6330475eb..46c1a83bcc4e 100644
--- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
+++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
@@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
u32 reg;
int i;
@@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
b1_reflist = ctx->h264_dec.reflists.b1;
p_reflist = ctx->h264_dec.reflists.p;
- reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
- VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
- VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
- VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
- VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
- VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
+ reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
+ VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
+ VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
+ VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
+ VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
+ VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
- reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
- VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
- VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
- VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
- VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
- VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
+ reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
+ VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
+ VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
+ VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
+ VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
+ VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
@@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
- reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
- VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
- VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
- VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
- VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
- VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
- reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
- VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
- VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
- VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
- VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
- VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
- reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
- VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
- VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
- VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
- reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
- VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
- VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
- VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
- VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
- VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
- reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
- VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
- VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
- VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
- VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
- VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
- reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
- VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
- VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
- VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
- reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
- VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
- VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
- VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
+ reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
+ VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
+ VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
+ VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 951e19231da2..3c7f3d87fab4 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
#define RKVDEC_H264_DPB_SIZE 16
struct rkvdec_h264_reflists {
- u8 p[RKVDEC_H264_DPB_SIZE];
- u8 b0[RKVDEC_H264_DPB_SIZE];
- u8 b1[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
u8 num_valid;
};
@@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
switch (j) {
case 0:
- idx = h264_ctx->reflists.p[i];
+ idx = h264_ctx->reflists.p[i].index;
break;
case 1:
- idx = h264_ctx->reflists.b0[i];
+ idx = h264_ctx->reflists.b0[i].index;
break;
case 2:
- idx = h264_ctx->reflists.b1[i];
+ idx = h264_ctx->reflists.b1[i].index;
break;
}
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 4b1c71c935e0..ef9a894e3c32 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
s32 cur_pic_order_count;
- u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
+ struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
u8 num_valid;
};
@@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist);
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist);
/**
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist);
+ struct v4l2_h264_reference *reflist);
#endif /* _MEDIA_V4L2_H264_H */
--
2.34.1
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
In preparation for adding field decoding support, convert the byte arrays
for reflist into array of struct v4l2_h264_reference. That struct will
allow us to mark which field of the reference picture is being referenced.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
.../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
.../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
drivers/staging/media/hantro/hantro_hw.h | 6 +-
.../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
include/media/v4l2-h264.h | 19 ++--
7 files changed, 116 insertions(+), 107 deletions(-)
diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
index 43542de11e9c..72c599e05a47 100644
--- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
+++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
@@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
/*
* The firmware expects unused reflist entries to have the value 0x20.
*/
-static void fixup_ref_list(u8 *ref_list, size_t num_valid)
+static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
{
- memset(&ref_list[num_valid], 0x20, 32 - num_valid);
+ u32 i;
+
+ /* FIXME mark the reference parity */
+ for (i = 0; i < b->num_valid; i++)
+ ref_list[i] = b->index;
+
+ for (; i < 32; i++)
+ ref_list[i] = 0x20;
}
static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
@@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
/* Adapt the built lists to the firmware's expectations */
- fixup_ref_list(p0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b0_reflist, reflist_builder.num_valid);
- fixup_ref_list(b1_reflist, reflist_builder.num_valid);
+ get_ref_list(p0_reflist, reflist_builder);
+ get_ref_list(b0_reflist, reflist_builder);
+ get_ref_list(b1_reflist, reflist_builder);
memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
sizeof(inst->vsi_ctx.h264_slice_params));
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index ac47519a9fbe..afbfcf78efe4 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
pic_order_count = dpb[i].top_field_order_cnt;
b->refs[i].pic_order_count = pic_order_count;
- b->unordered_reflist[b->num_valid] = i;
+ b->unordered_reflist[b->num_valid].index = i;
b->num_valid++;
}
for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
- b->unordered_reflist[i] = i;
+ b->unordered_reflist[i].index = i;
}
EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
@@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
const struct v4l2_h264_reflist_builder *builder = data;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
s32 poca, pocb;
u8 idxa, idxb;
- idxa = *((u8 *)ptra);
- idxb = *((u8 *)ptrb);
+ idxa = ((struct v4l2_h264_reference *)ptra)->index;
+ idxb = ((struct v4l2_h264_reference *)ptrb)->index;
if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
idxb >= V4L2_H264_NUM_DPB_ENTRIES))
@@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist)
+ struct v4l2_h264_reference *reflist)
{
memcpy(reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
@@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist)
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist)
{
memcpy(b0_reflist, builder->unordered_reflist,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
index f49dbfb8a843..9de7f05eff2a 100644
--- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
+++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
@@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
int reg_num;
u32 reg;
@@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 0; i < 15; i += 3) {
- reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
- G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
+ reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
+ G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
}
@@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
* of forward and backward reference picture lists and first 4 entries
* of P forward picture list.
*/
- reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
- G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
- G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
+ reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
+ G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
/*
@@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
*/
reg_num = 0;
for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
- reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
- G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
+ reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
+ G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
}
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index ed018e293ba0..2bc6b8f088f5 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
* @b1: B1 reflist
*/
struct hantro_h264_dec_reflists {
- u8 p[HANTRO_H264_DPB_SIZE];
- u8 b0[HANTRO_H264_DPB_SIZE];
- u8 b1[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
};
/**
diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
index 64a6330475eb..46c1a83bcc4e 100644
--- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
+++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
@@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
static void set_ref(struct hantro_ctx *ctx)
{
- const u8 *b0_reflist, *b1_reflist, *p_reflist;
+ const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
struct hantro_dev *vpu = ctx->dev;
u32 reg;
int i;
@@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
b1_reflist = ctx->h264_dec.reflists.b1;
p_reflist = ctx->h264_dec.reflists.p;
- reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
- VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
- VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
- VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
- VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
- VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
+ reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
+ VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
+ VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
+ VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
+ VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
+ VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
- reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
- VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
- VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
- VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
- VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
- VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
+ reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
+ VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
+ VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
+ VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
+ VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
+ VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
@@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
- reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
- VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
- VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
- VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
- VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
- VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
- reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
- VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
- VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
- VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
- VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
- VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
- reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
- VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
- VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
- VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
- reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
- VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
- VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
- VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
- VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
- VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
+ reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
+ VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
+ VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
+ VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
+ VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
+ VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
- reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
- VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
- VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
- VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
- VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
- VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
+ reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
+ VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
+ VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
+ VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
+ VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
+ VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
- reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
- VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
- VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
- VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
+ reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
+ VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
+ VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
+ VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
- reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
- VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
- VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
- VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
+ reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
+ VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
+ VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
+ VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 951e19231da2..3c7f3d87fab4 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
#define RKVDEC_H264_DPB_SIZE 16
struct rkvdec_h264_reflists {
- u8 p[RKVDEC_H264_DPB_SIZE];
- u8 b0[RKVDEC_H264_DPB_SIZE];
- u8 b1[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
u8 num_valid;
};
@@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
switch (j) {
case 0:
- idx = h264_ctx->reflists.p[i];
+ idx = h264_ctx->reflists.p[i].index;
break;
case 1:
- idx = h264_ctx->reflists.b0[i];
+ idx = h264_ctx->reflists.b0[i].index;
break;
case 2:
- idx = h264_ctx->reflists.b1[i];
+ idx = h264_ctx->reflists.b1[i].index;
break;
}
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 4b1c71c935e0..ef9a894e3c32 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
s32 cur_pic_order_count;
- u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
+ struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
u8 num_valid;
};
@@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
- * is an index in the DPB
- * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
- * is an index in the DPB
+ * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * is a v4l2_h264_reference structure
+ * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
* in section '8.2.4 Decoding process for reference picture lists construction'
@@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
*/
void
v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
- u8 *b0_reflist, u8 *b1_reflist);
+ struct v4l2_h264_reference *b0_reflist,
+ struct v4l2_h264_reference *b1_reflist);
/**
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16-bytes array used to store the P reference list. Each entry
- * is an index in the DPB
+ * @reflist: 16 sized array used to store the P reference list. Each entry
+ * is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
* section '8.2.4 Decoding process for reference picture lists construction'
@@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
*/
void
v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
- u8 *reflist);
+ struct v4l2_h264_reference *reflist);
#endif /* _MEDIA_V4L2_H264_H */
--
2.34.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 06/24] media: h264: Increase reference lists size to 32
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
This is to accommodate support for field decoding, which splits the top
and the bottom references into the reference list.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/media/v4l2-core/v4l2-h264.c | 6 +++---
drivers/staging/media/hantro/hantro_hw.h | 6 +++---
drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++---
include/media/v4l2-h264.h | 8 ++++----
4 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index afbfcf78efe4..4b46b36526c0 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16 sized array used to store the P reference list. Each entry
+ * @reflist: 32 sized array used to store the P reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
@@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry
* is a v4l2_h264_reference structure
- * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 2bc6b8f088f5..292aaaabaf24 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
* @b1: B1 reflist
*/
struct hantro_h264_dec_reflists {
- struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
- struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
- struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN];
};
/**
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 3c7f3d87fab4..dff89732ddd0 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
#define RKVDEC_H264_DPB_SIZE 16
struct rkvdec_h264_reflists {
- struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
- struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
- struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN];
u8 num_valid;
};
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index ef9a894e3c32..e282fb16ac58 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
s32 cur_pic_order_count;
- struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
+ struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN];
u8 num_valid;
};
@@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry
* is a v4l2_h264_reference structure
- * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
@@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16 sized array used to store the P reference list. Each entry
+ * @reflist: 32 sized array used to store the P reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 06/24] media: h264: Increase reference lists size to 32
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
This is to accommodate support for field decoding, which splits the top
and the bottom references into the reference list.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/media/v4l2-core/v4l2-h264.c | 6 +++---
drivers/staging/media/hantro/hantro_hw.h | 6 +++---
drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++---
include/media/v4l2-h264.h | 8 ++++----
4 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index afbfcf78efe4..4b46b36526c0 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16 sized array used to store the P reference list. Each entry
+ * @reflist: 32 sized array used to store the P reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
@@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry
* is a v4l2_h264_reference structure
- * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 2bc6b8f088f5..292aaaabaf24 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
* @b1: B1 reflist
*/
struct hantro_h264_dec_reflists {
- struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
- struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
- struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN];
};
/**
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 3c7f3d87fab4..dff89732ddd0 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
#define RKVDEC_H264_DPB_SIZE 16
struct rkvdec_h264_reflists {
- struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
- struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
- struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
+ struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN];
+ struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN];
u8 num_valid;
};
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index ef9a894e3c32..e282fb16ac58 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
s32 cur_pic_order_count;
- struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
+ struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN];
u8 num_valid;
};
@@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
* v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
*
* @builder: reference list builder context
- * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
+ * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry
* is a v4l2_h264_reference structure
- * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
+ * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the B0/B1 reference lists. This procedure is described
@@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
* @builder: reference list builder context
- * @reflist: 16 sized array used to store the P reference list. Each entry
+ * @reflist: 32 sized array used to store the P reference list. Each entry
* is a v4l2_h264_reference structure
*
* This functions builds the P reference lists. This procedure is describe in
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 07/24] media: h264: Store current picture fields
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (5 preceding siblings ...)
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 08/24] media: h264: Store all fields into the unordered list Nicolas Dufresne
` (18 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
This information, also called picture structure, is required in field
decoding mode to construct reference lists.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/media/v4l2-core/v4l2-h264.c | 10 +++++++---
include/media/v4l2-h264.h | 4 ++++
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index 4b46b36526c0..58f18bb0afb6 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -34,13 +34,17 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
cur_frame_num = dec_params->frame_num;
memset(b, 0, sizeof(*b));
- if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC))
+ if (!(dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC)) {
b->cur_pic_order_count = min(dec_params->bottom_field_order_cnt,
dec_params->top_field_order_cnt);
- else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD)
+ b->cur_pic_fields = V4L2_H264_FRAME_REF;
+ } else if (dec_params->flags & V4L2_H264_DECODE_PARAM_FLAG_BOTTOM_FIELD) {
b->cur_pic_order_count = dec_params->bottom_field_order_cnt;
- else
+ b->cur_pic_fields = V4L2_H264_BOTTOM_FIELD_REF;
+ } else {
b->cur_pic_order_count = dec_params->top_field_order_cnt;
+ b->cur_pic_fields = V4L2_H264_TOP_FIELD_REF;
+ }
for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) {
u32 pic_order_count;
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index e282fb16ac58..e165a54c68fa 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -21,6 +21,7 @@
* @refs.longterm: set to true for a long term reference
* @refs: array of references
* @cur_pic_order_count: picture order count of the frame being decoded
+ * @cur_pic_fields: fields present in the frame being decoded
* @unordered_reflist: unordered list of references. Will be used to generate
* ordered P/B0/B1 lists
* @num_valid: number of valid references in the refs array
@@ -36,7 +37,10 @@ struct v4l2_h264_reflist_builder {
u32 pic_num;
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
+
s32 cur_pic_order_count;
+ u8 cur_pic_fields;
+
struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN];
u8 num_valid;
};
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 08/24] media: h264: Store all fields into the unordered list
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (6 preceding siblings ...)
2022-04-05 20:44 ` [PATCH v3 07/24] media: h264: Store current picture fields Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist Nicolas Dufresne
` (17 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
When the current picture is a field, store each field into the
unordered_list and preserve both top and bottom picture order
count.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/media/v4l2-core/v4l2-h264.c | 65 +++++++++++++++++++++--------
include/media/v4l2-h264.h | 6 ++-
2 files changed, 51 insertions(+), 20 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index 58f18bb0afb6..38d8dbda0045 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -47,8 +47,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
}
for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) {
- u32 pic_order_count;
-
if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
continue;
@@ -59,8 +57,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
/*
* Handle frame_num wraparound as described in section
* '8.2.4.1 Decoding process for picture numbers' of the spec.
- * TODO: This logic will have to be adjusted when we start
- * supporting interlaced content.
* For long term references, frame_num is set to
* long_term_frame_idx which requires no wrapping.
*/
@@ -70,17 +66,33 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
else
b->refs[i].frame_num = dpb[i].frame_num;
- if (dpb[i].fields == V4L2_H264_FRAME_REF)
- pic_order_count = min(dpb[i].top_field_order_cnt,
- dpb[i].bottom_field_order_cnt);
- else if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF)
- pic_order_count = dpb[i].bottom_field_order_cnt;
- else
- pic_order_count = dpb[i].top_field_order_cnt;
+ b->refs[i].top_field_order_cnt = dpb[i].top_field_order_cnt;
+ b->refs[i].bottom_field_order_cnt = dpb[i].bottom_field_order_cnt;
+
+ if (b->cur_pic_fields == V4L2_H264_FRAME_REF) {
+ u8 fields = V4L2_H264_FRAME_REF;
+
+ b->unordered_reflist[b->num_valid].index = i;
+ b->unordered_reflist[b->num_valid].fields = fields;
+ b->num_valid++;
+ continue;
+ }
+
+ if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) {
+ u8 fields = V4L2_H264_TOP_FIELD_REF;
+
+ b->unordered_reflist[b->num_valid].index = i;
+ b->unordered_reflist[b->num_valid].fields = fields;
+ b->num_valid++;
+ }
- b->refs[i].pic_order_count = pic_order_count;
- b->unordered_reflist[b->num_valid].index = i;
- b->num_valid++;
+ if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) {
+ u8 fields = V4L2_H264_BOTTOM_FIELD_REF;
+
+ b->unordered_reflist[b->num_valid].index = i;
+ b->unordered_reflist[b->num_valid].fields = fields;
+ b->num_valid++;
+ }
}
for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
@@ -88,6 +100,23 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
}
EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
+static s32 v4l2_h264_get_poc(const struct v4l2_h264_reflist_builder *b,
+ const struct v4l2_h264_reference *ref)
+{
+ switch (ref->fields) {
+ case V4L2_H264_FRAME_REF:
+ return min(b->refs[ref->index].top_field_order_cnt,
+ b->refs[ref->index].bottom_field_order_cnt);
+ case V4L2_H264_TOP_FIELD_REF:
+ return b->refs[ref->index].top_field_order_cnt;
+ case V4L2_H264_BOTTOM_FIELD_REF:
+ return b->refs[ref->index].bottom_field_order_cnt;
+ }
+
+ /* not reached */
+ return 0;
+}
+
static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
const void *data)
{
@@ -150,8 +179,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
builder->refs[idxb].pic_num ?
-1 : 1;
- poca = builder->refs[idxa].pic_order_count;
- pocb = builder->refs[idxb].pic_order_count;
+ poca = v4l2_h264_get_poc(builder, ptra);
+ pocb = v4l2_h264_get_poc(builder, ptrb);
/*
* Short term pics with POC < cur POC first in POC descending order
@@ -195,8 +224,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
builder->refs[idxb].pic_num ?
-1 : 1;
- poca = builder->refs[idxa].pic_order_count;
- pocb = builder->refs[idxb].pic_order_count;
+ poca = v4l2_h264_get_poc(builder, ptra);
+ pocb = v4l2_h264_get_poc(builder, ptrb);
/*
* Short term pics with POC > cur POC first in POC ascending order
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index e165a54c68fa..4cef717b3f18 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -15,7 +15,8 @@
/**
* struct v4l2_h264_reflist_builder - Reference list builder object
*
- * @refs.pic_order_count: reference picture order count
+ * @refs.top_field_order_cnt: top field order count
+ * @refs.bottom_field_order_cnt: bottom field order count
* @refs.frame_num: reference frame number
* @refs.pic_num: reference picture number
* @refs.longterm: set to true for a long term reference
@@ -32,7 +33,8 @@
*/
struct v4l2_h264_reflist_builder {
struct {
- s32 pic_order_count;
+ s32 top_field_order_cnt;
+ s32 bottom_field_order_cnt;
int frame_num;
u32 pic_num;
u16 longterm : 1;
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (7 preceding siblings ...)
2022-04-05 20:44 ` [PATCH v3 08/24] media: h264: Store all fields into the unordered list Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 7:26 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 10/24] media: h264: Sort p/b reflist using frame_num Nicolas Dufresne
` (16 subsequent siblings)
25 siblings, 1 reply; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
Add debug print statements to print the content of P & B reference
lists, to verify that the ordering of the generated reference lists is
correct. This is especially important for the field decoding mode,
where sorting is more complex.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Tested-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/media/v4l2-core/v4l2-h264.c | 86 +++++++++++++++++++++++++++++
1 file changed, 86 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index 38d8dbda0045..bcf9b7774560 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -241,6 +241,87 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
return poca < pocb ? -1 : 1;
}
+static char ref_type_to_char (u8 ref_type)
+{
+ switch (ref_type) {
+ case V4L2_H264_FRAME_REF:
+ return 'f';
+ case V4L2_H264_TOP_FIELD_REF:
+ return 't';
+ case V4L2_H264_BOTTOM_FIELD_REF:
+ return 'b';
+ }
+
+ return '?';
+}
+
+static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
+ struct v4l2_h264_reference *reflist,
+ char *out_str, const int len)
+{
+ int n = 0, i;
+
+ n += snprintf(out_str + n, len - n, "|");
+
+ for (i = 0; i < builder->num_valid; i++) {
+ /* this is pic_num for frame and frame_num (wrapped) for field,
+ * but for frame pic_num is equal to frame_num (wrapped).
+ */
+ int frame_num = builder->refs[reflist[i].index].frame_num;
+ bool longterm = builder->refs[reflist[i].index].longterm;
+
+ n += scnprintf(out_str + n, len - n, "%i%c%c|",
+ frame_num, longterm ? 'l' : 's',
+ ref_type_to_char (reflist[i].fields));
+ }
+
+ return out_str;
+}
+
+static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
+ struct v4l2_h264_reference *reflist)
+{
+ char buf[1024];
+
+ pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n",
+ builder->cur_pic_order_count,
+ ref_type_to_char(builder->cur_pic_fields),
+ format_ref_list_p(builder, reflist, buf, sizeof(buf)));
+}
+
+static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
+ struct v4l2_h264_reference *reflist,
+ char *out_str, const int len)
+{
+ int n = 0, i;
+
+ n += snprintf(out_str + n, len - n, "|");
+
+ for (i = 0; i < builder->num_valid; i++) {
+ int frame_num = builder->refs[reflist[i].index].frame_num;
+ u32 poc = v4l2_h264_get_poc(builder, reflist + i);
+ bool longterm = builder->refs[reflist[i].index].longterm;
+
+ n += scnprintf(out_str + n, len - n, "%i%c%c|",
+ longterm ? frame_num : poc,
+ longterm ? 'l' : 's',
+ ref_type_to_char(reflist[i].fields));
+ }
+
+ return out_str;
+}
+
+static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
+ struct v4l2_h264_reference *reflist, u8 list_num)
+{
+ char buf[1024];
+
+ pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s",
+ list_num, builder->cur_pic_order_count,
+ ref_type_to_char (builder->cur_pic_fields),
+ format_ref_list_b(builder, reflist, buf, sizeof(buf)));
+}
+
/**
* v4l2_h264_build_p_ref_list() - Build the P reference list
*
@@ -261,6 +342,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
sizeof(builder->unordered_reflist[0]) * builder->num_valid);
sort_r(reflist, builder->num_valid, sizeof(*reflist),
v4l2_h264_p_ref_list_cmp, NULL, builder);
+
+ print_ref_list_p(builder, reflist);
}
EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
@@ -296,6 +379,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
if (builder->num_valid > 1 &&
!memcmp(b1_reflist, b0_reflist, builder->num_valid))
swap(b1_reflist[0], b1_reflist[1]);
+
+ print_ref_list_b(builder, b0_reflist, 0);
+ print_ref_list_b(builder, b1_reflist, 1);
}
EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 10/24] media: h264: Sort p/b reflist using frame_num
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (8 preceding siblings ...)
2022-04-05 20:44 ` [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 7:29 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 11/24] media: v4l2: Reorder field reflist Nicolas Dufresne
` (15 subsequent siblings)
25 siblings, 1 reply; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
In the reference list builder, frame_num refers to FrameNumWrap
in the spec, which is the same as the pic_num for frame decoding.
The same applies for long_term_pic_num and long_term_frame_idx.
Sort all type of references by frame_num so the sort can be reused
for fields reflist were the sorting is done using frame_num instead.
In short, pic_num is never actually used for building reference
lists.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/media/v4l2-core/v4l2-h264.c | 23 +++++++++++++----------
include/media/v4l2-h264.h | 2 --
2 files changed, 13 insertions(+), 12 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index bcf9b7774560..7e1eba03099a 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -50,7 +50,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
continue;
- b->refs[i].pic_num = dpb[i].pic_num;
if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
b->refs[i].longterm = true;
@@ -139,15 +138,19 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
}
/*
- * Short term pics in descending pic num order, long term ones in
- * ascending order.
+ * For frames, short term pics are in descending pic num order and long
+ * term ones in ascending order. For fields, the same direction is used
+ * but with frame_num (wrapped). For frames, the value of pic_num and
+ * frame_num are the same (see formula (8-28) and (8-29)). For this
+ * reason we can use frame_num only and share this funciton between
+ * frames and fields reflist.
*/
if (!builder->refs[idxa].longterm)
return builder->refs[idxb].frame_num <
builder->refs[idxa].frame_num ?
-1 : 1;
- return builder->refs[idxa].pic_num < builder->refs[idxb].pic_num ?
+ return builder->refs[idxa].frame_num < builder->refs[idxb].frame_num ?
-1 : 1;
}
@@ -173,10 +176,10 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
return 1;
}
- /* Long term pics in ascending pic num order. */
+ /* Long term pics in ascending frame num order. */
if (builder->refs[idxa].longterm)
- return builder->refs[idxa].pic_num <
- builder->refs[idxb].pic_num ?
+ return builder->refs[idxa].frame_num <
+ builder->refs[idxb].frame_num ?
-1 : 1;
poca = v4l2_h264_get_poc(builder, ptra);
@@ -218,10 +221,10 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
return 1;
}
- /* Long term pics in ascending pic num order. */
+ /* Long term pics in ascending frame num order. */
if (builder->refs[idxa].longterm)
- return builder->refs[idxa].pic_num <
- builder->refs[idxb].pic_num ?
+ return builder->refs[idxa].frame_num <
+ builder->refs[idxb].frame_num ?
-1 : 1;
poca = v4l2_h264_get_poc(builder, ptra);
diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
index 4cef717b3f18..0d9eaa956123 100644
--- a/include/media/v4l2-h264.h
+++ b/include/media/v4l2-h264.h
@@ -18,7 +18,6 @@
* @refs.top_field_order_cnt: top field order count
* @refs.bottom_field_order_cnt: bottom field order count
* @refs.frame_num: reference frame number
- * @refs.pic_num: reference picture number
* @refs.longterm: set to true for a long term reference
* @refs: array of references
* @cur_pic_order_count: picture order count of the frame being decoded
@@ -36,7 +35,6 @@ struct v4l2_h264_reflist_builder {
s32 top_field_order_cnt;
s32 bottom_field_order_cnt;
int frame_num;
- u32 pic_num;
u16 longterm : 1;
} refs[V4L2_H264_NUM_DPB_ENTRIES];
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 11/24] media: v4l2: Reorder field reflist
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (9 preceding siblings ...)
2022-04-05 20:44 ` [PATCH v3 10/24] media: h264: Sort p/b reflist using frame_num Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 7:31 ` Hans Verkuil
2022-04-05 20:44 ` Nicolas Dufresne
` (14 subsequent siblings)
25 siblings, 1 reply; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab; +Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
As per spec, the field reflist requires interleaving top and bottom
field in a specific way that does not fit inside the sort operation.
The process consist of alternating references parity, starting with
a reference of the same parity as the current picture. This
processs is done twice, once for short term references and a second
time for the long term references.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/media/v4l2-core/v4l2-h264.c | 42 +++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
index 7e1eba03099a..d2d1f3bed7e8 100644
--- a/drivers/media/v4l2-core/v4l2-h264.c
+++ b/drivers/media/v4l2-core/v4l2-h264.c
@@ -244,6 +244,40 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
return poca < pocb ? -1 : 1;
}
+/*
+ * The references need to be reordered so that references are alternating
+ * between top and bottom field references starting with the current picture
+ * parity. This has to be * done for short term and long term references
+ * separately.
+ */
+static void reorder_field_reflist(const struct v4l2_h264_reflist_builder *b,
+ struct v4l2_h264_reference *reflist)
+{
+ struct v4l2_h264_reference tmplist[V4L2_H264_REF_LIST_LEN];
+ u8 lt, i = 0, j = 0, k = 0;
+
+ memcpy(tmplist, reflist, sizeof(tmplist[0]) * b->num_valid);
+
+ for (lt = 0; lt <= 1; lt++) {
+ do {
+ for (; i < b->num_valid && b->refs[tmplist[i].index].longterm == lt; i++) {
+ if (tmplist[i].fields == b->cur_pic_fields) {
+ reflist[k++] = tmplist[i++];
+ break;
+ }
+ }
+
+ for (; j < b->num_valid && b->refs[tmplist[j].index].longterm == lt; j++) {
+ if (tmplist[j].fields != b->cur_pic_fields) {
+ reflist[k++] = tmplist[j++];
+ break;
+ }
+ }
+ } while ((i < b->num_valid && b->refs[tmplist[i].index].longterm == lt) ||
+ (j < b->num_valid && b->refs[tmplist[j].index].longterm == lt));
+ }
+}
+
static char ref_type_to_char (u8 ref_type)
{
switch (ref_type) {
@@ -346,6 +380,9 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
sort_r(reflist, builder->num_valid, sizeof(*reflist),
v4l2_h264_p_ref_list_cmp, NULL, builder);
+ if (builder->cur_pic_fields != V4L2_H264_FRAME_REF)
+ reorder_field_reflist(builder, reflist);
+
print_ref_list_p(builder, reflist);
}
EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
@@ -379,6 +416,11 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
sort_r(b1_reflist, builder->num_valid, sizeof(*b1_reflist),
v4l2_h264_b1_ref_list_cmp, NULL, builder);
+ if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) {
+ reorder_field_reflist(builder, b0_reflist);
+ reorder_field_reflist(builder, b1_reflist);
+ }
+
if (builder->num_valid > 1 &&
!memcmp(b1_reflist, b0_reflist, builder->num_valid))
swap(b1_reflist[0], b1_reflist[1]);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 12/24] media: rkvdec: Stop overclocking the decoder
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman,
Boris Brezillon, Hans Verkuil
Cc: kernel, linux-kernel, Sebastian Fricke, Mauro Carvalho Chehab,
linux-media, linux-rockchip, linux-staging
While this overclock hack seems to work on some implementations
(some ChromeBooks, RockPi4) it also causes instability on other
implementations (notably LibreComputer Renegade, but there were more
reports in the LibreELEC project, where this has been removed). While
performance is indeed affected (tested with GStreamer), 4K playback
still works as long as you don't operate in lock step and keep at
least 1 frame ahead of time in the decode queue.
After discussion with ChromeOS members, it would seem that their
implementation indeed used to synchronously decode each frames, so
this hack was simply compensating for their code being less
efficient. In my opinion, this hack should not have been included
upstream.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/staging/media/rkvdec/rkvdec.c | 6 ------
1 file changed, 6 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index c0cf3488f970..2df8cf4883e2 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev)
if (ret)
return ret;
- /*
- * Bump ACLK to max. possible freq. (500 MHz) to improve performance
- * When 4k video playback.
- */
- clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000);
-
rkvdec->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(rkvdec->regs))
return PTR_ERR(rkvdec->regs);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 12/24] media: rkvdec: Stop overclocking the decoder
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman,
Boris Brezillon, Hans Verkuil
Cc: kernel, linux-kernel, Sebastian Fricke, Mauro Carvalho Chehab,
linux-media, linux-rockchip, linux-staging
While this overclock hack seems to work on some implementations
(some ChromeBooks, RockPi4) it also causes instability on other
implementations (notably LibreComputer Renegade, but there were more
reports in the LibreELEC project, where this has been removed). While
performance is indeed affected (tested with GStreamer), 4K playback
still works as long as you don't operate in lock step and keep at
least 1 frame ahead of time in the decode queue.
After discussion with ChromeOS members, it would seem that their
implementation indeed used to synchronously decode each frames, so
this hack was simply compensating for their code being less
efficient. In my opinion, this hack should not have been included
upstream.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/staging/media/rkvdec/rkvdec.c | 6 ------
1 file changed, 6 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index c0cf3488f970..2df8cf4883e2 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev)
if (ret)
return ret;
- /*
- * Bump ACLK to max. possible freq. (500 MHz) to improve performance
- * When 4k video playback.
- */
- clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000);
-
rkvdec->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(rkvdec->regs))
return PTR_ERR(rkvdec->regs);
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 13/24] media: rkvdec: h264: Fix dpb_valid implementation
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman,
Hans Verkuil, Boris Brezillon
Cc: kernel, linux-kernel, Sebastian Fricke, Mauro Carvalho Chehab,
linux-media, linux-rockchip, linux-staging
The ref builder only provided references that are marked as valid in the
dpb. Thus the current implementation of dpb_valid would always set the
flag to 1. This is not representing missing frames (this is called
'non-existing' pictures in the spec). In some context, these non-existing
pictures still need to occupy a slot in the reference list according to
the spec.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------
1 file changed, 24 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index dff89732ddd0..bcde37d72244 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -112,6 +112,7 @@ struct rkvdec_h264_run {
const struct v4l2_ctrl_h264_sps *sps;
const struct v4l2_ctrl_h264_pps *pps;
const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix;
+ int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES];
};
struct rkvdec_h264_ctx {
@@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
}
}
+static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx,
+ struct rkvdec_h264_run *run)
+{
+ const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
+ u32 i;
+
+ for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) {
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb;
+ struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
+ int buf_idx = -1;
+
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
+ buf_idx = vb2_find_timestamp(cap_q,
+ dpb[i].reference_ts, 0);
+
+ run->ref_buf_idx[i] = buf_idx;
+ }
+}
+
static void assemble_hw_rps(struct rkvdec_ctx *ctx,
struct rkvdec_h264_run *run)
{
@@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
for (j = 0; j < RKVDEC_NUM_REFLIST; j++) {
for (i = 0; i < h264_ctx->reflists.num_valid; i++) {
- u8 dpb_valid = 0;
+ bool dpb_valid = run->ref_buf_idx[i] >= 0;
u8 idx = 0;
switch (j) {
@@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
if (idx >= ARRAY_SIZE(dec_params->dpb))
continue;
- dpb_valid = !!(dpb[idx].flags &
- V4L2_H264_DPB_ENTRY_FLAG_ACTIVE);
set_ps_field(hw_rps, DPB_INFO(i, j),
idx | dpb_valid << 4);
@@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run,
unsigned int dpb_idx)
{
struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
- const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb;
struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
- int buf_idx = -1;
-
- if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
- buf_idx = vb2_find_timestamp(cap_q,
- dpb[dpb_idx].reference_ts, 0);
+ int buf_idx = run->ref_buf_idx[dpb_idx];
/*
* If a DPB entry is unused or invalid, address of current destination
@@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
assemble_hw_scaling_list(ctx, &run);
assemble_hw_pps(ctx, &run);
+ lookup_ref_buf_idx(ctx, &run);
assemble_hw_rps(ctx, &run);
config_registers(ctx, &run);
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 13/24] media: rkvdec: h264: Fix dpb_valid implementation
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman,
Hans Verkuil, Boris Brezillon
Cc: kernel, linux-kernel, Sebastian Fricke, Mauro Carvalho Chehab,
linux-media, linux-rockchip, linux-staging
The ref builder only provided references that are marked as valid in the
dpb. Thus the current implementation of dpb_valid would always set the
flag to 1. This is not representing missing frames (this is called
'non-existing' pictures in the spec). In some context, these non-existing
pictures still need to occupy a slot in the reference list according to
the spec.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------
1 file changed, 24 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index dff89732ddd0..bcde37d72244 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -112,6 +112,7 @@ struct rkvdec_h264_run {
const struct v4l2_ctrl_h264_sps *sps;
const struct v4l2_ctrl_h264_pps *pps;
const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix;
+ int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES];
};
struct rkvdec_h264_ctx {
@@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
}
}
+static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx,
+ struct rkvdec_h264_run *run)
+{
+ const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
+ u32 i;
+
+ for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) {
+ struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
+ const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb;
+ struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
+ int buf_idx = -1;
+
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
+ buf_idx = vb2_find_timestamp(cap_q,
+ dpb[i].reference_ts, 0);
+
+ run->ref_buf_idx[i] = buf_idx;
+ }
+}
+
static void assemble_hw_rps(struct rkvdec_ctx *ctx,
struct rkvdec_h264_run *run)
{
@@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
for (j = 0; j < RKVDEC_NUM_REFLIST; j++) {
for (i = 0; i < h264_ctx->reflists.num_valid; i++) {
- u8 dpb_valid = 0;
+ bool dpb_valid = run->ref_buf_idx[i] >= 0;
u8 idx = 0;
switch (j) {
@@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
if (idx >= ARRAY_SIZE(dec_params->dpb))
continue;
- dpb_valid = !!(dpb[idx].flags &
- V4L2_H264_DPB_ENTRY_FLAG_ACTIVE);
set_ps_field(hw_rps, DPB_INFO(i, j),
idx | dpb_valid << 4);
@@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run,
unsigned int dpb_idx)
{
struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx;
- const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb;
struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
- int buf_idx = -1;
-
- if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
- buf_idx = vb2_find_timestamp(cap_q,
- dpb[dpb_idx].reference_ts, 0);
+ int buf_idx = run->ref_buf_idx[dpb_idx];
/*
* If a DPB entry is unused or invalid, address of current destination
@@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
assemble_hw_scaling_list(ctx, &run);
assemble_hw_pps(ctx, &run);
+ lookup_ref_buf_idx(ctx, &run);
assemble_hw_rps(ctx, &run);
config_registers(ctx, &run);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 14/24] media: rkvdec: h264: Fix bit depth wrap in pps packet
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman,
Hans Verkuil, Boris Brezillon
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Mauro Carvalho Chehab, linux-media, linux-rockchip,
linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
The luma and chroma bit depth fields in the pps packet are 3 bits wide.
8 is wrongly added to the bit depth values written to these 3 bit fields.
Because only the 3 LSB are written, the hardware was configured
correctly.
Correct this by not adding 8 to the luma and chroma bit depth value.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index bcde37d72244..8d44a884a52e 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
WRITE_PPS(0xff, PROFILE_IDC);
WRITE_PPS(1, CONSTRAINT_SET3_FLAG);
WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC);
- WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA);
- WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA);
+ WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA);
+ WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA);
WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG);
WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4);
WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES);
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 14/24] media: rkvdec: h264: Fix bit depth wrap in pps packet
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman,
Hans Verkuil, Boris Brezillon
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Mauro Carvalho Chehab, linux-media, linux-rockchip,
linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
The luma and chroma bit depth fields in the pps packet are 3 bits wide.
8 is wrongly added to the bit depth values written to these 3 bit fields.
Because only the 3 LSB are written, the hardware was configured
correctly.
Correct this by not adding 8 to the luma and chroma bit depth value.
Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index bcde37d72244..8d44a884a52e 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
WRITE_PPS(0xff, PROFILE_IDC);
WRITE_PPS(1, CONSTRAINT_SET3_FLAG);
WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC);
- WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA);
- WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA);
+ WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA);
+ WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA);
WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG);
WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4);
WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging
No function change, this moves H264 specific validation into the H264
specific code. This is in preparation of improving this validation and
reusing at streamone.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++
drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++----------------
drivers/staging/media/rkvdec/rkvdec.h | 1 +
3 files changed, 30 insertions(+), 17 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 8d44a884a52e..0dcbcb1bac80 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
return 0;
}
+static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
+{
+ if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
+ const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
+ /*
+ * TODO: The hardware supports 10-bit and 4:2:2 profiles,
+ * but it's currently broken in the driver.
+ * Reject them for now, until it's fixed.
+ */
+ if (sps->chroma_format_idc > 1)
+ /* Only 4:0:0 and 4:2:0 are supported */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
+ /* Luma and chroma bit depth mismatch */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != 0)
+ /* Only 8-bit is supported */
+ return -EINVAL;
+ }
+ return 0;
+}
+
const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
.adjust_fmt = rkvdec_h264_adjust_fmt,
.start = rkvdec_h264_start,
.stop = rkvdec_h264_stop,
.run = rkvdec_h264_run,
+ .try_ctrl = rkvdec_h264_try_ctrl,
};
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 2df8cf4883e2..e3d44d5b35f3 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -29,23 +29,12 @@
static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
{
- if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
- const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
- /*
- * TODO: The hardware supports 10-bit and 4:2:2 profiles,
- * but it's currently broken in the driver.
- * Reject them for now, until it's fixed.
- */
- if (sps->chroma_format_idc > 1)
- /* Only 4:0:0 and 4:2:0 are supported */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
- /* Luma and chroma bit depth mismatch */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
- return -EINVAL;
- }
+ struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+ if (desc->ops->try_ctrl)
+ return desc->ops->try_ctrl(ctx, ctrl);
+
return 0;
}
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 2f4ea1786b93..9df0fba799a4 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops {
void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf,
struct vb2_v4l2_buffer *dst_buf,
enum vb2_buffer_state result);
+ int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
};
struct rkvdec_coded_fmt_desc {
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging
No function change, this moves H264 specific validation into the H264
specific code. This is in preparation of improving this validation and
reusing at streamone.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++
drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++----------------
drivers/staging/media/rkvdec/rkvdec.h | 1 +
3 files changed, 30 insertions(+), 17 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 8d44a884a52e..0dcbcb1bac80 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
return 0;
}
+static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
+{
+ if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
+ const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
+ /*
+ * TODO: The hardware supports 10-bit and 4:2:2 profiles,
+ * but it's currently broken in the driver.
+ * Reject them for now, until it's fixed.
+ */
+ if (sps->chroma_format_idc > 1)
+ /* Only 4:0:0 and 4:2:0 are supported */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
+ /* Luma and chroma bit depth mismatch */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != 0)
+ /* Only 8-bit is supported */
+ return -EINVAL;
+ }
+ return 0;
+}
+
const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
.adjust_fmt = rkvdec_h264_adjust_fmt,
.start = rkvdec_h264_start,
.stop = rkvdec_h264_stop,
.run = rkvdec_h264_run,
+ .try_ctrl = rkvdec_h264_try_ctrl,
};
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 2df8cf4883e2..e3d44d5b35f3 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -29,23 +29,12 @@
static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
{
- if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
- const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
- /*
- * TODO: The hardware supports 10-bit and 4:2:2 profiles,
- * but it's currently broken in the driver.
- * Reject them for now, until it's fixed.
- */
- if (sps->chroma_format_idc > 1)
- /* Only 4:0:0 and 4:2:0 are supported */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
- /* Luma and chroma bit depth mismatch */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
- return -EINVAL;
- }
+ struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
+ const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
+
+ if (desc->ops->try_ctrl)
+ return desc->ops->try_ctrl(ctx, ctrl);
+
return 0;
}
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 2f4ea1786b93..9df0fba799a4 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops {
void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf,
struct vb2_v4l2_buffer *dst_buf,
enum vb2_buffer_state result);
+ int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
};
struct rkvdec_coded_fmt_desc {
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
The width and height in macroblocks is currently configured based on OUTPUT
buffer resolution, this works for frame pictures but can cause issues for
field pictures.
When frame_mbs_only_flag is 0 the height in mbs should be height of
the field instead of height of frame.
Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
against OUTPUT buffer resolution and use these values to configure HW.
The validation is happening in both try_ctrt() and start() since it is
otherwise possible to trick the driver during initialization by changing
the OUTPUT format after having set a valid control.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
1 file changed, 59 insertions(+), 19 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 0dcbcb1bac80..f081b476340f 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
- WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
- WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
+
+ /* Use the SPS values since they are already in macroblocks
+ * dimensions, height can be field height (halved) if
+ * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
+ * decoding smaller images into larger allocation which can be used
+ * to implementing SVC spatial layer support.
+ */
+ WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
+ WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
+
WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
FRAME_MBS_ONLY_FLAG);
WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
@@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
return 0;
}
+static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
+ const struct v4l2_ctrl_h264_sps *sps)
+{
+ unsigned int width, height;
+ /*
+ * TODO: The hardware supports 10-bit and 4:2:2 profiles,
+ * but it's currently broken in the driver.
+ * Reject them for now, until it's fixed.
+ */
+ if (sps->chroma_format_idc > 1)
+ /* Only 4:0:0 and 4:2:0 are supported */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
+ /* Luma and chroma bit depth mismatch */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != 0)
+ /* Only 8-bit is supported */
+ return -EINVAL;
+
+ width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
+ height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
+
+ /* when frame_mbs_only_flag is not set, this is field height,
+ * which is half the final height (see (7-18) in the
+ * specification)
+ */
+ if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
+ height *= 2;
+
+ if (width > ctx->coded_fmt.fmt.pix_mp.width ||
+ height > ctx->coded_fmt.fmt.pix_mp.height)
+ return -EINVAL;
+
+ return 0;
+}
+
static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
{
struct rkvdec_dev *rkvdec = ctx->dev;
struct rkvdec_h264_priv_tbl *priv_tbl;
struct rkvdec_h264_ctx *h264_ctx;
+ struct v4l2_ctrl *ctrl;
int ret;
+ ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+ V4L2_CID_STATELESS_H264_SPS);
+ if (!ctrl)
+ return -EINVAL;
+
+ ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
+ if (ret)
+ return ret;
+
h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
if (!h264_ctx)
return -ENOMEM;
@@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
{
- if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
- const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
- /*
- * TODO: The hardware supports 10-bit and 4:2:2 profiles,
- * but it's currently broken in the driver.
- * Reject them for now, until it's fixed.
- */
- if (sps->chroma_format_idc > 1)
- /* Only 4:0:0 and 4:2:0 are supported */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
- /* Luma and chroma bit depth mismatch */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
- return -EINVAL;
- }
+ if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
+ return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
+
return 0;
}
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
The width and height in macroblocks is currently configured based on OUTPUT
buffer resolution, this works for frame pictures but can cause issues for
field pictures.
When frame_mbs_only_flag is 0 the height in mbs should be height of
the field instead of height of frame.
Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
against OUTPUT buffer resolution and use these values to configure HW.
The validation is happening in both try_ctrt() and start() since it is
otherwise possible to trick the driver during initialization by changing
the OUTPUT format after having set a valid control.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
1 file changed, 59 insertions(+), 19 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 0dcbcb1bac80..f081b476340f 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
- WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
- WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
+
+ /* Use the SPS values since they are already in macroblocks
+ * dimensions, height can be field height (halved) if
+ * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
+ * decoding smaller images into larger allocation which can be used
+ * to implementing SVC spatial layer support.
+ */
+ WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
+ WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
+
WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
FRAME_MBS_ONLY_FLAG);
WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
@@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
return 0;
}
+static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
+ const struct v4l2_ctrl_h264_sps *sps)
+{
+ unsigned int width, height;
+ /*
+ * TODO: The hardware supports 10-bit and 4:2:2 profiles,
+ * but it's currently broken in the driver.
+ * Reject them for now, until it's fixed.
+ */
+ if (sps->chroma_format_idc > 1)
+ /* Only 4:0:0 and 4:2:0 are supported */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
+ /* Luma and chroma bit depth mismatch */
+ return -EINVAL;
+ if (sps->bit_depth_luma_minus8 != 0)
+ /* Only 8-bit is supported */
+ return -EINVAL;
+
+ width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
+ height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
+
+ /* when frame_mbs_only_flag is not set, this is field height,
+ * which is half the final height (see (7-18) in the
+ * specification)
+ */
+ if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
+ height *= 2;
+
+ if (width > ctx->coded_fmt.fmt.pix_mp.width ||
+ height > ctx->coded_fmt.fmt.pix_mp.height)
+ return -EINVAL;
+
+ return 0;
+}
+
static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
{
struct rkvdec_dev *rkvdec = ctx->dev;
struct rkvdec_h264_priv_tbl *priv_tbl;
struct rkvdec_h264_ctx *h264_ctx;
+ struct v4l2_ctrl *ctrl;
int ret;
+ ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
+ V4L2_CID_STATELESS_H264_SPS);
+ if (!ctrl)
+ return -EINVAL;
+
+ ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
+ if (ret)
+ return ret;
+
h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
if (!h264_ctx)
return -ENOMEM;
@@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
{
- if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
- const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
- /*
- * TODO: The hardware supports 10-bit and 4:2:2 profiles,
- * but it's currently broken in the driver.
- * Reject them for now, until it's fixed.
- */
- if (sps->chroma_format_idc > 1)
- /* Only 4:0:0 and 4:2:0 are supported */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
- /* Luma and chroma bit depth mismatch */
- return -EINVAL;
- if (sps->bit_depth_luma_minus8 != 0)
- /* Only 8-bit is supported */
- return -EINVAL;
- }
+ if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
+ return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
+
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
When decoding the second field in a complementary field pair the second
field is sharing the same frame_num with the first field.
Currently the frame_num for the first field is wrapped when it matches the
field being decoded, this cause issues to decode the second field in a
complementary field pair.
Fix this by using inclusive comparison, less than or equal.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index f081b476340f..60eaf06b6e25 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
continue;
if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
- dpb[i].frame_num < dec_params->frame_num) {
+ dpb[i].frame_num <= dec_params->frame_num) {
p[i] = dpb[i].frame_num;
continue;
}
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
When decoding the second field in a complementary field pair the second
field is sharing the same frame_num with the first field.
Currently the frame_num for the first field is wrapped when it matches the
field being decoded, this cause issues to decode the second field in a
complementary field pair.
Fix this by using inclusive comparison, less than or equal.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index f081b476340f..60eaf06b6e25 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
continue;
if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
- dpb[i].frame_num < dec_params->frame_num) {
+ dpb[i].frame_num <= dec_params->frame_num) {
p[i] = dpb[i].frame_num;
continue;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 18/24] media: rkvdec: Ensure decoded resolution fit coded resolution
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Sebastian Fricke,
linux-media, linux-rockchip, linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
Ensure decoded CAPTURE buffer resolution is larger or equal to the coded
OUTPUT buffer resolution.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index e3d44d5b35f3..2bc4b1a40989 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -257,6 +257,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
pix_mp->pixelformat = coded_desc->decoded_fmts[0];
/* Always apply the frmsize constraint of the coded end. */
+ pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
+ pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height);
v4l2_apply_frmsize_constraints(&pix_mp->width,
&pix_mp->height,
&coded_desc->frmsize);
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 18/24] media: rkvdec: Ensure decoded resolution fit coded resolution
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Sebastian Fricke,
linux-media, linux-rockchip, linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
Ensure decoded CAPTURE buffer resolution is larger or equal to the coded
OUTPUT buffer resolution.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index e3d44d5b35f3..2bc4b1a40989 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -257,6 +257,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv,
pix_mp->pixelformat = coded_desc->decoded_fmts[0];
/* Always apply the frmsize constraint of the coded end. */
+ pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width);
+ pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height);
v4l2_apply_frmsize_constraints(&pix_mp->width,
&pix_mp->height,
&coded_desc->frmsize);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 19/24] media: rkvdec-h264: Add field decoding support
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
This makes use of the new feature in the reference builder to program
up to 32 references when doing field decoding. It also signals the
parity (top or bottom) of the field to the hardware.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------
1 file changed, 21 insertions(+), 27 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 60eaf06b6e25..ef74a556c916 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl {
u8 err_info[RKV_ERROR_INFO_SIZE];
};
-#define RKVDEC_H264_DPB_SIZE 16
-
struct rkvdec_h264_reflists {
struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN];
struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN];
struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN];
- u8 num_valid;
};
struct rkvdec_h264_run {
@@ -746,23 +743,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx,
struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
int buf_idx = -1;
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) {
buf_idx = vb2_find_timestamp(cap_q,
dpb[i].reference_ts, 0);
+ if (buf_idx < 0)
+ pr_debug("No buffer for reference_ts %llu",
+ dpb[i].reference_ts);
+ }
run->ref_buf_idx[i] = buf_idx;
}
}
static void assemble_hw_rps(struct rkvdec_ctx *ctx,
+ struct v4l2_h264_reflist_builder *builder,
struct rkvdec_h264_run *run)
{
const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb;
struct rkvdec_h264_ctx *h264_ctx = ctx->priv;
- const struct v4l2_ctrl_h264_sps *sps = run->sps;
struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu;
- u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4);
u32 *hw_rps = priv_tbl->rps;
u32 i, j;
@@ -780,37 +780,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
continue;
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
- dpb[i].frame_num <= dec_params->frame_num) {
- p[i] = dpb[i].frame_num;
- continue;
- }
-
- p[i] = dpb[i].frame_num - max_frame_num;
+ p[i] = builder->refs[i].frame_num;
}
for (j = 0; j < RKVDEC_NUM_REFLIST; j++) {
- for (i = 0; i < h264_ctx->reflists.num_valid; i++) {
- bool dpb_valid = run->ref_buf_idx[i] >= 0;
- u8 idx = 0;
+ for (i = 0; i < builder->num_valid; i++) {
+ struct v4l2_h264_reference *ref;
+ bool dpb_valid;
+ bool bottom;
switch (j) {
case 0:
- idx = h264_ctx->reflists.p[i].index;
+ ref = &h264_ctx->reflists.p[i];
break;
case 1:
- idx = h264_ctx->reflists.b0[i].index;
+ ref = &h264_ctx->reflists.b0[i];
break;
case 2:
- idx = h264_ctx->reflists.b1[i].index;
+ ref = &h264_ctx->reflists.b1[i];
break;
}
- if (idx >= ARRAY_SIZE(dec_params->dpb))
+ if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb)))
continue;
+ dpb_valid = run->ref_buf_idx[ref->index] >= 0;
+ bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF;
+
set_ps_field(hw_rps, DPB_INFO(i, j),
- idx | dpb_valid << 4);
+ ref->index | dpb_valid << 4);
+ set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom);
}
}
}
@@ -998,10 +997,6 @@ static void config_registers(struct rkvdec_ctx *ctx,
rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15);
}
- /*
- * Since support frame mode only
- * top_field_order_cnt is the same as bottom_field_order_cnt
- */
reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt);
writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0);
@@ -1163,7 +1158,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
/* Build the P/B{0,1} ref lists. */
v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params,
run.sps, run.decode_params->dpb);
- h264_ctx->reflists.num_valid = reflist_builder.num_valid;
v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p);
v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0,
h264_ctx->reflists.b1);
@@ -1171,7 +1165,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
assemble_hw_scaling_list(ctx, &run);
assemble_hw_pps(ctx, &run);
lookup_ref_buf_idx(ctx, &run);
- assemble_hw_rps(ctx, &run);
+ assemble_hw_rps(ctx, &reflist_builder, &run);
config_registers(ctx, &run);
rkvdec_run_postamble(ctx, &run.base);
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 19/24] media: rkvdec-h264: Add field decoding support
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
This makes use of the new feature in the reference builder to program
up to 32 references when doing field decoding. It also signals the
parity (top or bottom) of the field to the hardware.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------
1 file changed, 21 insertions(+), 27 deletions(-)
diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
index 60eaf06b6e25..ef74a556c916 100644
--- a/drivers/staging/media/rkvdec/rkvdec-h264.c
+++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
@@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl {
u8 err_info[RKV_ERROR_INFO_SIZE];
};
-#define RKVDEC_H264_DPB_SIZE 16
-
struct rkvdec_h264_reflists {
struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN];
struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN];
struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN];
- u8 num_valid;
};
struct rkvdec_h264_run {
@@ -746,23 +743,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx,
struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q;
int buf_idx = -1;
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) {
buf_idx = vb2_find_timestamp(cap_q,
dpb[i].reference_ts, 0);
+ if (buf_idx < 0)
+ pr_debug("No buffer for reference_ts %llu",
+ dpb[i].reference_ts);
+ }
run->ref_buf_idx[i] = buf_idx;
}
}
static void assemble_hw_rps(struct rkvdec_ctx *ctx,
+ struct v4l2_h264_reflist_builder *builder,
struct rkvdec_h264_run *run)
{
const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params;
const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb;
struct rkvdec_h264_ctx *h264_ctx = ctx->priv;
- const struct v4l2_ctrl_h264_sps *sps = run->sps;
struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu;
- u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4);
u32 *hw_rps = priv_tbl->rps;
u32 i, j;
@@ -780,37 +780,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
continue;
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
- dpb[i].frame_num <= dec_params->frame_num) {
- p[i] = dpb[i].frame_num;
- continue;
- }
-
- p[i] = dpb[i].frame_num - max_frame_num;
+ p[i] = builder->refs[i].frame_num;
}
for (j = 0; j < RKVDEC_NUM_REFLIST; j++) {
- for (i = 0; i < h264_ctx->reflists.num_valid; i++) {
- bool dpb_valid = run->ref_buf_idx[i] >= 0;
- u8 idx = 0;
+ for (i = 0; i < builder->num_valid; i++) {
+ struct v4l2_h264_reference *ref;
+ bool dpb_valid;
+ bool bottom;
switch (j) {
case 0:
- idx = h264_ctx->reflists.p[i].index;
+ ref = &h264_ctx->reflists.p[i];
break;
case 1:
- idx = h264_ctx->reflists.b0[i].index;
+ ref = &h264_ctx->reflists.b0[i];
break;
case 2:
- idx = h264_ctx->reflists.b1[i].index;
+ ref = &h264_ctx->reflists.b1[i];
break;
}
- if (idx >= ARRAY_SIZE(dec_params->dpb))
+ if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb)))
continue;
+ dpb_valid = run->ref_buf_idx[ref->index] >= 0;
+ bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF;
+
set_ps_field(hw_rps, DPB_INFO(i, j),
- idx | dpb_valid << 4);
+ ref->index | dpb_valid << 4);
+ set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom);
}
}
}
@@ -998,10 +997,6 @@ static void config_registers(struct rkvdec_ctx *ctx,
rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15);
}
- /*
- * Since support frame mode only
- * top_field_order_cnt is the same as bottom_field_order_cnt
- */
reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt);
writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0);
@@ -1163,7 +1158,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
/* Build the P/B{0,1} ref lists. */
v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params,
run.sps, run.decode_params->dpb);
- h264_ctx->reflists.num_valid = reflist_builder.num_valid;
v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p);
v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0,
h264_ctx->reflists.b1);
@@ -1171,7 +1165,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
assemble_hw_scaling_list(ctx, &run);
assemble_hw_pps(ctx, &run);
lookup_ref_buf_idx(ctx, &run);
- assemble_hw_rps(ctx, &run);
+ assemble_hw_rps(ctx, &reflist_builder, &run);
config_registers(ctx, &run);
rkvdec_run_postamble(ctx, &run.base);
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 20/24] media: rkvdec: Enable capture buffer holding for H264
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
In order to support interlaced video decoding, the driver must
allow holding the capture buffer so that the second field can
be decoded into it.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec.c | 4 ++++
drivers/staging/media/rkvdec/rkvdec.h | 1 +
2 files changed, 5 insertions(+)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 2bc4b1a40989..7bab7586918c 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -127,6 +127,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
.ops = &rkvdec_h264_fmt_ops,
.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
.decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+ .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
},
{
.fourcc = V4L2_PIX_FMT_VP9_FRAME,
@@ -385,6 +386,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv,
cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+ /* Enable format specific queue features */
+ vq->subsystem_flags |= desc->subsystem_flags;
+
return 0;
}
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 9df0fba799a4..633335ebb9c4 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -82,6 +82,7 @@ struct rkvdec_coded_fmt_desc {
const struct rkvdec_coded_fmt_ops *ops;
unsigned int num_decoded_fmts;
const u32 *decoded_fmts;
+ u32 subsystem_flags;
};
struct rkvdec_dev {
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 20/24] media: rkvdec: Enable capture buffer holding for H264
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
In order to support interlaced video decoding, the driver must
allow holding the capture buffer so that the second field can
be decoded into it.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/rkvdec/rkvdec.c | 4 ++++
drivers/staging/media/rkvdec/rkvdec.h | 1 +
2 files changed, 5 insertions(+)
diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
index 2bc4b1a40989..7bab7586918c 100644
--- a/drivers/staging/media/rkvdec/rkvdec.c
+++ b/drivers/staging/media/rkvdec/rkvdec.c
@@ -127,6 +127,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = {
.ops = &rkvdec_h264_fmt_ops,
.num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts),
.decoded_fmts = rkvdec_h264_vp9_decoded_fmts,
+ .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF,
},
{
.fourcc = V4L2_PIX_FMT_VP9_FRAME,
@@ -385,6 +386,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv,
cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc;
cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization;
+ /* Enable format specific queue features */
+ vq->subsystem_flags |= desc->subsystem_flags;
+
return 0;
}
diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
index 9df0fba799a4..633335ebb9c4 100644
--- a/drivers/staging/media/rkvdec/rkvdec.h
+++ b/drivers/staging/media/rkvdec/rkvdec.h
@@ -82,6 +82,7 @@ struct rkvdec_coded_fmt_desc {
const struct rkvdec_coded_fmt_ops *ops;
unsigned int num_decoded_fmts;
const u32 *decoded_fmts;
+ u32 subsystem_flags;
};
struct rkvdec_dev {
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 21/24] media: hantro: Stop using H.264 parameter pic_num
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
The hardware expects FrameNumWrap or long_term_frame_idx. Picture
numbers are per field, and are mostly used during the memory
management process, which is done in userland. This fixes two
ITU conformance tests:
- MR6_BT_B
- MR8_BT_B
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/hantro/hantro_h264.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
index 0b4d2491be3b..228629fb3cdf 100644
--- a/drivers/staging/media/hantro/hantro_h264.c
+++ b/drivers/staging/media/hantro/hantro_h264.c
@@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
return 0;
- if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
- return dpb->pic_num;
return dpb->frame_num;
}
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 21/24] media: hantro: Stop using H.264 parameter pic_num
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
The hardware expects FrameNumWrap or long_term_frame_idx. Picture
numbers are per field, and are mostly used during the memory
management process, which is done in userland. This fixes two
ITU conformance tests:
- MR6_BT_B
- MR8_BT_B
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/hantro/hantro_h264.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
index 0b4d2491be3b..228629fb3cdf 100644
--- a/drivers/staging/media/hantro/hantro_h264.c
+++ b/drivers/staging/media/hantro/hantro_h264.c
@@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
return 0;
- if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
- return dpb->pic_num;
return dpb->frame_num;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 22/24] media: hantro: h264: Make dpb entry management more robust
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
The driver maintains stable slot locations for reference pictures. This
change makes the code more robust by using the reference_ts as key and
by marking all entries invalid right from the start.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/hantro/hantro_h264.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
index 228629fb3cdf..7377fc26f780 100644
--- a/drivers/staging/media/hantro/hantro_h264.c
+++ b/drivers/staging/media/hantro/hantro_h264.c
@@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx)
static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a,
const struct v4l2_h264_dpb_entry *b)
{
- return a->top_field_order_cnt == b->top_field_order_cnt &&
- a->bottom_field_order_cnt == b->bottom_field_order_cnt;
+ return a->reference_ts == b->reference_ts;
}
static void update_dpb(struct hantro_ctx *ctx)
@@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx)
/* Disable all entries by default. */
for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++)
- ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE;
+ ctx->h264_dec.dpb[i].flags = 0;
/* Try to match new DPB entries with existing ones by their POCs. */
for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i];
- if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+ if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID))
continue;
/*
@@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx)
struct v4l2_h264_dpb_entry *cdpb;
cdpb = &ctx->h264_dec.dpb[j];
- if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE ||
- !dpb_entry_match(cdpb, ndpb))
+ if (!dpb_entry_match(cdpb, ndpb))
continue;
*cdpb = *ndpb;
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 22/24] media: hantro: h264: Make dpb entry management more robust
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
From: Jonas Karlman <jonas@kwiboo.se>
The driver maintains stable slot locations for reference pictures. This
change makes the code more robust by using the reference_ts as key and
by marking all entries invalid right from the start.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/hantro/hantro_h264.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
index 228629fb3cdf..7377fc26f780 100644
--- a/drivers/staging/media/hantro/hantro_h264.c
+++ b/drivers/staging/media/hantro/hantro_h264.c
@@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx)
static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a,
const struct v4l2_h264_dpb_entry *b)
{
- return a->top_field_order_cnt == b->top_field_order_cnt &&
- a->bottom_field_order_cnt == b->bottom_field_order_cnt;
+ return a->reference_ts == b->reference_ts;
}
static void update_dpb(struct hantro_ctx *ctx)
@@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx)
/* Disable all entries by default. */
for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++)
- ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE;
+ ctx->h264_dec.dpb[i].flags = 0;
/* Try to match new DPB entries with existing ones by their POCs. */
for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) {
const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i];
- if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
+ if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID))
continue;
/*
@@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx)
struct v4l2_h264_dpb_entry *cdpb;
cdpb = &ctx->h264_dec.dpb[j];
- if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE ||
- !dpb_entry_match(cdpb, ndpb))
+ if (!dpb_entry_match(cdpb, ndpb))
continue;
*cdpb = *ndpb;
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 23/24] media: hantro: Add H.264 field decoding support
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
This adds the required code to support field decoding. While most of
the code is derived from Rockchip and VSI reference code, the
reduction of the reference list to 16 entries was found by
trial and errors. The list consists of all the references with the
opposite field parity.
The strategy is to deduplicate the reference picture that points
to the same storage (same index). The choice of opposite parity has
been made to keep the other field of the current field pair in the
list. This method may not be robust if a field was lost.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/hantro/hantro_h264.c | 107 ++++++++++++++++++---
drivers/staging/media/hantro/hantro_hw.h | 1 +
2 files changed, 94 insertions(+), 14 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
index 7377fc26f780..f6fc939aa726 100644
--- a/drivers/staging/media/hantro/hantro_h264.c
+++ b/drivers/staging/media/hantro/hantro_h264.c
@@ -22,6 +22,11 @@
#define POC_BUFFER_SIZE 34
#define SCALING_LIST_SIZE (6 * 16 + 2 * 64)
+/* For valid and long term reference marking, index are reversed, so bit 31
+ * indicates the status of the picture 0.
+ */
+#define REF_BIT(i) BIT(32 - 1 - (i))
+
/* Data structure describing auxiliary buffer format. */
struct hantro_h264_dec_priv_tbl {
u32 cabac_table[CABAC_INIT_BUFFER_SIZE];
@@ -227,6 +232,7 @@ static void prepare_table(struct hantro_ctx *ctx)
{
const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls;
const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode;
+ const struct v4l2_ctrl_h264_sps *sps = ctrls->sps;
struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu;
const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
u32 dpb_longterm = 0;
@@ -237,20 +243,45 @@ static void prepare_table(struct hantro_ctx *ctx)
tbl->poc[i * 2] = dpb[i].top_field_order_cnt;
tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt;
+ if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID))
+ continue;
+
/*
* Set up bit maps of valid and long term DPBs.
- * NOTE: The bits are reversed, i.e. MSb is DPB 0.
+ * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame
+ * decoding, bit 31 to 15 are used, while for field decoding,
+ * all bits are used, with bit 31 being a top field, 30 a bottom
+ * field and so on.
*/
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
- dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
- dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
+ if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) {
+ if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF)
+ dpb_valid |= REF_BIT(i * 2);
+
+ if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF)
+ dpb_valid |= REF_BIT(i * 2 + 1);
+
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) {
+ dpb_longterm |= REF_BIT(i * 2);
+ dpb_longterm |= REF_BIT(i * 2 + 1);
+ }
+ } else {
+ dpb_valid |= REF_BIT(i);
+
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
+ dpb_longterm |= REF_BIT(i);
+ }
+ }
+ ctx->h264_dec.dpb_valid = dpb_valid;
+ ctx->h264_dec.dpb_longterm = dpb_longterm;
+
+ if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) ||
+ !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) {
+ tbl->poc[32] = ctx->h264_dec.cur_poc;
+ tbl->poc[33] = 0;
+ } else {
+ tbl->poc[32] = dec_param->top_field_order_cnt;
+ tbl->poc[33] = dec_param->bottom_field_order_cnt;
}
- ctx->h264_dec.dpb_valid = dpb_valid << 16;
- ctx->h264_dec.dpb_longterm = dpb_longterm << 16;
-
- tbl->poc[32] = dec_param->top_field_order_cnt;
- tbl->poc[33] = dec_param->bottom_field_order_cnt;
assemble_scaling_list(ctx);
}
@@ -326,6 +357,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
{
struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
dma_addr_t dma_addr = 0;
+ s32 cur_poc = ctx->h264_dec.cur_poc;
+ u32 flags;
if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts);
@@ -343,7 +376,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
dma_addr = hantro_get_dec_buf_addr(ctx, buf);
}
- return dma_addr;
+ flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0;
+ flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) <
+ abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ?
+ 0x1 : 0;
+
+ return dma_addr | flags;
}
u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
@@ -355,6 +393,34 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
return dpb->frame_num;
}
+static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b,
+ struct v4l2_h264_reference *reflist)
+{
+ int write_idx = 0;
+ int i;
+
+ if (b->cur_pic_fields == V4L2_H264_FRAME_REF) {
+ write_idx = b->num_valid;
+ goto done;
+ }
+
+ for (i = 0; i < b->num_valid; i++) {
+ if (!(b->cur_pic_fields == reflist[i].fields)) {
+ reflist[write_idx++] = reflist[i];
+ continue;
+ }
+ }
+
+done:
+ /* Should not happen unless we have a bug in the reflist builder. */
+ if (WARN_ON(write_idx > 16))
+ write_idx = 16;
+
+ /* Clear the remaining, some streams fails otherwise */
+ for (; write_idx < 16; write_idx++)
+ reflist[write_idx].index = 15;
+}
+
int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
{
struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec;
@@ -386,15 +452,28 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
/* Update the DPB with new refs. */
update_dpb(ctx);
- /* Prepare data in memory. */
- prepare_table(ctx);
-
/* Build the P/B{0,1} ref lists. */
v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode,
ctrls->sps, ctx->h264_dec.dpb);
+ h264_ctx->cur_poc = reflist_builder.cur_pic_order_count;
+
+ /* Prepare data in memory. */
+ prepare_table(ctx);
+
v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p);
v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0,
h264_ctx->reflists.b1);
+
+ /* Reduce ref lists to at most 16 entries, Hantro hardware will deduce
+ * the actual picture lists in field through the dpb_valid,
+ * dpb_longterm bitmap along with the current frame parity.
+ */
+ if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) {
+ deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p);
+ deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0);
+ deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1);
+ }
+
return 0;
}
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 292aaaabaf24..fd869369fb97 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx {
struct hantro_h264_dec_ctrls ctrls;
u32 dpb_longterm;
u32 dpb_valid;
+ s32 cur_poc;
};
/**
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 23/24] media: hantro: Add H.264 field decoding support
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
This adds the required code to support field decoding. While most of
the code is derived from Rockchip and VSI reference code, the
reduction of the reference list to 16 entries was found by
trial and errors. The list consists of all the references with the
opposite field parity.
The strategy is to deduplicate the reference picture that points
to the same storage (same index). The choice of opposite parity has
been made to keep the other field of the current field pair in the
list. This method may not be robust if a field was lost.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
---
drivers/staging/media/hantro/hantro_h264.c | 107 ++++++++++++++++++---
drivers/staging/media/hantro/hantro_hw.h | 1 +
2 files changed, 94 insertions(+), 14 deletions(-)
diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
index 7377fc26f780..f6fc939aa726 100644
--- a/drivers/staging/media/hantro/hantro_h264.c
+++ b/drivers/staging/media/hantro/hantro_h264.c
@@ -22,6 +22,11 @@
#define POC_BUFFER_SIZE 34
#define SCALING_LIST_SIZE (6 * 16 + 2 * 64)
+/* For valid and long term reference marking, index are reversed, so bit 31
+ * indicates the status of the picture 0.
+ */
+#define REF_BIT(i) BIT(32 - 1 - (i))
+
/* Data structure describing auxiliary buffer format. */
struct hantro_h264_dec_priv_tbl {
u32 cabac_table[CABAC_INIT_BUFFER_SIZE];
@@ -227,6 +232,7 @@ static void prepare_table(struct hantro_ctx *ctx)
{
const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls;
const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode;
+ const struct v4l2_ctrl_h264_sps *sps = ctrls->sps;
struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu;
const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
u32 dpb_longterm = 0;
@@ -237,20 +243,45 @@ static void prepare_table(struct hantro_ctx *ctx)
tbl->poc[i * 2] = dpb[i].top_field_order_cnt;
tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt;
+ if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID))
+ continue;
+
/*
* Set up bit maps of valid and long term DPBs.
- * NOTE: The bits are reversed, i.e. MSb is DPB 0.
+ * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame
+ * decoding, bit 31 to 15 are used, while for field decoding,
+ * all bits are used, with bit 31 being a top field, 30 a bottom
+ * field and so on.
*/
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
- dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
- if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
- dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
+ if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) {
+ if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF)
+ dpb_valid |= REF_BIT(i * 2);
+
+ if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF)
+ dpb_valid |= REF_BIT(i * 2 + 1);
+
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) {
+ dpb_longterm |= REF_BIT(i * 2);
+ dpb_longterm |= REF_BIT(i * 2 + 1);
+ }
+ } else {
+ dpb_valid |= REF_BIT(i);
+
+ if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
+ dpb_longterm |= REF_BIT(i);
+ }
+ }
+ ctx->h264_dec.dpb_valid = dpb_valid;
+ ctx->h264_dec.dpb_longterm = dpb_longterm;
+
+ if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) ||
+ !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) {
+ tbl->poc[32] = ctx->h264_dec.cur_poc;
+ tbl->poc[33] = 0;
+ } else {
+ tbl->poc[32] = dec_param->top_field_order_cnt;
+ tbl->poc[33] = dec_param->bottom_field_order_cnt;
}
- ctx->h264_dec.dpb_valid = dpb_valid << 16;
- ctx->h264_dec.dpb_longterm = dpb_longterm << 16;
-
- tbl->poc[32] = dec_param->top_field_order_cnt;
- tbl->poc[33] = dec_param->bottom_field_order_cnt;
assemble_scaling_list(ctx);
}
@@ -326,6 +357,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
{
struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
dma_addr_t dma_addr = 0;
+ s32 cur_poc = ctx->h264_dec.cur_poc;
+ u32 flags;
if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts);
@@ -343,7 +376,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
dma_addr = hantro_get_dec_buf_addr(ctx, buf);
}
- return dma_addr;
+ flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0;
+ flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) <
+ abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ?
+ 0x1 : 0;
+
+ return dma_addr | flags;
}
u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
@@ -355,6 +393,34 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
return dpb->frame_num;
}
+static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b,
+ struct v4l2_h264_reference *reflist)
+{
+ int write_idx = 0;
+ int i;
+
+ if (b->cur_pic_fields == V4L2_H264_FRAME_REF) {
+ write_idx = b->num_valid;
+ goto done;
+ }
+
+ for (i = 0; i < b->num_valid; i++) {
+ if (!(b->cur_pic_fields == reflist[i].fields)) {
+ reflist[write_idx++] = reflist[i];
+ continue;
+ }
+ }
+
+done:
+ /* Should not happen unless we have a bug in the reflist builder. */
+ if (WARN_ON(write_idx > 16))
+ write_idx = 16;
+
+ /* Clear the remaining, some streams fails otherwise */
+ for (; write_idx < 16; write_idx++)
+ reflist[write_idx].index = 15;
+}
+
int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
{
struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec;
@@ -386,15 +452,28 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
/* Update the DPB with new refs. */
update_dpb(ctx);
- /* Prepare data in memory. */
- prepare_table(ctx);
-
/* Build the P/B{0,1} ref lists. */
v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode,
ctrls->sps, ctx->h264_dec.dpb);
+ h264_ctx->cur_poc = reflist_builder.cur_pic_order_count;
+
+ /* Prepare data in memory. */
+ prepare_table(ctx);
+
v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p);
v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0,
h264_ctx->reflists.b1);
+
+ /* Reduce ref lists to at most 16 entries, Hantro hardware will deduce
+ * the actual picture lists in field through the dpb_valid,
+ * dpb_longterm bitmap along with the current frame parity.
+ */
+ if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) {
+ deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p);
+ deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0);
+ deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1);
+ }
+
return 0;
}
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 292aaaabaf24..fd869369fb97 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx {
struct hantro_h264_dec_ctrls ctrls;
u32 dpb_longterm;
u32 dpb_valid;
+ s32 cur_poc;
};
/**
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 24/24] media: hantro: Enable HOLD_CAPTURE_BUF for H.264
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
` (24 subsequent siblings)
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
This is needed to optimize field decoding. Each field will be
decoded into the same capture buffer. To be able to queue multiple
buffers, we need to be able to ask the driver to hold the capture
buffer.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index 67148ba346f5..50d636678ff3 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc)
}
}
+static void
+hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc)
+{
+ struct vb2_queue *vq;
+
+ vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+ V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+
+ switch (fourcc) {
+ case V4L2_PIX_FMT_JPEG:
+ case V4L2_PIX_FMT_MPEG2_SLICE:
+ case V4L2_PIX_FMT_VP8_FRAME:
+ case V4L2_PIX_FMT_HEVC_SLICE:
+ case V4L2_PIX_FMT_VP9_FRAME:
+ vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF);
+ break;
+ case V4L2_PIX_FMT_H264_SLICE:
+ vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF;
+ break;
+ default:
+ break;
+ }
+}
+
static int hantro_set_fmt_out(struct hantro_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
@@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
ctx->dst_fmt.quantization = pix_mp->quantization;
hantro_update_requires_request(ctx, pix_mp->pixelformat);
+ hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat);
vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode);
vpu_debug(0, "fmt - w: %d, h: %d\n",
--
2.34.1
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply related [flat|nested] 86+ messages in thread
* [PATCH v3 24/24] media: hantro: Enable HOLD_CAPTURE_BUF for H.264
@ 2022-04-05 20:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-05 20:44 UTC (permalink / raw)
To: Ezequiel Garcia, Philipp Zabel, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media,
linux-rockchip, linux-staging
This is needed to optimize field decoding. Each field will be
decoded into the same capture buffer. To be able to queue multiple
buffers, we need to be able to ask the driver to hold the capture
buffer.
Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
---
drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index 67148ba346f5..50d636678ff3 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc)
}
}
+static void
+hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc)
+{
+ struct vb2_queue *vq;
+
+ vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+ V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE);
+
+ switch (fourcc) {
+ case V4L2_PIX_FMT_JPEG:
+ case V4L2_PIX_FMT_MPEG2_SLICE:
+ case V4L2_PIX_FMT_VP8_FRAME:
+ case V4L2_PIX_FMT_HEVC_SLICE:
+ case V4L2_PIX_FMT_VP9_FRAME:
+ vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF);
+ break;
+ case V4L2_PIX_FMT_H264_SLICE:
+ vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF;
+ break;
+ default:
+ break;
+ }
+}
+
static int hantro_set_fmt_out(struct hantro_ctx *ctx,
struct v4l2_pix_format_mplane *pix_mp)
{
@@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx,
ctx->dst_fmt.quantization = pix_mp->quantization;
hantro_update_requires_request(ctx, pix_mp->pixelformat);
+ hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat);
vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode);
vpu_debug(0, "fmt - w: %d, h: %d\n",
--
2.34.1
^ permalink raw reply related [flat|nested] 86+ messages in thread
* Re: [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-06 5:17 ` Ezequiel Garcia
-1 siblings, 0 replies; 86+ messages in thread
From: Ezequiel Garcia @ 2022-04-06 5:17 UTC (permalink / raw)
To: Nicolas Dufresne
Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, kernel, linux-kernel,
linux-media, linux-rockchip, linux-staging
On Tue, Apr 05, 2022 at 04:44:16PM -0400, Nicolas Dufresne wrote:
> No function change, this moves H264 specific validation into the H264
> specific code. This is in preparation of improving this validation and
> reusing at streamone.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++
> drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++----------------
> drivers/staging/media/rkvdec/rkvdec.h | 1 +
> 3 files changed, 30 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 8d44a884a52e..0dcbcb1bac80 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
> return 0;
> }
>
> +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> +{
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
> .adjust_fmt = rkvdec_h264_adjust_fmt,
> .start = rkvdec_h264_start,
> .stop = rkvdec_h264_stop,
> .run = rkvdec_h264_run,
> + .try_ctrl = rkvdec_h264_try_ctrl,
> };
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 2df8cf4883e2..e3d44d5b35f3 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -29,23 +29,12 @@
>
> static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
> + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +
> + if (desc->ops->try_ctrl)
> + return desc->ops->try_ctrl(ctx, ctrl);
> +
> return 0;
> }
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index 2f4ea1786b93..9df0fba799a4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops {
> void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf,
> struct vb2_v4l2_buffer *dst_buf,
> enum vb2_buffer_state result);
> + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
> };
>
> struct rkvdec_coded_fmt_desc {
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264
@ 2022-04-06 5:17 ` Ezequiel Garcia
0 siblings, 0 replies; 86+ messages in thread
From: Ezequiel Garcia @ 2022-04-06 5:17 UTC (permalink / raw)
To: Nicolas Dufresne
Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, kernel, linux-kernel,
linux-media, linux-rockchip, linux-staging
On Tue, Apr 05, 2022 at 04:44:16PM -0400, Nicolas Dufresne wrote:
> No function change, this moves H264 specific validation into the H264
> specific code. This is in preparation of improving this validation and
> reusing at streamone.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++
> drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++----------------
> drivers/staging/media/rkvdec/rkvdec.h | 1 +
> 3 files changed, 30 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 8d44a884a52e..0dcbcb1bac80 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
> return 0;
> }
>
> +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> +{
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
> .adjust_fmt = rkvdec_h264_adjust_fmt,
> .start = rkvdec_h264_start,
> .stop = rkvdec_h264_stop,
> .run = rkvdec_h264_run,
> + .try_ctrl = rkvdec_h264_try_ctrl,
> };
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 2df8cf4883e2..e3d44d5b35f3 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -29,23 +29,12 @@
>
> static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
> + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +
> + if (desc->ops->try_ctrl)
> + return desc->ops->try_ctrl(ctx, ctrl);
> +
> return 0;
> }
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index 2f4ea1786b93..9df0fba799a4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops {
> void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf,
> struct vb2_v4l2_buffer *dst_buf,
> enum vb2_buffer_state result);
> + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
> };
>
> struct rkvdec_coded_fmt_desc {
> --
> 2.34.1
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Fwd: [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (23 preceding siblings ...)
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-06 13:06 ` Nicolas Dufresne
2022-04-06 17:58 ` Nicolas Dufresne
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-06 13:06 UTC (permalink / raw)
To: linux-media
[-- Attachment #1: Type: text/plain, Size: 1 bytes --]
[-- Attachment #2: Message transféré — [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders --]
[-- Type: message/rfc822, Size: 8415 bytes --]
From: Nicolas Dufresne <nicolas.dufresne@collabora.com>
To:
Cc: kernel@collabora.com, linux-kernel@vger.kernel.org
Subject: [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders
Date: Tue, 5 Apr 2022 16:44:01 -0400
Message-ID: <20220405204426.259074-1-nicolas.dufresne@collabora.com>
Until now, only Cedrus (a slice base decoder) supported interlaced
decoding. In order to support field decoding in our frame-based decoder,
the v4l2-h264 library needed adaptation to produce the appropriate
reference lists.
This patch extends the v4l2-h264 library to produce the larger references
list needed to represent fields separately. Hantro, MTK-VCODEC and RKVDEC
drivers have been adapted to accommodate the larger lists. Though, only
Hantro and RKVDEC actually have HW support for field decoding. So only
these two have been updated to make use of the larger lists. All this work
has been done using the H.264 specification, LibreELEC downstream kernel
patches, Rockchip MPP reference software and Hantro reference software.
For reviewers, the following is the map of all commit. Patches that could
be merge independently of this serie are marked as independent. Note that
the test results do depend on the generic fixes.
01 : Documentation fix (independent)
02-03 : Improving some generic traces (independent)
04 : Minor v4l2-h264 fix (independent)
05-11 : v4l2-h264 field decoding support
12-16 : rkvdec h.264 generic fixes (independent)
17-20 : rkvdec h.264 field decoding support
21-24 : hantro h.264 field decoding support
All this work have been tested using GStreamer mainline implementation
but also with FFMPEG LibreELEC fork using the testing tool fluster
running through the ITU-T H.264 (2016-02) AVCv2 set of bitstream. Before
this patch, the scores were:
Hantro:
FFMPEG: 88/135
GSteamer: 90/135
RKVDEC:
FFMPEG: 73/135
GSteamer: 77/135
And after these changes:
Hantro:
FFMPEG: 118/135
GSteamer: 129/135
RKVDEC:
FFMPEG: 118/135
GSteamer: 129/135
Note that a bug in FFMPEG / LibreELEC fork was noticed and fixed with the
following change:
diff --git a/libavcodec/v4l2_request_h264.c b/libavcodec/v4l2_request_h264.c
index 88da8f0a2d..394bae0550 100644
--- a/libavcodec/v4l2_request_h264.c
+++ b/libavcodec/v4l2_request_h264.c
@@ -66,7 +66,7 @@ static void fill_dpb_entry(struct v4l2_h264_dpb_entry *entry, const H264Picture
{
entry->reference_ts = ff_v4l2_request_get_capture_timestamp(pic->f);
entry->pic_num = pic->pic_id;
- entry->frame_num = pic->frame_num;
+ entry->frame_num = pic->long_ref ? pic->pic_id : pic->frame_num;
entry->fields = pic->reference & V4L2_H264_FRAME_REF;
entry->flags = V4L2_H264_DPB_ENTRY_FLAG_VALID;
if (entry->fields)
Some useful links:
Detailed Hantro Results: https://gitlab.freedesktop.org/-/snippets/5189
Detailed RKVDEC Results: https://gitlab.freedesktop.org/-/snippets/5253
ITU-T H.264 (2016-02) AVCv2: https://www.itu.int/net/itu-t/sigdb/spevideo/VideoForm-s.aspx?val=102002641
Fluster: https://github.com/fluendo/fluster
GStreamer: https://gitlab.freedesktop.org/gstreamer/gstreamer/
FFMPEG Fork: https://github.com/jernejsk/FFmpeg/tree/v4l2-request-hwaccel-4.4
Rockchip MPP: https://github.com/rockchip-linux/mpp
Changes in v3:
- Improved debug message on timestamp miss-match
- Moved H264 SPS validation into rkvdec-h264
- Added more comments around H264 SPS validation
- Also validate at streamon (rkvdec start())
- Applied more Review-by and Fixes tag
- Fixed Signed-off-by chain in Jonas patch
Changes in v2:
- Applied most of Sebastian's suggestion in comments and commit messages.
- Use a bool for dpb_valid and dpb_bottom in rkvdec
- Dropped one wrong typo fix (media: v4l2-mem2mem: Fix typo in trace message)
- Dropped Alex fix (media: rkvdec-h264: Don't hardcode SPS/PPS parameters
+ I will carry this one later, it seems cosmetic
Jonas Karlman (5):
media: rkvdec: h264: Fix bit depth wrap in pps packet
media: rkvdec: h264: Validate and use pic width and height in mbs
media: rkvdec: h264: Fix reference frame_num wrap for second field
media: rkvdec: Ensure decoded resolution fit coded resolution
media: hantro: h264: Make dpb entry management more robust
Nicolas Dufresne (18):
media: doc: Document dual use of H.264 pic_num/frame_num
media: v4l2-mem2mem: Trace on implicit un-hold
media: h264: Avoid wrapping long_term_frame_idx
media: h264: Use v4l2_h264_reference for reflist
media: h264: Increase reference lists size to 32
media: h264: Store current picture fields
media: h264: Store all fields into the unordered list
media: v4l2: Trace calculated p/b0/b1 initial reflist
media: h264: Sort p/b reflist using frame_num
media: v4l2: Reorder field reflist
media: rkvdec: Stop overclocking the decoder
media: rkvdec: h264: Fix dpb_valid implementation
media: rkvdec: Move H264 SPS validation in rkvdec-h264
media: rkvdec-h264: Add field decoding support
media: rkvdec: Enable capture buffer holding for H264
media: hantro: Stop using H.264 parameter pic_num
media: hantro: Add H.264 field decoding support
media: hantro: Enable HOLD_CAPTURE_BUF for H.264
Sebastian Fricke (1):
media: videobuf2-v4l2: Warn on holding buffers without support
.../media/v4l/ext-ctrls-codec-stateless.rst | 10 +-
.../media/common/videobuf2/videobuf2-v4l2.c | 7 +-
.../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +-
drivers/media/v4l2-core/v4l2-h264.c | 261 ++++++++++++++----
drivers/media/v4l2-core/v4l2-mem2mem.c | 1 +
.../staging/media/hantro/hantro_g1_h264_dec.c | 38 +--
drivers/staging/media/hantro/hantro_h264.c | 119 ++++++--
drivers/staging/media/hantro/hantro_hw.h | 7 +-
drivers/staging/media/hantro/hantro_v4l2.c | 25 ++
.../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++----
drivers/staging/media/rkvdec/rkvdec-h264.c | 154 ++++++++---
drivers/staging/media/rkvdec/rkvdec.c | 35 +--
drivers/staging/media/rkvdec/rkvdec.h | 2 +
include/media/v4l2-h264.h | 31 ++-
14 files changed, 580 insertions(+), 225 deletions(-)
--
2.34.1
--
To unsubscribe, send mail to kernel-unsubscribe@lists.collabora.co.uk.
^ permalink raw reply related [flat|nested] 86+ messages in thread
* Re: [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-06 13:10 ` Nicolas Dufresne
-1 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-06 13:10 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
Le mardi 05 avril 2022 à 16:44 -0400, Nicolas Dufresne a écrit :
> From: Jonas Karlman <jonas@kwiboo.se>
>
> The width and height in macroblocks is currently configured based on OUTPUT
> buffer resolution, this works for frame pictures but can cause issues for
> field pictures.
>
> When frame_mbs_only_flag is 0 the height in mbs should be height of
> the field instead of height of frame.
>
> Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
> against OUTPUT buffer resolution and use these values to configure HW.
> The validation is happening in both try_ctrt() and start() since it is
> otherwise possible to trick the driver during initialization by changing
> the OUTPUT format after having set a valid control.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
> 1 file changed, 59 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 0dcbcb1bac80..f081b476340f 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
> LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
> DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
> +
> + /* Use the SPS values since they are already in macroblocks
> + * dimensions, height can be field height (halved) if
> + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
> + * decoding smaller images into larger allocation which can be used
> + * to implementing SVC spatial layer support.
> + */
> + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
> + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
> +
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
> FRAME_MBS_ONLY_FLAG);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
> @@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
> return 0;
> }
>
> +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
> + const struct v4l2_ctrl_h264_sps *sps)
> +{
> + unsigned int width, height;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> +
> + width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
> + height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
> +
> + /* when frame_mbs_only_flag is not set, this is field height,
> + * which is half the final height (see (7-18) in the
> + * specification)
> + */
> + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> + height *= 2;
> +
> + if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> + height > ctx->coded_fmt.fmt.pix_mp.height)
> + return -EINVAL;
I forgot to add traces for these as suggestion by Ezequiel in my review. I think
the patch is still acceptable as-is, I can later make a patchset on that
specific subject. I'll would add trace for validation taking place in the
generic control code, making this all much more consistent (and a smoother
experience for developers).
> +
> + return 0;
> +}
> +
> static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
> {
> struct rkvdec_dev *rkvdec = ctx->dev;
> struct rkvdec_h264_priv_tbl *priv_tbl;
> struct rkvdec_h264_ctx *h264_ctx;
> + struct v4l2_ctrl *ctrl;
> int ret;
>
> + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
> + V4L2_CID_STATELESS_H264_SPS);
> + if (!ctrl)
> + return -EINVAL;
> +
> + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> + if (ret)
> + return ret;
> +
> h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
> if (!h264_ctx)
> return -ENOMEM;
> @@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>
> static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
> + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> +
> return 0;
> }
>
> --
> 2.34.1
>
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
@ 2022-04-06 13:10 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-06 13:10 UTC (permalink / raw)
To: Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
Le mardi 05 avril 2022 à 16:44 -0400, Nicolas Dufresne a écrit :
> From: Jonas Karlman <jonas@kwiboo.se>
>
> The width and height in macroblocks is currently configured based on OUTPUT
> buffer resolution, this works for frame pictures but can cause issues for
> field pictures.
>
> When frame_mbs_only_flag is 0 the height in mbs should be height of
> the field instead of height of frame.
>
> Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
> against OUTPUT buffer resolution and use these values to configure HW.
> The validation is happening in both try_ctrt() and start() since it is
> otherwise possible to trick the driver during initialization by changing
> the OUTPUT format after having set a valid control.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
> 1 file changed, 59 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 0dcbcb1bac80..f081b476340f 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
> LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
> DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
> +
> + /* Use the SPS values since they are already in macroblocks
> + * dimensions, height can be field height (halved) if
> + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
> + * decoding smaller images into larger allocation which can be used
> + * to implementing SVC spatial layer support.
> + */
> + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
> + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
> +
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
> FRAME_MBS_ONLY_FLAG);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
> @@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
> return 0;
> }
>
> +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
> + const struct v4l2_ctrl_h264_sps *sps)
> +{
> + unsigned int width, height;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> +
> + width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
> + height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
> +
> + /* when frame_mbs_only_flag is not set, this is field height,
> + * which is half the final height (see (7-18) in the
> + * specification)
> + */
> + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> + height *= 2;
> +
> + if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> + height > ctx->coded_fmt.fmt.pix_mp.height)
> + return -EINVAL;
I forgot to add traces for these as suggestion by Ezequiel in my review. I think
the patch is still acceptable as-is, I can later make a patchset on that
specific subject. I'll would add trace for validation taking place in the
generic control code, making this all much more consistent (and a smoother
experience for developers).
> +
> + return 0;
> +}
> +
> static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
> {
> struct rkvdec_dev *rkvdec = ctx->dev;
> struct rkvdec_h264_priv_tbl *priv_tbl;
> struct rkvdec_h264_ctx *h264_ctx;
> + struct v4l2_ctrl *ctrl;
> int ret;
>
> + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
> + V4L2_CID_STATELESS_H264_SPS);
> + if (!ctrl)
> + return -EINVAL;
> +
> + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> + if (ret)
> + return ret;
> +
> h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
> if (!h264_ctx)
> return -ENOMEM;
> @@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>
> static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
> + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> +
> return 0;
> }
>
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-06 13:12 ` Ezequiel Garcia
-1 siblings, 0 replies; 86+ messages in thread
From: Ezequiel Garcia @ 2022-04-06 13:12 UTC (permalink / raw)
To: Nicolas Dufresne
Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, kernel, linux-kernel,
Jonas Karlman, linux-media, linux-rockchip, linux-staging
On Tue, Apr 05, 2022 at 04:44:17PM -0400, Nicolas Dufresne wrote:
> From: Jonas Karlman <jonas@kwiboo.se>
>
> The width and height in macroblocks is currently configured based on OUTPUT
> buffer resolution, this works for frame pictures but can cause issues for
> field pictures.
>
> When frame_mbs_only_flag is 0 the height in mbs should be height of
> the field instead of height of frame.
>
> Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
> against OUTPUT buffer resolution and use these values to configure HW.
> The validation is happening in both try_ctrt() and start() since it is
> otherwise possible to trick the driver during initialization by changing
> the OUTPUT format after having set a valid control.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
> 1 file changed, 59 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 0dcbcb1bac80..f081b476340f 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
> LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
> DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
> +
> + /* Use the SPS values since they are already in macroblocks
> + * dimensions, height can be field height (halved) if
> + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
> + * decoding smaller images into larger allocation which can be used
> + * to implementing SVC spatial layer support.
> + */
> + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
> + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
> +
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
> FRAME_MBS_ONLY_FLAG);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
> @@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
> return 0;
> }
>
> +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
> + const struct v4l2_ctrl_h264_sps *sps)
> +{
> + unsigned int width, height;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> +
> + width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
> + height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
> +
> + /* when frame_mbs_only_flag is not set, this is field height,
> + * which is half the final height (see (7-18) in the
> + * specification)
> + */
> + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> + height *= 2;
> +
> + if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> + height > ctx->coded_fmt.fmt.pix_mp.height)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
> {
> struct rkvdec_dev *rkvdec = ctx->dev;
> struct rkvdec_h264_priv_tbl *priv_tbl;
> struct rkvdec_h264_ctx *h264_ctx;
> + struct v4l2_ctrl *ctrl;
> int ret;
>
> + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
> + V4L2_CID_STATELESS_H264_SPS);
> + if (!ctrl)
> + return -EINVAL;
> +
> + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> + if (ret)
> + return ret;
> +
> h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
> if (!h264_ctx)
> return -ENOMEM;
> @@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>
> static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
> + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> +
> return 0;
> }
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
@ 2022-04-06 13:12 ` Ezequiel Garcia
0 siblings, 0 replies; 86+ messages in thread
From: Ezequiel Garcia @ 2022-04-06 13:12 UTC (permalink / raw)
To: Nicolas Dufresne
Cc: Mauro Carvalho Chehab, Greg Kroah-Hartman, kernel, linux-kernel,
Jonas Karlman, linux-media, linux-rockchip, linux-staging
On Tue, Apr 05, 2022 at 04:44:17PM -0400, Nicolas Dufresne wrote:
> From: Jonas Karlman <jonas@kwiboo.se>
>
> The width and height in macroblocks is currently configured based on OUTPUT
> buffer resolution, this works for frame pictures but can cause issues for
> field pictures.
>
> When frame_mbs_only_flag is 0 the height in mbs should be height of
> the field instead of height of frame.
>
> Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
> against OUTPUT buffer resolution and use these values to configure HW.
> The validation is happening in both try_ctrt() and start() since it is
> otherwise possible to trick the driver during initialization by changing
> the OUTPUT format after having set a valid control.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
> 1 file changed, 59 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 0dcbcb1bac80..f081b476340f 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
> LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
> DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
> +
> + /* Use the SPS values since they are already in macroblocks
> + * dimensions, height can be field height (halved) if
> + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
> + * decoding smaller images into larger allocation which can be used
> + * to implementing SVC spatial layer support.
> + */
> + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
> + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
> +
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
> FRAME_MBS_ONLY_FLAG);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
> @@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
> return 0;
> }
>
> +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
> + const struct v4l2_ctrl_h264_sps *sps)
> +{
> + unsigned int width, height;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> +
> + width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
> + height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
> +
> + /* when frame_mbs_only_flag is not set, this is field height,
> + * which is half the final height (see (7-18) in the
> + * specification)
> + */
> + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> + height *= 2;
> +
> + if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> + height > ctx->coded_fmt.fmt.pix_mp.height)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
> {
> struct rkvdec_dev *rkvdec = ctx->dev;
> struct rkvdec_h264_priv_tbl *priv_tbl;
> struct rkvdec_h264_ctx *h264_ctx;
> + struct v4l2_ctrl *ctrl;
> int ret;
>
> + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
> + V4L2_CID_STATELESS_H264_SPS);
> + if (!ctrl)
> + return -EINVAL;
> +
> + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> + if (ret)
> + return ret;
> +
> h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
> if (!h264_ctx)
> return -ENOMEM;
> @@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>
> static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
> + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> +
> return 0;
> }
>
> --
> 2.34.1
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
` (24 preceding siblings ...)
2022-04-06 13:06 ` Fwd: [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
@ 2022-04-06 17:58 ` Nicolas Dufresne
25 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-06 17:58 UTC (permalink / raw)
To: Nicolas Dufresne; +Cc: kernel, linux-kernel, linux-media
Hello everyone,
its not clear if that cover made it this time, though it will now. Adding a
comment below ...
Le mardi 05 avril 2022 à 16:44 -0400, Nicolas Dufresne a écrit :
> Until now, only Cedrus (a slice base decoder) supported interlaced
> decoding. In order to support field decoding in our frame-based decoder,
> the v4l2-h264 library needed adaptation to produce the appropriate
> reference lists.
>
> This patch extends the v4l2-h264 library to produce the larger references
> list needed to represent fields separately. Hantro, MTK-VCODEC and RKVDEC
> drivers have been adapted to accommodate the larger lists. Though, only
> Hantro and RKVDEC actually have HW support for field decoding. So only
> these two have been updated to make use of the larger lists. All this work
> has been done using the H.264 specification, LibreELEC downstream kernel
> patches, Rockchip MPP reference software and Hantro reference software.
>
> For reviewers, the following is the map of all commit. Patches that could
> be merge independently of this serie are marked as independent. Note that
> the test results do depend on the generic fixes.
>
> 01 : Documentation fix (independent)
> 02-03 : Improving some generic traces (independent)
> 04 : Minor v4l2-h264 fix (independent)
> 05-11 : v4l2-h264 field decoding support
> 12-16 : rkvdec h.264 generic fixes (independent)
> 17-20 : rkvdec h.264 field decoding support
> 21-24 : hantro h.264 field decoding support
>
> All this work have been tested using GStreamer mainline implementation
> but also with FFMPEG LibreELEC fork using the testing tool fluster
> running through the ITU-T H.264 (2016-02) AVCv2 set of bitstream. Before
> this patch, the scores were:
>
> Hantro:
> FFMPEG: 88/135
> GSteamer: 90/135
> RKVDEC:
> FFMPEG: 73/135
> GSteamer: 77/135
>
> And after these changes:
>
> Hantro:
> FFMPEG: 118/135
> GSteamer: 129/135
I have also tested on IMX8MQ now, same results. This exercise the
hantro_g1_h264.c code.
> RKVDEC:
> FFMPEG: 118/135
> GSteamer: 129/135
>
> Note that a bug in FFMPEG / LibreELEC fork was noticed and fixed with the
> following change:
>
> diff --git a/libavcodec/v4l2_request_h264.c b/libavcodec/v4l2_request_h264.c
> index 88da8f0a2d..394bae0550 100644
> --- a/libavcodec/v4l2_request_h264.c
> +++ b/libavcodec/v4l2_request_h264.c
> @@ -66,7 +66,7 @@ static void fill_dpb_entry(struct v4l2_h264_dpb_entry *entry, const H264Picture
> {
> entry->reference_ts = ff_v4l2_request_get_capture_timestamp(pic->f);
> entry->pic_num = pic->pic_id;
> - entry->frame_num = pic->frame_num;
> + entry->frame_num = pic->long_ref ? pic->pic_id : pic->frame_num;
> entry->fields = pic->reference & V4L2_H264_FRAME_REF;
> entry->flags = V4L2_H264_DPB_ENTRY_FLAG_VALID;
> if (entry->fields)
>
> Some useful links:
>
> Detailed Hantro Results: https://gitlab.freedesktop.org/-/snippets/5189
> Detailed RKVDEC Results: https://gitlab.freedesktop.org/-/snippets/5253
> ITU-T H.264 (2016-02) AVCv2: https://www.itu.int/net/itu-t/sigdb/spevideo/VideoForm-s.aspx?val=102002641
> Fluster: https://github.com/fluendo/fluster
> GStreamer: https://gitlab.freedesktop.org/gstreamer/gstreamer/
> FFMPEG Fork: https://github.com/jernejsk/FFmpeg/tree/v4l2-request-hwaccel-4.4
> Rockchip MPP: https://github.com/rockchip-linux/mpp
>
> Changes in v3:
> - Improved debug message on timestamp miss-match
> - Moved H264 SPS validation into rkvdec-h264
> - Added more comments around H264 SPS validation
> - Also validate at streamon (rkvdec start())
> - Applied more Review-by and Fixes tag
> - Fixed Signed-off-by chain in Jonas patch
>
> Changes in v2:
> - Applied most of Sebastian's suggestion in comments and commit messages.
> - Use a bool for dpb_valid and dpb_bottom in rkvdec
> - Dropped one wrong typo fix (media: v4l2-mem2mem: Fix typo in trace message)
> - Dropped Alex fix (media: rkvdec-h264: Don't hardcode SPS/PPS parameters
> + I will carry this one later, it seems cosmetic
>
> Jonas Karlman (5):
> media: rkvdec: h264: Fix bit depth wrap in pps packet
> media: rkvdec: h264: Validate and use pic width and height in mbs
> media: rkvdec: h264: Fix reference frame_num wrap for second field
> media: rkvdec: Ensure decoded resolution fit coded resolution
> media: hantro: h264: Make dpb entry management more robust
>
> Nicolas Dufresne (18):
> media: doc: Document dual use of H.264 pic_num/frame_num
> media: v4l2-mem2mem: Trace on implicit un-hold
> media: h264: Avoid wrapping long_term_frame_idx
> media: h264: Use v4l2_h264_reference for reflist
> media: h264: Increase reference lists size to 32
> media: h264: Store current picture fields
> media: h264: Store all fields into the unordered list
> media: v4l2: Trace calculated p/b0/b1 initial reflist
> media: h264: Sort p/b reflist using frame_num
> media: v4l2: Reorder field reflist
> media: rkvdec: Stop overclocking the decoder
> media: rkvdec: h264: Fix dpb_valid implementation
> media: rkvdec: Move H264 SPS validation in rkvdec-h264
> media: rkvdec-h264: Add field decoding support
> media: rkvdec: Enable capture buffer holding for H264
> media: hantro: Stop using H.264 parameter pic_num
> media: hantro: Add H.264 field decoding support
> media: hantro: Enable HOLD_CAPTURE_BUF for H.264
>
> Sebastian Fricke (1):
> media: videobuf2-v4l2: Warn on holding buffers without support
>
> .../media/v4l/ext-ctrls-codec-stateless.rst | 10 +-
> .../media/common/videobuf2/videobuf2-v4l2.c | 7 +-
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +-
> drivers/media/v4l2-core/v4l2-h264.c | 261 ++++++++++++++----
> drivers/media/v4l2-core/v4l2-mem2mem.c | 1 +
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +--
> drivers/staging/media/hantro/hantro_h264.c | 119 ++++++--
> drivers/staging/media/hantro/hantro_hw.h | 7 +-
> drivers/staging/media/hantro/hantro_v4l2.c | 25 ++
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++----
> drivers/staging/media/rkvdec/rkvdec-h264.c | 154 ++++++++---
> drivers/staging/media/rkvdec/rkvdec.c | 35 +--
> drivers/staging/media/rkvdec/rkvdec.h | 2 +
> include/media/v4l2-h264.h | 31 ++-
> 14 files changed, 580 insertions(+), 225 deletions(-)
>
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
2022-04-05 20:44 ` Nicolas Dufresne
(?)
(?)
@ 2022-04-22 5:42 ` Hans Verkuil
-1 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 5:42 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Hi Nicolas,
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
After applying this patch building the mediatek driver fails:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
451 | ref_list[i] = b->index;
| ^~
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
| ^~~~~~~~~~
| |
| u8 * {aka unsigned char *}
(dropped a lot more errors)
Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
66 | struct v4l2_h264_reference *b0_reflist,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
67 | struct v4l2_h264_reference *b1_reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
| ~~~~~~~~^~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
83 | struct v4l2_h264_reference *reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
It looks like you didn't build the tegra-vde driver either.
Regards,
Hans
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 5:42 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 5:42 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Hi Nicolas,
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
After applying this patch building the mediatek driver fails:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
451 | ref_list[i] = b->index;
| ^~
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
| ^~~~~~~~~~
| |
| u8 * {aka unsigned char *}
(dropped a lot more errors)
Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
66 | struct v4l2_h264_reference *b0_reflist,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
67 | struct v4l2_h264_reference *b1_reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
| ~~~~~~~~^~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
83 | struct v4l2_h264_reference *reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
It looks like you didn't build the tegra-vde driver either.
Regards,
Hans
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 5:42 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 5:42 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Hi Nicolas,
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
After applying this patch building the mediatek driver fails:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
451 | ref_list[i] = b->index;
| ^~
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
| ^~~~~~~~~~
| |
| u8 * {aka unsigned char *}
(dropped a lot more errors)
Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
66 | struct v4l2_h264_reference *b0_reflist,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
67 | struct v4l2_h264_reference *b1_reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
| ~~~~~~~~^~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
83 | struct v4l2_h264_reference *reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
It looks like you didn't build the tegra-vde driver either.
Regards,
Hans
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 5:42 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 5:42 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Hi Nicolas,
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
After applying this patch building the mediatek driver fails:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
451 | ref_list[i] = b->index;
| ^~
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
| ^~~~~~~~~~
| |
| u8 * {aka unsigned char *}
(dropped a lot more errors)
Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
66 | struct v4l2_h264_reference *b0_reflist,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
| ~~~~~~~~^~~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
67 | struct v4l2_h264_reference *b1_reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
| ~~~~~~~~^~
| |
| u8 * {aka unsigned char *}
In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
83 | struct v4l2_h264_reference *reflist);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
It looks like you didn't build the tegra-vde driver either.
Regards,
Hans
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num
2022-04-05 20:44 ` [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num Nicolas Dufresne
@ 2022-04-22 6:31 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 6:31 UTC (permalink / raw)
To: Nicolas Dufresne, Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> These two fields need documentation as they have dual meaning. It is also
> confusing since pic_num is a derived value from frame_num, so this should
> help application developers. If we ever need to make a V2 of this API, I
> would suggest to remove pic_num entirely.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> ---
> .../media/v4l/ext-ctrls-codec-stateless.rst | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
> index 6541e4c32b26..49f89b702068 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec-stateless.rst
> @@ -649,10 +649,16 @@ Stateless Codec Control ID
> :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64.
> * - __u32
> - ``pic_num``
> - -
> + - For short term references, this should match the derived value PicNum
shouldn't 'should' be 'must'? Same elsewhere below.
> + (8-28) and for long term references it should match the derived value
> + LongTermPicNum (8-29). Note that pic_num is the same as FrameNumWrap
> + for frame decoding.
I think this last sentence is a bit confusing. How about:
"When decoding frames (as opposed to fields) pic_num is the same as FrameNumWrap."
> * - __u16
> - ``frame_num``
> - -
> + - For short term references, this should match the frame_num value from
> + the slice header syntax (the driver will wrap the value if neeeded). For
neeeded -> needed
> + long term references, this should be set to the value of
> + long_term_frame_idx described in the dec_ref_pic_marking() syntax.
> * - __u8
> - ``fields``
> - Specifies how the DPB entry is referenced. See :ref:`Reference Fields <h264_ref_fields>`
Regards,
Hans
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
2022-04-05 20:44 ` Nicolas Dufresne
(?)
(?)
@ 2022-04-22 6:59 ` Hans Verkuil
-1 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 6:59 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
It's not clear how serious this FIXME is. Either it should be fixed (here or
later in the series), or explain why this isn't fixed yet. Presumably because
it is not a problem now, but it might be later when some other feature is
enabled.
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
How about:
for (i = 0; i < 32; i++)
ref_list[i] = i < b->num_valid ? b->index : 0x20;
Or:
memset(ref_list, b->index, b->num_valid);
memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
Either approach is better IMHO than using two for-loops.
Regards,
Hans
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 6:59 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 6:59 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
It's not clear how serious this FIXME is. Either it should be fixed (here or
later in the series), or explain why this isn't fixed yet. Presumably because
it is not a problem now, but it might be later when some other feature is
enabled.
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
How about:
for (i = 0; i < 32; i++)
ref_list[i] = i < b->num_valid ? b->index : 0x20;
Or:
memset(ref_list, b->index, b->num_valid);
memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
Either approach is better IMHO than using two for-loops.
Regards,
Hans
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 6:59 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 6:59 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
It's not clear how serious this FIXME is. Either it should be fixed (here or
later in the series), or explain why this isn't fixed yet. Presumably because
it is not a problem now, but it might be later when some other feature is
enabled.
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
How about:
for (i = 0; i < 32; i++)
ref_list[i] = i < b->num_valid ? b->index : 0x20;
Or:
memset(ref_list, b->index, b->num_valid);
memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
Either approach is better IMHO than using two for-loops.
Regards,
Hans
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 6:59 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 6:59 UTC (permalink / raw)
To: Nicolas Dufresne, Tiffany Lin, Andrew-CT Chen,
Mauro Carvalho Chehab, Ezequiel Garcia, Philipp Zabel,
Greg Kroah-Hartman, Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In preparation for adding field decoding support, convert the byte arrays
> for reflist into array of struct v4l2_h264_reference. That struct will
> allow us to mark which field of the reference picture is being referenced.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> drivers/staging/media/hantro/hantro_hw.h | 6 +-
> .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> include/media/v4l2-h264.h | 19 ++--
> 7 files changed, 116 insertions(+), 107 deletions(-)
>
> diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> index 43542de11e9c..72c599e05a47 100644
> --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> /*
> * The firmware expects unused reflist entries to have the value 0x20.
> */
> -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> {
> - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> + u32 i;
> +
> + /* FIXME mark the reference parity */
It's not clear how serious this FIXME is. Either it should be fixed (here or
later in the series), or explain why this isn't fixed yet. Presumably because
it is not a problem now, but it might be later when some other feature is
enabled.
> + for (i = 0; i < b->num_valid; i++)
> + ref_list[i] = b->index;
> +
> + for (; i < 32; i++)
> + ref_list[i] = 0x20;
How about:
for (i = 0; i < 32; i++)
ref_list[i] = i < b->num_valid ? b->index : 0x20;
Or:
memset(ref_list, b->index, b->num_valid);
memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
Either approach is better IMHO than using two for-loops.
Regards,
Hans
> }
>
> static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> /* Adapt the built lists to the firmware's expectations */
> - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> + get_ref_list(p0_reflist, reflist_builder);
> + get_ref_list(b0_reflist, reflist_builder);
> + get_ref_list(b1_reflist, reflist_builder);
>
> memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> sizeof(inst->vsi_ctx.h264_slice_params));
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index ac47519a9fbe..afbfcf78efe4 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> pic_order_count = dpb[i].top_field_order_cnt;
>
> b->refs[i].pic_order_count = pic_order_count;
> - b->unordered_reflist[b->num_valid] = i;
> + b->unordered_reflist[b->num_valid].index = i;
> b->num_valid++;
> }
>
> for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> - b->unordered_reflist[i] = i;
> + b->unordered_reflist[i].index = i;
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
>
> @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> const struct v4l2_h264_reflist_builder *builder = data;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> s32 poca, pocb;
> u8 idxa, idxb;
>
> - idxa = *((u8 *)ptra);
> - idxb = *((u8 *)ptrb);
> + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
>
> if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist)
> + struct v4l2_h264_reference *reflist)
> {
> memcpy(reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist)
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist)
> {
> memcpy(b0_reflist, builder->unordered_reflist,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> index f49dbfb8a843..9de7f05eff2a 100644
> --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> int reg_num;
> u32 reg;
> @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 0; i < 15; i += 3) {
> - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> }
>
> @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> * of forward and backward reference picture lists and first 4 entries
> * of P forward picture list.
> */
> - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
>
> /*
> @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> */
> reg_num = 0;
> for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index ed018e293ba0..2bc6b8f088f5 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> * @b1: B1 reflist
> */
> struct hantro_h264_dec_reflists {
> - u8 p[HANTRO_H264_DPB_SIZE];
> - u8 b0[HANTRO_H264_DPB_SIZE];
> - u8 b1[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> };
>
> /**
> diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> index 64a6330475eb..46c1a83bcc4e 100644
> --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
>
> static void set_ref(struct hantro_ctx *ctx)
> {
> - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> struct hantro_dev *vpu = ctx->dev;
> u32 reg;
> int i;
> @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> b1_reflist = ctx->h264_dec.reflists.b1;
> p_reflist = ctx->h264_dec.reflists.p;
>
> - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
>
> - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
>
> reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
>
> - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
>
> - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
>
> - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
>
> - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
>
> - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
>
> - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
>
> - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
>
> reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 951e19231da2..3c7f3d87fab4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> #define RKVDEC_H264_DPB_SIZE 16
>
> struct rkvdec_h264_reflists {
> - u8 p[RKVDEC_H264_DPB_SIZE];
> - u8 b0[RKVDEC_H264_DPB_SIZE];
> - u8 b1[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> u8 num_valid;
> };
>
> @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
>
> switch (j) {
> case 0:
> - idx = h264_ctx->reflists.p[i];
> + idx = h264_ctx->reflists.p[i].index;
> break;
> case 1:
> - idx = h264_ctx->reflists.b0[i];
> + idx = h264_ctx->reflists.b0[i].index;
> break;
> case 2:
> - idx = h264_ctx->reflists.b1[i];
> + idx = h264_ctx->reflists.b1[i].index;
> break;
> }
>
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4b1c71c935e0..ef9a894e3c32 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
> s32 cur_pic_order_count;
> - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> u8 num_valid;
> };
>
> @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> *
> * @builder: reference list builder context
> - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> - * is an index in the DPB
> - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> - * is an index in the DPB
> + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> + * is a v4l2_h264_reference structure
> + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the B0/B1 reference lists. This procedure is described
> * in section '8.2.4 Decoding process for reference picture lists construction'
> @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> */
> void
> v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> - u8 *b0_reflist, u8 *b1_reflist);
> + struct v4l2_h264_reference *b0_reflist,
> + struct v4l2_h264_reference *b1_reflist);
>
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> * @builder: reference list builder context
> - * @reflist: 16-bytes array used to store the P reference list. Each entry
> - * is an index in the DPB
> + * @reflist: 16 sized array used to store the P reference list. Each entry
> + * is a v4l2_h264_reference structure
> *
> * This functions builds the P reference lists. This procedure is describe in
> * section '8.2.4 Decoding process for reference picture lists construction'
> @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> */
> void
> v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> - u8 *reflist);
> + struct v4l2_h264_reference *reflist);
>
> #endif /* _MEDIA_V4L2_H264_H */
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist
2022-04-05 20:44 ` [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist Nicolas Dufresne
@ 2022-04-22 7:26 ` Hans Verkuil
2022-04-22 14:58 ` Nicolas Dufresne
0 siblings, 1 reply; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:26 UTC (permalink / raw)
To: Nicolas Dufresne, Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> Add debug print statements to print the content of P & B reference
> lists, to verify that the ordering of the generated reference lists is
> correct. This is especially important for the field decoding mode,
> where sorting is more complex.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Tested-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> ---
> drivers/media/v4l2-core/v4l2-h264.c | 86 +++++++++++++++++++++++++++++
> 1 file changed, 86 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index 38d8dbda0045..bcf9b7774560 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -241,6 +241,87 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> return poca < pocb ? -1 : 1;
> }
>
> +static char ref_type_to_char (u8 ref_type)
Spurious space before (.
Odd that checkpatch didn't catch that.
> +{
> + switch (ref_type) {
> + case V4L2_H264_FRAME_REF:
> + return 'f';
> + case V4L2_H264_TOP_FIELD_REF:
> + return 't';
> + case V4L2_H264_BOTTOM_FIELD_REF:
> + return 'b';
> + }
> +
> + return '?';
> +}
> +
> +static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
> + struct v4l2_h264_reference *reflist,
> + char *out_str, const int len)
> +{
> + int n = 0, i;
> +
> + n += snprintf(out_str + n, len - n, "|");
> +
> + for (i = 0; i < builder->num_valid; i++) {
> + /* this is pic_num for frame and frame_num (wrapped) for field,
> + * but for frame pic_num is equal to frame_num (wrapped).
> + */
> + int frame_num = builder->refs[reflist[i].index].frame_num;
> + bool longterm = builder->refs[reflist[i].index].longterm;
> +
> + n += scnprintf(out_str + n, len - n, "%i%c%c|",
> + frame_num, longterm ? 'l' : 's',
> + ref_type_to_char (reflist[i].fields));
> + }
> +
> + return out_str;
> +}
> +
> +static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
> + struct v4l2_h264_reference *reflist)
> +{
> + char buf[1024];
> +
> + pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n",
> + builder->cur_pic_order_count,
> + ref_type_to_char(builder->cur_pic_fields),
> + format_ref_list_p(builder, reflist, buf, sizeof(buf)));
> +}
> +
> +static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
> + struct v4l2_h264_reference *reflist,
> + char *out_str, const int len)
> +{
> + int n = 0, i;
> +
> + n += snprintf(out_str + n, len - n, "|");
> +
> + for (i = 0; i < builder->num_valid; i++) {
> + int frame_num = builder->refs[reflist[i].index].frame_num;
> + u32 poc = v4l2_h264_get_poc(builder, reflist + i);
> + bool longterm = builder->refs[reflist[i].index].longterm;
> +
> + n += scnprintf(out_str + n, len - n, "%i%c%c|",
> + longterm ? frame_num : poc,
> + longterm ? 'l' : 's',
> + ref_type_to_char(reflist[i].fields));
> + }
> +
> + return out_str;
> +}
> +
> +static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
> + struct v4l2_h264_reference *reflist, u8 list_num)
> +{
> + char buf[1024];
I really don't like placing 1024 bytes on the stack. Can you find another way
of doing this? Perhaps using pr_cont or writing each format_ref_list item
on a separate line.
Regards,
Hans
> +
> + pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s",
> + list_num, builder->cur_pic_order_count,
> + ref_type_to_char (builder->cur_pic_fields),
> + format_ref_list_b(builder, reflist, buf, sizeof(buf)));
> +}
> +
> /**
> * v4l2_h264_build_p_ref_list() - Build the P reference list
> *
> @@ -261,6 +342,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> sort_r(reflist, builder->num_valid, sizeof(*reflist),
> v4l2_h264_p_ref_list_cmp, NULL, builder);
> +
> + print_ref_list_p(builder, reflist);
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
>
> @@ -296,6 +379,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> if (builder->num_valid > 1 &&
> !memcmp(b1_reflist, b0_reflist, builder->num_valid))
> swap(b1_reflist[0], b1_reflist[1]);
> +
> + print_ref_list_b(builder, b0_reflist, 0);
> + print_ref_list_b(builder, b1_reflist, 1);
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists);
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 10/24] media: h264: Sort p/b reflist using frame_num
2022-04-05 20:44 ` [PATCH v3 10/24] media: h264: Sort p/b reflist using frame_num Nicolas Dufresne
@ 2022-04-22 7:29 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:29 UTC (permalink / raw)
To: Nicolas Dufresne, Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> In the reference list builder, frame_num refers to FrameNumWrap
> in the spec, which is the same as the pic_num for frame decoding.
> The same applies for long_term_pic_num and long_term_frame_idx.
>
> Sort all type of references by frame_num so the sort can be reused
> for fields reflist were the sorting is done using frame_num instead.
> In short, pic_num is never actually used for building reference
> lists.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> ---
> drivers/media/v4l2-core/v4l2-h264.c | 23 +++++++++++++----------
> include/media/v4l2-h264.h | 2 --
> 2 files changed, 13 insertions(+), 12 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index bcf9b7774560..7e1eba03099a 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -50,7 +50,6 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE))
> continue;
>
> - b->refs[i].pic_num = dpb[i].pic_num;
> if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
> b->refs[i].longterm = true;
>
> @@ -139,15 +138,19 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> }
>
> /*
> - * Short term pics in descending pic num order, long term ones in
> - * ascending order.
> + * For frames, short term pics are in descending pic num order and long
> + * term ones in ascending order. For fields, the same direction is used
> + * but with frame_num (wrapped). For frames, the value of pic_num and
> + * frame_num are the same (see formula (8-28) and (8-29)). For this
> + * reason we can use frame_num only and share this funciton between
funciton -> function
Regards,
Hans
> + * frames and fields reflist.
> */
> if (!builder->refs[idxa].longterm)
> return builder->refs[idxb].frame_num <
> builder->refs[idxa].frame_num ?
> -1 : 1;
>
> - return builder->refs[idxa].pic_num < builder->refs[idxb].pic_num ?
> + return builder->refs[idxa].frame_num < builder->refs[idxb].frame_num ?
> -1 : 1;
> }
>
> @@ -173,10 +176,10 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> return 1;
> }
>
> - /* Long term pics in ascending pic num order. */
> + /* Long term pics in ascending frame num order. */
> if (builder->refs[idxa].longterm)
> - return builder->refs[idxa].pic_num <
> - builder->refs[idxb].pic_num ?
> + return builder->refs[idxa].frame_num <
> + builder->refs[idxb].frame_num ?
> -1 : 1;
>
> poca = v4l2_h264_get_poc(builder, ptra);
> @@ -218,10 +221,10 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> return 1;
> }
>
> - /* Long term pics in ascending pic num order. */
> + /* Long term pics in ascending frame num order. */
> if (builder->refs[idxa].longterm)
> - return builder->refs[idxa].pic_num <
> - builder->refs[idxb].pic_num ?
> + return builder->refs[idxa].frame_num <
> + builder->refs[idxb].frame_num ?
> -1 : 1;
>
> poca = v4l2_h264_get_poc(builder, ptra);
> diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> index 4cef717b3f18..0d9eaa956123 100644
> --- a/include/media/v4l2-h264.h
> +++ b/include/media/v4l2-h264.h
> @@ -18,7 +18,6 @@
> * @refs.top_field_order_cnt: top field order count
> * @refs.bottom_field_order_cnt: bottom field order count
> * @refs.frame_num: reference frame number
> - * @refs.pic_num: reference picture number
> * @refs.longterm: set to true for a long term reference
> * @refs: array of references
> * @cur_pic_order_count: picture order count of the frame being decoded
> @@ -36,7 +35,6 @@ struct v4l2_h264_reflist_builder {
> s32 top_field_order_cnt;
> s32 bottom_field_order_cnt;
> int frame_num;
> - u32 pic_num;
> u16 longterm : 1;
> } refs[V4L2_H264_NUM_DPB_ENTRIES];
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 11/24] media: v4l2: Reorder field reflist
2022-04-05 20:44 ` [PATCH v3 11/24] media: v4l2: Reorder field reflist Nicolas Dufresne
@ 2022-04-22 7:31 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:31 UTC (permalink / raw)
To: Nicolas Dufresne, Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> As per spec, the field reflist requires interleaving top and bottom
> field in a specific way that does not fit inside the sort operation.
> The process consist of alternating references parity, starting with
> a reference of the same parity as the current picture. This
> processs is done twice, once for short term references and a second
> time for the long term references.
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> ---
> drivers/media/v4l2-core/v4l2-h264.c | 42 +++++++++++++++++++++++++++++
> 1 file changed, 42 insertions(+)
>
> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> index 7e1eba03099a..d2d1f3bed7e8 100644
> --- a/drivers/media/v4l2-core/v4l2-h264.c
> +++ b/drivers/media/v4l2-core/v4l2-h264.c
> @@ -244,6 +244,40 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> return poca < pocb ? -1 : 1;
> }
>
> +/*
> + * The references need to be reordered so that references are alternating
> + * between top and bottom field references starting with the current picture
> + * parity. This has to be * done for short term and long term references
^
Spurious *.
Regards,
Hans
> + * separately.
> + */
> +static void reorder_field_reflist(const struct v4l2_h264_reflist_builder *b,
> + struct v4l2_h264_reference *reflist)
> +{
> + struct v4l2_h264_reference tmplist[V4L2_H264_REF_LIST_LEN];
> + u8 lt, i = 0, j = 0, k = 0;
> +
> + memcpy(tmplist, reflist, sizeof(tmplist[0]) * b->num_valid);
> +
> + for (lt = 0; lt <= 1; lt++) {
> + do {
> + for (; i < b->num_valid && b->refs[tmplist[i].index].longterm == lt; i++) {
> + if (tmplist[i].fields == b->cur_pic_fields) {
> + reflist[k++] = tmplist[i++];
> + break;
> + }
> + }
> +
> + for (; j < b->num_valid && b->refs[tmplist[j].index].longterm == lt; j++) {
> + if (tmplist[j].fields != b->cur_pic_fields) {
> + reflist[k++] = tmplist[j++];
> + break;
> + }
> + }
> + } while ((i < b->num_valid && b->refs[tmplist[i].index].longterm == lt) ||
> + (j < b->num_valid && b->refs[tmplist[j].index].longterm == lt));
> + }
> +}
> +
> static char ref_type_to_char (u8 ref_type)
> {
> switch (ref_type) {
> @@ -346,6 +380,9 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> sort_r(reflist, builder->num_valid, sizeof(*reflist),
> v4l2_h264_p_ref_list_cmp, NULL, builder);
>
> + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF)
> + reorder_field_reflist(builder, reflist);
> +
> print_ref_list_p(builder, reflist);
> }
> EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> @@ -379,6 +416,11 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> sort_r(b1_reflist, builder->num_valid, sizeof(*b1_reflist),
> v4l2_h264_b1_ref_list_cmp, NULL, builder);
>
> + if (builder->cur_pic_fields != V4L2_H264_FRAME_REF) {
> + reorder_field_reflist(builder, b0_reflist);
> + reorder_field_reflist(builder, b1_reflist);
> + }
> +
> if (builder->num_valid > 1 &&
> !memcmp(b1_reflist, b0_reflist, builder->num_valid))
> swap(b1_reflist[0], b1_reflist[1]);
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 12/24] media: rkvdec: Stop overclocking the decoder
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-22 7:33 ` Hans Verkuil
-1 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:33 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Boris Brezillon
Cc: kernel, linux-kernel, Sebastian Fricke, Mauro Carvalho Chehab,
linux-media, linux-rockchip, linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> While this overclock hack seems to work on some implementations
> (some ChromeBooks, RockPi4) it also causes instability on other
> implementations (notably LibreComputer Renegade, but there were more
> reports in the LibreELEC project, where this has been removed). While
> performance is indeed affected (tested with GStreamer), 4K playback
> still works as long as you don't operate in lock step and keep at
> least 1 frame ahead of time in the decode queue.
>
> After discussion with ChromeOS members, it would seem that their
> implementation indeed used to synchronously decode each frames, so
frames -> frame
Regards,
Hans
> this hack was simply compensating for their code being less
> efficient. In my opinion, this hack should not have been included
> upstream.
>
> Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> drivers/staging/media/rkvdec/rkvdec.c | 6 ------
> 1 file changed, 6 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index c0cf3488f970..2df8cf4883e2 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev)
> if (ret)
> return ret;
>
> - /*
> - * Bump ACLK to max. possible freq. (500 MHz) to improve performance
> - * When 4k video playback.
> - */
> - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000);
> -
> rkvdec->regs = devm_platform_ioremap_resource(pdev, 0);
> if (IS_ERR(rkvdec->regs))
> return PTR_ERR(rkvdec->regs);
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 12/24] media: rkvdec: Stop overclocking the decoder
@ 2022-04-22 7:33 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:33 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman, Boris Brezillon
Cc: kernel, linux-kernel, Sebastian Fricke, Mauro Carvalho Chehab,
linux-media, linux-rockchip, linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> While this overclock hack seems to work on some implementations
> (some ChromeBooks, RockPi4) it also causes instability on other
> implementations (notably LibreComputer Renegade, but there were more
> reports in the LibreELEC project, where this has been removed). While
> performance is indeed affected (tested with GStreamer), 4K playback
> still works as long as you don't operate in lock step and keep at
> least 1 frame ahead of time in the decode queue.
>
> After discussion with ChromeOS members, it would seem that their
> implementation indeed used to synchronously decode each frames, so
frames -> frame
Regards,
Hans
> this hack was simply compensating for their code being less
> efficient. In my opinion, this hack should not have been included
> upstream.
>
> Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver")
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> ---
> drivers/staging/media/rkvdec/rkvdec.c | 6 ------
> 1 file changed, 6 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index c0cf3488f970..2df8cf4883e2 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev)
> if (ret)
> return ret;
>
> - /*
> - * Bump ACLK to max. possible freq. (500 MHz) to improve performance
> - * When 4k video playback.
> - */
> - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000);
> -
> rkvdec->regs = devm_platform_ioremap_resource(pdev, 0);
> if (IS_ERR(rkvdec->regs))
> return PTR_ERR(rkvdec->regs);
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-22 7:36 ` Hans Verkuil
-1 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:36 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> No function change, this moves H264 specific validation into the H264
function -> functional
> specific code. This is in preparation of improving this validation and
> reusing at streamone.
streamone? I guess you mean "stream on"? Or perhaps "streaming"?
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++
> drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++----------------
> drivers/staging/media/rkvdec/rkvdec.h | 1 +
> 3 files changed, 30 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 8d44a884a52e..0dcbcb1bac80 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
> return 0;
> }
>
> +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> +{
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
> .adjust_fmt = rkvdec_h264_adjust_fmt,
> .start = rkvdec_h264_start,
> .stop = rkvdec_h264_stop,
> .run = rkvdec_h264_run,
> + .try_ctrl = rkvdec_h264_try_ctrl,
> };
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 2df8cf4883e2..e3d44d5b35f3 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -29,23 +29,12 @@
>
> static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
> + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +
> + if (desc->ops->try_ctrl)
> + return desc->ops->try_ctrl(ctx, ctrl);
> +
> return 0;
> }
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index 2f4ea1786b93..9df0fba799a4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops {
> void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf,
> struct vb2_v4l2_buffer *dst_buf,
> enum vb2_buffer_state result);
> + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
> };
>
> struct rkvdec_coded_fmt_desc {
Regards,
Hans
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264
@ 2022-04-22 7:36 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:36 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> No function change, this moves H264 specific validation into the H264
function -> functional
> specific code. This is in preparation of improving this validation and
> reusing at streamone.
streamone? I guess you mean "stream on"? Or perhaps "streaming"?
>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++
> drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++----------------
> drivers/staging/media/rkvdec/rkvdec.h | 1 +
> 3 files changed, 30 insertions(+), 17 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 8d44a884a52e..0dcbcb1bac80 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
> return 0;
> }
>
> +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> +{
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = {
> .adjust_fmt = rkvdec_h264_adjust_fmt,
> .start = rkvdec_h264_start,
> .stop = rkvdec_h264_stop,
> .run = rkvdec_h264_run,
> + .try_ctrl = rkvdec_h264_try_ctrl,
> };
> diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c
> index 2df8cf4883e2..e3d44d5b35f3 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.c
> +++ b/drivers/staging/media/rkvdec/rkvdec.c
> @@ -29,23 +29,12 @@
>
> static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl);
> + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +
> + if (desc->ops->try_ctrl)
> + return desc->ops->try_ctrl(ctx, ctrl);
> +
> return 0;
> }
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h
> index 2f4ea1786b93..9df0fba799a4 100644
> --- a/drivers/staging/media/rkvdec/rkvdec.h
> +++ b/drivers/staging/media/rkvdec/rkvdec.h
> @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops {
> void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf,
> struct vb2_v4l2_buffer *dst_buf,
> enum vb2_buffer_state result);
> + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl);
> };
>
> struct rkvdec_coded_fmt_desc {
Regards,
Hans
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-22 7:39 ` Hans Verkuil
-1 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:39 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> From: Jonas Karlman <jonas@kwiboo.se>
>
> The width and height in macroblocks is currently configured based on OUTPUT
> buffer resolution, this works for frame pictures but can cause issues for
> field pictures.
>
> When frame_mbs_only_flag is 0 the height in mbs should be height of
> the field instead of height of frame.
>
> Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
> against OUTPUT buffer resolution and use these values to configure HW.
> The validation is happening in both try_ctrt() and start() since it is
> otherwise possible to trick the driver during initialization by changing
> the OUTPUT format after having set a valid control.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
> 1 file changed, 59 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 0dcbcb1bac80..f081b476340f 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
> LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
> DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
> +
> + /* Use the SPS values since they are already in macroblocks
Move /* to a line by itself.
> + * dimensions, height can be field height (halved) if
> + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
> + * decoding smaller images into larger allocation which can be used
> + * to implementing SVC spatial layer support.
> + */
> + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
> + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
> +
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
> FRAME_MBS_ONLY_FLAG);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
> @@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
> return 0;
> }
>
> +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
> + const struct v4l2_ctrl_h264_sps *sps)
> +{
> + unsigned int width, height;
Add empty line here.
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> +
> + width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
> + height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
> +
> + /* when frame_mbs_only_flag is not set, this is field height,
Place /* on a line by itself. And 'when' should be 'When'.
Regards,
Hans
> + * which is half the final height (see (7-18) in the
> + * specification)
> + */
> + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> + height *= 2;
> +
> + if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> + height > ctx->coded_fmt.fmt.pix_mp.height)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
> {
> struct rkvdec_dev *rkvdec = ctx->dev;
> struct rkvdec_h264_priv_tbl *priv_tbl;
> struct rkvdec_h264_ctx *h264_ctx;
> + struct v4l2_ctrl *ctrl;
> int ret;
>
> + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
> + V4L2_CID_STATELESS_H264_SPS);
> + if (!ctrl)
> + return -EINVAL;
> +
> + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> + if (ret)
> + return ret;
> +
> h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
> if (!h264_ctx)
> return -ENOMEM;
> @@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>
> static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
> + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> +
> return 0;
> }
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs
@ 2022-04-22 7:39 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:39 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> From: Jonas Karlman <jonas@kwiboo.se>
>
> The width and height in macroblocks is currently configured based on OUTPUT
> buffer resolution, this works for frame pictures but can cause issues for
> field pictures.
>
> When frame_mbs_only_flag is 0 the height in mbs should be height of
> the field instead of height of frame.
>
> Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1
> against OUTPUT buffer resolution and use these values to configure HW.
> The validation is happening in both try_ctrt() and start() since it is
> otherwise possible to trick the driver during initialization by changing
> the OUTPUT format after having set a valid control.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 78 ++++++++++++++++------
> 1 file changed, 59 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index 0dcbcb1bac80..f081b476340f 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -672,8 +672,16 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx,
> LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO),
> DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS);
> - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS);
> +
> + /* Use the SPS values since they are already in macroblocks
Move /* to a line by itself.
> + * dimensions, height can be field height (halved) if
> + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows
> + * decoding smaller images into larger allocation which can be used
> + * to implementing SVC spatial layer support.
> + */
> + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS);
> + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS);
> +
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY),
> FRAME_MBS_ONLY_FLAG);
> WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD),
> @@ -1035,13 +1043,59 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx,
> return 0;
> }
>
> +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx,
> + const struct v4l2_ctrl_h264_sps *sps)
> +{
> + unsigned int width, height;
Add empty line here.
> + /*
> + * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> + * but it's currently broken in the driver.
> + * Reject them for now, until it's fixed.
> + */
> + if (sps->chroma_format_idc > 1)
> + /* Only 4:0:0 and 4:2:0 are supported */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> + /* Luma and chroma bit depth mismatch */
> + return -EINVAL;
> + if (sps->bit_depth_luma_minus8 != 0)
> + /* Only 8-bit is supported */
> + return -EINVAL;
> +
> + width = (sps->pic_width_in_mbs_minus1 + 1) * 16;
> + height = (sps->pic_height_in_map_units_minus1 + 1) * 16;
> +
> + /* when frame_mbs_only_flag is not set, this is field height,
Place /* on a line by itself. And 'when' should be 'When'.
Regards,
Hans
> + * which is half the final height (see (7-18) in the
> + * specification)
> + */
> + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
> + height *= 2;
> +
> + if (width > ctx->coded_fmt.fmt.pix_mp.width ||
> + height > ctx->coded_fmt.fmt.pix_mp.height)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> static int rkvdec_h264_start(struct rkvdec_ctx *ctx)
> {
> struct rkvdec_dev *rkvdec = ctx->dev;
> struct rkvdec_h264_priv_tbl *priv_tbl;
> struct rkvdec_h264_ctx *h264_ctx;
> + struct v4l2_ctrl *ctrl;
> int ret;
>
> + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl,
> + V4L2_CID_STATELESS_H264_SPS);
> + if (!ctrl)
> + return -EINVAL;
> +
> + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> + if (ret)
> + return ret;
> +
> h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL);
> if (!h264_ctx)
> return -ENOMEM;
> @@ -1139,23 +1193,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx)
>
> static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl)
> {
> - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) {
> - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps;
> - /*
> - * TODO: The hardware supports 10-bit and 4:2:2 profiles,
> - * but it's currently broken in the driver.
> - * Reject them for now, until it's fixed.
> - */
> - if (sps->chroma_format_idc > 1)
> - /* Only 4:0:0 and 4:2:0 are supported */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> - /* Luma and chroma bit depth mismatch */
> - return -EINVAL;
> - if (sps->bit_depth_luma_minus8 != 0)
> - /* Only 8-bit is supported */
> - return -EINVAL;
> - }
> + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS)
> + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps);
> +
> return 0;
> }
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-22 7:43 ` Hans Verkuil
-1 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:43 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> From: Jonas Karlman <jonas@kwiboo.se>
>
> When decoding the second field in a complementary field pair the second
> field is sharing the same frame_num with the first field.
>
> Currently the frame_num for the first field is wrapped when it matches the
> field being decoded, this cause issues to decode the second field in a
cause issues to decode -> caused issues decoding
> complementary field pair.
>
> Fix this by using inclusive comparison, less than or equal.
I would change this last sentence to:
Fix this by using inclusive comparison: 'less than or equal'.
It makes it a bit easier to parse.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index f081b476340f..60eaf06b6e25 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> continue;
>
> if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
> - dpb[i].frame_num < dec_params->frame_num) {
> + dpb[i].frame_num <= dec_params->frame_num) {
I wonder if a comment should be added here, explaining the reason for '<='.
It doesn't seem obvious to me. Up to you, though.
> p[i] = dpb[i].frame_num;
> continue;
> }
Regards,
Hans
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
@ 2022-04-22 7:43 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:43 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Mauro Carvalho Chehab,
Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> From: Jonas Karlman <jonas@kwiboo.se>
>
> When decoding the second field in a complementary field pair the second
> field is sharing the same frame_num with the first field.
>
> Currently the frame_num for the first field is wrapped when it matches the
> field being decoded, this cause issues to decode the second field in a
cause issues to decode -> caused issues decoding
> complementary field pair.
>
> Fix this by using inclusive comparison, less than or equal.
I would change this last sentence to:
Fix this by using inclusive comparison: 'less than or equal'.
It makes it a bit easier to parse.
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> ---
> drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> index f081b476340f..60eaf06b6e25 100644
> --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> @@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> continue;
>
> if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
> - dpb[i].frame_num < dec_params->frame_num) {
> + dpb[i].frame_num <= dec_params->frame_num) {
I wonder if a comment should be added here, explaining the reason for '<='.
It doesn't seem obvious to me. Up to you, though.
> p[i] = dpb[i].frame_num;
> continue;
> }
Regards,
Hans
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 23/24] media: hantro: Add H.264 field decoding support
2022-04-05 20:44 ` Nicolas Dufresne
@ 2022-04-22 7:49 ` Hans Verkuil
-1 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:49 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> This adds the required code to support field decoding. While most of
> the code is derived from Rockchip and VSI reference code, the
> reduction of the reference list to 16 entries was found by
> trial and errors. The list consists of all the references with the
> opposite field parity.
>
> The strategy is to deduplicate the reference picture that points
> to the same storage (same index). The choice of opposite parity has
> been made to keep the other field of the current field pair in the
> list. This method may not be robust if a field was lost.
Should this 'not robust if field is lost' be mentioned in a comment with
the code?
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/hantro/hantro_h264.c | 107 ++++++++++++++++++---
> drivers/staging/media/hantro/hantro_hw.h | 1 +
> 2 files changed, 94 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
> index 7377fc26f780..f6fc939aa726 100644
> --- a/drivers/staging/media/hantro/hantro_h264.c
> +++ b/drivers/staging/media/hantro/hantro_h264.c
> @@ -22,6 +22,11 @@
> #define POC_BUFFER_SIZE 34
> #define SCALING_LIST_SIZE (6 * 16 + 2 * 64)
>
> +/* For valid and long term reference marking, index are reversed, so bit 31
Keep /* on a line by its own.
> + * indicates the status of the picture 0.
> + */
> +#define REF_BIT(i) BIT(32 - 1 - (i))
> +
> /* Data structure describing auxiliary buffer format. */
> struct hantro_h264_dec_priv_tbl {
> u32 cabac_table[CABAC_INIT_BUFFER_SIZE];
> @@ -227,6 +232,7 @@ static void prepare_table(struct hantro_ctx *ctx)
> {
> const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls;
> const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode;
> + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps;
> struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu;
> const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
> u32 dpb_longterm = 0;
> @@ -237,20 +243,45 @@ static void prepare_table(struct hantro_ctx *ctx)
> tbl->poc[i * 2] = dpb[i].top_field_order_cnt;
> tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt;
>
> + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID))
> + continue;
> +
> /*
> * Set up bit maps of valid and long term DPBs.
> - * NOTE: The bits are reversed, i.e. MSb is DPB 0.
> + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame
> + * decoding, bit 31 to 15 are used, while for field decoding,
> + * all bits are used, with bit 31 being a top field, 30 a bottom
> + * field and so on.
> */
> - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
> - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
> - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
> - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
> + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) {
> + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF)
> + dpb_valid |= REF_BIT(i * 2);
> +
> + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF)
> + dpb_valid |= REF_BIT(i * 2 + 1);
> +
> + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) {
> + dpb_longterm |= REF_BIT(i * 2);
> + dpb_longterm |= REF_BIT(i * 2 + 1);
> + }
> + } else {
> + dpb_valid |= REF_BIT(i);
> +
> + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
> + dpb_longterm |= REF_BIT(i);
> + }
> + }
> + ctx->h264_dec.dpb_valid = dpb_valid;
> + ctx->h264_dec.dpb_longterm = dpb_longterm;
> +
> + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) ||
> + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) {
> + tbl->poc[32] = ctx->h264_dec.cur_poc;
> + tbl->poc[33] = 0;
> + } else {
> + tbl->poc[32] = dec_param->top_field_order_cnt;
> + tbl->poc[33] = dec_param->bottom_field_order_cnt;
> }
> - ctx->h264_dec.dpb_valid = dpb_valid << 16;
> - ctx->h264_dec.dpb_longterm = dpb_longterm << 16;
> -
> - tbl->poc[32] = dec_param->top_field_order_cnt;
> - tbl->poc[33] = dec_param->bottom_field_order_cnt;
>
> assemble_scaling_list(ctx);
> }
> @@ -326,6 +357,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
> {
> struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
> dma_addr_t dma_addr = 0;
> + s32 cur_poc = ctx->h264_dec.cur_poc;
> + u32 flags;
>
> if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
> dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts);
> @@ -343,7 +376,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
> dma_addr = hantro_get_dec_buf_addr(ctx, buf);
> }
>
> - return dma_addr;
> + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0;
> + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) <
> + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ?
> + 0x1 : 0;
> +
> + return dma_addr | flags;
> }
>
> u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
> @@ -355,6 +393,34 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
> return dpb->frame_num;
> }
>
> +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b,
> + struct v4l2_h264_reference *reflist)
> +{
> + int write_idx = 0;
> + int i;
> +
> + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) {
> + write_idx = b->num_valid;
> + goto done;
> + }
> +
> + for (i = 0; i < b->num_valid; i++) {
> + if (!(b->cur_pic_fields == reflist[i].fields)) {
> + reflist[write_idx++] = reflist[i];
> + continue;
> + }
> + }
> +
> +done:
> + /* Should not happen unless we have a bug in the reflist builder. */
> + if (WARN_ON(write_idx > 16))
> + write_idx = 16;
> +
> + /* Clear the remaining, some streams fails otherwise */
> + for (; write_idx < 16; write_idx++)
> + reflist[write_idx].index = 15;
> +}
> +
> int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
> {
> struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec;
> @@ -386,15 +452,28 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
> /* Update the DPB with new refs. */
> update_dpb(ctx);
>
> - /* Prepare data in memory. */
> - prepare_table(ctx);
> -
> /* Build the P/B{0,1} ref lists. */
> v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode,
> ctrls->sps, ctx->h264_dec.dpb);
> + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count;
> +
> + /* Prepare data in memory. */
> + prepare_table(ctx);
> +
> v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p);
> v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0,
> h264_ctx->reflists.b1);
> +
> + /* Reduce ref lists to at most 16 entries, Hantro hardware will deduce
/* on a line by its own.
> + * the actual picture lists in field through the dpb_valid,
> + * dpb_longterm bitmap along with the current frame parity.
> + */
> + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) {
> + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p);
> + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0);
> + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1);
> + }
> +
> return 0;
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index 292aaaabaf24..fd869369fb97 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx {
> struct hantro_h264_dec_ctrls ctrls;
> u32 dpb_longterm;
> u32 dpb_valid;
> + s32 cur_poc;
> };
>
> /**
Regards,
Hans
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 23/24] media: hantro: Add H.264 field decoding support
@ 2022-04-22 7:49 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 7:49 UTC (permalink / raw)
To: Nicolas Dufresne, Ezequiel Garcia, Philipp Zabel,
Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, linux-media, linux-rockchip,
linux-staging
On 05/04/2022 22:44, Nicolas Dufresne wrote:
> This adds the required code to support field decoding. While most of
> the code is derived from Rockchip and VSI reference code, the
> reduction of the reference list to 16 entries was found by
> trial and errors. The list consists of all the references with the
> opposite field parity.
>
> The strategy is to deduplicate the reference picture that points
> to the same storage (same index). The choice of opposite parity has
> been made to keep the other field of the current field pair in the
> list. This method may not be robust if a field was lost.
Should this 'not robust if field is lost' be mentioned in a comment with
the code?
>
> Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> ---
> drivers/staging/media/hantro/hantro_h264.c | 107 ++++++++++++++++++---
> drivers/staging/media/hantro/hantro_hw.h | 1 +
> 2 files changed, 94 insertions(+), 14 deletions(-)
>
> diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c
> index 7377fc26f780..f6fc939aa726 100644
> --- a/drivers/staging/media/hantro/hantro_h264.c
> +++ b/drivers/staging/media/hantro/hantro_h264.c
> @@ -22,6 +22,11 @@
> #define POC_BUFFER_SIZE 34
> #define SCALING_LIST_SIZE (6 * 16 + 2 * 64)
>
> +/* For valid and long term reference marking, index are reversed, so bit 31
Keep /* on a line by its own.
> + * indicates the status of the picture 0.
> + */
> +#define REF_BIT(i) BIT(32 - 1 - (i))
> +
> /* Data structure describing auxiliary buffer format. */
> struct hantro_h264_dec_priv_tbl {
> u32 cabac_table[CABAC_INIT_BUFFER_SIZE];
> @@ -227,6 +232,7 @@ static void prepare_table(struct hantro_ctx *ctx)
> {
> const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls;
> const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode;
> + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps;
> struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu;
> const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
> u32 dpb_longterm = 0;
> @@ -237,20 +243,45 @@ static void prepare_table(struct hantro_ctx *ctx)
> tbl->poc[i * 2] = dpb[i].top_field_order_cnt;
> tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt;
>
> + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID))
> + continue;
> +
> /*
> * Set up bit maps of valid and long term DPBs.
> - * NOTE: The bits are reversed, i.e. MSb is DPB 0.
> + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame
> + * decoding, bit 31 to 15 are used, while for field decoding,
> + * all bits are used, with bit 31 being a top field, 30 a bottom
> + * field and so on.
> */
> - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
> - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
> - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
> - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i);
> + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) {
> + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF)
> + dpb_valid |= REF_BIT(i * 2);
> +
> + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF)
> + dpb_valid |= REF_BIT(i * 2 + 1);
> +
> + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) {
> + dpb_longterm |= REF_BIT(i * 2);
> + dpb_longterm |= REF_BIT(i * 2 + 1);
> + }
> + } else {
> + dpb_valid |= REF_BIT(i);
> +
> + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM)
> + dpb_longterm |= REF_BIT(i);
> + }
> + }
> + ctx->h264_dec.dpb_valid = dpb_valid;
> + ctx->h264_dec.dpb_longterm = dpb_longterm;
> +
> + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) ||
> + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) {
> + tbl->poc[32] = ctx->h264_dec.cur_poc;
> + tbl->poc[33] = 0;
> + } else {
> + tbl->poc[32] = dec_param->top_field_order_cnt;
> + tbl->poc[33] = dec_param->bottom_field_order_cnt;
> }
> - ctx->h264_dec.dpb_valid = dpb_valid << 16;
> - ctx->h264_dec.dpb_longterm = dpb_longterm << 16;
> -
> - tbl->poc[32] = dec_param->top_field_order_cnt;
> - tbl->poc[33] = dec_param->bottom_field_order_cnt;
>
> assemble_scaling_list(ctx);
> }
> @@ -326,6 +357,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
> {
> struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb;
> dma_addr_t dma_addr = 0;
> + s32 cur_poc = ctx->h264_dec.cur_poc;
> + u32 flags;
>
> if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)
> dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts);
> @@ -343,7 +376,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx,
> dma_addr = hantro_get_dec_buf_addr(ctx, buf);
> }
>
> - return dma_addr;
> + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0;
> + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) <
> + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ?
> + 0x1 : 0;
> +
> + return dma_addr | flags;
> }
>
> u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
> @@ -355,6 +393,34 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx)
> return dpb->frame_num;
> }
>
> +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b,
> + struct v4l2_h264_reference *reflist)
> +{
> + int write_idx = 0;
> + int i;
> +
> + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) {
> + write_idx = b->num_valid;
> + goto done;
> + }
> +
> + for (i = 0; i < b->num_valid; i++) {
> + if (!(b->cur_pic_fields == reflist[i].fields)) {
> + reflist[write_idx++] = reflist[i];
> + continue;
> + }
> + }
> +
> +done:
> + /* Should not happen unless we have a bug in the reflist builder. */
> + if (WARN_ON(write_idx > 16))
> + write_idx = 16;
> +
> + /* Clear the remaining, some streams fails otherwise */
> + for (; write_idx < 16; write_idx++)
> + reflist[write_idx].index = 15;
> +}
> +
> int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
> {
> struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec;
> @@ -386,15 +452,28 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx)
> /* Update the DPB with new refs. */
> update_dpb(ctx);
>
> - /* Prepare data in memory. */
> - prepare_table(ctx);
> -
> /* Build the P/B{0,1} ref lists. */
> v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode,
> ctrls->sps, ctx->h264_dec.dpb);
> + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count;
> +
> + /* Prepare data in memory. */
> + prepare_table(ctx);
> +
> v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p);
> v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0,
> h264_ctx->reflists.b1);
> +
> + /* Reduce ref lists to at most 16 entries, Hantro hardware will deduce
/* on a line by its own.
> + * the actual picture lists in field through the dpb_valid,
> + * dpb_longterm bitmap along with the current frame parity.
> + */
> + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) {
> + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p);
> + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0);
> + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1);
> + }
> +
> return 0;
> }
>
> diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> index 292aaaabaf24..fd869369fb97 100644
> --- a/drivers/staging/media/hantro/hantro_hw.h
> +++ b/drivers/staging/media/hantro/hantro_hw.h
> @@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx {
> struct hantro_h264_dec_ctrls ctrls;
> u32 dpb_longterm;
> u32 dpb_valid;
> + s32 cur_poc;
> };
>
> /**
Regards,
Hans
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
2022-04-22 6:59 ` Hans Verkuil
(?)
(?)
@ 2022-04-22 14:44 ` Nicolas Dufresne
-1 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 14:44 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 08:59 +0200, Hans Verkuil a écrit :
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
>
> It's not clear how serious this FIXME is. Either it should be fixed (here or
> later in the series), or explain why this isn't fixed yet. Presumably because
> it is not a problem now, but it might be later when some other feature is
> enabled.
I will improve this comment. I have spoken to Yunfei, and his firmware have no
interlacing support (despite the reference list size being 32 entry, so
interlaced ready). Though he said this is something he'd like to work in the
future, after this patchset have been merged of course. Sot he purpose of the
FIXME was really to ease locating were the field parity information is being
lost.
>
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
>
> How about:
>
> for (i = 0; i < 32; i++)
> ref_list[i] = i < b->num_valid ? b->index : 0x20;
>
> Or:
>
> memset(ref_list, b->index, b->num_valid);
> memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
>
> Either approach is better IMHO than using two for-loops.
Thanks, I have no strong opinion, though I think I'll use the second, with weak
rationale that is looks more like the original style.
>
> Regards,
>
> Hans
>
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 14:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 14:44 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 08:59 +0200, Hans Verkuil a écrit :
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
>
> It's not clear how serious this FIXME is. Either it should be fixed (here or
> later in the series), or explain why this isn't fixed yet. Presumably because
> it is not a problem now, but it might be later when some other feature is
> enabled.
I will improve this comment. I have spoken to Yunfei, and his firmware have no
interlacing support (despite the reference list size being 32 entry, so
interlaced ready). Though he said this is something he'd like to work in the
future, after this patchset have been merged of course. Sot he purpose of the
FIXME was really to ease locating were the field parity information is being
lost.
>
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
>
> How about:
>
> for (i = 0; i < 32; i++)
> ref_list[i] = i < b->num_valid ? b->index : 0x20;
>
> Or:
>
> memset(ref_list, b->index, b->num_valid);
> memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
>
> Either approach is better IMHO than using two for-loops.
Thanks, I have no strong opinion, though I think I'll use the second, with weak
rationale that is looks more like the original style.
>
> Regards,
>
> Hans
>
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 14:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 14:44 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 08:59 +0200, Hans Verkuil a écrit :
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
>
> It's not clear how serious this FIXME is. Either it should be fixed (here or
> later in the series), or explain why this isn't fixed yet. Presumably because
> it is not a problem now, but it might be later when some other feature is
> enabled.
I will improve this comment. I have spoken to Yunfei, and his firmware have no
interlacing support (despite the reference list size being 32 entry, so
interlaced ready). Though he said this is something he'd like to work in the
future, after this patchset have been merged of course. Sot he purpose of the
FIXME was really to ease locating were the field parity information is being
lost.
>
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
>
> How about:
>
> for (i = 0; i < 32; i++)
> ref_list[i] = i < b->num_valid ? b->index : 0x20;
>
> Or:
>
> memset(ref_list, b->index, b->num_valid);
> memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
>
> Either approach is better IMHO than using two for-loops.
Thanks, I have no strong opinion, though I think I'll use the second, with weak
rationale that is looks more like the original style.
>
> Regards,
>
> Hans
>
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 14:44 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 14:44 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 08:59 +0200, Hans Verkuil a écrit :
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
>
> It's not clear how serious this FIXME is. Either it should be fixed (here or
> later in the series), or explain why this isn't fixed yet. Presumably because
> it is not a problem now, but it might be later when some other feature is
> enabled.
I will improve this comment. I have spoken to Yunfei, and his firmware have no
interlacing support (despite the reference list size being 32 entry, so
interlaced ready). Though he said this is something he'd like to work in the
future, after this patchset have been merged of course. Sot he purpose of the
FIXME was really to ease locating were the field parity information is being
lost.
>
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
>
> How about:
>
> for (i = 0; i < 32; i++)
> ref_list[i] = i < b->num_valid ? b->index : 0x20;
>
> Or:
>
> memset(ref_list, b->index, b->num_valid);
> memset(ref_list + b->num_valid, 0x20, 32 - b->num_valid);
>
> Either approach is better IMHO than using two for-loops.
Thanks, I have no strong opinion, though I think I'll use the second, with weak
rationale that is looks more like the original style.
>
> Regards,
>
> Hans
>
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist
2022-04-22 7:26 ` Hans Verkuil
@ 2022-04-22 14:58 ` Nicolas Dufresne
2022-04-22 16:33 ` Hans Verkuil
0 siblings, 1 reply; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 14:58 UTC (permalink / raw)
To: Hans Verkuil, Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
Le vendredi 22 avril 2022 à 09:26 +0200, Hans Verkuil a écrit :
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > Add debug print statements to print the content of P & B reference
> > lists, to verify that the ordering of the generated reference lists is
> > correct. This is especially important for the field decoding mode,
> > where sorting is more complex.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Tested-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> > Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> > ---
> > drivers/media/v4l2-core/v4l2-h264.c | 86 +++++++++++++++++++++++++++++
> > 1 file changed, 86 insertions(+)
> >
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index 38d8dbda0045..bcf9b7774560 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -241,6 +241,87 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > return poca < pocb ? -1 : 1;
> > }
> >
> > +static char ref_type_to_char (u8 ref_type)
>
> Spurious space before (.
>
> Odd that checkpatch didn't catch that.
>
> > +{
> > + switch (ref_type) {
> > + case V4L2_H264_FRAME_REF:
> > + return 'f';
> > + case V4L2_H264_TOP_FIELD_REF:
> > + return 't';
> > + case V4L2_H264_BOTTOM_FIELD_REF:
> > + return 'b';
> > + }
> > +
> > + return '?';
> > +}
> > +
> > +static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
> > + struct v4l2_h264_reference *reflist,
> > + char *out_str, const int len)
> > +{
> > + int n = 0, i;
> > +
> > + n += snprintf(out_str + n, len - n, "|");
> > +
> > + for (i = 0; i < builder->num_valid; i++) {
> > + /* this is pic_num for frame and frame_num (wrapped) for field,
> > + * but for frame pic_num is equal to frame_num (wrapped).
> > + */
> > + int frame_num = builder->refs[reflist[i].index].frame_num;
> > + bool longterm = builder->refs[reflist[i].index].longterm;
> > +
> > + n += scnprintf(out_str + n, len - n, "%i%c%c|",
> > + frame_num, longterm ? 'l' : 's',
> > + ref_type_to_char (reflist[i].fields));
> > + }
> > +
> > + return out_str;
> > +}
> > +
> > +static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
> > + struct v4l2_h264_reference *reflist)
> > +{
> > + char buf[1024];
> > +
> > + pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n",
> > + builder->cur_pic_order_count,
> > + ref_type_to_char(builder->cur_pic_fields),
> > + format_ref_list_p(builder, reflist, buf, sizeof(buf)));
> > +}
> > +
> > +static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
> > + struct v4l2_h264_reference *reflist,
> > + char *out_str, const int len)
> > +{
> > + int n = 0, i;
> > +
> > + n += snprintf(out_str + n, len - n, "|");
> > +
> > + for (i = 0; i < builder->num_valid; i++) {
> > + int frame_num = builder->refs[reflist[i].index].frame_num;
> > + u32 poc = v4l2_h264_get_poc(builder, reflist + i);
> > + bool longterm = builder->refs[reflist[i].index].longterm;
> > +
> > + n += scnprintf(out_str + n, len - n, "%i%c%c|",
> > + longterm ? frame_num : poc,
> > + longterm ? 'l' : 's',
> > + ref_type_to_char(reflist[i].fields));
> > + }
> > +
> > + return out_str;
> > +}
> > +
> > +static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
> > + struct v4l2_h264_reference *reflist, u8 list_num)
> > +{
> > + char buf[1024];
>
> I really don't like placing 1024 bytes on the stack. Can you find another way
> of doing this? Perhaps using pr_cont or writing each format_ref_list item
> on a separate line.
Thanks, I was strongly discourage of using pr_cont (which was my first
approach). Rationales are well covered on LKLM and in the pr_cont documentation,
so I won't say more then its not visually thread safe.
I would like to decline the second proposition, as having the lists spread out
on up to 32 lines will make the trace very hard to use. What I may suggest, as I
would really prefer keeping this trace useful, is to use an allocation instead.
The performance does not matter, and I explicitly call this function inside the
pr_debug call so it can be compiled out.
My last resort otherwise would be to use 32 %s formaters, and pass each of the
possible 32 entry (or empty string "") manually.
let me know what you believe is acceptable for you,
Nicolas
>
> Regards,
>
> Hans
>
> > +
> > + pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s",
> > + list_num, builder->cur_pic_order_count,
> > + ref_type_to_char (builder->cur_pic_fields),
> > + format_ref_list_b(builder, reflist, buf, sizeof(buf)));
> > +}
> > +
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > @@ -261,6 +342,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > sort_r(reflist, builder->num_valid, sizeof(*reflist),
> > v4l2_h264_p_ref_list_cmp, NULL, builder);
> > +
> > + print_ref_list_p(builder, reflist);
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> >
> > @@ -296,6 +379,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > if (builder->num_valid > 1 &&
> > !memcmp(b1_reflist, b0_reflist, builder->num_valid))
> > swap(b1_reflist[0], b1_reflist[1]);
> > +
> > + print_ref_list_b(builder, b0_reflist, 0);
> > + print_ref_list_b(builder, b1_reflist, 1);
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists);
> >
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
2022-04-22 5:42 ` Hans Verkuil
(?)
(?)
@ 2022-04-22 15:23 ` Nicolas Dufresne
-1 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 15:23 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 07:42 +0200, Hans Verkuil a écrit :
> Hi Nicolas,
>
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
>
> After applying this patch building the mediatek driver fails:
>
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
> 451 | ref_list[i] = b->index;
> | ^~
Sorry about this, I normally try to enable all the drivers, but failed at it,
and forgot about this. I'll fix and I will even test for regressions as I now
have some hardware.
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> | ^~~~~~~~~~
> | |
> | u8 * {aka unsigned char *}
>
> (dropped a lot more errors)
>
> Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
>
> drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 66 | struct v4l2_h264_reference *b0_reflist,
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 67 | struct v4l2_h264_reference *b1_reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
> | ~~~~~~~~^~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 83 | struct v4l2_h264_reference *reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
>
> It looks like you didn't build the tegra-vde driver either.
Driver was added recently, I will enable and port it, I'll check with Dmitry if
he can validate, though I won't includ interlaced support there, I'll just port
to the new structure.
>
> Regards,
>
> Hans
>
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 15:23 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 15:23 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 07:42 +0200, Hans Verkuil a écrit :
> Hi Nicolas,
>
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
>
> After applying this patch building the mediatek driver fails:
>
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
> 451 | ref_list[i] = b->index;
> | ^~
Sorry about this, I normally try to enable all the drivers, but failed at it,
and forgot about this. I'll fix and I will even test for regressions as I now
have some hardware.
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> | ^~~~~~~~~~
> | |
> | u8 * {aka unsigned char *}
>
> (dropped a lot more errors)
>
> Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
>
> drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 66 | struct v4l2_h264_reference *b0_reflist,
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 67 | struct v4l2_h264_reference *b1_reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
> | ~~~~~~~~^~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 83 | struct v4l2_h264_reference *reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
>
> It looks like you didn't build the tegra-vde driver either.
Driver was added recently, I will enable and port it, I'll check with Dmitry if
he can validate, though I won't includ interlaced support there, I'll just port
to the new structure.
>
> Regards,
>
> Hans
>
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 15:23 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 15:23 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 07:42 +0200, Hans Verkuil a écrit :
> Hi Nicolas,
>
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
>
> After applying this patch building the mediatek driver fails:
>
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
> 451 | ref_list[i] = b->index;
> | ^~
Sorry about this, I normally try to enable all the drivers, but failed at it,
and forgot about this. I'll fix and I will even test for regressions as I now
have some hardware.
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> | ^~~~~~~~~~
> | |
> | u8 * {aka unsigned char *}
>
> (dropped a lot more errors)
>
> Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
>
> drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 66 | struct v4l2_h264_reference *b0_reflist,
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 67 | struct v4l2_h264_reference *b1_reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
> | ~~~~~~~~^~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 83 | struct v4l2_h264_reference *reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
>
> It looks like you didn't build the tegra-vde driver either.
Driver was added recently, I will enable and port it, I'll check with Dmitry if
he can validate, though I won't includ interlaced support there, I'll just port
to the new structure.
>
> Regards,
>
> Hans
>
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist
@ 2022-04-22 15:23 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-22 15:23 UTC (permalink / raw)
To: Hans Verkuil, Tiffany Lin, Andrew-CT Chen, Mauro Carvalho Chehab,
Ezequiel Garcia, Philipp Zabel, Greg Kroah-Hartman,
Heiko Stuebner, Matthias Brugger
Cc: kernel, linux-kernel, linux-media, linux-rockchip, linux-staging,
linux-arm-kernel, linux-mediatek
Le vendredi 22 avril 2022 à 07:42 +0200, Hans Verkuil a écrit :
> Hi Nicolas,
>
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > In preparation for adding field decoding support, convert the byte arrays
> > for reflist into array of struct v4l2_h264_reference. That struct will
> > allow us to mark which field of the reference picture is being referenced.
> >
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>
> > ---
> > .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +++-
>
> After applying this patch building the mediatek driver fails:
>
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_ref_list’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:451:32: error: ‘struct v4l2_h264_reflist_builder’ has no member named ‘index’
> 451 | ref_list[i] = b->index;
> | ^~
Sorry about this, I normally try to enable all the drivers, but failed at it,
and forgot about this. I'll fix and I will even test for regressions as I now
have some hardware.
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c: In function ‘get_vdec_decode_parameters’:
> drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c:485:54: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 485 | v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> | ^~~~~~~~~~
> | |
> | u8 * {aka unsigned char *}
>
> (dropped a lot more errors)
>
> Ditto for drivers/media/platform/nvidia/tegra-vde/h264.c:
>
> drivers/media/platform/nvidia/tegra-vde/h264.c: In function ‘tegra_vde_h264_setup_frames’:
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:57: error: passing argument 2 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:66:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 66 | struct v4l2_h264_reference *b0_reflist,
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:806:70: error: passing argument 3 of ‘v4l2_h264_build_b_ref_lists’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 806 | v4l2_h264_build_b_ref_lists(&b, reflists.b0, reflists.b1);
> | ~~~~~~~~^~~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:67:57: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 67 | struct v4l2_h264_reference *b1_reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
> drivers/media/platform/nvidia/tegra-vde/h264.c:809:56: error: passing argument 2 of ‘v4l2_h264_build_p_ref_list’ from incompatible pointer type [-Werror=incompatible-pointer-types]
> 809 | v4l2_h264_build_p_ref_list(&b, reflists.p);
> | ~~~~~~~~^~
> | |
> | u8 * {aka unsigned char *}
> In file included from drivers/media/platform/nvidia/tegra-vde/h264.c:14:
> ./include/media/v4l2-h264.h:83:56: note: expected ‘struct v4l2_h264_reference *’ but argument is of type ‘u8 *’ {aka ‘unsigned char *’}
> 83 | struct v4l2_h264_reference *reflist);
> | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
>
> It looks like you didn't build the tegra-vde driver either.
Driver was added recently, I will enable and port it, I'll check with Dmitry if
he can validate, though I won't includ interlaced support there, I'll just port
to the new structure.
>
> Regards,
>
> Hans
>
> > drivers/media/v4l2-core/v4l2-h264.c | 33 ++++---
> > .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++----
> > drivers/staging/media/hantro/hantro_hw.h | 6 +-
> > .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++----------
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +--
> > include/media/v4l2-h264.h | 19 ++--
> > 7 files changed, 116 insertions(+), 107 deletions(-)
> >
> > diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > index 43542de11e9c..72c599e05a47 100644
> > --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c
> > @@ -442,9 +442,16 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param,
> > /*
> > * The firmware expects unused reflist entries to have the value 0x20.
> > */
> > -static void fixup_ref_list(u8 *ref_list, size_t num_valid)
> > +static void get_ref_list(u8 *ref_list, struct v4l2_h264_reflist_builder *b)
> > {
> > - memset(&ref_list[num_valid], 0x20, 32 - num_valid);
> > + u32 i;
> > +
> > + /* FIXME mark the reference parity */
> > + for (i = 0; i < b->num_valid; i++)
> > + ref_list[i] = b->index;
> > +
> > + for (; i < 32; i++)
> > + ref_list[i] = 0x20;
> > }
> >
> > static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > @@ -478,9 +485,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst)
> > v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist);
> > v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist);
> > /* Adapt the built lists to the firmware's expectations */
> > - fixup_ref_list(p0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b0_reflist, reflist_builder.num_valid);
> > - fixup_ref_list(b1_reflist, reflist_builder.num_valid);
> > + get_ref_list(p0_reflist, reflist_builder);
> > + get_ref_list(b0_reflist, reflist_builder);
> > + get_ref_list(b1_reflist, reflist_builder);
> >
> > memcpy(&inst->vsi_ctx.h264_slice_params, slice_param,
> > sizeof(inst->vsi_ctx.h264_slice_params));
> > diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
> > index ac47519a9fbe..afbfcf78efe4 100644
> > --- a/drivers/media/v4l2-core/v4l2-h264.c
> > +++ b/drivers/media/v4l2-core/v4l2-h264.c
> > @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > pic_order_count = dpb[i].top_field_order_cnt;
> >
> > b->refs[i].pic_order_count = pic_order_count;
> > - b->unordered_reflist[b->num_valid] = i;
> > + b->unordered_reflist[b->num_valid].index = i;
> > b->num_valid++;
> > }
> >
> > for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++)
> > - b->unordered_reflist[i] = i;
> > + b->unordered_reflist[i].index = i;
> > }
> > EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder);
> >
> > @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb,
> > const struct v4l2_h264_reflist_builder *builder = data;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > s32 poca, pocb;
> > u8 idxa, idxb;
> >
> > - idxa = *((u8 *)ptra);
> > - idxb = *((u8 *)ptrb);
> > + idxa = ((struct v4l2_h264_reference *)ptra)->index;
> > + idxb = ((struct v4l2_h264_reference *)ptrb)->index;
> >
> > if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES ||
> > idxb >= V4L2_H264_NUM_DPB_ENTRIES))
> > @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist)
> > + struct v4l2_h264_reference *reflist)
> > {
> > memcpy(reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist)
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist)
> > {
> > memcpy(b0_reflist, builder->unordered_reflist,
> > sizeof(builder->unordered_reflist[0]) * builder->num_valid);
> > diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > index f49dbfb8a843..9de7f05eff2a 100644
> > --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c
> > @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > int reg_num;
> > u32 reg;
> > @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 0; i < 15; i += 3) {
> > - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) |
> > - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]);
> > + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) |
> > + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++));
> > }
> >
> > @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > * of forward and backward reference picture lists and first 4 entries
> > * of P forward picture list.
> > */
> > - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) |
> > - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]);
> > + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) |
> > + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC);
> >
> > /*
> > @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx)
> > */
> > reg_num = 0;
> > for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) {
> > - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) |
> > - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]);
> > + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) |
> > + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index);
> > vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++));
> > }
> >
> > diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
> > index ed018e293ba0..2bc6b8f088f5 100644
> > --- a/drivers/staging/media/hantro/hantro_hw.h
> > +++ b/drivers/staging/media/hantro/hantro_hw.h
> > @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls {
> > * @b1: B1 reflist
> > */
> > struct hantro_h264_dec_reflists {
> > - u8 p[HANTRO_H264_DPB_SIZE];
> > - u8 b0[HANTRO_H264_DPB_SIZE];
> > - u8 b1[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE];
> > };
> >
> > /**
> > diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > index 64a6330475eb..46c1a83bcc4e 100644
> > --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c
> > @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf)
> >
> > static void set_ref(struct hantro_ctx *ctx)
> > {
> > - const u8 *b0_reflist, *b1_reflist, *p_reflist;
> > + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist;
> > struct hantro_dev *vpu = ctx->dev;
> > u32 reg;
> > int i;
> > @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx)
> > b1_reflist = ctx->h264_dec.reflists.b1;
> > p_reflist = ctx->h264_dec.reflists.p;
> >
> > - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) |
> > - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) |
> > - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) |
> > - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) |
> > - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) |
> > - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]);
> > + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) |
> > + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) |
> > + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) |
> > + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) |
> > + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) |
> > + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) |
> > - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) |
> > - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) |
> > - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) |
> > - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) |
> > - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]);
> > + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) |
> > + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) |
> > + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) |
> > + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) |
> > + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) |
> > + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75));
> >
> > reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) |
> > @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx)
> > VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14));
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101));
> >
> > - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) |
> > - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) |
> > - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) |
> > - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) |
> > - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) |
> > - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]);
> > + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) |
> > + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) |
> > + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) |
> > + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) |
> > + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) |
> > + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) |
> > - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) |
> > - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) |
> > - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) |
> > - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) |
> > - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]);
> > + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) |
> > + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) |
> > + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) |
> > + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) |
> > + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) |
> > + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104));
> >
> > - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) |
> > - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) |
> > - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) |
> > - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]);
> > + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) |
> > + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) |
> > + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) |
> > + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105));
> >
> > - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) |
> > - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) |
> > - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) |
> > - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]);
> > + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) |
> > + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) |
> > + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) |
> > + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index);
> > vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106));
> >
> > reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm);
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index 951e19231da2..3c7f3d87fab4 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl {
> > #define RKVDEC_H264_DPB_SIZE 16
> >
> > struct rkvdec_h264_reflists {
> > - u8 p[RKVDEC_H264_DPB_SIZE];
> > - u8 b0[RKVDEC_H264_DPB_SIZE];
> > - u8 b1[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE];
> > + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE];
> > u8 num_valid;
> > };
> >
> > @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> >
> > switch (j) {
> > case 0:
> > - idx = h264_ctx->reflists.p[i];
> > + idx = h264_ctx->reflists.p[i].index;
> > break;
> > case 1:
> > - idx = h264_ctx->reflists.b0[i];
> > + idx = h264_ctx->reflists.b0[i].index;
> > break;
> > case 2:
> > - idx = h264_ctx->reflists.b1[i];
> > + idx = h264_ctx->reflists.b1[i].index;
> > break;
> > }
> >
> > diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h
> > index 4b1c71c935e0..ef9a894e3c32 100644
> > --- a/include/media/v4l2-h264.h
> > +++ b/include/media/v4l2-h264.h
> > @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder {
> > u16 longterm : 1;
> > } refs[V4L2_H264_NUM_DPB_ENTRIES];
> > s32 cur_pic_order_count;
> > - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES];
> > u8 num_valid;
> > };
> >
> > @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists
> > *
> > * @builder: reference list builder context
> > - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry
> > - * is an index in the DPB
> > - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry
> > - * is an index in the DPB
> > + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the B0/B1 reference lists. This procedure is described
> > * in section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b,
> > */
> > void
> > v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *b0_reflist, u8 *b1_reflist);
> > + struct v4l2_h264_reference *b0_reflist,
> > + struct v4l2_h264_reference *b1_reflist);
> >
> > /**
> > * v4l2_h264_build_p_ref_list() - Build the P reference list
> > *
> > * @builder: reference list builder context
> > - * @reflist: 16-bytes array used to store the P reference list. Each entry
> > - * is an index in the DPB
> > + * @reflist: 16 sized array used to store the P reference list. Each entry
> > + * is a v4l2_h264_reference structure
> > *
> > * This functions builds the P reference lists. This procedure is describe in
> > * section '8.2.4 Decoding process for reference picture lists construction'
> > @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
> > */
> > void
> > v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
> > - u8 *reflist);
> > + struct v4l2_h264_reference *reflist);
> >
> > #endif /* _MEDIA_V4L2_H264_H */
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist
2022-04-22 14:58 ` Nicolas Dufresne
@ 2022-04-22 16:33 ` Hans Verkuil
0 siblings, 0 replies; 86+ messages in thread
From: Hans Verkuil @ 2022-04-22 16:33 UTC (permalink / raw)
To: Nicolas Dufresne, Mauro Carvalho Chehab
Cc: kernel, linux-kernel, Sebastian Fricke, linux-media
On 22/04/2022 16:58, Nicolas Dufresne wrote:
> Le vendredi 22 avril 2022 à 09:26 +0200, Hans Verkuil a écrit :
>> On 05/04/2022 22:44, Nicolas Dufresne wrote:
>>> Add debug print statements to print the content of P & B reference
>>> lists, to verify that the ordering of the generated reference lists is
>>> correct. This is especially important for the field decoding mode,
>>> where sorting is more complex.
>>>
>>> Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
>>> Tested-by: Sebastian Fricke <sebastian.fricke@collabora.com>
>>> Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
>>> ---
>>> drivers/media/v4l2-core/v4l2-h264.c | 86 +++++++++++++++++++++++++++++
>>> 1 file changed, 86 insertions(+)
>>>
>>> diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c
>>> index 38d8dbda0045..bcf9b7774560 100644
>>> --- a/drivers/media/v4l2-core/v4l2-h264.c
>>> +++ b/drivers/media/v4l2-core/v4l2-h264.c
>>> @@ -241,6 +241,87 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb,
>>> return poca < pocb ? -1 : 1;
>>> }
>>>
>>> +static char ref_type_to_char (u8 ref_type)
>>
>> Spurious space before (.
>>
>> Odd that checkpatch didn't catch that.
>>
>>> +{
>>> + switch (ref_type) {
>>> + case V4L2_H264_FRAME_REF:
>>> + return 'f';
>>> + case V4L2_H264_TOP_FIELD_REF:
>>> + return 't';
>>> + case V4L2_H264_BOTTOM_FIELD_REF:
>>> + return 'b';
>>> + }
>>> +
>>> + return '?';
>>> +}
>>> +
>>> +static const char *format_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
>>> + struct v4l2_h264_reference *reflist,
>>> + char *out_str, const int len)
>>> +{
>>> + int n = 0, i;
>>> +
>>> + n += snprintf(out_str + n, len - n, "|");
>>> +
>>> + for (i = 0; i < builder->num_valid; i++) {
>>> + /* this is pic_num for frame and frame_num (wrapped) for field,
>>> + * but for frame pic_num is equal to frame_num (wrapped).
>>> + */
>>> + int frame_num = builder->refs[reflist[i].index].frame_num;
>>> + bool longterm = builder->refs[reflist[i].index].longterm;
>>> +
>>> + n += scnprintf(out_str + n, len - n, "%i%c%c|",
>>> + frame_num, longterm ? 'l' : 's',
>>> + ref_type_to_char (reflist[i].fields));
>>> + }
>>> +
>>> + return out_str;
>>> +}
>>> +
>>> +static void print_ref_list_p(const struct v4l2_h264_reflist_builder *builder,
>>> + struct v4l2_h264_reference *reflist)
>>> +{
>>> + char buf[1024];
>>> +
>>> + pr_debug("ref_pic_list_p (cur_poc %u%c) %s\n",
>>> + builder->cur_pic_order_count,
>>> + ref_type_to_char(builder->cur_pic_fields),
>>> + format_ref_list_p(builder, reflist, buf, sizeof(buf)));
>>> +}
>>> +
>>> +static const char *format_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
>>> + struct v4l2_h264_reference *reflist,
>>> + char *out_str, const int len)
>>> +{
>>> + int n = 0, i;
>>> +
>>> + n += snprintf(out_str + n, len - n, "|");
>>> +
>>> + for (i = 0; i < builder->num_valid; i++) {
>>> + int frame_num = builder->refs[reflist[i].index].frame_num;
>>> + u32 poc = v4l2_h264_get_poc(builder, reflist + i);
>>> + bool longterm = builder->refs[reflist[i].index].longterm;
>>> +
>>> + n += scnprintf(out_str + n, len - n, "%i%c%c|",
>>> + longterm ? frame_num : poc,
>>> + longterm ? 'l' : 's',
>>> + ref_type_to_char(reflist[i].fields));
>>> + }
>>> +
>>> + return out_str;
>>> +}
>>> +
>>> +static void print_ref_list_b(const struct v4l2_h264_reflist_builder *builder,
>>> + struct v4l2_h264_reference *reflist, u8 list_num)
>>> +{
>>> + char buf[1024];
>>
>> I really don't like placing 1024 bytes on the stack. Can you find another way
>> of doing this? Perhaps using pr_cont or writing each format_ref_list item
>> on a separate line.
>
> Thanks, I was strongly discourage of using pr_cont (which was my first
> approach). Rationales are well covered on LKLM and in the pr_cont documentation,
> so I won't say more then its not visually thread safe.
>
> I would like to decline the second proposition, as having the lists spread out
> on up to 32 lines will make the trace very hard to use. What I may suggest, as I
> would really prefer keeping this trace useful, is to use an allocation instead.
> The performance does not matter, and I explicitly call this function inside the
> pr_debug call so it can be compiled out.
Allocation is probably best here.
Regards,
Hans
>
> My last resort otherwise would be to use 32 %s formaters, and pass each of the
> possible 32 entry (or empty string "") manually.
>
> let me know what you believe is acceptable for you,
> Nicolas
>
>>
>> Regards,
>>
>> Hans
>>
>>> +
>>> + pr_debug("ref_pic_list_b%u (cur_poc %u%c) %s",
>>> + list_num, builder->cur_pic_order_count,
>>> + ref_type_to_char (builder->cur_pic_fields),
>>> + format_ref_list_b(builder, reflist, buf, sizeof(buf)));
>>> +}
>>> +
>>> /**
>>> * v4l2_h264_build_p_ref_list() - Build the P reference list
>>> *
>>> @@ -261,6 +342,8 @@ v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder,
>>> sizeof(builder->unordered_reflist[0]) * builder->num_valid);
>>> sort_r(reflist, builder->num_valid, sizeof(*reflist),
>>> v4l2_h264_p_ref_list_cmp, NULL, builder);
>>> +
>>> + print_ref_list_p(builder, reflist);
>>> }
>>> EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list);
>>>
>>> @@ -296,6 +379,9 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder,
>>> if (builder->num_valid > 1 &&
>>> !memcmp(b1_reflist, b0_reflist, builder->num_valid))
>>> swap(b1_reflist[0], b1_reflist[1]);
>>> +
>>> + print_ref_list_b(builder, b0_reflist, 0);
>>> + print_ref_list_b(builder, b1_reflist, 1);
>>> }
>>> EXPORT_SYMBOL_GPL(v4l2_h264_build_b_ref_lists);
>>>
>>
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
2022-04-22 7:43 ` Hans Verkuil
@ 2022-04-25 18:55 ` Nicolas Dufresne
-1 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-25 18:55 UTC (permalink / raw)
To: Hans Verkuil, Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
Le vendredi 22 avril 2022 à 09:43 +0200, Hans Verkuil a écrit :
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > From: Jonas Karlman <jonas@kwiboo.se>
> >
> > When decoding the second field in a complementary field pair the second
> > field is sharing the same frame_num with the first field.
> >
> > Currently the frame_num for the first field is wrapped when it matches the
> > field being decoded, this cause issues to decode the second field in a
>
> cause issues to decode -> caused issues decoding
>
> > complementary field pair.
> >
> > Fix this by using inclusive comparison, less than or equal.
>
> I would change this last sentence to:
>
> Fix this by using inclusive comparison: 'less than or equal'.
>
> It makes it a bit easier to parse.
>
> >
> > Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
> > Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> > ---
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index f081b476340f..60eaf06b6e25 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> > continue;
> >
> > if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
> > - dpb[i].frame_num < dec_params->frame_num) {
> > + dpb[i].frame_num <= dec_params->frame_num) {
>
> I wonder if a comment should be added here, explaining the reason for '<='.
>
> It doesn't seem obvious to me. Up to you, though.
I guess I could, the algo for wrapping in the spec is (formula 8-27):
if( FrameNum > frame_num )
FrameNumWrap = FrameNum − MaxFrameNum
else
FrameNumWrap = FrameNum
Our implementation has the branch condition flip over, and the flipped version of that is:
if( FrameNum <= frame_num )
FrameNumWrap = FrameNum
else
FrameNumWrap = FrameNum − MaxFrameNum
There is no deeper rationale since we simply follow the recipe described in the
spec. This is done so that we can share that condition with that long term
reference handling.
>
> > p[i] = dpb[i].frame_num;
> > continue;
> > }
>
> Regards,
>
> Hans
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
@ 2022-04-25 18:55 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-25 18:55 UTC (permalink / raw)
To: Hans Verkuil, Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
Le vendredi 22 avril 2022 à 09:43 +0200, Hans Verkuil a écrit :
> On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > From: Jonas Karlman <jonas@kwiboo.se>
> >
> > When decoding the second field in a complementary field pair the second
> > field is sharing the same frame_num with the first field.
> >
> > Currently the frame_num for the first field is wrapped when it matches the
> > field being decoded, this cause issues to decode the second field in a
>
> cause issues to decode -> caused issues decoding
>
> > complementary field pair.
> >
> > Fix this by using inclusive comparison, less than or equal.
>
> I would change this last sentence to:
>
> Fix this by using inclusive comparison: 'less than or equal'.
>
> It makes it a bit easier to parse.
>
> >
> > Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
> > Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> > ---
> > drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > index f081b476340f..60eaf06b6e25 100644
> > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > @@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> > continue;
> >
> > if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
> > - dpb[i].frame_num < dec_params->frame_num) {
> > + dpb[i].frame_num <= dec_params->frame_num) {
>
> I wonder if a comment should be added here, explaining the reason for '<='.
>
> It doesn't seem obvious to me. Up to you, though.
I guess I could, the algo for wrapping in the spec is (formula 8-27):
if( FrameNum > frame_num )
FrameNumWrap = FrameNum − MaxFrameNum
else
FrameNumWrap = FrameNum
Our implementation has the branch condition flip over, and the flipped version of that is:
if( FrameNum <= frame_num )
FrameNumWrap = FrameNum
else
FrameNumWrap = FrameNum − MaxFrameNum
There is no deeper rationale since we simply follow the recipe described in the
spec. This is done so that we can share that condition with that long term
reference handling.
>
> > p[i] = dpb[i].frame_num;
> > continue;
> > }
>
> Regards,
>
> Hans
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
2022-04-25 18:55 ` Nicolas Dufresne
@ 2022-04-25 19:00 ` Nicolas Dufresne
-1 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-25 19:00 UTC (permalink / raw)
To: Hans Verkuil, Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
Le lundi 25 avril 2022 à 14:55 -0400, Nicolas Dufresne a écrit :
> Le vendredi 22 avril 2022 à 09:43 +0200, Hans Verkuil a écrit :
> > On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > > From: Jonas Karlman <jonas@kwiboo.se>
> > >
> > > When decoding the second field in a complementary field pair the second
> > > field is sharing the same frame_num with the first field.
> > >
> > > Currently the frame_num for the first field is wrapped when it matches the
> > > field being decoded, this cause issues to decode the second field in a
> >
> > cause issues to decode -> caused issues decoding
> >
> > > complementary field pair.
> > >
> > > Fix this by using inclusive comparison, less than or equal.
> >
> > I would change this last sentence to:
> >
> > Fix this by using inclusive comparison: 'less than or equal'.
> >
> > It makes it a bit easier to parse.
> >
> > >
> > > Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> > > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > > Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
> > > Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> > > ---
> > > drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > > index f081b476340f..60eaf06b6e25 100644
> > > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > > @@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> > > continue;
> > >
> > > if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
> > > - dpb[i].frame_num < dec_params->frame_num) {
> > > + dpb[i].frame_num <= dec_params->frame_num) {
> >
> > I wonder if a comment should be added here, explaining the reason for '<='.
> >
> > It doesn't seem obvious to me. Up to you, though.
>
> I guess I could, the algo for wrapping in the spec is (formula 8-27):
>
> if( FrameNum > frame_num )
> FrameNumWrap = FrameNum − MaxFrameNum
> else
> FrameNumWrap = FrameNum
>
> Our implementation has the branch condition flip over, and the flipped version of that is:
>
> if( FrameNum <= frame_num )
> FrameNumWrap = FrameNum
> else
> FrameNumWrap = FrameNum − MaxFrameNum
>
> There is no deeper rationale since we simply follow the recipe described in the
> spec. This is done so that we can share that condition with that long term
> reference handling.
Now I come to realize that in patch "[v3,19/24] media: rkvdec-h264: Add field
decoding support" all this code is removed. This is because the wrapping is
already done by the ref-builder, so while enabling field decoding, I now use the
wrapped value from the ref builder. I'm mostly keeping the patch so that this
fix is well documented. I will leave it like this then.
>
> >
> > > p[i] = dpb[i].frame_num;
> > > continue;
> > > }
> >
> > Regards,
> >
> > Hans
>
^ permalink raw reply [flat|nested] 86+ messages in thread
* Re: [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field
@ 2022-04-25 19:00 ` Nicolas Dufresne
0 siblings, 0 replies; 86+ messages in thread
From: Nicolas Dufresne @ 2022-04-25 19:00 UTC (permalink / raw)
To: Hans Verkuil, Ezequiel Garcia, Mauro Carvalho Chehab, Greg Kroah-Hartman
Cc: kernel, linux-kernel, Jonas Karlman, Ezequiel Garcia,
Sebastian Fricke, linux-media, linux-rockchip, linux-staging
Le lundi 25 avril 2022 à 14:55 -0400, Nicolas Dufresne a écrit :
> Le vendredi 22 avril 2022 à 09:43 +0200, Hans Verkuil a écrit :
> > On 05/04/2022 22:44, Nicolas Dufresne wrote:
> > > From: Jonas Karlman <jonas@kwiboo.se>
> > >
> > > When decoding the second field in a complementary field pair the second
> > > field is sharing the same frame_num with the first field.
> > >
> > > Currently the frame_num for the first field is wrapped when it matches the
> > > field being decoded, this cause issues to decode the second field in a
> >
> > cause issues to decode -> caused issues decoding
> >
> > > complementary field pair.
> > >
> > > Fix this by using inclusive comparison, less than or equal.
> >
> > I would change this last sentence to:
> >
> > Fix this by using inclusive comparison: 'less than or equal'.
> >
> > It makes it a bit easier to parse.
> >
> > >
> > > Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
> > > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
> > > Reviewed-by: Ezequiel Garcia <ezequiel@collabora.com>
> > > Reviewed-by: Sebastian Fricke <sebastian.fricke@collabora.com>
> > > ---
> > > drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > > index f081b476340f..60eaf06b6e25 100644
> > > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c
> > > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c
> > > @@ -781,7 +781,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx,
> > > continue;
> > >
> > > if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM ||
> > > - dpb[i].frame_num < dec_params->frame_num) {
> > > + dpb[i].frame_num <= dec_params->frame_num) {
> >
> > I wonder if a comment should be added here, explaining the reason for '<='.
> >
> > It doesn't seem obvious to me. Up to you, though.
>
> I guess I could, the algo for wrapping in the spec is (formula 8-27):
>
> if( FrameNum > frame_num )
> FrameNumWrap = FrameNum − MaxFrameNum
> else
> FrameNumWrap = FrameNum
>
> Our implementation has the branch condition flip over, and the flipped version of that is:
>
> if( FrameNum <= frame_num )
> FrameNumWrap = FrameNum
> else
> FrameNumWrap = FrameNum − MaxFrameNum
>
> There is no deeper rationale since we simply follow the recipe described in the
> spec. This is done so that we can share that condition with that long term
> reference handling.
Now I come to realize that in patch "[v3,19/24] media: rkvdec-h264: Add field
decoding support" all this code is removed. This is because the wrapping is
already done by the ref-builder, so while enabling field decoding, I now use the
wrapped value from the ref builder. I'm mostly keeping the patch so that this
fix is well documented. I will leave it like this then.
>
> >
> > > p[i] = dpb[i].frame_num;
> > > continue;
> > > }
> >
> > Regards,
> >
> > Hans
>
_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip
^ permalink raw reply [flat|nested] 86+ messages in thread
end of thread, other threads:[~2022-04-25 19:04 UTC | newest]
Thread overview: 86+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-05 20:44 [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 01/24] media: doc: Document dual use of H.264 pic_num/frame_num Nicolas Dufresne
2022-04-22 6:31 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 02/24] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 03/24] media: videobuf2-v4l2: Warn on holding buffers without support Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 04/24] media: h264: Avoid wrapping long_term_frame_idx Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 05/24] media: h264: Use v4l2_h264_reference for reflist Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 5:42 ` Hans Verkuil
2022-04-22 5:42 ` Hans Verkuil
2022-04-22 5:42 ` Hans Verkuil
2022-04-22 5:42 ` Hans Verkuil
2022-04-22 15:23 ` Nicolas Dufresne
2022-04-22 15:23 ` Nicolas Dufresne
2022-04-22 15:23 ` Nicolas Dufresne
2022-04-22 15:23 ` Nicolas Dufresne
2022-04-22 6:59 ` Hans Verkuil
2022-04-22 6:59 ` Hans Verkuil
2022-04-22 6:59 ` Hans Verkuil
2022-04-22 6:59 ` Hans Verkuil
2022-04-22 14:44 ` Nicolas Dufresne
2022-04-22 14:44 ` Nicolas Dufresne
2022-04-22 14:44 ` Nicolas Dufresne
2022-04-22 14:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 06/24] media: h264: Increase reference lists size to 32 Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 07/24] media: h264: Store current picture fields Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 08/24] media: h264: Store all fields into the unordered list Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 09/24] media: v4l2: Trace calculated p/b0/b1 initial reflist Nicolas Dufresne
2022-04-22 7:26 ` Hans Verkuil
2022-04-22 14:58 ` Nicolas Dufresne
2022-04-22 16:33 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 10/24] media: h264: Sort p/b reflist using frame_num Nicolas Dufresne
2022-04-22 7:29 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 11/24] media: v4l2: Reorder field reflist Nicolas Dufresne
2022-04-22 7:31 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 12/24] media: rkvdec: Stop overclocking the decoder Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 7:33 ` Hans Verkuil
2022-04-22 7:33 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 13/24] media: rkvdec: h264: Fix dpb_valid implementation Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 14/24] media: rkvdec: h264: Fix bit depth wrap in pps packet Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264 Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-06 5:17 ` Ezequiel Garcia
2022-04-06 5:17 ` Ezequiel Garcia
2022-04-22 7:36 ` Hans Verkuil
2022-04-22 7:36 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-06 13:10 ` Nicolas Dufresne
2022-04-06 13:10 ` Nicolas Dufresne
2022-04-06 13:12 ` Ezequiel Garcia
2022-04-06 13:12 ` Ezequiel Garcia
2022-04-22 7:39 ` Hans Verkuil
2022-04-22 7:39 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 7:43 ` Hans Verkuil
2022-04-22 7:43 ` Hans Verkuil
2022-04-25 18:55 ` Nicolas Dufresne
2022-04-25 18:55 ` Nicolas Dufresne
2022-04-25 19:00 ` Nicolas Dufresne
2022-04-25 19:00 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 18/24] media: rkvdec: Ensure decoded resolution fit coded resolution Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 19/24] media: rkvdec-h264: Add field decoding support Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 20/24] media: rkvdec: Enable capture buffer holding for H264 Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 21/24] media: hantro: Stop using H.264 parameter pic_num Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 22/24] media: hantro: h264: Make dpb entry management more robust Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-05 20:44 ` [PATCH v3 23/24] media: hantro: Add H.264 field decoding support Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-22 7:49 ` Hans Verkuil
2022-04-22 7:49 ` Hans Verkuil
2022-04-05 20:44 ` [PATCH v3 24/24] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Nicolas Dufresne
2022-04-05 20:44 ` Nicolas Dufresne
2022-04-06 13:06 ` Fwd: [PATCH v3 00/24] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
2022-04-06 17:58 ` Nicolas Dufresne
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.