From: Nicolas Dufresne <nicolas.dufresne@collabora.com>
To: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: kernel@collabora.com,
Sebastian Fricke <sebastian.fricke@collabora.com>,
linux-media@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v2 00/23] H.264 Field Decoding Support for Frame-based Decoders
Date: Thu, 31 Mar 2022 16:02:01 -0400 [thread overview]
Message-ID: <16d0c5c4178b9c91f843e57472e1f0212d7364e5.camel@collabora.com> (raw)
In-Reply-To: <20220331193726.289559-2-nicolas.dufresne@collabora.com>
(re-sending as the cover letter didn't make it for me)
Until now, only Cedrus (a slice base decoder) supported interlaced
decoding. In order to support field decoding in our frame-based decoder,
the v4l2-h264 library needed adaptation to produce the appropriate
reference lists.
This patch extends the v4l2-h264 library to produce the larger references
list needed to represent fields separately. Hantro, MTK-VCODEC and RKVDEC
drivers have been adapted to accommodate the larger lists. Though, only
Hantro and RKVDEC actually have HW support for field decoding. So only
these two have been updated to make use of the larger lists. All this work
has been done using the H.264 specification, LibreELEC downstream kernel
patches, Rockchip MPP reference software and Hantro reference software.
For reviewers, the following is the map of all commit. Patches that could
be merge independently of this serie are marked as independent. Note that
the test results do depend on the generic fixes.
01 : Documentation fix (independent)
02-03 : Improving some generic traces (independent)
04 : Minor v4l2-h264 fix (independent)
05-11 : v4l2-h264 field decoding support
12-15 : rkvdec h.264 generic fixes (independent)
16-19 : rkvdec h.264 field decoding support
20-23 : hantro h.264 field decoding support
All this work have been tested using GStreamer mainline implementation
but also with FFMPEG LibreELEC fork using the testing tool fluster
running through the ITU-T H.264 (2016-02) AVCv2 set of bitstream. Before
this patch, the scores were:
Hantro:
FFMPEG: 88/135
GSteamer: 90/135
RKVDEC:
FFMPEG: 73/135
GSteamer: 77/135
And after these changes:
Hantro:
FFMPEG: 118/135
GSteamer: 129/135
RKVDEC:
FFMPEG: 118/135
GSteamer: 129/135
Note that a bug in FFMPEG / LibreELEC fork was noticed and fixed with the
following change:
diff --git a/libavcodec/v4l2_request_h264.c b/libavcodec/v4l2_request_h264.c
index 88da8f0a2d..394bae0550 100644
--- a/libavcodec/v4l2_request_h264.c
+++ b/libavcodec/v4l2_request_h264.c
@@ -66,7 +66,7 @@ static void fill_dpb_entry(struct v4l2_h264_dpb_entry *entry,
const H264Picture
{
entry->reference_ts = ff_v4l2_request_get_capture_timestamp(pic->f);
entry->pic_num = pic->pic_id;
- entry->frame_num = pic->frame_num;
+ entry->frame_num = pic->long_ref ? pic->pic_id : pic->frame_num;
entry->fields = pic->reference & V4L2_H264_FRAME_REF;
entry->flags = V4L2_H264_DPB_ENTRY_FLAG_VALID;
if (entry->fields)
Some useful links:
Detailed Hantro Results: https://gitlab.freedesktop.org/-/snippets/5189
Detailed RKVDEC Results: https://gitlab.freedesktop.org/-/snippets/5253
ITU-T H.264 (2016-02) AVCv2:
https://www.itu.int/net/itu-t/sigdb/spevideo/VideoForm-s.aspx?val=102002641
Fluster: https://github.com/fluendo/fluster
GStreamer: https://gitlab.freedesktop.org/gstreamer/gstreamer/
FFMPEG Fork:
https://github.com/jernejsk/FFmpeg/tree/v4l2-request-hwaccel-4.4
Rockchip MPP: https://github.com/rockchip-linux/mpp
Changes in v2:
- Applied most of Sebastian's suggestion in comments and commit messages.
- Use a bool for dpb_valid and dpb_bottom in rkvdec
- Dropped one wrong typo fix (media: v4l2-mem2mem: Fix typo in trace message)
- Dropped Alex fix (media: rkvdec-h264: Don't hardcode SPS/PPS parameters
+ I will carry this one later, it seems cosmetic
Jonas Karlman (5):
media: rkvdec: h264: Fix bit depth wrap in pps packet
media: rkvdec: h264: Validate and use pic width and height in mbs
media: rkvdec: h264: Fix reference frame_num wrap for second field
media: rkvdec: Ensure decoded resolution fit coded resolution
media: hantro: h264: Make dpb entry management more robust
Nicolas Dufresne (17):
media: doc: Document dual use of H.264 pic_num/frame_num
media: v4l2-mem2mem: Trace on implicit un-hold
media: h264: Avoid wrapping long_term_frame_idx
media: h264: Use v4l2_h264_reference for reflist
media: h264: Increase reference lists size to 32
media: h264: Store current picture fields
media: h264: Store all fields into the unordered list
media: v4l2: Trace calculated p/b0/b1 initial reflist
media: h264: Sort p/b reflist using frame_num
media: v4l2: Reorder field reflist
media: rkvdec: Stop overclocking the decoder
media: rkvdec: h264: Fix dpb_valid implementation
media: rkvdec: Enable capture buffer holding for H264
media: rkvdec-h264: Add field decoding support
media: hantro: Enable HOLD_CAPTURE_BUF for H.264
media: hantro: Stop using H.264 parameter pic_num
media: hantro: Add H.264 field decoding support
Sebastian Fricke (1):
media: videobuf2-v4l2: Warn on holding buffers without support
.../media/v4l/ext-ctrls-codec-stateless.rst | 10 +-
.../media/common/videobuf2/videobuf2-v4l2.c | 7 +-
.../mediatek/vcodec/vdec/vdec_h264_req_if.c | 17 +-
drivers/media/v4l2-core/v4l2-h264.c | 261 ++++++++++++++----
drivers/media/v4l2-core/v4l2-mem2mem.c | 1 +
.../staging/media/hantro/hantro_g1_h264_dec.c | 38 +--
drivers/staging/media/hantro/hantro_h264.c | 119 ++++++--
drivers/staging/media/hantro/hantro_hw.h | 7 +-
drivers/staging/media/hantro/hantro_v4l2.c | 25 ++
.../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++----
drivers/staging/media/rkvdec/rkvdec-h264.c | 91 +++---
drivers/staging/media/rkvdec/rkvdec.c | 22 +-
drivers/staging/media/rkvdec/rkvdec.h | 1 +
include/media/v4l2-h264.h | 31 ++-
14 files changed, 520 insertions(+), 208 deletions(-)
--
2.34.1
next prev parent reply other threads:[~2022-03-31 20:02 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-31 19:37 [PATCH v2 00/23] H.264 Field Decoding Support for Frame-based Decoders Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 01/23] media: doc: Document dual use of H.264 pic_num/frame_num Nicolas Dufresne
2022-03-31 20:02 ` Nicolas Dufresne [this message]
2022-03-31 19:37 ` [PATCH v2 02/23] media: v4l2-mem2mem: Trace on implicit un-hold Nicolas Dufresne
2022-04-02 10:47 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 03/23] media: videobuf2-v4l2: Warn on holding buffers without support Nicolas Dufresne
2022-04-02 10:52 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 04/23] media: h264: Avoid wrapping long_term_frame_idx Nicolas Dufresne
2022-04-02 11:01 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 05/23] media: h264: Use v4l2_h264_reference for reflist Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 06/23] media: h264: Increase reference lists size to 32 Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-04-02 12:03 ` Ezequiel Garcia
2022-04-02 12:03 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 07/23] media: h264: Store current picture fields Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 08/23] media: h264: Store all fields into the unordered list Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 09/23] media: v4l2: Trace calculated p/b0/b1 initial reflist Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 10/23] media: h264: Sort p/b reflist using frame_num Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 11/23] media: v4l2: Reorder field reflist Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 12/23] media: rkvdec: Stop overclocking the decoder Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-04-02 11:05 ` Ezequiel Garcia
2022-04-02 11:05 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 13/23] media: rkvdec: h264: Fix dpb_valid implementation Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-04-02 11:16 ` Ezequiel Garcia
2022-04-02 11:16 ` Ezequiel Garcia
2022-04-05 15:10 ` Nicolas Dufresne
2022-04-05 15:10 ` Nicolas Dufresne
2022-04-05 15:34 ` Ezequiel Garcia
2022-04-05 15:34 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 14/23] media: rkvdec: h264: Fix bit depth wrap in pps packet Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-04-02 11:20 ` Ezequiel Garcia
2022-04-02 11:20 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 15/23] media: rkvdec: h264: Validate and use pic width and height in mbs Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-04-02 11:32 ` Ezequiel Garcia
2022-04-02 11:32 ` Ezequiel Garcia
2022-04-05 15:44 ` Nicolas Dufresne
2022-04-05 15:44 ` Nicolas Dufresne
2022-04-05 16:04 ` Nicolas Dufresne
2022-04-05 16:04 ` Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 16/23] media: rkvdec: h264: Fix reference frame_num wrap for second field Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-04-02 11:33 ` Ezequiel Garcia
2022-04-02 11:33 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 17/23] media: rkvdec: Enable capture buffer holding for H264 Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-04-02 11:35 ` Ezequiel Garcia
2022-04-02 11:35 ` Ezequiel Garcia
2022-03-31 19:37 ` [PATCH v2 18/23] media: rkvdec: Ensure decoded resolution fit coded resolution Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 19/23] media: rkvdec-h264: Add field decoding support Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 20/23] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 21/23] media: hantro: Stop using H.264 parameter pic_num Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 22/23] media: hantro: h264: Make dpb entry management more robust Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
2022-03-31 19:37 ` [PATCH v2 23/23] media: hantro: Add H.264 field decoding support Nicolas Dufresne
2022-03-31 19:37 ` Nicolas Dufresne
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=16d0c5c4178b9c91f843e57472e1f0212d7364e5.camel@collabora.com \
--to=nicolas.dufresne@collabora.com \
--cc=kernel@collabora.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=sebastian.fricke@collabora.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.