* [PATCH v3 0/2] media: cedrus: Add H264 decoding support
@ 2019-02-11 14:39 Maxime Ripard
2019-02-11 14:39 ` [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls Maxime Ripard
` (2 more replies)
0 siblings, 3 replies; 24+ messages in thread
From: Maxime Ripard @ 2019-02-11 14:39 UTC (permalink / raw)
To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart
Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel,
linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske,
jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni,
Maxime Ripard
Hi,
Here is a new version of the H264 decoding support in the cedrus
driver.
As you might already know, the cedrus driver relies on the Request
API, and is a reverse engineered driver for the video decoding engine
found on the Allwinner SoCs.
This work has been possible thanks to the work done by the people
behind libvdpau-sunxi found here:
https://github.com/linux-sunxi/libvdpau-sunxi/
I've tested the various ABI using this gdb script:
http://code.bulix.org/jl4se4-505620?raw
And this test script:
http://code.bulix.org/8zle4s-505623?raw
The application compiled is quite trivial:
http://code.bulix.org/e34zp8-505624?raw
The output is:
arm: builds/arm-test-v4l2-h264-structures
SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318
x86: builds/x86-test-v4l2-h264-structures
SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318
x64: builds/x64-test-v4l2-h264-structures
SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318
arm64: builds/arm64-test-v4l2-h264-structures
SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318
Let me know if there's any flaw using that test setup, or if you have
any comments on the patches.
Maxime
Changes from v2:
- Simplified _cedrus_write_ref_list as suggested by Jernej
- Set whether the frame is used as reference using nal_ref_idc
- Respect chroma_format_idc
- Fixes for the scaling list and prediction tables
- Wrote the documentation for the flags
- Added a bunch of defines to the driver bit fields
- Reworded the controls and data format descriptions as suggested
by Hans
- Reworked the controls' structure field size to avoid padding
- Removed the long term reference flag
- Reintroduced the neighbor info buffer
- Removed the ref_pic_list_p0/b0/b1 arrays that are redundant with the
one in the DPB
- used the timestamps instead of tags
- Rebased on 5.0-rc1
Changes from v1:
- Rebased on 4.20
- Did the documentation for the userspace API
- Used the tags instead of buffer IDs
- Added a comment to explain why we still needed the swdec trigger
- Reworked the MV col buffer in order to have one slot per frame
- Removed the unused neighbor info buffer
- Made sure to have the same structure offset and alignments across
32 bits and 64 bits architecture
Maxime Ripard (1):
media: cedrus: Add H264 decoding support
Pawel Osciak (1):
media: uapi: Add H264 low-level decoder API compound controls.
Documentation/media/uapi/v4l/biblio.rst | 9 +-
Documentation/media/uapi/v4l/extended-controls.rst | 530 +++++++++++++-
Documentation/media/uapi/v4l/pixfmt-compressed.rst | 20 +-
Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 30 +-
Documentation/media/videodev2.h.rst.exceptions | 5 +-
drivers/media/v4l2-core/v4l2-ctrls.c | 42 +-
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +-
drivers/staging/media/sunxi/cedrus/Makefile | 3 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 31 +-
drivers/staging/media/sunxi/cedrus/cedrus.h | 38 +-
drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 15 +-
drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 589 ++++++++++++++-
drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 4 +-
drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 91 ++-
drivers/staging/media/sunxi/cedrus/cedrus_video.c | 9 +-
include/media/h264-ctrls.h | 179 ++++-
include/media/v4l2-ctrls.h | 13 +-
include/uapi/linux/videodev2.h | 1 +-
18 files changed, 1607 insertions(+), 3 deletions(-)
create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_h264.c
create mode 100644 include/media/h264-ctrls.h
base-commit: e7552cc33320523b660dd1891bceb616ced7b47c
--
git-series 0.9.1
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 14:39 [PATCH v3 0/2] media: cedrus: Add H264 decoding support Maxime Ripard @ 2019-02-11 14:39 ` Maxime Ripard 2019-02-11 15:16 ` Hans Verkuil ` (2 more replies) 2019-02-11 14:39 ` [PATCH v3 2/2] media: cedrus: Add H264 decoding support Maxime Ripard 2019-02-12 12:50 ` [PATCH v3 0/2] " Tomasz Figa 2 siblings, 3 replies; 24+ messages in thread From: Maxime Ripard @ 2019-02-11 14:39 UTC (permalink / raw) To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni, Maxime Ripard, Guenter Roeck From: Pawel Osciak <posciak@chromium.org> Stateless video codecs will require both the H264 metadata and slices in order to be able to decode frames. This introduces the definitions for a new pixel format for H264 slices that have been parsed, as well as the structures used to pass the metadata from the userspace to the kernel. Co-Developed-by: Maxime Ripard <maxime.ripard@bootlin.com> Signed-off-by: Pawel Osciak <posciak@chromium.org> Signed-off-by: Guenter Roeck <groeck@chromium.org> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> --- Documentation/media/uapi/v4l/biblio.rst | 9 +- Documentation/media/uapi/v4l/extended-controls.rst | 530 ++++++++++++++- Documentation/media/uapi/v4l/pixfmt-compressed.rst | 20 +- Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 30 +- Documentation/media/videodev2.h.rst.exceptions | 5 +- drivers/media/v4l2-core/v4l2-ctrls.c | 42 +- drivers/media/v4l2-core/v4l2-ioctl.c | 1 +- include/media/h264-ctrls.h | 179 +++++- include/media/v4l2-ctrls.h | 13 +- include/uapi/linux/videodev2.h | 1 +- 10 files changed, 829 insertions(+), 1 deletion(-) create mode 100644 include/media/h264-ctrls.h diff --git a/Documentation/media/uapi/v4l/biblio.rst b/Documentation/media/uapi/v4l/biblio.rst index ec33768c055e..3fc3f7ff338a 100644 --- a/Documentation/media/uapi/v4l/biblio.rst +++ b/Documentation/media/uapi/v4l/biblio.rst @@ -122,6 +122,15 @@ ITU BT.1119 :author: International Telecommunication Union (http://www.itu.ch) +.. _h264: + +ITU H.264 +========= + +:title: ITU-T Recommendation H.264 "Advanced Video Coding for Generic Audiovisual Services" + +:author: International Telecommunication Union (http://www.itu.ch) + .. _jfif: JFIF diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst index af4273aa5e85..7e306aa0d0a6 100644 --- a/Documentation/media/uapi/v4l/extended-controls.rst +++ b/Documentation/media/uapi/v4l/extended-controls.rst @@ -1703,6 +1703,536 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - non-intra-coded frames, in zigzag scanning order. Only relevant for non-4:2:0 YUV formats. +.. _v4l2-mpeg-h264: + +``V4L2_CID_MPEG_VIDEO_H264_SPS (struct)`` + Specifies the sequence parameter set (as extracted from the + bitstream) for the associated H264 slice data. This includes the + necessary parameters for configuring a stateless hardware decoding + pipeline for H264. The bitstream parameters are defined according + to :ref:`h264`. Unless there's a specific comment, refer to the + specification for the documentation of these fields, section 7.4.2.1.1 + "Sequence Parameter Set Data Semantics". + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_h264_sps + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_h264_sps + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``profile_idc`` + - + * - __u8 + - ``constraint_set_flags`` + - See :ref:`Sequence Parameter Set Constraints Set Flags <h264_sps_constraints_set_flags>` + * - __u8 + - ``level_idc`` + - + * - __u8 + - ``seq_parameter_set_id`` + - + * - __u8 + - ``chroma_format_idc`` + - + * - __u8 + - ``bit_depth_luma_minus8`` + - + * - __u8 + - ``bit_depth_chroma_minus8`` + - + * - __u8 + - ``log2_max_frame_num_minus4`` + - + * - __u8 + - ``pic_order_cnt_type`` + - + * - __u8 + - ``log2_max_pic_order_cnt_lsb_minus4`` + - + * - __u8 + - ``max_num_ref_frames`` + - + * - __u8 + - ``num_ref_frames_in_pic_order_cnt_cycle`` + - + * - __s32 + - ``offset_for_ref_frame[255]`` + - + * - __s32 + - ``offset_for_non_ref_pic`` + - + * - __s32 + - ``offset_for_top_to_bottom_field`` + - + * - __u16 + - ``pic_width_in_mbs_minus1`` + - + * - __u16 + - ``pic_height_in_map_units_minus1`` + - + * - __u32 + - ``flags`` + - See :ref:`Sequence Parameter Set Flags <h264_sps_flags>` + +.. _h264_sps_constraints_set_flags: + +``Sequence Parameter Set Constraints Set Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_H264_SPS_CONSTRAINT_SET0_FLAG`` + - 0x00000001 + - + * - ``V4L2_H264_SPS_CONSTRAINT_SET1_FLAG`` + - 0x00000002 + - + * - ``V4L2_H264_SPS_CONSTRAINT_SET2_FLAG`` + - 0x00000004 + - + * - ``V4L2_H264_SPS_CONSTRAINT_SET3_FLAG`` + - 0x00000008 + - + * - ``V4L2_H264_SPS_CONSTRAINT_SET4_FLAG`` + - 0x00000010 + - + * - ``V4L2_H264_SPS_CONSTRAINT_SET5_FLAG`` + - 0x00000020 + - + +.. _h264_sps_flags: + +``Sequence Parameter Set Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE`` + - 0x00000001 + - + * - ``V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS`` + - 0x00000002 + - + * - ``V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO`` + - 0x00000004 + - + * - ``V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED`` + - 0x00000008 + - + * - ``V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY`` + - 0x00000010 + - + * - ``V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD`` + - 0x00000020 + - + * - ``V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE`` + - 0x00000040 + - + +``V4L2_CID_MPEG_VIDEO_H264_PPS (struct)`` + Specifies the picture parameter set (as extracted from the + bitstream) for the associated H264 slice data. This includes the + necessary parameters for configuring a stateless hardware decoding + pipeline for H264. The bitstream parameters are defined according + to :ref:`h264`. Unless there's a specific comment, refer to the + specification for the documentation of these fields, section 7.4.2.2 + "Picture Parameter Set RBSP Semantics". + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_h264_pps + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_h264_pps + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``pic_parameter_set_id`` + - + * - __u8 + - ``seq_parameter_set_id`` + - + * - __u8 + - ``num_slice_groups_minus1`` + - + * - __u8 + - ``num_ref_idx_l0_default_active_minus1`` + - + * - __u8 + - ``num_ref_idx_l1_default_active_minus1`` + - + * - __u8 + - ``weighted_bipred_idc`` + - + * - __s8 + - ``pic_init_qp_minus26`` + - + * - __s8 + - ``pic_init_qs_minus26`` + - + * - __s8 + - ``chroma_qp_index_offset`` + - + * - __s8 + - ``second_chroma_qp_index_offset`` + - + * - __u16 + - ``flags`` + - See :ref:`Picture Parameter Set Flags <h264_pps_flags>` + +.. _h264_pps_flags: + +``Picture Parameter Set Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE`` + - 0x00000001 + - + * - ``V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT`` + - 0x00000002 + - + * - ``V4L2_H264_PPS_FLAG_WEIGHTED_PRED`` + - 0x00000004 + - + * - ``V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT`` + - 0x00000008 + - + * - ``V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED`` + - 0x00000010 + - + * - ``V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT`` + - 0x00000020 + - + * - ``V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE`` + - 0x00000040 + - + * - ``V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT`` + - 0x00000080 + - + +``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (struct)`` + Specifies the scaling matrix (as extracted from the bitstream) for + the associated H264 slice data. The bitstream parameters are + defined according to :ref:`h264`. Unless there's a specific + comment, refer to the specification for the documentation of these + fields, section 7.4.2.1.1.1 "Scaling List Semantics". + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_h264_scaling_matrix + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_h264_scaling_matrix + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``scaling_list_4x4[6][16]`` + - + * - __u8 + - ``scaling_list_8x8[6][64]`` + - + +``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (struct)`` + Specifies the slice parameters (as extracted from the bitstream) + for the associated H264 slice data. This includes the necessary + parameters for configuring a stateless hardware decoding pipeline + for H264. The bitstream parameters are defined according to + :ref:`h264`. Unless there's a specific comment, refer to the + specification for the documentation of these fields, section 7.4.3 + "Slice Header Semantics". + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_h264_slice_param + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_h264_slice_param + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``size`` + - + * - __u32 + - ``header_bit_size`` + - + * - __u16 + - ``first_mb_in_slice`` + - + * - __u8 + - ``slice_type`` + - + * - __u8 + - ``pic_parameter_set_id`` + - + * - __u8 + - ``colour_plane_id`` + - + * - __u16 + - ``frame_num`` + - + * - __u16 + - ``idr_pic_id`` + - + * - __u16 + - ``pic_order_cnt_lsb`` + - + * - __s32 + - ``delta_pic_order_cnt_bottom`` + - + * - __s32 + - ``delta_pic_order_cnt0`` + - + * - __s32 + - ``delta_pic_order_cnt1`` + - + * - __u8 + - ``redundant_pic_cnt`` + - + * - struct :c:type:`v4l2_h264_pred_weight_table` + - ``pred_weight_table`` + - + * - __u32 + - ``dec_ref_pic_marking_bit_size`` + - + * - __u32 + - ``pic_order_cnt_bit_size`` + - + * - __u8 + - ``cabac_init_idc`` + - + * - __s8 + - ``slice_qp_delta`` + - + * - __s8 + - ``slice_qs_delta`` + - + * - __u8 + - ``disable_deblocking_filter_idc`` + - + * - __s8 + - ``slice_alpha_c0_offset_div2`` + - + * - __s8 + - ``slice_beta_offset_div2`` + - + * - __u32 + - ``slice_group_change_cycle`` + - + * - __u8 + - ``num_ref_idx_l0_active_minus1`` + - + * - __u8 + - ``num_ref_idx_l1_active_minus1`` + - + * - __u8 + - ``ref_pic_list0[32]`` + - + * - __u8 + - ``ref_pic_list1[32]`` + - + * - __u32 + - ``flags`` + - See :ref:`Slice Parameter Flags <h264_slice_flags>` + +.. _h264_slice_flags: + +``Slice Parameter Set Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_H264_SLICE_FLAG_FIELD_PIC`` + - 0x00000001 + - + * - ``V4L2_H264_SLICE_FLAG_BOTTOM_FIELD`` + - 0x00000002 + - + * - ``V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED`` + - 0x00000004 + - + * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH`` + - 0x00000008 + - + +``Prediction Weight Table`` + + Unless there's a specific comment, refer to the specification for + the documentation of these fields, section 7.4.3.2 "Prediction + Weight Table Semantics". + +.. c:type:: v4l2_h264_pred_weight_table + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_h264_pred_weight_table + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u16 + - ``luma_log2_weight_denom`` + - + * - __u16 + - ``chroma_log2_weight_denom`` + - + * - struct :c:type:`v4l2_h264_weight_factors` + - ``weight_factors[2]`` + - + +.. c:type:: v4l2_h264_weight_factors + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_h264_weight_factors + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __s8 + - ``luma_weight[32]`` + - + * - __s8 + - ``luma_offset[32]`` + - + * - __s8 + - ``chroma_weight[32][2]`` + - + * - __s8 + - ``chroma_offset[32][2]`` + - + +``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)`` + Specifies the decode parameters (as extracted from the bitstream) + for the associated H264 slice data. This includes the necessary + parameters for configuring a stateless hardware decoding pipeline + for H264. The bitstream parameters are defined according to + :ref:`h264`. Unless there's a specific comment, refer to the + specification for the documentation of these fields. + + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_ctrl_h264_decode_param + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_ctrl_h264_decode_param + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``num_slices`` + - Number of slices needed to decode the current frame + * - __u16 + - ``idr_pic_flag`` + - Is the picture an IDR picture? + * - __u16 + - ``nal_ref_idc`` + - NAL reference ID value coming from the NAL Unit header + * - __s32 + - ``top_field_order_cnt`` + - Picture Order Count for the coded top field + * - __s32 + - ``bottom_field_order_cnt`` + - Picture Order Count for the coded bottom field + * - struct :c:type:`v4l2_h264_dpb_entry` + - ``dpb[16]`` + - + +.. c:type:: v4l2_h264_dpb_entry + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_h264_dpb_entry + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``tag`` + - tag to identify the buffer containing the reference frame + * - __u16 + - ``frame_num`` + - + * - __u16 + - ``pic_num`` + - + * - __s32 + - ``top_field_order_cnt`` + - + * - __s32 + - ``bottom_field_order_cnt`` + - + * - __u32 + - ``flags`` + - See :ref:`DPB Entry Flags <h264_dpb_flags>` + +.. _h264_dpb_flags: + +``DPB Entries Flags`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_H264_DPB_ENTRY_FLAG_VALID`` + - 0x00000001 + - + * - ``V4L2_H264_DPB_ENTRY_FLAG_ACTIVE`` + - 0x00000002 + - + MFC 5.1 MPEG Controls --------------------- diff --git a/Documentation/media/uapi/v4l/pixfmt-compressed.rst b/Documentation/media/uapi/v4l/pixfmt-compressed.rst index e4c5e456df59..95905b967693 100644 --- a/Documentation/media/uapi/v4l/pixfmt-compressed.rst +++ b/Documentation/media/uapi/v4l/pixfmt-compressed.rst @@ -52,6 +52,26 @@ Compressed Formats - ``V4L2_PIX_FMT_H264_MVC`` - 'M264' - H264 MVC video elementary stream. + * .. _V4L2-PIX-FMT-H264-SLICE: + + - ``V4L2_PIX_FMT_H264_SLICE`` + - 'S264' + - H264 parsed slice data, as extracted from the H264 bitstream. + This format is adapted for stateless video decoders that + implement an H264 pipeline (using the :ref:`codec` and + :ref:`media-request-api`). Metadata associated with the frame + to decode are required to be passed through the + ``V4L2_CID_MPEG_VIDEO_H264_SPS``, + ``V4L2_CID_MPEG_VIDEO_H264_PPS``, + ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` and + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS`` controls and + scaling matrices can optionally be specified through the + ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX`` control. See the + :ref:`associated Codec Control IDs <v4l2-mpeg-h264>`. + Exactly one output and one capture buffer must be provided for + use with this pixel format. The output buffer must contain the + appropriate number of macroblocks to decode a full + corresponding frame to the matching capture buffer. * .. _V4L2-PIX-FMT-H263: - ``V4L2_PIX_FMT_H263`` diff --git a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst index f824162d0ea9..bf29dc5b9758 100644 --- a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst +++ b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst @@ -443,6 +443,36 @@ See also the examples in :ref:`control`. - n/a - A struct :c:type:`v4l2_ctrl_mpeg2_quantization`, containing MPEG-2 quantization matrices for stateless video decoders. + * - ``V4L2_CTRL_TYPE_H264_SPS`` + - n/a + - n/a + - n/a + - A struct :c:type:`v4l2_ctrl_h264_sps`, containing H264 + sequence parameters for stateless video decoders. + * - ``V4L2_CTRL_TYPE_H264_PPS`` + - n/a + - n/a + - n/a + - A struct :c:type:`v4l2_ctrl_h264_pps`, containing H264 + picture parameters for stateless video decoders. + * - ``V4L2_CTRL_TYPE_H264_SCALING_MATRIX`` + - n/a + - n/a + - n/a + - A struct :c:type:`v4l2_ctrl_h264_scaling_matrix`, containing H264 + scaling matrices for stateless video decoders. + * - ``V4L2_CTRL_TYPE_H264_SLICE_PARAMS`` + - n/a + - n/a + - n/a + - A struct :c:type:`v4l2_ctrl_h264_slice_param`, containing H264 + slice parameters for stateless video decoders. + * - ``V4L2_CTRL_TYPE_H264_DECODE_PARAMS`` + - n/a + - n/a + - n/a + - A struct :c:type:`v4l2_ctrl_h264_decode_param`, containing H264 + decode parameters for stateless video decoders. .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}| diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions index 64d348e67df9..55cbe324b9fc 100644 --- a/Documentation/media/videodev2.h.rst.exceptions +++ b/Documentation/media/videodev2.h.rst.exceptions @@ -136,6 +136,11 @@ replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type` replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type` replace symbol V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS :c:type:`v4l2_ctrl_type` replace symbol V4L2_CTRL_TYPE_MPEG2_QUANTIZATION :c:type:`v4l2_ctrl_type` +replace symbol V4L2_CTRL_TYPE_H264_SPS :c:type:`v4l2_ctrl_type` +replace symbol V4L2_CTRL_TYPE_H264_PPS :c:type:`v4l2_ctrl_type` +replace symbol V4L2_CTRL_TYPE_H264_SCALING_MATRIX :c:type:`v4l2_ctrl_type` +replace symbol V4L2_CTRL_TYPE_H264_SLICE_PARAMS :c:type:`v4l2_ctrl_type` +replace symbol V4L2_CTRL_TYPE_H264_DECODE_PARAMS :c:type:`v4l2_ctrl_type` # V4L2 capability defines replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index e3bd441fa29a..fe9f9447caba 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -825,6 +825,11 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers"; case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP: return "H264 Set QP Value for HC Layers"; + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 Sequence Parameter Set"; + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 Picture Parameter Set"; + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix"; + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters"; + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters"; case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value"; case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value"; case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value"; @@ -1300,6 +1305,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION: *type = V4L2_CTRL_TYPE_MPEG2_QUANTIZATION; break; + case V4L2_CID_MPEG_VIDEO_H264_SPS: + *type = V4L2_CTRL_TYPE_H264_SPS; + break; + case V4L2_CID_MPEG_VIDEO_H264_PPS: + *type = V4L2_CTRL_TYPE_H264_PPS; + break; + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX; + break; + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAMS; + break; + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAMS; + break; default: *type = V4L2_CTRL_TYPE_INTEGER; break; @@ -1666,6 +1686,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx, case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: return 0; + case V4L2_CTRL_TYPE_H264_SPS: + case V4L2_CTRL_TYPE_H264_PPS: + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: + return 0; + default: return -EINVAL; } @@ -2246,6 +2273,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization); break; + case V4L2_CTRL_TYPE_H264_SPS: + elem_size = sizeof(struct v4l2_ctrl_h264_sps); + break; + case V4L2_CTRL_TYPE_H264_PPS: + elem_size = sizeof(struct v4l2_ctrl_h264_pps); + break; + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix); + break; + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param); + break; + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param); + break; default: if (type < V4L2_CTRL_COMPOUND_TYPES) elem_size = sizeof(s32); diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 1441a73ce64c..f42d6e7049eb 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1313,6 +1313,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_H264: descr = "H.264"; break; case V4L2_PIX_FMT_H264_NO_SC: descr = "H.264 (No Start Codes)"; break; case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break; + case V4L2_PIX_FMT_H264_SLICE: descr = "H.264 Parsed Slice Data"; break; case V4L2_PIX_FMT_H263: descr = "H.263"; break; case V4L2_PIX_FMT_MPEG1: descr = "MPEG-1 ES"; break; case V4L2_PIX_FMT_MPEG2: descr = "MPEG-2 ES"; break; diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h new file mode 100644 index 000000000000..7b0563dfa05f --- /dev/null +++ b/include/media/h264-ctrls.h @@ -0,0 +1,179 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * These are the H.264 state controls for use with stateless H.264 + * codec drivers. + * + * It turns out that these structs are not stable yet and will undergo + * more changes. So keep them private until they are stable and ready to + * become part of the official public API. + */ + +#ifndef _H264_CTRLS_H_ +#define _H264_CTRLS_H_ + +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383) +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384) +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385) +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+386) +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+387) + +/* enum v4l2_ctrl_type type values */ +#define V4L2_CTRL_TYPE_H264_SPS 0x0105 +#define V4L2_CTRL_TYPE_H264_PPS 0x0106 +#define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0107 +#define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0108 +#define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0109 + +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01 +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02 +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04 +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08 +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10 +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20 + +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01 +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02 +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04 +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08 +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10 +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20 +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40 + +struct v4l2_ctrl_h264_sps { + __u8 profile_idc; + __u8 constraint_set_flags; + __u8 level_idc; + __u8 seq_parameter_set_id; + __u8 chroma_format_idc; + __u8 bit_depth_luma_minus8; + __u8 bit_depth_chroma_minus8; + __u8 log2_max_frame_num_minus4; + __u8 pic_order_cnt_type; + __u8 log2_max_pic_order_cnt_lsb_minus4; + __u8 max_num_ref_frames; + __u8 num_ref_frames_in_pic_order_cnt_cycle; + __s32 offset_for_ref_frame[255]; + __s32 offset_for_non_ref_pic; + __s32 offset_for_top_to_bottom_field; + __u16 pic_width_in_mbs_minus1; + __u16 pic_height_in_map_units_minus1; + __u32 flags; +}; + +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001 +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002 +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004 +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008 +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010 +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020 +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040 +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080 + +struct v4l2_ctrl_h264_pps { + __u8 pic_parameter_set_id; + __u8 seq_parameter_set_id; + __u8 num_slice_groups_minus1; + __u8 num_ref_idx_l0_default_active_minus1; + __u8 num_ref_idx_l1_default_active_minus1; + __u8 weighted_bipred_idc; + __s8 pic_init_qp_minus26; + __s8 pic_init_qs_minus26; + __s8 chroma_qp_index_offset; + __s8 second_chroma_qp_index_offset; + __u16 flags; +}; + +struct v4l2_ctrl_h264_scaling_matrix { + __u8 scaling_list_4x4[6][16]; + __u8 scaling_list_8x8[6][64]; +}; + +struct v4l2_h264_weight_factors { + __s8 luma_weight[32]; + __s8 luma_offset[32]; + __s8 chroma_weight[32][2]; + __s8 chroma_offset[32][2]; +}; + +struct v4l2_h264_pred_weight_table { + __u16 luma_log2_weight_denom; + __u16 chroma_log2_weight_denom; + struct v4l2_h264_weight_factors weight_factors[2]; +}; + +#define V4L2_H264_SLICE_TYPE_P 0 +#define V4L2_H264_SLICE_TYPE_B 1 +#define V4L2_H264_SLICE_TYPE_I 2 +#define V4L2_H264_SLICE_TYPE_SP 3 +#define V4L2_H264_SLICE_TYPE_SI 4 + +#define V4L2_H264_SLICE_FLAG_FIELD_PIC 0x01 +#define V4L2_H264_SLICE_FLAG_BOTTOM_FIELD 0x02 +#define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04 +#define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08 + +struct v4l2_ctrl_h264_slice_param { + /* Size in bytes, including header */ + __u32 size; + /* Offset in bits to slice_data() from the beginning of this slice. */ + __u32 header_bit_size; + + __u16 first_mb_in_slice; + __u8 slice_type; + __u8 pic_parameter_set_id; + __u8 colour_plane_id; + __u8 redundant_pic_cnt; + __u16 frame_num; + __u16 idr_pic_id; + __u16 pic_order_cnt_lsb; + __s32 delta_pic_order_cnt_bottom; + __s32 delta_pic_order_cnt0; + __s32 delta_pic_order_cnt1; + + struct v4l2_h264_pred_weight_table pred_weight_table; + /* Size in bits of dec_ref_pic_marking() syntax element. */ + __u32 dec_ref_pic_marking_bit_size; + /* Size in bits of pic order count syntax. */ + __u32 pic_order_cnt_bit_size; + + __u8 cabac_init_idc; + __s8 slice_qp_delta; + __s8 slice_qs_delta; + __u8 disable_deblocking_filter_idc; + __s8 slice_alpha_c0_offset_div2; + __s8 slice_beta_offset_div2; + __u8 num_ref_idx_l0_active_minus1; + __u8 num_ref_idx_l1_active_minus1; + __u32 slice_group_change_cycle; + + /* Entries on each list are indices + * into v4l2_ctrl_h264_decode_param.dpb[]. */ + __u8 ref_pic_list0[32]; + __u8 ref_pic_list1[32]; + + __u32 flags; +}; + +#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01 +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x02 + +struct v4l2_h264_dpb_entry { + __u64 timestamp; + __u16 frame_num; + __u16 pic_num; + /* Note that field is indicated by v4l2_buffer.field */ + __s32 top_field_order_cnt; + __s32 bottom_field_order_cnt; + __u32 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */ +}; + +struct v4l2_ctrl_h264_decode_param { + __u32 num_slices; + __u16 idr_pic_flag; + __u16 nal_ref_idc; + __s32 top_field_order_cnt; + __s32 bottom_field_order_cnt; + struct v4l2_h264_dpb_entry dpb[16]; +}; + +#endif diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h index d63cf227b0ab..22b6d09c4764 100644 --- a/include/media/v4l2-ctrls.h +++ b/include/media/v4l2-ctrls.h @@ -23,10 +23,11 @@ #include <media/media-request.h> /* - * Include the mpeg2 stateless codec compound control definitions. + * Include the stateless codec compound control definitions. * This will move to the public headers once this API is fully stable. */ #include <media/mpeg2-ctrls.h> +#include <media/h264-ctrls.h> /* forward references */ struct file; @@ -49,6 +50,11 @@ struct poll_table_struct; * @p_char: Pointer to a string. * @p_mpeg2_slice_params: Pointer to a MPEG2 slice parameters structure. * @p_mpeg2_quantization: Pointer to a MPEG2 quantization data structure. + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps. + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps. + * @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix. + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param. + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param. * @p: Pointer to a compound value. */ union v4l2_ctrl_ptr { @@ -60,6 +66,11 @@ union v4l2_ctrl_ptr { char *p_char; struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params; struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quantization; + struct v4l2_ctrl_h264_sps *p_h264_sps; + struct v4l2_ctrl_h264_pps *p_h264_pps; + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix; + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param; + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param; void *p; }; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index d6eed479c3a6..6fc955926bdb 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -645,6 +645,7 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */ #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */ #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */ +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */ #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */ #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */ -- git-series 0.9.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 14:39 ` [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls Maxime Ripard @ 2019-02-11 15:16 ` Hans Verkuil 2019-02-11 15:21 ` Hans Verkuil 2019-02-11 17:16 ` Nicolas Dufresne 2019-02-11 17:12 ` Ezequiel Garcia 2019-02-11 19:53 ` Ezequiel Garcia 2 siblings, 2 replies; 24+ messages in thread From: Hans Verkuil @ 2019-02-11 15:16 UTC (permalink / raw) To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni, Guenter Roeck Hi Maxime, A quick review below. Note that I am no expert on the codec details, so I leave that to others. I'm mainly concentrating on the structs, flags, etc. On 2/11/19 3:39 PM, Maxime Ripard wrote: > From: Pawel Osciak <posciak@chromium.org> > > Stateless video codecs will require both the H264 metadata and slices in > order to be able to decode frames. > > This introduces the definitions for a new pixel format for H264 slices that > have been parsed, as well as the structures used to pass the metadata from > the userspace to the kernel. > > Co-Developed-by: Maxime Ripard <maxime.ripard@bootlin.com> > Signed-off-by: Pawel Osciak <posciak@chromium.org> > Signed-off-by: Guenter Roeck <groeck@chromium.org> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> > --- > Documentation/media/uapi/v4l/biblio.rst | 9 +- > Documentation/media/uapi/v4l/extended-controls.rst | 530 ++++++++++++++- > Documentation/media/uapi/v4l/pixfmt-compressed.rst | 20 +- > Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 30 +- > Documentation/media/videodev2.h.rst.exceptions | 5 +- > drivers/media/v4l2-core/v4l2-ctrls.c | 42 +- > drivers/media/v4l2-core/v4l2-ioctl.c | 1 +- > include/media/h264-ctrls.h | 179 +++++- > include/media/v4l2-ctrls.h | 13 +- > include/uapi/linux/videodev2.h | 1 +- > 10 files changed, 829 insertions(+), 1 deletion(-) > create mode 100644 include/media/h264-ctrls.h > > diff --git a/Documentation/media/uapi/v4l/biblio.rst b/Documentation/media/uapi/v4l/biblio.rst > index ec33768c055e..3fc3f7ff338a 100644 > --- a/Documentation/media/uapi/v4l/biblio.rst > +++ b/Documentation/media/uapi/v4l/biblio.rst > @@ -122,6 +122,15 @@ ITU BT.1119 > > :author: International Telecommunication Union (http://www.itu.ch) > > +.. _h264: > + > +ITU H.264 > +========= > + > +:title: ITU-T Recommendation H.264 "Advanced Video Coding for Generic Audiovisual Services" > + > +:author: International Telecommunication Union (http://www.itu.ch) > + > .. _jfif: > > JFIF > diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst > index af4273aa5e85..7e306aa0d0a6 100644 > --- a/Documentation/media/uapi/v4l/extended-controls.rst > +++ b/Documentation/media/uapi/v4l/extended-controls.rst > @@ -1703,6 +1703,536 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - > non-intra-coded frames, in zigzag scanning order. Only relevant for > non-4:2:0 YUV formats. > > +.. _v4l2-mpeg-h264: > + > +``V4L2_CID_MPEG_VIDEO_H264_SPS (struct)`` > + Specifies the sequence parameter set (as extracted from the > + bitstream) for the associated H264 slice data. This includes the > + necessary parameters for configuring a stateless hardware decoding > + pipeline for H264. The bitstream parameters are defined according > + to :ref:`h264`. Unless there's a specific comment, refer to the > + specification for the documentation of these fields, section 7.4.2.1.1 > + "Sequence Parameter Set Data Semantics". > + > + .. note:: > + > + This compound control is not yet part of the public kernel API and > + it is expected to change. > + > +.. c:type:: v4l2_ctrl_h264_sps > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_h264_sps > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __u8 > + - ``profile_idc`` > + - > + * - __u8 > + - ``constraint_set_flags`` > + - See :ref:`Sequence Parameter Set Constraints Set Flags <h264_sps_constraints_set_flags>` > + * - __u8 > + - ``level_idc`` > + - > + * - __u8 > + - ``seq_parameter_set_id`` > + - > + * - __u8 > + - ``chroma_format_idc`` > + - > + * - __u8 > + - ``bit_depth_luma_minus8`` > + - > + * - __u8 > + - ``bit_depth_chroma_minus8`` > + - > + * - __u8 > + - ``log2_max_frame_num_minus4`` > + - > + * - __u8 > + - ``pic_order_cnt_type`` > + - > + * - __u8 > + - ``log2_max_pic_order_cnt_lsb_minus4`` > + - > + * - __u8 > + - ``max_num_ref_frames`` > + - > + * - __u8 > + - ``num_ref_frames_in_pic_order_cnt_cycle`` > + - > + * - __s32 > + - ``offset_for_ref_frame[255]`` > + - > + * - __s32 > + - ``offset_for_non_ref_pic`` > + - > + * - __s32 > + - ``offset_for_top_to_bottom_field`` > + - > + * - __u16 > + - ``pic_width_in_mbs_minus1`` > + - > + * - __u16 > + - ``pic_height_in_map_units_minus1`` > + - > + * - __u32 > + - ``flags`` > + - See :ref:`Sequence Parameter Set Flags <h264_sps_flags>` > + > +.. _h264_sps_constraints_set_flags: > + > +``Sequence Parameter Set Constraints Set Flags`` > + > +.. cssclass:: longtable > + > +.. flat-table:: > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - ``V4L2_H264_SPS_CONSTRAINT_SET0_FLAG`` > + - 0x00000001 > + - > + * - ``V4L2_H264_SPS_CONSTRAINT_SET1_FLAG`` > + - 0x00000002 > + - > + * - ``V4L2_H264_SPS_CONSTRAINT_SET2_FLAG`` > + - 0x00000004 > + - > + * - ``V4L2_H264_SPS_CONSTRAINT_SET3_FLAG`` > + - 0x00000008 > + - > + * - ``V4L2_H264_SPS_CONSTRAINT_SET4_FLAG`` > + - 0x00000010 > + - > + * - ``V4L2_H264_SPS_CONSTRAINT_SET5_FLAG`` > + - 0x00000020 > + - > + > +.. _h264_sps_flags: > + > +``Sequence Parameter Set Flags`` > + > +.. cssclass:: longtable > + > +.. flat-table:: > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - ``V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE`` > + - 0x00000001 > + - > + * - ``V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS`` > + - 0x00000002 > + - > + * - ``V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO`` > + - 0x00000004 > + - > + * - ``V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED`` > + - 0x00000008 > + - > + * - ``V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY`` > + - 0x00000010 > + - > + * - ``V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD`` > + - 0x00000020 > + - > + * - ``V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE`` > + - 0x00000040 > + - > + > +``V4L2_CID_MPEG_VIDEO_H264_PPS (struct)`` > + Specifies the picture parameter set (as extracted from the > + bitstream) for the associated H264 slice data. This includes the > + necessary parameters for configuring a stateless hardware decoding > + pipeline for H264. The bitstream parameters are defined according > + to :ref:`h264`. Unless there's a specific comment, refer to the > + specification for the documentation of these fields, section 7.4.2.2 > + "Picture Parameter Set RBSP Semantics". > + > + .. note:: > + > + This compound control is not yet part of the public kernel API and > + it is expected to change. > + > +.. c:type:: v4l2_ctrl_h264_pps > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_h264_pps > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __u8 > + - ``pic_parameter_set_id`` > + - > + * - __u8 > + - ``seq_parameter_set_id`` > + - > + * - __u8 > + - ``num_slice_groups_minus1`` > + - > + * - __u8 > + - ``num_ref_idx_l0_default_active_minus1`` > + - > + * - __u8 > + - ``num_ref_idx_l1_default_active_minus1`` > + - > + * - __u8 > + - ``weighted_bipred_idc`` > + - > + * - __s8 > + - ``pic_init_qp_minus26`` > + - > + * - __s8 > + - ``pic_init_qs_minus26`` > + - > + * - __s8 > + - ``chroma_qp_index_offset`` > + - > + * - __s8 > + - ``second_chroma_qp_index_offset`` > + - > + * - __u16 > + - ``flags`` > + - See :ref:`Picture Parameter Set Flags <h264_pps_flags>` > + > +.. _h264_pps_flags: > + > +``Picture Parameter Set Flags`` > + > +.. cssclass:: longtable > + > +.. flat-table:: > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - ``V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE`` > + - 0x00000001 > + - > + * - ``V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT`` > + - 0x00000002 > + - > + * - ``V4L2_H264_PPS_FLAG_WEIGHTED_PRED`` > + - 0x00000004 > + - > + * - ``V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT`` > + - 0x00000008 > + - > + * - ``V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED`` > + - 0x00000010 > + - > + * - ``V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT`` > + - 0x00000020 > + - > + * - ``V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE`` > + - 0x00000040 > + - > + * - ``V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT`` > + - 0x00000080 > + - > + > +``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (struct)`` > + Specifies the scaling matrix (as extracted from the bitstream) for > + the associated H264 slice data. The bitstream parameters are > + defined according to :ref:`h264`. Unless there's a specific > + comment, refer to the specification for the documentation of these > + fields, section 7.4.2.1.1.1 "Scaling List Semantics". > + > + .. note:: > + > + This compound control is not yet part of the public kernel API and > + it is expected to change. > + > +.. c:type:: v4l2_ctrl_h264_scaling_matrix > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_h264_scaling_matrix > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __u8 > + - ``scaling_list_4x4[6][16]`` > + - > + * - __u8 > + - ``scaling_list_8x8[6][64]`` > + - > + > +``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (struct)`` > + Specifies the slice parameters (as extracted from the bitstream) > + for the associated H264 slice data. This includes the necessary > + parameters for configuring a stateless hardware decoding pipeline > + for H264. The bitstream parameters are defined according to > + :ref:`h264`. Unless there's a specific comment, refer to the > + specification for the documentation of these fields, section 7.4.3 > + "Slice Header Semantics". > + > + .. note:: > + > + This compound control is not yet part of the public kernel API and > + it is expected to change. > + > +.. c:type:: v4l2_ctrl_h264_slice_param > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_h264_slice_param > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __u32 > + - ``size`` > + - > + * - __u32 > + - ``header_bit_size`` > + - > + * - __u16 > + - ``first_mb_in_slice`` > + - > + * - __u8 > + - ``slice_type`` > + - > + * - __u8 > + - ``pic_parameter_set_id`` > + - > + * - __u8 > + - ``colour_plane_id`` > + - > + * - __u16 > + - ``frame_num`` > + - > + * - __u16 > + - ``idr_pic_id`` > + - > + * - __u16 > + - ``pic_order_cnt_lsb`` > + - > + * - __s32 > + - ``delta_pic_order_cnt_bottom`` > + - > + * - __s32 > + - ``delta_pic_order_cnt0`` > + - > + * - __s32 > + - ``delta_pic_order_cnt1`` > + - > + * - __u8 > + - ``redundant_pic_cnt`` This should be moved to just before 'frame_num' as per the actual struct. > + - > + * - struct :c:type:`v4l2_h264_pred_weight_table` > + - ``pred_weight_table`` > + - > + * - __u32 > + - ``dec_ref_pic_marking_bit_size`` > + - > + * - __u32 > + - ``pic_order_cnt_bit_size`` > + - > + * - __u8 > + - ``cabac_init_idc`` > + - > + * - __s8 > + - ``slice_qp_delta`` > + - > + * - __s8 > + - ``slice_qs_delta`` > + - > + * - __u8 > + - ``disable_deblocking_filter_idc`` > + - > + * - __s8 > + - ``slice_alpha_c0_offset_div2`` > + - > + * - __s8 > + - ``slice_beta_offset_div2`` > + - > + * - __u32 > + - ``slice_group_change_cycle`` > + - > + * - __u8 > + - ``num_ref_idx_l0_active_minus1`` > + - > + * - __u8 > + - ``num_ref_idx_l1_active_minus1`` These two fields should be moved to just before the slice_group_change_cycle field as per the actual struct. > + - > + * - __u8 > + - ``ref_pic_list0[32]`` > + - > + * - __u8 > + - ``ref_pic_list1[32]`` > + - > + * - __u32 > + - ``flags`` > + - See :ref:`Slice Parameter Flags <h264_slice_flags>` > + > +.. _h264_slice_flags: > + > +``Slice Parameter Set Flags`` > + > +.. cssclass:: longtable > + > +.. flat-table:: > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - ``V4L2_H264_SLICE_FLAG_FIELD_PIC`` > + - 0x00000001 > + - > + * - ``V4L2_H264_SLICE_FLAG_BOTTOM_FIELD`` > + - 0x00000002 > + - > + * - ``V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED`` > + - 0x00000004 > + - > + * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH`` > + - 0x00000008 > + - > + > +``Prediction Weight Table`` > + > + Unless there's a specific comment, refer to the specification for > + the documentation of these fields, section 7.4.3.2 "Prediction > + Weight Table Semantics". > + > +.. c:type:: v4l2_h264_pred_weight_table > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_h264_pred_weight_table > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __u16 > + - ``luma_log2_weight_denom`` > + - > + * - __u16 > + - ``chroma_log2_weight_denom`` > + - > + * - struct :c:type:`v4l2_h264_weight_factors` > + - ``weight_factors[2]`` > + - > + > +.. c:type:: v4l2_h264_weight_factors > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_h264_weight_factors > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __s8 > + - ``luma_weight[32]`` > + - > + * - __s8 > + - ``luma_offset[32]`` > + - > + * - __s8 > + - ``chroma_weight[32][2]`` > + - > + * - __s8 > + - ``chroma_offset[32][2]`` > + - > + > +``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)`` > + Specifies the decode parameters (as extracted from the bitstream) > + for the associated H264 slice data. This includes the necessary > + parameters for configuring a stateless hardware decoding pipeline > + for H264. The bitstream parameters are defined according to > + :ref:`h264`. Unless there's a specific comment, refer to the > + specification for the documentation of these fields. > + > + .. note:: > + > + This compound control is not yet part of the public kernel API and > + it is expected to change. > + > +.. c:type:: v4l2_ctrl_h264_decode_param > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_ctrl_h264_decode_param > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __u32 > + - ``num_slices`` > + - Number of slices needed to decode the current frame > + * - __u16 > + - ``idr_pic_flag`` > + - Is the picture an IDR picture? > + * - __u16 > + - ``nal_ref_idc`` > + - NAL reference ID value coming from the NAL Unit header > + * - __s32 > + - ``top_field_order_cnt`` > + - Picture Order Count for the coded top field > + * - __s32 > + - ``bottom_field_order_cnt`` > + - Picture Order Count for the coded bottom field > + * - struct :c:type:`v4l2_h264_dpb_entry` > + - ``dpb[16]`` > + - > + > +.. c:type:: v4l2_h264_dpb_entry > + > +.. cssclass:: longtable > + > +.. flat-table:: struct v4l2_h264_dpb_entry > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - __u32 > + - ``tag`` > + - tag to identify the buffer containing the reference frame This is outdated: it's a __u64 timestamp. You can copy the struct backward_ref_ts documentation from struct v4l2_ctrl_mpeg2_slice_params and edit it a bit for H.264. > + * - __u16 > + - ``frame_num`` > + - > + * - __u16 > + - ``pic_num`` > + - > + * - __s32 > + - ``top_field_order_cnt`` > + - > + * - __s32 > + - ``bottom_field_order_cnt`` > + - > + * - __u32 > + - ``flags`` > + - See :ref:`DPB Entry Flags <h264_dpb_flags>` > + > +.. _h264_dpb_flags: > + > +``DPB Entries Flags`` > + > +.. cssclass:: longtable > + > +.. flat-table:: > + :header-rows: 0 > + :stub-columns: 0 > + :widths: 1 1 2 > + > + * - ``V4L2_H264_DPB_ENTRY_FLAG_VALID`` > + - 0x00000001 > + - > + * - ``V4L2_H264_DPB_ENTRY_FLAG_ACTIVE`` > + - 0x00000002 > + - > + > MFC 5.1 MPEG Controls > --------------------- > > diff --git a/Documentation/media/uapi/v4l/pixfmt-compressed.rst b/Documentation/media/uapi/v4l/pixfmt-compressed.rst > index e4c5e456df59..95905b967693 100644 > --- a/Documentation/media/uapi/v4l/pixfmt-compressed.rst > +++ b/Documentation/media/uapi/v4l/pixfmt-compressed.rst > @@ -52,6 +52,26 @@ Compressed Formats > - ``V4L2_PIX_FMT_H264_MVC`` > - 'M264' > - H264 MVC video elementary stream. > + * .. _V4L2-PIX-FMT-H264-SLICE: > + > + - ``V4L2_PIX_FMT_H264_SLICE`` > + - 'S264' > + - H264 parsed slice data, as extracted from the H264 bitstream. > + This format is adapted for stateless video decoders that > + implement an H264 pipeline (using the :ref:`codec` and > + :ref:`media-request-api`). Metadata associated with the frame > + to decode are required to be passed through the > + ``V4L2_CID_MPEG_VIDEO_H264_SPS``, > + ``V4L2_CID_MPEG_VIDEO_H264_PPS``, > + ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` and > + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS`` controls and > + scaling matrices can optionally be specified through the > + ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX`` control. See the > + :ref:`associated Codec Control IDs <v4l2-mpeg-h264>`. > + Exactly one output and one capture buffer must be provided for > + use with this pixel format. The output buffer must contain the > + appropriate number of macroblocks to decode a full > + corresponding frame to the matching capture buffer. > * .. _V4L2-PIX-FMT-H263: > > - ``V4L2_PIX_FMT_H263`` > diff --git a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > index f824162d0ea9..bf29dc5b9758 100644 > --- a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > +++ b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > @@ -443,6 +443,36 @@ See also the examples in :ref:`control`. > - n/a > - A struct :c:type:`v4l2_ctrl_mpeg2_quantization`, containing MPEG-2 > quantization matrices for stateless video decoders. > + * - ``V4L2_CTRL_TYPE_H264_SPS`` > + - n/a > + - n/a > + - n/a > + - A struct :c:type:`v4l2_ctrl_h264_sps`, containing H264 > + sequence parameters for stateless video decoders. > + * - ``V4L2_CTRL_TYPE_H264_PPS`` > + - n/a > + - n/a > + - n/a > + - A struct :c:type:`v4l2_ctrl_h264_pps`, containing H264 > + picture parameters for stateless video decoders. > + * - ``V4L2_CTRL_TYPE_H264_SCALING_MATRIX`` > + - n/a > + - n/a > + - n/a > + - A struct :c:type:`v4l2_ctrl_h264_scaling_matrix`, containing H264 > + scaling matrices for stateless video decoders. > + * - ``V4L2_CTRL_TYPE_H264_SLICE_PARAMS`` > + - n/a > + - n/a > + - n/a > + - A struct :c:type:`v4l2_ctrl_h264_slice_param`, containing H264 > + slice parameters for stateless video decoders. > + * - ``V4L2_CTRL_TYPE_H264_DECODE_PARAMS`` > + - n/a > + - n/a > + - n/a > + - A struct :c:type:`v4l2_ctrl_h264_decode_param`, containing H264 > + decode parameters for stateless video decoders. > > .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}| > > diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions > index 64d348e67df9..55cbe324b9fc 100644 > --- a/Documentation/media/videodev2.h.rst.exceptions > +++ b/Documentation/media/videodev2.h.rst.exceptions > @@ -136,6 +136,11 @@ replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type` > replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type` > replace symbol V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS :c:type:`v4l2_ctrl_type` > replace symbol V4L2_CTRL_TYPE_MPEG2_QUANTIZATION :c:type:`v4l2_ctrl_type` > +replace symbol V4L2_CTRL_TYPE_H264_SPS :c:type:`v4l2_ctrl_type` > +replace symbol V4L2_CTRL_TYPE_H264_PPS :c:type:`v4l2_ctrl_type` > +replace symbol V4L2_CTRL_TYPE_H264_SCALING_MATRIX :c:type:`v4l2_ctrl_type` > +replace symbol V4L2_CTRL_TYPE_H264_SLICE_PARAMS :c:type:`v4l2_ctrl_type` > +replace symbol V4L2_CTRL_TYPE_H264_DECODE_PARAMS :c:type:`v4l2_ctrl_type` > > # V4L2 capability defines > replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c > index e3bd441fa29a..fe9f9447caba 100644 > --- a/drivers/media/v4l2-core/v4l2-ctrls.c > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c > @@ -825,6 +825,11 @@ const char *v4l2_ctrl_get_name(u32 id) > case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers"; > case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP: > return "H264 Set QP Value for HC Layers"; > + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 Sequence Parameter Set"; > + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 Picture Parameter Set"; > + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix"; > + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters"; > + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters"; > case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value"; > case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value"; > case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value"; > @@ -1300,6 +1305,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > case V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION: > *type = V4L2_CTRL_TYPE_MPEG2_QUANTIZATION; > break; > + case V4L2_CID_MPEG_VIDEO_H264_SPS: > + *type = V4L2_CTRL_TYPE_H264_SPS; > + break; > + case V4L2_CID_MPEG_VIDEO_H264_PPS: > + *type = V4L2_CTRL_TYPE_H264_PPS; > + break; > + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: > + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX; > + break; > + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: > + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAMS; > + break; > + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: > + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAMS; > + break; > default: > *type = V4L2_CTRL_TYPE_INTEGER; > break; > @@ -1666,6 +1686,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx, > case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: > return 0; > > + case V4L2_CTRL_TYPE_H264_SPS: > + case V4L2_CTRL_TYPE_H264_PPS: > + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: > + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: > + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: > + return 0; > + > default: > return -EINVAL; > } > @@ -2246,6 +2273,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, > case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: > elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization); > break; > + case V4L2_CTRL_TYPE_H264_SPS: > + elem_size = sizeof(struct v4l2_ctrl_h264_sps); > + break; > + case V4L2_CTRL_TYPE_H264_PPS: > + elem_size = sizeof(struct v4l2_ctrl_h264_pps); > + break; > + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: > + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix); > + break; > + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: > + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param); > + break; > + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: > + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param); > + break; > default: > if (type < V4L2_CTRL_COMPOUND_TYPES) > elem_size = sizeof(s32); > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > index 1441a73ce64c..f42d6e7049eb 100644 > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > @@ -1313,6 +1313,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) > case V4L2_PIX_FMT_H264: descr = "H.264"; break; > case V4L2_PIX_FMT_H264_NO_SC: descr = "H.264 (No Start Codes)"; break; > case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break; > + case V4L2_PIX_FMT_H264_SLICE: descr = "H.264 Parsed Slice Data"; break; > case V4L2_PIX_FMT_H263: descr = "H.263"; break; > case V4L2_PIX_FMT_MPEG1: descr = "MPEG-1 ES"; break; > case V4L2_PIX_FMT_MPEG2: descr = "MPEG-2 ES"; break; > diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h > new file mode 100644 > index 000000000000..7b0563dfa05f > --- /dev/null > +++ b/include/media/h264-ctrls.h > @@ -0,0 +1,179 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * These are the H.264 state controls for use with stateless H.264 > + * codec drivers. > + * > + * It turns out that these structs are not stable yet and will undergo > + * more changes. So keep them private until they are stable and ready to > + * become part of the official public API. > + */ > + > +#ifndef _H264_CTRLS_H_ > +#define _H264_CTRLS_H_ > + > +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383) > +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384) > +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385) > +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+386) > +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+387) > + > +/* enum v4l2_ctrl_type type values */ > +#define V4L2_CTRL_TYPE_H264_SPS 0x0105 > +#define V4L2_CTRL_TYPE_H264_PPS 0x0106 > +#define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0107 > +#define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0108 > +#define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0109 > + > +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01 > +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02 > +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04 > +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08 > +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10 > +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20 > + > +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01 > +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02 > +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04 > +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08 > +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10 > +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20 > +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40 > + > +struct v4l2_ctrl_h264_sps { > + __u8 profile_idc; > + __u8 constraint_set_flags; > + __u8 level_idc; > + __u8 seq_parameter_set_id; > + __u8 chroma_format_idc; > + __u8 bit_depth_luma_minus8; > + __u8 bit_depth_chroma_minus8; > + __u8 log2_max_frame_num_minus4; > + __u8 pic_order_cnt_type; > + __u8 log2_max_pic_order_cnt_lsb_minus4; > + __u8 max_num_ref_frames; > + __u8 num_ref_frames_in_pic_order_cnt_cycle; > + __s32 offset_for_ref_frame[255]; Where does 255 come from? Should this be a define? If this is not based on the standard but instead a driver limitation, then I think that should be documented. In that case I assume that if more is needed in the future, then that will be done through additional controls. > + __s32 offset_for_non_ref_pic; > + __s32 offset_for_top_to_bottom_field; > + __u16 pic_width_in_mbs_minus1; > + __u16 pic_height_in_map_units_minus1; > + __u32 flags; > +}; > + > +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001 > +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002 > +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004 > +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008 > +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010 > +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020 > +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040 > +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080 > + > +struct v4l2_ctrl_h264_pps { > + __u8 pic_parameter_set_id; > + __u8 seq_parameter_set_id; > + __u8 num_slice_groups_minus1; > + __u8 num_ref_idx_l0_default_active_minus1; > + __u8 num_ref_idx_l1_default_active_minus1; > + __u8 weighted_bipred_idc; > + __s8 pic_init_qp_minus26; > + __s8 pic_init_qs_minus26; > + __s8 chroma_qp_index_offset; > + __s8 second_chroma_qp_index_offset; > + __u16 flags; > +}; > + > +struct v4l2_ctrl_h264_scaling_matrix { > + __u8 scaling_list_4x4[6][16]; > + __u8 scaling_list_8x8[6][64]; > +}; > + > +struct v4l2_h264_weight_factors { > + __s8 luma_weight[32]; > + __s8 luma_offset[32]; > + __s8 chroma_weight[32][2]; > + __s8 chroma_offset[32][2]; Same question here. > +}; > + > +struct v4l2_h264_pred_weight_table { > + __u16 luma_log2_weight_denom; > + __u16 chroma_log2_weight_denom; > + struct v4l2_h264_weight_factors weight_factors[2]; I assume [0] is for luma and [1] is for chroma? Should be documented. > +}; > + > +#define V4L2_H264_SLICE_TYPE_P 0 > +#define V4L2_H264_SLICE_TYPE_B 1 > +#define V4L2_H264_SLICE_TYPE_I 2 > +#define V4L2_H264_SLICE_TYPE_SP 3 > +#define V4L2_H264_SLICE_TYPE_SI 4 > + > +#define V4L2_H264_SLICE_FLAG_FIELD_PIC 0x01 > +#define V4L2_H264_SLICE_FLAG_BOTTOM_FIELD 0x02 > +#define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04 > +#define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08 > + > +struct v4l2_ctrl_h264_slice_param { > + /* Size in bytes, including header */ > + __u32 size; > + /* Offset in bits to slice_data() from the beginning of this slice. */ > + __u32 header_bit_size; > + > + __u16 first_mb_in_slice; > + __u8 slice_type; > + __u8 pic_parameter_set_id; > + __u8 colour_plane_id; > + __u8 redundant_pic_cnt; > + __u16 frame_num; > + __u16 idr_pic_id; > + __u16 pic_order_cnt_lsb; > + __s32 delta_pic_order_cnt_bottom; > + __s32 delta_pic_order_cnt0; > + __s32 delta_pic_order_cnt1; > + > + struct v4l2_h264_pred_weight_table pred_weight_table; > + /* Size in bits of dec_ref_pic_marking() syntax element. */ > + __u32 dec_ref_pic_marking_bit_size; > + /* Size in bits of pic order count syntax. */ > + __u32 pic_order_cnt_bit_size; > + > + __u8 cabac_init_idc; > + __s8 slice_qp_delta; > + __s8 slice_qs_delta; > + __u8 disable_deblocking_filter_idc; > + __s8 slice_alpha_c0_offset_div2; > + __s8 slice_beta_offset_div2; > + __u8 num_ref_idx_l0_active_minus1; > + __u8 num_ref_idx_l1_active_minus1; > + __u32 slice_group_change_cycle; > + > + /* Entries on each list are indices > + * into v4l2_ctrl_h264_decode_param.dpb[]. */ > + __u8 ref_pic_list0[32]; > + __u8 ref_pic_list1[32]; Where does 32 come from? > + > + __u32 flags; > +}; > + > +#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01 > +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x02 > + > +struct v4l2_h264_dpb_entry { > + __u64 timestamp; > + __u16 frame_num; > + __u16 pic_num; > + /* Note that field is indicated by v4l2_buffer.field */ > + __s32 top_field_order_cnt; > + __s32 bottom_field_order_cnt; > + __u32 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */ > +}; > + > +struct v4l2_ctrl_h264_decode_param { > + __u32 num_slices; > + __u16 idr_pic_flag; > + __u16 nal_ref_idc; > + __s32 top_field_order_cnt; > + __s32 bottom_field_order_cnt; > + struct v4l2_h264_dpb_entry dpb[16]; And 16? Nothing special needs to be done for those constants that are based on the standard. But if it isn't part of the standard but a driver constraint, then that should be documented, and there should be an idea on how it can be extended for future drivers without that constraint. I think the API should be designed with 4k video in mind. So if some of these constants would be too small when dealing with 4k (even if the current HW doesn't support this yet), then these constants would have to be increased. And yes, I know 8k video is starting to appear, but I think it is OK that additional control(s) would be needed to support 8k. > +}; > + > +#endif > diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h > index d63cf227b0ab..22b6d09c4764 100644 > --- a/include/media/v4l2-ctrls.h > +++ b/include/media/v4l2-ctrls.h > @@ -23,10 +23,11 @@ > #include <media/media-request.h> > > /* > - * Include the mpeg2 stateless codec compound control definitions. > + * Include the stateless codec compound control definitions. > * This will move to the public headers once this API is fully stable. > */ > #include <media/mpeg2-ctrls.h> > +#include <media/h264-ctrls.h> > > /* forward references */ > struct file; > @@ -49,6 +50,11 @@ struct poll_table_struct; > * @p_char: Pointer to a string. > * @p_mpeg2_slice_params: Pointer to a MPEG2 slice parameters structure. > * @p_mpeg2_quantization: Pointer to a MPEG2 quantization data structure. > + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps. > + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps. > + * @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix. > + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param. > + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param. > * @p: Pointer to a compound value. > */ > union v4l2_ctrl_ptr { > @@ -60,6 +66,11 @@ union v4l2_ctrl_ptr { > char *p_char; > struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params; > struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quantization; > + struct v4l2_ctrl_h264_sps *p_h264_sps; > + struct v4l2_ctrl_h264_pps *p_h264_pps; > + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix; > + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param; > + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param; > void *p; > }; > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index d6eed479c3a6..6fc955926bdb 100644 > --- a/include/uapi/linux/videodev2.h > +++ b/include/uapi/linux/videodev2.h > @@ -645,6 +645,7 @@ struct v4l2_pix_format { > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */ > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */ > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */ > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ > #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */ > #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */ > #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */ > Regards, Hans ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 15:16 ` Hans Verkuil @ 2019-02-11 15:21 ` Hans Verkuil 2019-02-12 13:05 ` Maxime Ripard 2019-02-11 17:16 ` Nicolas Dufresne 1 sibling, 1 reply; 24+ messages in thread From: Hans Verkuil @ 2019-02-11 15:21 UTC (permalink / raw) To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni, Guenter Roeck On 2/11/19 4:16 PM, Hans Verkuil wrote: > Hi Maxime, > > A quick review below. Note that I am no expert on the codec details, so > I leave that to others. I'm mainly concentrating on the structs, flags, etc. > > On 2/11/19 3:39 PM, Maxime Ripard wrote: >> From: Pawel Osciak <posciak@chromium.org> >> >> Stateless video codecs will require both the H264 metadata and slices in >> order to be able to decode frames. >> >> This introduces the definitions for a new pixel format for H264 slices that >> have been parsed, as well as the structures used to pass the metadata from >> the userspace to the kernel. >> >> Co-Developed-by: Maxime Ripard <maxime.ripard@bootlin.com> >> Signed-off-by: Pawel Osciak <posciak@chromium.org> >> Signed-off-by: Guenter Roeck <groeck@chromium.org> >> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> >> --- >> Documentation/media/uapi/v4l/biblio.rst | 9 +- >> Documentation/media/uapi/v4l/extended-controls.rst | 530 ++++++++++++++- >> Documentation/media/uapi/v4l/pixfmt-compressed.rst | 20 +- >> Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 30 +- >> Documentation/media/videodev2.h.rst.exceptions | 5 +- >> drivers/media/v4l2-core/v4l2-ctrls.c | 42 +- >> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +- >> include/media/h264-ctrls.h | 179 +++++- >> include/media/v4l2-ctrls.h | 13 +- >> include/uapi/linux/videodev2.h | 1 +- >> 10 files changed, 829 insertions(+), 1 deletion(-) >> create mode 100644 include/media/h264-ctrls.h >> >> diff --git a/Documentation/media/uapi/v4l/biblio.rst b/Documentation/media/uapi/v4l/biblio.rst >> index ec33768c055e..3fc3f7ff338a 100644 >> --- a/Documentation/media/uapi/v4l/biblio.rst >> +++ b/Documentation/media/uapi/v4l/biblio.rst >> @@ -122,6 +122,15 @@ ITU BT.1119 >> >> :author: International Telecommunication Union (http://www.itu.ch) >> >> +.. _h264: >> + >> +ITU H.264 >> +========= >> + >> +:title: ITU-T Recommendation H.264 "Advanced Video Coding for Generic Audiovisual Services" >> + >> +:author: International Telecommunication Union (http://www.itu.ch) >> + >> .. _jfif: >> >> JFIF >> diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst >> index af4273aa5e85..7e306aa0d0a6 100644 >> --- a/Documentation/media/uapi/v4l/extended-controls.rst >> +++ b/Documentation/media/uapi/v4l/extended-controls.rst >> @@ -1703,6 +1703,536 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - >> non-intra-coded frames, in zigzag scanning order. Only relevant for >> non-4:2:0 YUV formats. >> >> +.. _v4l2-mpeg-h264: >> + >> +``V4L2_CID_MPEG_VIDEO_H264_SPS (struct)`` >> + Specifies the sequence parameter set (as extracted from the >> + bitstream) for the associated H264 slice data. This includes the >> + necessary parameters for configuring a stateless hardware decoding >> + pipeline for H264. The bitstream parameters are defined according >> + to :ref:`h264`. Unless there's a specific comment, refer to the >> + specification for the documentation of these fields, section 7.4.2.1.1 >> + "Sequence Parameter Set Data Semantics". >> + >> + .. note:: >> + >> + This compound control is not yet part of the public kernel API and >> + it is expected to change. >> + >> +.. c:type:: v4l2_ctrl_h264_sps >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_ctrl_h264_sps >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __u8 >> + - ``profile_idc`` >> + - >> + * - __u8 >> + - ``constraint_set_flags`` >> + - See :ref:`Sequence Parameter Set Constraints Set Flags <h264_sps_constraints_set_flags>` >> + * - __u8 >> + - ``level_idc`` >> + - >> + * - __u8 >> + - ``seq_parameter_set_id`` >> + - >> + * - __u8 >> + - ``chroma_format_idc`` >> + - >> + * - __u8 >> + - ``bit_depth_luma_minus8`` >> + - >> + * - __u8 >> + - ``bit_depth_chroma_minus8`` >> + - >> + * - __u8 >> + - ``log2_max_frame_num_minus4`` >> + - >> + * - __u8 >> + - ``pic_order_cnt_type`` >> + - >> + * - __u8 >> + - ``log2_max_pic_order_cnt_lsb_minus4`` >> + - >> + * - __u8 >> + - ``max_num_ref_frames`` >> + - >> + * - __u8 >> + - ``num_ref_frames_in_pic_order_cnt_cycle`` >> + - >> + * - __s32 >> + - ``offset_for_ref_frame[255]`` >> + - >> + * - __s32 >> + - ``offset_for_non_ref_pic`` >> + - >> + * - __s32 >> + - ``offset_for_top_to_bottom_field`` >> + - >> + * - __u16 >> + - ``pic_width_in_mbs_minus1`` >> + - >> + * - __u16 >> + - ``pic_height_in_map_units_minus1`` >> + - >> + * - __u32 >> + - ``flags`` >> + - See :ref:`Sequence Parameter Set Flags <h264_sps_flags>` >> + >> +.. _h264_sps_constraints_set_flags: >> + >> +``Sequence Parameter Set Constraints Set Flags`` >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - ``V4L2_H264_SPS_CONSTRAINT_SET0_FLAG`` >> + - 0x00000001 >> + - >> + * - ``V4L2_H264_SPS_CONSTRAINT_SET1_FLAG`` >> + - 0x00000002 >> + - >> + * - ``V4L2_H264_SPS_CONSTRAINT_SET2_FLAG`` >> + - 0x00000004 >> + - >> + * - ``V4L2_H264_SPS_CONSTRAINT_SET3_FLAG`` >> + - 0x00000008 >> + - >> + * - ``V4L2_H264_SPS_CONSTRAINT_SET4_FLAG`` >> + - 0x00000010 >> + - >> + * - ``V4L2_H264_SPS_CONSTRAINT_SET5_FLAG`` >> + - 0x00000020 >> + - >> + >> +.. _h264_sps_flags: >> + >> +``Sequence Parameter Set Flags`` >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - ``V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE`` >> + - 0x00000001 >> + - >> + * - ``V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS`` >> + - 0x00000002 >> + - >> + * - ``V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO`` >> + - 0x00000004 >> + - >> + * - ``V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED`` >> + - 0x00000008 >> + - >> + * - ``V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY`` >> + - 0x00000010 >> + - >> + * - ``V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD`` >> + - 0x00000020 >> + - >> + * - ``V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE`` >> + - 0x00000040 >> + - >> + >> +``V4L2_CID_MPEG_VIDEO_H264_PPS (struct)`` >> + Specifies the picture parameter set (as extracted from the >> + bitstream) for the associated H264 slice data. This includes the >> + necessary parameters for configuring a stateless hardware decoding >> + pipeline for H264. The bitstream parameters are defined according >> + to :ref:`h264`. Unless there's a specific comment, refer to the >> + specification for the documentation of these fields, section 7.4.2.2 >> + "Picture Parameter Set RBSP Semantics". >> + >> + .. note:: >> + >> + This compound control is not yet part of the public kernel API and >> + it is expected to change. >> + >> +.. c:type:: v4l2_ctrl_h264_pps >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_ctrl_h264_pps >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __u8 >> + - ``pic_parameter_set_id`` >> + - >> + * - __u8 >> + - ``seq_parameter_set_id`` >> + - >> + * - __u8 >> + - ``num_slice_groups_minus1`` >> + - >> + * - __u8 >> + - ``num_ref_idx_l0_default_active_minus1`` >> + - >> + * - __u8 >> + - ``num_ref_idx_l1_default_active_minus1`` >> + - >> + * - __u8 >> + - ``weighted_bipred_idc`` >> + - >> + * - __s8 >> + - ``pic_init_qp_minus26`` >> + - >> + * - __s8 >> + - ``pic_init_qs_minus26`` >> + - >> + * - __s8 >> + - ``chroma_qp_index_offset`` >> + - >> + * - __s8 >> + - ``second_chroma_qp_index_offset`` >> + - >> + * - __u16 >> + - ``flags`` >> + - See :ref:`Picture Parameter Set Flags <h264_pps_flags>` >> + >> +.. _h264_pps_flags: >> + >> +``Picture Parameter Set Flags`` >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - ``V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE`` >> + - 0x00000001 >> + - >> + * - ``V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT`` >> + - 0x00000002 >> + - >> + * - ``V4L2_H264_PPS_FLAG_WEIGHTED_PRED`` >> + - 0x00000004 >> + - >> + * - ``V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT`` >> + - 0x00000008 >> + - >> + * - ``V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED`` >> + - 0x00000010 >> + - >> + * - ``V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT`` >> + - 0x00000020 >> + - >> + * - ``V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE`` >> + - 0x00000040 >> + - >> + * - ``V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT`` >> + - 0x00000080 >> + - >> + >> +``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (struct)`` >> + Specifies the scaling matrix (as extracted from the bitstream) for >> + the associated H264 slice data. The bitstream parameters are >> + defined according to :ref:`h264`. Unless there's a specific >> + comment, refer to the specification for the documentation of these >> + fields, section 7.4.2.1.1.1 "Scaling List Semantics". >> + >> + .. note:: >> + >> + This compound control is not yet part of the public kernel API and >> + it is expected to change. >> + >> +.. c:type:: v4l2_ctrl_h264_scaling_matrix >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_ctrl_h264_scaling_matrix >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __u8 >> + - ``scaling_list_4x4[6][16]`` >> + - >> + * - __u8 >> + - ``scaling_list_8x8[6][64]`` >> + - >> + >> +``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (struct)`` >> + Specifies the slice parameters (as extracted from the bitstream) >> + for the associated H264 slice data. This includes the necessary >> + parameters for configuring a stateless hardware decoding pipeline >> + for H264. The bitstream parameters are defined according to >> + :ref:`h264`. Unless there's a specific comment, refer to the >> + specification for the documentation of these fields, section 7.4.3 >> + "Slice Header Semantics". >> + >> + .. note:: >> + >> + This compound control is not yet part of the public kernel API and >> + it is expected to change. >> + >> +.. c:type:: v4l2_ctrl_h264_slice_param >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_ctrl_h264_slice_param >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __u32 >> + - ``size`` >> + - >> + * - __u32 >> + - ``header_bit_size`` >> + - >> + * - __u16 >> + - ``first_mb_in_slice`` >> + - >> + * - __u8 >> + - ``slice_type`` >> + - >> + * - __u8 >> + - ``pic_parameter_set_id`` >> + - >> + * - __u8 >> + - ``colour_plane_id`` >> + - >> + * - __u16 >> + - ``frame_num`` >> + - >> + * - __u16 >> + - ``idr_pic_id`` >> + - >> + * - __u16 >> + - ``pic_order_cnt_lsb`` >> + - >> + * - __s32 >> + - ``delta_pic_order_cnt_bottom`` >> + - >> + * - __s32 >> + - ``delta_pic_order_cnt0`` >> + - >> + * - __s32 >> + - ``delta_pic_order_cnt1`` >> + - >> + * - __u8 >> + - ``redundant_pic_cnt`` > > This should be moved to just before 'frame_num' as per the actual struct. > >> + - >> + * - struct :c:type:`v4l2_h264_pred_weight_table` >> + - ``pred_weight_table`` >> + - >> + * - __u32 >> + - ``dec_ref_pic_marking_bit_size`` >> + - >> + * - __u32 >> + - ``pic_order_cnt_bit_size`` >> + - >> + * - __u8 >> + - ``cabac_init_idc`` >> + - >> + * - __s8 >> + - ``slice_qp_delta`` >> + - >> + * - __s8 >> + - ``slice_qs_delta`` >> + - >> + * - __u8 >> + - ``disable_deblocking_filter_idc`` >> + - >> + * - __s8 >> + - ``slice_alpha_c0_offset_div2`` >> + - >> + * - __s8 >> + - ``slice_beta_offset_div2`` >> + - >> + * - __u32 >> + - ``slice_group_change_cycle`` >> + - >> + * - __u8 >> + - ``num_ref_idx_l0_active_minus1`` >> + - >> + * - __u8 >> + - ``num_ref_idx_l1_active_minus1`` > > These two fields should be moved to just before the slice_group_change_cycle > field as per the actual struct. > >> + - >> + * - __u8 >> + - ``ref_pic_list0[32]`` >> + - >> + * - __u8 >> + - ``ref_pic_list1[32]`` >> + - >> + * - __u32 >> + - ``flags`` >> + - See :ref:`Slice Parameter Flags <h264_slice_flags>` >> + >> +.. _h264_slice_flags: >> + >> +``Slice Parameter Set Flags`` >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - ``V4L2_H264_SLICE_FLAG_FIELD_PIC`` >> + - 0x00000001 >> + - >> + * - ``V4L2_H264_SLICE_FLAG_BOTTOM_FIELD`` >> + - 0x00000002 >> + - >> + * - ``V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED`` >> + - 0x00000004 >> + - >> + * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH`` >> + - 0x00000008 >> + - >> + >> +``Prediction Weight Table`` >> + >> + Unless there's a specific comment, refer to the specification for >> + the documentation of these fields, section 7.4.3.2 "Prediction >> + Weight Table Semantics". >> + >> +.. c:type:: v4l2_h264_pred_weight_table >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_h264_pred_weight_table >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __u16 >> + - ``luma_log2_weight_denom`` >> + - >> + * - __u16 >> + - ``chroma_log2_weight_denom`` >> + - >> + * - struct :c:type:`v4l2_h264_weight_factors` >> + - ``weight_factors[2]`` >> + - >> + >> +.. c:type:: v4l2_h264_weight_factors >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_h264_weight_factors >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __s8 >> + - ``luma_weight[32]`` >> + - >> + * - __s8 >> + - ``luma_offset[32]`` >> + - >> + * - __s8 >> + - ``chroma_weight[32][2]`` >> + - >> + * - __s8 >> + - ``chroma_offset[32][2]`` >> + - >> + >> +``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)`` >> + Specifies the decode parameters (as extracted from the bitstream) >> + for the associated H264 slice data. This includes the necessary >> + parameters for configuring a stateless hardware decoding pipeline >> + for H264. The bitstream parameters are defined according to >> + :ref:`h264`. Unless there's a specific comment, refer to the >> + specification for the documentation of these fields. >> + >> + .. note:: >> + >> + This compound control is not yet part of the public kernel API and >> + it is expected to change. >> + >> +.. c:type:: v4l2_ctrl_h264_decode_param >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_ctrl_h264_decode_param >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __u32 >> + - ``num_slices`` >> + - Number of slices needed to decode the current frame >> + * - __u16 >> + - ``idr_pic_flag`` >> + - Is the picture an IDR picture? >> + * - __u16 >> + - ``nal_ref_idc`` >> + - NAL reference ID value coming from the NAL Unit header >> + * - __s32 >> + - ``top_field_order_cnt`` >> + - Picture Order Count for the coded top field >> + * - __s32 >> + - ``bottom_field_order_cnt`` >> + - Picture Order Count for the coded bottom field >> + * - struct :c:type:`v4l2_h264_dpb_entry` >> + - ``dpb[16]`` >> + - >> + >> +.. c:type:: v4l2_h264_dpb_entry >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: struct v4l2_h264_dpb_entry >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - __u32 >> + - ``tag`` >> + - tag to identify the buffer containing the reference frame > > This is outdated: it's a __u64 timestamp. > > You can copy the struct backward_ref_ts documentation from struct > v4l2_ctrl_mpeg2_slice_params and edit it a bit for H.264. > >> + * - __u16 >> + - ``frame_num`` >> + - >> + * - __u16 >> + - ``pic_num`` >> + - >> + * - __s32 >> + - ``top_field_order_cnt`` >> + - >> + * - __s32 >> + - ``bottom_field_order_cnt`` >> + - >> + * - __u32 >> + - ``flags`` >> + - See :ref:`DPB Entry Flags <h264_dpb_flags>` >> + >> +.. _h264_dpb_flags: >> + >> +``DPB Entries Flags`` >> + >> +.. cssclass:: longtable >> + >> +.. flat-table:: >> + :header-rows: 0 >> + :stub-columns: 0 >> + :widths: 1 1 2 >> + >> + * - ``V4L2_H264_DPB_ENTRY_FLAG_VALID`` >> + - 0x00000001 >> + - >> + * - ``V4L2_H264_DPB_ENTRY_FLAG_ACTIVE`` >> + - 0x00000002 >> + - >> + >> MFC 5.1 MPEG Controls >> --------------------- >> >> diff --git a/Documentation/media/uapi/v4l/pixfmt-compressed.rst b/Documentation/media/uapi/v4l/pixfmt-compressed.rst >> index e4c5e456df59..95905b967693 100644 >> --- a/Documentation/media/uapi/v4l/pixfmt-compressed.rst >> +++ b/Documentation/media/uapi/v4l/pixfmt-compressed.rst >> @@ -52,6 +52,26 @@ Compressed Formats >> - ``V4L2_PIX_FMT_H264_MVC`` >> - 'M264' >> - H264 MVC video elementary stream. >> + * .. _V4L2-PIX-FMT-H264-SLICE: >> + >> + - ``V4L2_PIX_FMT_H264_SLICE`` >> + - 'S264' >> + - H264 parsed slice data, as extracted from the H264 bitstream. >> + This format is adapted for stateless video decoders that >> + implement an H264 pipeline (using the :ref:`codec` and >> + :ref:`media-request-api`). Metadata associated with the frame >> + to decode are required to be passed through the >> + ``V4L2_CID_MPEG_VIDEO_H264_SPS``, >> + ``V4L2_CID_MPEG_VIDEO_H264_PPS``, >> + ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` and >> + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS`` controls and >> + scaling matrices can optionally be specified through the >> + ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX`` control. See the >> + :ref:`associated Codec Control IDs <v4l2-mpeg-h264>`. >> + Exactly one output and one capture buffer must be provided for >> + use with this pixel format. The output buffer must contain the >> + appropriate number of macroblocks to decode a full >> + corresponding frame to the matching capture buffer. >> * .. _V4L2-PIX-FMT-H263: >> >> - ``V4L2_PIX_FMT_H263`` >> diff --git a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst >> index f824162d0ea9..bf29dc5b9758 100644 >> --- a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst >> +++ b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst >> @@ -443,6 +443,36 @@ See also the examples in :ref:`control`. >> - n/a >> - A struct :c:type:`v4l2_ctrl_mpeg2_quantization`, containing MPEG-2 >> quantization matrices for stateless video decoders. >> + * - ``V4L2_CTRL_TYPE_H264_SPS`` >> + - n/a >> + - n/a >> + - n/a >> + - A struct :c:type:`v4l2_ctrl_h264_sps`, containing H264 >> + sequence parameters for stateless video decoders. >> + * - ``V4L2_CTRL_TYPE_H264_PPS`` >> + - n/a >> + - n/a >> + - n/a >> + - A struct :c:type:`v4l2_ctrl_h264_pps`, containing H264 >> + picture parameters for stateless video decoders. >> + * - ``V4L2_CTRL_TYPE_H264_SCALING_MATRIX`` >> + - n/a >> + - n/a >> + - n/a >> + - A struct :c:type:`v4l2_ctrl_h264_scaling_matrix`, containing H264 >> + scaling matrices for stateless video decoders. >> + * - ``V4L2_CTRL_TYPE_H264_SLICE_PARAMS`` >> + - n/a >> + - n/a >> + - n/a >> + - A struct :c:type:`v4l2_ctrl_h264_slice_param`, containing H264 >> + slice parameters for stateless video decoders. >> + * - ``V4L2_CTRL_TYPE_H264_DECODE_PARAMS`` >> + - n/a >> + - n/a >> + - n/a >> + - A struct :c:type:`v4l2_ctrl_h264_decode_param`, containing H264 >> + decode parameters for stateless video decoders. >> >> .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}| >> >> diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions >> index 64d348e67df9..55cbe324b9fc 100644 >> --- a/Documentation/media/videodev2.h.rst.exceptions >> +++ b/Documentation/media/videodev2.h.rst.exceptions >> @@ -136,6 +136,11 @@ replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type` >> replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type` >> replace symbol V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS :c:type:`v4l2_ctrl_type` >> replace symbol V4L2_CTRL_TYPE_MPEG2_QUANTIZATION :c:type:`v4l2_ctrl_type` >> +replace symbol V4L2_CTRL_TYPE_H264_SPS :c:type:`v4l2_ctrl_type` >> +replace symbol V4L2_CTRL_TYPE_H264_PPS :c:type:`v4l2_ctrl_type` >> +replace symbol V4L2_CTRL_TYPE_H264_SCALING_MATRIX :c:type:`v4l2_ctrl_type` >> +replace symbol V4L2_CTRL_TYPE_H264_SLICE_PARAMS :c:type:`v4l2_ctrl_type` >> +replace symbol V4L2_CTRL_TYPE_H264_DECODE_PARAMS :c:type:`v4l2_ctrl_type` >> >> # V4L2 capability defines >> replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities >> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c >> index e3bd441fa29a..fe9f9447caba 100644 >> --- a/drivers/media/v4l2-core/v4l2-ctrls.c >> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c >> @@ -825,6 +825,11 @@ const char *v4l2_ctrl_get_name(u32 id) >> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers"; >> case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP: >> return "H264 Set QP Value for HC Layers"; >> + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 Sequence Parameter Set"; >> + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 Picture Parameter Set"; >> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix"; >> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters"; >> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters"; >> case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value"; >> case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value"; >> case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value"; >> @@ -1300,6 +1305,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, >> case V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION: >> *type = V4L2_CTRL_TYPE_MPEG2_QUANTIZATION; >> break; >> + case V4L2_CID_MPEG_VIDEO_H264_SPS: >> + *type = V4L2_CTRL_TYPE_H264_SPS; >> + break; >> + case V4L2_CID_MPEG_VIDEO_H264_PPS: >> + *type = V4L2_CTRL_TYPE_H264_PPS; >> + break; >> + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: >> + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX; >> + break; >> + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: >> + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAMS; >> + break; >> + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: >> + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAMS; >> + break; >> default: >> *type = V4L2_CTRL_TYPE_INTEGER; >> break; >> @@ -1666,6 +1686,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx, >> case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: >> return 0; >> >> + case V4L2_CTRL_TYPE_H264_SPS: >> + case V4L2_CTRL_TYPE_H264_PPS: >> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: >> + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: >> + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: >> + return 0; >> + >> default: >> return -EINVAL; >> } >> @@ -2246,6 +2273,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, >> case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: >> elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization); >> break; >> + case V4L2_CTRL_TYPE_H264_SPS: >> + elem_size = sizeof(struct v4l2_ctrl_h264_sps); >> + break; >> + case V4L2_CTRL_TYPE_H264_PPS: >> + elem_size = sizeof(struct v4l2_ctrl_h264_pps); >> + break; >> + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: >> + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix); >> + break; >> + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: >> + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param); >> + break; >> + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: >> + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param); >> + break; >> default: >> if (type < V4L2_CTRL_COMPOUND_TYPES) >> elem_size = sizeof(s32); >> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c >> index 1441a73ce64c..f42d6e7049eb 100644 >> --- a/drivers/media/v4l2-core/v4l2-ioctl.c >> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c >> @@ -1313,6 +1313,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) >> case V4L2_PIX_FMT_H264: descr = "H.264"; break; >> case V4L2_PIX_FMT_H264_NO_SC: descr = "H.264 (No Start Codes)"; break; >> case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break; >> + case V4L2_PIX_FMT_H264_SLICE: descr = "H.264 Parsed Slice Data"; break; >> case V4L2_PIX_FMT_H263: descr = "H.263"; break; >> case V4L2_PIX_FMT_MPEG1: descr = "MPEG-1 ES"; break; >> case V4L2_PIX_FMT_MPEG2: descr = "MPEG-2 ES"; break; >> diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h >> new file mode 100644 >> index 000000000000..7b0563dfa05f >> --- /dev/null >> +++ b/include/media/h264-ctrls.h >> @@ -0,0 +1,179 @@ >> +/* SPDX-License-Identifier: GPL-2.0 */ >> +/* >> + * These are the H.264 state controls for use with stateless H.264 >> + * codec drivers. >> + * >> + * It turns out that these structs are not stable yet and will undergo >> + * more changes. So keep them private until they are stable and ready to >> + * become part of the official public API. >> + */ >> + >> +#ifndef _H264_CTRLS_H_ >> +#define _H264_CTRLS_H_ >> + >> +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383) >> +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384) >> +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385) >> +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+386) >> +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+387) >> + >> +/* enum v4l2_ctrl_type type values */ >> +#define V4L2_CTRL_TYPE_H264_SPS 0x0105 >> +#define V4L2_CTRL_TYPE_H264_PPS 0x0106 >> +#define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0107 >> +#define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0108 >> +#define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0109 >> + >> +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01 >> +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02 >> +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04 >> +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08 >> +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10 >> +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20 >> + >> +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01 >> +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02 >> +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04 >> +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08 >> +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10 >> +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20 >> +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40 >> + >> +struct v4l2_ctrl_h264_sps { >> + __u8 profile_idc; >> + __u8 constraint_set_flags; >> + __u8 level_idc; >> + __u8 seq_parameter_set_id; >> + __u8 chroma_format_idc; >> + __u8 bit_depth_luma_minus8; >> + __u8 bit_depth_chroma_minus8; >> + __u8 log2_max_frame_num_minus4; >> + __u8 pic_order_cnt_type; >> + __u8 log2_max_pic_order_cnt_lsb_minus4; >> + __u8 max_num_ref_frames; >> + __u8 num_ref_frames_in_pic_order_cnt_cycle; >> + __s32 offset_for_ref_frame[255]; > > Where does 255 come from? Should this be a define? > > If this is not based on the standard but instead a driver limitation, then > I think that should be documented. In that case I assume that if more is needed > in the future, then that will be done through additional controls. > >> + __s32 offset_for_non_ref_pic; >> + __s32 offset_for_top_to_bottom_field; >> + __u16 pic_width_in_mbs_minus1; >> + __u16 pic_height_in_map_units_minus1; >> + __u32 flags; >> +}; >> + >> +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001 >> +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002 >> +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004 >> +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008 >> +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010 >> +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020 >> +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040 >> +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080 >> + >> +struct v4l2_ctrl_h264_pps { >> + __u8 pic_parameter_set_id; >> + __u8 seq_parameter_set_id; >> + __u8 num_slice_groups_minus1; >> + __u8 num_ref_idx_l0_default_active_minus1; >> + __u8 num_ref_idx_l1_default_active_minus1; >> + __u8 weighted_bipred_idc; >> + __s8 pic_init_qp_minus26; >> + __s8 pic_init_qs_minus26; >> + __s8 chroma_qp_index_offset; >> + __s8 second_chroma_qp_index_offset; >> + __u16 flags; >> +}; >> + >> +struct v4l2_ctrl_h264_scaling_matrix { >> + __u8 scaling_list_4x4[6][16]; >> + __u8 scaling_list_8x8[6][64]; >> +}; >> + >> +struct v4l2_h264_weight_factors { >> + __s8 luma_weight[32]; >> + __s8 luma_offset[32]; >> + __s8 chroma_weight[32][2]; >> + __s8 chroma_offset[32][2]; > > Same question here. > >> +}; >> + >> +struct v4l2_h264_pred_weight_table { >> + __u16 luma_log2_weight_denom; >> + __u16 chroma_log2_weight_denom; >> + struct v4l2_h264_weight_factors weight_factors[2]; > > I assume [0] is for luma and [1] is for chroma? Should be documented. > >> +}; >> + >> +#define V4L2_H264_SLICE_TYPE_P 0 >> +#define V4L2_H264_SLICE_TYPE_B 1 >> +#define V4L2_H264_SLICE_TYPE_I 2 >> +#define V4L2_H264_SLICE_TYPE_SP 3 >> +#define V4L2_H264_SLICE_TYPE_SI 4 >> + >> +#define V4L2_H264_SLICE_FLAG_FIELD_PIC 0x01 >> +#define V4L2_H264_SLICE_FLAG_BOTTOM_FIELD 0x02 >> +#define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04 >> +#define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08 >> + >> +struct v4l2_ctrl_h264_slice_param { >> + /* Size in bytes, including header */ >> + __u32 size; >> + /* Offset in bits to slice_data() from the beginning of this slice. */ >> + __u32 header_bit_size; >> + >> + __u16 first_mb_in_slice; >> + __u8 slice_type; >> + __u8 pic_parameter_set_id; >> + __u8 colour_plane_id; >> + __u8 redundant_pic_cnt; >> + __u16 frame_num; >> + __u16 idr_pic_id; >> + __u16 pic_order_cnt_lsb; >> + __s32 delta_pic_order_cnt_bottom; >> + __s32 delta_pic_order_cnt0; >> + __s32 delta_pic_order_cnt1; >> + >> + struct v4l2_h264_pred_weight_table pred_weight_table; >> + /* Size in bits of dec_ref_pic_marking() syntax element. */ >> + __u32 dec_ref_pic_marking_bit_size; >> + /* Size in bits of pic order count syntax. */ >> + __u32 pic_order_cnt_bit_size; >> + >> + __u8 cabac_init_idc; >> + __s8 slice_qp_delta; >> + __s8 slice_qs_delta; >> + __u8 disable_deblocking_filter_idc; >> + __s8 slice_alpha_c0_offset_div2; >> + __s8 slice_beta_offset_div2; >> + __u8 num_ref_idx_l0_active_minus1; >> + __u8 num_ref_idx_l1_active_minus1; >> + __u32 slice_group_change_cycle; >> + >> + /* Entries on each list are indices >> + * into v4l2_ctrl_h264_decode_param.dpb[]. */ >> + __u8 ref_pic_list0[32]; >> + __u8 ref_pic_list1[32]; > > Where does 32 come from? > >> + >> + __u32 flags; >> +}; >> + >> +#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01 >> +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x02 >> + >> +struct v4l2_h264_dpb_entry { >> + __u64 timestamp; >> + __u16 frame_num; >> + __u16 pic_num; >> + /* Note that field is indicated by v4l2_buffer.field */ >> + __s32 top_field_order_cnt; >> + __s32 bottom_field_order_cnt; >> + __u32 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */ >> +}; >> + >> +struct v4l2_ctrl_h264_decode_param { >> + __u32 num_slices; >> + __u16 idr_pic_flag; >> + __u16 nal_ref_idc; >> + __s32 top_field_order_cnt; >> + __s32 bottom_field_order_cnt; >> + struct v4l2_h264_dpb_entry dpb[16]; > > And 16? > > Nothing special needs to be done for those constants that are based on the > standard. But if it isn't part of the standard but a driver constraint, then > that should be documented, and there should be an idea on how it can be > extended for future drivers without that constraint. > > I think the API should be designed with 4k video in mind. So if some of > these constants would be too small when dealing with 4k (even if the > current HW doesn't support this yet), then these constants would have to > be increased. > > And yes, I know 8k video is starting to appear, but I think it is OK > that additional control(s) would be needed to support 8k. Hmm, 4k (and up) is much more likely to use HEVC. So perhaps designing this for 4k is overkill. Does anyone know if H.264 is used for 4k video at all? If not (or if very rare), then just ignore this. Regards, Hans > >> +}; >> + >> +#endif >> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h >> index d63cf227b0ab..22b6d09c4764 100644 >> --- a/include/media/v4l2-ctrls.h >> +++ b/include/media/v4l2-ctrls.h >> @@ -23,10 +23,11 @@ >> #include <media/media-request.h> >> >> /* >> - * Include the mpeg2 stateless codec compound control definitions. >> + * Include the stateless codec compound control definitions. >> * This will move to the public headers once this API is fully stable. >> */ >> #include <media/mpeg2-ctrls.h> >> +#include <media/h264-ctrls.h> >> >> /* forward references */ >> struct file; >> @@ -49,6 +50,11 @@ struct poll_table_struct; >> * @p_char: Pointer to a string. >> * @p_mpeg2_slice_params: Pointer to a MPEG2 slice parameters structure. >> * @p_mpeg2_quantization: Pointer to a MPEG2 quantization data structure. >> + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps. >> + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps. >> + * @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix. >> + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param. >> + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param. >> * @p: Pointer to a compound value. >> */ >> union v4l2_ctrl_ptr { >> @@ -60,6 +66,11 @@ union v4l2_ctrl_ptr { >> char *p_char; >> struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params; >> struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quantization; >> + struct v4l2_ctrl_h264_sps *p_h264_sps; >> + struct v4l2_ctrl_h264_pps *p_h264_pps; >> + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix; >> + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param; >> + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param; >> void *p; >> }; >> >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h >> index d6eed479c3a6..6fc955926bdb 100644 >> --- a/include/uapi/linux/videodev2.h >> +++ b/include/uapi/linux/videodev2.h >> @@ -645,6 +645,7 @@ struct v4l2_pix_format { >> #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */ >> #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */ >> #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */ >> +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ >> #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */ >> #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */ >> #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */ >> > > Regards, > > Hans > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 15:21 ` Hans Verkuil @ 2019-02-12 13:05 ` Maxime Ripard 2019-02-12 21:17 ` Ezequiel Garcia 0 siblings, 1 reply; 24+ messages in thread From: Maxime Ripard @ 2019-02-12 13:05 UTC (permalink / raw) To: Hans Verkuil Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni, Guenter Roeck [-- Attachment #1: Type: text/plain, Size: 1004 bytes --] Hi, On Mon, Feb 11, 2019 at 04:21:47PM +0100, Hans Verkuil wrote: > > I think the API should be designed with 4k video in mind. So if some of > > these constants would be too small when dealing with 4k (even if the > > current HW doesn't support this yet), then these constants would have to > > be increased. > > > > And yes, I know 8k video is starting to appear, but I think it is OK > > that additional control(s) would be needed to support 8k. > > Hmm, 4k (and up) is much more likely to use HEVC. So perhaps designing this > for 4k is overkill. > > Does anyone know if H.264 is used for 4k video at all? If not (or if very > rare), then just ignore this. I don't know the state of it right now, but until quite recently youtube at least was encoding their 4k videos in both VP9 and H264. They might have moved to h265 since, but considering 4k doesn't seem unreasonable. Maxime -- Maxime Ripard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-12 13:05 ` Maxime Ripard @ 2019-02-12 21:17 ` Ezequiel Garcia 0 siblings, 0 replies; 24+ messages in thread From: Ezequiel Garcia @ 2019-02-12 21:17 UTC (permalink / raw) To: Maxime Ripard, Hans Verkuil Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, linux-sunxi, Thomas Petazzoni, Guenter Roeck On Tue, 2019-02-12 at 14:05 +0100, Maxime Ripard wrote: > Hi, > > On Mon, Feb 11, 2019 at 04:21:47PM +0100, Hans Verkuil wrote: > > > I think the API should be designed with 4k video in mind. So if some of > > > these constants would be too small when dealing with 4k (even if the > > > current HW doesn't support this yet), then these constants would have to > > > be increased. > > > > > > And yes, I know 8k video is starting to appear, but I think it is OK > > > that additional control(s) would be needed to support 8k. > > > > Hmm, 4k (and up) is much more likely to use HEVC. So perhaps designing this > > for 4k is overkill. > > > > Does anyone know if H.264 is used for 4k video at all? If not (or if very > > rare), then just ignore this. > > I don't know the state of it right now, but until quite recently > youtube at least was encoding their 4k videos in both VP9 and > H264. They might have moved to h265 since, but considering 4k doesn't > seem unreasonable. > Just asked the multimedia team here, and they say that 4k videos are in fact widely used for both H264 and VP9 because of the HEVC licensing issues, and because gains in compressions are not so impressive. Regards, Ezequiel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 15:16 ` Hans Verkuil 2019-02-11 15:21 ` Hans Verkuil @ 2019-02-11 17:16 ` Nicolas Dufresne 1 sibling, 0 replies; 24+ messages in thread From: Nicolas Dufresne @ 2019-02-11 17:16 UTC (permalink / raw) To: Hans Verkuil, Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, jenskuske, jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni, Guenter Roeck [-- Attachment #1: Type: text/plain, Size: 38845 bytes --] Le lundi 11 février 2019 à 16:16 +0100, Hans Verkuil a écrit : > Hi Maxime, > > A quick review below. Note that I am no expert on the codec details, so > I leave that to others. I'm mainly concentrating on the structs, flags, etc. > > On 2/11/19 3:39 PM, Maxime Ripard wrote: > > From: Pawel Osciak <posciak@chromium.org> > > > > Stateless video codecs will require both the H264 metadata and slices in > > order to be able to decode frames. > > > > This introduces the definitions for a new pixel format for H264 slices that > > have been parsed, as well as the structures used to pass the metadata from > > the userspace to the kernel. > > > > Co-Developed-by: Maxime Ripard <maxime.ripard@bootlin.com> > > Signed-off-by: Pawel Osciak <posciak@chromium.org> > > Signed-off-by: Guenter Roeck <groeck@chromium.org> > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> > > --- > > Documentation/media/uapi/v4l/biblio.rst | 9 +- > > Documentation/media/uapi/v4l/extended-controls.rst | 530 ++++++++++++++- > > Documentation/media/uapi/v4l/pixfmt-compressed.rst | 20 +- > > Documentation/media/uapi/v4l/vidioc-queryctrl.rst | 30 +- > > Documentation/media/videodev2.h.rst.exceptions | 5 +- > > drivers/media/v4l2-core/v4l2-ctrls.c | 42 +- > > drivers/media/v4l2-core/v4l2-ioctl.c | 1 +- > > include/media/h264-ctrls.h | 179 +++++- > > include/media/v4l2-ctrls.h | 13 +- > > include/uapi/linux/videodev2.h | 1 +- > > 10 files changed, 829 insertions(+), 1 deletion(-) > > create mode 100644 include/media/h264-ctrls.h > > > > diff --git a/Documentation/media/uapi/v4l/biblio.rst b/Documentation/media/uapi/v4l/biblio.rst > > index ec33768c055e..3fc3f7ff338a 100644 > > --- a/Documentation/media/uapi/v4l/biblio.rst > > +++ b/Documentation/media/uapi/v4l/biblio.rst > > @@ -122,6 +122,15 @@ ITU BT.1119 > > > > :author: International Telecommunication Union (http://www.itu.ch) > > > > +.. _h264: > > + > > +ITU H.264 > > +========= > > + > > +:title: ITU-T Recommendation H.264 "Advanced Video Coding for Generic Audiovisual Services" > > + > > +:author: International Telecommunication Union (http://www.itu.ch) > > + > > .. _jfif: > > > > JFIF > > diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst > > index af4273aa5e85..7e306aa0d0a6 100644 > > --- a/Documentation/media/uapi/v4l/extended-controls.rst > > +++ b/Documentation/media/uapi/v4l/extended-controls.rst > > @@ -1703,6 +1703,536 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - > > non-intra-coded frames, in zigzag scanning order. Only relevant for > > non-4:2:0 YUV formats. > > > > +.. _v4l2-mpeg-h264: > > + > > +``V4L2_CID_MPEG_VIDEO_H264_SPS (struct)`` > > + Specifies the sequence parameter set (as extracted from the > > + bitstream) for the associated H264 slice data. This includes the > > + necessary parameters for configuring a stateless hardware decoding > > + pipeline for H264. The bitstream parameters are defined according > > + to :ref:`h264`. Unless there's a specific comment, refer to the > > + specification for the documentation of these fields, section 7.4.2.1.1 > > + "Sequence Parameter Set Data Semantics". > > + > > + .. note:: > > + > > + This compound control is not yet part of the public kernel API and > > + it is expected to change. > > + > > +.. c:type:: v4l2_ctrl_h264_sps > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_ctrl_h264_sps > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u8 > > + - ``profile_idc`` > > + - > > + * - __u8 > > + - ``constraint_set_flags`` > > + - See :ref:`Sequence Parameter Set Constraints Set Flags <h264_sps_constraints_set_flags>` > > + * - __u8 > > + - ``level_idc`` > > + - > > + * - __u8 > > + - ``seq_parameter_set_id`` > > + - > > + * - __u8 > > + - ``chroma_format_idc`` > > + - > > + * - __u8 > > + - ``bit_depth_luma_minus8`` > > + - > > + * - __u8 > > + - ``bit_depth_chroma_minus8`` > > + - > > + * - __u8 > > + - ``log2_max_frame_num_minus4`` > > + - > > + * - __u8 > > + - ``pic_order_cnt_type`` > > + - > > + * - __u8 > > + - ``log2_max_pic_order_cnt_lsb_minus4`` > > + - > > + * - __u8 > > + - ``max_num_ref_frames`` > > + - > > + * - __u8 > > + - ``num_ref_frames_in_pic_order_cnt_cycle`` > > + - > > + * - __s32 > > + - ``offset_for_ref_frame[255]`` > > + - > > + * - __s32 > > + - ``offset_for_non_ref_pic`` > > + - > > + * - __s32 > > + - ``offset_for_top_to_bottom_field`` > > + - > > + * - __u16 > > + - ``pic_width_in_mbs_minus1`` > > + - > > + * - __u16 > > + - ``pic_height_in_map_units_minus1`` > > + - > > + * - __u32 > > + - ``flags`` > > + - See :ref:`Sequence Parameter Set Flags <h264_sps_flags>` > > + > > +.. _h264_sps_constraints_set_flags: > > + > > +``Sequence Parameter Set Constraints Set Flags`` > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - ``V4L2_H264_SPS_CONSTRAINT_SET0_FLAG`` > > + - 0x00000001 > > + - > > + * - ``V4L2_H264_SPS_CONSTRAINT_SET1_FLAG`` > > + - 0x00000002 > > + - > > + * - ``V4L2_H264_SPS_CONSTRAINT_SET2_FLAG`` > > + - 0x00000004 > > + - > > + * - ``V4L2_H264_SPS_CONSTRAINT_SET3_FLAG`` > > + - 0x00000008 > > + - > > + * - ``V4L2_H264_SPS_CONSTRAINT_SET4_FLAG`` > > + - 0x00000010 > > + - > > + * - ``V4L2_H264_SPS_CONSTRAINT_SET5_FLAG`` > > + - 0x00000020 > > + - > > + > > +.. _h264_sps_flags: > > + > > +``Sequence Parameter Set Flags`` > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - ``V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE`` > > + - 0x00000001 > > + - > > + * - ``V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS`` > > + - 0x00000002 > > + - > > + * - ``V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO`` > > + - 0x00000004 > > + - > > + * - ``V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED`` > > + - 0x00000008 > > + - > > + * - ``V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY`` > > + - 0x00000010 > > + - > > + * - ``V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD`` > > + - 0x00000020 > > + - > > + * - ``V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE`` > > + - 0x00000040 > > + - > > + > > +``V4L2_CID_MPEG_VIDEO_H264_PPS (struct)`` > > + Specifies the picture parameter set (as extracted from the > > + bitstream) for the associated H264 slice data. This includes the > > + necessary parameters for configuring a stateless hardware decoding > > + pipeline for H264. The bitstream parameters are defined according > > + to :ref:`h264`. Unless there's a specific comment, refer to the > > + specification for the documentation of these fields, section 7.4.2.2 > > + "Picture Parameter Set RBSP Semantics". > > + > > + .. note:: > > + > > + This compound control is not yet part of the public kernel API and > > + it is expected to change. > > + > > +.. c:type:: v4l2_ctrl_h264_pps > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_ctrl_h264_pps > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u8 > > + - ``pic_parameter_set_id`` > > + - > > + * - __u8 > > + - ``seq_parameter_set_id`` > > + - > > + * - __u8 > > + - ``num_slice_groups_minus1`` > > + - > > + * - __u8 > > + - ``num_ref_idx_l0_default_active_minus1`` > > + - > > + * - __u8 > > + - ``num_ref_idx_l1_default_active_minus1`` > > + - > > + * - __u8 > > + - ``weighted_bipred_idc`` > > + - > > + * - __s8 > > + - ``pic_init_qp_minus26`` > > + - > > + * - __s8 > > + - ``pic_init_qs_minus26`` > > + - > > + * - __s8 > > + - ``chroma_qp_index_offset`` > > + - > > + * - __s8 > > + - ``second_chroma_qp_index_offset`` > > + - > > + * - __u16 > > + - ``flags`` > > + - See :ref:`Picture Parameter Set Flags <h264_pps_flags>` > > + > > +.. _h264_pps_flags: > > + > > +``Picture Parameter Set Flags`` > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - ``V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE`` > > + - 0x00000001 > > + - > > + * - ``V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT`` > > + - 0x00000002 > > + - > > + * - ``V4L2_H264_PPS_FLAG_WEIGHTED_PRED`` > > + - 0x00000004 > > + - > > + * - ``V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT`` > > + - 0x00000008 > > + - > > + * - ``V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED`` > > + - 0x00000010 > > + - > > + * - ``V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT`` > > + - 0x00000020 > > + - > > + * - ``V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE`` > > + - 0x00000040 > > + - > > + * - ``V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT`` > > + - 0x00000080 > > + - > > + > > +``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (struct)`` > > + Specifies the scaling matrix (as extracted from the bitstream) for > > + the associated H264 slice data. The bitstream parameters are > > + defined according to :ref:`h264`. Unless there's a specific > > + comment, refer to the specification for the documentation of these > > + fields, section 7.4.2.1.1.1 "Scaling List Semantics". > > + > > + .. note:: > > + > > + This compound control is not yet part of the public kernel API and > > + it is expected to change. > > + > > +.. c:type:: v4l2_ctrl_h264_scaling_matrix > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_ctrl_h264_scaling_matrix > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u8 > > + - ``scaling_list_4x4[6][16]`` > > + - > > + * - __u8 > > + - ``scaling_list_8x8[6][64]`` > > + - > > + > > +``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (struct)`` > > + Specifies the slice parameters (as extracted from the bitstream) > > + for the associated H264 slice data. This includes the necessary > > + parameters for configuring a stateless hardware decoding pipeline > > + for H264. The bitstream parameters are defined according to > > + :ref:`h264`. Unless there's a specific comment, refer to the > > + specification for the documentation of these fields, section 7.4.3 > > + "Slice Header Semantics". > > + > > + .. note:: > > + > > + This compound control is not yet part of the public kernel API and > > + it is expected to change. > > + > > +.. c:type:: v4l2_ctrl_h264_slice_param > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_ctrl_h264_slice_param > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u32 > > + - ``size`` > > + - > > + * - __u32 > > + - ``header_bit_size`` > > + - > > + * - __u16 > > + - ``first_mb_in_slice`` > > + - > > + * - __u8 > > + - ``slice_type`` > > + - > > + * - __u8 > > + - ``pic_parameter_set_id`` > > + - > > + * - __u8 > > + - ``colour_plane_id`` > > + - > > + * - __u16 > > + - ``frame_num`` > > + - > > + * - __u16 > > + - ``idr_pic_id`` > > + - > > + * - __u16 > > + - ``pic_order_cnt_lsb`` > > + - > > + * - __s32 > > + - ``delta_pic_order_cnt_bottom`` > > + - > > + * - __s32 > > + - ``delta_pic_order_cnt0`` > > + - > > + * - __s32 > > + - ``delta_pic_order_cnt1`` > > + - > > + * - __u8 > > + - ``redundant_pic_cnt`` > > This should be moved to just before 'frame_num' as per the actual struct. > > > + - > > + * - struct :c:type:`v4l2_h264_pred_weight_table` > > + - ``pred_weight_table`` > > + - > > + * - __u32 > > + - ``dec_ref_pic_marking_bit_size`` > > + - > > + * - __u32 > > + - ``pic_order_cnt_bit_size`` > > + - > > + * - __u8 > > + - ``cabac_init_idc`` > > + - > > + * - __s8 > > + - ``slice_qp_delta`` > > + - > > + * - __s8 > > + - ``slice_qs_delta`` > > + - > > + * - __u8 > > + - ``disable_deblocking_filter_idc`` > > + - > > + * - __s8 > > + - ``slice_alpha_c0_offset_div2`` > > + - > > + * - __s8 > > + - ``slice_beta_offset_div2`` > > + - > > + * - __u32 > > + - ``slice_group_change_cycle`` > > + - > > + * - __u8 > > + - ``num_ref_idx_l0_active_minus1`` > > + - > > + * - __u8 > > + - ``num_ref_idx_l1_active_minus1`` > > These two fields should be moved to just before the slice_group_change_cycle > field as per the actual struct. > > > + - > > + * - __u8 > > + - ``ref_pic_list0[32]`` > > + - > > + * - __u8 > > + - ``ref_pic_list1[32]`` > > + - > > + * - __u32 > > + - ``flags`` > > + - See :ref:`Slice Parameter Flags <h264_slice_flags>` > > + > > +.. _h264_slice_flags: > > + > > +``Slice Parameter Set Flags`` > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - ``V4L2_H264_SLICE_FLAG_FIELD_PIC`` > > + - 0x00000001 > > + - > > + * - ``V4L2_H264_SLICE_FLAG_BOTTOM_FIELD`` > > + - 0x00000002 > > + - > > + * - ``V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED`` > > + - 0x00000004 > > + - > > + * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH`` > > + - 0x00000008 > > + - > > + > > +``Prediction Weight Table`` > > + > > + Unless there's a specific comment, refer to the specification for > > + the documentation of these fields, section 7.4.3.2 "Prediction > > + Weight Table Semantics". > > + > > +.. c:type:: v4l2_h264_pred_weight_table > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_h264_pred_weight_table > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u16 > > + - ``luma_log2_weight_denom`` > > + - > > + * - __u16 > > + - ``chroma_log2_weight_denom`` > > + - > > + * - struct :c:type:`v4l2_h264_weight_factors` > > + - ``weight_factors[2]`` > > + - > > + > > +.. c:type:: v4l2_h264_weight_factors > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_h264_weight_factors > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __s8 > > + - ``luma_weight[32]`` > > + - > > + * - __s8 > > + - ``luma_offset[32]`` > > + - > > + * - __s8 > > + - ``chroma_weight[32][2]`` > > + - > > + * - __s8 > > + - ``chroma_offset[32][2]`` > > + - > > + > > +``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)`` > > + Specifies the decode parameters (as extracted from the bitstream) > > + for the associated H264 slice data. This includes the necessary > > + parameters for configuring a stateless hardware decoding pipeline > > + for H264. The bitstream parameters are defined according to > > + :ref:`h264`. Unless there's a specific comment, refer to the > > + specification for the documentation of these fields. > > + > > + .. note:: > > + > > + This compound control is not yet part of the public kernel API and > > + it is expected to change. > > + > > +.. c:type:: v4l2_ctrl_h264_decode_param > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_ctrl_h264_decode_param > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u32 > > + - ``num_slices`` > > + - Number of slices needed to decode the current frame > > + * - __u16 > > + - ``idr_pic_flag`` > > + - Is the picture an IDR picture? > > + * - __u16 > > + - ``nal_ref_idc`` > > + - NAL reference ID value coming from the NAL Unit header > > + * - __s32 > > + - ``top_field_order_cnt`` > > + - Picture Order Count for the coded top field > > + * - __s32 > > + - ``bottom_field_order_cnt`` > > + - Picture Order Count for the coded bottom field > > + * - struct :c:type:`v4l2_h264_dpb_entry` > > + - ``dpb[16]`` > > + - > > + > > +.. c:type:: v4l2_h264_dpb_entry > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: struct v4l2_h264_dpb_entry > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - __u32 > > + - ``tag`` > > + - tag to identify the buffer containing the reference frame > > This is outdated: it's a __u64 timestamp. > > You can copy the struct backward_ref_ts documentation from struct > v4l2_ctrl_mpeg2_slice_params and edit it a bit for H.264. > > > + * - __u16 > > + - ``frame_num`` > > + - > > + * - __u16 > > + - ``pic_num`` > > + - > > + * - __s32 > > + - ``top_field_order_cnt`` > > + - > > + * - __s32 > > + - ``bottom_field_order_cnt`` > > + - > > + * - __u32 > > + - ``flags`` > > + - See :ref:`DPB Entry Flags <h264_dpb_flags>` > > + > > +.. _h264_dpb_flags: > > + > > +``DPB Entries Flags`` > > + > > +.. cssclass:: longtable > > + > > +.. flat-table:: > > + :header-rows: 0 > > + :stub-columns: 0 > > + :widths: 1 1 2 > > + > > + * - ``V4L2_H264_DPB_ENTRY_FLAG_VALID`` > > + - 0x00000001 > > + - > > + * - ``V4L2_H264_DPB_ENTRY_FLAG_ACTIVE`` > > + - 0x00000002 > > + - > > + > > MFC 5.1 MPEG Controls > > --------------------- > > > > diff --git a/Documentation/media/uapi/v4l/pixfmt-compressed.rst b/Documentation/media/uapi/v4l/pixfmt-compressed.rst > > index e4c5e456df59..95905b967693 100644 > > --- a/Documentation/media/uapi/v4l/pixfmt-compressed.rst > > +++ b/Documentation/media/uapi/v4l/pixfmt-compressed.rst > > @@ -52,6 +52,26 @@ Compressed Formats > > - ``V4L2_PIX_FMT_H264_MVC`` > > - 'M264' > > - H264 MVC video elementary stream. > > + * .. _V4L2-PIX-FMT-H264-SLICE: > > + > > + - ``V4L2_PIX_FMT_H264_SLICE`` > > + - 'S264' > > + - H264 parsed slice data, as extracted from the H264 bitstream. > > + This format is adapted for stateless video decoders that > > + implement an H264 pipeline (using the :ref:`codec` and > > + :ref:`media-request-api`). Metadata associated with the frame > > + to decode are required to be passed through the > > + ``V4L2_CID_MPEG_VIDEO_H264_SPS``, > > + ``V4L2_CID_MPEG_VIDEO_H264_PPS``, > > + ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` and > > + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS`` controls and > > + scaling matrices can optionally be specified through the > > + ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX`` control. See the > > + :ref:`associated Codec Control IDs <v4l2-mpeg-h264>`. > > + Exactly one output and one capture buffer must be provided for > > + use with this pixel format. The output buffer must contain the > > + appropriate number of macroblocks to decode a full > > + corresponding frame to the matching capture buffer. > > * .. _V4L2-PIX-FMT-H263: > > > > - ``V4L2_PIX_FMT_H263`` > > diff --git a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > > index f824162d0ea9..bf29dc5b9758 100644 > > --- a/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > > +++ b/Documentation/media/uapi/v4l/vidioc-queryctrl.rst > > @@ -443,6 +443,36 @@ See also the examples in :ref:`control`. > > - n/a > > - A struct :c:type:`v4l2_ctrl_mpeg2_quantization`, containing MPEG-2 > > quantization matrices for stateless video decoders. > > + * - ``V4L2_CTRL_TYPE_H264_SPS`` > > + - n/a > > + - n/a > > + - n/a > > + - A struct :c:type:`v4l2_ctrl_h264_sps`, containing H264 > > + sequence parameters for stateless video decoders. > > + * - ``V4L2_CTRL_TYPE_H264_PPS`` > > + - n/a > > + - n/a > > + - n/a > > + - A struct :c:type:`v4l2_ctrl_h264_pps`, containing H264 > > + picture parameters for stateless video decoders. > > + * - ``V4L2_CTRL_TYPE_H264_SCALING_MATRIX`` > > + - n/a > > + - n/a > > + - n/a > > + - A struct :c:type:`v4l2_ctrl_h264_scaling_matrix`, containing H264 > > + scaling matrices for stateless video decoders. > > + * - ``V4L2_CTRL_TYPE_H264_SLICE_PARAMS`` > > + - n/a > > + - n/a > > + - n/a > > + - A struct :c:type:`v4l2_ctrl_h264_slice_param`, containing H264 > > + slice parameters for stateless video decoders. > > + * - ``V4L2_CTRL_TYPE_H264_DECODE_PARAMS`` > > + - n/a > > + - n/a > > + - n/a > > + - A struct :c:type:`v4l2_ctrl_h264_decode_param`, containing H264 > > + decode parameters for stateless video decoders. > > > > .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}| > > > > diff --git a/Documentation/media/videodev2.h.rst.exceptions b/Documentation/media/videodev2.h.rst.exceptions > > index 64d348e67df9..55cbe324b9fc 100644 > > --- a/Documentation/media/videodev2.h.rst.exceptions > > +++ b/Documentation/media/videodev2.h.rst.exceptions > > @@ -136,6 +136,11 @@ replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type` > > replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type` > > replace symbol V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS :c:type:`v4l2_ctrl_type` > > replace symbol V4L2_CTRL_TYPE_MPEG2_QUANTIZATION :c:type:`v4l2_ctrl_type` > > +replace symbol V4L2_CTRL_TYPE_H264_SPS :c:type:`v4l2_ctrl_type` > > +replace symbol V4L2_CTRL_TYPE_H264_PPS :c:type:`v4l2_ctrl_type` > > +replace symbol V4L2_CTRL_TYPE_H264_SCALING_MATRIX :c:type:`v4l2_ctrl_type` > > +replace symbol V4L2_CTRL_TYPE_H264_SLICE_PARAMS :c:type:`v4l2_ctrl_type` > > +replace symbol V4L2_CTRL_TYPE_H264_DECODE_PARAMS :c:type:`v4l2_ctrl_type` > > > > # V4L2 capability defines > > replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities > > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c > > index e3bd441fa29a..fe9f9447caba 100644 > > --- a/drivers/media/v4l2-core/v4l2-ctrls.c > > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c > > @@ -825,6 +825,11 @@ const char *v4l2_ctrl_get_name(u32 id) > > case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER:return "H264 Number of HC Layers"; > > case V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER_QP: > > return "H264 Set QP Value for HC Layers"; > > + case V4L2_CID_MPEG_VIDEO_H264_SPS: return "H264 Sequence Parameter Set"; > > + case V4L2_CID_MPEG_VIDEO_H264_PPS: return "H264 Picture Parameter Set"; > > + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix"; > > + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters"; > > + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters"; > > case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value"; > > case V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP: return "MPEG4 P-Frame QP Value"; > > case V4L2_CID_MPEG_VIDEO_MPEG4_B_FRAME_QP: return "MPEG4 B-Frame QP Value"; > > @@ -1300,6 +1305,21 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, > > case V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION: > > *type = V4L2_CTRL_TYPE_MPEG2_QUANTIZATION; > > break; > > + case V4L2_CID_MPEG_VIDEO_H264_SPS: > > + *type = V4L2_CTRL_TYPE_H264_SPS; > > + break; > > + case V4L2_CID_MPEG_VIDEO_H264_PPS: > > + *type = V4L2_CTRL_TYPE_H264_PPS; > > + break; > > + case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: > > + *type = V4L2_CTRL_TYPE_H264_SCALING_MATRIX; > > + break; > > + case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: > > + *type = V4L2_CTRL_TYPE_H264_SLICE_PARAMS; > > + break; > > + case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: > > + *type = V4L2_CTRL_TYPE_H264_DECODE_PARAMS; > > + break; > > default: > > *type = V4L2_CTRL_TYPE_INTEGER; > > break; > > @@ -1666,6 +1686,13 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx, > > case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: > > return 0; > > > > + case V4L2_CTRL_TYPE_H264_SPS: > > + case V4L2_CTRL_TYPE_H264_PPS: > > + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: > > + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: > > + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: > > + return 0; > > + > > default: > > return -EINVAL; > > } > > @@ -2246,6 +2273,21 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, > > case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION: > > elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization); > > break; > > + case V4L2_CTRL_TYPE_H264_SPS: > > + elem_size = sizeof(struct v4l2_ctrl_h264_sps); > > + break; > > + case V4L2_CTRL_TYPE_H264_PPS: > > + elem_size = sizeof(struct v4l2_ctrl_h264_pps); > > + break; > > + case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: > > + elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix); > > + break; > > + case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: > > + elem_size = sizeof(struct v4l2_ctrl_h264_slice_param); > > + break; > > + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: > > + elem_size = sizeof(struct v4l2_ctrl_h264_decode_param); > > + break; > > default: > > if (type < V4L2_CTRL_COMPOUND_TYPES) > > elem_size = sizeof(s32); > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > > index 1441a73ce64c..f42d6e7049eb 100644 > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > > @@ -1313,6 +1313,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) > > case V4L2_PIX_FMT_H264: descr = "H.264"; break; > > case V4L2_PIX_FMT_H264_NO_SC: descr = "H.264 (No Start Codes)"; break; > > case V4L2_PIX_FMT_H264_MVC: descr = "H.264 MVC"; break; > > + case V4L2_PIX_FMT_H264_SLICE: descr = "H.264 Parsed Slice Data"; break; > > case V4L2_PIX_FMT_H263: descr = "H.263"; break; > > case V4L2_PIX_FMT_MPEG1: descr = "MPEG-1 ES"; break; > > case V4L2_PIX_FMT_MPEG2: descr = "MPEG-2 ES"; break; > > diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h > > new file mode 100644 > > index 000000000000..7b0563dfa05f > > --- /dev/null > > +++ b/include/media/h264-ctrls.h > > @@ -0,0 +1,179 @@ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > +/* > > + * These are the H.264 state controls for use with stateless H.264 > > + * codec drivers. > > + * > > + * It turns out that these structs are not stable yet and will undergo > > + * more changes. So keep them private until they are stable and ready to > > + * become part of the official public API. > > + */ > > + > > +#ifndef _H264_CTRLS_H_ > > +#define _H264_CTRLS_H_ > > + > > +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383) > > +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384) > > +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385) > > +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+386) > > +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+387) > > + > > +/* enum v4l2_ctrl_type type values */ > > +#define V4L2_CTRL_TYPE_H264_SPS 0x0105 > > +#define V4L2_CTRL_TYPE_H264_PPS 0x0106 > > +#define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0107 > > +#define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0108 > > +#define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0109 > > + > > +#define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01 > > +#define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02 > > +#define V4L2_H264_SPS_CONSTRAINT_SET2_FLAG 0x04 > > +#define V4L2_H264_SPS_CONSTRAINT_SET3_FLAG 0x08 > > +#define V4L2_H264_SPS_CONSTRAINT_SET4_FLAG 0x10 > > +#define V4L2_H264_SPS_CONSTRAINT_SET5_FLAG 0x20 > > + > > +#define V4L2_H264_SPS_FLAG_SEPARATE_COLOUR_PLANE 0x01 > > +#define V4L2_H264_SPS_FLAG_QPPRIME_Y_ZERO_TRANSFORM_BYPASS 0x02 > > +#define V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO 0x04 > > +#define V4L2_H264_SPS_FLAG_GAPS_IN_FRAME_NUM_VALUE_ALLOWED 0x08 > > +#define V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY 0x10 > > +#define V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD 0x20 > > +#define V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE 0x40 > > + > > +struct v4l2_ctrl_h264_sps { > > + __u8 profile_idc; > > + __u8 constraint_set_flags; > > + __u8 level_idc; > > + __u8 seq_parameter_set_id; > > + __u8 chroma_format_idc; > > + __u8 bit_depth_luma_minus8; > > + __u8 bit_depth_chroma_minus8; > > + __u8 log2_max_frame_num_minus4; > > + __u8 pic_order_cnt_type; > > + __u8 log2_max_pic_order_cnt_lsb_minus4; > > + __u8 max_num_ref_frames; > > + __u8 num_ref_frames_in_pic_order_cnt_cycle; > > + __s32 offset_for_ref_frame[255]; > > Where does 255 come from? Should this be a define? this is indexed relative to num_ref_frames_in_pic_order_cnt_cycle (right before), which is defined in 7.4.2.1.1 as being between 0 and 255. > > If this is not based on the standard but instead a driver limitation, then > I think that should be documented. In that case I assume that if more is needed > in the future, then that will be done through additional controls. > > > + __s32 offset_for_non_ref_pic; > > + __s32 offset_for_top_to_bottom_field; > > + __u16 pic_width_in_mbs_minus1; > > + __u16 pic_height_in_map_units_minus1; > > + __u32 flags; > > +}; > > + > > +#define V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE 0x0001 > > +#define V4L2_H264_PPS_FLAG_BOTTOM_FIELD_PIC_ORDER_IN_FRAME_PRESENT 0x0002 > > +#define V4L2_H264_PPS_FLAG_WEIGHTED_PRED 0x0004 > > +#define V4L2_H264_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT 0x0008 > > +#define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010 > > +#define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020 > > +#define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040 > > +#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080 > > + > > +struct v4l2_ctrl_h264_pps { > > + __u8 pic_parameter_set_id; > > + __u8 seq_parameter_set_id; > > + __u8 num_slice_groups_minus1; > > + __u8 num_ref_idx_l0_default_active_minus1; > > + __u8 num_ref_idx_l1_default_active_minus1; > > + __u8 weighted_bipred_idc; > > + __s8 pic_init_qp_minus26; > > + __s8 pic_init_qs_minus26; > > + __s8 chroma_qp_index_offset; > > + __s8 second_chroma_qp_index_offset; > > + __u16 flags; > > +}; > > + > > +struct v4l2_ctrl_h264_scaling_matrix { > > + __u8 scaling_list_4x4[6][16]; > > + __u8 scaling_list_8x8[6][64]; > > +}; > > + > > +struct v4l2_h264_weight_factors { > > + __s8 luma_weight[32]; > > + __s8 luma_offset[32]; > > + __s8 chroma_weight[32][2]; > > + __s8 chroma_offset[32][2]; > > Same question here. As per 7.4.3.1 the second index refer to Cb (0) and 1 Cr(). And the 32 is the maximum number of RefPicList0, as limited by the range of num_ref_idx_l0_active_minus1 (if I read properly). This will never change, so if we add defines it would only be for documentation purpose I think. > > > +}; > > + > > +struct v4l2_h264_pred_weight_table { > > + __u16 luma_log2_weight_denom; > > + __u16 chroma_log2_weight_denom; > > + struct v4l2_h264_weight_factors weight_factors[2]; > > I assume [0] is for luma and [1] is for chroma? Should be documented. Yes. > > > +}; > > + > > +#define V4L2_H264_SLICE_TYPE_P 0 > > +#define V4L2_H264_SLICE_TYPE_B 1 > > +#define V4L2_H264_SLICE_TYPE_I 2 > > +#define V4L2_H264_SLICE_TYPE_SP 3 > > +#define V4L2_H264_SLICE_TYPE_SI 4 > > + > > +#define V4L2_H264_SLICE_FLAG_FIELD_PIC 0x01 > > +#define V4L2_H264_SLICE_FLAG_BOTTOM_FIELD 0x02 > > +#define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04 > > +#define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08 > > + > > +struct v4l2_ctrl_h264_slice_param { > > + /* Size in bytes, including header */ > > + __u32 size; > > + /* Offset in bits to slice_data() from the beginning of this slice. */ > > + __u32 header_bit_size; > > + > > + __u16 first_mb_in_slice; > > + __u8 slice_type; > > + __u8 pic_parameter_set_id; > > + __u8 colour_plane_id; > > + __u8 redundant_pic_cnt; > > + __u16 frame_num; > > + __u16 idr_pic_id; > > + __u16 pic_order_cnt_lsb; > > + __s32 delta_pic_order_cnt_bottom; > > + __s32 delta_pic_order_cnt0; > > + __s32 delta_pic_order_cnt1; > > + > > + struct v4l2_h264_pred_weight_table pred_weight_table; > > + /* Size in bits of dec_ref_pic_marking() syntax element. */ > > + __u32 dec_ref_pic_marking_bit_size; > > + /* Size in bits of pic order count syntax. */ > > + __u32 pic_order_cnt_bit_size; > > + > > + __u8 cabac_init_idc; > > + __s8 slice_qp_delta; > > + __s8 slice_qs_delta; > > + __u8 disable_deblocking_filter_idc; > > + __s8 slice_alpha_c0_offset_div2; > > + __s8 slice_beta_offset_div2; > > + __u8 num_ref_idx_l0_active_minus1; > > + __u8 num_ref_idx_l1_active_minus1; > > + __u32 slice_group_change_cycle; > > + > > + /* Entries on each list are indices > > + * into v4l2_ctrl_h264_decode_param.dpb[]. */ > > + __u8 ref_pic_list0[32]; > > + __u8 ref_pic_list1[32]; > > Where does 32 come from? It's the range of num_ref_idx_l0_active_minus1, > > > + > > + __u32 flags; > > +}; > > + > > +#define V4L2_H264_DPB_ENTRY_FLAG_VALID 0x01 > > +#define V4L2_H264_DPB_ENTRY_FLAG_ACTIVE 0x02 > > + > > +struct v4l2_h264_dpb_entry { > > + __u64 timestamp; > > + __u16 frame_num; > > + __u16 pic_num; > > + /* Note that field is indicated by v4l2_buffer.field */ > > + __s32 top_field_order_cnt; > > + __s32 bottom_field_order_cnt; > > + __u32 flags; /* V4L2_H264_DPB_ENTRY_FLAG_* */ > > +}; > > + > > +struct v4l2_ctrl_h264_decode_param { > > + __u32 num_slices; > > + __u16 idr_pic_flag; > > + __u16 nal_ref_idc; > > + __s32 top_field_order_cnt; > > + __s32 bottom_field_order_cnt; > > + struct v4l2_h264_dpb_entry dpb[16]; > > And 16? H264 DPB size cannot exceed 16, that's well known normally. > > Nothing special needs to be done for those constants that are based on the > standard. But if it isn't part of the standard but a driver constraint, then > that should be documented, and there should be an idea on how it can be > extended for future drivers without that constraint. > > I think the API should be designed with 4k video in mind. So if some of > these constants would be too small when dealing with 4k (even if the > current HW doesn't support this yet), then these constants would have to > be increased. > > And yes, I know 8k video is starting to appear, but I think it is OK > that additional control(s) would be needed to support 8k. > > > +}; > > + > > +#endif > > diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h > > index d63cf227b0ab..22b6d09c4764 100644 > > --- a/include/media/v4l2-ctrls.h > > +++ b/include/media/v4l2-ctrls.h > > @@ -23,10 +23,11 @@ > > #include <media/media-request.h> > > > > /* > > - * Include the mpeg2 stateless codec compound control definitions. > > + * Include the stateless codec compound control definitions. > > * This will move to the public headers once this API is fully stable. > > */ > > #include <media/mpeg2-ctrls.h> > > +#include <media/h264-ctrls.h> > > > > /* forward references */ > > struct file; > > @@ -49,6 +50,11 @@ struct poll_table_struct; > > * @p_char: Pointer to a string. > > * @p_mpeg2_slice_params: Pointer to a MPEG2 slice parameters structure. > > * @p_mpeg2_quantization: Pointer to a MPEG2 quantization data structure. > > + * @p_h264_sps: Pointer to a struct v4l2_ctrl_h264_sps. > > + * @p_h264_pps: Pointer to a struct v4l2_ctrl_h264_pps. > > + * @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix. > > + * @p_h264_slice_param: Pointer to a struct v4l2_ctrl_h264_slice_param. > > + * @p_h264_decode_param: Pointer to a struct v4l2_ctrl_h264_decode_param. > > * @p: Pointer to a compound value. > > */ > > union v4l2_ctrl_ptr { > > @@ -60,6 +66,11 @@ union v4l2_ctrl_ptr { > > char *p_char; > > struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params; > > struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quantization; > > + struct v4l2_ctrl_h264_sps *p_h264_sps; > > + struct v4l2_ctrl_h264_pps *p_h264_pps; > > + struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix; > > + struct v4l2_ctrl_h264_slice_param *p_h264_slice_param; > > + struct v4l2_ctrl_h264_decode_param *p_h264_decode_param; > > void *p; > > }; > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > > index d6eed479c3a6..6fc955926bdb 100644 > > --- a/include/uapi/linux/videodev2.h > > +++ b/include/uapi/linux/videodev2.h > > @@ -645,6 +645,7 @@ struct v4l2_pix_format { > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */ > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */ > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */ > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ > > #define V4L2_PIX_FMT_H263 v4l2_fourcc('H', '2', '6', '3') /* H263 */ > > #define V4L2_PIX_FMT_MPEG1 v4l2_fourcc('M', 'P', 'G', '1') /* MPEG-1 ES */ > > #define V4L2_PIX_FMT_MPEG2 v4l2_fourcc('M', 'P', 'G', '2') /* MPEG-2 ES */ > > > > Regards, > > Hans [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 14:39 ` [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls Maxime Ripard 2019-02-11 15:16 ` Hans Verkuil @ 2019-02-11 17:12 ` Ezequiel Garcia 2019-02-12 10:23 ` Maxime Ripard 2019-02-11 19:53 ` Ezequiel Garcia 2 siblings, 1 reply; 24+ messages in thread From: Ezequiel Garcia @ 2019-02-11 17:12 UTC (permalink / raw) To: nicolas.dufresne, tfiga, posciak, Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, jenskuske, jernej.skrabec, jonas, linux-sunxi, Thomas Petazzoni, Guenter Roeck On Mon, 2019-02-11 at 15:39 +0100, Maxime Ripard wrote: > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index d6eed479c3a6..6fc955926bdb 100644 > --- a/include/uapi/linux/videodev2.h > +++ b/include/uapi/linux/videodev2.h > @@ -645,6 +645,7 @@ struct v4l2_pix_format { > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */ > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */ > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */ > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ Nicolas and I have discussed the pixel format, and came up with the following proposal. Given this format represents H264 parsed slices, without any start code, perhpaps we name it as: V4L2_PIX_FMT_H264_SLICE_RAW Then, we'd also add: V4L2_PIX_FMT_H264_SLICE_ANNEX_B To represent H264 parsed slices with annex B (3- or 4-byte) start code. This one is what the Rockchip VPU driver would expose. Ideas? ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 17:12 ` Ezequiel Garcia @ 2019-02-12 10:23 ` Maxime Ripard 0 siblings, 0 replies; 24+ messages in thread From: Maxime Ripard @ 2019-02-12 10:23 UTC (permalink / raw) To: Ezequiel Garcia Cc: nicolas.dufresne, tfiga, posciak, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, jenskuske, jernej.skrabec, jonas, linux-sunxi, Thomas Petazzoni, Guenter Roeck [-- Attachment #1: Type: text/plain, Size: 1389 bytes --] On Mon, Feb 11, 2019 at 02:12:25PM -0300, Ezequiel Garcia wrote: > On Mon, 2019-02-11 at 15:39 +0100, Maxime Ripard wrote: > > > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > > index d6eed479c3a6..6fc955926bdb 100644 > > --- a/include/uapi/linux/videodev2.h > > +++ b/include/uapi/linux/videodev2.h > > @@ -645,6 +645,7 @@ struct v4l2_pix_format { > > #define V4L2_PIX_FMT_H264 v4l2_fourcc('H', '2', '6', '4') /* H264 with start codes */ > > #define V4L2_PIX_FMT_H264_NO_SC v4l2_fourcc('A', 'V', 'C', '1') /* H264 without start codes */ > > #define V4L2_PIX_FMT_H264_MVC v4l2_fourcc('M', '2', '6', '4') /* H264 MVC */ > > +#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ > > Nicolas and I have discussed the pixel format, and came up with > the following proposal. > > Given this format represents H264 parsed slices, without any start code, > perhpaps we name it as: > > V4L2_PIX_FMT_H264_SLICE_RAW > > Then, we'd also add: > > V4L2_PIX_FMT_H264_SLICE_ANNEX_B > > To represent H264 parsed slices with annex B (3- or 4-byte) start code. > This one is what the Rockchip VPU driver would expose. > > Ideas? I think we discussed that idea already, and I'm all for it. Maxime -- Maxime Ripard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls. 2019-02-11 14:39 ` [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls Maxime Ripard 2019-02-11 15:16 ` Hans Verkuil 2019-02-11 17:12 ` Ezequiel Garcia @ 2019-02-11 19:53 ` Ezequiel Garcia 2 siblings, 0 replies; 24+ messages in thread From: Ezequiel Garcia @ 2019-02-11 19:53 UTC (permalink / raw) To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, linux-sunxi, Thomas Petazzoni, Guenter Roeck On Mon, 2019-02-11 at 15:39 +0100, Maxime Ripard wrote: > From: Pawel Osciak <posciak@chromium.org> > > Stateless video codecs will require both the H264 metadata and slices in > order to be able to decode frames. > > This introduces the definitions for a new pixel format for H264 slices that > have been parsed, as well as the structures used to pass the metadata from > the userspace to the kernel. > > Co-Developed-by: Maxime Ripard <maxime.ripard@bootlin.com> > Signed-off-by: Pawel Osciak <posciak@chromium.org> > Signed-off-by: Guenter Roeck <groeck@chromium.org> > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> [..] > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * These are the H.264 state controls for use with stateless H.264 > + * codec drivers. > + * > + * It turns out that these structs are not stable yet and will undergo > + * more changes. So keep them private until they are stable and ready to > + * become part of the official public API. > + */ > + > +#ifndef _H264_CTRLS_H_ > +#define _H264_CTRLS_H_ > + > +#define V4L2_CID_MPEG_VIDEO_H264_SPS (V4L2_CID_MPEG_BASE+383) > +#define V4L2_CID_MPEG_VIDEO_H264_PPS (V4L2_CID_MPEG_BASE+384) > +#define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+385) > +#define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+386) > +#define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+387) > + Note that these integers (+383, ..) are already in-use. Since the controls are non-public, perhaps we could use upper integers, say +1000 or something? ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-11 14:39 [PATCH v3 0/2] media: cedrus: Add H264 decoding support Maxime Ripard 2019-02-11 14:39 ` [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls Maxime Ripard @ 2019-02-11 14:39 ` Maxime Ripard 2019-02-11 19:21 ` [linux-sunxi] " Jernej Škrabec ` (3 more replies) 2019-02-12 12:50 ` [PATCH v3 0/2] " Tomasz Figa 2 siblings, 4 replies; 24+ messages in thread From: Maxime Ripard @ 2019-02-11 14:39 UTC (permalink / raw) To: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, ezequiel, linux-sunxi, Thomas Petazzoni, Maxime Ripard Introduce some basic H264 decoding support in cedrus. So far, only the baseline profile videos have been tested, and some more advanced features used in higher profiles are not even implemented. Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> --- drivers/staging/media/sunxi/cedrus/Makefile | 3 +- drivers/staging/media/sunxi/cedrus/cedrus.c | 31 +- drivers/staging/media/sunxi/cedrus/cedrus.h | 38 +- drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 15 +- drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 589 +++++++++++++++- drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 4 +- drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 91 ++- drivers/staging/media/sunxi/cedrus/cedrus_video.c | 9 +- 8 files changed, 778 insertions(+), 2 deletions(-) create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_h264.c diff --git a/drivers/staging/media/sunxi/cedrus/Makefile b/drivers/staging/media/sunxi/cedrus/Makefile index e9dc68b7bcb6..aaf141fc58b6 100644 --- a/drivers/staging/media/sunxi/cedrus/Makefile +++ b/drivers/staging/media/sunxi/cedrus/Makefile @@ -1,3 +1,4 @@ obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o -sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o cedrus_mpeg2.o +sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o \ + cedrus_mpeg2.o cedrus_h264.o diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index ff11cbeba205..b275607b8111 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -40,6 +40,36 @@ static const struct cedrus_control cedrus_controls[] = { .codec = CEDRUS_CODEC_MPEG2, .required = false, }, + { + .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS, + .elem_size = sizeof(struct v4l2_ctrl_h264_decode_param), + .codec = CEDRUS_CODEC_H264, + .required = true, + }, + { + .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, + .elem_size = sizeof(struct v4l2_ctrl_h264_slice_param), + .codec = CEDRUS_CODEC_H264, + .required = true, + }, + { + .id = V4L2_CID_MPEG_VIDEO_H264_SPS, + .elem_size = sizeof(struct v4l2_ctrl_h264_sps), + .codec = CEDRUS_CODEC_H264, + .required = true, + }, + { + .id = V4L2_CID_MPEG_VIDEO_H264_PPS, + .elem_size = sizeof(struct v4l2_ctrl_h264_pps), + .codec = CEDRUS_CODEC_H264, + .required = true, + }, + { + .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, + .elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix), + .codec = CEDRUS_CODEC_H264, + .required = true, + }, }; #define CEDRUS_CONTROLS_COUNT ARRAY_SIZE(cedrus_controls) @@ -278,6 +308,7 @@ static int cedrus_probe(struct platform_device *pdev) } dev->dec_ops[CEDRUS_CODEC_MPEG2] = &cedrus_dec_ops_mpeg2; + dev->dec_ops[CEDRUS_CODEC_H264] = &cedrus_dec_ops_h264; mutex_init(&dev->dev_mutex); diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h index 4aedd24a9848..8c64f9a27e9d 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h @@ -30,7 +30,7 @@ enum cedrus_codec { CEDRUS_CODEC_MPEG2, - + CEDRUS_CODEC_H264, CEDRUS_CODEC_LAST, }; @@ -40,6 +40,12 @@ enum cedrus_irq_status { CEDRUS_IRQ_OK, }; +enum cedrus_h264_pic_type { + CEDRUS_H264_PIC_TYPE_FRAME = 0, + CEDRUS_H264_PIC_TYPE_FIELD, + CEDRUS_H264_PIC_TYPE_MBAFF, +}; + struct cedrus_control { u32 id; u32 elem_size; @@ -47,6 +53,14 @@ struct cedrus_control { unsigned char required:1; }; +struct cedrus_h264_run { + const struct v4l2_ctrl_h264_decode_param *decode_param; + const struct v4l2_ctrl_h264_pps *pps; + const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; + const struct v4l2_ctrl_h264_slice_param *slice_param; + const struct v4l2_ctrl_h264_sps *sps; +}; + struct cedrus_mpeg2_run { const struct v4l2_ctrl_mpeg2_slice_params *slice_params; const struct v4l2_ctrl_mpeg2_quantization *quantization; @@ -57,12 +71,20 @@ struct cedrus_run { struct vb2_v4l2_buffer *dst; union { + struct cedrus_h264_run h264; struct cedrus_mpeg2_run mpeg2; }; }; struct cedrus_buffer { struct v4l2_m2m_buffer m2m_buf; + + union { + struct { + unsigned int position; + enum cedrus_h264_pic_type pic_type; + } h264; + } codec; }; struct cedrus_ctx { @@ -77,6 +99,19 @@ struct cedrus_ctx { struct v4l2_ctrl **ctrls; struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME]; + + union { + struct { + void *mv_col_buf; + dma_addr_t mv_col_buf_dma; + ssize_t mv_col_buf_field_size; + ssize_t mv_col_buf_size; + void *pic_info_buf; + dma_addr_t pic_info_buf_dma; + void *neighbor_info_buf; + dma_addr_t neighbor_info_buf_dma; + } h264; + } codec; }; struct cedrus_dec_ops { @@ -118,6 +153,7 @@ struct cedrus_dev { }; extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2; +extern struct cedrus_dec_ops cedrus_dec_ops_h264; static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val) { diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c index 443fb037e1cf..4c33e3b1283f 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c @@ -46,6 +46,21 @@ void cedrus_device_run(void *priv) V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION); break; + case V4L2_PIX_FMT_H264_SLICE: + run.h264.decode_param = cedrus_find_control_data(ctx, + V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS); + run.h264.pps = cedrus_find_control_data(ctx, + V4L2_CID_MPEG_VIDEO_H264_PPS); + run.h264.scaling_matrix = cedrus_find_control_data(ctx, + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); + run.h264.slice_param = cedrus_find_control_data(ctx, + V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS); + run.h264.scaling_matrix = cedrus_find_control_data(ctx, + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); + run.h264.sps = cedrus_find_control_data(ctx, + V4L2_CID_MPEG_VIDEO_H264_SPS); + break; + default: break; } diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c new file mode 100644 index 000000000000..a5c5f13ffecb --- /dev/null +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -0,0 +1,589 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Cedrus VPU driver + * + * Copyright (c) 2013 Jens Kuske <jenskuske@gmail.com> + * Copyright (c) 2018 Bootlin + */ + +#include <linux/types.h> + +#include <media/videobuf2-dma-contig.h> + +#include "cedrus.h" +#include "cedrus_hw.h" +#include "cedrus_regs.h" + +enum cedrus_h264_sram_off { + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE = 0x000, + CEDRUS_SRAM_H264_FRAMEBUFFER_LIST = 0x100, + CEDRUS_SRAM_H264_REF_LIST_0 = 0x190, + CEDRUS_SRAM_H264_REF_LIST_1 = 0x199, + CEDRUS_SRAM_H264_SCALING_LIST_8x8_0 = 0x200, + CEDRUS_SRAM_H264_SCALING_LIST_8x8_1 = 0x210, + CEDRUS_SRAM_H264_SCALING_LIST_4x4 = 0x220, +}; + +struct cedrus_h264_sram_ref_pic { + __le32 top_field_order_cnt; + __le32 bottom_field_order_cnt; + __le32 frame_info; + __le32 luma_ptr; + __le32 chroma_ptr; + __le32 mv_col_top_ptr; + __le32 mv_col_bot_ptr; + __le32 reserved; +} __packed; + +#define CEDRUS_H264_FRAME_NUM 18 + +#define CEDRUS_NEIGHBOR_INFO_BUF_SIZE (16 * SZ_1K) +#define CEDRUS_PIC_INFO_BUF_SIZE (128 * SZ_1K) + +static void cedrus_h264_write_sram(struct cedrus_dev *dev, + enum cedrus_h264_sram_off off, + const void *data, size_t len) +{ + const u32 *buffer = data; + size_t count = DIV_ROUND_UP(len, 4); + + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET, off << 2); + + do { + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, *buffer++); + } while (--count); +} + +static dma_addr_t cedrus_h264_mv_col_buf_addr(struct cedrus_ctx *ctx, + unsigned int position, + unsigned int field) +{ + dma_addr_t addr = ctx->codec.h264.mv_col_buf_dma; + + /* Adjust for the position */ + addr += position * ctx->codec.h264.mv_col_buf_field_size * 2; + + /* Adjust for the field */ + addr += field * ctx->codec.h264.mv_col_buf_field_size; + + return addr; +} + +static void cedrus_fill_ref_pic(struct cedrus_ctx *ctx, + struct cedrus_buffer *buf, + unsigned int top_field_order_cnt, + unsigned int bottom_field_order_cnt, + struct cedrus_h264_sram_ref_pic *pic) +{ + struct vb2_buffer *vbuf = &buf->m2m_buf.vb.vb2_buf; + unsigned int position = buf->codec.h264.position; + + pic->top_field_order_cnt = top_field_order_cnt; + pic->bottom_field_order_cnt = bottom_field_order_cnt; + pic->frame_info = buf->codec.h264.pic_type << 8; + + pic->luma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 0); + pic->chroma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 1); + pic->mv_col_top_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 0); + pic->mv_col_bot_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 1); +} + +static void cedrus_write_frame_list(struct cedrus_ctx *ctx, + struct cedrus_run *run) +{ + struct cedrus_h264_sram_ref_pic pic_list[CEDRUS_H264_FRAME_NUM]; + const struct v4l2_ctrl_h264_decode_param *dec_param = run->h264.decode_param; + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param; + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps; + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; + struct cedrus_buffer *output_buf; + struct cedrus_dev *dev = ctx->dev; + unsigned long used_dpbs = 0; + unsigned int position; + unsigned int output = 0; + unsigned int i; + + memset(pic_list, 0, sizeof(pic_list)); + + for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { + const struct v4l2_h264_dpb_entry *dpb = &dec_param->dpb[i]; + struct cedrus_buffer *cedrus_buf; + int buf_idx; + + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) + continue; + + if (dst_buf->timestamp == dpb->timestamp) + buf_idx = dst_buf->index; + else + buf_idx = vb2_find_timestamp(cap_q, dpb->timestamp, 0); + + if (buf_idx < 0) + continue; + + cedrus_buf = vb2_to_cedrus_buffer(ctx->dst_bufs[buf_idx]); + position = cedrus_buf->codec.h264.position; + used_dpbs |= BIT(position); + + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + continue; + + cedrus_fill_ref_pic(ctx, cedrus_buf, + dpb->top_field_order_cnt, + dpb->bottom_field_order_cnt, + &pic_list[position]); + + output = max(position, output); + } + + position = find_next_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM, + output); + if (position >= CEDRUS_H264_FRAME_NUM) + position = find_first_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM); + + output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf); + output_buf->codec.h264.position = position; + + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FIELD; + else if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD) + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_MBAFF; + else + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FRAME; + + cedrus_fill_ref_pic(ctx, output_buf, + dec_param->top_field_order_cnt, + dec_param->bottom_field_order_cnt, + &pic_list[position]); + + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_FRAMEBUFFER_LIST, + pic_list, sizeof(pic_list)); + + cedrus_write(dev, VE_H264_OUTPUT_FRAME_IDX, position); +} + +#define CEDRUS_MAX_REF_IDX 32 + +static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, + struct cedrus_run *run, + const u8 *ref_list, u8 num_ref, + enum cedrus_h264_sram_off sram) +{ + const struct v4l2_ctrl_h264_decode_param *decode = run->h264.decode_param; + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; + struct cedrus_dev *dev = ctx->dev; + u8 sram_array[CEDRUS_MAX_REF_IDX]; + unsigned int size, i; + + memset(sram_array, 0, sizeof(sram_array)); + + for (i = 0; i < num_ref; i++) { + const struct v4l2_h264_dpb_entry *dpb; + const struct cedrus_buffer *cedrus_buf; + const struct vb2_v4l2_buffer *ref_buf; + unsigned int position; + int buf_idx; + u8 dpb_idx; + + dpb_idx = ref_list[i]; + dpb = &decode->dpb[dpb_idx]; + + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + continue; + + if (dst_buf->timestamp == dpb->timestamp) + buf_idx = dst_buf->index; + else + buf_idx = vb2_find_timestamp(cap_q, dpb->timestamp, 0); + + if (buf_idx < 0) + continue; + + ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[buf_idx]); + cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf); + position = cedrus_buf->codec.h264.position; + + sram_array[i] |= position << 1; + if (ref_buf->field == V4L2_FIELD_BOTTOM) + sram_array[i] |= BIT(0); + } + + size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array)); + cedrus_h264_write_sram(dev, sram, &sram_array, size); +} + +static void cedrus_write_ref_list0(struct cedrus_ctx *ctx, + struct cedrus_run *run) +{ + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param; + + _cedrus_write_ref_list(ctx, run, + slice->ref_pic_list0, + slice->num_ref_idx_l0_active_minus1 + 1, + CEDRUS_SRAM_H264_REF_LIST_0); +} + +static void cedrus_write_ref_list1(struct cedrus_ctx *ctx, + struct cedrus_run *run) +{ + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param; + + _cedrus_write_ref_list(ctx, run, + slice->ref_pic_list1, + slice->num_ref_idx_l1_active_minus1 + 1, + CEDRUS_SRAM_H264_REF_LIST_1); +} + +static void cedrus_write_scaling_lists(struct cedrus_ctx *ctx, + struct cedrus_run *run) +{ + const struct v4l2_ctrl_h264_scaling_matrix *scaling = + run->h264.scaling_matrix; + struct cedrus_dev *dev = ctx->dev; + + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_0, + scaling->scaling_list_8x8[0], + sizeof(scaling->scaling_list_8x8[0])); + + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_1, + scaling->scaling_list_8x8[1], + sizeof(scaling->scaling_list_8x8[1])); + + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_4x4, + scaling->scaling_list_4x4, + sizeof(scaling->scaling_list_4x4)); +} + +static void cedrus_write_pred_weight_table(struct cedrus_ctx *ctx, + struct cedrus_run *run) +{ + const struct v4l2_ctrl_h264_slice_param *slice = + run->h264.slice_param; + const struct v4l2_h264_pred_weight_table *pred_weight = + &slice->pred_weight_table; + struct cedrus_dev *dev = ctx->dev; + int i, j, k; + + cedrus_write(dev, VE_H264_SHS_WP, + ((pred_weight->chroma_log2_weight_denom & 0xf) << 4) | + ((pred_weight->luma_log2_weight_denom & 0xf) << 0)); + + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET, + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE << 2); + + for (i = 0; i < ARRAY_SIZE(pred_weight->weight_factors); i++) { + const struct v4l2_h264_weight_factors *factors = + &pred_weight->weight_factors[i]; + + for (j = 0; j < ARRAY_SIZE(factors->luma_weight); j++) { + u32 val; + + val = ((factors->luma_offset[j] & 0x1ff) << 16) | + (factors->luma_weight[j] & 0x1ff); + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val); + } + + for (j = 0; j < ARRAY_SIZE(factors->chroma_weight); j++) { + for (k = 0; k < ARRAY_SIZE(factors->chroma_weight[0]); k++) { + u32 val; + + val = ((factors->chroma_offset[j][k] & 0x1ff) << 16) | + (factors->chroma_weight[j][k] & 0x1ff); + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val); + } + } + } +} + +static void cedrus_set_params(struct cedrus_ctx *ctx, + struct cedrus_run *run) +{ + const struct v4l2_ctrl_h264_decode_param *decode = run->h264.decode_param; + const struct v4l2_ctrl_h264_slice_param *slice = run->h264.slice_param; + const struct v4l2_ctrl_h264_pps *pps = run->h264.pps; + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps; + struct vb2_buffer *src_buf = &run->src->vb2_buf; + struct cedrus_dev *dev = ctx->dev; + dma_addr_t src_buf_addr; + u32 offset = slice->header_bit_size; + u32 len = (slice->size * 8) - offset; + u32 reg; + + cedrus_write(dev, VE_H264_VLD_LEN, len); + cedrus_write(dev, VE_H264_VLD_OFFSET, offset); + + src_buf_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0); + cedrus_write(dev, VE_H264_VLD_END, + src_buf_addr + vb2_get_plane_payload(src_buf, 0)); + cedrus_write(dev, VE_H264_VLD_ADDR, + VE_H264_VLD_ADDR_VAL(src_buf_addr) | + VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID | + VE_H264_VLD_ADDR_LAST); + + /* + * FIXME: Since the bitstream parsing is done in software, and + * in userspace, this shouldn't be needed anymore. But it + * turns out that removing it breaks the decoding process, + * without any clear indication why. + */ + cedrus_write(dev, VE_H264_TRIGGER_TYPE, + VE_H264_TRIGGER_TYPE_INIT_SWDEC); + + if (((pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) && + (slice->slice_type == V4L2_H264_SLICE_TYPE_P || + slice->slice_type == V4L2_H264_SLICE_TYPE_SP)) || + (pps->weighted_bipred_idc == 1 && + slice->slice_type == V4L2_H264_SLICE_TYPE_B)) + cedrus_write_pred_weight_table(ctx, run); + + if ((slice->slice_type == V4L2_H264_SLICE_TYPE_P) || + (slice->slice_type == V4L2_H264_SLICE_TYPE_SP) || + (slice->slice_type == V4L2_H264_SLICE_TYPE_B)) + cedrus_write_ref_list0(ctx, run); + + if (slice->slice_type == V4L2_H264_SLICE_TYPE_B) + cedrus_write_ref_list1(ctx, run); + + // picture parameters + reg = 0; + /* + * FIXME: the kernel headers are allowing the default value to + * be passed, but the libva doesn't give us that. + */ + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 10; + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 5; + reg |= (pps->weighted_bipred_idc & 0x3) << 2; + if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE) + reg |= VE_H264_PPS_ENTROPY_CODING_MODE; + if (pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) + reg |= VE_H264_PPS_WEIGHTED_PRED; + if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED) + reg |= VE_H264_PPS_CONSTRAINED_INTRA_PRED; + if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE) + reg |= VE_H264_PPS_TRANSFORM_8X8_MODE; + cedrus_write(dev, VE_H264_PPS, reg); + + // sequence parameters + reg = 0; + reg |= (sps->chroma_format_idc & 0x7) << 19; + reg |= (sps->pic_width_in_mbs_minus1 & 0xff) << 8; + reg |= sps->pic_height_in_map_units_minus1 & 0xff; + if (sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) + reg |= VE_H264_SPS_MBS_ONLY; + if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD) + reg |= VE_H264_SPS_MB_ADAPTIVE_FRAME_FIELD; + if (sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE) + reg |= VE_H264_SPS_DIRECT_8X8_INFERENCE; + cedrus_write(dev, VE_H264_SPS, reg); + + // slice parameters + reg = 0; + reg |= decode->nal_ref_idc ? BIT(12) : 0; + reg |= (slice->slice_type & 0xf) << 8; + reg |= slice->cabac_init_idc & 0x3; + reg |= VE_H264_SHS_FIRST_SLICE_IN_PIC; + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) + reg |= VE_H264_SHS_FIELD_PIC; + if (slice->flags & V4L2_H264_SLICE_FLAG_BOTTOM_FIELD) + reg |= VE_H264_SHS_BOTTOM_FIELD; + if (slice->flags & V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED) + reg |= VE_H264_SHS_DIRECT_SPATIAL_MV_PRED; + cedrus_write(dev, VE_H264_SHS, reg); + + reg = 0; + reg |= VE_H264_SHS2_NUM_REF_IDX_ACTIVE_OVRD; + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 24; + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 16; + reg |= (slice->disable_deblocking_filter_idc & 0x3) << 8; + reg |= (slice->slice_alpha_c0_offset_div2 & 0xf) << 4; + reg |= slice->slice_beta_offset_div2 & 0xf; + cedrus_write(dev, VE_H264_SHS2, reg); + + reg = 0; + /* + * FIXME: This bit tells the video engine to use the default + * quantization matrices. This will obviously need to be + * changed to support the profiles supporting custom + * quantization matrices. + */ + reg |= VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT; + reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16; + reg |= (pps->chroma_qp_index_offset & 0x3f) << 8; + reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f; + cedrus_write(dev, VE_H264_SHS_QP, reg); + + // clear status flags + cedrus_write(dev, VE_H264_STATUS, cedrus_read(dev, VE_H264_STATUS)); + + // enable int + reg = cedrus_read(dev, VE_H264_CTRL); + cedrus_write(dev, VE_H264_CTRL, reg | + VE_H264_CTRL_SLICE_DECODE_INT | + VE_H264_CTRL_DECODE_ERR_INT | + VE_H264_CTRL_VLD_DATA_REQ_INT); +} + +static enum cedrus_irq_status +cedrus_h264_irq_status(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + u32 reg = cedrus_read(dev, VE_H264_STATUS); + + if (reg & (VE_H264_STATUS_DECODE_ERR_INT | + VE_H264_STATUS_VLD_DATA_REQ_INT)) + return CEDRUS_IRQ_ERROR; + + if (reg & VE_H264_CTRL_SLICE_DECODE_INT) + return CEDRUS_IRQ_OK; + + return CEDRUS_IRQ_NONE; +} + +static void cedrus_h264_irq_clear(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + + cedrus_write(dev, VE_H264_STATUS, + VE_H264_STATUS_INT_MASK); +} + +static void cedrus_h264_irq_disable(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + u32 reg = cedrus_read(dev, VE_H264_CTRL); + + cedrus_write(dev, VE_H264_CTRL, + reg & ~VE_H264_CTRL_INT_MASK); +} + +static void cedrus_h264_setup(struct cedrus_ctx *ctx, + struct cedrus_run *run) +{ + struct cedrus_dev *dev = ctx->dev; + + cedrus_engine_enable(dev, CEDRUS_CODEC_H264); + + cedrus_write(dev, VE_H264_SDROT_CTRL, 0); + cedrus_write(dev, VE_H264_EXTRA_BUFFER1, + ctx->codec.h264.pic_info_buf_dma); + cedrus_write(dev, VE_H264_EXTRA_BUFFER2, + ctx->codec.h264.neighbor_info_buf_dma); + + cedrus_write_scaling_lists(ctx, run); + cedrus_write_frame_list(ctx, run); + + cedrus_set_params(ctx, run); +} + +static int cedrus_h264_start(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + unsigned int field_size; + unsigned int mv_col_size; + int ret; + + /* + * FIXME: It seems that the H6 cedarX code is using a formula + * here based on the size of the frame, while all the older + * code is using a fixed size, so that might need to be + * changed at some point. + */ + ctx->codec.h264.pic_info_buf = + dma_alloc_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, + &ctx->codec.h264.pic_info_buf_dma, + GFP_KERNEL); + if (!ctx->codec.h264.pic_info_buf) + return -ENOMEM; + + /* + * That buffer is supposed to be 16kiB in size, and be aligned + * on 16kiB as well. However, dma_alloc_coherent provides the + * guarantee that we'll have a CPU and DMA address aligned on + * the smallest page order that is greater to the requested + * size, so we don't have to overallocate. + */ + ctx->codec.h264.neighbor_info_buf = + dma_alloc_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, + &ctx->codec.h264.neighbor_info_buf_dma, + GFP_KERNEL); + if (!ctx->codec.h264.neighbor_info_buf) { + ret = -ENOMEM; + goto err_pic_buf; + } + + field_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) * + DIV_ROUND_UP(ctx->src_fmt.height, 16) * 16; + + /* + * FIXME: This is actually conditional to + * V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE not being set, we + * might have to rework this if memory efficiency ever is + * something we need to work on. + */ + field_size = field_size * 2; + + /* + * FIXME: This is actually conditional to + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY not being set, we might + * have to rework this if memory efficiency ever is something + * we need to work on. + */ + field_size = field_size * 2; + ctx->codec.h264.mv_col_buf_field_size = field_size; + + mv_col_size = field_size * 2 * CEDRUS_H264_FRAME_NUM; + ctx->codec.h264.mv_col_buf_size = mv_col_size; + ctx->codec.h264.mv_col_buf = dma_alloc_coherent(dev->dev, + ctx->codec.h264.mv_col_buf_size, + &ctx->codec.h264.mv_col_buf_dma, + GFP_KERNEL); + if (!ctx->codec.h264.mv_col_buf) { + ret = -ENOMEM; + goto err_neighbor_buf; + } + + return 0; + +err_neighbor_buf: + dma_free_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, + ctx->codec.h264.neighbor_info_buf, + ctx->codec.h264.neighbor_info_buf_dma); + +err_pic_buf: + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, + ctx->codec.h264.pic_info_buf, + ctx->codec.h264.pic_info_buf_dma); + return ret; +} + +static void cedrus_h264_stop(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + + dma_free_coherent(dev->dev, ctx->codec.h264.mv_col_buf_size, + ctx->codec.h264.mv_col_buf, + ctx->codec.h264.mv_col_buf_dma); + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, + ctx->codec.h264.pic_info_buf, + ctx->codec.h264.pic_info_buf_dma); +} + +static void cedrus_h264_trigger(struct cedrus_ctx *ctx) +{ + struct cedrus_dev *dev = ctx->dev; + + cedrus_write(dev, VE_H264_TRIGGER_TYPE, + VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE); +} + +struct cedrus_dec_ops cedrus_dec_ops_h264 = { + .irq_clear = cedrus_h264_irq_clear, + .irq_disable = cedrus_h264_irq_disable, + .irq_status = cedrus_h264_irq_status, + .setup = cedrus_h264_setup, + .start = cedrus_h264_start, + .stop = cedrus_h264_stop, + .trigger = cedrus_h264_trigger, +}; diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c index 300339fee1bc..1e11f44adaf8 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c @@ -46,6 +46,10 @@ int cedrus_engine_enable(struct cedrus_dev *dev, enum cedrus_codec codec) reg |= VE_MODE_DEC_MPEG; break; + case CEDRUS_CODEC_H264: + reg |= VE_MODE_DEC_H264; + break; + default: return -EINVAL; } diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h index de2d6b6f64bf..3e9931416e45 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h @@ -232,4 +232,95 @@ #define VE_DEC_MPEG_ROT_LUMA (VE_ENGINE_DEC_MPEG + 0xcc) #define VE_DEC_MPEG_ROT_CHROMA (VE_ENGINE_DEC_MPEG + 0xd0) +#define VE_H264_SPS 0x200 +#define VE_H264_SPS_MBS_ONLY BIT(18) +#define VE_H264_SPS_MB_ADAPTIVE_FRAME_FIELD BIT(17) +#define VE_H264_SPS_DIRECT_8X8_INFERENCE BIT(16) + +#define VE_H264_PPS 0x204 +#define VE_H264_PPS_ENTROPY_CODING_MODE BIT(15) +#define VE_H264_PPS_WEIGHTED_PRED BIT(4) +#define VE_H264_PPS_CONSTRAINED_INTRA_PRED BIT(1) +#define VE_H264_PPS_TRANSFORM_8X8_MODE BIT(0) + +#define VE_H264_SHS 0x208 +#define VE_H264_SHS_FIRST_SLICE_IN_PIC BIT(5) +#define VE_H264_SHS_FIELD_PIC BIT(4) +#define VE_H264_SHS_BOTTOM_FIELD BIT(3) +#define VE_H264_SHS_DIRECT_SPATIAL_MV_PRED BIT(2) + +#define VE_H264_SHS2 0x20c +#define VE_H264_SHS2_NUM_REF_IDX_ACTIVE_OVRD BIT(12) + +#define VE_H264_SHS_WP 0x210 + +#define VE_H264_SHS_QP 0x21c +#define VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT BIT(24) + +#define VE_H264_CTRL 0x220 +#define VE_H264_CTRL_VLD_DATA_REQ_INT BIT(2) +#define VE_H264_CTRL_DECODE_ERR_INT BIT(1) +#define VE_H264_CTRL_SLICE_DECODE_INT BIT(0) + +#define VE_H264_CTRL_INT_MASK (VE_H264_CTRL_VLD_DATA_REQ_INT | \ + VE_H264_CTRL_DECODE_ERR_INT | \ + VE_H264_CTRL_SLICE_DECODE_INT) + +#define VE_H264_TRIGGER_TYPE 0x224 +#define VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE (8 << 0) +#define VE_H264_TRIGGER_TYPE_INIT_SWDEC (7 << 0) + +#define VE_H264_STATUS 0x228 +#define VE_H264_STATUS_VLD_DATA_REQ_INT VE_H264_CTRL_VLD_DATA_REQ_INT +#define VE_H264_STATUS_DECODE_ERR_INT VE_H264_CTRL_DECODE_ERR_INT +#define VE_H264_STATUS_SLICE_DECODE_INT VE_H264_CTRL_SLICE_DECODE_INT + +#define VE_H264_STATUS_INT_MASK VE_H264_CTRL_INT_MASK + +#define VE_H264_CUR_MB_NUM 0x22c + +#define VE_H264_VLD_ADDR 0x230 +#define VE_H264_VLD_ADDR_FIRST BIT(30) +#define VE_H264_VLD_ADDR_LAST BIT(29) +#define VE_H264_VLD_ADDR_VALID BIT(28) +#define VE_H264_VLD_ADDR_VAL(x) (((x) & 0x0ffffff0) | ((x) >> 28)) + +#define VE_H264_VLD_OFFSET 0x234 +#define VE_H264_VLD_LEN 0x238 +#define VE_H264_VLD_END 0x23c +#define VE_H264_SDROT_CTRL 0x240 +#define VE_H264_OUTPUT_FRAME_IDX 0x24c +#define VE_H264_EXTRA_BUFFER1 0x250 +#define VE_H264_EXTRA_BUFFER2 0x254 +#define VE_H264_BASIC_BITS 0x2dc +#define VE_AVC_SRAM_PORT_OFFSET 0x2e0 +#define VE_AVC_SRAM_PORT_DATA 0x2e4 + +#define VE_ISP_INPUT_SIZE 0xa00 +#define VE_ISP_INPUT_STRIDE 0xa04 +#define VE_ISP_CTRL 0xa08 +#define VE_ISP_INPUT_LUMA 0xa78 +#define VE_ISP_INPUT_CHROMA 0xa7c + +#define VE_AVC_PARAM 0xb04 +#define VE_AVC_QP 0xb08 +#define VE_AVC_MOTION_EST 0xb10 +#define VE_AVC_CTRL 0xb14 +#define VE_AVC_TRIGGER 0xb18 +#define VE_AVC_STATUS 0xb1c +#define VE_AVC_BASIC_BITS 0xb20 +#define VE_AVC_UNK_BUF 0xb60 +#define VE_AVC_VLE_ADDR 0xb80 +#define VE_AVC_VLE_END 0xb84 +#define VE_AVC_VLE_OFFSET 0xb88 +#define VE_AVC_VLE_MAX 0xb8c +#define VE_AVC_VLE_LENGTH 0xb90 +#define VE_AVC_REF_LUMA 0xba0 +#define VE_AVC_REF_CHROMA 0xba4 +#define VE_AVC_REC_LUMA 0xbb0 +#define VE_AVC_REC_CHROMA 0xbb4 +#define VE_AVC_REF_SLUMA 0xbb8 +#define VE_AVC_REC_SLUMA 0xbbc +#define VE_AVC_MB_INFO 0xbc0 + #endif diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c index 8721b4a7d496..bca025505b46 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c @@ -38,6 +38,10 @@ static struct cedrus_format cedrus_formats[] = { .directions = CEDRUS_DECODE_SRC, }, { + .pixelformat = V4L2_PIX_FMT_H264_SLICE, + .directions = CEDRUS_DECODE_SRC, + }, + { .pixelformat = V4L2_PIX_FMT_SUNXI_TILED_NV12, .directions = CEDRUS_DECODE_DST, }, @@ -100,6 +104,7 @@ static void cedrus_prepare_format(struct v4l2_pix_format *pix_fmt) switch (pix_fmt->pixelformat) { case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_H264_SLICE: /* Zero bytes per line for encoded source. */ bytesperline = 0; @@ -446,6 +451,10 @@ static int cedrus_start_streaming(struct vb2_queue *vq, unsigned int count) ctx->current_codec = CEDRUS_CODEC_MPEG2; break; + case V4L2_PIX_FMT_H264_SLICE: + ctx->current_codec = CEDRUS_CODEC_H264; + break; + default: return -EINVAL; } -- git-series 0.9.1 ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [linux-sunxi] [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-11 14:39 ` [PATCH v3 2/2] media: cedrus: Add H264 decoding support Maxime Ripard @ 2019-02-11 19:21 ` Jernej Škrabec 2019-02-12 10:43 ` Maxime Ripard 2019-02-11 19:48 ` Ezequiel Garcia ` (2 subsequent siblings) 3 siblings, 1 reply; 24+ messages in thread From: Jernej Škrabec @ 2019-02-11 19:21 UTC (permalink / raw) To: linux-sunxi, maxime.ripard Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jonas, ezequiel, Thomas Petazzoni Hi Maxime, I spotted few minor issues, but otherwise it looks very well. I'll do detailed review at a later time. Dne ponedeljek, 11. februar 2019 ob 15:39:03 CET je Maxime Ripard napisal(a): > Introduce some basic H264 decoding support in cedrus. So far, only the > baseline profile videos have been tested, and some more advanced features > used in higher profiles are not even implemented. > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> > --- > drivers/staging/media/sunxi/cedrus/Makefile | 3 +- > drivers/staging/media/sunxi/cedrus/cedrus.c | 31 +- > drivers/staging/media/sunxi/cedrus/cedrus.h | 38 +- > drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 15 +- > drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 589 +++++++++++++++- > drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 4 +- > drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 91 ++- > drivers/staging/media/sunxi/cedrus/cedrus_video.c | 9 +- > 8 files changed, 778 insertions(+), 2 deletions(-) > create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_h264.c > > diff --git a/drivers/staging/media/sunxi/cedrus/Makefile > b/drivers/staging/media/sunxi/cedrus/Makefile index > e9dc68b7bcb6..aaf141fc58b6 100644 > --- a/drivers/staging/media/sunxi/cedrus/Makefile > +++ b/drivers/staging/media/sunxi/cedrus/Makefile > @@ -1,3 +1,4 @@ > obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o > > -sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o > cedrus_mpeg2.o +sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o > cedrus_dec.o \ + cedrus_mpeg2.o cedrus_h264.o > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c > b/drivers/staging/media/sunxi/cedrus/cedrus.c index > ff11cbeba205..b275607b8111 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c > @@ -40,6 +40,36 @@ static const struct cedrus_control cedrus_controls[] = { > .codec = CEDRUS_CODEC_MPEG2, > .required = false, > }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_decode_param), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_slice_param), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SPS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_sps), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_PPS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_pps), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, > + .elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > }; > > #define CEDRUS_CONTROLS_COUNT ARRAY_SIZE(cedrus_controls) > @@ -278,6 +308,7 @@ static int cedrus_probe(struct platform_device *pdev) > } > > dev->dec_ops[CEDRUS_CODEC_MPEG2] = &cedrus_dec_ops_mpeg2; > + dev->dec_ops[CEDRUS_CODEC_H264] = &cedrus_dec_ops_h264; > > mutex_init(&dev->dev_mutex); > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h > b/drivers/staging/media/sunxi/cedrus/cedrus.h index > 4aedd24a9848..8c64f9a27e9d 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus.h > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h > @@ -30,7 +30,7 @@ > > enum cedrus_codec { > CEDRUS_CODEC_MPEG2, > - > + CEDRUS_CODEC_H264, > CEDRUS_CODEC_LAST, > }; > > @@ -40,6 +40,12 @@ enum cedrus_irq_status { > CEDRUS_IRQ_OK, > }; > > +enum cedrus_h264_pic_type { > + CEDRUS_H264_PIC_TYPE_FRAME = 0, > + CEDRUS_H264_PIC_TYPE_FIELD, > + CEDRUS_H264_PIC_TYPE_MBAFF, > +}; > + > struct cedrus_control { > u32 id; > u32 elem_size; > @@ -47,6 +53,14 @@ struct cedrus_control { > unsigned char required:1; > }; > > +struct cedrus_h264_run { > + const struct v4l2_ctrl_h264_decode_param *decode_param; > + const struct v4l2_ctrl_h264_pps *pps; > + const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; > + const struct v4l2_ctrl_h264_slice_param *slice_param; > + const struct v4l2_ctrl_h264_sps *sps; > +}; > + > struct cedrus_mpeg2_run { > const struct v4l2_ctrl_mpeg2_slice_params *slice_params; > const struct v4l2_ctrl_mpeg2_quantization *quantization; > @@ -57,12 +71,20 @@ struct cedrus_run { > struct vb2_v4l2_buffer *dst; > > union { > + struct cedrus_h264_run h264; > struct cedrus_mpeg2_run mpeg2; > }; > }; > > struct cedrus_buffer { > struct v4l2_m2m_buffer m2m_buf; > + > + union { > + struct { > + unsigned int position; > + enum cedrus_h264_pic_type pic_type; > + } h264; > + } codec; > }; > > struct cedrus_ctx { > @@ -77,6 +99,19 @@ struct cedrus_ctx { > struct v4l2_ctrl **ctrls; > > struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME]; > + > + union { > + struct { > + void *mv_col_buf; > + dma_addr_t mv_col_buf_dma; > + ssize_t mv_col_buf_field_size; > + ssize_t mv_col_buf_size; > + void *pic_info_buf; > + dma_addr_t pic_info_buf_dma; > + void *neighbor_info_buf; > + dma_addr_t neighbor_info_buf_dma; > + } h264; > + } codec; > }; > > struct cedrus_dec_ops { > @@ -118,6 +153,7 @@ struct cedrus_dev { > }; > > extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2; > +extern struct cedrus_dec_ops cedrus_dec_ops_h264; > > static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val) > { > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c index > 443fb037e1cf..4c33e3b1283f 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > @@ -46,6 +46,21 @@ void cedrus_device_run(void *priv) > V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION); > break; > > + case V4L2_PIX_FMT_H264_SLICE: > + run.h264.decode_param = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS); > + run.h264.pps = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_PPS); > + run.h264.scaling_matrix = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); > + run.h264.slice_param = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS); > + run.h264.scaling_matrix = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); > + run.h264.sps = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SPS); > + break; > + > default: > break; > } > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c > b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c new file mode 100644 > index 000000000000..a5c5f13ffecb > --- /dev/null > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c > @@ -0,0 +1,589 @@ > +// SPDX-License-Identifier: GPL-2.0-or-later > +/* > + * Cedrus VPU driver > + * > + * Copyright (c) 2013 Jens Kuske <jenskuske@gmail.com> > + * Copyright (c) 2018 Bootlin > + */ > + > +#include <linux/types.h> > + > +#include <media/videobuf2-dma-contig.h> > + > +#include "cedrus.h" > +#include "cedrus_hw.h" > +#include "cedrus_regs.h" > + > +enum cedrus_h264_sram_off { > + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE = 0x000, > + CEDRUS_SRAM_H264_FRAMEBUFFER_LIST = 0x100, > + CEDRUS_SRAM_H264_REF_LIST_0 = 0x190, > + CEDRUS_SRAM_H264_REF_LIST_1 = 0x199, > + CEDRUS_SRAM_H264_SCALING_LIST_8x8_0 = 0x200, > + CEDRUS_SRAM_H264_SCALING_LIST_8x8_1 = 0x210, > + CEDRUS_SRAM_H264_SCALING_LIST_4x4 = 0x220, > +}; > + > +struct cedrus_h264_sram_ref_pic { > + __le32 top_field_order_cnt; > + __le32 bottom_field_order_cnt; > + __le32 frame_info; > + __le32 luma_ptr; > + __le32 chroma_ptr; > + __le32 mv_col_top_ptr; > + __le32 mv_col_bot_ptr; > + __le32 reserved; > +} __packed; > + > +#define CEDRUS_H264_FRAME_NUM 18 > + > +#define CEDRUS_NEIGHBOR_INFO_BUF_SIZE (16 * SZ_1K) > +#define CEDRUS_PIC_INFO_BUF_SIZE (128 * SZ_1K) > + > +static void cedrus_h264_write_sram(struct cedrus_dev *dev, > + enum cedrus_h264_sram_off off, > + const void *data, size_t len) > +{ > + const u32 *buffer = data; > + size_t count = DIV_ROUND_UP(len, 4); > + > + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET, off << 2); > + > + do { > + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, *buffer++); > + } while (--count); > +} > + > +static dma_addr_t cedrus_h264_mv_col_buf_addr(struct cedrus_ctx *ctx, > + unsigned int position, > + unsigned int field) > +{ > + dma_addr_t addr = ctx->codec.h264.mv_col_buf_dma; > + > + /* Adjust for the position */ > + addr += position * ctx->codec.h264.mv_col_buf_field_size * 2; > + > + /* Adjust for the field */ > + addr += field * ctx->codec.h264.mv_col_buf_field_size; > + > + return addr; > +} > + > +static void cedrus_fill_ref_pic(struct cedrus_ctx *ctx, > + struct cedrus_buffer *buf, > + unsigned int top_field_order_cnt, > + unsigned int bottom_field_order_cnt, > + struct cedrus_h264_sram_ref_pic *pic) > +{ > + struct vb2_buffer *vbuf = &buf->m2m_buf.vb.vb2_buf; > + unsigned int position = buf->codec.h264.position; > + > + pic->top_field_order_cnt = top_field_order_cnt; > + pic->bottom_field_order_cnt = bottom_field_order_cnt; > + pic->frame_info = buf->codec.h264.pic_type << 8; > + > + pic->luma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 0); > + pic->chroma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 1); > + pic->mv_col_top_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 0); > + pic->mv_col_bot_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 1); > +} > + > +static void cedrus_write_frame_list(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + struct cedrus_h264_sram_ref_pic pic_list[CEDRUS_H264_FRAME_NUM]; > + const struct v4l2_ctrl_h264_decode_param *dec_param = > run->h264.decode_param; + const struct v4l2_ctrl_h264_slice_param *slice = > run->h264.slice_param; + const struct v4l2_ctrl_h264_sps *sps = > run->h264.sps; > + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; > + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; > + struct cedrus_buffer *output_buf; > + struct cedrus_dev *dev = ctx->dev; > + unsigned long used_dpbs = 0; > + unsigned int position; > + unsigned int output = 0; > + unsigned int i; > + > + memset(pic_list, 0, sizeof(pic_list)); > + > + for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { > + const struct v4l2_h264_dpb_entry *dpb = &dec_param- >dpb[i]; > + struct cedrus_buffer *cedrus_buf; > + int buf_idx; > + > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) > + continue; > + > + if (dst_buf->timestamp == dpb->timestamp) > + buf_idx = dst_buf->index; > + else > + buf_idx = vb2_find_timestamp(cap_q, dpb- >timestamp, 0); Above "if" is no longer needed with current media-next branch. Hans relaxed conditions in vb2_find_timestamp(). > + > + if (buf_idx < 0) > + continue; > + > + cedrus_buf = vb2_to_cedrus_buffer(ctx- >dst_bufs[buf_idx]); > + position = cedrus_buf->codec.h264.position; > + used_dpbs |= BIT(position); > + > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) > + continue; > + > + cedrus_fill_ref_pic(ctx, cedrus_buf, > + dpb->top_field_order_cnt, > + dpb->bottom_field_order_cnt, > + &pic_list[position]); > + > + output = max(position, output); > + } > + > + position = find_next_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM, > + output); > + if (position >= CEDRUS_H264_FRAME_NUM) > + position = find_first_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM); > + > + output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf); > + output_buf->codec.h264.position = position; > + > + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) > + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FIELD; > + else if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD) > + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_MBAFF; > + else > + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FRAME; > + > + cedrus_fill_ref_pic(ctx, output_buf, > + dec_param->top_field_order_cnt, > + dec_param->bottom_field_order_cnt, > + &pic_list[position]); > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_FRAMEBUFFER_LIST, > + pic_list, sizeof(pic_list)); > + > + cedrus_write(dev, VE_H264_OUTPUT_FRAME_IDX, position); > +} > + > +#define CEDRUS_MAX_REF_IDX 32 > + > +static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, > + struct cedrus_run *run, > + const u8 *ref_list, u8 num_ref, > + enum cedrus_h264_sram_off sram) > +{ > + const struct v4l2_ctrl_h264_decode_param *decode = run- >h264.decode_param; > + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; > + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; > + struct cedrus_dev *dev = ctx->dev; > + u8 sram_array[CEDRUS_MAX_REF_IDX]; > + unsigned int size, i; > + > + memset(sram_array, 0, sizeof(sram_array)); > + > + for (i = 0; i < num_ref; i++) { > + const struct v4l2_h264_dpb_entry *dpb; > + const struct cedrus_buffer *cedrus_buf; > + const struct vb2_v4l2_buffer *ref_buf; > + unsigned int position; > + int buf_idx; > + u8 dpb_idx; > + > + dpb_idx = ref_list[i]; > + dpb = &decode->dpb[dpb_idx]; > + > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) > + continue; > + > + if (dst_buf->timestamp == dpb->timestamp) > + buf_idx = dst_buf->index; > + else > + buf_idx = vb2_find_timestamp(cap_q, dpb- >timestamp, 0); Ditto. > + > + if (buf_idx < 0) > + continue; > + > + ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[buf_idx]); > + cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf); > + position = cedrus_buf->codec.h264.position; > + > + sram_array[i] |= position << 1; > + if (ref_buf->field == V4L2_FIELD_BOTTOM) > + sram_array[i] |= BIT(0); > + } > + > + size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array)); > + cedrus_h264_write_sram(dev, sram, &sram_array, size); > +} > + > +static void cedrus_write_ref_list0(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_slice_param *slice = run- >h264.slice_param; > + > + _cedrus_write_ref_list(ctx, run, > + slice->ref_pic_list0, > + slice->num_ref_idx_l0_active_minus1 + 1, > + CEDRUS_SRAM_H264_REF_LIST_0); > +} > + > +static void cedrus_write_ref_list1(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_slice_param *slice = run- >h264.slice_param; > + > + _cedrus_write_ref_list(ctx, run, > + slice->ref_pic_list1, > + slice->num_ref_idx_l1_active_minus1 + 1, > + CEDRUS_SRAM_H264_REF_LIST_1); > +} > + > +static void cedrus_write_scaling_lists(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_scaling_matrix *scaling = > + run->h264.scaling_matrix; > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_0, > + scaling->scaling_list_8x8[0], > + sizeof(scaling->scaling_list_8x8[0])); > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_1, > + scaling->scaling_list_8x8[1], > + sizeof(scaling->scaling_list_8x8[1])); > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_4x4, > + scaling->scaling_list_4x4, > + sizeof(scaling->scaling_list_4x4)); > +} > + > +static void cedrus_write_pred_weight_table(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_slice_param *slice = > + run->h264.slice_param; > + const struct v4l2_h264_pred_weight_table *pred_weight = > + &slice->pred_weight_table; > + struct cedrus_dev *dev = ctx->dev; > + int i, j, k; > + > + cedrus_write(dev, VE_H264_SHS_WP, > + ((pred_weight->chroma_log2_weight_denom & 0xf) << 4) | > + ((pred_weight->luma_log2_weight_denom & 0xf) << 0)); > + > + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET, > + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE << 2); > + > + for (i = 0; i < ARRAY_SIZE(pred_weight->weight_factors); i++) { > + const struct v4l2_h264_weight_factors *factors = > + &pred_weight->weight_factors[i]; > + > + for (j = 0; j < ARRAY_SIZE(factors->luma_weight); j++) { > + u32 val; > + > + val = ((factors->luma_offset[j] & 0x1ff) << 16) | > + (factors->luma_weight[j] & 0x1ff); > + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val); > + } > + > + for (j = 0; j < ARRAY_SIZE(factors->chroma_weight); j++) { > + for (k = 0; k < ARRAY_SIZE(factors- >chroma_weight[0]); k++) { > + u32 val; > + > + val = ((factors->chroma_offset[j] [k] & 0x1ff) << 16) | > + (factors- >chroma_weight[j][k] & 0x1ff); > + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val); > + } > + } > + } > +} > + > +static void cedrus_set_params(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_decode_param *decode = run- >h264.decode_param; > + const struct v4l2_ctrl_h264_slice_param *slice = run- >h264.slice_param; > + const struct v4l2_ctrl_h264_pps *pps = run->h264.pps; > + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps; > + struct vb2_buffer *src_buf = &run->src->vb2_buf; > + struct cedrus_dev *dev = ctx->dev; > + dma_addr_t src_buf_addr; > + u32 offset = slice->header_bit_size; > + u32 len = (slice->size * 8) - offset; > + u32 reg; > + > + cedrus_write(dev, VE_H264_VLD_LEN, len); > + cedrus_write(dev, VE_H264_VLD_OFFSET, offset); > + > + src_buf_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0); > + cedrus_write(dev, VE_H264_VLD_END, > + src_buf_addr + vb2_get_plane_payload(src_buf, 0)); > + cedrus_write(dev, VE_H264_VLD_ADDR, > + VE_H264_VLD_ADDR_VAL(src_buf_addr) | > + VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID | > + VE_H264_VLD_ADDR_LAST); > + > + /* > + * FIXME: Since the bitstream parsing is done in software, and > + * in userspace, this shouldn't be needed anymore. But it > + * turns out that removing it breaks the decoding process, > + * without any clear indication why. > + */ > + cedrus_write(dev, VE_H264_TRIGGER_TYPE, > + VE_H264_TRIGGER_TYPE_INIT_SWDEC); > + > + if (((pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) && > + (slice->slice_type == V4L2_H264_SLICE_TYPE_P || > + slice->slice_type == V4L2_H264_SLICE_TYPE_SP)) || > + (pps->weighted_bipred_idc == 1 && > + slice->slice_type == V4L2_H264_SLICE_TYPE_B)) > + cedrus_write_pred_weight_table(ctx, run); > + > + if ((slice->slice_type == V4L2_H264_SLICE_TYPE_P) || > + (slice->slice_type == V4L2_H264_SLICE_TYPE_SP) || > + (slice->slice_type == V4L2_H264_SLICE_TYPE_B)) > + cedrus_write_ref_list0(ctx, run); > + > + if (slice->slice_type == V4L2_H264_SLICE_TYPE_B) > + cedrus_write_ref_list1(ctx, run); > + > + // picture parameters > + reg = 0; > + /* > + * FIXME: the kernel headers are allowing the default value to > + * be passed, but the libva doesn't give us that. > + */ > + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 10; > + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 5; > + reg |= (pps->weighted_bipred_idc & 0x3) << 2; > + if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE) > + reg |= VE_H264_PPS_ENTROPY_CODING_MODE; > + if (pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) > + reg |= VE_H264_PPS_WEIGHTED_PRED; > + if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED) > + reg |= VE_H264_PPS_CONSTRAINED_INTRA_PRED; > + if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE) > + reg |= VE_H264_PPS_TRANSFORM_8X8_MODE; > + cedrus_write(dev, VE_H264_PPS, reg); > + > + // sequence parameters > + reg = 0; > + reg |= (sps->chroma_format_idc & 0x7) << 19; > + reg |= (sps->pic_width_in_mbs_minus1 & 0xff) << 8; > + reg |= sps->pic_height_in_map_units_minus1 & 0xff; > + if (sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) > + reg |= VE_H264_SPS_MBS_ONLY; > + if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD) > + reg |= VE_H264_SPS_MB_ADAPTIVE_FRAME_FIELD; > + if (sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE) > + reg |= VE_H264_SPS_DIRECT_8X8_INFERENCE; > + cedrus_write(dev, VE_H264_SPS, reg); > + > + // slice parameters > + reg = 0; > + reg |= decode->nal_ref_idc ? BIT(12) : 0; > + reg |= (slice->slice_type & 0xf) << 8; > + reg |= slice->cabac_init_idc & 0x3; > + reg |= VE_H264_SHS_FIRST_SLICE_IN_PIC; > + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) > + reg |= VE_H264_SHS_FIELD_PIC; > + if (slice->flags & V4L2_H264_SLICE_FLAG_BOTTOM_FIELD) > + reg |= VE_H264_SHS_BOTTOM_FIELD; > + if (slice->flags & V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED) > + reg |= VE_H264_SHS_DIRECT_SPATIAL_MV_PRED; > + cedrus_write(dev, VE_H264_SHS, reg); > + > + reg = 0; > + reg |= VE_H264_SHS2_NUM_REF_IDX_ACTIVE_OVRD; > + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 24; > + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 16; > + reg |= (slice->disable_deblocking_filter_idc & 0x3) << 8; > + reg |= (slice->slice_alpha_c0_offset_div2 & 0xf) << 4; > + reg |= slice->slice_beta_offset_div2 & 0xf; > + cedrus_write(dev, VE_H264_SHS2, reg); > + > + reg = 0; > + /* > + * FIXME: This bit tells the video engine to use the default > + * quantization matrices. This will obviously need to be > + * changed to support the profiles supporting custom > + * quantization matrices. > + */ > + reg |= VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT; This flag should not be needed anymore. From what I see, you correctly set scaling matrix every time. Best regards, Jernej > + reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16; > + reg |= (pps->chroma_qp_index_offset & 0x3f) << 8; > + reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f; > + cedrus_write(dev, VE_H264_SHS_QP, reg); > + > + // clear status flags > + cedrus_write(dev, VE_H264_STATUS, cedrus_read(dev, VE_H264_STATUS)); > + > + // enable int > + reg = cedrus_read(dev, VE_H264_CTRL); > + cedrus_write(dev, VE_H264_CTRL, reg | > + VE_H264_CTRL_SLICE_DECODE_INT | > + VE_H264_CTRL_DECODE_ERR_INT | > + VE_H264_CTRL_VLD_DATA_REQ_INT); > +} > + > +static enum cedrus_irq_status > +cedrus_h264_irq_status(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + u32 reg = cedrus_read(dev, VE_H264_STATUS); > + > + if (reg & (VE_H264_STATUS_DECODE_ERR_INT | > + VE_H264_STATUS_VLD_DATA_REQ_INT)) > + return CEDRUS_IRQ_ERROR; > + > + if (reg & VE_H264_CTRL_SLICE_DECODE_INT) > + return CEDRUS_IRQ_OK; > + > + return CEDRUS_IRQ_NONE; > +} > + > +static void cedrus_h264_irq_clear(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_write(dev, VE_H264_STATUS, > + VE_H264_STATUS_INT_MASK); > +} > + > +static void cedrus_h264_irq_disable(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + u32 reg = cedrus_read(dev, VE_H264_CTRL); > + > + cedrus_write(dev, VE_H264_CTRL, > + reg & ~VE_H264_CTRL_INT_MASK); > +} > + > +static void cedrus_h264_setup(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_engine_enable(dev, CEDRUS_CODEC_H264); > + > + cedrus_write(dev, VE_H264_SDROT_CTRL, 0); > + cedrus_write(dev, VE_H264_EXTRA_BUFFER1, > + ctx->codec.h264.pic_info_buf_dma); > + cedrus_write(dev, VE_H264_EXTRA_BUFFER2, > + ctx->codec.h264.neighbor_info_buf_dma); > + > + cedrus_write_scaling_lists(ctx, run); > + cedrus_write_frame_list(ctx, run); > + > + cedrus_set_params(ctx, run); > +} > + > +static int cedrus_h264_start(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + unsigned int field_size; > + unsigned int mv_col_size; > + int ret; > + > + /* > + * FIXME: It seems that the H6 cedarX code is using a formula > + * here based on the size of the frame, while all the older > + * code is using a fixed size, so that might need to be > + * changed at some point. > + */ > + ctx->codec.h264.pic_info_buf = > + dma_alloc_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, > + &ctx- >codec.h264.pic_info_buf_dma, > + GFP_KERNEL); > + if (!ctx->codec.h264.pic_info_buf) > + return -ENOMEM; > + > + /* > + * That buffer is supposed to be 16kiB in size, and be aligned > + * on 16kiB as well. However, dma_alloc_coherent provides the > + * guarantee that we'll have a CPU and DMA address aligned on > + * the smallest page order that is greater to the requested > + * size, so we don't have to overallocate. > + */ > + ctx->codec.h264.neighbor_info_buf = > + dma_alloc_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, > + &ctx- >codec.h264.neighbor_info_buf_dma, > + GFP_KERNEL); > + if (!ctx->codec.h264.neighbor_info_buf) { > + ret = -ENOMEM; > + goto err_pic_buf; > + } > + > + field_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) * > + DIV_ROUND_UP(ctx->src_fmt.height, 16) * 16; > + > + /* > + * FIXME: This is actually conditional to > + * V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE not being set, we > + * might have to rework this if memory efficiency ever is > + * something we need to work on. > + */ > + field_size = field_size * 2; > + > + /* > + * FIXME: This is actually conditional to > + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY not being set, we might > + * have to rework this if memory efficiency ever is something > + * we need to work on. > + */ > + field_size = field_size * 2; > + ctx->codec.h264.mv_col_buf_field_size = field_size; > + > + mv_col_size = field_size * 2 * CEDRUS_H264_FRAME_NUM; > + ctx->codec.h264.mv_col_buf_size = mv_col_size; > + ctx->codec.h264.mv_col_buf = dma_alloc_coherent(dev->dev, > + ctx->codec.h264.mv_col_buf_size, > + &ctx->codec.h264.mv_col_buf_dma, > + GFP_KERNEL); > + if (!ctx->codec.h264.mv_col_buf) { > + ret = -ENOMEM; > + goto err_neighbor_buf; > + } > + > + return 0; > + > +err_neighbor_buf: > + dma_free_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, > + ctx->codec.h264.neighbor_info_buf, > + ctx->codec.h264.neighbor_info_buf_dma); > + > +err_pic_buf: > + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, > + ctx->codec.h264.pic_info_buf, > + ctx->codec.h264.pic_info_buf_dma); > + return ret; > +} > + > +static void cedrus_h264_stop(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + dma_free_coherent(dev->dev, ctx->codec.h264.mv_col_buf_size, > + ctx->codec.h264.mv_col_buf, > + ctx->codec.h264.mv_col_buf_dma); > + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, > + ctx->codec.h264.pic_info_buf, > + ctx->codec.h264.pic_info_buf_dma); > +} > + > +static void cedrus_h264_trigger(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_write(dev, VE_H264_TRIGGER_TYPE, > + VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE); > +} > + > +struct cedrus_dec_ops cedrus_dec_ops_h264 = { > + .irq_clear = cedrus_h264_irq_clear, > + .irq_disable = cedrus_h264_irq_disable, > + .irq_status = cedrus_h264_irq_status, > + .setup = cedrus_h264_setup, > + .start = cedrus_h264_start, > + .stop = cedrus_h264_stop, > + .trigger = cedrus_h264_trigger, > +}; > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c index > 300339fee1bc..1e11f44adaf8 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c > @@ -46,6 +46,10 @@ int cedrus_engine_enable(struct cedrus_dev *dev, enum > cedrus_codec codec) reg |= VE_MODE_DEC_MPEG; > break; > > + case CEDRUS_CODEC_H264: > + reg |= VE_MODE_DEC_H264; > + break; > + > default: > return -EINVAL; > } > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h index > de2d6b6f64bf..3e9931416e45 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_regs.h > @@ -232,4 +232,95 @@ > #define VE_DEC_MPEG_ROT_LUMA (VE_ENGINE_DEC_MPEG + 0xcc) > #define VE_DEC_MPEG_ROT_CHROMA (VE_ENGINE_DEC_MPEG + 0xd0) > > +#define VE_H264_SPS 0x200 > +#define VE_H264_SPS_MBS_ONLY BIT(18) > +#define VE_H264_SPS_MB_ADAPTIVE_FRAME_FIELD BIT(17) > +#define VE_H264_SPS_DIRECT_8X8_INFERENCE BIT(16) > + > +#define VE_H264_PPS 0x204 > +#define VE_H264_PPS_ENTROPY_CODING_MODE BIT(15) > +#define VE_H264_PPS_WEIGHTED_PRED BIT(4) > +#define VE_H264_PPS_CONSTRAINED_INTRA_PRED BIT(1) > +#define VE_H264_PPS_TRANSFORM_8X8_MODE BIT(0) > + > +#define VE_H264_SHS 0x208 > +#define VE_H264_SHS_FIRST_SLICE_IN_PIC BIT(5) > +#define VE_H264_SHS_FIELD_PIC BIT(4) > +#define VE_H264_SHS_BOTTOM_FIELD BIT(3) > +#define VE_H264_SHS_DIRECT_SPATIAL_MV_PRED BIT(2) > + > +#define VE_H264_SHS2 0x20c > +#define VE_H264_SHS2_NUM_REF_IDX_ACTIVE_OVRD BIT(12) > + > +#define VE_H264_SHS_WP 0x210 > + > +#define VE_H264_SHS_QP 0x21c > +#define VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT BIT(24) > + > +#define VE_H264_CTRL 0x220 > +#define VE_H264_CTRL_VLD_DATA_REQ_INT BIT(2) > +#define VE_H264_CTRL_DECODE_ERR_INT BIT(1) > +#define VE_H264_CTRL_SLICE_DECODE_INT BIT(0) > + > +#define VE_H264_CTRL_INT_MASK (VE_H264_CTRL_VLD_DATA_REQ_INT | \ > + VE_H264_CTRL_DECODE_ERR_INT | \ > + VE_H264_CTRL_SLICE_DECODE_INT) > + > +#define VE_H264_TRIGGER_TYPE 0x224 > +#define VE_H264_TRIGGER_TYPE_AVC_SLICE_DECODE (8 << 0) > +#define VE_H264_TRIGGER_TYPE_INIT_SWDEC (7 << 0) > + > +#define VE_H264_STATUS 0x228 > +#define VE_H264_STATUS_VLD_DATA_REQ_INT VE_H264_CTRL_VLD_DATA_REQ_INT > +#define VE_H264_STATUS_DECODE_ERR_INT VE_H264_CTRL_DECODE_ERR_INT > +#define VE_H264_STATUS_SLICE_DECODE_INT VE_H264_CTRL_SLICE_DECODE_INT > + > +#define VE_H264_STATUS_INT_MASK VE_H264_CTRL_INT_MASK > + > +#define VE_H264_CUR_MB_NUM 0x22c > + > +#define VE_H264_VLD_ADDR 0x230 > +#define VE_H264_VLD_ADDR_FIRST BIT(30) > +#define VE_H264_VLD_ADDR_LAST BIT(29) > +#define VE_H264_VLD_ADDR_VALID BIT(28) > +#define VE_H264_VLD_ADDR_VAL(x) (((x) & 0x0ffffff0) | ((x) >> 28)) > + > +#define VE_H264_VLD_OFFSET 0x234 > +#define VE_H264_VLD_LEN 0x238 > +#define VE_H264_VLD_END 0x23c > +#define VE_H264_SDROT_CTRL 0x240 > +#define VE_H264_OUTPUT_FRAME_IDX 0x24c > +#define VE_H264_EXTRA_BUFFER1 0x250 > +#define VE_H264_EXTRA_BUFFER2 0x254 > +#define VE_H264_BASIC_BITS 0x2dc > +#define VE_AVC_SRAM_PORT_OFFSET 0x2e0 > +#define VE_AVC_SRAM_PORT_DATA 0x2e4 > + > +#define VE_ISP_INPUT_SIZE 0xa00 > +#define VE_ISP_INPUT_STRIDE 0xa04 > +#define VE_ISP_CTRL 0xa08 > +#define VE_ISP_INPUT_LUMA 0xa78 > +#define VE_ISP_INPUT_CHROMA 0xa7c > + > +#define VE_AVC_PARAM 0xb04 > +#define VE_AVC_QP 0xb08 > +#define VE_AVC_MOTION_EST 0xb10 > +#define VE_AVC_CTRL 0xb14 > +#define VE_AVC_TRIGGER 0xb18 > +#define VE_AVC_STATUS 0xb1c > +#define VE_AVC_BASIC_BITS 0xb20 > +#define VE_AVC_UNK_BUF 0xb60 > +#define VE_AVC_VLE_ADDR 0xb80 > +#define VE_AVC_VLE_END 0xb84 > +#define VE_AVC_VLE_OFFSET 0xb88 > +#define VE_AVC_VLE_MAX 0xb8c > +#define VE_AVC_VLE_LENGTH 0xb90 > +#define VE_AVC_REF_LUMA 0xba0 > +#define VE_AVC_REF_CHROMA 0xba4 > +#define VE_AVC_REC_LUMA 0xbb0 > +#define VE_AVC_REC_CHROMA 0xbb4 > +#define VE_AVC_REF_SLUMA 0xbb8 > +#define VE_AVC_REC_SLUMA 0xbbc > +#define VE_AVC_MB_INFO 0xbc0 > + > #endif > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c > b/drivers/staging/media/sunxi/cedrus/cedrus_video.c index > 8721b4a7d496..bca025505b46 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c > @@ -38,6 +38,10 @@ static struct cedrus_format cedrus_formats[] = { > .directions = CEDRUS_DECODE_SRC, > }, > { > + .pixelformat = V4L2_PIX_FMT_H264_SLICE, > + .directions = CEDRUS_DECODE_SRC, > + }, > + { > .pixelformat = V4L2_PIX_FMT_SUNXI_TILED_NV12, > .directions = CEDRUS_DECODE_DST, > }, > @@ -100,6 +104,7 @@ static void cedrus_prepare_format(struct v4l2_pix_format > *pix_fmt) > > switch (pix_fmt->pixelformat) { > case V4L2_PIX_FMT_MPEG2_SLICE: > + case V4L2_PIX_FMT_H264_SLICE: > /* Zero bytes per line for encoded source. */ > bytesperline = 0; > > @@ -446,6 +451,10 @@ static int cedrus_start_streaming(struct vb2_queue *vq, > unsigned int count) ctx->current_codec = CEDRUS_CODEC_MPEG2; > break; > > + case V4L2_PIX_FMT_H264_SLICE: > + ctx->current_codec = CEDRUS_CODEC_H264; > + break; > + > default: > return -EINVAL; > } ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [linux-sunxi] [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-11 19:21 ` [linux-sunxi] " Jernej Škrabec @ 2019-02-12 10:43 ` Maxime Ripard 2019-02-12 17:46 ` Jernej Škrabec 0 siblings, 1 reply; 24+ messages in thread From: Maxime Ripard @ 2019-02-12 10:43 UTC (permalink / raw) To: Jernej Škrabec Cc: linux-sunxi, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jonas, ezequiel, Thomas Petazzoni [-- Attachment #1: Type: text/plain, Size: 765 bytes --] Hi, On Mon, Feb 11, 2019 at 08:21:31PM +0100, Jernej Škrabec wrote: > > + reg = 0; > > + /* > > + * FIXME: This bit tells the video engine to use the default > > + * quantization matrices. This will obviously need to be > > + * changed to support the profiles supporting custom > > + * quantization matrices. > > + */ > > + reg |= VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT; > > This flag should not be needed anymore. From what I see, you correctly set > scaling matrix every time. The scaling matrix control is optional, so I guess we should protect that by a check on whether that control has been set or not. What do you think? Thanks! Maxime -- Maxime Ripard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 228 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [linux-sunxi] [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-12 10:43 ` Maxime Ripard @ 2019-02-12 17:46 ` Jernej Škrabec 0 siblings, 0 replies; 24+ messages in thread From: Jernej Škrabec @ 2019-02-12 17:46 UTC (permalink / raw) To: linux-sunxi, maxime.ripard Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jonas, ezequiel, Thomas Petazzoni Dne torek, 12. februar 2019 ob 11:43:14 CET je Maxime Ripard napisal(a): > Hi, > > On Mon, Feb 11, 2019 at 08:21:31PM +0100, Jernej Škrabec wrote: > > > + reg = 0; > > > + /* > > > + * FIXME: This bit tells the video engine to use the default > > > + * quantization matrices. This will obviously need to be > > > + * changed to support the profiles supporting custom > > > + * quantization matrices. > > > + */ > > > + reg |= VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT; > > > > This flag should not be needed anymore. From what I see, you correctly set > > scaling matrix every time. > > The scaling matrix control is optional, so I guess we should protect > that by a check on whether that control has been set or not. What do > you think? Is it? Consider following snippet from your patch: <snip> { .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, .elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix), .codec = CEDRUS_CODEC_H264, .required = true, }, <snip> Doesn't "required = true" mean that it's mandatory? But yes, if it's optional, then you should add a check. Best regards, Jernej ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-11 14:39 ` [PATCH v3 2/2] media: cedrus: Add H264 decoding support Maxime Ripard 2019-02-11 19:21 ` [linux-sunxi] " Jernej Škrabec @ 2019-02-11 19:48 ` Ezequiel Garcia 2019-02-12 12:47 ` Maxime Ripard 2019-02-14 20:42 ` Jernej Škrabec 2019-02-14 21:27 ` Jernej Škrabec 3 siblings, 1 reply; 24+ messages in thread From: Ezequiel Garcia @ 2019-02-11 19:48 UTC (permalink / raw) To: Maxime Ripard, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart Cc: tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, linux-sunxi, Thomas Petazzoni On Mon, 2019-02-11 at 15:39 +0100, Maxime Ripard wrote: > Introduce some basic H264 decoding support in cedrus. So far, only the > baseline profile videos have been tested, and some more advanced features > used in higher profiles are not even implemented. > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> [..] > +static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, > + struct cedrus_run *run, > + const u8 *ref_list, u8 num_ref, > + enum cedrus_h264_sram_off sram) > +{ > + const struct v4l2_ctrl_h264_decode_param *decode = run->h264.decode_param; > + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; > + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; > + struct cedrus_dev *dev = ctx->dev; > + u8 sram_array[CEDRUS_MAX_REF_IDX]; > + unsigned int size, i; > + > + memset(sram_array, 0, sizeof(sram_array)); > + > + for (i = 0; i < num_ref; i++) { > + const struct v4l2_h264_dpb_entry *dpb; > + const struct cedrus_buffer *cedrus_buf; > + const struct vb2_v4l2_buffer *ref_buf; > + unsigned int position; > + int buf_idx; > + u8 dpb_idx; > + > + dpb_idx = ref_list[i]; > + dpb = &decode->dpb[dpb_idx]; > + > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) > + continue; > + > + if (dst_buf->timestamp == dpb->timestamp) > + buf_idx = dst_buf->index; > + else > + buf_idx = vb2_find_timestamp(cap_q, dpb->timestamp, 0); > + > + if (buf_idx < 0) > + continue; > + > + ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[buf_idx]); > + cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf); > + position = cedrus_buf->codec.h264.position; > + > + sram_array[i] |= position << 1; > + if (ref_buf->field == V4L2_FIELD_BOTTOM) > + sram_array[i] |= BIT(0); > + } > + > + size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array)); Perhaps s/unsigned int/size_t, so the arguments to min() have the same type? Otherwise, I got this warning: /home/zeta/repos/linux/media_tree/drivers/staging/media/sunxi/cedrus/cedrus_h264.c: In function ‘_cedrus_write_ref_list’: /home/zeta/repos/linux/media_tree/include/linux/kernel.h:846:29: warning: comparison of distinct pointer types lacks a cast (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) Regards, Ez ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-11 19:48 ` Ezequiel Garcia @ 2019-02-12 12:47 ` Maxime Ripard 2019-02-12 17:40 ` Jernej Škrabec 0 siblings, 1 reply; 24+ messages in thread From: Maxime Ripard @ 2019-02-12 12:47 UTC (permalink / raw) To: Ezequiel Garcia Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jernej.skrabec, jonas, linux-sunxi, Thomas Petazzoni On Mon, Feb 11, 2019 at 04:48:17PM -0300, Ezequiel Garcia wrote: > On Mon, 2019-02-11 at 15:39 +0100, Maxime Ripard wrote: > > Introduce some basic H264 decoding support in cedrus. So far, only the > > baseline profile videos have been tested, and some more advanced features > > used in higher profiles are not even implemented. > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> > [..] > > > > +static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, > > + struct cedrus_run *run, > > + const u8 *ref_list, u8 num_ref, > > + enum cedrus_h264_sram_off sram) > > +{ > > + const struct v4l2_ctrl_h264_decode_param *decode = run->h264.decode_param; > > + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; > > + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; > > + struct cedrus_dev *dev = ctx->dev; > > + u8 sram_array[CEDRUS_MAX_REF_IDX]; > > + unsigned int size, i; > > + > > + memset(sram_array, 0, sizeof(sram_array)); > > + > > + for (i = 0; i < num_ref; i++) { > > + const struct v4l2_h264_dpb_entry *dpb; > > + const struct cedrus_buffer *cedrus_buf; > > + const struct vb2_v4l2_buffer *ref_buf; > > + unsigned int position; > > + int buf_idx; > > + u8 dpb_idx; > > + > > + dpb_idx = ref_list[i]; > > + dpb = &decode->dpb[dpb_idx]; > > + > > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) > > + continue; > > + > > + if (dst_buf->timestamp == dpb->timestamp) > > + buf_idx = dst_buf->index; > > + else > > + buf_idx = vb2_find_timestamp(cap_q, dpb->timestamp, 0); > > + > > + if (buf_idx < 0) > > + continue; > > + > > + ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[buf_idx]); > > + cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf); > > + position = cedrus_buf->codec.h264.position; > > + > > + sram_array[i] |= position << 1; > > + if (ref_buf->field == V4L2_FIELD_BOTTOM) > > + sram_array[i] |= BIT(0); > > + } > > + > > + size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array)); > > Perhaps s/unsigned int/size_t, so the arguments to min() have the same type? > > Otherwise, I got this warning: > > /home/zeta/repos/linux/media_tree/drivers/staging/media/sunxi/cedrus/cedrus_h264.c: In function ‘_cedrus_write_ref_list’: > /home/zeta/repos/linux/media_tree/include/linux/kernel.h:846:29: warning: comparison of distinct pointer types lacks a cast > (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) Strange, I didn't notice any warning. I'll make sure to fix it. Thanks! Maxime -- Maxime Ripard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-12 12:47 ` Maxime Ripard @ 2019-02-12 17:40 ` Jernej Škrabec 0 siblings, 0 replies; 24+ messages in thread From: Jernej Škrabec @ 2019-02-12 17:40 UTC (permalink / raw) To: Maxime Ripard Cc: Ezequiel Garcia, hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jonas, linux-sunxi, Thomas Petazzoni Dne torek, 12. februar 2019 ob 13:47:13 CET je Maxime Ripard napisal(a): > On Mon, Feb 11, 2019 at 04:48:17PM -0300, Ezequiel Garcia wrote: > > On Mon, 2019-02-11 at 15:39 +0100, Maxime Ripard wrote: > > > Introduce some basic H264 decoding support in cedrus. So far, only the > > > baseline profile videos have been tested, and some more advanced > > > features > > > used in higher profiles are not even implemented. > > > > > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> > > > > [..] > > > > > +static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, > > > + struct cedrus_run *run, > > > + const u8 *ref_list, u8 num_ref, > > > + enum cedrus_h264_sram_off sram) > > > +{ > > > + const struct v4l2_ctrl_h264_decode_param *decode = > > > run->h264.decode_param; + struct vb2_queue *cap_q = > > > &ctx->fh.m2m_ctx->cap_q_ctx.q; > > > + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; > > > + struct cedrus_dev *dev = ctx->dev; > > > + u8 sram_array[CEDRUS_MAX_REF_IDX]; > > > + unsigned int size, i; > > > + > > > + memset(sram_array, 0, sizeof(sram_array)); > > > + > > > + for (i = 0; i < num_ref; i++) { > > > + const struct v4l2_h264_dpb_entry *dpb; > > > + const struct cedrus_buffer *cedrus_buf; > > > + const struct vb2_v4l2_buffer *ref_buf; > > > + unsigned int position; > > > + int buf_idx; > > > + u8 dpb_idx; > > > + > > > + dpb_idx = ref_list[i]; > > > + dpb = &decode->dpb[dpb_idx]; > > > + > > > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) > > > + continue; > > > + > > > + if (dst_buf->timestamp == dpb->timestamp) > > > + buf_idx = dst_buf->index; > > > + else > > > + buf_idx = vb2_find_timestamp(cap_q, dpb- >timestamp, 0); > > > + > > > + if (buf_idx < 0) > > > + continue; > > > + > > > + ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[buf_idx]); > > > + cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf); > > > + position = cedrus_buf->codec.h264.position; > > > + > > > + sram_array[i] |= position << 1; > > > + if (ref_buf->field == V4L2_FIELD_BOTTOM) > > > + sram_array[i] |= BIT(0); > > > + } > > > + > > > + size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array)); > > > > Perhaps s/unsigned int/size_t, so the arguments to min() have the same > > type? > > > > Otherwise, I got this warning: > > > > /home/zeta/repos/linux/media_tree/drivers/staging/media/sunxi/cedrus/cedru > > s_h264.c: In function ‘_cedrus_write_ref_list’: > > /home/zeta/repos/linux/media_tree/include/linux/kernel.h:846:29: warning: > > comparison of distinct pointer types lacks a cast> > > (!!(sizeof((typeof(x) *)1 == (typeof(y) *)1))) > > Strange, I didn't notice any warning. I'll make sure to fix it. I guess this is one of those warnings which pops out only on 64-bit platforms. Best regards, Jernej ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-11 14:39 ` [PATCH v3 2/2] media: cedrus: Add H264 decoding support Maxime Ripard 2019-02-11 19:21 ` [linux-sunxi] " Jernej Škrabec 2019-02-11 19:48 ` Ezequiel Garcia @ 2019-02-14 20:42 ` Jernej Škrabec 2019-02-14 21:27 ` Jernej Škrabec 3 siblings, 0 replies; 24+ messages in thread From: Jernej Škrabec @ 2019-02-14 20:42 UTC (permalink / raw) To: Maxime Ripard Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jonas, ezequiel, linux-sunxi, Thomas Petazzoni Dne ponedeljek, 11. februar 2019 ob 15:39:03 CET je Maxime Ripard napisal(a): > Introduce some basic H264 decoding support in cedrus. So far, only the > baseline profile videos have been tested, and some more advanced features > used in higher profiles are not even implemented. > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> > --- > drivers/staging/media/sunxi/cedrus/Makefile | 3 +- > drivers/staging/media/sunxi/cedrus/cedrus.c | 31 +- > drivers/staging/media/sunxi/cedrus/cedrus.h | 38 +- > drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 15 +- > drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 589 +++++++++++++++- > drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 4 +- > drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 91 ++- > drivers/staging/media/sunxi/cedrus/cedrus_video.c | 9 +- > 8 files changed, 778 insertions(+), 2 deletions(-) > create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_h264.c > > diff --git a/drivers/staging/media/sunxi/cedrus/Makefile > b/drivers/staging/media/sunxi/cedrus/Makefile index > e9dc68b7bcb6..aaf141fc58b6 100644 > --- a/drivers/staging/media/sunxi/cedrus/Makefile > +++ b/drivers/staging/media/sunxi/cedrus/Makefile > @@ -1,3 +1,4 @@ > obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o > > -sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o > cedrus_mpeg2.o +sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o > cedrus_dec.o \ + cedrus_mpeg2.o cedrus_h264.o > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c > b/drivers/staging/media/sunxi/cedrus/cedrus.c index > ff11cbeba205..b275607b8111 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c > @@ -40,6 +40,36 @@ static const struct cedrus_control cedrus_controls[] = { > .codec = CEDRUS_CODEC_MPEG2, > .required = false, > }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_decode_param), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_slice_param), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SPS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_sps), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_PPS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_pps), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, > + .elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > }; > > #define CEDRUS_CONTROLS_COUNT ARRAY_SIZE(cedrus_controls) > @@ -278,6 +308,7 @@ static int cedrus_probe(struct platform_device *pdev) > } > > dev->dec_ops[CEDRUS_CODEC_MPEG2] = &cedrus_dec_ops_mpeg2; > + dev->dec_ops[CEDRUS_CODEC_H264] = &cedrus_dec_ops_h264; > > mutex_init(&dev->dev_mutex); > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h > b/drivers/staging/media/sunxi/cedrus/cedrus.h index > 4aedd24a9848..8c64f9a27e9d 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus.h > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h > @@ -30,7 +30,7 @@ > > enum cedrus_codec { > CEDRUS_CODEC_MPEG2, > - > + CEDRUS_CODEC_H264, > CEDRUS_CODEC_LAST, > }; > > @@ -40,6 +40,12 @@ enum cedrus_irq_status { > CEDRUS_IRQ_OK, > }; > > +enum cedrus_h264_pic_type { > + CEDRUS_H264_PIC_TYPE_FRAME = 0, > + CEDRUS_H264_PIC_TYPE_FIELD, > + CEDRUS_H264_PIC_TYPE_MBAFF, > +}; > + > struct cedrus_control { > u32 id; > u32 elem_size; > @@ -47,6 +53,14 @@ struct cedrus_control { > unsigned char required:1; > }; > > +struct cedrus_h264_run { > + const struct v4l2_ctrl_h264_decode_param *decode_param; > + const struct v4l2_ctrl_h264_pps *pps; > + const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; > + const struct v4l2_ctrl_h264_slice_param *slice_param; > + const struct v4l2_ctrl_h264_sps *sps; > +}; > + > struct cedrus_mpeg2_run { > const struct v4l2_ctrl_mpeg2_slice_params *slice_params; > const struct v4l2_ctrl_mpeg2_quantization *quantization; > @@ -57,12 +71,20 @@ struct cedrus_run { > struct vb2_v4l2_buffer *dst; > > union { > + struct cedrus_h264_run h264; > struct cedrus_mpeg2_run mpeg2; > }; > }; > > struct cedrus_buffer { > struct v4l2_m2m_buffer m2m_buf; > + > + union { > + struct { > + unsigned int position; > + enum cedrus_h264_pic_type pic_type; > + } h264; > + } codec; > }; > > struct cedrus_ctx { > @@ -77,6 +99,19 @@ struct cedrus_ctx { > struct v4l2_ctrl **ctrls; > > struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME]; > + > + union { > + struct { > + void *mv_col_buf; > + dma_addr_t mv_col_buf_dma; > + ssize_t mv_col_buf_field_size; > + ssize_t mv_col_buf_size; > + void *pic_info_buf; > + dma_addr_t pic_info_buf_dma; > + void *neighbor_info_buf; > + dma_addr_t neighbor_info_buf_dma; > + } h264; > + } codec; > }; > > struct cedrus_dec_ops { > @@ -118,6 +153,7 @@ struct cedrus_dev { > }; > > extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2; > +extern struct cedrus_dec_ops cedrus_dec_ops_h264; > > static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val) > { > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c index > 443fb037e1cf..4c33e3b1283f 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > @@ -46,6 +46,21 @@ void cedrus_device_run(void *priv) > V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION); > break; > > + case V4L2_PIX_FMT_H264_SLICE: > + run.h264.decode_param = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS); > + run.h264.pps = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_PPS); > + run.h264.scaling_matrix = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); > + run.h264.slice_param = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS); > + run.h264.scaling_matrix = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); You already assigned scaling_matrix few lines above. Best regards, Jernej ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 2/2] media: cedrus: Add H264 decoding support 2019-02-11 14:39 ` [PATCH v3 2/2] media: cedrus: Add H264 decoding support Maxime Ripard ` (2 preceding siblings ...) 2019-02-14 20:42 ` Jernej Škrabec @ 2019-02-14 21:27 ` Jernej Škrabec 3 siblings, 0 replies; 24+ messages in thread From: Jernej Škrabec @ 2019-02-14 21:27 UTC (permalink / raw) To: Maxime Ripard Cc: hans.verkuil, acourbot, sakari.ailus, Laurent Pinchart, tfiga, posciak, Paul Kocialkowski, Chen-Yu Tsai, linux-kernel, linux-arm-kernel, linux-media, nicolas.dufresne, jenskuske, jonas, ezequiel, linux-sunxi, Thomas Petazzoni Dne ponedeljek, 11. februar 2019 ob 15:39:03 CET je Maxime Ripard napisal(a): > Introduce some basic H264 decoding support in cedrus. So far, only the > baseline profile videos have been tested, and some more advanced features > used in higher profiles are not even implemented. > > Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> > --- > drivers/staging/media/sunxi/cedrus/Makefile | 3 +- > drivers/staging/media/sunxi/cedrus/cedrus.c | 31 +- > drivers/staging/media/sunxi/cedrus/cedrus.h | 38 +- > drivers/staging/media/sunxi/cedrus/cedrus_dec.c | 15 +- > drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 589 +++++++++++++++- > drivers/staging/media/sunxi/cedrus/cedrus_hw.c | 4 +- > drivers/staging/media/sunxi/cedrus/cedrus_regs.h | 91 ++- > drivers/staging/media/sunxi/cedrus/cedrus_video.c | 9 +- > 8 files changed, 778 insertions(+), 2 deletions(-) > create mode 100644 drivers/staging/media/sunxi/cedrus/cedrus_h264.c > > diff --git a/drivers/staging/media/sunxi/cedrus/Makefile > b/drivers/staging/media/sunxi/cedrus/Makefile index > e9dc68b7bcb6..aaf141fc58b6 100644 > --- a/drivers/staging/media/sunxi/cedrus/Makefile > +++ b/drivers/staging/media/sunxi/cedrus/Makefile > @@ -1,3 +1,4 @@ > obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o > > -sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o cedrus_dec.o > cedrus_mpeg2.o +sunxi-cedrus-y = cedrus.o cedrus_video.o cedrus_hw.o > cedrus_dec.o \ + cedrus_mpeg2.o cedrus_h264.o > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c > b/drivers/staging/media/sunxi/cedrus/cedrus.c index > ff11cbeba205..b275607b8111 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c > @@ -40,6 +40,36 @@ static const struct cedrus_control cedrus_controls[] = { > .codec = CEDRUS_CODEC_MPEG2, > .required = false, > }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_decode_param), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_slice_param), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SPS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_sps), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_PPS, > + .elem_size = sizeof(struct v4l2_ctrl_h264_pps), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > + { > + .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, > + .elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix), > + .codec = CEDRUS_CODEC_H264, > + .required = true, > + }, > }; > > #define CEDRUS_CONTROLS_COUNT ARRAY_SIZE(cedrus_controls) > @@ -278,6 +308,7 @@ static int cedrus_probe(struct platform_device *pdev) > } > > dev->dec_ops[CEDRUS_CODEC_MPEG2] = &cedrus_dec_ops_mpeg2; > + dev->dec_ops[CEDRUS_CODEC_H264] = &cedrus_dec_ops_h264; > > mutex_init(&dev->dev_mutex); > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h > b/drivers/staging/media/sunxi/cedrus/cedrus.h index > 4aedd24a9848..8c64f9a27e9d 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus.h > +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h > @@ -30,7 +30,7 @@ > > enum cedrus_codec { > CEDRUS_CODEC_MPEG2, > - > + CEDRUS_CODEC_H264, > CEDRUS_CODEC_LAST, > }; > > @@ -40,6 +40,12 @@ enum cedrus_irq_status { > CEDRUS_IRQ_OK, > }; > > +enum cedrus_h264_pic_type { > + CEDRUS_H264_PIC_TYPE_FRAME = 0, > + CEDRUS_H264_PIC_TYPE_FIELD, > + CEDRUS_H264_PIC_TYPE_MBAFF, > +}; > + > struct cedrus_control { > u32 id; > u32 elem_size; > @@ -47,6 +53,14 @@ struct cedrus_control { > unsigned char required:1; > }; > > +struct cedrus_h264_run { > + const struct v4l2_ctrl_h264_decode_param *decode_param; > + const struct v4l2_ctrl_h264_pps *pps; > + const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; > + const struct v4l2_ctrl_h264_slice_param *slice_param; > + const struct v4l2_ctrl_h264_sps *sps; > +}; > + > struct cedrus_mpeg2_run { > const struct v4l2_ctrl_mpeg2_slice_params *slice_params; > const struct v4l2_ctrl_mpeg2_quantization *quantization; > @@ -57,12 +71,20 @@ struct cedrus_run { > struct vb2_v4l2_buffer *dst; > > union { > + struct cedrus_h264_run h264; > struct cedrus_mpeg2_run mpeg2; > }; > }; > > struct cedrus_buffer { > struct v4l2_m2m_buffer m2m_buf; > + > + union { > + struct { > + unsigned int position; > + enum cedrus_h264_pic_type pic_type; > + } h264; > + } codec; > }; > > struct cedrus_ctx { > @@ -77,6 +99,19 @@ struct cedrus_ctx { > struct v4l2_ctrl **ctrls; > > struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME]; > + > + union { > + struct { > + void *mv_col_buf; > + dma_addr_t mv_col_buf_dma; > + ssize_t mv_col_buf_field_size; > + ssize_t mv_col_buf_size; > + void *pic_info_buf; > + dma_addr_t pic_info_buf_dma; > + void *neighbor_info_buf; > + dma_addr_t neighbor_info_buf_dma; > + } h264; > + } codec; > }; > > struct cedrus_dec_ops { > @@ -118,6 +153,7 @@ struct cedrus_dev { > }; > > extern struct cedrus_dec_ops cedrus_dec_ops_mpeg2; > +extern struct cedrus_dec_ops cedrus_dec_ops_h264; > > static inline void cedrus_write(struct cedrus_dev *dev, u32 reg, u32 val) > { > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c index > 443fb037e1cf..4c33e3b1283f 100644 > --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c > @@ -46,6 +46,21 @@ void cedrus_device_run(void *priv) > V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION); > break; > > + case V4L2_PIX_FMT_H264_SLICE: > + run.h264.decode_param = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS); > + run.h264.pps = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_PPS); > + run.h264.scaling_matrix = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); > + run.h264.slice_param = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS); > + run.h264.scaling_matrix = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX); > + run.h264.sps = cedrus_find_control_data(ctx, > + V4L2_CID_MPEG_VIDEO_H264_SPS); > + break; > + > default: > break; > } > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c > b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c new file mode 100644 > index 000000000000..a5c5f13ffecb > --- /dev/null > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c > @@ -0,0 +1,589 @@ > +// SPDX-License-Identifier: GPL-2.0-or-later > +/* > + * Cedrus VPU driver > + * > + * Copyright (c) 2013 Jens Kuske <jenskuske@gmail.com> > + * Copyright (c) 2018 Bootlin > + */ > + > +#include <linux/types.h> > + > +#include <media/videobuf2-dma-contig.h> > + > +#include "cedrus.h" > +#include "cedrus_hw.h" > +#include "cedrus_regs.h" > + > +enum cedrus_h264_sram_off { > + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE = 0x000, > + CEDRUS_SRAM_H264_FRAMEBUFFER_LIST = 0x100, > + CEDRUS_SRAM_H264_REF_LIST_0 = 0x190, > + CEDRUS_SRAM_H264_REF_LIST_1 = 0x199, > + CEDRUS_SRAM_H264_SCALING_LIST_8x8_0 = 0x200, > + CEDRUS_SRAM_H264_SCALING_LIST_8x8_1 = 0x210, > + CEDRUS_SRAM_H264_SCALING_LIST_4x4 = 0x220, > +}; > + > +struct cedrus_h264_sram_ref_pic { > + __le32 top_field_order_cnt; > + __le32 bottom_field_order_cnt; > + __le32 frame_info; > + __le32 luma_ptr; > + __le32 chroma_ptr; > + __le32 mv_col_top_ptr; > + __le32 mv_col_bot_ptr; > + __le32 reserved; > +} __packed; > + > +#define CEDRUS_H264_FRAME_NUM 18 > + > +#define CEDRUS_NEIGHBOR_INFO_BUF_SIZE (16 * SZ_1K) > +#define CEDRUS_PIC_INFO_BUF_SIZE (128 * SZ_1K) > + > +static void cedrus_h264_write_sram(struct cedrus_dev *dev, > + enum cedrus_h264_sram_off off, > + const void *data, size_t len) > +{ > + const u32 *buffer = data; > + size_t count = DIV_ROUND_UP(len, 4); > + > + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET, off << 2); > + > + do { > + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, *buffer++); > + } while (--count); > +} > + > +static dma_addr_t cedrus_h264_mv_col_buf_addr(struct cedrus_ctx *ctx, > + unsigned int position, > + unsigned int field) > +{ > + dma_addr_t addr = ctx->codec.h264.mv_col_buf_dma; > + > + /* Adjust for the position */ > + addr += position * ctx->codec.h264.mv_col_buf_field_size * 2; > + > + /* Adjust for the field */ > + addr += field * ctx->codec.h264.mv_col_buf_field_size; > + > + return addr; > +} > + > +static void cedrus_fill_ref_pic(struct cedrus_ctx *ctx, > + struct cedrus_buffer *buf, > + unsigned int top_field_order_cnt, > + unsigned int bottom_field_order_cnt, > + struct cedrus_h264_sram_ref_pic *pic) > +{ > + struct vb2_buffer *vbuf = &buf->m2m_buf.vb.vb2_buf; > + unsigned int position = buf->codec.h264.position; > + > + pic->top_field_order_cnt = top_field_order_cnt; > + pic->bottom_field_order_cnt = bottom_field_order_cnt; > + pic->frame_info = buf->codec.h264.pic_type << 8; > + > + pic->luma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 0); > + pic->chroma_ptr = cedrus_buf_addr(vbuf, &ctx->dst_fmt, 1); > + pic->mv_col_top_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 0); > + pic->mv_col_bot_ptr = cedrus_h264_mv_col_buf_addr(ctx, position, 1); > +} > + > +static void cedrus_write_frame_list(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + struct cedrus_h264_sram_ref_pic pic_list[CEDRUS_H264_FRAME_NUM]; > + const struct v4l2_ctrl_h264_decode_param *dec_param = > run->h264.decode_param; + const struct v4l2_ctrl_h264_slice_param *slice = > run->h264.slice_param; + const struct v4l2_ctrl_h264_sps *sps = > run->h264.sps; > + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; > + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; > + struct cedrus_buffer *output_buf; > + struct cedrus_dev *dev = ctx->dev; > + unsigned long used_dpbs = 0; > + unsigned int position; > + unsigned int output = 0; > + unsigned int i; > + > + memset(pic_list, 0, sizeof(pic_list)); > + > + for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { > + const struct v4l2_h264_dpb_entry *dpb = &dec_param- >dpb[i]; > + struct cedrus_buffer *cedrus_buf; > + int buf_idx; > + > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) > + continue; > + > + if (dst_buf->timestamp == dpb->timestamp) > + buf_idx = dst_buf->index; > + else > + buf_idx = vb2_find_timestamp(cap_q, dpb- >timestamp, 0); > + > + if (buf_idx < 0) > + continue; > + > + cedrus_buf = vb2_to_cedrus_buffer(ctx- >dst_bufs[buf_idx]); > + position = cedrus_buf->codec.h264.position; > + used_dpbs |= BIT(position); > + > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) > + continue; > + > + cedrus_fill_ref_pic(ctx, cedrus_buf, > + dpb->top_field_order_cnt, > + dpb->bottom_field_order_cnt, > + &pic_list[position]); > + > + output = max(position, output); > + } > + > + position = find_next_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM, > + output); > + if (position >= CEDRUS_H264_FRAME_NUM) > + position = find_first_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM); > + > + output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf); > + output_buf->codec.h264.position = position; > + > + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) > + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FIELD; > + else if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD) > + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_MBAFF; > + else > + output_buf->codec.h264.pic_type = CEDRUS_H264_PIC_TYPE_FRAME; > + > + cedrus_fill_ref_pic(ctx, output_buf, > + dec_param->top_field_order_cnt, > + dec_param->bottom_field_order_cnt, > + &pic_list[position]); > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_FRAMEBUFFER_LIST, > + pic_list, sizeof(pic_list)); > + > + cedrus_write(dev, VE_H264_OUTPUT_FRAME_IDX, position); > +} > + > +#define CEDRUS_MAX_REF_IDX 32 > + > +static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, > + struct cedrus_run *run, > + const u8 *ref_list, u8 num_ref, > + enum cedrus_h264_sram_off sram) > +{ > + const struct v4l2_ctrl_h264_decode_param *decode = run- >h264.decode_param; > + struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q; > + const struct vb2_buffer *dst_buf = &run->dst->vb2_buf; > + struct cedrus_dev *dev = ctx->dev; > + u8 sram_array[CEDRUS_MAX_REF_IDX]; > + unsigned int size, i; > + > + memset(sram_array, 0, sizeof(sram_array)); > + > + for (i = 0; i < num_ref; i++) { > + const struct v4l2_h264_dpb_entry *dpb; > + const struct cedrus_buffer *cedrus_buf; > + const struct vb2_v4l2_buffer *ref_buf; > + unsigned int position; > + int buf_idx; > + u8 dpb_idx; > + > + dpb_idx = ref_list[i]; > + dpb = &decode->dpb[dpb_idx]; > + > + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) > + continue; > + > + if (dst_buf->timestamp == dpb->timestamp) > + buf_idx = dst_buf->index; > + else > + buf_idx = vb2_find_timestamp(cap_q, dpb- >timestamp, 0); > + > + if (buf_idx < 0) > + continue; > + > + ref_buf = to_vb2_v4l2_buffer(ctx->dst_bufs[buf_idx]); > + cedrus_buf = vb2_v4l2_to_cedrus_buffer(ref_buf); > + position = cedrus_buf->codec.h264.position; > + > + sram_array[i] |= position << 1; > + if (ref_buf->field == V4L2_FIELD_BOTTOM) > + sram_array[i] |= BIT(0); > + } > + > + size = min((unsigned int)ALIGN(num_ref, 4), sizeof(sram_array)); > + cedrus_h264_write_sram(dev, sram, &sram_array, size); > +} > + > +static void cedrus_write_ref_list0(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_slice_param *slice = run- >h264.slice_param; > + > + _cedrus_write_ref_list(ctx, run, > + slice->ref_pic_list0, > + slice->num_ref_idx_l0_active_minus1 + 1, > + CEDRUS_SRAM_H264_REF_LIST_0); > +} > + > +static void cedrus_write_ref_list1(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_slice_param *slice = run- >h264.slice_param; > + > + _cedrus_write_ref_list(ctx, run, > + slice->ref_pic_list1, > + slice->num_ref_idx_l1_active_minus1 + 1, > + CEDRUS_SRAM_H264_REF_LIST_1); > +} > + > +static void cedrus_write_scaling_lists(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_scaling_matrix *scaling = > + run->h264.scaling_matrix; > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_0, > + scaling->scaling_list_8x8[0], > + sizeof(scaling->scaling_list_8x8[0])); > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_1, > + scaling->scaling_list_8x8[1], > + sizeof(scaling->scaling_list_8x8[1])); > + > + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_4x4, > + scaling->scaling_list_4x4, > + sizeof(scaling->scaling_list_4x4)); > +} > + > +static void cedrus_write_pred_weight_table(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_slice_param *slice = > + run->h264.slice_param; > + const struct v4l2_h264_pred_weight_table *pred_weight = > + &slice->pred_weight_table; > + struct cedrus_dev *dev = ctx->dev; > + int i, j, k; > + > + cedrus_write(dev, VE_H264_SHS_WP, > + ((pred_weight->chroma_log2_weight_denom & 0xf) << 4) | > + ((pred_weight->luma_log2_weight_denom & 0xf) << 0)); > + > + cedrus_write(dev, VE_AVC_SRAM_PORT_OFFSET, > + CEDRUS_SRAM_H264_PRED_WEIGHT_TABLE << 2); > + > + for (i = 0; i < ARRAY_SIZE(pred_weight->weight_factors); i++) { > + const struct v4l2_h264_weight_factors *factors = > + &pred_weight->weight_factors[i]; > + > + for (j = 0; j < ARRAY_SIZE(factors->luma_weight); j++) { > + u32 val; > + > + val = ((factors->luma_offset[j] & 0x1ff) << 16) | > + (factors->luma_weight[j] & 0x1ff); > + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val); > + } > + > + for (j = 0; j < ARRAY_SIZE(factors->chroma_weight); j++) { > + for (k = 0; k < ARRAY_SIZE(factors- >chroma_weight[0]); k++) { > + u32 val; > + > + val = ((factors->chroma_offset[j] [k] & 0x1ff) << 16) | > + (factors- >chroma_weight[j][k] & 0x1ff); > + cedrus_write(dev, VE_AVC_SRAM_PORT_DATA, val); > + } > + } > + } > +} > + > +static void cedrus_set_params(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + const struct v4l2_ctrl_h264_decode_param *decode = run- >h264.decode_param; > + const struct v4l2_ctrl_h264_slice_param *slice = run- >h264.slice_param; > + const struct v4l2_ctrl_h264_pps *pps = run->h264.pps; > + const struct v4l2_ctrl_h264_sps *sps = run->h264.sps; > + struct vb2_buffer *src_buf = &run->src->vb2_buf; > + struct cedrus_dev *dev = ctx->dev; > + dma_addr_t src_buf_addr; > + u32 offset = slice->header_bit_size; > + u32 len = (slice->size * 8) - offset; > + u32 reg; > + > + cedrus_write(dev, VE_H264_VLD_LEN, len); > + cedrus_write(dev, VE_H264_VLD_OFFSET, offset); > + > + src_buf_addr = vb2_dma_contig_plane_dma_addr(src_buf, 0); > + cedrus_write(dev, VE_H264_VLD_END, > + src_buf_addr + vb2_get_plane_payload(src_buf, 0)); > + cedrus_write(dev, VE_H264_VLD_ADDR, > + VE_H264_VLD_ADDR_VAL(src_buf_addr) | > + VE_H264_VLD_ADDR_FIRST | VE_H264_VLD_ADDR_VALID | > + VE_H264_VLD_ADDR_LAST); > + > + /* > + * FIXME: Since the bitstream parsing is done in software, and > + * in userspace, this shouldn't be needed anymore. But it > + * turns out that removing it breaks the decoding process, > + * without any clear indication why. > + */ > + cedrus_write(dev, VE_H264_TRIGGER_TYPE, > + VE_H264_TRIGGER_TYPE_INIT_SWDEC); > + > + if (((pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) && > + (slice->slice_type == V4L2_H264_SLICE_TYPE_P || > + slice->slice_type == V4L2_H264_SLICE_TYPE_SP)) || > + (pps->weighted_bipred_idc == 1 && > + slice->slice_type == V4L2_H264_SLICE_TYPE_B)) > + cedrus_write_pred_weight_table(ctx, run); > + > + if ((slice->slice_type == V4L2_H264_SLICE_TYPE_P) || > + (slice->slice_type == V4L2_H264_SLICE_TYPE_SP) || > + (slice->slice_type == V4L2_H264_SLICE_TYPE_B)) > + cedrus_write_ref_list0(ctx, run); > + > + if (slice->slice_type == V4L2_H264_SLICE_TYPE_B) > + cedrus_write_ref_list1(ctx, run); > + > + // picture parameters > + reg = 0; > + /* > + * FIXME: the kernel headers are allowing the default value to > + * be passed, but the libva doesn't give us that. > + */ > + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 10; > + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 5; > + reg |= (pps->weighted_bipred_idc & 0x3) << 2; > + if (pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE) > + reg |= VE_H264_PPS_ENTROPY_CODING_MODE; > + if (pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) > + reg |= VE_H264_PPS_WEIGHTED_PRED; > + if (pps->flags & V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED) > + reg |= VE_H264_PPS_CONSTRAINED_INTRA_PRED; > + if (pps->flags & V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE) > + reg |= VE_H264_PPS_TRANSFORM_8X8_MODE; > + cedrus_write(dev, VE_H264_PPS, reg); > + > + // sequence parameters > + reg = 0; > + reg |= (sps->chroma_format_idc & 0x7) << 19; > + reg |= (sps->pic_width_in_mbs_minus1 & 0xff) << 8; > + reg |= sps->pic_height_in_map_units_minus1 & 0xff; > + if (sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) > + reg |= VE_H264_SPS_MBS_ONLY; > + if (sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD) > + reg |= VE_H264_SPS_MB_ADAPTIVE_FRAME_FIELD; > + if (sps->flags & V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE) > + reg |= VE_H264_SPS_DIRECT_8X8_INFERENCE; > + cedrus_write(dev, VE_H264_SPS, reg); > + > + // slice parameters > + reg = 0; > + reg |= decode->nal_ref_idc ? BIT(12) : 0; > + reg |= (slice->slice_type & 0xf) << 8; > + reg |= slice->cabac_init_idc & 0x3; > + reg |= VE_H264_SHS_FIRST_SLICE_IN_PIC; > + if (slice->flags & V4L2_H264_SLICE_FLAG_FIELD_PIC) > + reg |= VE_H264_SHS_FIELD_PIC; > + if (slice->flags & V4L2_H264_SLICE_FLAG_BOTTOM_FIELD) > + reg |= VE_H264_SHS_BOTTOM_FIELD; > + if (slice->flags & V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED) > + reg |= VE_H264_SHS_DIRECT_SPATIAL_MV_PRED; > + cedrus_write(dev, VE_H264_SHS, reg); > + > + reg = 0; > + reg |= VE_H264_SHS2_NUM_REF_IDX_ACTIVE_OVRD; > + reg |= (slice->num_ref_idx_l0_active_minus1 & 0x1f) << 24; > + reg |= (slice->num_ref_idx_l1_active_minus1 & 0x1f) << 16; > + reg |= (slice->disable_deblocking_filter_idc & 0x3) << 8; > + reg |= (slice->slice_alpha_c0_offset_div2 & 0xf) << 4; > + reg |= slice->slice_beta_offset_div2 & 0xf; > + cedrus_write(dev, VE_H264_SHS2, reg); > + > + reg = 0; > + /* > + * FIXME: This bit tells the video engine to use the default > + * quantization matrices. This will obviously need to be > + * changed to support the profiles supporting custom > + * quantization matrices. > + */ > + reg |= VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT; > + reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16; > + reg |= (pps->chroma_qp_index_offset & 0x3f) << 8; > + reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f; > + cedrus_write(dev, VE_H264_SHS_QP, reg); > + > + // clear status flags > + cedrus_write(dev, VE_H264_STATUS, cedrus_read(dev, VE_H264_STATUS)); > + > + // enable int > + reg = cedrus_read(dev, VE_H264_CTRL); > + cedrus_write(dev, VE_H264_CTRL, reg | > + VE_H264_CTRL_SLICE_DECODE_INT | > + VE_H264_CTRL_DECODE_ERR_INT | > + VE_H264_CTRL_VLD_DATA_REQ_INT); > +} > + > +static enum cedrus_irq_status > +cedrus_h264_irq_status(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + u32 reg = cedrus_read(dev, VE_H264_STATUS); > + > + if (reg & (VE_H264_STATUS_DECODE_ERR_INT | > + VE_H264_STATUS_VLD_DATA_REQ_INT)) > + return CEDRUS_IRQ_ERROR; > + > + if (reg & VE_H264_CTRL_SLICE_DECODE_INT) > + return CEDRUS_IRQ_OK; > + > + return CEDRUS_IRQ_NONE; > +} > + > +static void cedrus_h264_irq_clear(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_write(dev, VE_H264_STATUS, > + VE_H264_STATUS_INT_MASK); > +} > + > +static void cedrus_h264_irq_disable(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + u32 reg = cedrus_read(dev, VE_H264_CTRL); > + > + cedrus_write(dev, VE_H264_CTRL, > + reg & ~VE_H264_CTRL_INT_MASK); > +} > + > +static void cedrus_h264_setup(struct cedrus_ctx *ctx, > + struct cedrus_run *run) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + cedrus_engine_enable(dev, CEDRUS_CODEC_H264); > + > + cedrus_write(dev, VE_H264_SDROT_CTRL, 0); > + cedrus_write(dev, VE_H264_EXTRA_BUFFER1, > + ctx->codec.h264.pic_info_buf_dma); > + cedrus_write(dev, VE_H264_EXTRA_BUFFER2, > + ctx->codec.h264.neighbor_info_buf_dma); > + > + cedrus_write_scaling_lists(ctx, run); > + cedrus_write_frame_list(ctx, run); > + > + cedrus_set_params(ctx, run); > +} > + > +static int cedrus_h264_start(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + unsigned int field_size; > + unsigned int mv_col_size; > + int ret; > + > + /* > + * FIXME: It seems that the H6 cedarX code is using a formula > + * here based on the size of the frame, while all the older > + * code is using a fixed size, so that might need to be > + * changed at some point. > + */ > + ctx->codec.h264.pic_info_buf = > + dma_alloc_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, > + &ctx- >codec.h264.pic_info_buf_dma, > + GFP_KERNEL); > + if (!ctx->codec.h264.pic_info_buf) > + return -ENOMEM; > + > + /* > + * That buffer is supposed to be 16kiB in size, and be aligned > + * on 16kiB as well. However, dma_alloc_coherent provides the > + * guarantee that we'll have a CPU and DMA address aligned on > + * the smallest page order that is greater to the requested > + * size, so we don't have to overallocate. > + */ > + ctx->codec.h264.neighbor_info_buf = > + dma_alloc_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, > + &ctx- >codec.h264.neighbor_info_buf_dma, > + GFP_KERNEL); > + if (!ctx->codec.h264.neighbor_info_buf) { > + ret = -ENOMEM; > + goto err_pic_buf; > + } > + > + field_size = DIV_ROUND_UP(ctx->src_fmt.width, 16) * > + DIV_ROUND_UP(ctx->src_fmt.height, 16) * 16; > + > + /* > + * FIXME: This is actually conditional to > + * V4L2_H264_SPS_FLAG_DIRECT_8X8_INFERENCE not being set, we > + * might have to rework this if memory efficiency ever is > + * something we need to work on. > + */ > + field_size = field_size * 2; > + > + /* > + * FIXME: This is actually conditional to > + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY not being set, we might > + * have to rework this if memory efficiency ever is something > + * we need to work on. > + */ > + field_size = field_size * 2; > + ctx->codec.h264.mv_col_buf_field_size = field_size; > + > + mv_col_size = field_size * 2 * CEDRUS_H264_FRAME_NUM; > + ctx->codec.h264.mv_col_buf_size = mv_col_size; > + ctx->codec.h264.mv_col_buf = dma_alloc_coherent(dev->dev, > + ctx->codec.h264.mv_col_buf_size, > + &ctx->codec.h264.mv_col_buf_dma, > + GFP_KERNEL); > + if (!ctx->codec.h264.mv_col_buf) { > + ret = -ENOMEM; > + goto err_neighbor_buf; > + } > + > + return 0; > + > +err_neighbor_buf: > + dma_free_coherent(dev->dev, CEDRUS_NEIGHBOR_INFO_BUF_SIZE, > + ctx->codec.h264.neighbor_info_buf, > + ctx->codec.h264.neighbor_info_buf_dma); > + > +err_pic_buf: > + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, > + ctx->codec.h264.pic_info_buf, > + ctx->codec.h264.pic_info_buf_dma); > + return ret; > +} > + > +static void cedrus_h264_stop(struct cedrus_ctx *ctx) > +{ > + struct cedrus_dev *dev = ctx->dev; > + > + dma_free_coherent(dev->dev, ctx->codec.h264.mv_col_buf_size, > + ctx->codec.h264.mv_col_buf, > + ctx->codec.h264.mv_col_buf_dma); > + dma_free_coherent(dev->dev, CEDRUS_PIC_INFO_BUF_SIZE, > + ctx->codec.h264.pic_info_buf, > + ctx->codec.h264.pic_info_buf_dma); You forgot to free neighbor_info_buf. Best regards, Jernej ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 0/2] media: cedrus: Add H264 decoding support 2019-02-11 14:39 [PATCH v3 0/2] media: cedrus: Add H264 decoding support Maxime Ripard 2019-02-11 14:39 ` [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls Maxime Ripard 2019-02-11 14:39 ` [PATCH v3 2/2] media: cedrus: Add H264 decoding support Maxime Ripard @ 2019-02-12 12:50 ` Tomasz Figa 2019-02-12 21:22 ` Ezequiel Garcia 2 siblings, 1 reply; 24+ messages in thread From: Tomasz Figa @ 2019-02-12 12:50 UTC (permalink / raw) To: Maxime Ripard Cc: Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Pawel Osciak, Paul Kocialkowski, Chen-Yu Tsai, Linux Kernel Mailing List, list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>, Joerg Roedel <joro@8bytes.org>,, Linux Media Mailing List, Nicolas Dufresne, jenskuske, Jernej Skrabec, Jonas Karlman, Ezequiel Garcia, linux-sunxi, Thomas Petazzoni Hi Maxime, On Mon, Feb 11, 2019 at 11:39 PM Maxime Ripard <maxime.ripard@bootlin.com> wrote: > > Hi, > > Here is a new version of the H264 decoding support in the cedrus > driver. Thanks for working on this. Please see my comments below. > > As you might already know, the cedrus driver relies on the Request > API, and is a reverse engineered driver for the video decoding engine > found on the Allwinner SoCs. > > This work has been possible thanks to the work done by the people > behind libvdpau-sunxi found here: > https://github.com/linux-sunxi/libvdpau-sunxi/ > > I've tested the various ABI using this gdb script: > http://code.bulix.org/jl4se4-505620?raw > > And this test script: > http://code.bulix.org/8zle4s-505623?raw > > The application compiled is quite trivial: > http://code.bulix.org/e34zp8-505624?raw > > The output is: > arm: builds/arm-test-v4l2-h264-structures > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > x86: builds/x86-test-v4l2-h264-structures > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > x64: builds/x64-test-v4l2-h264-structures > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > arm64: builds/arm64-test-v4l2-h264-structures > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > Let me know if there's any flaw using that test setup, or if you have > any comments on the patches. > > Maxime > > Changes from v2: > - Simplified _cedrus_write_ref_list as suggested by Jernej > - Set whether the frame is used as reference using nal_ref_idc > - Respect chroma_format_idc > - Fixes for the scaling list and prediction tables > - Wrote the documentation for the flags > - Added a bunch of defines to the driver bit fields > - Reworded the controls and data format descriptions as suggested > by Hans > - Reworked the controls' structure field size to avoid padding > - Removed the long term reference flag This and... > - Reintroduced the neighbor info buffer > - Removed the ref_pic_list_p0/b0/b1 arrays that are redundant with the > one in the DPB these are used in our Rockchip VDEC driver. Could you elaborate on the reasons why they got removed? Best regards, Tomasz ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 0/2] media: cedrus: Add H264 decoding support 2019-02-12 12:50 ` [PATCH v3 0/2] " Tomasz Figa @ 2019-02-12 21:22 ` Ezequiel Garcia 2019-02-13 3:02 ` Tomasz Figa 0 siblings, 1 reply; 24+ messages in thread From: Ezequiel Garcia @ 2019-02-12 21:22 UTC (permalink / raw) To: Tomasz Figa, Maxime Ripard Cc: Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Pawel Osciak, Paul Kocialkowski, Chen-Yu Tsai, Linux Kernel Mailing List, list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>, Joerg Roedel <joro@8bytes.org>,, Linux Media Mailing List, Nicolas Dufresne, jenskuske, Jernej Skrabec, Jonas Karlman, linux-sunxi, Thomas Petazzoni Hey Tomasz, On Tue, 2019-02-12 at 21:50 +0900, Tomasz Figa wrote: > Hi Maxime, > > On Mon, Feb 11, 2019 at 11:39 PM Maxime Ripard > <maxime.ripard@bootlin.com> wrote: > > Hi, > > > > Here is a new version of the H264 decoding support in the cedrus > > driver. > > Thanks for working on this. Please see my comments below. > > > As you might already know, the cedrus driver relies on the Request > > API, and is a reverse engineered driver for the video decoding engine > > found on the Allwinner SoCs. > > > > This work has been possible thanks to the work done by the people > > behind libvdpau-sunxi found here: > > https://github.com/linux-sunxi/libvdpau-sunxi/ > > > > I've tested the various ABI using this gdb script: > > http://code.bulix.org/jl4se4-505620?raw > > > > And this test script: > > http://code.bulix.org/8zle4s-505623?raw > > > > The application compiled is quite trivial: > > http://code.bulix.org/e34zp8-505624?raw > > > > The output is: > > arm: builds/arm-test-v4l2-h264-structures > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > x86: builds/x86-test-v4l2-h264-structures > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > x64: builds/x64-test-v4l2-h264-structures > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > arm64: builds/arm64-test-v4l2-h264-structures > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > Let me know if there's any flaw using that test setup, or if you have > > any comments on the patches. > > > > Maxime > > > > Changes from v2: > > - Simplified _cedrus_write_ref_list as suggested by Jernej > > - Set whether the frame is used as reference using nal_ref_idc > > - Respect chroma_format_idc > > - Fixes for the scaling list and prediction tables > > - Wrote the documentation for the flags > > - Added a bunch of defines to the driver bit fields > > - Reworded the controls and data format descriptions as suggested > > by Hans > > - Reworked the controls' structure field size to avoid padding > > - Removed the long term reference flag > > This and... > Maxime has dropped this because of Ayaka's mail about long term references not making much sense in stateless decoders. I noticed that RK3399 TRM has a field to specify long term refs and so was wondering about this item as well. > > - Reintroduced the neighbor info buffer > > - Removed the ref_pic_list_p0/b0/b1 arrays that are redundant with the > > one in the DPB > > these are used in our Rockchip VDEC driver. > > Could you elaborate on the reasons why they got removed? > If I understood correctly, there are two reference picture lists. P-frames will populate ref_pic_list0 and B-frames will populate both. According to this, v4l2_ctrl_h264_slice_param.ref_pic_list0 and .ref_pic_list1 should be enough and ref_pic_list_p0/b0/b1 are not needed. What do you think? Regards, Ezequiel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 0/2] media: cedrus: Add H264 decoding support 2019-02-12 21:22 ` Ezequiel Garcia @ 2019-02-13 3:02 ` Tomasz Figa 2019-02-13 16:28 ` Ezequiel Garcia 0 siblings, 1 reply; 24+ messages in thread From: Tomasz Figa @ 2019-02-13 3:02 UTC (permalink / raw) To: Ezequiel Garcia Cc: Maxime Ripard, Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Pawel Osciak, Paul Kocialkowski, Chen-Yu Tsai, Linux Kernel Mailing List, list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>, Joerg Roedel <joro@8bytes.org>,, Linux Media Mailing List, Nicolas Dufresne, jenskuske, Jernej Skrabec, Jonas Karlman, linux-sunxi, Thomas Petazzoni On Wed, Feb 13, 2019 at 6:22 AM Ezequiel Garcia <ezequiel@collabora.com> wrote: > > Hey Tomasz, > > On Tue, 2019-02-12 at 21:50 +0900, Tomasz Figa wrote: > > Hi Maxime, > > > > On Mon, Feb 11, 2019 at 11:39 PM Maxime Ripard > > <maxime.ripard@bootlin.com> wrote: > > > Hi, > > > > > > Here is a new version of the H264 decoding support in the cedrus > > > driver. > > > > Thanks for working on this. Please see my comments below. > > > > > As you might already know, the cedrus driver relies on the Request > > > API, and is a reverse engineered driver for the video decoding engine > > > found on the Allwinner SoCs. > > > > > > This work has been possible thanks to the work done by the people > > > behind libvdpau-sunxi found here: > > > https://github.com/linux-sunxi/libvdpau-sunxi/ > > > > > > I've tested the various ABI using this gdb script: > > > http://code.bulix.org/jl4se4-505620?raw > > > > > > And this test script: > > > http://code.bulix.org/8zle4s-505623?raw > > > > > > The application compiled is quite trivial: > > > http://code.bulix.org/e34zp8-505624?raw > > > > > > The output is: > > > arm: builds/arm-test-v4l2-h264-structures > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > x86: builds/x86-test-v4l2-h264-structures > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > x64: builds/x64-test-v4l2-h264-structures > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > arm64: builds/arm64-test-v4l2-h264-structures > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > > > Let me know if there's any flaw using that test setup, or if you have > > > any comments on the patches. > > > > > > Maxime > > > > > > Changes from v2: > > > - Simplified _cedrus_write_ref_list as suggested by Jernej > > > - Set whether the frame is used as reference using nal_ref_idc > > > - Respect chroma_format_idc > > > - Fixes for the scaling list and prediction tables > > > - Wrote the documentation for the flags > > > - Added a bunch of defines to the driver bit fields > > > - Reworded the controls and data format descriptions as suggested > > > by Hans > > > - Reworked the controls' structure field size to avoid padding > > > - Removed the long term reference flag > > > > This and... > > > > Maxime has dropped this because of Ayaka's mail about long term references > not making much sense in stateless decoders. I haven't seen any argument confirming that thesis, though. I should have kicked in earlier, sorry. > > I noticed that RK3399 TRM has a field to specify long term refs and > so was wondering about this item as well. > > > > - Reintroduced the neighbor info buffer > > > - Removed the ref_pic_list_p0/b0/b1 arrays that are redundant with the > > > one in the DPB > > > > these are used in our Rockchip VDEC driver. > > > > Could you elaborate on the reasons why they got removed? > > > > If I understood correctly, there are two reference picture lists. > P-frames will populate ref_pic_list0 and B-frames will populate both. > > According to this, v4l2_ctrl_h264_slice_param.ref_pic_list0 and .ref_pic_list1 > should be enough and ref_pic_list_p0/b0/b1 are not needed. > > What do you think? The lists in v4l2_ctrl_h264_slice_param are expected to be past the per-slice modification stage (which is quite complicated and better done in userspace), while the ones in v4l2_ctrl_h264_decode_param just in the original order. Rockchip VPU expects them in the original order and does the modification in the hardware. Best regards, Tomasz ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 0/2] media: cedrus: Add H264 decoding support 2019-02-13 3:02 ` Tomasz Figa @ 2019-02-13 16:28 ` Ezequiel Garcia 2019-02-14 15:47 ` Maxime Ripard 0 siblings, 1 reply; 24+ messages in thread From: Ezequiel Garcia @ 2019-02-13 16:28 UTC (permalink / raw) To: Tomasz Figa Cc: Maxime Ripard, Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Pawel Osciak, Paul Kocialkowski, Chen-Yu Tsai, Linux Kernel Mailing List, list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>, Joerg Roedel <joro@8bytes.org>,, Linux Media Mailing List, Nicolas Dufresne, jenskuske, Jernej Skrabec, Jonas Karlman, linux-sunxi, Thomas Petazzoni On Wed, 2019-02-13 at 12:02 +0900, Tomasz Figa wrote: > On Wed, Feb 13, 2019 at 6:22 AM Ezequiel Garcia <ezequiel@collabora.com> wrote: > > Hey Tomasz, > > > > On Tue, 2019-02-12 at 21:50 +0900, Tomasz Figa wrote: > > > Hi Maxime, > > > > > > On Mon, Feb 11, 2019 at 11:39 PM Maxime Ripard > > > <maxime.ripard@bootlin.com> wrote: > > > > Hi, > > > > > > > > Here is a new version of the H264 decoding support in the cedrus > > > > driver. > > > > > > Thanks for working on this. Please see my comments below. > > > > > > > As you might already know, the cedrus driver relies on the Request > > > > API, and is a reverse engineered driver for the video decoding engine > > > > found on the Allwinner SoCs. > > > > > > > > This work has been possible thanks to the work done by the people > > > > behind libvdpau-sunxi found here: > > > > https://github.com/linux-sunxi/libvdpau-sunxi/ > > > > > > > > I've tested the various ABI using this gdb script: > > > > http://code.bulix.org/jl4se4-505620?raw > > > > > > > > And this test script: > > > > http://code.bulix.org/8zle4s-505623?raw > > > > > > > > The application compiled is quite trivial: > > > > http://code.bulix.org/e34zp8-505624?raw > > > > > > > > The output is: > > > > arm: builds/arm-test-v4l2-h264-structures > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > x86: builds/x86-test-v4l2-h264-structures > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > x64: builds/x64-test-v4l2-h264-structures > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > arm64: builds/arm64-test-v4l2-h264-structures > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > > > > > Let me know if there's any flaw using that test setup, or if you have > > > > any comments on the patches. > > > > > > > > Maxime > > > > > > > > Changes from v2: > > > > - Simplified _cedrus_write_ref_list as suggested by Jernej > > > > - Set whether the frame is used as reference using nal_ref_idc > > > > - Respect chroma_format_idc > > > > - Fixes for the scaling list and prediction tables > > > > - Wrote the documentation for the flags > > > > - Added a bunch of defines to the driver bit fields > > > > - Reworded the controls and data format descriptions as suggested > > > > by Hans > > > > - Reworked the controls' structure field size to avoid padding > > > > - Removed the long term reference flag > > > > > > This and... > > > > > > > Maxime has dropped this because of Ayaka's mail about long term references > > not making much sense in stateless decoders. > > I haven't seen any argument confirming that thesis, though. I should > have kicked in earlier, sorry. > OK, in that case, we need to have this flag back. > > I noticed that RK3399 TRM has a field to specify long term refs and > > so was wondering about this item as well. > > > > > > - Reintroduced the neighbor info buffer > > > > - Removed the ref_pic_list_p0/b0/b1 arrays that are redundant with the > > > > one in the DPB > > > > > > these are used in our Rockchip VDEC driver. > > > > > > Could you elaborate on the reasons why they got removed? > > > > > > > If I understood correctly, there are two reference picture lists. > > P-frames will populate ref_pic_list0 and B-frames will populate both. > > > > According to this, v4l2_ctrl_h264_slice_param.ref_pic_list0 and .ref_pic_list1 > > should be enough and ref_pic_list_p0/b0/b1 are not needed. > > > > What do you think? > > The lists in v4l2_ctrl_h264_slice_param are expected to be past the > per-slice modification stage (which is quite complicated and better > done in userspace), The fact that these are RefPicList0 and RefPicList1, after the reordering stage should be better documented. > while the ones in v4l2_ctrl_h264_decode_param just > in the original order. Rockchip VPU expects them in the original order > and does the modification in the hardware. > OK, I see. So, we have RefPicList0 and RefPicList1, and there is an initialization stage and a modification/reordering process. One could argue that it's more generic to just pass the initial list, but that would mean doing in the kernel something that is easier done in userspace (and parsers doing this are already available). The question would be what is the most generic way of passing the RefPicList0 and RefPicList1 in its initial state. 1/ We create additional controls for these. 2/ We put them on some of the other controls. Putting them on v4l2_ctrl_h264_decode_param didn't seem too wrong. Any objections to put them back in there? Regards, Ezequiel ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v3 0/2] media: cedrus: Add H264 decoding support 2019-02-13 16:28 ` Ezequiel Garcia @ 2019-02-14 15:47 ` Maxime Ripard 0 siblings, 0 replies; 24+ messages in thread From: Maxime Ripard @ 2019-02-14 15:47 UTC (permalink / raw) To: Ezequiel Garcia Cc: Tomasz Figa, Hans Verkuil, Alexandre Courbot, Sakari Ailus, Laurent Pinchart, Pawel Osciak, Paul Kocialkowski, Chen-Yu Tsai, Linux Kernel Mailing List, list@263.net:IOMMU DRIVERS <iommu@lists.linux-foundation.org>, Joerg Roedel <joro@8bytes.org>,, Linux Media Mailing List, Nicolas Dufresne, jenskuske, Jernej Skrabec, Jonas Karlman, linux-sunxi, Thomas Petazzoni On Wed, Feb 13, 2019 at 01:28:34PM -0300, Ezequiel Garcia wrote: > On Wed, 2019-02-13 at 12:02 +0900, Tomasz Figa wrote: > > On Wed, Feb 13, 2019 at 6:22 AM Ezequiel Garcia <ezequiel@collabora.com> wrote: > > > Hey Tomasz, > > > > > > On Tue, 2019-02-12 at 21:50 +0900, Tomasz Figa wrote: > > > > Hi Maxime, > > > > > > > > On Mon, Feb 11, 2019 at 11:39 PM Maxime Ripard > > > > <maxime.ripard@bootlin.com> wrote: > > > > > Hi, > > > > > > > > > > Here is a new version of the H264 decoding support in the cedrus > > > > > driver. > > > > > > > > Thanks for working on this. Please see my comments below. > > > > > > > > > As you might already know, the cedrus driver relies on the Request > > > > > API, and is a reverse engineered driver for the video decoding engine > > > > > found on the Allwinner SoCs. > > > > > > > > > > This work has been possible thanks to the work done by the people > > > > > behind libvdpau-sunxi found here: > > > > > https://github.com/linux-sunxi/libvdpau-sunxi/ > > > > > > > > > > I've tested the various ABI using this gdb script: > > > > > http://code.bulix.org/jl4se4-505620?raw > > > > > > > > > > And this test script: > > > > > http://code.bulix.org/8zle4s-505623?raw > > > > > > > > > > The application compiled is quite trivial: > > > > > http://code.bulix.org/e34zp8-505624?raw > > > > > > > > > > The output is: > > > > > arm: builds/arm-test-v4l2-h264-structures > > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > > x86: builds/x86-test-v4l2-h264-structures > > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > > x64: builds/x64-test-v4l2-h264-structures > > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > > arm64: builds/arm64-test-v4l2-h264-structures > > > > > SHA1: 88cbf7485ba81831fc3b93772b215599b3b38318 > > > > > > > > > > Let me know if there's any flaw using that test setup, or if you have > > > > > any comments on the patches. > > > > > > > > > > Maxime > > > > > > > > > > Changes from v2: > > > > > - Simplified _cedrus_write_ref_list as suggested by Jernej > > > > > - Set whether the frame is used as reference using nal_ref_idc > > > > > - Respect chroma_format_idc > > > > > - Fixes for the scaling list and prediction tables > > > > > - Wrote the documentation for the flags > > > > > - Added a bunch of defines to the driver bit fields > > > > > - Reworded the controls and data format descriptions as suggested > > > > > by Hans > > > > > - Reworked the controls' structure field size to avoid padding > > > > > - Removed the long term reference flag > > > > > > > > This and... > > > > > > > > > > Maxime has dropped this because of Ayaka's mail about long term references > > > not making much sense in stateless decoders. > > > > I haven't seen any argument confirming that thesis, though. I should > > have kicked in earlier, sorry. > > > > OK, in that case, we need to have this flag back. > > > > I noticed that RK3399 TRM has a field to specify long term refs and > > > so was wondering about this item as well. > > > > > > > > - Reintroduced the neighbor info buffer > > > > > - Removed the ref_pic_list_p0/b0/b1 arrays that are redundant with the > > > > > one in the DPB > > > > > > > > these are used in our Rockchip VDEC driver. > > > > > > > > Could you elaborate on the reasons why they got removed? > > > > > > > > > > If I understood correctly, there are two reference picture lists. > > > P-frames will populate ref_pic_list0 and B-frames will populate both. > > > > > > According to this, v4l2_ctrl_h264_slice_param.ref_pic_list0 and .ref_pic_list1 > > > should be enough and ref_pic_list_p0/b0/b1 are not needed. > > > > > > What do you think? > > > > The lists in v4l2_ctrl_h264_slice_param are expected to be past the > > per-slice modification stage (which is quite complicated and better > > done in userspace), > > The fact that these are RefPicList0 and RefPicList1, after > the reordering stage should be better documented. > > > while the ones in v4l2_ctrl_h264_decode_param just > > in the original order. Rockchip VPU expects them in the original order > > and does the modification in the hardware. > > > > OK, I see. > > So, we have RefPicList0 and RefPicList1, and there is an initialization > stage and a modification/reordering process. > > One could argue that it's more generic to just pass the initial list, > but that would mean doing in the kernel something that is easier > done in userspace (and parsers doing this are already available). > > The question would be what is the most generic way of passing > the RefPicList0 and RefPicList1 in its initial state. > > 1/ We create additional controls for these. > > 2/ We put them on some of the other controls. Putting them on > v4l2_ctrl_h264_decode_param didn't seem too wrong. > > Any objections to put them back in there? None. i'll put them back in. Maxime -- Maxime Ripard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2019-02-14 21:27 UTC | newest] Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-02-11 14:39 [PATCH v3 0/2] media: cedrus: Add H264 decoding support Maxime Ripard 2019-02-11 14:39 ` [PATCH v3 1/2] media: uapi: Add H264 low-level decoder API compound controls Maxime Ripard 2019-02-11 15:16 ` Hans Verkuil 2019-02-11 15:21 ` Hans Verkuil 2019-02-12 13:05 ` Maxime Ripard 2019-02-12 21:17 ` Ezequiel Garcia 2019-02-11 17:16 ` Nicolas Dufresne 2019-02-11 17:12 ` Ezequiel Garcia 2019-02-12 10:23 ` Maxime Ripard 2019-02-11 19:53 ` Ezequiel Garcia 2019-02-11 14:39 ` [PATCH v3 2/2] media: cedrus: Add H264 decoding support Maxime Ripard 2019-02-11 19:21 ` [linux-sunxi] " Jernej Škrabec 2019-02-12 10:43 ` Maxime Ripard 2019-02-12 17:46 ` Jernej Škrabec 2019-02-11 19:48 ` Ezequiel Garcia 2019-02-12 12:47 ` Maxime Ripard 2019-02-12 17:40 ` Jernej Škrabec 2019-02-14 20:42 ` Jernej Škrabec 2019-02-14 21:27 ` Jernej Škrabec 2019-02-12 12:50 ` [PATCH v3 0/2] " Tomasz Figa 2019-02-12 21:22 ` Ezequiel Garcia 2019-02-13 3:02 ` Tomasz Figa 2019-02-13 16:28 ` Ezequiel Garcia 2019-02-14 15:47 ` Maxime Ripard
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).