[PATCH v3 0/9] Add HANTRO G2/HEVC decoder support for IMX8MQ

[PATCH v3 0/9] Add HANTRO G2/HEVC decoder support for IMX8MQ

[Benjamin Gaignard]

The IMX8MQ got two VPUs but until now only G1 has been enabled.
This series aim to add the second VPU (aka G2) and provide basic 
HEVC decoding support.

To be able to decode HEVC it is needed to add/update some of the
structures in the uapi. In addition of them one HANTRO dedicated
control is required to inform the driver of the numbre of bits to skip
at the beginning of the slice header.
The hardware require to allocate few auxiliary buffers to store the
references frame or tile size data.

The driver has been tested with fluster test suite stream.
For example with this command: ./fluster.py run -ts JCT-VC-HEVC_V1 -d GStreamer-H.265-V4L2SL-Gst1.0
 
This series depends of the reset rework posted here: https://www.spinics.net/lists/arm-kernel/msg875766.html

Finally the both VPUs will have a node the device-tree and be
independent from v4l2 point of view.

A branch with all the dev is available here:
https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/upstream_g2_v2

version 3:
- Fix typo in Hantro v4l2 dedicated control
- Add documentation for the new structures and fields
- Rebased on top of media_tree for-linus-5.12-rc1 tag

version 2:
- remove all change related to scaling
- squash commits to a coherent split
- be more verbose about the added fields
- fix the comments done by Ezequiel about dma_alloc_coherent usage
- fix Dan's comments about control copy, reverse the test logic
in tile_buffer_reallocate, rework some goto and return cases.
- be more verbose about why I change the bindings
- remove all sign-off expect mime since it is confusing
- remove useless clocks in VPUs nodes

Benjamin

Benjamin Gaignard (9):
  media: hevc: Modify structures to follow H265 ITU spec
  media: hantro: Define HEVC codec profiles and supported features
  media: hantro: Add a field to distinguish the hardware versions
  media: uapi: Add a control for HANTRO driver
  media: hantro: Introduce G2/HEVC decoder
  media: hantro: handle V4L2_PIX_FMT_HEVC_SLICE control
  media: hantro: IMX8M: add variant for G2/HEVC codec
  dt-bindings: media: nxp,imx8mq-vpu: Update bindings
  arm64: dts: imx8mq: Add node to G2 hardware

 .../bindings/media/nxp,imx8mq-vpu.yaml        |  54 +-
 .../media/v4l/ext-ctrls-codec.rst             | 126 +++-
 .../media/v4l/vidioc-queryctrl.rst            |   6 +
 arch/arm64/boot/dts/freescale/imx8mq.dtsi     |  41 +-
 drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +-
 drivers/staging/media/hantro/Makefile         |   2 +
 drivers/staging/media/hantro/hantro.h         |  34 +-
 drivers/staging/media/hantro/hantro_drv.c     | 103 +++
 .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++++++++++++++++++
 drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++++++
 drivers/staging/media/hantro/hantro_hevc.c    | 321 ++++++++++
 drivers/staging/media/hantro/hantro_hw.h      |  48 ++
 .../staging/media/hantro/hantro_postproc.c    |  17 +
 drivers/staging/media/hantro/hantro_v4l2.c    |   1 +
 drivers/staging/media/hantro/imx8m_vpu_hw.c   |  95 ++-
 drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
 drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
 .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
 .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
 include/media/hevc-ctrls.h                    |  45 +-
 include/uapi/linux/hantro-v4l2-controls.h     |  20 +
 include/uapi/linux/v4l2-controls.h            |   5 +
 22 files changed, 1674 insertions(+), 70 deletions(-)
 create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
 create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
 create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
 create mode 100644 include/uapi/linux/hantro-v4l2-controls.h

-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[Benjamin Gaignard]

The H.265 ITU specification (section 7.4) define the general
slice segment header semantics.
Modified/added fields are:
- video_parameter_set_id: (7.4.3.1) identifies the VPS for
reference by other syntax elements.
- seq_parameter_set_id: (7.4.3.2.1) specifies the value of
the vps_video_parameter_set_id of the active VPS.
- chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
 relative to the luma sampling
- pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
reference by other syntax elements
- num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
the inferred value of num_ref_idx_l0_active_minus1
- num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
the inferred value of num_ref_idx_l1_active_minus1
- slice_segment_addr: (7.4.7.1) specifies the address of
the first coding tree block in the slice segment
- num_entry_point_offsets: (7.4.7.1) specifies the number of
entry_point_offset_minus1[ i ] syntax elements in the slice header

Add HEVC decode params contains the information used in section
"8.3 Slice decoding process" of the specification to let the hardware
perform decoding of a slices.

Adapt Cedrus driver according to these changes.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 3:
- Add documentation about the new structuers and fields.

version 2:
- remove all change related to scaling
- squash commits to a coherent split
- be more verbose about the added fields

 .../media/v4l/ext-ctrls-codec.rst             | 126 +++++++++++++++---
 .../media/v4l/vidioc-queryctrl.rst            |   6 +
 drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
 drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
 drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
 .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
 .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
 include/media/hevc-ctrls.h                    |  45 +++++--
 8 files changed, 186 insertions(+), 32 deletions(-)

diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 00944e97d638..5e6d77e858c0 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
     :stub-columns: 0
     :widths:       1 1 2
 
+    * - __u8
+      - ``video_parameter_set_id``
+      - Identifies the VPS for reference by other syntax elements
+    * - __u8
+      - ``seq_parameter_set_id̀``
+      - Specifies the value of the vps_video_parameter_set_id of the active VPS
+    * - __u8
+      - ``chroma_format_idc``
+      - Specifies the chroma sampling relative to the luma sampling
     * - __u16
       - ``pic_width_in_luma_samples``
       -
@@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
     * - __u8
       - ``chroma_format_idc``
       -
+    * - __u8
+      - ``num_slices``
+      -
     * - __u64
       - ``flags``
       - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
@@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
     :stub-columns: 0
     :widths:       1 1 2
 
+    * - __u8
+      - ``pic_parameter_set_id``
+      - Identifies the PPS for reference by other syntax elements
     * - __u8
       - ``num_extra_slice_header_bits``
       -
+    * - __u8
+      - ``num_ref_idx_l0_default_active_minus1``
+      - Specifies the inferred value of num_ref_idx_l0_active_minus1
+    * - __u8
+      - ``num_ref_idx_l1_default_active_minus1``
+      - Specifies the inferred value of num_ref_idx_l1_active_minus1
     * - __s8
       - ``init_qp_minus26``
       -
@@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
     * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
       - 0x00040000
       -
+    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
+      - 0x00080000
+      -
+    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
+      - 0x00100000
+      -
 
 ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
     Specifies various slice-specific parameters, especially from the NAL unit
@@ -3366,6 +3393,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
     * - __u32
       - ``data_bit_offset``
       - Offset (in bits) to the video data in the current slice data.
+    * - __u32
+      - ``slice_segment_addr``
+      - Specifies the address of the first coding tree block in the slice segment
+    * - __u32
+      - ``num_entry_point_offsets``
+      - Specifies the number of entry_point_offset_minus1[ i ] syntax elements in the slice header
     * - __u8
       - ``nal_unit_type``
       -
@@ -3422,28 +3455,20 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
     * - __u8
       - ``pic_struct``
       -
-    * - __u8
-      - ``num_active_dpb_entries``
-      - The number of entries in ``dpb``.
     * - __u8
       - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
       - The list of L0 reference elements as indices in the DPB.
     * - __u8
       - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
       - The list of L1 reference elements as indices in the DPB.
+    * - __u16
+      - ``short_term_ref_pic_set_size``
+      -
+    * - __u16
+      - ``long_term_ref_pic_set_size``
+      -
     * - __u8
-      - ``num_rps_poc_st_curr_before``
-      - The number of reference pictures in the short-term set that come before
-        the current frame.
-    * - __u8
-      - ``num_rps_poc_st_curr_after``
-      - The number of reference pictures in the short-term set that come after
-        the current frame.
-    * - __u8
-      - ``num_rps_poc_lt_curr``
-      - The number of reference pictures in the long-term set.
-    * - __u8
-      - ``padding[7]``
+      - ``padding``
       - Applications and drivers must set this to zero.
     * - struct :c:type:`v4l2_hevc_dpb_entry`
       - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
@@ -3646,3 +3671,74 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
     so this has to come from client.
     This is applicable to H264 and valid Range is from 0 to 63.
     Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
+
+``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
+    Specifies various decode parameters, especially the references picture order
+    count (POC) for all the lists (short, long, before, current, after) and the
+    number of entries for each of them.
+    These parameters are defined according to :ref:`hevc`.
+    They are described in section 8.3 "Slice decoding process" of the
+    specification.
+
+.. c:type:: v4l2_ctrl_hevc_decode_params
+
+.. cssclass:: longtable
+
+.. flat-table:: struct v4l2_ctrl_hevc_decode_params
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       1 1 2
+
+    * - __s32
+      - ``pic_order_cnt_val``
+      -
+    * - __u8
+      - ``num_active_dpb_entries``
+      - The number of entries in ``dpb``.
+    * - struct :c:type:`v4l2_hevc_dpb_entry`
+      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
+      - The decoded picture buffer, for meta-data about reference frames.
+    * - __u8
+      - ``num_rps_poc_st_curr_before``
+      - The number of reference pictures in the short-term set that come before
+        the current frame.
+    * - __u8
+      - ``num_rps_poc_st_curr_after``
+      - The number of reference pictures in the short-term set that come after
+        the current frame.
+    * - __u8
+      - ``num_rps_poc_lt_curr``
+      - The number of reference pictures in the long-term set.
+    * - __u8
+      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
+      -
+    * - __u8
+      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
+      -
+    * - __u8
+      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
+      -
+    * - __u64
+      - ``flags``
+      - See :ref:`Decode Parameters Flags <hevc_decode_params_flags>`
+
+.. _hevc_decode_params_flags:
+
+``Decode Parameters Flags``
+
+.. cssclass:: longtable
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       1 1 2
+
+    * - ``V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC``
+      - 0x00000001
+      -
+    * - ``V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC``
+      - 0x00000002
+      -
+    * - ``V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR``
+      - 0x00000004
+      -
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 82f61f1e2fb8..d84ae255bc79 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -486,6 +486,12 @@ See also the examples in :ref:`control`.
       - n/a
       - A struct :c:type:`v4l2_ctrl_hevc_slice_params`, containing HEVC
 	slice parameters for stateless video decoders.
+    * - ``V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS``
+      - n/a
+      - n/a
+      - n/a
+      - A struct :c:type:`v4l2_ctrl_hevc_decode_params`, containing HEVC
+	decoding parameters for stateless video decoders.
 
 .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
 
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 016cf6204cbb..4060b5bcc3c0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1028,6 +1028,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_MPEG_VIDEO_HEVC_SPS:			return "HEVC Sequence Parameter Set";
 	case V4L2_CID_MPEG_VIDEO_HEVC_PPS:			return "HEVC Picture Parameter Set";
 	case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:		return "HEVC Slice Parameters";
+	case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:		return "HEVC Decode Parameters";
 	case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:		return "HEVC Decode Mode";
 	case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:		return "HEVC Start Code";
 
@@ -1482,6 +1483,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
 		*type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
 		break;
+	case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
+		*type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
+		break;
 	case V4L2_CID_UNIT_CELL_SIZE:
 		*type = V4L2_CTRL_TYPE_AREA;
 		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
@@ -1833,6 +1837,7 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
 	struct v4l2_ctrl_hevc_sps *p_hevc_sps;
 	struct v4l2_ctrl_hevc_pps *p_hevc_pps;
 	struct v4l2_ctrl_hevc_slice_params *p_hevc_slice_params;
+	struct v4l2_ctrl_hevc_decode_params *p_hevc_decode_params;
 	struct v4l2_area *area;
 	void *p = ptr.p + idx * ctrl->elem_size;
 	unsigned int i;
@@ -2108,23 +2113,27 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx,
 		zero_padding(*p_hevc_pps);
 		break;
 
-	case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
-		p_hevc_slice_params = p;
+	case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
+		p_hevc_decode_params = p;
 
-		if (p_hevc_slice_params->num_active_dpb_entries >
+		if (p_hevc_decode_params->num_active_dpb_entries >
 		    V4L2_HEVC_DPB_ENTRIES_NUM_MAX)
 			return -EINVAL;
 
-		zero_padding(p_hevc_slice_params->pred_weight_table);
-
-		for (i = 0; i < p_hevc_slice_params->num_active_dpb_entries;
+		for (i = 0; i < p_hevc_decode_params->num_active_dpb_entries;
 		     i++) {
 			struct v4l2_hevc_dpb_entry *dpb_entry =
-				&p_hevc_slice_params->dpb[i];
+				&p_hevc_decode_params->dpb[i];
 
 			zero_padding(*dpb_entry);
 		}
 
+		break;
+
+	case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
+		p_hevc_slice_params = p;
+
+		zero_padding(p_hevc_slice_params->pred_weight_table);
 		zero_padding(*p_hevc_slice_params);
 		break;
 
@@ -2821,6 +2830,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 	case V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS:
 		elem_size = sizeof(struct v4l2_ctrl_hevc_slice_params);
 		break;
+	case V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS:
+		elem_size = sizeof(struct v4l2_ctrl_hevc_decode_params);
+		break;
 	case V4L2_CTRL_TYPE_AREA:
 		elem_size = sizeof(struct v4l2_area);
 		break;
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index 7bd9291c8d5f..4cd3cab1a257 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -151,6 +151,12 @@ static const struct cedrus_control cedrus_controls[] = {
 		},
 		.codec		= CEDRUS_CODEC_VP8,
 	},
+	{
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
+		},
+		.codec		= CEDRUS_CODEC_H265,
+	},
 };
 
 #define CEDRUS_CONTROLS_COUNT	ARRAY_SIZE(cedrus_controls)
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h
index 251a6a660351..2ca33ac38b9a 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.h
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.h
@@ -76,6 +76,7 @@ struct cedrus_h265_run {
 	const struct v4l2_ctrl_hevc_sps			*sps;
 	const struct v4l2_ctrl_hevc_pps			*pps;
 	const struct v4l2_ctrl_hevc_slice_params	*slice_params;
+	const struct v4l2_ctrl_hevc_decode_params	*decode_params;
 };
 
 struct cedrus_vp8_run {
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
index a9090daf626a..cd821f417a14 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
@@ -68,6 +68,8 @@ void cedrus_device_run(void *priv)
 			V4L2_CID_MPEG_VIDEO_HEVC_PPS);
 		run.h265.slice_params = cedrus_find_control_data(ctx,
 			V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS);
+		run.h265.decode_params = cedrus_find_control_data(ctx,
+			V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
 		break;
 
 	case V4L2_PIX_FMT_VP8_FRAME:
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
index ce497d0197df..dce5db6be13a 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
@@ -245,6 +245,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 	const struct v4l2_ctrl_hevc_sps *sps;
 	const struct v4l2_ctrl_hevc_pps *pps;
 	const struct v4l2_ctrl_hevc_slice_params *slice_params;
+	const struct v4l2_ctrl_hevc_decode_params *decode_params;
 	const struct v4l2_hevc_pred_weight_table *pred_weight_table;
 	dma_addr_t src_buf_addr;
 	dma_addr_t src_buf_end_addr;
@@ -256,6 +257,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 	sps = run->h265.sps;
 	pps = run->h265.pps;
 	slice_params = run->h265.slice_params;
+	decode_params = run->h265.decode_params;
 	pred_weight_table = &slice_params->pred_weight_table;
 
 	/* MV column buffer size and allocation. */
@@ -487,7 +489,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 
 	reg = VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_TC_OFFSET_DIV2(slice_params->slice_tc_offset_div2) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_BETA_OFFSET_DIV2(slice_params->slice_beta_offset_div2) |
-	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_POC_BIGEST_IN_RPS_ST(slice_params->num_rps_poc_st_curr_after == 0) |
+	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_POC_BIGEST_IN_RPS_ST(decode_params->num_rps_poc_st_curr_after == 0) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_CR_QP_OFFSET(slice_params->slice_cr_qp_offset) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_CB_QP_OFFSET(slice_params->slice_cb_qp_offset) |
 	      VE_DEC_H265_DEC_SLICE_HDR_INFO1_SLICE_QP_DELTA(slice_params->slice_qp_delta);
@@ -528,7 +530,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
 
 	/* Write decoded picture buffer in pic list. */
 	cedrus_h265_frame_info_write_dpb(ctx, slice_params->dpb,
-					 slice_params->num_active_dpb_entries);
+					 decode_params->num_active_dpb_entries);
 
 	/* Output frame. */
 
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
index b4cb2ef02f17..7fe704a08f77 100644
--- a/include/media/hevc-ctrls.h
+++ b/include/media/hevc-ctrls.h
@@ -19,6 +19,7 @@
 #define V4L2_CID_MPEG_VIDEO_HEVC_SPS		(V4L2_CID_CODEC_BASE + 1008)
 #define V4L2_CID_MPEG_VIDEO_HEVC_PPS		(V4L2_CID_CODEC_BASE + 1009)
 #define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS	(V4L2_CID_CODEC_BASE + 1010)
+#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS	(V4L2_CID_CODEC_BASE + 1012)
 #define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE	(V4L2_CID_CODEC_BASE + 1015)
 #define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE	(V4L2_CID_CODEC_BASE + 1016)
 
@@ -26,6 +27,7 @@
 #define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
 #define V4L2_CTRL_TYPE_HEVC_PPS 0x0121
 #define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122
+#define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
 
 enum v4l2_mpeg_video_hevc_decode_mode {
 	V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
@@ -54,6 +56,9 @@ enum v4l2_mpeg_video_hevc_start_code {
 /* The controls are not stable at the moment and will likely be reworked. */
 struct v4l2_ctrl_hevc_sps {
 	/* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
+	__u8	video_parameter_set_id;
+	__u8	seq_parameter_set_id;
+	__u8	chroma_format_idc;
 	__u16	pic_width_in_luma_samples;
 	__u16	pic_height_in_luma_samples;
 	__u8	bit_depth_luma_minus8;
@@ -74,9 +79,9 @@ struct v4l2_ctrl_hevc_sps {
 	__u8	log2_diff_max_min_pcm_luma_coding_block_size;
 	__u8	num_short_term_ref_pic_sets;
 	__u8	num_long_term_ref_pics_sps;
-	__u8	chroma_format_idc;
 
-	__u8	padding;
+	__u8	num_slices;
+	__u8	padding[6];
 
 	__u64	flags;
 };
@@ -100,10 +105,15 @@ struct v4l2_ctrl_hevc_sps {
 #define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER	(1ULL << 16)
 #define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT		(1ULL << 17)
 #define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL << 18)
+#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT	(1ULL << 19)
+#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING			(1ULL << 20)
 
 struct v4l2_ctrl_hevc_pps {
 	/* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
+	__u8	pic_parameter_set_id;
 	__u8	num_extra_slice_header_bits;
+	__u8	num_ref_idx_l0_default_active_minus1;
+	__u8	num_ref_idx_l1_default_active_minus1;
 	__s8	init_qp_minus26;
 	__u8	diff_cu_qp_delta_depth;
 	__s8	pps_cb_qp_offset;
@@ -116,7 +126,7 @@ struct v4l2_ctrl_hevc_pps {
 	__s8	pps_tc_offset_div2;
 	__u8	log2_parallel_merge_level_minus2;
 
-	__u8	padding[4];
+	__u8	padding;
 	__u64	flags;
 };
 
@@ -165,6 +175,10 @@ struct v4l2_ctrl_hevc_slice_params {
 	__u32	bit_size;
 	__u32	data_bit_offset;
 
+	/* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
+	__u32	slice_segment_addr;
+	__u32	num_entry_point_offsets;
+
 	/* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
 	__u8	nal_unit_type;
 	__u8	nuh_temporal_id_plus1;
@@ -190,15 +204,13 @@ struct v4l2_ctrl_hevc_slice_params {
 	__u8	pic_struct;
 
 	/* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
-	__u8	num_active_dpb_entries;
 	__u8	ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
 	__u8	ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
 
-	__u8	num_rps_poc_st_curr_before;
-	__u8	num_rps_poc_st_curr_after;
-	__u8	num_rps_poc_lt_curr;
+	__u16	short_term_ref_pic_set_size;
+	__u16	long_term_ref_pic_set_size;
 
-	__u8	padding;
+	__u8	padding[5];
 
 	/* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
 	struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
@@ -209,4 +221,21 @@ struct v4l2_ctrl_hevc_slice_params {
 	__u64	flags;
 };
 
+#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC		0x1
+#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC		0x2
+#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR  0x4
+
+struct v4l2_ctrl_hevc_decode_params {
+	__s32	pic_order_cnt_val;
+	__u8	num_active_dpb_entries;
+	struct	v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+	__u8	num_rps_poc_st_curr_before;
+	__u8	num_rps_poc_st_curr_after;
+	__u8	num_rps_poc_lt_curr;
+	__u8	rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+	__u8	rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+	__u8	rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+	__u64	flags;
+};
+
 #endif
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 2/9] media: hantro: Define HEVC codec profiles and supported features

[Benjamin Gaignard]

Define which HEVC profiles (up to level 5.1) and features
(no scaling, no 10 bits) are supported by the driver.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
 drivers/staging/media/hantro/hantro.h     |  2 +
 drivers/staging/media/hantro/hantro_drv.c | 58 +++++++++++++++++++++++
 2 files changed, 60 insertions(+)

diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index 65f9f7ea7dcf..bde65231f22f 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -99,6 +99,7 @@ struct hantro_variant {
  * @HANTRO_MODE_H264_DEC: H264 decoder.
  * @HANTRO_MODE_MPEG2_DEC: MPEG-2 decoder.
  * @HANTRO_MODE_VP8_DEC: VP8 decoder.
+ * @HANTRO_MODE_HEVC_DEC: HEVC decoder.
  */
 enum hantro_codec_mode {
 	HANTRO_MODE_NONE = -1,
@@ -106,6 +107,7 @@ enum hantro_codec_mode {
 	HANTRO_MODE_H264_DEC,
 	HANTRO_MODE_MPEG2_DEC,
 	HANTRO_MODE_VP8_DEC,
+	HANTRO_MODE_HEVC_DEC,
 };
 
 /*
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index e5f200e64993..d86e322a5980 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -243,6 +243,18 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
 		if (sps->bit_depth_luma_minus8 != 0)
 			/* Only 8-bit is supported */
 			return -EINVAL;
+	} else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
+		const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
+
+		if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
+			/* Luma and chroma bit depth mismatch */
+			return -EINVAL;
+		if (sps->bit_depth_luma_minus8 != 0)
+			/* Only 8-bit is supported */
+			return -EINVAL;
+		if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
+			/* No scaling support */
+			return -EINVAL;
 	}
 	return 0;
 }
@@ -349,6 +361,52 @@ static const struct hantro_ctrl controls[] = {
 			.def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
 		}
 	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
+			.min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
+			.max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
+			.def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
+		},
+	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
+			.min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
+			.max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
+			.def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
+		},
+	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+			.min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+			.max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
+			.def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
+		},
+	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+			.min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
+			.max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
+		},
+	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
+			.ops = &hantro_ctrl_ops,
+		},
+	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
+		},
+	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
+		},
 	},
 };
 
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 3/9] media: hantro: Add a field to distinguish the hardware versions

[Benjamin Gaignard]

Decoders hardware blocks could exist in multiple versions: add
a field to distinguish them at runtime.
G2 hardware block doesn't have postprocessor hantro_needs_postproc
function should always returns false in for this hardware.
hantro_needs_postproc function becoming to much complex to
stay inline in .h file move it to .c file.

Keep the default behavoir to be G1 hardware.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2:
- squash this commit with the post processor change
to show one usage of core_hw_dec_rev

 drivers/staging/media/hantro/hantro.h          | 13 +++++++------
 drivers/staging/media/hantro/hantro_drv.c      |  2 ++
 drivers/staging/media/hantro/hantro_postproc.c | 17 +++++++++++++++++
 3 files changed, 26 insertions(+), 6 deletions(-)

diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index bde65231f22f..352930bd49fc 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -36,6 +36,9 @@ struct hantro_codec_ops;
 #define HANTRO_H264_DECODER	BIT(18)
 #define HANTRO_DECODERS		0xffff0000
 
+#define HANTRO_G1_REV		0x6731
+#define HANTRO_G2_REV		0x6732
+
 /**
  * struct hantro_irq - irq handler and name
  *
@@ -170,6 +173,7 @@ hantro_vdev_to_func(struct video_device *vdev)
  * @enc_base:		Mapped address of VPU encoder register for convenience.
  * @dec_base:		Mapped address of VPU decoder register for convenience.
  * @ctrl_base:		Mapped address of VPU control block.
+ * @core_hw_dec_rev	Runtime detected HW decoder core revision
  * @vpu_mutex:		Mutex to synchronize V4L2 calls.
  * @irqlock:		Spinlock to synchronize access to data structures
  *			shared with interrupt handlers.
@@ -189,6 +193,7 @@ struct hantro_dev {
 	void __iomem *enc_base;
 	void __iomem *dec_base;
 	void __iomem *ctrl_base;
+	u32 core_hw_dec_rev;
 
 	struct mutex vpu_mutex;	/* video_device lock */
 	spinlock_t irqlock;
@@ -411,12 +416,8 @@ hantro_get_dst_buf(struct hantro_ctx *ctx)
 	return v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 }
 
-static inline bool
-hantro_needs_postproc(const struct hantro_ctx *ctx,
-		      const struct hantro_fmt *fmt)
-{
-	return !ctx->is_encoder && fmt->fourcc != V4L2_PIX_FMT_NV12;
-}
+bool hantro_needs_postproc(const struct hantro_ctx *ctx,
+			   const struct hantro_fmt *fmt);
 
 static inline dma_addr_t
 hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index d86e322a5980..0a62beb0bfbd 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -834,6 +834,8 @@ static int hantro_probe(struct platform_device *pdev)
 	}
 	vpu->enc_base = vpu->reg_bases[0] + vpu->variant->enc_offset;
 	vpu->dec_base = vpu->reg_bases[0] + vpu->variant->dec_offset;
+	/* by default decoder is G1 */
+	vpu->core_hw_dec_rev = HANTRO_G1_REV;
 
 	ret = dma_set_coherent_mask(vpu->dev, DMA_BIT_MASK(32));
 	if (ret) {
diff --git a/drivers/staging/media/hantro/hantro_postproc.c b/drivers/staging/media/hantro/hantro_postproc.c
index 6d2a8f2a8f0b..050880f720d6 100644
--- a/drivers/staging/media/hantro/hantro_postproc.c
+++ b/drivers/staging/media/hantro/hantro_postproc.c
@@ -50,6 +50,23 @@ const struct hantro_postproc_regs hantro_g1_postproc_regs = {
 	.display_width = {G1_REG_PP_DISPLAY_WIDTH, 0, 0xfff},
 };
 
+bool hantro_needs_postproc(const struct hantro_ctx *ctx,
+			   const struct hantro_fmt *fmt)
+{
+	struct hantro_dev *vpu = ctx->dev;
+
+	if (ctx->is_encoder)
+		return false;
+
+	if (vpu->core_hw_dec_rev == HANTRO_G1_REV)
+		return fmt->fourcc != V4L2_PIX_FMT_NV12;
+
+	if (vpu->core_hw_dec_rev == HANTRO_G2_REV)
+		return false;
+
+	return false;
+}
+
 void hantro_postproc_enable(struct hantro_ctx *ctx)
 {
 	struct hantro_dev *vpu = ctx->dev;
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 4/9] media: uapi: Add a control for HANTRO driver

[Benjamin Gaignard]

The HEVC HANTRO driver needs to know the number of bits to skip at
the beginning of the slice header.
That is a hardware specific requirement so create a dedicated control
that this purpose.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 3:
- Fix typo in field name

 include/uapi/linux/hantro-v4l2-controls.h | 20 ++++++++++++++++++++
 include/uapi/linux/v4l2-controls.h        |  5 +++++
 2 files changed, 25 insertions(+)
 create mode 100644 include/uapi/linux/hantro-v4l2-controls.h

diff --git a/include/uapi/linux/hantro-v4l2-controls.h b/include/uapi/linux/hantro-v4l2-controls.h
new file mode 100644
index 000000000000..a8dfd6b1a2a9
--- /dev/null
+++ b/include/uapi/linux/hantro-v4l2-controls.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+
+#ifndef __UAPI_HANTRO_V4L2_CONYTROLS_H__
+#define __UAPI_HANTRO_V4L2_CONYTROLS_H__
+
+#include <linux/v4l2-controls.h>
+#include <media/hevc-ctrls.h>
+
+#define V4L2_CID_HANTRO_HEVC_EXTRA_DECODE_PARAMS	(V4L2_CID_USER_HANTRO_BASE + 0)
+
+/**
+ * struct hantro_hevc_extra_decode_params - extra decode parameters for hantro driver
+ * @hevc_hdr_skip_length:	header first bits offset
+ */
+struct hantro_hevc_extra_decode_params {
+	__u32	hevc_hdr_skip_length;
+	__u8	padding[4];
+};
+
+#endif
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 039c0d7add1b..ced7486c7f46 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -209,6 +209,11 @@ enum v4l2_colorfx {
  * We reserve 128 controls for this driver.
  */
 #define V4L2_CID_USER_CCS_BASE			(V4L2_CID_USER_BASE + 0x10f0)
+/*
+ * The base for HANTRO driver controls.
+ * We reserve 32 controls for this driver.
+ */
+#define V4L2_CID_USER_HANTRO_BASE		(V4L2_CID_USER_BASE + 0x1170)
 
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 5/9] media: hantro: Introduce G2/HEVC decoder

[Benjamin Gaignard]

Implement all the logic to get G2 hardware decoding HEVC frames.
It support up level 5.1 HEVC stream.
It doesn't support yet 10 bits formats or scaling feature.

Add HANTRO HEVC dedicated control to skip some bits at the beginning
of the slice header. That is very specific to this hardware so can't
go into uapi structures. Compute the needed value is complex and require
information from the stream that only the userland knows so let it
provide the correct value to the driver.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2:
- squash multiple commits in this one.
- fix the comments done by Ezequiel about dma_alloc_coherent usage
- fix Dan's comments about control copy, reverse the test logic
in tile_buffer_reallocate, rework some goto and return cases.

 drivers/staging/media/hantro/Makefile         |   2 +
 drivers/staging/media/hantro/hantro.h         |  19 +
 drivers/staging/media/hantro/hantro_drv.c     |  42 ++
 .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++++++++++++++++++
 drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++++++
 drivers/staging/media/hantro/hantro_hevc.c    | 321 ++++++++++
 drivers/staging/media/hantro/hantro_hw.h      |  47 ++
 7 files changed, 1216 insertions(+)
 create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
 create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
 create mode 100644 drivers/staging/media/hantro/hantro_hevc.c

diff --git a/drivers/staging/media/hantro/Makefile b/drivers/staging/media/hantro/Makefile
index 743ce08eb184..0357f1772267 100644
--- a/drivers/staging/media/hantro/Makefile
+++ b/drivers/staging/media/hantro/Makefile
@@ -9,12 +9,14 @@ hantro-vpu-y += \
 		hantro_h1_jpeg_enc.o \
 		hantro_g1_h264_dec.o \
 		hantro_g1_mpeg2_dec.o \
+		hantro_g2_hevc_dec.o \
 		hantro_g1_vp8_dec.o \
 		rk3399_vpu_hw_jpeg_enc.o \
 		rk3399_vpu_hw_mpeg2_dec.o \
 		rk3399_vpu_hw_vp8_dec.o \
 		hantro_jpeg.o \
 		hantro_h264.o \
+		hantro_hevc.o \
 		hantro_mpeg2.o \
 		hantro_vp8.o
 
diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
index 352930bd49fc..a9b80b2c9124 100644
--- a/drivers/staging/media/hantro/hantro.h
+++ b/drivers/staging/media/hantro/hantro.h
@@ -34,6 +34,7 @@ struct hantro_codec_ops;
 #define HANTRO_MPEG2_DECODER	BIT(16)
 #define HANTRO_VP8_DECODER	BIT(17)
 #define HANTRO_H264_DECODER	BIT(18)
+#define HANTRO_HEVC_DECODER	BIT(19)
 #define HANTRO_DECODERS		0xffff0000
 
 #define HANTRO_G1_REV		0x6731
@@ -224,6 +225,7 @@ struct hantro_dev {
  * @jpeg_enc:		JPEG-encoding context.
  * @mpeg2_dec:		MPEG-2-decoding context.
  * @vp8_dec:		VP8-decoding context.
+ * @hevc_dec:		HEVC-decoding context.
  */
 struct hantro_ctx {
 	struct hantro_dev *dev;
@@ -250,6 +252,7 @@ struct hantro_ctx {
 		struct hantro_jpeg_enc_hw_ctx jpeg_enc;
 		struct hantro_mpeg2_dec_hw_ctx mpeg2_dec;
 		struct hantro_vp8_dec_hw_ctx vp8_dec;
+		struct hantro_hevc_dec_hw_ctx hevc_dec;
 	};
 };
 
@@ -427,6 +430,22 @@ hantro_get_dec_buf_addr(struct hantro_ctx *ctx, struct vb2_buffer *vb)
 	return vb2_dma_contig_plane_dma_addr(vb, 0);
 }
 
+static inline size_t
+hantro_get_dec_buf_size(struct hantro_ctx *ctx, struct vb2_buffer *vb)
+{
+	if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
+		return ctx->postproc.dec_q[vb->index].size;
+	return vb2_plane_size(vb, 0);
+}
+
+static inline void *
+hantro_get_dec_buf(struct hantro_ctx *ctx, struct vb2_buffer *vb)
+{
+	if (hantro_needs_postproc(ctx, ctx->vpu_dst_fmt))
+		return ctx->postproc.dec_q[vb->index].cpu;
+	return vb2_plane_vaddr(vb, 0);
+}
+
 void hantro_postproc_disable(struct hantro_ctx *ctx);
 void hantro_postproc_enable(struct hantro_ctx *ctx);
 void hantro_postproc_free(struct hantro_ctx *ctx);
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index 0a62beb0bfbd..c009941dff3e 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -279,6 +279,21 @@ static int hantro_jpeg_s_ctrl(struct v4l2_ctrl *ctrl)
 	return 0;
 }
 
+static int hantro_extra_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct hantro_hevc_extra_decode_params *extra_params;
+	struct hantro_ctx *ctx;
+
+	ctx = container_of(ctrl->handler,
+			   struct hantro_ctx, ctrl_handler);
+	extra_params =
+		(struct hantro_hevc_extra_decode_params *)&ctx->hevc_dec.ctrls.extra_params;
+
+	memcpy(extra_params, ctrl->p_new.p_u8, sizeof(*extra_params));
+
+	return 0;
+}
+
 static const struct v4l2_ctrl_ops hantro_ctrl_ops = {
 	.try_ctrl = hantro_try_ctrl,
 };
@@ -287,6 +302,10 @@ static const struct v4l2_ctrl_ops hantro_jpeg_ctrl_ops = {
 	.s_ctrl = hantro_jpeg_s_ctrl,
 };
 
+static const struct v4l2_ctrl_ops hantro_extra_ctrl_ops = {
+	.s_ctrl = hantro_extra_s_ctrl,
+};
+
 static const struct hantro_ctrl controls[] = {
 	{
 		.codec = HANTRO_JPEG_ENCODER,
@@ -407,6 +426,29 @@ static const struct hantro_ctrl controls[] = {
 		.cfg = {
 			.id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
 		},
+	}, {
+		.codec = HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_HANTRO_HEVC_EXTRA_DECODE_PARAMS,
+			.name = "HANTRO extra decode params",
+			.type = V4L2_CTRL_TYPE_U8,
+			.min = 0,
+			.def = 0,
+			.max = 255,
+			.step = 1,
+			.dims = { sizeof(struct hantro_hevc_extra_decode_params) },
+			.ops = &hantro_extra_ctrl_ops,
+		},
+	}, {
+		.codec = HANTRO_JPEG_ENCODER | HANTRO_MPEG2_DECODER |
+			 HANTRO_VP8_DECODER | HANTRO_H264_DECODER |
+			 HANTRO_HEVC_DECODER,
+		.cfg = {
+			.id = V4L2_CID_USER_CLASS,
+			.name = "HANTRO controls",
+			.type = V4L2_CTRL_TYPE_CTRL_CLASS,
+			.flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY,
+		},
 	},
 };
 
diff --git a/drivers/staging/media/hantro/hantro_g2_hevc_dec.c b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
new file mode 100644
index 000000000000..ab91dab04a3c
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_g2_hevc_dec.c
@@ -0,0 +1,587 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VPU HEVC codec driver
+ *
+ * Copyright (C) 2020 Safran Passenger Innovations LLC
+ */
+
+#include "hantro_hw.h"
+#include "hantro_g2_regs.h"
+
+#define HEVC_DEC_MODE	0xC
+
+#define BUS_WIDTH_32		0
+#define BUS_WIDTH_64		1
+#define BUS_WIDTH_128		2
+#define BUS_WIDTH_256		3
+
+static inline void hantro_write_addr(struct hantro_dev *vpu,
+				     unsigned long offset,
+				     dma_addr_t addr)
+{
+	vdpu_write(vpu, addr & 0xffffffff, offset);
+}
+
+static void prepare_tile_info_buffer(struct hantro_ctx *ctx)
+{
+	struct hantro_dev *vpu = ctx->dev;
+	const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+	const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
+	const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+	u16 *p = (u16 *)((u8 *)ctx->hevc_dec.tile_sizes.cpu);
+	unsigned int num_tile_rows = pps->num_tile_rows_minus1 + 1;
+	unsigned int num_tile_cols = pps->num_tile_columns_minus1 + 1;
+	unsigned int pic_width_in_ctbs, pic_height_in_ctbs;
+	unsigned int max_log2_ctb_size, ctb_size;
+	bool tiles_enabled, uniform_spacing;
+	u32 no_chroma = 0;
+
+	tiles_enabled = !!(pps->flags & V4L2_HEVC_PPS_FLAG_TILES_ENABLED);
+	uniform_spacing = !!(pps->flags & V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING);
+
+	hantro_reg_write(vpu, hevc_tile_e, tiles_enabled);
+
+	max_log2_ctb_size = sps->log2_min_luma_coding_block_size_minus3 + 3 +
+			    sps->log2_diff_max_min_luma_coding_block_size;
+	pic_width_in_ctbs = (sps->pic_width_in_luma_samples +
+			    (1 << max_log2_ctb_size) - 1) >> max_log2_ctb_size;
+	pic_height_in_ctbs = (sps->pic_height_in_luma_samples + (1 << max_log2_ctb_size) - 1)
+			     >> max_log2_ctb_size;
+	ctb_size = 1 << max_log2_ctb_size;
+
+	vpu_debug(1, "Preparing tile sizes buffer for %dx%d CTBs (CTB size %d)\n",
+		  pic_width_in_ctbs, pic_height_in_ctbs, ctb_size);
+
+	if (tiles_enabled) {
+		unsigned int i, j, h;
+
+		vpu_debug(1, "Tiles enabled! %dx%d\n", num_tile_cols, num_tile_rows);
+
+		hantro_reg_write(vpu, hevc_num_tile_rows, num_tile_rows);
+		hantro_reg_write(vpu, hevc_num_tile_cols, num_tile_cols);
+
+		/* write width + height for each tile in pic */
+		if (!uniform_spacing) {
+			u32 tmp_w = 0, tmp_h = 0;
+
+			for (i = 0; i < num_tile_rows; i++) {
+				if (i == num_tile_rows - 1)
+					h = pic_height_in_ctbs - tmp_h;
+				else
+					h = pps->row_height_minus1[i] + 1;
+				tmp_h += h;
+				if (i == 0 && h == 1 && ctb_size == 16)
+					no_chroma = 1;
+				for (j = 0, tmp_w = 0; j < num_tile_cols - 1; j++) {
+					tmp_w += pps->column_width_minus1[j] + 1;
+					*p++ = pps->column_width_minus1[j + 1];
+					*p++ = h;
+					if (i == 0 && h == 1 && ctb_size == 16)
+						no_chroma = 1;
+				}
+				/* last column */
+				*p++ = pic_width_in_ctbs - tmp_w;
+				*p++ = h;
+			}
+		} else { /* uniform spacing */
+			u32 tmp, prev_h, prev_w;
+
+			for (i = 0, prev_h = 0; i < num_tile_rows; i++) {
+				tmp = (i + 1) * pic_height_in_ctbs / num_tile_rows;
+				h = tmp - prev_h;
+				prev_h = tmp;
+				if (i == 0 && h == 1 && ctb_size == 16)
+					no_chroma = 1;
+				for (j = 0, prev_w = 0; j < num_tile_cols; j++) {
+					tmp = (j + 1) * pic_width_in_ctbs / num_tile_cols;
+					*p++ = tmp - prev_w;
+					*p++ = h;
+					if (j == 0 &&
+					    (pps->column_width_minus1[0] + 1) == 1 &&
+					    ctb_size == 16)
+						no_chroma = 1;
+					prev_w = tmp;
+				}
+			}
+		}
+	} else {
+		hantro_reg_write(vpu, hevc_num_tile_rows, 1);
+		hantro_reg_write(vpu, hevc_num_tile_cols, 1);
+
+		/* There's one tile, with dimensions equal to pic size. */
+		p[0] = pic_width_in_ctbs;
+		p[1] = pic_height_in_ctbs;
+	}
+
+	if (no_chroma)
+		vpu_debug(1, "%s: no chroma!\n", __func__);
+}
+
+static void set_params(struct hantro_ctx *ctx)
+{
+	const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+	const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+	const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
+	const struct v4l2_ctrl_hevc_decode_params *decode_params = ctrls->decode_params;
+	const struct hantro_hevc_extra_decode_params *extra_params = &ctrls->extra_params;
+	struct hantro_dev *vpu = ctx->dev;
+	u32 min_log2_cb_size, max_log2_ctb_size, min_cb_size, max_ctb_size;
+	u32 pic_width_in_min_cbs, pic_height_in_min_cbs;
+	u32 pic_width_aligned, pic_height_aligned;
+	u32 partial_ctb_x, partial_ctb_y;
+
+	hantro_reg_write(vpu, hevc_bit_depth_y_minus8, sps->bit_depth_luma_minus8);
+	hantro_reg_write(vpu, hevc_bit_depth_c_minus8, sps->bit_depth_chroma_minus8);
+
+	hantro_reg_write(vpu, hevc_output_8_bits, 0);
+
+	hantro_reg_write(vpu, hevc_hdr_skip_length, extra_params->hevc_hdr_skip_length);
+
+	min_log2_cb_size = sps->log2_min_luma_coding_block_size_minus3 + 3;
+	max_log2_ctb_size = min_log2_cb_size + sps->log2_diff_max_min_luma_coding_block_size;
+
+	hantro_reg_write(vpu, hevc_min_cb_size, min_log2_cb_size);
+	hantro_reg_write(vpu, hevc_max_cb_size, max_log2_ctb_size);
+
+	min_cb_size = 1 << min_log2_cb_size;
+	max_ctb_size = 1 << max_log2_ctb_size;
+
+	pic_width_in_min_cbs = sps->pic_width_in_luma_samples / min_cb_size;
+	pic_height_in_min_cbs = sps->pic_height_in_luma_samples / min_cb_size;
+	pic_width_aligned = ALIGN(sps->pic_width_in_luma_samples, max_ctb_size);
+	pic_height_aligned = ALIGN(sps->pic_height_in_luma_samples, max_ctb_size);
+
+	partial_ctb_x = !!(sps->pic_width_in_luma_samples != pic_width_aligned);
+	partial_ctb_y = !!(sps->pic_height_in_luma_samples != pic_height_aligned);
+
+	hantro_reg_write(vpu, hevc_partial_ctb_x, partial_ctb_x);
+	hantro_reg_write(vpu, hevc_partial_ctb_y, partial_ctb_y);
+
+	hantro_reg_write(vpu, hevc_pic_width_in_cbs, pic_width_in_min_cbs);
+	hantro_reg_write(vpu, hevc_pic_height_in_cbs, pic_height_in_min_cbs);
+
+	hantro_reg_write(vpu, hevc_pic_width_4x4,
+			 (pic_width_in_min_cbs * min_cb_size) / 4);
+	hantro_reg_write(vpu, hevc_pic_height_4x4,
+			 (pic_height_in_min_cbs * min_cb_size) / 4);
+
+	hantro_reg_write(vpu, hevc_max_inter_hierdepth,
+			 sps->max_transform_hierarchy_depth_inter);
+	hantro_reg_write(vpu, hevc_max_intra_hierdepth,
+			 sps->max_transform_hierarchy_depth_intra);
+	hantro_reg_write(vpu, hevc_min_trb_size,
+			 sps->log2_min_luma_transform_block_size_minus2 + 2);
+	hantro_reg_write(vpu, hevc_max_trb_size,
+			 sps->log2_min_luma_transform_block_size_minus2 + 2 +
+			 sps->log2_diff_max_min_luma_transform_block_size);
+
+	hantro_reg_write(vpu, hevc_tempor_mvp_e,
+			 !!(sps->flags & V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED) &&
+			 !(decode_params->flags & V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC));
+	hantro_reg_write(vpu, hevc_strong_smooth_e,
+			 !!(sps->flags & V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED));
+	hantro_reg_write(vpu, hevc_asym_pred_e,
+			 !!(sps->flags & V4L2_HEVC_SPS_FLAG_AMP_ENABLED));
+	hantro_reg_write(vpu, hevc_sao_e,
+			 !!(sps->flags & V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET));
+	hantro_reg_write(vpu, hevc_sign_data_hide,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED));
+
+	if (pps->flags & V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED) {
+		hantro_reg_write(vpu, hevc_cu_qpd_e, 1);
+		hantro_reg_write(vpu, hevc_max_cu_qpd_depth, pps->diff_cu_qp_delta_depth);
+	} else {
+		hantro_reg_write(vpu, hevc_cu_qpd_e, 0);
+		hantro_reg_write(vpu, hevc_max_cu_qpd_depth, 0);
+	}
+
+	if (pps->flags & V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT) {
+		hantro_reg_write(vpu, hevc_cb_qp_offset, pps->pps_cb_qp_offset);
+		hantro_reg_write(vpu, hevc_cr_qp_offset, pps->pps_cr_qp_offset);
+	} else {
+		hantro_reg_write(vpu, hevc_cb_qp_offset, 0);
+		hantro_reg_write(vpu, hevc_cr_qp_offset, 0);
+	}
+
+	hantro_reg_write(vpu, hevc_filt_offset_beta, pps->pps_beta_offset_div2);
+	hantro_reg_write(vpu, hevc_filt_offset_tc, pps->pps_tc_offset_div2);
+	hantro_reg_write(vpu, hevc_slice_hdr_ext_e,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT));
+	hantro_reg_write(vpu, hevc_slice_hdr_ext_bits, pps->num_extra_slice_header_bits);
+	hantro_reg_write(vpu, hevc_slice_chqp_present,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT));
+	hantro_reg_write(vpu, hevc_weight_bipr_idc,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED));
+	hantro_reg_write(vpu, hevc_transq_bypass,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED));
+	hantro_reg_write(vpu, hevc_list_mod_e,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT));
+	hantro_reg_write(vpu, hevc_entropy_sync_e,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED));
+	hantro_reg_write(vpu, hevc_cabac_init_present,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT));
+	hantro_reg_write(vpu, hevc_idr_pic_e,
+			 !!(decode_params->flags & V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC));
+	hantro_reg_write(vpu, hevc_parallel_merge,
+			 pps->log2_parallel_merge_level_minus2 + 2);
+	hantro_reg_write(vpu, hevc_pcm_filt_d,
+			 !!(sps->flags & V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED));
+	hantro_reg_write(vpu, hevc_pcm_e,
+			 !!(sps->flags & V4L2_HEVC_SPS_FLAG_PCM_ENABLED));
+	if (sps->flags & V4L2_HEVC_SPS_FLAG_PCM_ENABLED) {
+		hantro_reg_write(vpu, hevc_max_pcm_size,
+				 sps->log2_diff_max_min_pcm_luma_coding_block_size +
+				 sps->log2_min_pcm_luma_coding_block_size_minus3 + 3);
+		hantro_reg_write(vpu, hevc_min_pcm_size,
+				 sps->log2_min_pcm_luma_coding_block_size_minus3 + 3);
+		hantro_reg_write(vpu, hevc_bit_depth_pcm_y,
+				 sps->pcm_sample_bit_depth_luma_minus1 + 1);
+		hantro_reg_write(vpu, hevc_bit_depth_pcm_c,
+				 sps->pcm_sample_bit_depth_chroma_minus1 + 1);
+	} else {
+		hantro_reg_write(vpu, hevc_max_pcm_size, 0);
+		hantro_reg_write(vpu, hevc_min_pcm_size, 0);
+		hantro_reg_write(vpu, hevc_bit_depth_pcm_y, 0);
+		hantro_reg_write(vpu, hevc_bit_depth_pcm_c, 0);
+	}
+
+	hantro_reg_write(vpu, hevc_start_code_e, 1);
+	hantro_reg_write(vpu, hevc_init_qp, pps->init_qp_minus26 + 26);
+	hantro_reg_write(vpu, hevc_weight_pred_e,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED));
+	hantro_reg_write(vpu, hevc_cabac_init_present,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT));
+	hantro_reg_write(vpu, hevc_const_intra_e,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED));
+	hantro_reg_write(vpu, hevc_transform_skip,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED));
+	hantro_reg_write(vpu, hevc_out_filtering_dis,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER));
+	hantro_reg_write(vpu, hevc_filt_ctrl_pres,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT));
+	hantro_reg_write(vpu, hevc_dependent_slice,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT));
+	hantro_reg_write(vpu, hevc_filter_override,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED));
+	hantro_reg_write(vpu, hevc_refidx0_active,
+			 pps->num_ref_idx_l0_default_active_minus1 + 1);
+	hantro_reg_write(vpu, hevc_refidx1_active,
+			 pps->num_ref_idx_l1_default_active_minus1 + 1);
+	hantro_reg_write(vpu, hevc_apf_threshold, 8);
+}
+
+static int find_ref_pic_index(const struct v4l2_hevc_dpb_entry *dpb, int pic_order_cnt)
+{
+	int i;
+
+	for (i = 0; i < V4L2_HEVC_DPB_ENTRIES_NUM_MAX; i++) {
+		if (dpb[i].pic_order_cnt[0] == pic_order_cnt)
+			return i;
+	}
+
+	return 0x0;
+}
+
+static void set_ref_pic_list(struct hantro_ctx *ctx)
+{
+	const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+	struct hantro_dev *vpu = ctx->dev;
+	const struct v4l2_ctrl_hevc_decode_params *decode_params = ctrls->decode_params;
+	const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb;
+	u32 list0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX] = {0};
+	u32 list1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX] = {0};
+	const struct hantro_reg *ref_pic_regs0[] = {
+		hevc_rlist_f0,
+		hevc_rlist_f1,
+		hevc_rlist_f2,
+		hevc_rlist_f3,
+		hevc_rlist_f4,
+		hevc_rlist_f5,
+		hevc_rlist_f6,
+		hevc_rlist_f7,
+		hevc_rlist_f8,
+		hevc_rlist_f9,
+		hevc_rlist_f10,
+		hevc_rlist_f11,
+		hevc_rlist_f12,
+		hevc_rlist_f13,
+		hevc_rlist_f14,
+		hevc_rlist_f15,
+	};
+	const struct hantro_reg *ref_pic_regs1[] = {
+		hevc_rlist_b0,
+		hevc_rlist_b1,
+		hevc_rlist_b2,
+		hevc_rlist_b3,
+		hevc_rlist_b4,
+		hevc_rlist_b5,
+		hevc_rlist_b6,
+		hevc_rlist_b7,
+		hevc_rlist_b8,
+		hevc_rlist_b9,
+		hevc_rlist_b10,
+		hevc_rlist_b11,
+		hevc_rlist_b12,
+		hevc_rlist_b13,
+		hevc_rlist_b14,
+		hevc_rlist_b15,
+	};
+	unsigned int i, j;
+
+	/* List 0 contains: short term before, short term after and long term */
+	j = 0;
+	for (i = 0; i < decode_params->num_rps_poc_st_curr_before && j < ARRAY_SIZE(list0); i++)
+		list0[j++] = find_ref_pic_index(dpb, decode_params->rps_st_curr_before[i]);
+	for (i = 0; i < decode_params->num_rps_poc_st_curr_after && j < ARRAY_SIZE(list0); i++)
+		list0[j++] = find_ref_pic_index(dpb, decode_params->rps_st_curr_after[i]);
+	for (i = 0; i < decode_params->num_rps_poc_lt_curr && j < ARRAY_SIZE(list0); i++)
+		list0[j++] = find_ref_pic_index(dpb, decode_params->rps_lt_curr[i]);
+
+	/* Fill the list, copying over and over */
+	i = 0;
+	while (j < ARRAY_SIZE(list0))
+		list0[j++] = list0[i++];
+
+	j = 0;
+	for (i = 0; i < decode_params->num_rps_poc_st_curr_after && j < ARRAY_SIZE(list1); i++)
+		list1[j++] = find_ref_pic_index(dpb, decode_params->rps_st_curr_after[i]);
+	for (i = 0; i < decode_params->num_rps_poc_st_curr_before && j < ARRAY_SIZE(list1); i++)
+		list1[j++] = find_ref_pic_index(dpb, decode_params->rps_st_curr_before[i]);
+	for (i = 0; i < decode_params->num_rps_poc_lt_curr && j < ARRAY_SIZE(list1); i++)
+		list1[j++] = find_ref_pic_index(dpb, decode_params->rps_lt_curr[i]);
+
+	i = 0;
+	while (j < ARRAY_SIZE(list1))
+		list1[j++] = list1[i++];
+
+	for (i = 0; i < V4L2_HEVC_DPB_ENTRIES_NUM_MAX; i++) {
+		hantro_reg_write(vpu, ref_pic_regs0[i], list0[i]);
+		hantro_reg_write(vpu, ref_pic_regs1[i], list1[i]);
+	}
+}
+
+static int set_ref(struct hantro_ctx *ctx)
+{
+	const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+	const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+	const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
+	const struct v4l2_ctrl_hevc_decode_params *decode_params = ctrls->decode_params;
+	const struct v4l2_hevc_dpb_entry *dpb = decode_params->dpb;
+	dma_addr_t luma_addr, chroma_addr, mv_addr = 0;
+	struct hantro_dev *vpu = ctx->dev;
+	size_t cr_offset = hantro_hevc_chroma_offset(sps);
+	size_t mv_offset = hantro_hevc_motion_vectors_offset(sps);
+	u32 max_ref_frames;
+	u16 dpb_longterm_e;
+
+	const struct hantro_reg *cur_poc[] = {
+		hevc_cur_poc_00,
+		hevc_cur_poc_01,
+		hevc_cur_poc_02,
+		hevc_cur_poc_03,
+		hevc_cur_poc_04,
+		hevc_cur_poc_05,
+		hevc_cur_poc_06,
+		hevc_cur_poc_07,
+		hevc_cur_poc_08,
+		hevc_cur_poc_09,
+		hevc_cur_poc_10,
+		hevc_cur_poc_11,
+		hevc_cur_poc_12,
+		hevc_cur_poc_13,
+		hevc_cur_poc_14,
+		hevc_cur_poc_15,
+	};
+	unsigned int i;
+
+	max_ref_frames = decode_params->num_rps_poc_lt_curr +
+		decode_params->num_rps_poc_st_curr_before +
+		decode_params->num_rps_poc_st_curr_after;
+	/*
+	 * Set max_ref_frames to non-zero to avoid HW hang when decoding
+	 * badly marked I-frames.
+	 */
+	max_ref_frames = max_ref_frames ? max_ref_frames : 1;
+	hantro_reg_write(vpu, hevc_num_ref_frames, max_ref_frames);
+	hantro_reg_write(vpu, hevc_filter_over_slices,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED));
+	hantro_reg_write(vpu, hevc_filter_over_tiles,
+			 !!(pps->flags & V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED));
+
+	/*
+	 * Write POC count diff from current pic. For frame decoding only compute
+	 * pic_order_cnt[0] and ignore pic_order_cnt[1] used in field-coding.
+	 */
+	for (i = 0; i < decode_params->num_active_dpb_entries && i < ARRAY_SIZE(cur_poc); i++) {
+		char poc_diff = decode_params->pic_order_cnt_val - dpb[i].pic_order_cnt[0];
+
+		hantro_reg_write(vpu, cur_poc[i], poc_diff);
+	}
+
+	if (i < ARRAY_SIZE(cur_poc)) {
+		/*
+		 * After the references, fill one entry pointing to itself,
+		 * i.e. difference is zero.
+		 */
+		hantro_reg_write(vpu, cur_poc[i], 0);
+		i++;
+	}
+
+	/* Fill the rest with the current picture */
+	for (; i < ARRAY_SIZE(cur_poc); i++)
+		hantro_reg_write(vpu, cur_poc[i], decode_params->pic_order_cnt_val);
+
+	set_ref_pic_list(ctx);
+
+	/* We will only keep the references picture that are still used */
+	ctx->hevc_dec.ref_bufs_used = 0;
+
+	/* Set up addresses of DPB buffers */
+	dpb_longterm_e = 0;
+	for (i = 0; i < decode_params->num_active_dpb_entries &&
+	     i < (V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1); i++) {
+		luma_addr = hantro_hevc_get_ref_buf(ctx, dpb[i].pic_order_cnt[0]);
+		if (!luma_addr)
+			return -ENOMEM;
+
+		chroma_addr = luma_addr + cr_offset;
+		mv_addr = luma_addr + mv_offset;
+
+		if (dpb[i].rps == V4L2_HEVC_DPB_ENTRY_RPS_LT_CURR)
+			dpb_longterm_e |= BIT(V4L2_HEVC_DPB_ENTRIES_NUM_MAX - 1 - i);
+
+		hantro_write_addr(vpu, HEVC_REG_ADDR_REF(i), luma_addr);
+		hantro_write_addr(vpu, HEVC_REG_CHR_REF(i), chroma_addr);
+		hantro_write_addr(vpu, HEVC_REG_DMV_REF(i), mv_addr);
+	}
+
+	luma_addr = hantro_hevc_get_ref_buf(ctx, decode_params->pic_order_cnt_val);
+	if (!luma_addr)
+		return -ENOMEM;
+
+	chroma_addr = luma_addr + cr_offset;
+	mv_addr = luma_addr + mv_offset;
+
+	hantro_write_addr(vpu, HEVC_REG_ADDR_REF(i), luma_addr);
+	hantro_write_addr(vpu, HEVC_REG_CHR_REF(i), chroma_addr);
+	hantro_write_addr(vpu, HEVC_REG_DMV_REF(i++), mv_addr);
+
+	hantro_write_addr(vpu, HEVC_ADDR_DST, luma_addr);
+	hantro_write_addr(vpu, HEVC_ADDR_DST_CHR, chroma_addr);
+	hantro_write_addr(vpu, HEVC_ADDR_DST_MV, mv_addr);
+
+	hantro_hevc_ref_remove_unused(ctx);
+
+	for (; i < V4L2_HEVC_DPB_ENTRIES_NUM_MAX; i++) {
+		hantro_write_addr(vpu, HEVC_REG_ADDR_REF(i), 0);
+		hantro_write_addr(vpu, HEVC_REG_CHR_REF(i), 0);
+		hantro_write_addr(vpu, HEVC_REG_DMV_REF(i), 0);
+	}
+
+	hantro_reg_write(vpu, hevc_refer_lterm_e, dpb_longterm_e);
+
+	return 0;
+}
+
+static void set_buffers(struct hantro_ctx *ctx)
+{
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct hantro_dev *vpu = ctx->dev;
+	const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+	const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+	size_t cr_offset = hantro_hevc_chroma_offset(sps);
+	dma_addr_t src_dma, dst_dma;
+	u32 src_len, src_buf_len;
+
+	src_buf = hantro_get_src_buf(ctx);
+	dst_buf = hantro_get_dst_buf(ctx);
+
+	/* Source (stream) buffer. */
+	src_dma = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+	src_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+	src_buf_len = vb2_plane_size(&src_buf->vb2_buf, 0);
+
+	hantro_write_addr(vpu, HEVC_ADDR_STR, src_dma);
+	hantro_reg_write(vpu, hevc_stream_len, src_len);
+	hantro_reg_write(vpu, hevc_strm_buffer_len, src_buf_len);
+	hantro_reg_write(vpu, hevc_strm_start_offset, 0);
+	hantro_reg_write(vpu, hevc_write_mvs_e, 1);
+
+	/* Destination (decoded frame) buffer. */
+	dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf);
+
+	hantro_write_addr(vpu, HEVC_RASTER_SCAN, dst_dma);
+	hantro_write_addr(vpu, HEVC_RASTER_SCAN_CHR, dst_dma + cr_offset);
+	hantro_write_addr(vpu, HEVC_ADDR_TILE_SIZE, ctx->hevc_dec.tile_sizes.dma);
+	hantro_write_addr(vpu, HEVC_TILE_FILTER, ctx->hevc_dec.tile_filter.dma);
+	hantro_write_addr(vpu, HEVC_TILE_SAO, ctx->hevc_dec.tile_sao.dma);
+	hantro_write_addr(vpu, HEVC_TILE_BSD, ctx->hevc_dec.tile_bsd.dma);
+}
+
+void hantro_g2_check_idle(struct hantro_dev *vpu)
+{
+	int i;
+
+	for (i = 0; i < 3; i++) {
+		u32 status;
+
+		/* Make sure the VPU is idle */
+		status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
+		if (status & HEVC_REG_INTERRUPT_DEC_E) {
+			pr_warn("%s: still enabled!!! resetting.\n", __func__);
+			status |= HEVC_REG_INTERRUPT_DEC_ABORT_E | HEVC_REG_INTERRUPT_DEC_IRQ_DIS;
+			vdpu_write(vpu, status, HEVC_REG_INTERRUPT);
+		}
+	}
+}
+
+void hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
+{
+	struct hantro_dev *vpu = ctx->dev;
+
+	hantro_g2_check_idle(vpu);
+
+	/* Prepare HEVC decoder context. */
+	if (hantro_hevc_dec_prepare_run(ctx))
+		return;
+
+	/* Configure hardware registers. */
+	set_params(ctx);
+
+	/* set reference pictures */
+	if (set_ref(ctx))
+		/* something goes wrong */
+		hantro_irq_done(vpu, VB2_BUF_STATE_ERROR);
+
+	set_buffers(ctx);
+	prepare_tile_info_buffer(ctx);
+
+	hantro_end_prepare_run(ctx);
+
+	hantro_reg_write(vpu, hevc_mode, HEVC_DEC_MODE);
+	hantro_reg_write(vpu, hevc_clk_gate_e, 1);
+
+	/* Don't disable output */
+	hantro_reg_write(vpu, hevc_out_dis, 0);
+
+	/* Don't compress buffers */
+	hantro_reg_write(vpu, hevc_ref_compress_bypass, 1);
+
+	/* use NV12 as output format */
+	hantro_reg_write(vpu, hevc_tile_e, 0);
+	hantro_reg_write(vpu, hevc_out_rs_e, 1);
+	hantro_reg_write(vpu, hevc_num_tile_rows, 1);
+	hantro_reg_write(vpu, hevc_num_tile_cols, 1);
+
+	/* Bus width and max burst */
+	hantro_reg_write(vpu, hevc_buswidth, BUS_WIDTH_128);
+	hantro_reg_write(vpu, hevc_max_burst, 16);
+
+	/* Swap */
+	hantro_reg_write(vpu, hevc_strm_swap, 0xf);
+	hantro_reg_write(vpu, hevc_dirmv_swap, 0xf);
+	hantro_reg_write(vpu, hevc_compress_swap, 0xf);
+
+	/* Start decoding! */
+	vdpu_write(vpu, HEVC_REG_INTERRUPT_DEC_E, HEVC_REG_INTERRUPT);
+}
diff --git a/drivers/staging/media/hantro/hantro_g2_regs.h b/drivers/staging/media/hantro/hantro_g2_regs.h
new file mode 100644
index 000000000000..a361c9ba911d
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_g2_regs.h
@@ -0,0 +1,198 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2021, Collabora
+ *
+ * Author: Benjamin Gaignard <benjamin.gaignard@collabora.com>
+ */
+
+#ifndef HANTRO_G2_REGS_H_
+#define HANTRO_G2_REGS_H_
+
+#include "hantro.h"
+
+#define G2_SWREG(nr)	((nr) * 4)
+
+#define HEVC_DEC_REG(name, base, shift, mask) \
+	static const struct hantro_reg _hevc_##name[] = { \
+		{ G2_SWREG(base), (shift), (mask) } \
+	}; \
+	static const struct hantro_reg __maybe_unused *hevc_##name = &_hevc_##name[0];
+
+#define HEVC_REG_VERSION		G2_SWREG(0)
+
+#define HEVC_REG_INTERRUPT		G2_SWREG(1)
+#define HEVC_REG_INTERRUPT_DEC_RDY_INT	BIT(12)
+#define HEVC_REG_INTERRUPT_DEC_ABORT_E	BIT(5)
+#define HEVC_REG_INTERRUPT_DEC_IRQ_DIS	BIT(4)
+#define HEVC_REG_INTERRUPT_DEC_E	BIT(0)
+
+HEVC_DEC_REG(strm_swap,		2, 28,	0xf)
+HEVC_DEC_REG(dirmv_swap,	2, 20,	0xf)
+
+HEVC_DEC_REG(mode,		  3, 27, 0x1f)
+HEVC_DEC_REG(compress_swap,	  3, 20, 0xf)
+HEVC_DEC_REG(ref_compress_bypass, 3, 17, 0x1)
+HEVC_DEC_REG(out_rs_e,		  3, 16, 0x1)
+HEVC_DEC_REG(out_dis,		  3, 15, 0x1)
+HEVC_DEC_REG(out_filtering_dis,   3, 14, 0x1)
+HEVC_DEC_REG(write_mvs_e,	  3, 12, 0x1)
+
+HEVC_DEC_REG(pic_width_in_cbs,	4, 19,	0x1ff)
+HEVC_DEC_REG(pic_height_in_cbs,	4, 6,	0x1ff)
+HEVC_DEC_REG(num_ref_frames,	4, 0,	0x1f)
+
+HEVC_DEC_REG(scaling_list_e,	5, 24,	0x1)
+HEVC_DEC_REG(cb_qp_offset,	5, 19,	0x1f)
+HEVC_DEC_REG(cr_qp_offset,	5, 14,	0x1f)
+HEVC_DEC_REG(sign_data_hide,	5, 12,	0x1)
+HEVC_DEC_REG(tempor_mvp_e,	5, 11,	0x1)
+HEVC_DEC_REG(max_cu_qpd_depth,	5, 5,	0x3f)
+HEVC_DEC_REG(cu_qpd_e,		5, 4,	0x1)
+
+HEVC_DEC_REG(stream_len,	6, 0,	0xffffffff)
+
+HEVC_DEC_REG(cabac_init_present, 7, 31, 0x1)
+HEVC_DEC_REG(weight_pred_e,	 7, 28, 0x1)
+HEVC_DEC_REG(weight_bipr_idc,	 7, 26, 0x3)
+HEVC_DEC_REG(filter_over_slices, 7, 25, 0x1)
+HEVC_DEC_REG(filter_over_tiles,  7, 24, 0x1)
+HEVC_DEC_REG(asym_pred_e,	 7, 23, 0x1)
+HEVC_DEC_REG(sao_e,		 7, 22, 0x1)
+HEVC_DEC_REG(pcm_filt_d,	 7, 21, 0x1)
+HEVC_DEC_REG(slice_chqp_present, 7, 20, 0x1)
+HEVC_DEC_REG(dependent_slice,	 7, 19, 0x1)
+HEVC_DEC_REG(filter_override,	 7, 18, 0x1)
+HEVC_DEC_REG(strong_smooth_e,	 7, 17, 0x1)
+HEVC_DEC_REG(filt_offset_beta,	 7, 12, 0x1f)
+HEVC_DEC_REG(filt_offset_tc,	 7, 7,  0x1f)
+HEVC_DEC_REG(slice_hdr_ext_e,	 7, 6,	0x1)
+HEVC_DEC_REG(slice_hdr_ext_bits, 7, 3,	0x7)
+
+HEVC_DEC_REG(const_intra_e,	 8, 31, 0x1)
+HEVC_DEC_REG(filt_ctrl_pres,	 8, 30, 0x1)
+HEVC_DEC_REG(idr_pic_e,		 8, 16, 0x1)
+HEVC_DEC_REG(bit_depth_pcm_y,	 8, 12, 0xf)
+HEVC_DEC_REG(bit_depth_pcm_c,	 8, 8,  0xf)
+HEVC_DEC_REG(bit_depth_y_minus8, 8, 6,  0x3)
+HEVC_DEC_REG(bit_depth_c_minus8, 8, 4,  0x3)
+HEVC_DEC_REG(output_8_bits,	 8, 3,  0x1)
+
+HEVC_DEC_REG(refidx1_active,	9, 19,	0x1f)
+HEVC_DEC_REG(refidx0_active,	9, 14,	0x1f)
+HEVC_DEC_REG(hdr_skip_length,	9, 0,	0x3fff)
+
+HEVC_DEC_REG(start_code_e,	10, 31, 0x1)
+HEVC_DEC_REG(init_qp,		10, 24, 0x3f)
+HEVC_DEC_REG(num_tile_cols,	10, 19, 0x1f)
+HEVC_DEC_REG(num_tile_rows,	10, 14, 0x1f)
+HEVC_DEC_REG(tile_e,		10, 1,	0x1)
+HEVC_DEC_REG(entropy_sync_e,	10, 0,	0x1)
+
+HEVC_DEC_REG(refer_lterm_e,	12, 16, 0xffff)
+HEVC_DEC_REG(min_cb_size,	12, 13, 0x7)
+HEVC_DEC_REG(max_cb_size,	12, 10, 0x7)
+HEVC_DEC_REG(min_pcm_size,	12, 7,  0x7)
+HEVC_DEC_REG(max_pcm_size,	12, 4,  0x7)
+HEVC_DEC_REG(pcm_e,		12, 3,  0x1)
+HEVC_DEC_REG(transform_skip,	12, 2,	0x1)
+HEVC_DEC_REG(transq_bypass,	12, 1,	0x1)
+HEVC_DEC_REG(list_mod_e,	12, 0,	0x1)
+
+HEVC_DEC_REG(min_trb_size,	  13, 13, 0x7)
+HEVC_DEC_REG(max_trb_size,	  13, 10, 0x7)
+HEVC_DEC_REG(max_intra_hierdepth, 13, 7,  0x7)
+HEVC_DEC_REG(max_inter_hierdepth, 13, 4,  0x7)
+HEVC_DEC_REG(parallel_merge,	  13, 0,  0xf)
+
+HEVC_DEC_REG(rlist_f0,		14, 0,	0x1f)
+HEVC_DEC_REG(rlist_f1,		14, 10,	0x1f)
+HEVC_DEC_REG(rlist_f2,		14, 20,	0x1f)
+HEVC_DEC_REG(rlist_b0,		14, 5,	0x1f)
+HEVC_DEC_REG(rlist_b1,		14, 15, 0x1f)
+HEVC_DEC_REG(rlist_b2,		14, 25, 0x1f)
+
+HEVC_DEC_REG(rlist_f3,		15, 0,	0x1f)
+HEVC_DEC_REG(rlist_f4,		15, 10, 0x1f)
+HEVC_DEC_REG(rlist_f5,		15, 20, 0x1f)
+HEVC_DEC_REG(rlist_b3,		15, 5,	0x1f)
+HEVC_DEC_REG(rlist_b4,		15, 15, 0x1f)
+HEVC_DEC_REG(rlist_b5,		15, 25, 0x1f)
+
+HEVC_DEC_REG(rlist_f6,		16, 0,	0x1f)
+HEVC_DEC_REG(rlist_f7,		16, 10, 0x1f)
+HEVC_DEC_REG(rlist_f8,		16, 20, 0x1f)
+HEVC_DEC_REG(rlist_b6,		16, 5,	0x1f)
+HEVC_DEC_REG(rlist_b7,		16, 15, 0x1f)
+HEVC_DEC_REG(rlist_b8,		16, 25, 0x1f)
+
+HEVC_DEC_REG(rlist_f9,		17, 0,	0x1f)
+HEVC_DEC_REG(rlist_f10,		17, 10, 0x1f)
+HEVC_DEC_REG(rlist_f11,		17, 20, 0x1f)
+HEVC_DEC_REG(rlist_b9,		17, 5,	0x1f)
+HEVC_DEC_REG(rlist_b10,		17, 15, 0x1f)
+HEVC_DEC_REG(rlist_b11,		17, 25, 0x1f)
+
+HEVC_DEC_REG(rlist_f12,		18, 0,	0x1f)
+HEVC_DEC_REG(rlist_f13,		18, 10, 0x1f)
+HEVC_DEC_REG(rlist_f14,		18, 20, 0x1f)
+HEVC_DEC_REG(rlist_b12,		18, 5,	0x1f)
+HEVC_DEC_REG(rlist_b13,		18, 15, 0x1f)
+HEVC_DEC_REG(rlist_b14,		18, 25, 0x1f)
+
+HEVC_DEC_REG(rlist_f15,		19, 0,	0x1f)
+HEVC_DEC_REG(rlist_b15,		19, 5,	0x1f)
+
+HEVC_DEC_REG(partial_ctb_x,	20, 31, 0x1)
+HEVC_DEC_REG(partial_ctb_y,	20, 30, 0x1)
+HEVC_DEC_REG(pic_width_4x4,	20, 16, 0xfff)
+HEVC_DEC_REG(pic_height_4x4,	20, 0,  0xfff)
+
+HEVC_DEC_REG(cur_poc_00,	46, 24,	0xff)
+HEVC_DEC_REG(cur_poc_01,	46, 16,	0xff)
+HEVC_DEC_REG(cur_poc_02,	46, 8,	0xff)
+HEVC_DEC_REG(cur_poc_03,	46, 0,	0xff)
+
+HEVC_DEC_REG(cur_poc_04,	47, 24,	0xff)
+HEVC_DEC_REG(cur_poc_05,	47, 16,	0xff)
+HEVC_DEC_REG(cur_poc_06,	47, 8,	0xff)
+HEVC_DEC_REG(cur_poc_07,	47, 0,	0xff)
+
+HEVC_DEC_REG(cur_poc_08,	48, 24,	0xff)
+HEVC_DEC_REG(cur_poc_09,	48, 16,	0xff)
+HEVC_DEC_REG(cur_poc_10,	48, 8,	0xff)
+HEVC_DEC_REG(cur_poc_11,	48, 0,	0xff)
+
+HEVC_DEC_REG(cur_poc_12,	49, 24, 0xff)
+HEVC_DEC_REG(cur_poc_13,	49, 16, 0xff)
+HEVC_DEC_REG(cur_poc_14,	49, 8,	0xff)
+HEVC_DEC_REG(cur_poc_15,	49, 0,	0xff)
+
+HEVC_DEC_REG(apf_threshold,	55, 0,	0xffff)
+
+HEVC_DEC_REG(clk_gate_e,	58, 16,	0x1)
+HEVC_DEC_REG(buswidth,		58, 8,	0x7)
+HEVC_DEC_REG(max_burst,		58, 0,	0xff)
+
+#define HEVC_REG_CONFIG				G2_SWREG(58)
+#define HEVC_REG_CONFIG_DEC_CLK_GATE_E		BIT(16)
+#define HEVC_REG_CONFIG_DEC_CLK_GATE_IDLE_E	BIT(17)
+
+#define HEVC_ADDR_DST		(G2_SWREG(65))
+#define HEVC_REG_ADDR_REF(i)	(G2_SWREG(67)  + ((i) * 0x8))
+#define HEVC_ADDR_DST_CHR	(G2_SWREG(99))
+#define HEVC_REG_CHR_REF(i)	(G2_SWREG(101) + ((i) * 0x8))
+#define HEVC_ADDR_DST_MV	(G2_SWREG(133))
+#define HEVC_REG_DMV_REF(i)	(G2_SWREG(135) + ((i) * 0x8))
+#define HEVC_ADDR_TILE_SIZE	(G2_SWREG(167))
+#define HEVC_ADDR_STR		(G2_SWREG(169))
+#define HEVC_SCALING_LIST	(G2_SWREG(171))
+#define HEVC_RASTER_SCAN	(G2_SWREG(175))
+#define HEVC_RASTER_SCAN_CHR	(G2_SWREG(177))
+#define HEVC_TILE_FILTER	(G2_SWREG(179))
+#define HEVC_TILE_SAO		(G2_SWREG(181))
+#define HEVC_TILE_BSD		(G2_SWREG(183))
+
+HEVC_DEC_REG(strm_buffer_len,	258, 0,	0xffffffff)
+HEVC_DEC_REG(strm_start_offset,	259, 0,	0xffffffff)
+
+#endif
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
new file mode 100644
index 000000000000..8e319a837ff3
--- /dev/null
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -0,0 +1,321 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hantro VPU HEVC codec driver
+ *
+ * Copyright (C) 2020 Safran Passenger Innovations LLC
+ */
+
+#include <linux/types.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "hantro.h"
+#include "hantro_hw.h"
+
+#define VERT_FILTER_RAM_SIZE 8 /* bytes per pixel row */
+/*
+ * BSD control data of current picture at tile border
+ * 128 bits per 4x4 tile = 128/(8*4) bytes per row
+ */
+#define BSD_CTRL_RAM_SIZE 4 /* bytes per pixel row */
+/* tile border coefficients of filter */
+#define VERT_SAO_RAM_SIZE 48 /* bytes per pixel */
+
+#define MAX_TILE_COLS 20
+#define MAX_TILE_ROWS 22
+
+#define UNUSED_REF	-1
+
+#define G2_ALIGN		16
+#define MC_WORD_SIZE		32
+
+size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps)
+{
+	int bytes_per_pixel = sps->bit_depth_luma_minus8 == 0 ? 1 : 2;
+
+	return sps->pic_width_in_luma_samples *
+		sps->pic_height_in_luma_samples * bytes_per_pixel;
+}
+
+size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps)
+{
+	size_t cr_offset = hantro_hevc_chroma_offset(sps);
+
+	return ALIGN((cr_offset * 3) / 2, G2_ALIGN) + MC_WORD_SIZE;
+}
+
+static size_t hantro_hevc_mv_size(const struct v4l2_ctrl_hevc_sps *sps)
+{
+	u32 pic_width_in_ctb64 = (sps->pic_width_in_luma_samples + (1 << 8) - 1) >> 8;
+	u32 pic_height_in_ctb64 = (sps->pic_height_in_luma_samples  + (1 << 8) - 1) >> 8;
+	size_t mv_size;
+
+	mv_size = (pic_width_in_ctb64 * pic_height_in_ctb64 *
+		  (1 << (2 * (8 - 4))) * 16) + 32;
+
+	vpu_debug(4, "%dx%d (CTBs) %lu MV bytes\n",
+		  pic_width_in_ctb64, pic_height_in_ctb64, mv_size);
+
+	return mv_size;
+}
+
+static size_t hantro_hevc_ref_size(struct hantro_ctx *ctx)
+{
+	const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+	const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+
+	return hantro_hevc_motion_vectors_offset(sps) + hantro_hevc_mv_size(sps);
+}
+
+static void hantro_hevc_ref_free(struct hantro_ctx *ctx)
+{
+	struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+	struct hantro_dev *vpu = ctx->dev;
+	int i;
+
+	/* Just tag buffer as unused, do not free them */
+	for (i = 0;  i < NUM_REF_PICTURES; i++) {
+		if (hevc_dec->ref_bufs[i].cpu) {
+			memset(hevc_dec->ref_bufs[i].cpu, 0, hantro_hevc_ref_size(ctx));
+			dma_free_coherent(vpu->dev, hevc_dec->ref_bufs[i].size,
+					  hevc_dec->ref_bufs[i].cpu,
+					  hevc_dec->ref_bufs[i].dma);
+		}
+	}
+}
+
+static void hantro_hevc_ref_init(struct hantro_ctx *ctx)
+{
+	struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+	int i;
+
+	for (i = 0;  i < NUM_REF_PICTURES; i++)
+		hevc_dec->ref_bufs_poc[i] = UNUSED_REF;
+}
+
+dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx,
+				   int poc)
+{
+	struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+	int i;
+
+	/* Find the reference buffer in already know ones */
+	for (i = 0;  i < NUM_REF_PICTURES; i++) {
+		if (hevc_dec->ref_bufs_poc[i] == poc) {
+			hevc_dec->ref_bufs_used |= 1 << i;
+			return hevc_dec->ref_bufs[i].dma;
+		}
+	}
+
+	/* Allocate a new reference buffer */
+	for (i = 0; i < NUM_REF_PICTURES; i++) {
+		if (hevc_dec->ref_bufs_poc[i] == UNUSED_REF) {
+			if (!hevc_dec->ref_bufs[i].cpu) {
+				struct hantro_dev *vpu = ctx->dev;
+
+				hevc_dec->ref_bufs[i].cpu =
+					dma_alloc_coherent(vpu->dev,
+							   hantro_hevc_ref_size(ctx),
+							   &hevc_dec->ref_bufs[i].dma,
+							   GFP_KERNEL);
+				if (!hevc_dec->ref_bufs[i].cpu)
+					return 0;
+
+				hevc_dec->ref_bufs[i].size = hantro_hevc_ref_size(ctx);
+			}
+			hevc_dec->ref_bufs_used |= 1 << i;
+			memset(hevc_dec->ref_bufs[i].cpu, 0, hantro_hevc_ref_size(ctx));
+			hevc_dec->ref_bufs_poc[i] = poc;
+
+			return hevc_dec->ref_bufs[i].dma;
+		}
+	}
+
+	return 0;
+}
+
+void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx)
+{
+	struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+	int i;
+
+	/* Just tag buffer as unused, do not free them */
+	for (i = 0;  i < NUM_REF_PICTURES; i++) {
+		if (hevc_dec->ref_bufs_poc[i] == UNUSED_REF)
+			continue;
+
+		if (hevc_dec->ref_bufs_used & (1 << i))
+			continue;
+
+		hevc_dec->ref_bufs_poc[i] = UNUSED_REF;
+	}
+}
+
+static int tile_buffer_reallocate(struct hantro_ctx *ctx)
+{
+	struct hantro_dev *vpu = ctx->dev;
+	struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+	const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
+	const struct v4l2_ctrl_hevc_pps *pps = ctrls->pps;
+	const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
+	unsigned int num_tile_cols = pps->num_tile_columns_minus1 + 1;
+	unsigned int height64 = (sps->pic_height_in_luma_samples + 63) & ~63;
+	unsigned int size;
+
+	if (num_tile_cols <= 1 ||
+	    num_tile_cols <= hevc_dec->num_tile_cols_allocated)
+		return 0;
+
+	/* Need to reallocate due to tiles passed via PPS */
+	if (hevc_dec->tile_filter.size)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_filter.size,
+				  hevc_dec->tile_filter.cpu,
+				  hevc_dec->tile_filter.dma);
+
+	if (hevc_dec->tile_sao.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_sao.size,
+				  hevc_dec->tile_sao.cpu,
+				  hevc_dec->tile_sao.dma);
+
+	if (hevc_dec->tile_bsd.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_bsd.size,
+				  hevc_dec->tile_bsd.cpu,
+				  hevc_dec->tile_bsd.dma);
+
+	size = VERT_FILTER_RAM_SIZE * height64 * (num_tile_cols - 1);
+	hevc_dec->tile_filter.cpu = dma_alloc_coherent(vpu->dev, size,
+						       &hevc_dec->tile_filter.dma,
+						       GFP_KERNEL);
+	if (!hevc_dec->tile_filter.cpu)
+		goto err_free_tile_buffers;
+	hevc_dec->tile_filter.size = size;
+
+	size = VERT_SAO_RAM_SIZE * height64 * (num_tile_cols - 1);
+	hevc_dec->tile_sao.cpu = dma_alloc_coherent(vpu->dev, size,
+						    &hevc_dec->tile_sao.dma,
+						    GFP_KERNEL);
+	if (!hevc_dec->tile_sao.cpu)
+		goto err_free_tile_buffers;
+	hevc_dec->tile_sao.size = size;
+
+	size = BSD_CTRL_RAM_SIZE * height64 * (num_tile_cols - 1);
+	hevc_dec->tile_bsd.cpu = dma_alloc_coherent(vpu->dev, size,
+						    &hevc_dec->tile_bsd.dma,
+						    GFP_KERNEL);
+	if (!hevc_dec->tile_bsd.cpu)
+		goto err_free_tile_buffers;
+	hevc_dec->tile_bsd.size = size;
+
+	hevc_dec->num_tile_cols_allocated = num_tile_cols;
+
+	return 0;
+
+err_free_tile_buffers:
+	if (hevc_dec->tile_filter.size)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_filter.size,
+				  hevc_dec->tile_filter.cpu,
+				  hevc_dec->tile_filter.dma);
+	hevc_dec->tile_filter.cpu = 0;
+
+	if (hevc_dec->tile_sao.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_sao.size,
+				  hevc_dec->tile_sao.cpu,
+				  hevc_dec->tile_sao.dma);
+	hevc_dec->tile_sao.cpu = 0;
+
+	if (hevc_dec->tile_bsd.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_bsd.size,
+				  hevc_dec->tile_bsd.cpu,
+				  hevc_dec->tile_bsd.dma);
+	hevc_dec->tile_bsd.cpu = 0;
+
+	return -ENOMEM;
+}
+
+int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx)
+{
+	struct hantro_hevc_dec_hw_ctx *hevc_ctx = &ctx->hevc_dec;
+	struct hantro_hevc_dec_ctrls *ctrls = &hevc_ctx->ctrls;
+	int ret;
+
+	hantro_start_prepare_run(ctx);
+
+	ctrls->decode_params =
+		hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
+	if (WARN_ON(!ctrls->decode_params))
+		return -EINVAL;
+
+	ctrls->sps =
+		hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SPS);
+	if (WARN_ON(!ctrls->sps))
+		return -EINVAL;
+
+	ctrls->pps =
+		hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_PPS);
+	if (WARN_ON(!ctrls->pps))
+		return -EINVAL;
+
+	ret = tile_buffer_reallocate(ctx);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+void hantro_hevc_dec_exit(struct hantro_ctx *ctx)
+{
+	struct hantro_dev *vpu = ctx->dev;
+	struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+
+	if (hevc_dec->tile_sizes.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_sizes.size,
+				  hevc_dec->tile_sizes.cpu,
+				  hevc_dec->tile_sizes.dma);
+	hevc_dec->tile_sizes.cpu = 0;
+
+	if (hevc_dec->tile_filter.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_filter.size,
+				  hevc_dec->tile_filter.cpu,
+				  hevc_dec->tile_filter.dma);
+	hevc_dec->tile_filter.cpu = 0;
+
+	if (hevc_dec->tile_sao.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_sao.size,
+				  hevc_dec->tile_sao.cpu,
+				  hevc_dec->tile_sao.dma);
+	hevc_dec->tile_sao.cpu = 0;
+
+	if (hevc_dec->tile_bsd.cpu)
+		dma_free_coherent(vpu->dev, hevc_dec->tile_bsd.size,
+				  hevc_dec->tile_bsd.cpu,
+				  hevc_dec->tile_bsd.dma);
+	hevc_dec->tile_bsd.cpu = 0;
+
+	hantro_hevc_ref_free(ctx);
+}
+
+int hantro_hevc_dec_init(struct hantro_ctx *ctx)
+{
+	struct hantro_dev *vpu = ctx->dev;
+	struct hantro_hevc_dec_hw_ctx *hevc_dec = &ctx->hevc_dec;
+	unsigned int size;
+
+	memset(hevc_dec, 0, sizeof(*hevc_dec));
+
+	/*
+	 * Maximum number of tiles times width and height (2 bytes each),
+	 * rounding up to next 16 bytes boundary + one extra 16 byte
+	 * chunk (HW guys wanted to have this).
+	 */
+	size = round_up(MAX_TILE_COLS * MAX_TILE_ROWS * 4 * sizeof(u16) + 16, 16);
+	hevc_dec->tile_sizes.cpu = dma_alloc_coherent(vpu->dev, size,
+						      &hevc_dec->tile_sizes.dma,
+						      GFP_KERNEL);
+	if (!hevc_dec->tile_sizes.cpu)
+		return -ENOMEM;
+
+	hevc_dec->tile_sizes.size = size;
+
+	hantro_hevc_ref_init(ctx);
+
+	return 0;
+}
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 34c9e4649a25..82e19b477173 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -9,6 +9,7 @@
 #ifndef HANTRO_HW_H_
 #define HANTRO_HW_H_
 
+#include <linux/hantro-v4l2-controls.h>
 #include <linux/interrupt.h>
 #include <linux/v4l2-controls.h>
 #include <media/v4l2-ctrls.h>
@@ -20,6 +21,8 @@
 #define MB_WIDTH(w)		DIV_ROUND_UP(w, MB_DIM)
 #define MB_HEIGHT(h)		DIV_ROUND_UP(h, MB_DIM)
 
+#define NUM_REF_PICTURES	(V4L2_HEVC_DPB_ENTRIES_NUM_MAX + 1)
+
 struct hantro_dev;
 struct hantro_ctx;
 struct hantro_buf;
@@ -90,6 +93,41 @@ struct hantro_h264_dec_hw_ctx {
 	struct hantro_h264_dec_ctrls ctrls;
 };
 
+/**
+ * struct hantro_hevc_dec_ctrls
+ * @decode_params: Decode params
+ * @sps:	SPS info
+ * @pps:	PPS info
+ */
+struct hantro_hevc_dec_ctrls {
+	const struct v4l2_ctrl_hevc_decode_params *decode_params;
+	const struct v4l2_ctrl_hevc_sps *sps;
+	const struct v4l2_ctrl_hevc_pps *pps;
+	const struct hantro_hevc_extra_decode_params extra_params;
+};
+
+/**
+ * struct hantro_hevc_dec_hw_ctx
+ * @tile_sizes:		Tile sizes buffer
+ * @tile_filter:	Tile vertical filter buffer
+ * @tile_sao:		Tile SAO buffer
+ * @tile_bsd:		Tile BSD control buffer
+ * @dpb:	DPB
+ * @reflists:	P/B0/B1 reflists
+ * @ctrls:	V4L2 controls attached to a run
+ */
+struct hantro_hevc_dec_hw_ctx {
+	struct hantro_aux_buf tile_sizes;
+	struct hantro_aux_buf tile_filter;
+	struct hantro_aux_buf tile_sao;
+	struct hantro_aux_buf tile_bsd;
+	struct hantro_aux_buf ref_bufs[NUM_REF_PICTURES];
+	int ref_bufs_poc[NUM_REF_PICTURES];
+	u32 ref_bufs_used;
+	struct hantro_hevc_dec_ctrls ctrls;
+	unsigned int num_tile_cols_allocated;
+};
+
 /**
  * struct hantro_mpeg2_dec_hw_ctx
  * @qtable:		Quantization table
@@ -177,6 +215,15 @@ void hantro_g1_h264_dec_run(struct hantro_ctx *ctx);
 int hantro_h264_dec_init(struct hantro_ctx *ctx);
 void hantro_h264_dec_exit(struct hantro_ctx *ctx);
 
+int hantro_hevc_dec_init(struct hantro_ctx *ctx);
+void hantro_hevc_dec_exit(struct hantro_ctx *ctx);
+void hantro_g2_hevc_dec_run(struct hantro_ctx *ctx);
+int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx);
+dma_addr_t hantro_hevc_get_ref_buf(struct hantro_ctx *ctx, int poc);
+void hantro_hevc_ref_remove_unused(struct hantro_ctx *ctx);
+size_t hantro_hevc_chroma_offset(const struct v4l2_ctrl_hevc_sps *sps);
+size_t hantro_hevc_motion_vectors_offset(const struct v4l2_ctrl_hevc_sps *sps);
+
 static inline size_t
 hantro_h264_mv_size(unsigned int width, unsigned int height)
 {
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 6/9] media: hantro: handle V4L2_PIX_FMT_HEVC_SLICE control

[Benjamin Gaignard]

Make sure that V4L2_PIX_FMT_HEVC_SLICE is correctly handle by v4l2
of the driver.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
 drivers/staging/media/hantro/hantro_v4l2.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c
index 1bc118e375a1..e16d5fd0b9f7 100644
--- a/drivers/staging/media/hantro/hantro_v4l2.c
+++ b/drivers/staging/media/hantro/hantro_v4l2.c
@@ -390,6 +390,7 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc)
 	case V4L2_PIX_FMT_MPEG2_SLICE:
 	case V4L2_PIX_FMT_VP8_FRAME:
 	case V4L2_PIX_FMT_H264_SLICE:
+	case V4L2_PIX_FMT_HEVC_SLICE:
 		ctx->fh.m2m_ctx->out_q_ctx.q.requires_requests = true;
 		break;
 	default:
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 7/9] media: hantro: IMX8M: add variant for G2/HEVC codec

[Benjamin Gaignard]

Add variant to IMX8M to enable G2/HEVC codec.
Define the capabilities for the hardware up to 3840x2160.
Retrieve the hardware version at init to distinguish G1 from G2.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2:
- remove useless clocks

 drivers/staging/media/hantro/hantro_drv.c   |  1 +
 drivers/staging/media/hantro/hantro_hw.h    |  1 +
 drivers/staging/media/hantro/imx8m_vpu_hw.c | 95 ++++++++++++++++++++-
 3 files changed, 93 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index c009941dff3e..b6eb933b008c 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -578,6 +578,7 @@ static const struct of_device_id of_hantro_match[] = {
 #endif
 #ifdef CONFIG_VIDEO_HANTRO_IMX8M
 	{ .compatible = "nxp,imx8mq-vpu", .data = &imx8mq_vpu_variant, },
+	{ .compatible = "nxp,imx8mq-vpu-g2", .data = &imx8mq_vpu_g2_variant },
 #endif
 	{ /* sentinel */ }
 };
diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h
index 82e19b477173..77d6c065cea1 100644
--- a/drivers/staging/media/hantro/hantro_hw.h
+++ b/drivers/staging/media/hantro/hantro_hw.h
@@ -190,6 +190,7 @@ extern const struct hantro_variant rk3399_vpu_variant;
 extern const struct hantro_variant rk3328_vpu_variant;
 extern const struct hantro_variant rk3288_vpu_variant;
 extern const struct hantro_variant imx8mq_vpu_variant;
+extern const struct hantro_variant imx8mq_vpu_g2_variant;
 
 extern const struct hantro_postproc_regs hantro_g1_postproc_regs;
 
diff --git a/drivers/staging/media/hantro/imx8m_vpu_hw.c b/drivers/staging/media/hantro/imx8m_vpu_hw.c
index d5b4312b9391..46b33531be85 100644
--- a/drivers/staging/media/hantro/imx8m_vpu_hw.c
+++ b/drivers/staging/media/hantro/imx8m_vpu_hw.c
@@ -12,6 +12,7 @@
 #include "hantro.h"
 #include "hantro_jpeg.h"
 #include "hantro_g1_regs.h"
+#include "hantro_g2_regs.h"
 
 static int imx8mq_runtime_resume(struct hantro_dev *vpu)
 {
@@ -90,6 +91,26 @@ static const struct hantro_fmt imx8m_vpu_dec_fmts[] = {
 	},
 };
 
+static const struct hantro_fmt imx8m_vpu_g2_dec_fmts[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.codec_mode = HANTRO_MODE_NONE,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_HEVC_SLICE,
+		.codec_mode = HANTRO_MODE_HEVC_DEC,
+		.max_depth = 2,
+		.frmsize = {
+			.min_width = 48,
+			.max_width = 3840,
+			.step_width = MB_DIM,
+			.min_height = 48,
+			.max_height = 2160,
+			.step_height = MB_DIM,
+		},
+	},
+};
+
 static irqreturn_t imx8m_vpu_g1_irq(int irq, void *dev_id)
 {
 	struct hantro_dev *vpu = dev_id;
@@ -108,9 +129,42 @@ static irqreturn_t imx8m_vpu_g1_irq(int irq, void *dev_id)
 	return IRQ_HANDLED;
 }
 
+static irqreturn_t imx8m_vpu_g2_irq(int irq, void *dev_id)
+{
+	struct hantro_dev *vpu = dev_id;
+	enum vb2_buffer_state state;
+	u32 status;
+
+	status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
+	state = (status & HEVC_REG_INTERRUPT_DEC_RDY_INT) ?
+		 VB2_BUF_STATE_DONE : VB2_BUF_STATE_ERROR;
+
+	vdpu_write(vpu, 0, HEVC_REG_INTERRUPT);
+	vdpu_write(vpu, HEVC_REG_CONFIG_DEC_CLK_GATE_E, HEVC_REG_CONFIG);
+
+	hantro_irq_done(vpu, state);
+
+	return IRQ_HANDLED;
+}
+
 static int imx8mq_vpu_hw_init(struct hantro_dev *vpu)
 {
-	vpu->dec_base = vpu->reg_bases[0];
+	int ret;
+
+	/* Check variant version */
+	ret = clk_bulk_prepare_enable(vpu->variant->num_clocks, vpu->clocks);
+	if (ret) {
+		dev_err(vpu->dev, "Failed to enable clocks\n");
+		return ret;
+	}
+
+	/* Make that the device has been reset before read it id */
+	ret = device_reset(vpu->dev);
+	if (ret)
+		dev_err(vpu->dev, "Failed to reset Hantro VPU\n");
+
+	vpu->core_hw_dec_rev = (vdpu_read(vpu, HEVC_REG_VERSION) >> 16) & 0xffff;
+	clk_bulk_disable_unprepare(vpu->variant->num_clocks, vpu->clocks);
 
 	return 0;
 }
@@ -149,17 +203,32 @@ static const struct hantro_codec_ops imx8mq_vpu_codec_ops[] = {
 	},
 };
 
+static const struct hantro_codec_ops imx8mq_vpu_g2_codec_ops[] = {
+	[HANTRO_MODE_HEVC_DEC] = {
+		.run = hantro_g2_hevc_dec_run,
+		.reset = imx8mq_vpu_reset,
+		.init = hantro_hevc_dec_init,
+		.exit = hantro_hevc_dec_exit,
+	},
+};
+
 /*
  * VPU variants.
  */
 
 static const struct hantro_irq imx8mq_irqs[] = {
 	{ "g1", imx8m_vpu_g1_irq },
-	{ "g2", NULL /* TODO: imx8m_vpu_g2_irq */ },
 };
 
-static const char * const imx8mq_clk_names[] = { "g1", "g2", "bus" };
-static const char * const imx8mq_reg_names[] = { "g1", "g2", "ctrl" };
+static const struct hantro_irq imx8mq_g2_irqs[] = {
+	{ "g2", imx8m_vpu_g2_irq },
+};
+
+static const char * const imx8mq_clk_names[] = { "g1", "bus"};
+static const char * const imx8mq_reg_names[] = { "g1"};
+
+static const char * const imx8mq_g2_clk_names[] = { "g2", "bus"};
+static const char * const imx8mq_g2_reg_names[] = { "g2"};
 
 const struct hantro_variant imx8mq_vpu_variant = {
 	.dec_fmts = imx8m_vpu_dec_fmts,
@@ -179,3 +248,21 @@ const struct hantro_variant imx8mq_vpu_variant = {
 	.reg_names = imx8mq_reg_names,
 	.num_regs = ARRAY_SIZE(imx8mq_reg_names)
 };
+
+const struct hantro_variant imx8mq_vpu_g2_variant = {
+	.dec_offset = 0x0,
+	.dec_fmts = imx8m_vpu_g2_dec_fmts,
+	.num_dec_fmts = ARRAY_SIZE(imx8m_vpu_g2_dec_fmts),
+	.postproc_fmts = imx8m_vpu_postproc_fmts,
+	.num_postproc_fmts = ARRAY_SIZE(imx8m_vpu_postproc_fmts),
+	.codec = HANTRO_HEVC_DECODER,
+	.codec_ops = imx8mq_vpu_g2_codec_ops,
+	.init = imx8mq_vpu_hw_init,
+	.runtime_resume = imx8mq_runtime_resume,
+	.irqs = imx8mq_g2_irqs,
+	.num_irqs = ARRAY_SIZE(imx8mq_g2_irqs),
+	.clk_names = imx8mq_g2_clk_names,
+	.num_clocks = ARRAY_SIZE(imx8mq_g2_clk_names),
+	.reg_names = imx8mq_g2_reg_names,
+	.num_regs = ARRAY_SIZE(imx8mq_g2_reg_names),
+};
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 8/9] dt-bindings: media: nxp,imx8mq-vpu: Update bindings

[Benjamin Gaignard]

The current bindings seem to make the assumption that the
two VPUs hardware blocks (G1 and G2) are only one set of
registers.
After implementing the VPU reset driver and G2 decoder driver
it shows that all the VPUs are independent and don't need to
know about the registers of the other blocks.
Remove from the bindings the need to set all blocks register
but keep reg-names property because removing it from the driver
may affect other variants.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2:
- be more verbose about why I change the bindings
Keep in mind that series comes after: https://www.spinics.net/lists/arm-kernel/msg875766.html
without that review and ack it won't work

 .../bindings/media/nxp,imx8mq-vpu.yaml        | 54 ++++++++++++-------
 1 file changed, 36 insertions(+), 18 deletions(-)

diff --git a/Documentation/devicetree/bindings/media/nxp,imx8mq-vpu.yaml b/Documentation/devicetree/bindings/media/nxp,imx8mq-vpu.yaml
index 762be3f96ce9..468435c70eef 100644
--- a/Documentation/devicetree/bindings/media/nxp,imx8mq-vpu.yaml
+++ b/Documentation/devicetree/bindings/media/nxp,imx8mq-vpu.yaml
@@ -15,24 +15,25 @@ description:
 
 properties:
   compatible:
-    const: nxp,imx8mq-vpu
+    enum:
+      - nxp,imx8mq-vpu
+      - nxp,imx8mq-vpu-g2
 
   reg:
-    maxItems: 3
+    maxItems: 1
 
   reg-names:
-    items:
-      - const: g1
-      - const: g2
-      - const: ctrl
+    enum:
+      - g1
+      - g2
 
   interrupts:
-    maxItems: 2
+    maxItems: 1
 
   interrupt-names:
-    items:
-      - const: g1
-      - const: g2
+    enum:
+      - g1
+      - g2
 
   clocks:
     maxItems: 3
@@ -46,6 +47,9 @@ properties:
   power-domains:
     maxItems: 1
 
+  resets:
+    maxItems: 1
+
 required:
   - compatible
   - reg
@@ -54,6 +58,7 @@ required:
   - interrupt-names
   - clocks
   - clock-names
+  - resets
 
 additionalProperties: false
 
@@ -61,19 +66,32 @@ examples:
   - |
         #include <dt-bindings/clock/imx8mq-clock.h>
         #include <dt-bindings/interrupt-controller/arm-gic.h>
+        #include <dt-bindings/reset/imx8mq-vpu-reset.h>
 
-        vpu: video-codec@38300000 {
+        vpu_g1: video-codec@38300000 {
                 compatible = "nxp,imx8mq-vpu";
-                reg = <0x38300000 0x10000>,
-                      <0x38310000 0x10000>,
-                      <0x38320000 0x10000>;
-                reg-names = "g1", "g2", "ctrl";
-                interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
-                             <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
-                interrupt-names = "g1", "g2";
+                reg = <0x38300000 0x10000>;
+                reg-names = "g1";
+                interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
+                interrupt-names = "g1";
+                clocks = <&clk IMX8MQ_CLK_VPU_G1_ROOT>,
+                         <&clk IMX8MQ_CLK_VPU_G2_ROOT>,
+                         <&clk IMX8MQ_CLK_VPU_DEC_ROOT>;
+                clock-names = "g1", "g2", "bus";
+                power-domains = <&pgc_vpu>;
+                resets = <&vpu_reset IMX8MQ_RESET_VPU_RESET_G1>;
+        };
+
+        vpu_g2: video-codec@38310000 {
+                compatible = "nxp,imx8mq-vpu-g2";
+                reg = <0x38310000 0x10000>;
+                reg-names = "g2";
+                interrupts = <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
+                interrupt-names = "g2";
                 clocks = <&clk IMX8MQ_CLK_VPU_G1_ROOT>,
                          <&clk IMX8MQ_CLK_VPU_G2_ROOT>,
                          <&clk IMX8MQ_CLK_VPU_DEC_ROOT>;
                 clock-names = "g1", "g2", "bus";
                 power-domains = <&pgc_vpu>;
+                resets = <&vpu_reset IMX8MQ_RESET_VPU_RESET_G2>;
         };
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

[PATCH v3 9/9] arm64: dts: imx8mq: Add node to G2 hardware

[Benjamin Gaignard]

Split VPU node in two: one for G1 and one for G2 since they are
different hardware blocks.

Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2:
- remove useless clocks in VPUs nodes

 arch/arm64/boot/dts/freescale/imx8mq.dtsi | 41 +++++++++++++++++------
 1 file changed, 31 insertions(+), 10 deletions(-)

diff --git a/arch/arm64/boot/dts/freescale/imx8mq.dtsi b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
index d9d9efc8592d..8358e214d696 100644
--- a/arch/arm64/boot/dts/freescale/imx8mq.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mq.dtsi
@@ -1287,17 +1287,15 @@ vpu_reset: vpu-reset@38320000 {
 			#reset-cells = <1>;
 		};
 
-		vpu: video-codec@38300000 {
+		vpu_g1: video-codec@38300000 {
 			compatible = "nxp,imx8mq-vpu";
-			reg = <0x38300000 0x10000>,
-			      <0x38310000 0x10000>;
-			reg-names = "g1", "g2";
-			interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>,
-				     <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
-			interrupt-names = "g1", "g2";
+			reg = <0x38300000 0x10000>;
+			reg-names = "g1";
+			interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
+			interrupt-names = "g1";
 			clocks = <&clk IMX8MQ_CLK_VPU_G1_ROOT>,
-				 <&clk IMX8MQ_CLK_VPU_G2_ROOT>;
-			clock-names = "g1", "g2";
+				 <&clk IMX8MQ_CLK_VPU_DEC_ROOT>;
+			clock-names = "g1", "bus";
 			assigned-clocks = <&clk IMX8MQ_CLK_VPU_G1>,
 					  <&clk IMX8MQ_CLK_VPU_G2>,
 					  <&clk IMX8MQ_CLK_VPU_BUS>,
@@ -1306,12 +1304,35 @@ vpu: video-codec@38300000 {
 						 <&clk IMX8MQ_VPU_PLL_OUT>,
 						 <&clk IMX8MQ_SYS1_PLL_800M>,
 						 <&clk IMX8MQ_VPU_PLL>;
-			assigned-clock-rates = <600000000>, <600000000>,
+			assigned-clock-rates = <600000000>, <300000000>,
 					       <800000000>, <0>;
 			resets = <&vpu_reset IMX8MQ_RESET_VPU_RESET_G1>;
 			power-domains = <&pgc_vpu>;
 		};
 
+		vpu_g2: video-codec@38310000 {
+			compatible = "nxp,imx8mq-vpu-g2";
+			reg = <0x38310000 0x10000>;
+			reg-names = "g2";
+			interrupts = <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
+			interrupt-names = "g2";
+			clocks = <&clk IMX8MQ_CLK_VPU_G2_ROOT>,
+				 <&clk IMX8MQ_CLK_VPU_DEC_ROOT>;
+			clock-names = "g2",  "bus";
+			assigned-clocks = <&clk IMX8MQ_CLK_VPU_G1>,
+					  <&clk IMX8MQ_CLK_VPU_G2>,
+					  <&clk IMX8MQ_CLK_VPU_BUS>,
+					  <&clk IMX8MQ_VPU_PLL_BYPASS>;
+			assigned-clock-parents = <&clk IMX8MQ_VPU_PLL_OUT>,
+						 <&clk IMX8MQ_VPU_PLL_OUT>,
+						 <&clk IMX8MQ_SYS1_PLL_800M>,
+						 <&clk IMX8MQ_VPU_PLL>;
+			assigned-clock-rates = <600000000>, <300000000>,
+					       <800000000>, <0>;
+			resets = <&vpu_reset IMX8MQ_RESET_VPU_RESET_G2>;
+			power-domains = <&pgc_vpu>;
+		};
+
 		pcie0: pcie@33800000 {
 			compatible = "fsl,imx8mq-pcie";
 			reg = <0x33800000 0x400000>,
-- 
2.25.1


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 8/9] dt-bindings: media: nxp,imx8mq-vpu: Update bindings

[Rob Herring]

On Mon, Feb 22, 2021 at 01:24:05PM +0100, Benjamin Gaignard wrote:
> The current bindings seem to make the assumption that the
> two VPUs hardware blocks (G1 and G2) are only one set of
> registers.
> After implementing the VPU reset driver and G2 decoder driver
> it shows that all the VPUs are independent and don't need to
> know about the registers of the other blocks.
> Remove from the bindings the need to set all blocks register
> but keep reg-names property because removing it from the driver
> may affect other variants.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> version 2:
> - be more verbose about why I change the bindings
> Keep in mind that series comes after: https://www.spinics.net/lists/arm-kernel/msg875766.html
> without that review and ack it won't work

Better, but you've still mentioned nothing about breaking compatibility.
Why is that okay?

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 8/9] dt-bindings: media: nxp,imx8mq-vpu: Update bindings

[Benjamin Gaignard]

Le 23/02/2021 à 01:34, Rob Herring a écrit :
> On Mon, Feb 22, 2021 at 01:24:05PM +0100, Benjamin Gaignard wrote:
>> The current bindings seem to make the assumption that the
>> two VPUs hardware blocks (G1 and G2) are only one set of
>> registers.
>> After implementing the VPU reset driver and G2 decoder driver
>> it shows that all the VPUs are independent and don't need to
>> know about the registers of the other blocks.
>> Remove from the bindings the need to set all blocks register
>> but keep reg-names property because removing it from the driver
>> may affect other variants.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> version 2:
>> - be more verbose about why I change the bindings
>> Keep in mind that series comes after: https://www.spinics.net/lists/arm-kernel/msg875766.html
>> without that review and ack it won't work
> Better, but you've still mentioned nothing about breaking compatibility.
> Why is that okay?

Because this reg-names wasn't used before for this variant so remove it won't change anything.

>

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 8/9] dt-bindings: media: nxp, imx8mq-vpu: Update bindings

[Rob Herring]

On Tue, Feb 23, 2021 at 2:04 AM Benjamin Gaignard
<benjamin.gaignard@collabora.com> wrote:
>
>
> Le 23/02/2021 à 01:34, Rob Herring a écrit :
> > On Mon, Feb 22, 2021 at 01:24:05PM +0100, Benjamin Gaignard wrote:
> >> The current bindings seem to make the assumption that the
> >> two VPUs hardware blocks (G1 and G2) are only one set of
> >> registers.
> >> After implementing the VPU reset driver and G2 decoder driver
> >> it shows that all the VPUs are independent and don't need to
> >> know about the registers of the other blocks.
> >> Remove from the bindings the need to set all blocks register
> >> but keep reg-names property because removing it from the driver
> >> may affect other variants.
> >>
> >> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >> ---
> >> version 2:
> >> - be more verbose about why I change the bindings
> >> Keep in mind that series comes after: https://www.spinics.net/lists/arm-kernel/msg875766.html
> >> without that review and ack it won't work
> > Better, but you've still mentioned nothing about breaking compatibility.
> > Why is that okay?
>
> Because this reg-names wasn't used before for this variant so remove it won't change anything.

It is the reset changes in the driver that break. The driver
previously got the 'ctrl' registers whether it went by name or index,
right? With an old DTB and a kernel with the changes (and vice-versa),
you'll have nothing to handle the VPU resets because the VPU reset
node doesn't exist. It could work if the default state is not held in
reset.

At least the removal of 'ctrl' registers belongs in the reset changes series.

Rob

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 8/9] dt-bindings: media: nxp,imx8mq-vpu: Update bindings

[Ezequiel Garcia]

Hi Rob,

On Tue, 2021-02-23 at 08:31 -0600, Rob Herring wrote:
> On Tue, Feb 23, 2021 at 2:04 AM Benjamin Gaignard
> <benjamin.gaignard@collabora.com> wrote:
> > 
> > 
> > Le 23/02/2021 à 01:34, Rob Herring a écrit :
> > > On Mon, Feb 22, 2021 at 01:24:05PM +0100, Benjamin Gaignard wrote:
> > > > The current bindings seem to make the assumption that the
> > > > two VPUs hardware blocks (G1 and G2) are only one set of
> > > > registers.
> > > > After implementing the VPU reset driver and G2 decoder driver
> > > > it shows that all the VPUs are independent and don't need to
> > > > know about the registers of the other blocks.
> > > > Remove from the bindings the need to set all blocks register
> > > > but keep reg-names property because removing it from the driver
> > > > may affect other variants.
> > > > 
> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > ---
> > > > version 2:
> > > > - be more verbose about why I change the bindings
> > > > Keep in mind that series comes after: https://www.spinics.net/lists/arm-kernel/msg875766.html
> > > > without that review and ack it won't work
> > > Better, but you've still mentioned nothing about breaking compatibility.
> > > Why is that okay?
> > 

Indeed, the commit description should be clearer about breaking compatibility.

> > Because this reg-names wasn't used before for this variant so remove it won't change anything.
> 
> It is the reset changes in the driver that break. The driver
> previously got the 'ctrl' registers whether it went by name or index,
> right? With an old DTB and a kernel with the changes (and vice-versa),
> you'll have nothing to handle the VPU resets because the VPU reset
> node doesn't exist. It could work if the default state is not held in
> reset.
> 
> At least the removal of 'ctrl' registers belongs in the reset changes series.
> 
> 

Considering that FFMPEG patches that are required to support this driver
are still floating around, and GStreamer's implementation is also still
a bit under discussion, we are certain there aren't many upstreams users
(leaving ChromiumOS aside which mostly care for Rockchip variants).

So, given the driver is in staging, and that there aren't users of the
i.MX8MQ G1 variant just yet, I think we are safe breaking the compatibility
(and I'm not taking it lightly).

It would be important to detect an old devicetree and do some pr_warn about
the driver not supporting it.

Thanks,
Ezequiel


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 0/9] Add HANTRO G2/HEVC decoder support for IMX8MQ

[Ezequiel Garcia]

Hi Benjamin,

On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> The IMX8MQ got two VPUs but until now only G1 has been enabled.
> This series aim to add the second VPU (aka G2) and provide basic 
> HEVC decoding support.
> 
> To be able to decode HEVC it is needed to add/update some of the
> structures in the uapi. In addition of them one HANTRO dedicated
> control is required to inform the driver of the numbre of bits to skip
> at the beginning of the slice header.
> The hardware require to allocate few auxiliary buffers to store the
> references frame or tile size data.
> 
> The driver has been tested with fluster test suite stream.
> For example with this command: ./fluster.py run -ts JCT-VC-HEVC_V1 -d GStreamer-H.265-V4L2SL-Gst1.0
>  
> This series depends of the reset rework posted here: https://www.spinics.net/lists/arm-kernel/msg875766.html
> 
> Finally the both VPUs will have a node the device-tree and be
> independent from v4l2 point of view.
> 
> A branch with all the dev is available here:
> https://gitlab.collabora.com/benjamin.gaignard/for-upstream/-/commits/upstream_g2_v2
> 
> version 3:
> - Fix typo in Hantro v4l2 dedicated control
> - Add documentation for the new structures and fields
> - Rebased on top of media_tree for-linus-5.12-rc1 tag
> 
> version 2:
> - remove all change related to scaling
> - squash commits to a coherent split
> - be more verbose about the added fields
> - fix the comments done by Ezequiel about dma_alloc_coherent usage
> - fix Dan's comments about control copy, reverse the test logic
> in tile_buffer_reallocate, rework some goto and return cases.
> - be more verbose about why I change the bindings
> - remove all sign-off expect mime since it is confusing

I don't think it's about having less confusing commits, but
trying to describe fairly accurately the origin of the work
and the authorship of changes.

In this series I believe we have three cases, please
correct me if I'm wrong:

* Changes that you have authored, for instance it would
seem to me that "dt-bindings: media: nxp,imx8mq-vpu: Update bindings",
and "media: uapi: Add a control for HANTRO driver" would be in
that category.

* Changes that Adrian and/or me have initially authored,
where both Adrian and me did further work, and also
where you did significant work on as well.

IOW, changes where the three of us did significant work.

I guess in this case, the original author could be the
author of the patch (git's Author tag), and additionally
authorship would be indicated with Co-developed-by tags.

commit 494adacd844e5656b570895a82bc343438b23023
Author: Adrian Ratiu <adrian.ratiu@collabora.com>
Date:   Thu Feb 18 12:17:37 2021 +0100

    media: hantro: Introduce G2/HEVC decoder

    ...

    Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
    Co-developed-by: Ezequiel Garcia <ezequiel@collabora.com>
    Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
    Co-developed-by: Benjamin Gaignard <benjamin.gaignard@collabora.com> 
    Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>

* And finally changes with an original author (Adrian or me),
which you haven't changed, but you are just submitting
(or you did minor work).

Maybe with the current way of splitting commits we don't have
these type of patches, but just for the sake of it, let's say
there's a commit authored by Adrian that you are mostly picking up,
it would be:

commit 896776d3bcd032808e4d5772e6749da5dd4eec42
Author: Adrian Ratiu <adrian.ratiu@collabora.com>
Date:   Fri Feb 5 12:14:16 2021 +0100

    media: hantro: Some change
    
    ...

    Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>
    Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>


Thanks a lot!
Ezequiel

> - remove useless clocks in VPUs nodes
> 
> Benjamin
> 
> Benjamin Gaignard (9):
>   media: hevc: Modify structures to follow H265 ITU spec
>   media: hantro: Define HEVC codec profiles and supported features
>   media: hantro: Add a field to distinguish the hardware versions
>   media: uapi: Add a control for HANTRO driver
>   media: hantro: Introduce G2/HEVC decoder
>   media: hantro: handle V4L2_PIX_FMT_HEVC_SLICE control
>   media: hantro: IMX8M: add variant for G2/HEVC codec
>   dt-bindings: media: nxp,imx8mq-vpu: Update bindings
>   arm64: dts: imx8mq: Add node to G2 hardware
> 
>  .../bindings/media/nxp,imx8mq-vpu.yaml        |  54 +-
>  .../media/v4l/ext-ctrls-codec.rst             | 126 +++-
>  .../media/v4l/vidioc-queryctrl.rst            |   6 +
>  arch/arm64/boot/dts/freescale/imx8mq.dtsi     |  41 +-
>  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +-
>  drivers/staging/media/hantro/Makefile         |   2 +
>  drivers/staging/media/hantro/hantro.h         |  34 +-
>  drivers/staging/media/hantro/hantro_drv.c     | 103 +++
>  .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++++++++++++++++++
>  drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++++++
>  drivers/staging/media/hantro/hantro_hevc.c    | 321 ++++++++++
>  drivers/staging/media/hantro/hantro_hw.h      |  48 ++
>  .../staging/media/hantro/hantro_postproc.c    |  17 +
>  drivers/staging/media/hantro/hantro_v4l2.c    |   1 +
>  drivers/staging/media/hantro/imx8m_vpu_hw.c   |  95 ++-
>  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
>  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
>  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
>  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
>  include/media/hevc-ctrls.h                    |  45 +-
>  include/uapi/linux/hantro-v4l2-controls.h     |  20 +
>  include/uapi/linux/v4l2-controls.h            |   5 +
>  22 files changed, 1674 insertions(+), 70 deletions(-)
>  create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>  create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
>  create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
>  create mode 100644 include/uapi/linux/hantro-v4l2-controls.h
> 



_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 2/9] media: hantro: Define HEVC codec profiles and supported features

[Ezequiel Garcia]

On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> Define which HEVC profiles (up to level 5.1) and features
> (no scaling, no 10 bits) are supported by the driver.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
>  drivers/staging/media/hantro/hantro.h     |  2 +
>  drivers/staging/media/hantro/hantro_drv.c | 58 +++++++++++++++++++++++
>  2 files changed, 60 insertions(+)
> 
> diff --git a/drivers/staging/media/hantro/hantro.h b/drivers/staging/media/hantro/hantro.h
> index 65f9f7ea7dcf..bde65231f22f 100644
> --- a/drivers/staging/media/hantro/hantro.h
> +++ b/drivers/staging/media/hantro/hantro.h
> @@ -99,6 +99,7 @@ struct hantro_variant {
>   * @HANTRO_MODE_H264_DEC: H264 decoder.
>   * @HANTRO_MODE_MPEG2_DEC: MPEG-2 decoder.
>   * @HANTRO_MODE_VP8_DEC: VP8 decoder.
> + * @HANTRO_MODE_HEVC_DEC: HEVC decoder.
>   */
>  enum hantro_codec_mode {
>         HANTRO_MODE_NONE = -1,
> @@ -106,6 +107,7 @@ enum hantro_codec_mode {
>         HANTRO_MODE_H264_DEC,
>         HANTRO_MODE_MPEG2_DEC,
>         HANTRO_MODE_VP8_DEC,
> +       HANTRO_MODE_HEVC_DEC,
>  };
>  
>  /*
> diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
> index e5f200e64993..d86e322a5980 100644
> --- a/drivers/staging/media/hantro/hantro_drv.c
> +++ b/drivers/staging/media/hantro/hantro_drv.c
> @@ -243,6 +243,18 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
>                 if (sps->bit_depth_luma_minus8 != 0)
>                         /* Only 8-bit is supported */
>                         return -EINVAL;
> +       } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
> +               const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
> +
> +               if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
> +                       /* Luma and chroma bit depth mismatch */
> +                       return -EINVAL;
> +               if (sps->bit_depth_luma_minus8 != 0)
> +                       /* Only 8-bit is supported */
> +                       return -EINVAL;
> +               if (sps->flags & V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED)
> +                       /* No scaling support */
> +                       return -EINVAL;
>         }
>         return 0;
>  }
> @@ -349,6 +361,52 @@ static const struct hantro_ctrl controls[] = {
>                         .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
>                 }
>         }, {
> +               .codec = HANTRO_HEVC_DECODER,

Silly nitpick. Looks like this is not defined yet?

I'm getting:

drivers/staging/media/hantro/hantro_drv.c:364:12: error: ‘HANTRO_HEVC_DECODER’ undeclared here (not in a function); did you mean
‘HANTRO_H264_DECODER’?
  364 |   .codec = HANTRO_HEVC_DECODER,
      |            ^~~~~~~~~~~~~~~~~~~
      |            HANTRO_H264_DECODER

I'll review the G2 driver soon :-)

Thanks,
Ezequiel

> +               .cfg = {
> +                       .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
> +                       .min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> +                       .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> +                       .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> +               },
> +       }, {
> +               .codec = HANTRO_HEVC_DECODER,
> +               .cfg = {
> +                       .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
> +                       .min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> +                       .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> +                       .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> +               },
> +       }, {
> +               .codec = HANTRO_HEVC_DECODER,
> +               .cfg = {
> +                       .id = V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
> +                       .min = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> +                       .max = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10,
> +                       .def = V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN,
> +               },
> +       }, {
> +               .codec = HANTRO_HEVC_DECODER,
> +               .cfg = {
> +                       .id = V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
> +                       .min = V4L2_MPEG_VIDEO_HEVC_LEVEL_1,
> +                       .max = V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1,
> +               },
> +       }, {
> +               .codec = HANTRO_HEVC_DECODER,
> +               .cfg = {
> +                       .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
> +                       .ops = &hantro_ctrl_ops,
> +               },
> +       }, {
> +               .codec = HANTRO_HEVC_DECODER,
> +               .cfg = {
> +                       .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
> +               },
> +       }, {
> +               .codec = HANTRO_HEVC_DECODER,
> +               .cfg = {
> +                       .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
> +               },
>         },
>  };
>  



_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[Ezequiel Garcia]

Hi Benjamin,

Thanks for the good work.

On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> The H.265 ITU specification (section 7.4) define the general
> slice segment header semantics.
> Modified/added fields are:
> - video_parameter_set_id: (7.4.3.1) identifies the VPS for
> reference by other syntax elements.
> - seq_parameter_set_id: (7.4.3.2.1) specifies the value of
> the vps_video_parameter_set_id of the active VPS.
> - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
>  relative to the luma sampling
> - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
> reference by other syntax elements
> - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
> the inferred value of num_ref_idx_l0_active_minus1
> - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
> the inferred value of num_ref_idx_l1_active_minus1
> - slice_segment_addr: (7.4.7.1) specifies the address of
> the first coding tree block in the slice segment
> - num_entry_point_offsets: (7.4.7.1) specifies the number of
> entry_point_offset_minus1[ i ] syntax elements in the slice header
> 
> Add HEVC decode params contains the information used in section
> "8.3 Slice decoding process" of the specification to let the hardware
> perform decoding of a slices.
> 
> Adapt Cedrus driver according to these changes.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> version 3:
> - Add documentation about the new structuers and fields.
> 
> version 2:
> - remove all change related to scaling
> - squash commits to a coherent split
> - be more verbose about the added fields
> 
>  .../media/v4l/ext-ctrls-codec.rst             | 126 +++++++++++++++---
>  .../media/v4l/vidioc-queryctrl.rst            |   6 +
>  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
>  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
>  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
>  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
>  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
>  include/media/hevc-ctrls.h                    |  45 +++++--
>  8 files changed, 186 insertions(+), 32 deletions(-)
> 
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 00944e97d638..5e6d77e858c0 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>      :stub-columns: 0
>      :widths:       1 1 2
>  
> +    * - __u8
> +      - ``video_parameter_set_id``
> +      - Identifies the VPS for reference by other syntax elements
> +    * - __u8
> +      - ``seq_parameter_set_id̀``
> +      - Specifies the value of the vps_video_parameter_set_id of the active VPS
> +    * - __u8
> +      - ``chroma_format_idc``
> +      - Specifies the chroma sampling relative to the luma sampling

None of these fields seem needed for the Hantro G2 driver,
so I suggest you drop them for now.

>      * - __u16
>        - ``pic_width_in_luma_samples``
>        -
> @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>      * - __u8
>        - ``chroma_format_idc``
>        -
> +    * - __u8
> +      - ``num_slices``
> +

Not used, but also doesn't seem part of the SPS syntax. If we have to
pass the number of slices, we'll need another mechanism.

>       -
>      * - __u64
>        - ``flags``
>        - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
> @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>      :stub-columns: 0
>      :widths:       1 1 2
>  
> +    * - __u8
> +      - ``pic_parameter_set_id``
> +      - Identifies the PPS for reference by other syntax elements

Not used.

>      * - __u8
>        - ``num_extra_slice_header_bits``
>        -
> +    * - __u8
> +      - ``num_ref_idx_l0_default_active_minus1``
> +      - Specifies the inferred value of num_ref_idx_l0_active_minus1
> +    * - __u8
> +      - ``num_ref_idx_l1_default_active_minus1``
> +      - Specifies the inferred value of num_ref_idx_l1_active_minus1
>      * - __s8
>        - ``init_qp_minus26``
>        -
> @@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>      * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
>        - 0x00040000
>        -
> +    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> +      - 0x00080000
> +      -
> +    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
> +      - 0x00100000
> +      -
>  

I suggest to do all the PPS control changes in a separate patch,
feels easier to review and cleaner as you can explain the
changes with more detail in the commit description.

Looking at the PPS syntax for tiles, I'm wondering if these
deserve their own control, which would be used if tiles are enabled,
i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set.

        __u8    num_tile_columns_minus1;                                         
        __u8    num_tile_rows_minus1;                                            
        __u8    column_width_minus1[20];                                         
        __u8    row_height_minus1[22];    

Not something we necessarily have to tackle now.

>  ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
>      Specifies various slice-specific parameters, especially from the NAL unit
> @@ -3366,6 +3393,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>      * - __u32
>        - ``data_bit_offset``
>        - Offset (in bits) to the video data in the current slice data.
> +    * - __u32
> +      - ``slice_segment_addr``
> +      - Specifies the address of the first coding tree block in the slice segment

Not used.

> +    * - __u32
> +      - ``num_entry_point_offsets``
> +      - Specifies the number of entry_point_offset_minus1[ i ] syntax elements in the slice header

Not used.

>      * - __u8
>        - ``nal_unit_type``
>        -
> @@ -3422,28 +3455,20 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>      * - __u8
>        - ``pic_struct``
>        -
> -    * - __u8
> -      - ``num_active_dpb_entries``
> -      - The number of entries in ``dpb``.

Need to explain in the commit description why this field is moved.

>      * - __u8
>        - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>        - The list of L0 reference elements as indices in the DPB.
>      * - __u8
>        - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>        - The list of L1 reference elements as indices in the DPB.
> +    * - __u16
> +      - ``short_term_ref_pic_set_size``
> +

Not used.

>       -
> +    * - __u16
> +      - ``long_term_ref_pic_set_size``
> +      -

Not used.

>      * - __u8
> -      - ``num_rps_poc_st_curr_before``
> -      - The number of reference pictures in the short-term set that come before
> -        the current frame.

If this matches NumPocStCurrBefore from section 8.3.2 "Decoding process for reference picture set"
then I would document that. And perhaps rename it to num_poc_st_curr_before.

> -    * - __u8
> -      - ``num_rps_poc_st_curr_after``
> -      - The number of reference pictures in the short-term set that come after
> -        the current frame.

Ditto.

> -    * - __u8
> -      - ``num_rps_poc_lt_curr``
> -      - The number of reference pictures in the long-term set.

Ditto.

Also, I'd like the changes that move fields from V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS
to the new V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS control, to be in their
patch.

That will allow us to put in the commit description a proper
explanation of why are fields being moved. Nothing fancy, simply
explaining that these variables come from section 8.3.2
"Decoding process for reference picture set", which describes
a process invoked once per picture, so they are not per-slice.

> -    * - __u8
> -      - ``padding[7]``
> +      - ``padding``
>        - Applications and drivers must set this to zero.
>      * - struct :c:type:`v4l2_hevc_dpb_entry`
>        - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> @@ -3646,3 +3671,74 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>      so this has to come from client.
>      This is applicable to H264 and valid Range is from 0 to 63.
>      Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
> +
> +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> +    Specifies various decode parameters, especially the references picture order
> +    count (POC) for all the lists (short, long, before, current, after) and the
> +    number of entries for each of them.
> +    These parameters are defined according to :ref:`hevc`.
> +    They are described in section 8.3 "Slice decoding process" of the
> +    specification.
> +
> +.. c:type:: v4l2_ctrl_hevc_decode_params
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: struct v4l2_ctrl_hevc_decode_params
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       1 1 2
> +
> +    * - __s32
> +      - ``pic_order_cnt_val``
> +      -

Can be documented as:

"""
PicOrderCntVal as described in section 8.3.1 "Decoding process
for picture order count" of the specification.
"""

Note that snake case is used to match the kernel style,
but other than that we try to keep the HEVC spec variable
names.

> +    * - __u8
> +      - ``num_active_dpb_entries``
> +      - The number of entries in ``dpb``.
> +    * - struct :c:type:`v4l2_hevc_dpb_entry`
> +      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> +      - The decoded picture buffer, for meta-data about reference frames.

The DPB is here, but it seems it's also in the slice control?

> +    * - __u8
> +      - ``num_rps_poc_st_curr_before``
> +      - The number of reference pictures in the short-term set that come before
> +        the current frame.
> +    * - __u8
> +      - ``num_rps_poc_st_curr_after``
> +      - The number of reference pictures in the short-term set that come after
> +        the current frame.
> +    * - __u8
> +      - ``num_rps_poc_lt_curr``
> +      - The number of reference pictures in the long-term set.
> +    * - __u8
> +      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> +      -
> +    * - __u8
> +      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> +      -
> +    * - __u8
> +      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> +      -

Could you document these as well?

Thanks a lot,
Ezequiel


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 4/9] media: uapi: Add a control for HANTRO driver

[Ezequiel Garcia]

Hi Benjamin,

On Mon, 2021-02-22 at 13:24 +0100, Benjamin Gaignard wrote:
> The HEVC HANTRO driver needs to know the number of bits to skip at

s/HANTRO/Hantro

> the beginning of the slice header.

As discussed in a different thread, we should describe exactly
what the hardware is expecting, so applications can parse that
and pass a correct value.

> That is a hardware specific requirement so create a dedicated control
> that this purpose.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> version 3:
> - Fix typo in field name
> 
>  include/uapi/linux/hantro-v4l2-controls.h | 20 ++++++++++++++++++++
>  include/uapi/linux/v4l2-controls.h        |  5 +++++
>  2 files changed, 25 insertions(+)
>  create mode 100644 include/uapi/linux/hantro-v4l2-controls.h
> 
> diff --git a/include/uapi/linux/hantro-v4l2-controls.h b/include/uapi/linux/hantro-v4l2-controls.h
> new file mode 100644
> index 000000000000..a8dfd6b1a2a9
> --- /dev/null
> +++ b/include/uapi/linux/hantro-v4l2-controls.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +
> +#ifndef __UAPI_HANTRO_V4L2_CONYTROLS_H__
> +#define __UAPI_HANTRO_V4L2_CONYTROLS_H__
> +
> +#include <linux/v4l2-controls.h>
> +#include <media/hevc-ctrls.h>
> +
> +#define V4L2_CID_HANTRO_HEVC_EXTRA_DECODE_PARAMS       (V4L2_CID_USER_HANTRO_BASE + 0)
> +
> +/**
> + * struct hantro_hevc_extra_decode_params - extra decode parameters for hantro driver
> + * @hevc_hdr_skip_length:      header first bits offset
> + */
> +struct hantro_hevc_extra_decode_params {
> +       __u32   hevc_hdr_skip_length;
> +       __u8    padding[4];
> +};
> +

I think we can get away with a simpler solution. Since it's just one integer
we need, there's no need for a compound control. Something like this:

                .codec = HANTRO_HEVC_DECODER,                                    
                .cfg = {                                                         
                        .id = V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP,            
                        .name = "Hantro HEVC slice header skip bytes",           
                        .type = V4L2_CTRL_TYPE_INTEGER,                          
                        .min = 0,                                                
                        .max = 0x7fffffff,                                       
                        .step = 1,                                               
                },     

Also see V4L2_CID_CODA_MB_ERR_CNT which is defined in drivers/media/platform/coda/coda.h.
The control is sufficiently special that it could be kept in an internal driver header.

> +#endif
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 039c0d7add1b..ced7486c7f46 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -209,6 +209,11 @@ enum v4l2_colorfx {
>   * We reserve 128 controls for this driver.
>   */
>  #define V4L2_CID_USER_CCS_BASE                 (V4L2_CID_USER_BASE + 0x10f0)
> +/*
> + * The base for HANTRO driver controls.
> + * We reserve 32 controls for this driver.
> + */
> +#define V4L2_CID_USER_HANTRO_BASE              (V4L2_CID_USER_BASE + 0x1170)
>  
>  /* MPEG-class control IDs */
>  /* The MPEG controls are applicable to all codec controls

Thanks,
Ezequiel


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[Jernej Škrabec]

Hi Ezequiel,

Dne četrtek, 25. februar 2021 ob 14:09:52 CET je Ezequiel Garcia napisal(a):
> Hi Benjamin,
> 
> Thanks for the good work.
> 
> On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> > The H.265 ITU specification (section 7.4) define the general
> > slice segment header semantics.
> > Modified/added fields are:
> > - video_parameter_set_id: (7.4.3.1) identifies the VPS for
> > reference by other syntax elements.
> > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of
> > the vps_video_parameter_set_id of the active VPS.
> > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
> >  relative to the luma sampling
> > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
> > reference by other syntax elements
> > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
> > the inferred value of num_ref_idx_l0_active_minus1
> > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
> > the inferred value of num_ref_idx_l1_active_minus1
> > - slice_segment_addr: (7.4.7.1) specifies the address of
> > the first coding tree block in the slice segment
> > - num_entry_point_offsets: (7.4.7.1) specifies the number of
> > entry_point_offset_minus1[ i ] syntax elements in the slice header
> > 
> > Add HEVC decode params contains the information used in section
> > "8.3 Slice decoding process" of the specification to let the hardware
> > perform decoding of a slices.
> > 
> > Adapt Cedrus driver according to these changes.
> > 
> > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > ---
> > version 3:
> > - Add documentation about the new structuers and fields.
> > 
> > version 2:
> > - remove all change related to scaling
> > - squash commits to a coherent split
> > - be more verbose about the added fields
> > 
> >  .../media/v4l/ext-ctrls-codec.rst             | 126 +++++++++++++++---
> >  .../media/v4l/vidioc-queryctrl.rst            |   6 +
> >  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
> >  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
> >  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
> >  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
> >  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
> >  include/media/hevc-ctrls.h                    |  45 +++++--
> >  8 files changed, 186 insertions(+), 32 deletions(-)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > index 00944e97d638..5e6d77e858c0 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      :stub-columns: 0
> >      :widths:       1 1 2
> >  
> > +    * - __u8
> > +      - ``video_parameter_set_id``
> > +      - Identifies the VPS for reference by other syntax elements
> > +    * - __u8
> > +      - ``seq_parameter_set_id̀``
> > +      - Specifies the value of the vps_video_parameter_set_id of the 
active VPS
> > +    * - __u8
> > +      - ``chroma_format_idc``
> > +      - Specifies the chroma sampling relative to the luma sampling
> 
> None of these fields seem needed for the Hantro G2 driver,
> so I suggest you drop them for now.
> 
> >      * - __u16
> >        - ``pic_width_in_luma_samples``
> >        -
> > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - __u8
> >        - ``chroma_format_idc``
> >        -
> > +    * - __u8
> > +      - ``num_slices``
> > +
> 
> Not used, but also doesn't seem part of the SPS syntax. If we have to
> pass the number of slices, we'll need another mechanism.
> 
> >       -
> >      * - __u64
> >        - ``flags``
> >        - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
> > @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      :stub-columns: 0
> >      :widths:       1 1 2
> >  
> > +    * - __u8
> > +      - ``pic_parameter_set_id``
> > +      - Identifies the PPS for reference by other syntax elements
> 
> Not used.
> 
> >      * - __u8
> >        - ``num_extra_slice_header_bits``
> >        -
> > +    * - __u8
> > +      - ``num_ref_idx_l0_default_active_minus1``
> > +      - Specifies the inferred value of num_ref_idx_l0_active_minus1
> > +    * - __u8
> > +      - ``num_ref_idx_l1_default_active_minus1``
> > +      - Specifies the inferred value of num_ref_idx_l1_active_minus1
> >      * - __s8
> >        - ``init_qp_minus26``
> >        -
> > @@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
> >        - 0x00040000
> >        -
> > +    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> > +      - 0x00080000
> > +      -
> > +    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
> > +      - 0x00100000
> > +      -
> >  
> 
> I suggest to do all the PPS control changes in a separate patch,
> feels easier to review and cleaner as you can explain the
> changes with more detail in the commit description.
> 
> Looking at the PPS syntax for tiles, I'm wondering if these
> deserve their own control, which would be used if tiles are enabled,
> i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set.
> 
>         __u8    num_tile_columns_minus1;                                         
>         __u8    num_tile_rows_minus1;                                            
>         __u8    column_width_minus1[20];                                         
>         __u8    row_height_minus1[22];    
> 
> Not something we necessarily have to tackle now.
> 
> >  ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
> >      Specifies various slice-specific parameters, especially from the NAL 
unit
> > @@ -3366,6 +3393,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - __u32
> >        - ``data_bit_offset``
> >        - Offset (in bits) to the video data in the current slice data.
> > +    * - __u32
> > +      - ``slice_segment_addr``
> > +      - Specifies the address of the first coding tree block in the slice 
segment
> 
> Not used.
> 
> > +    * - __u32
> > +      - ``num_entry_point_offsets``
> > +      - Specifies the number of entry_point_offset_minus1[ i ] syntax 
elements in the slice header
> 
> Not used.

While above two fields may not be used in Hantro, they are for sure useful for 
Cedrus and RPi4. I would like to keep them, otherwise with such approach HEVC 
will stay in staging for a long time. I'm still baffled why scaling matrix 
control was dropped. It would fit well in Cedrus and RPi4 driver and after a 
quick look, it seems that it was used in driver in later patch.

Best regards,
Jernej

> 
> >      * - __u8
> >        - ``nal_unit_type``
> >        -
> > @@ -3422,28 +3455,20 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - __u8
> >        - ``pic_struct``
> >        -
> > -    * - __u8
> > -      - ``num_active_dpb_entries``
> > -      - The number of entries in ``dpb``.
> 
> Need to explain in the commit description why this field is moved.
> 
> >      * - __u8
> >        - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> >        - The list of L0 reference elements as indices in the DPB.
> >      * - __u8
> >        - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> >        - The list of L1 reference elements as indices in the DPB.
> > +    * - __u16
> > +      - ``short_term_ref_pic_set_size``
> > +
> 
> Not used.
> 
> >       -
> > +    * - __u16
> > +      - ``long_term_ref_pic_set_size``
> > +      -
> 
> Not used.
> 
> >      * - __u8
> > -      - ``num_rps_poc_st_curr_before``
> > -      - The number of reference pictures in the short-term set that come 
before
> > -        the current frame.
> 
> If this matches NumPocStCurrBefore from section 8.3.2 "Decoding process for 
reference picture set"
> then I would document that. And perhaps rename it to num_poc_st_curr_before.
> 
> > -    * - __u8
> > -      - ``num_rps_poc_st_curr_after``
> > -      - The number of reference pictures in the short-term set that come 
after
> > -        the current frame.
> 
> Ditto.
> 
> > -    * - __u8
> > -      - ``num_rps_poc_lt_curr``
> > -      - The number of reference pictures in the long-term set.
> 
> Ditto.
> 
> Also, I'd like the changes that move fields from 
V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS
> to the new V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS control, to be in their
> patch.
> 
> That will allow us to put in the commit description a proper
> explanation of why are fields being moved. Nothing fancy, simply
> explaining that these variables come from section 8.3.2
> "Decoding process for reference picture set", which describes
> a process invoked once per picture, so they are not per-slice.
> 
> > -    * - __u8
> > -      - ``padding[7]``
> > +      - ``padding``
> >        - Applications and drivers must set this to zero.
> >      * - struct :c:type:`v4l2_hevc_dpb_entry`
> >        - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > @@ -3646,3 +3671,74 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      so this has to come from client.
> >      This is applicable to H264 and valid Range is from 0 to 63.
> >      Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
> > +
> > +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> > +    Specifies various decode parameters, especially the references picture 
order
> > +    count (POC) for all the lists (short, long, before, current, after) 
and the
> > +    number of entries for each of them.
> > +    These parameters are defined according to :ref:`hevc`.
> > +    They are described in section 8.3 "Slice decoding process" of the
> > +    specification.
> > +
> > +.. c:type:: v4l2_ctrl_hevc_decode_params
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: struct v4l2_ctrl_hevc_decode_params
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths:       1 1 2
> > +
> > +    * - __s32
> > +      - ``pic_order_cnt_val``
> > +      -
> 
> Can be documented as:
> 
> """
> PicOrderCntVal as described in section 8.3.1 "Decoding process
> for picture order count" of the specification.
> """
> 
> Note that snake case is used to match the kernel style,
> but other than that we try to keep the HEVC spec variable
> names.
> 
> > +    * - __u8
> > +      - ``num_active_dpb_entries``
> > +      - The number of entries in ``dpb``.
> > +    * - struct :c:type:`v4l2_hevc_dpb_entry`
> > +      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      - The decoded picture buffer, for meta-data about reference frames.
> 
> The DPB is here, but it seems it's also in the slice control?
> 
> > +    * - __u8
> > +      - ``num_rps_poc_st_curr_before``
> > +      - The number of reference pictures in the short-term set that come 
before
> > +        the current frame.
> > +    * - __u8
> > +      - ``num_rps_poc_st_curr_after``
> > +      - The number of reference pictures in the short-term set that come 
after
> > +        the current frame.
> > +    * - __u8
> > +      - ``num_rps_poc_lt_curr``
> > +      - The number of reference pictures in the long-term set.
> > +    * - __u8
> > +      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      -
> > +    * - __u8
> > +      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      -
> > +    * - __u8
> > +      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      -
> 
> Could you document these as well?
> 
> Thanks a lot,
> Ezequiel
> 
> 



_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[Ezequiel Garcia]

Hey Jernej,

On Thu, 2021-02-25 at 18:01 +0100, Jernej Škrabec wrote:
> Hi Ezequiel,
> 
> Dne četrtek, 25. februar 2021 ob 14:09:52 CET je Ezequiel Garcia napisal(a):
> > Hi Benjamin,
> > 
> > Thanks for the good work.
> > 
> > On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> > > The H.265 ITU specification (section 7.4) define the general
> > > slice segment header semantics.
> > > Modified/added fields are:
> > > - video_parameter_set_id: (7.4.3.1) identifies the VPS for
> > > reference by other syntax elements.
> > > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of
> > > the vps_video_parameter_set_id of the active VPS.
> > > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
> > >  relative to the luma sampling
> > > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
> > > reference by other syntax elements
> > > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
> > > the inferred value of num_ref_idx_l0_active_minus1
> > > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
> > > the inferred value of num_ref_idx_l1_active_minus1
> > > - slice_segment_addr: (7.4.7.1) specifies the address of
> > > the first coding tree block in the slice segment
> > > - num_entry_point_offsets: (7.4.7.1) specifies the number of
> > > entry_point_offset_minus1[ i ] syntax elements in the slice header
> > > 
> > > Add HEVC decode params contains the information used in section
> > > "8.3 Slice decoding process" of the specification to let the hardware
> > > perform decoding of a slices.
> > > 
> > > Adapt Cedrus driver according to these changes.
> > > 
> > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > ---
> > > version 3:
> > > - Add documentation about the new structuers and fields.
> > > 
> > > version 2:
> > > - remove all change related to scaling
> > > - squash commits to a coherent split
> > > - be more verbose about the added fields
> > > 
> > >  .../media/v4l/ext-ctrls-codec.rst             | 126 +++++++++++++++---
> > >  .../media/v4l/vidioc-queryctrl.rst            |   6 +
> > >  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
> > >  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
> > >  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
> > >  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
> > >  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
> > >  include/media/hevc-ctrls.h                    |  45 +++++--
> > >  8 files changed, 186 insertions(+), 32 deletions(-)
> > > 
> > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/
> Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > index 00944e97d638..5e6d77e858c0 100644
> > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >      :stub-columns: 0
> > >      :widths:       1 1 2
> > >  
> > > +    * - __u8
> > > +      - ``video_parameter_set_id``
> > > +      - Identifies the VPS for reference by other syntax elements
> > > +    * - __u8
> > > +      - ``seq_parameter_set_id̀``
> > > +      - Specifies the value of the vps_video_parameter_set_id of the 
> active VPS
> > > +    * - __u8
> > > +      - ``chroma_format_idc``
> > > +      - Specifies the chroma sampling relative to the luma sampling
> > 
> > None of these fields seem needed for the Hantro G2 driver,
> > so I suggest you drop them for now.
> > 
> > >      * - __u16
> > >        - ``pic_width_in_luma_samples``
> > >        -
> > > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >      * - __u8
> > >        - ``chroma_format_idc``
> > >        -
> > > +    * - __u8
> > > +      - ``num_slices``
> > > +
> > 
> > Not used, but also doesn't seem part of the SPS syntax. If we have to
> > pass the number of slices, we'll need another mechanism.
> > 
> > >       -
> > >      * - __u64
> > >        - ``flags``
> > >        - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
> > > @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >      :stub-columns: 0
> > >      :widths:       1 1 2
> > >  
> > > +    * - __u8
> > > +      - ``pic_parameter_set_id``
> > > +      - Identifies the PPS for reference by other syntax elements
> > 
> > Not used.
> > 
> > >      * - __u8
> > >        - ``num_extra_slice_header_bits``
> > >        -
> > > +    * - __u8
> > > +      - ``num_ref_idx_l0_default_active_minus1``
> > > +      - Specifies the inferred value of num_ref_idx_l0_active_minus1
> > > +    * - __u8
> > > +      - ``num_ref_idx_l1_default_active_minus1``
> > > +      - Specifies the inferred value of num_ref_idx_l1_active_minus1
> > >      * - __s8
> > >        - ``init_qp_minus26``
> > >        -
> > > @@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >      * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
> > >        - 0x00040000
> > >        -
> > > +    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> > > +      - 0x00080000
> > > +      -
> > > +    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
> > > +      - 0x00100000
> > > +      -
> > >  
> > 
> > I suggest to do all the PPS control changes in a separate patch,
> > feels easier to review and cleaner as you can explain the
> > changes with more detail in the commit description.
> > 
> > Looking at the PPS syntax for tiles, I'm wondering if these
> > deserve their own control, which would be used if tiles are enabled,
> > i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set.
> > 
> >         __u8    num_tile_columns_minus1;                                         
> >         __u8    num_tile_rows_minus1;                                            
> >         __u8    column_width_minus1[20];                                         
> >         __u8    row_height_minus1[22];    
> > 
> > Not something we necessarily have to tackle now.
> > 
> > >  ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
> > >      Specifies various slice-specific parameters, especially from the NAL 
> unit
> > > @@ -3366,6 +3393,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >      * - __u32
> > >        - ``data_bit_offset``
> > >        - Offset (in bits) to the video data in the current slice data.
> > > +    * - __u32
> > > +      - ``slice_segment_addr``
> > > +      - Specifies the address of the first coding tree block in the slice 
> segment
> > 
> > Not used.
> > 
> > > +    * - __u32
> > > +      - ``num_entry_point_offsets``
> > > +      - Specifies the number of entry_point_offset_minus1[ i ] syntax 
> elements in the slice header
> > 
> > Not used.
> 
> While above two fields may not be used in Hantro, they are for sure useful for 
> Cedrus and RPi4. I would like to keep them, otherwise with such approach HEVC 
> will stay in staging for a long time. I'm still baffled why scaling matrix 
> control was dropped. It would fit well in Cedrus and RPi4 driver and after a 
> quick look, it seems that it was used in driver in later patch.
> 

I'd like to make sure each modification we are making to the uAPI
goes in the right direction, that is in the direction of moving
the API out of staging.

Since reviewing each field is quite hard, and opens some discussions,
I wanted to keep this patchset specific to what's needed for Hantro G2.

The Scaling matrix control is certainly a good one, as well as the ones
needed for Cedrus and RPi4. However, I feel it's better to discuss
them in their own "uAPI review" series so we can review all the changes
with an API hat.

This way we decouple the Hantro G2 discussion and work from the API work.

Also please feel free to submit RFC patches fo Cedrus and RPi4
(API and driver changes). We can certainly start the discussion around that,
with driver changes in context.

Hope I'm making sense here :)

Thanks,
Ezequiel

> Best regards,
> Jernej
> 
> > 
> > >      * - __u8
> > >        - ``nal_unit_type``
> > >        -
> > > @@ -3422,28 +3455,20 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >      * - __u8
> > >        - ``pic_struct``
> > >        -
> > > -    * - __u8
> > > -      - ``num_active_dpb_entries``
> > > -      - The number of entries in ``dpb``.
> > 
> > Need to explain in the commit description why this field is moved.
> > 
> > >      * - __u8
> > >        - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > >        - The list of L0 reference elements as indices in the DPB.
> > >      * - __u8
> > >        - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > >        - The list of L1 reference elements as indices in the DPB.
> > > +    * - __u16
> > > +      - ``short_term_ref_pic_set_size``
> > > +
> > 
> > Not used.
> > 
> > >       -
> > > +    * - __u16
> > > +      - ``long_term_ref_pic_set_size``
> > > +      -
> > 
> > Not used.
> > 
> > >      * - __u8
> > > -      - ``num_rps_poc_st_curr_before``
> > > -      - The number of reference pictures in the short-term set that come 
> before
> > > -        the current frame.
> > 
> > If this matches NumPocStCurrBefore from section 8.3.2 "Decoding process for 
> reference picture set"
> > then I would document that. And perhaps rename it to num_poc_st_curr_before.
> > 
> > > -    * - __u8
> > > -      - ``num_rps_poc_st_curr_after``
> > > -      - The number of reference pictures in the short-term set that come 
> after
> > > -        the current frame.
> > 
> > Ditto.
> > 
> > > -    * - __u8
> > > -      - ``num_rps_poc_lt_curr``
> > > -      - The number of reference pictures in the long-term set.
> > 
> > Ditto.
> > 
> > Also, I'd like the changes that move fields from 
> V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS
> > to the new V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS control, to be in their
> > patch.
> > 
> > That will allow us to put in the commit description a proper
> > explanation of why are fields being moved. Nothing fancy, simply
> > explaining that these variables come from section 8.3.2
> > "Decoding process for reference picture set", which describes
> > a process invoked once per picture, so they are not per-slice.
> > 
> > > -    * - __u8
> > > -      - ``padding[7]``
> > > +      - ``padding``
> > >        - Applications and drivers must set this to zero.
> > >      * - struct :c:type:`v4l2_hevc_dpb_entry`
> > >        - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > @@ -3646,3 +3671,74 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > >      so this has to come from client.
> > >      This is applicable to H264 and valid Range is from 0 to 63.
> > >      Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
> > > +
> > > +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> > > +    Specifies various decode parameters, especially the references picture 
> order
> > > +    count (POC) for all the lists (short, long, before, current, after) 
> and the
> > > +    number of entries for each of them.
> > > +    These parameters are defined according to :ref:`hevc`.
> > > +    They are described in section 8.3 "Slice decoding process" of the
> > > +    specification.
> > > +
> > > +.. c:type:: v4l2_ctrl_hevc_decode_params
> > > +
> > > +.. cssclass:: longtable
> > > +
> > > +.. flat-table:: struct v4l2_ctrl_hevc_decode_params
> > > +    :header-rows:  0
> > > +    :stub-columns: 0
> > > +    :widths:       1 1 2
> > > +
> > > +    * - __s32
> > > +      - ``pic_order_cnt_val``
> > > +      -
> > 
> > Can be documented as:
> > 
> > """
> > PicOrderCntVal as described in section 8.3.1 "Decoding process
> > for picture order count" of the specification.
> > """
> > 
> > Note that snake case is used to match the kernel style,
> > but other than that we try to keep the HEVC spec variable
> > names.
> > 
> > > +    * - __u8
> > > +      - ``num_active_dpb_entries``
> > > +      - The number of entries in ``dpb``.
> > > +    * - struct :c:type:`v4l2_hevc_dpb_entry`
> > > +      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > +      - The decoded picture buffer, for meta-data about reference frames.
> > 
> > The DPB is here, but it seems it's also in the slice control?
> > 
> > > +    * - __u8
> > > +      - ``num_rps_poc_st_curr_before``
> > > +      - The number of reference pictures in the short-term set that come 
> before
> > > +        the current frame.
> > > +    * - __u8
> > > +      - ``num_rps_poc_st_curr_after``
> > > +      - The number of reference pictures in the short-term set that come 
> after
> > > +        the current frame.
> > > +    * - __u8
> > > +      - ``num_rps_poc_lt_curr``
> > > +      - The number of reference pictures in the long-term set.
> > > +    * - __u8
> > > +      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > +      -
> > > +    * - __u8
> > > +      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > +      -
> > > +    * - __u8
> > > +      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > +      -
> > 
> > Could you document these as well?
> > 
> > Thanks a lot,
> > Ezequiel
> > 
> > 
> 
> 



_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 5/9] media: hantro: Introduce G2/HEVC decoder

[Ezequiel Garcia]

Hi Benjamin,

Thanks for the good work!
I mostly have two concerns with this implementation,
the tiled output and the allocation path.

More below.

On Mon, 2021-02-22 at 13:24 +0100, Benjamin Gaignard wrote:
> Implement all the logic to get G2 hardware decoding HEVC frames.
> It support up level 5.1 HEVC stream.
> It doesn't support yet 10 bits formats or scaling feature.
> 
> Add HANTRO HEVC dedicated control to skip some bits at the beginning
> of the slice header. That is very specific to this hardware so can't
> go into uapi structures. Compute the needed value is complex and require
> information from the stream that only the userland knows so let it
> provide the correct value to the driver.
> 
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> version 2:
> - squash multiple commits in this one.
> - fix the comments done by Ezequiel about dma_alloc_coherent usage
> - fix Dan's comments about control copy, reverse the test logic
> in tile_buffer_reallocate, rework some goto and return cases.
> 
>  drivers/staging/media/hantro/Makefile         |   2 +
>  drivers/staging/media/hantro/hantro.h         |  19 +
>  drivers/staging/media/hantro/hantro_drv.c     |  42 ++
>  .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++++++++++++++++++
>  drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++++++
>  drivers/staging/media/hantro/hantro_hevc.c    | 321 ++++++++++
>  drivers/staging/media/hantro/hantro_hw.h      |  47 ++
>  7 files changed, 1216 insertions(+)
>  create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>  create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
>  create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
> 
> 
[snip]
> +
> +static void set_buffers(struct hantro_ctx *ctx)
> +{
> +       struct vb2_v4l2_buffer *src_buf, *dst_buf;
> +       struct hantro_dev *vpu = ctx->dev;
> +       const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
> +       const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
> +       size_t cr_offset = hantro_hevc_chroma_offset(sps);
> +       dma_addr_t src_dma, dst_dma;
> +       u32 src_len, src_buf_len;
> +
> +       src_buf = hantro_get_src_buf(ctx);
> +       dst_buf = hantro_get_dst_buf(ctx);
> +
> +       /* Source (stream) buffer. */
> +       src_dma = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
> +       src_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
> +       src_buf_len = vb2_plane_size(&src_buf->vb2_buf, 0);
> +
> +       hantro_write_addr(vpu, HEVC_ADDR_STR, src_dma);
> +       hantro_reg_write(vpu, hevc_stream_len, src_len);
> +       hantro_reg_write(vpu, hevc_strm_buffer_len, src_buf_len);
> +       hantro_reg_write(vpu, hevc_strm_start_offset, 0);
> +       hantro_reg_write(vpu, hevc_write_mvs_e, 1);
> +
> +       /* Destination (decoded frame) buffer. */
> +       dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf);
> +
> +       hantro_write_addr(vpu, HEVC_RASTER_SCAN, dst_dma);
> +       hantro_write_addr(vpu, HEVC_RASTER_SCAN_CHR, dst_dma + cr_offset);

The way this is done the raster-scan output is the only
output, and the tiled ouput it kept entirely internal, in hantro_hevc_dec_hw_ctx.ref_bufs.

This means there's no way to expose tiled NV12 in i.MX8M VPU tiled format
to userspace, which could be desirable for some use cases.

I'm not suggesting to actually expose tiled NV12 in this patch, but to prepare 
the driver to be able to support that easily in the future.

It should be possible to consider this detiling as
post-processing, adding some code in hantro_postproc.c
leveraging the existing post-proc support.

IOW, hantro_postproc_ctx.dec_q would hold the tiled output,
hantro_postproc_enable() writes the raster-scan
buffer destination address, and so on.

> +       hantro_write_addr(vpu, HEVC_ADDR_TILE_SIZE, ctx->hevc_dec.tile_sizes.dma);
> +       hantro_write_addr(vpu, HEVC_TILE_FILTER, ctx->hevc_dec.tile_filter.dma);
> +       hantro_write_addr(vpu, HEVC_TILE_SAO, ctx->hevc_dec.tile_sao.dma);
> +       hantro_write_addr(vpu, HEVC_TILE_BSD, ctx->hevc_dec.tile_bsd.dma);
> +}
> +
> +void hantro_g2_check_idle(struct hantro_dev *vpu)
> +{
> +       int i;
> +
> +       for (i = 0; i < 3; i++) {
> +               u32 status;
> +
> +               /* Make sure the VPU is idle */
> +               status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
> +               if (status & HEVC_REG_INTERRUPT_DEC_E) {
> +                       pr_warn("%s: still enabled!!! resetting.\n", __func__);
> +                       status |= HEVC_REG_INTERRUPT_DEC_ABORT_E | HEVC_REG_INTERRUPT_DEC_IRQ_DIS;
> +                       vdpu_write(vpu, status, HEVC_REG_INTERRUPT);
> +               }
> +       }
> +}
> +
> +void hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
> +{
> +       struct hantro_dev *vpu = ctx->dev;
> +
> +       hantro_g2_check_idle(vpu);
> +
> +       /* Prepare HEVC decoder context. */
> +       if (hantro_hevc_dec_prepare_run(ctx))
> +               return;
> +
> +       /* Configure hardware registers. */
> +       set_params(ctx);
> +
> +       /* set reference pictures */
> +       if (set_ref(ctx))
> +               /* something goes wrong */
> +               hantro_irq_done(vpu, VB2_BUF_STATE_ERROR);
> +

I don't think we want to allow the _run() operation to fail like this.
In other words, I would avoid allocating buffers in the _run() path,
and doing all allocation at start() time.

That's why hantro_start_streaming() calls hantro_postproc_alloc() if needed.

> +       set_buffers(ctx);
> +       prepare_tile_info_buffer(ctx);
> +
> +       hantro_end_prepare_run(ctx);
> +
> +       hantro_reg_write(vpu, hevc_mode, HEVC_DEC_MODE);
> +       hantro_reg_write(vpu, hevc_clk_gate_e, 1);
> +
> +       /* Don't disable output */
> +       hantro_reg_write(vpu, hevc_out_dis, 0);
> +
> +       /* Don't compress buffers */
> +       hantro_reg_write(vpu, hevc_ref_compress_bypass, 1);
> +
> +       /* use NV12 as output format */
> +       hantro_reg_write(vpu, hevc_tile_e, 0);

Unless I'm missing something, this ^

> +       hantro_reg_write(vpu, hevc_out_rs_e, 1);

> +       hantro_reg_write(vpu, hevc_num_tile_rows, 1);
> +       hantro_reg_write(vpu, hevc_num_tile_cols, 1);
> +

and this ^ these shouldn't be here.

HEVC tiles are handled already. See prepare_tile_info_buffer().
Note that HEVC "tiles" are not related to NV12 "tiled" format.

Thanks!
Ezequiel


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: Re: Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[Jernej Škrabec]

Dne četrtek, 25. februar 2021 ob 18:34:48 CET je Ezequiel Garcia napisal(a):
> Hey Jernej,
> 
> On Thu, 2021-02-25 at 18:01 +0100, Jernej Škrabec wrote:
> > Hi Ezequiel,
> > 
> > Dne četrtek, 25. februar 2021 ob 14:09:52 CET je Ezequiel Garcia 
napisal(a):
> > > Hi Benjamin,
> > > 
> > > Thanks for the good work.
> > > 
> > > On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> > > > The H.265 ITU specification (section 7.4) define the general
> > > > slice segment header semantics.
> > > > Modified/added fields are:
> > > > - video_parameter_set_id: (7.4.3.1) identifies the VPS for
> > > > reference by other syntax elements.
> > > > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of
> > > > the vps_video_parameter_set_id of the active VPS.
> > > > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
> > > >  relative to the luma sampling
> > > > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
> > > > reference by other syntax elements
> > > > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
> > > > the inferred value of num_ref_idx_l0_active_minus1
> > > > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
> > > > the inferred value of num_ref_idx_l1_active_minus1
> > > > - slice_segment_addr: (7.4.7.1) specifies the address of
> > > > the first coding tree block in the slice segment
> > > > - num_entry_point_offsets: (7.4.7.1) specifies the number of
> > > > entry_point_offset_minus1[ i ] syntax elements in the slice header
> > > > 
> > > > Add HEVC decode params contains the information used in section
> > > > "8.3 Slice decoding process" of the specification to let the hardware
> > > > perform decoding of a slices.
> > > > 
> > > > Adapt Cedrus driver according to these changes.
> > > > 
> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > ---
> > > > version 3:
> > > > - Add documentation about the new structuers and fields.
> > > > 
> > > > version 2:
> > > > - remove all change related to scaling
> > > > - squash commits to a coherent split
> > > > - be more verbose about the added fields
> > > > 
> > > >  .../media/v4l/ext-ctrls-codec.rst             | 126 ++++++++++++++
+---
> > > >  .../media/v4l/vidioc-queryctrl.rst            |   6 +
> > > >  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
> > > >  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
> > > >  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
> > > >  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
> > > >  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
> > > >  include/media/hevc-ctrls.h                    |  45 +++++--
> > > >  8 files changed, 186 insertions(+), 32 deletions(-)
> > > > 
> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst 
b/
> > Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > index 00944e97d638..5e6d77e858c0 100644
> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > > >      :stub-columns: 0
> > > >      :widths:       1 1 2
> > > >  
> > > > +    * - __u8
> > > > +      - ``video_parameter_set_id``
> > > > +      - Identifies the VPS for reference by other syntax elements
> > > > +    * - __u8
> > > > +      - ``seq_parameter_set_id̀``
> > > > +      - Specifies the value of the vps_video_parameter_set_id of the 
> > active VPS
> > > > +    * - __u8
> > > > +      - ``chroma_format_idc``
> > > > +      - Specifies the chroma sampling relative to the luma sampling
> > > 
> > > None of these fields seem needed for the Hantro G2 driver,
> > > so I suggest you drop them for now.
> > > 
> > > >      * - __u16
> > > >        - ``pic_width_in_luma_samples``
> > > >        -
> > > > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > > >      * - __u8
> > > >        - ``chroma_format_idc``
> > > >        -
> > > > +    * - __u8
> > > > +      - ``num_slices``
> > > > +
> > > 
> > > Not used, but also doesn't seem part of the SPS syntax. If we have to
> > > pass the number of slices, we'll need another mechanism.
> > > 
> > > >       -
> > > >      * - __u64
> > > >        - ``flags``
> > > >        - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
> > > > @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > > >      :stub-columns: 0
> > > >      :widths:       1 1 2
> > > >  
> > > > +    * - __u8
> > > > +      - ``pic_parameter_set_id``
> > > > +      - Identifies the PPS for reference by other syntax elements
> > > 
> > > Not used.
> > > 
> > > >      * - __u8
> > > >        - ``num_extra_slice_header_bits``
> > > >        -
> > > > +    * - __u8
> > > > +      - ``num_ref_idx_l0_default_active_minus1``
> > > > +      - Specifies the inferred value of num_ref_idx_l0_active_minus1
> > > > +    * - __u8
> > > > +      - ``num_ref_idx_l1_default_active_minus1``
> > > > +      - Specifies the inferred value of num_ref_idx_l1_active_minus1
> > > >      * - __s8
> > > >        - ``init_qp_minus26``
> > > >        -
> > > > @@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > > >      * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
> > > >        - 0x00040000
> > > >        -
> > > > +    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> > > > +      - 0x00080000
> > > > +      -
> > > > +    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
> > > > +      - 0x00100000
> > > > +      -
> > > >  
> > > 
> > > I suggest to do all the PPS control changes in a separate patch,
> > > feels easier to review and cleaner as you can explain the
> > > changes with more detail in the commit description.
> > > 
> > > Looking at the PPS syntax for tiles, I'm wondering if these
> > > deserve their own control, which would be used if tiles are enabled,
> > > i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set.
> > > 
> > >         __u8    
num_tile_columns_minus1;                                         
> > >         __u8    
num_tile_rows_minus1;                                            
> > >         __u8    
column_width_minus1[20];                                         
> > >         __u8    row_height_minus1[22];    
> > > 
> > > Not something we necessarily have to tackle now.
> > > 
> > > >  ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
> > > >      Specifies various slice-specific parameters, especially from the 
NAL 
> > unit
> > > > @@ -3366,6 +3393,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > > >      * - __u32
> > > >        - ``data_bit_offset``
> > > >        - Offset (in bits) to the video data in the current slice data.
> > > > +    * - __u32
> > > > +      - ``slice_segment_addr``
> > > > +      - Specifies the address of the first coding tree block in the 
slice 
> > segment
> > > 
> > > Not used.
> > > 
> > > > +    * - __u32
> > > > +      - ``num_entry_point_offsets``
> > > > +      - Specifies the number of entry_point_offset_minus1[ i ] syntax 
> > elements in the slice header
> > > 
> > > Not used.
> > 
> > While above two fields may not be used in Hantro, they are for sure useful 
for 
> > Cedrus and RPi4. I would like to keep them, otherwise with such approach 
HEVC 
> > will stay in staging for a long time. I'm still baffled why scaling matrix 
> > control was dropped. It would fit well in Cedrus and RPi4 driver and after 
a 
> > quick look, it seems that it was used in driver in later patch.
> > 
> 
> I'd like to make sure each modification we are making to the uAPI
> goes in the right direction, that is in the direction of moving
> the API out of staging.
> 
> Since reviewing each field is quite hard, and opens some discussions,
> I wanted to keep this patchset specific to what's needed for Hantro G2.
> 
> The Scaling matrix control is certainly a good one, as well as the ones
> needed for Cedrus and RPi4. However, I feel it's better to discuss
> them in their own "uAPI review" series so we can review all the changes
> with an API hat.
> 
> This way we decouple the Hantro G2 discussion and work from the API work.
> 
> Also please feel free to submit RFC patches fo Cedrus and RPi4
> (API and driver changes). We can certainly start the discussion around that,
> with driver changes in context.

I don't know much about RPi4 driver, only few implementation details, so 
you'll have to ping developer who wrote it. Regarding HEVC on Cedrus - it has 
one pain point - it needs entry point table which in turn needs support for 
variable arrays in order to be feasable AFAIK. I don't plan to develop that. 
Patches for scaling matrix and segment address were sent a bit more than a 
year ago but were turned down because they change control structures (among 
other things). Sorry to say, but I work on other things now, so Cedrus will 
have to wait. Alternatively, someone can take my patches from LibreELEC, 
update and submit them. They are in use for a long time.

Best regards,
Jernej

> 
> Hope I'm making sense here :)
> 
> Thanks,
> Ezequiel
> 
> > Best regards,
> > Jernej
> > 
> > > 
> > > >      * - __u8
> > > >        - ``nal_unit_type``
> > > >        -
> > > > @@ -3422,28 +3455,20 @@ enum v4l2_mpeg_video_hevc_size_of_length_field 
-
> > > >      * - __u8
> > > >        - ``pic_struct``
> > > >        -
> > > > -    * - __u8
> > > > -      - ``num_active_dpb_entries``
> > > > -      - The number of entries in ``dpb``.
> > > 
> > > Need to explain in the commit description why this field is moved.
> > > 
> > > >      * - __u8
> > > >        - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > >        - The list of L0 reference elements as indices in the DPB.
> > > >      * - __u8
> > > >        - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > >        - The list of L1 reference elements as indices in the DPB.
> > > > +    * - __u16
> > > > +      - ``short_term_ref_pic_set_size``
> > > > +
> > > 
> > > Not used.
> > > 
> > > >       -
> > > > +    * - __u16
> > > > +      - ``long_term_ref_pic_set_size``
> > > > +      -
> > > 
> > > Not used.
> > > 
> > > >      * - __u8
> > > > -      - ``num_rps_poc_st_curr_before``
> > > > -      - The number of reference pictures in the short-term set that 
come 
> > before
> > > > -        the current frame.
> > > 
> > > If this matches NumPocStCurrBefore from section 8.3.2 "Decoding process 
for 
> > reference picture set"
> > > then I would document that. And perhaps rename it to 
num_poc_st_curr_before.
> > > 
> > > > -    * - __u8
> > > > -      - ``num_rps_poc_st_curr_after``
> > > > -      - The number of reference pictures in the short-term set that 
come 
> > after
> > > > -        the current frame.
> > > 
> > > Ditto.
> > > 
> > > > -    * - __u8
> > > > -      - ``num_rps_poc_lt_curr``
> > > > -      - The number of reference pictures in the long-term set.
> > > 
> > > Ditto.
> > > 
> > > Also, I'd like the changes that move fields from 
> > V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS
> > > to the new V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS control, to be in 
their
> > > patch.
> > > 
> > > That will allow us to put in the commit description a proper
> > > explanation of why are fields being moved. Nothing fancy, simply
> > > explaining that these variables come from section 8.3.2
> > > "Decoding process for reference picture set", which describes
> > > a process invoked once per picture, so they are not per-slice.
> > > 
> > > > -    * - __u8
> > > > -      - ``padding[7]``
> > > > +      - ``padding``
> > > >        - Applications and drivers must set this to zero.
> > > >      * - struct :c:type:`v4l2_hevc_dpb_entry`
> > > >        - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > @@ -3646,3 +3671,74 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > > >      so this has to come from client.
> > > >      This is applicable to H264 and valid Range is from 0 to 63.
> > > >      Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
> > > > +
> > > > +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> > > > +    Specifies various decode parameters, especially the references 
picture 
> > order
> > > > +    count (POC) for all the lists (short, long, before, current, 
after) 
> > and the
> > > > +    number of entries for each of them.
> > > > +    These parameters are defined according to :ref:`hevc`.
> > > > +    They are described in section 8.3 "Slice decoding process" of the
> > > > +    specification.
> > > > +
> > > > +.. c:type:: v4l2_ctrl_hevc_decode_params
> > > > +
> > > > +.. cssclass:: longtable
> > > > +
> > > > +.. flat-table:: struct v4l2_ctrl_hevc_decode_params
> > > > +    :header-rows:  0
> > > > +    :stub-columns: 0
> > > > +    :widths:       1 1 2
> > > > +
> > > > +    * - __s32
> > > > +      - ``pic_order_cnt_val``
> > > > +      -
> > > 
> > > Can be documented as:
> > > 
> > > """
> > > PicOrderCntVal as described in section 8.3.1 "Decoding process
> > > for picture order count" of the specification.
> > > """
> > > 
> > > Note that snake case is used to match the kernel style,
> > > but other than that we try to keep the HEVC spec variable
> > > names.
> > > 
> > > > +    * - __u8
> > > > +      - ``num_active_dpb_entries``
> > > > +      - The number of entries in ``dpb``.
> > > > +    * - struct :c:type:`v4l2_hevc_dpb_entry`
> > > > +      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > +      - The decoded picture buffer, for meta-data about reference 
frames.
> > > 
> > > The DPB is here, but it seems it's also in the slice control?
> > > 
> > > > +    * - __u8
> > > > +      - ``num_rps_poc_st_curr_before``
> > > > +      - The number of reference pictures in the short-term set that 
come 
> > before
> > > > +        the current frame.
> > > > +    * - __u8
> > > > +      - ``num_rps_poc_st_curr_after``
> > > > +      - The number of reference pictures in the short-term set that 
come 
> > after
> > > > +        the current frame.
> > > > +    * - __u8
> > > > +      - ``num_rps_poc_lt_curr``
> > > > +      - The number of reference pictures in the long-term set.
> > > > +    * - __u8
> > > > +      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > +      -
> > > > +    * - __u8
> > > > +      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > +      -
> > > > +    * - __u8
> > > > +      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > +      -
> > > 
> > > Could you document these as well?
> > > 
> > > Thanks a lot,
> > > Ezequiel
> > > 
> > > 
> > 
> > 
> 
> 
> 



_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[John Cox]

On Thu, 25 Feb 2021 19:05:55 +0100, you wrote:

>Dne ?etrtek, 25. februar 2021 ob 18:34:48 CET je Ezequiel Garcia napisal(a):
>> Hey Jernej,
>> 
>> On Thu, 2021-02-25 at 18:01 +0100, Jernej Škrabec wrote:
>> > Hi Ezequiel,
>> > 
>> > Dne ?etrtek, 25. februar 2021 ob 14:09:52 CET je Ezequiel Garcia 
>napisal(a):
>> > > Hi Benjamin,
>> > > 
>> > > Thanks for the good work.
>> > > 
>> > > On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
>> > > > The H.265 ITU specification (section 7.4) define the general
>> > > > slice segment header semantics.
>> > > > Modified/added fields are:
>> > > > - video_parameter_set_id: (7.4.3.1) identifies the VPS for
>> > > > reference by other syntax elements.
>> > > > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of
>> > > > the vps_video_parameter_set_id of the active VPS.
>> > > > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
>> > > >  relative to the luma sampling
>> > > > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
>> > > > reference by other syntax elements
>> > > > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
>> > > > the inferred value of num_ref_idx_l0_active_minus1
>> > > > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
>> > > > the inferred value of num_ref_idx_l1_active_minus1
>> > > > - slice_segment_addr: (7.4.7.1) specifies the address of
>> > > > the first coding tree block in the slice segment
>> > > > - num_entry_point_offsets: (7.4.7.1) specifies the number of
>> > > > entry_point_offset_minus1[ i ] syntax elements in the slice header
>> > > > 
>> > > > Add HEVC decode params contains the information used in section
>> > > > "8.3 Slice decoding process" of the specification to let the hardware
>> > > > perform decoding of a slices.
>> > > > 
>> > > > Adapt Cedrus driver according to these changes.
>> > > > 
>> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> > > > ---
>> > > > version 3:
>> > > > - Add documentation about the new structuers and fields.
>> > > > 
>> > > > version 2:
>> > > > - remove all change related to scaling
>> > > > - squash commits to a coherent split
>> > > > - be more verbose about the added fields
>> > > > 
>> > > >  .../media/v4l/ext-ctrls-codec.rst             | 126 ++++++++++++++
>+---
>> > > >  .../media/v4l/vidioc-queryctrl.rst            |   6 +
>> > > >  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
>> > > >  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
>> > > >  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
>> > > >  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
>> > > >  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
>> > > >  include/media/hevc-ctrls.h                    |  45 +++++--
>> > > >  8 files changed, 186 insertions(+), 32 deletions(-)
>> > > > 
>> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst 
>b/
>> > Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > > index 00944e97d638..5e6d77e858c0 100644
>> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > > @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > > >      :stub-columns: 0
>> > > >      :widths:       1 1 2
>> > > >  
>> > > > +    * - __u8
>> > > > +      - ``video_parameter_set_id``
>> > > > +      - Identifies the VPS for reference by other syntax elements
>> > > > +    * - __u8
>> > > > +      - ``seq_parameter_set_id?``
>> > > > +      - Specifies the value of the vps_video_parameter_set_id of the 
>> > active VPS
>> > > > +    * - __u8
>> > > > +      - ``chroma_format_idc``
>> > > > +      - Specifies the chroma sampling relative to the luma sampling
>> > > 
>> > > None of these fields seem needed for the Hantro G2 driver,
>> > > so I suggest you drop them for now.
>> > > 
>> > > >      * - __u16
>> > > >        - ``pic_width_in_luma_samples``
>> > > >        -
>> > > > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > > >      * - __u8
>> > > >        - ``chroma_format_idc``
>> > > >        -
>> > > > +    * - __u8
>> > > > +      - ``num_slices``
>> > > > +
>> > > 
>> > > Not used, but also doesn't seem part of the SPS syntax. If we have to
>> > > pass the number of slices, we'll need another mechanism.
>> > > 
>> > > >       -
>> > > >      * - __u64
>> > > >        - ``flags``
>> > > >        - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
>> > > > @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > > >      :stub-columns: 0
>> > > >      :widths:       1 1 2
>> > > >  
>> > > > +    * - __u8
>> > > > +      - ``pic_parameter_set_id``
>> > > > +      - Identifies the PPS for reference by other syntax elements
>> > > 
>> > > Not used.
>> > > 
>> > > >      * - __u8
>> > > >        - ``num_extra_slice_header_bits``
>> > > >        -
>> > > > +    * - __u8
>> > > > +      - ``num_ref_idx_l0_default_active_minus1``
>> > > > +      - Specifies the inferred value of num_ref_idx_l0_active_minus1
>> > > > +    * - __u8
>> > > > +      - ``num_ref_idx_l1_default_active_minus1``
>> > > > +      - Specifies the inferred value of num_ref_idx_l1_active_minus1
>> > > >      * - __s8
>> > > >        - ``init_qp_minus26``
>> > > >        -
>> > > > @@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > > >      * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
>> > > >        - 0x00040000
>> > > >        -
>> > > > +    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
>> > > > +      - 0x00080000
>> > > > +      -
>> > > > +    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
>> > > > +      - 0x00100000
>> > > > +      -
>> > > >  
>> > > 
>> > > I suggest to do all the PPS control changes in a separate patch,
>> > > feels easier to review and cleaner as you can explain the
>> > > changes with more detail in the commit description.
>> > > 
>> > > Looking at the PPS syntax for tiles, I'm wondering if these
>> > > deserve their own control, which would be used if tiles are enabled,
>> > > i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set.
>> > > 
>> > >         __u8    
>num_tile_columns_minus1;                                         
>> > >         __u8    
>num_tile_rows_minus1;                                            
>> > >         __u8    
>column_width_minus1[20];                                         
>> > >         __u8    row_height_minus1[22];    
>> > > 
>> > > Not something we necessarily have to tackle now.
>> > > 
>> > > >  ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
>> > > >      Specifies various slice-specific parameters, especially from the 
>NAL 
>> > unit
>> > > > @@ -3366,6 +3393,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > > >      * - __u32
>> > > >        - ``data_bit_offset``
>> > > >        - Offset (in bits) to the video data in the current slice data.
>> > > > +    * - __u32
>> > > > +      - ``slice_segment_addr``
>> > > > +      - Specifies the address of the first coding tree block in the 
>slice 
>> > segment
>> > > 
>> > > Not used.
>> > > 
>> > > > +    * - __u32
>> > > > +      - ``num_entry_point_offsets``
>> > > > +      - Specifies the number of entry_point_offset_minus1[ i ] syntax 
>> > elements in the slice header
>> > > 
>> > > Not used.
>> > 
>> > While above two fields may not be used in Hantro, they are for sure useful 
>for 
>> > Cedrus and RPi4. I would like to keep them, otherwise with such approach 
>HEVC 
>> > will stay in staging for a long time. I'm still baffled why scaling matrix 
>> > control was dropped. It would fit well in Cedrus and RPi4 driver and after 
>a 
>> > quick look, it seems that it was used in driver in later patch.
>> > 
>> 
>> I'd like to make sure each modification we are making to the uAPI
>> goes in the right direction, that is in the direction of moving
>> the API out of staging.
>> 
>> Since reviewing each field is quite hard, and opens some discussions,
>> I wanted to keep this patchset specific to what's needed for Hantro G2.
>> 
>> The Scaling matrix control is certainly a good one, as well as the ones
>> needed for Cedrus and RPi4. However, I feel it's better to discuss
>> them in their own "uAPI review" series so we can review all the changes
>> with an API hat.
>> 
>> This way we decouple the Hantro G2 discussion and work from the API work.
>> 
>> Also please feel free to submit RFC patches fo Cedrus and RPi4
>> (API and driver changes). We can certainly start the discussion around that,
>> with driver changes in context.
>
>I don't know much about RPi4 driver, only few implementation details, so 
>you'll have to ping developer who wrote it. Regarding HEVC on Cedrus - it has 
>one pain point - it needs entry point table which in turn needs support for 
>variable arrays in order to be feasable AFAIK. I don't plan to develop that. 
>Patches for scaling matrix and segment address were sent a bit more than a 
>year ago but were turned down because they change control structures (among 
>other things). Sorry to say, but I work on other things now, so Cedrus will 
>have to wait. Alternatively, someone can take my patches from LibreELEC, 
>update and submit them. They are in use for a long time.

It seems to me that H.265 should be the source for what fields are
needed in the uapi - not whatever happens to be supported by current
h/w. The standard defines the list of fields that can occur in the
parameter sets and headers, and everything that is needed to decode 
any slice_seqgment_data should be in the upai.  Eventually all those
fields are going to be needed (they aren't in the standard just to look
pretty) and given the reluctance I've observed to change the uapi once
released they should be there from the start.

Now some hardware requires more fields that aren't in the standard -
those can (and should) be added in h/w specific extensions.

Regards

JC

>Best regards,
>Jernej
>
>> 
>> Hope I'm making sense here :)
>> 
>> Thanks,
>> Ezequiel
>> 
>> > Best regards,
>> > Jernej
>> > 
>> > > 
>> > > >      * - __u8
>> > > >        - ``nal_unit_type``
>> > > >        -
>> > > > @@ -3422,28 +3455,20 @@ enum v4l2_mpeg_video_hevc_size_of_length_field 
>-
>> > > >      * - __u8
>> > > >        - ``pic_struct``
>> > > >        -
>> > > > -    * - __u8
>> > > > -      - ``num_active_dpb_entries``
>> > > > -      - The number of entries in ``dpb``.
>> > > 
>> > > Need to explain in the commit description why this field is moved.
>> > > 
>> > > >      * - __u8
>> > > >        - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> > > >        - The list of L0 reference elements as indices in the DPB.
>> > > >      * - __u8
>> > > >        - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> > > >        - The list of L1 reference elements as indices in the DPB.
>> > > > +    * - __u16
>> > > > +      - ``short_term_ref_pic_set_size``
>> > > > +
>> > > 
>> > > Not used.
>> > > 
>> > > >       -
>> > > > +    * - __u16
>> > > > +      - ``long_term_ref_pic_set_size``
>> > > > +      -
>> > > 
>> > > Not used.
>> > > 
>> > > >      * - __u8
>> > > > -      - ``num_rps_poc_st_curr_before``
>> > > > -      - The number of reference pictures in the short-term set that 
>come 
>> > before
>> > > > -        the current frame.
>> > > 
>> > > If this matches NumPocStCurrBefore from section 8.3.2 "Decoding process 
>for 
>> > reference picture set"
>> > > then I would document that. And perhaps rename it to 
>num_poc_st_curr_before.
>> > > 
>> > > > -    * - __u8
>> > > > -      - ``num_rps_poc_st_curr_after``
>> > > > -      - The number of reference pictures in the short-term set that 
>come 
>> > after
>> > > > -        the current frame.
>> > > 
>> > > Ditto.
>> > > 
>> > > > -    * - __u8
>> > > > -      - ``num_rps_poc_lt_curr``
>> > > > -      - The number of reference pictures in the long-term set.
>> > > 
>> > > Ditto.
>> > > 
>> > > Also, I'd like the changes that move fields from 
>> > V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS
>> > > to the new V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS control, to be in 
>their
>> > > patch.
>> > > 
>> > > That will allow us to put in the commit description a proper
>> > > explanation of why are fields being moved. Nothing fancy, simply
>> > > explaining that these variables come from section 8.3.2
>> > > "Decoding process for reference picture set", which describes
>> > > a process invoked once per picture, so they are not per-slice.
>> > > 
>> > > > -    * - __u8
>> > > > -      - ``padding[7]``
>> > > > +      - ``padding``
>> > > >        - Applications and drivers must set this to zero.
>> > > >      * - struct :c:type:`v4l2_hevc_dpb_entry`
>> > > >        - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> > > > @@ -3646,3 +3671,74 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > > >      so this has to come from client.
>> > > >      This is applicable to H264 and valid Range is from 0 to 63.
>> > > >      Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
>> > > > +
>> > > > +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
>> > > > +    Specifies various decode parameters, especially the references 
>picture 
>> > order
>> > > > +    count (POC) for all the lists (short, long, before, current, 
>after) 
>> > and the
>> > > > +    number of entries for each of them.
>> > > > +    These parameters are defined according to :ref:`hevc`.
>> > > > +    They are described in section 8.3 "Slice decoding process" of the
>> > > > +    specification.
>> > > > +
>> > > > +.. c:type:: v4l2_ctrl_hevc_decode_params
>> > > > +
>> > > > +.. cssclass:: longtable
>> > > > +
>> > > > +.. flat-table:: struct v4l2_ctrl_hevc_decode_params
>> > > > +    :header-rows:  0
>> > > > +    :stub-columns: 0
>> > > > +    :widths:       1 1 2
>> > > > +
>> > > > +    * - __s32
>> > > > +      - ``pic_order_cnt_val``
>> > > > +      -
>> > > 
>> > > Can be documented as:
>> > > 
>> > > """
>> > > PicOrderCntVal as described in section 8.3.1 "Decoding process
>> > > for picture order count" of the specification.
>> > > """
>> > > 
>> > > Note that snake case is used to match the kernel style,
>> > > but other than that we try to keep the HEVC spec variable
>> > > names.
>> > > 
>> > > > +    * - __u8
>> > > > +      - ``num_active_dpb_entries``
>> > > > +      - The number of entries in ``dpb``.
>> > > > +    * - struct :c:type:`v4l2_hevc_dpb_entry`
>> > > > +      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> > > > +      - The decoded picture buffer, for meta-data about reference 
>frames.
>> > > 
>> > > The DPB is here, but it seems it's also in the slice control?
>> > > 
>> > > > +    * - __u8
>> > > > +      - ``num_rps_poc_st_curr_before``
>> > > > +      - The number of reference pictures in the short-term set that 
>come 
>> > before
>> > > > +        the current frame.
>> > > > +    * - __u8
>> > > > +      - ``num_rps_poc_st_curr_after``
>> > > > +      - The number of reference pictures in the short-term set that 
>come 
>> > after
>> > > > +        the current frame.
>> > > > +    * - __u8
>> > > > +      - ``num_rps_poc_lt_curr``
>> > > > +      - The number of reference pictures in the long-term set.
>> > > > +    * - __u8
>> > > > +      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> > > > +      -
>> > > > +    * - __u8
>> > > > +      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> > > > +      -
>> > > > +    * - __u8
>> > > > +      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> > > > +      -
>> > > 
>> > > Could you document these as well?
>> > > 
>> > > Thanks a lot,
>> > > Ezequiel
>> > > 
>> > > 
>> > 
>> > 
>> 
>> 
>> 
>

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[Nicolas Dufresne]

Le jeudi 25 février 2021 à 10:09 -0300, Ezequiel Garcia a écrit :
> Hi Benjamin,
> 
> Thanks for the good work.
> 
> On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> > The H.265 ITU specification (section 7.4) define the general
> > slice segment header semantics.
> > Modified/added fields are:
> > - video_parameter_set_id: (7.4.3.1) identifies the VPS for
> > reference by other syntax elements.
> > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of
> > the vps_video_parameter_set_id of the active VPS.
> > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
> >  relative to the luma sampling
> > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
> > reference by other syntax elements
> > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
> > the inferred value of num_ref_idx_l0_active_minus1
> > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
> > the inferred value of num_ref_idx_l1_active_minus1
> > - slice_segment_addr: (7.4.7.1) specifies the address of
> > the first coding tree block in the slice segment
> > - num_entry_point_offsets: (7.4.7.1) specifies the number of
> > entry_point_offset_minus1[ i ] syntax elements in the slice header
> > 
> > Add HEVC decode params contains the information used in section
> > "8.3 Slice decoding process" of the specification to let the hardware
> > perform decoding of a slices.
> > 
> > Adapt Cedrus driver according to these changes.
> > 
> > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > ---
> > version 3:
> > - Add documentation about the new structuers and fields.
> > 
> > version 2:
> > - remove all change related to scaling
> > - squash commits to a coherent split
> > - be more verbose about the added fields
> > 
> >  .../media/v4l/ext-ctrls-codec.rst             | 126 +++++++++++++++---
> >  .../media/v4l/vidioc-queryctrl.rst            |   6 +
> >  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
> >  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
> >  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
> >  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
> >  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
> >  include/media/hevc-ctrls.h                    |  45 +++++--
> >  8 files changed, 186 insertions(+), 32 deletions(-)
> > 
> > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > index 00944e97d638..5e6d77e858c0 100644
> > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > @@ -3109,6 +3109,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      :stub-columns: 0
> >      :widths:       1 1 2
> >  
> > +    * - __u8
> > +      - ``video_parameter_set_id``
> > +      - Identifies the VPS for reference by other syntax elements
> > +    * - __u8
> > +      - ``seq_parameter_set_id̀``
> > +      - Specifies the value of the vps_video_parameter_set_id of the active
> > VPS
> > +    * - __u8
> > +      - ``chroma_format_idc``
> > +      - Specifies the chroma sampling relative to the luma sampling
> 
> None of these fields seem needed for the Hantro G2 driver,
> so I suggest you drop them for now.

You should be using chroma_format_idc in your validation. I think Hantro G2 does
not do YUV 4:4:4, which is what chroma_format_idc tells the driver. Our
implementation might also be missing 4:0:0 (black and white).

As this value will aftect the buffer size, please keep them, and ideally
validate them. Note that it feels rather problematic / unsafe situation as we
endup having to trust userspace, hopefully the HW validates it's buffer size ?
(do we tell the HW the buffer sizes ?)

> 
> >      * - __u16
> >        - ``pic_width_in_luma_samples``
> >        -
> > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - __u8
> >        - ``chroma_format_idc``
> >        -
> > +    * - __u8
> > +      - ``num_slices``
> > +
> 
> Not used, but also doesn't seem part of the SPS syntax. If we have to
> pass the number of slices, we'll need another mechanism.
> 
> >       -
> >      * - __u64
> >        - ``flags``
> >        - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
> > @@ -3231,9 +3243,18 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      :stub-columns: 0
> >      :widths:       1 1 2
> >  
> > +    * - __u8
> > +      - ``pic_parameter_set_id``
> > +      - Identifies the PPS for reference by other syntax elements
> 
> Not used.
> 
> >      * - __u8
> >        - ``num_extra_slice_header_bits``
> >        -
> > +    * - __u8
> > +      - ``num_ref_idx_l0_default_active_minus1``
> > +      - Specifies the inferred value of num_ref_idx_l0_active_minus1
> > +    * - __u8
> > +      - ``num_ref_idx_l1_default_active_minus1``
> > +      - Specifies the inferred value of num_ref_idx_l1_active_minus1
> >      * - __s8
> >        - ``init_qp_minus26``
> >        -
> > @@ -3342,6 +3363,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
> >        - 0x00040000
> >        -
> > +    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> > +      - 0x00080000
> > +      -
> > +    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
> > +      - 0x00100000
> > +      -
> >  
> 
> I suggest to do all the PPS control changes in a separate patch,
> feels easier to review and cleaner as you can explain the
> changes with more detail in the commit description.
> 
> Looking at the PPS syntax for tiles, I'm wondering if these
> deserve their own control, which would be used if tiles are enabled,
> i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set.
> 
>         __u8   
> num_tile_columns_minus1;                                         
>         __u8   
> num_tile_rows_minus1;                                            
>         __u8   
> column_width_minus1[20];                                         
>         __u8    row_height_minus1[22];    
> 
> Not something we necessarily have to tackle now.
> 
> >  ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
> >      Specifies various slice-specific parameters, especially from the NAL
> > unit
> > @@ -3366,6 +3393,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - __u32
> >        - ``data_bit_offset``
> >        - Offset (in bits) to the video data in the current slice data.
> > +    * - __u32
> > +      - ``slice_segment_addr``
> > +      - Specifies the address of the first coding tree block in the slice
> > segment
> 
> Not used.
> 
> > +    * - __u32
> > +      - ``num_entry_point_offsets``
> > +      - Specifies the number of entry_point_offset_minus1[ i ] syntax
> > elements in the slice header
> 
> Not used.
> 
> >      * - __u8
> >        - ``nal_unit_type``
> >        -
> > @@ -3422,28 +3455,20 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      * - __u8
> >        - ``pic_struct``
> >        -
> > -    * - __u8
> > -      - ``num_active_dpb_entries``
> > -      - The number of entries in ``dpb``.
> 
> Need to explain in the commit description why this field is moved.
> 
> >      * - __u8
> >        - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> >        - The list of L0 reference elements as indices in the DPB.
> >      * - __u8
> >        - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> >        - The list of L1 reference elements as indices in the DPB.
> > +    * - __u16
> > +      - ``short_term_ref_pic_set_size``
> > +
> 
> Not used.
> 
> >       -
> > +    * - __u16
> > +      - ``long_term_ref_pic_set_size``
> > +      -
> 
> Not used.
> 
> >      * - __u8
> > -      - ``num_rps_poc_st_curr_before``
> > -      - The number of reference pictures in the short-term set that come
> > before
> > -        the current frame.
> 
> If this matches NumPocStCurrBefore from section 8.3.2 "Decoding process for
> reference picture set"
> then I would document that. And perhaps rename it to num_poc_st_curr_before.
> 
> > -    * - __u8
> > -      - ``num_rps_poc_st_curr_after``
> > -      - The number of reference pictures in the short-term set that come
> > after
> > -        the current frame.
> 
> Ditto.
> 
> > -    * - __u8
> > -      - ``num_rps_poc_lt_curr``
> > -      - The number of reference pictures in the long-term set.
> 
> Ditto.
> 
> Also, I'd like the changes that move fields from
> V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS
> to the new V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS control, to be in their
> patch.
> 
> That will allow us to put in the commit description a proper
> explanation of why are fields being moved. Nothing fancy, simply
> explaining that these variables come from section 8.3.2
> "Decoding process for reference picture set", which describes
> a process invoked once per picture, so they are not per-slice.
> 
> > -    * - __u8
> > -      - ``padding[7]``
> > +      - ``padding``
> >        - Applications and drivers must set this to zero.
> >      * - struct :c:type:`v4l2_hevc_dpb_entry`
> >        - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > @@ -3646,3 +3671,74 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> >      so this has to come from client.
> >      This is applicable to H264 and valid Range is from 0 to 63.
> >      Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
> > +
> > +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> > +    Specifies various decode parameters, especially the references picture
> > order
> > +    count (POC) for all the lists (short, long, before, current, after) and
> > the
> > +    number of entries for each of them.
> > +    These parameters are defined according to :ref:`hevc`.
> > +    They are described in section 8.3 "Slice decoding process" of the
> > +    specification.
> > +
> > +.. c:type:: v4l2_ctrl_hevc_decode_params
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: struct v4l2_ctrl_hevc_decode_params
> > +    :header-rows:  0
> > +    :stub-columns: 0
> > +    :widths:       1 1 2
> > +
> > +    * - __s32
> > +      - ``pic_order_cnt_val``
> > +      -
> 
> Can be documented as:
> 
> """
> PicOrderCntVal as described in section 8.3.1 "Decoding process
> for picture order count" of the specification.
> """
> 
> Note that snake case is used to match the kernel style,
> but other than that we try to keep the HEVC spec variable
> names.
> 
> > +    * - __u8
> > +      - ``num_active_dpb_entries``
> > +      - The number of entries in ``dpb``.
> > +    * - struct :c:type:`v4l2_hevc_dpb_entry`
> > +      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      - The decoded picture buffer, for meta-data about reference frames.
> 
> The DPB is here, but it seems it's also in the slice control?
> 
> > +    * - __u8
> > +      - ``num_rps_poc_st_curr_before``
> > +      - The number of reference pictures in the short-term set that come
> > before
> > +        the current frame.
> > +    * - __u8
> > +      - ``num_rps_poc_st_curr_after``
> > +      - The number of reference pictures in the short-term set that come
> > after
> > +        the current frame.
> > +    * - __u8
> > +      - ``num_rps_poc_lt_curr``
> > +      - The number of reference pictures in the long-term set.
> > +    * - __u8
> > +      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      -
> > +    * - __u8
> > +      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      -
> > +    * - __u8
> > +      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > +      -
> 
> Could you document these as well?
> 
> Thanks a lot,
> Ezequiel
> 
> 



_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 1/9] media: hevc: Modify structures to follow H265 ITU spec

[Nicolas Dufresne]

Le jeudi 25 février 2021 à 18:34 +0000, John Cox a écrit :
> On Thu, 25 Feb 2021 19:05:55 +0100, you wrote:
> 
> > Dne ?etrtek, 25. februar 2021 ob 18:34:48 CET je Ezequiel Garcia napisal(a):
> > > Hey Jernej,
> > > 
> > > On Thu, 2021-02-25 at 18:01 +0100, Jernej Škrabec wrote:
> > > > Hi Ezequiel,
> > > > 
> > > > Dne ?etrtek, 25. februar 2021 ob 14:09:52 CET je Ezequiel Garcia 
> > napisal(a):
> > > > > Hi Benjamin,
> > > > > 
> > > > > Thanks for the good work.
> > > > > 
> > > > > On Mon, 2021-02-22 at 13:23 +0100, Benjamin Gaignard wrote:
> > > > > > The H.265 ITU specification (section 7.4) define the general
> > > > > > slice segment header semantics.
> > > > > > Modified/added fields are:
> > > > > > - video_parameter_set_id: (7.4.3.1) identifies the VPS for
> > > > > > reference by other syntax elements.
> > > > > > - seq_parameter_set_id: (7.4.3.2.1) specifies the value of
> > > > > > the vps_video_parameter_set_id of the active VPS.
> > > > > > - chroma_format_idc: (7.4.3.2.1) specifies the chroma sampling
> > > > > >  relative to the luma sampling
> > > > > > - pic_parameter_set_id: (7.4.3.3.1) identifies the PPS for
> > > > > > reference by other syntax elements
> > > > > > - num_ref_idx_l0_default_active_minus1: (7.4.3.3.1) specifies
> > > > > > the inferred value of num_ref_idx_l0_active_minus1
> > > > > > - num_ref_idx_l1_default_active_minus1: (7.4.3.3.1) specifies
> > > > > > the inferred value of num_ref_idx_l1_active_minus1
> > > > > > - slice_segment_addr: (7.4.7.1) specifies the address of
> > > > > > the first coding tree block in the slice segment
> > > > > > - num_entry_point_offsets: (7.4.7.1) specifies the number of
> > > > > > entry_point_offset_minus1[ i ] syntax elements in the slice header
> > > > > > 
> > > > > > Add HEVC decode params contains the information used in section
> > > > > > "8.3 Slice decoding process" of the specification to let the
> > > > > > hardware
> > > > > > perform decoding of a slices.
> > > > > > 
> > > > > > Adapt Cedrus driver according to these changes.
> > > > > > 
> > > > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > > > ---
> > > > > > version 3:
> > > > > > - Add documentation about the new structuers and fields.
> > > > > > 
> > > > > > version 2:
> > > > > > - remove all change related to scaling
> > > > > > - squash commits to a coherent split
> > > > > > - be more verbose about the added fields
> > > > > > 
> > > > > >  .../media/v4l/ext-ctrls-codec.rst             | 126 ++++++++++++++
> > +---
> > > > > >  .../media/v4l/vidioc-queryctrl.rst            |   6 +
> > > > > >  drivers/media/v4l2-core/v4l2-ctrls.c          |  26 +++-
> > > > > >  drivers/staging/media/sunxi/cedrus/cedrus.c   |   6 +
> > > > > >  drivers/staging/media/sunxi/cedrus/cedrus.h   |   1 +
> > > > > >  .../staging/media/sunxi/cedrus/cedrus_dec.c   |   2 +
> > > > > >  .../staging/media/sunxi/cedrus/cedrus_h265.c  |   6 +-
> > > > > >  include/media/hevc-ctrls.h                    |  45 +++++--
> > > > > >  8 files changed, 186 insertions(+), 32 deletions(-)
> > > > > > 
> > > > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-
> > > > > > codec.rst 
> > b/
> > > > Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > > > index 00944e97d638..5e6d77e858c0 100644
> > > > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > > > @@ -3109,6 +3109,15 @@ enum
> > > > > > v4l2_mpeg_video_hevc_size_of_length_field -
> > > > > >      :stub-columns: 0
> > > > > >      :widths:       1 1 2
> > > > > >  
> > > > > > +    * - __u8
> > > > > > +      - ``video_parameter_set_id``
> > > > > > +      - Identifies the VPS for reference by other syntax elements
> > > > > > +    * - __u8
> > > > > > +      - ``seq_parameter_set_id?``
> > > > > > +      - Specifies the value of the vps_video_parameter_set_id of
> > > > > > the 
> > > > active VPS
> > > > > > +    * - __u8
> > > > > > +      - ``chroma_format_idc``
> > > > > > +      - Specifies the chroma sampling relative to the luma sampling
> > > > > 
> > > > > None of these fields seem needed for the Hantro G2 driver,
> > > > > so I suggest you drop them for now.
> > > > > 
> > > > > >      * - __u16
> > > > > >        - ``pic_width_in_luma_samples``
> > > > > >        -
> > > > > > @@ -3172,6 +3181,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field
> > > > > > -
> > > > > >      * - __u8
> > > > > >        - ``chroma_format_idc``
> > > > > >        -
> > > > > > +    * - __u8
> > > > > > +      - ``num_slices``
> > > > > > +
> > > > > 
> > > > > Not used, but also doesn't seem part of the SPS syntax. If we have to
> > > > > pass the number of slices, we'll need another mechanism.
> > > > > 
> > > > > >       -
> > > > > >      * - __u64
> > > > > >        - ``flags``
> > > > > >        - See :ref:`Sequence Parameter Set Flags <hevc_sps_flags>`
> > > > > > @@ -3231,9 +3243,18 @@ enum
> > > > > > v4l2_mpeg_video_hevc_size_of_length_field -
> > > > > >      :stub-columns: 0
> > > > > >      :widths:       1 1 2
> > > > > >  
> > > > > > +    * - __u8
> > > > > > +      - ``pic_parameter_set_id``
> > > > > > +      - Identifies the PPS for reference by other syntax elements
> > > > > 
> > > > > Not used.
> > > > > 
> > > > > >      * - __u8
> > > > > >        - ``num_extra_slice_header_bits``
> > > > > >        -
> > > > > > +    * - __u8
> > > > > > +      - ``num_ref_idx_l0_default_active_minus1``
> > > > > > +      - Specifies the inferred value of
> > > > > > num_ref_idx_l0_active_minus1
> > > > > > +    * - __u8
> > > > > > +      - ``num_ref_idx_l1_default_active_minus1``
> > > > > > +      - Specifies the inferred value of
> > > > > > num_ref_idx_l1_active_minus1
> > > > > >      * - __s8
> > > > > >        - ``init_qp_minus26``
> > > > > >        -
> > > > > > @@ -3342,6 +3363,12 @@ enum
> > > > > > v4l2_mpeg_video_hevc_size_of_length_field -
> > > > > >      * -
> > > > > > ``V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT``
> > > > > >        - 0x00040000
> > > > > >        -
> > > > > > +    * - ``V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT``
> > > > > > +      - 0x00080000
> > > > > > +      -
> > > > > > +    * - ``V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING``
> > > > > > +      - 0x00100000
> > > > > > +      -
> > > > > >  
> > > > > 
> > > > > I suggest to do all the PPS control changes in a separate patch,
> > > > > feels easier to review and cleaner as you can explain the
> > > > > changes with more detail in the commit description.
> > > > > 
> > > > > Looking at the PPS syntax for tiles, I'm wondering if these
> > > > > deserve their own control, which would be used if tiles are enabled,
> > > > > i.e. V4L2_HEVC_PPS_FLAG_TILES_ENABLED is set.
> > > > > 
> > > > >         __u8    
> > num_tile_columns_minus1;                                         
> > > > >         __u8    
> > num_tile_rows_minus1;                                            
> > > > >         __u8    
> > column_width_minus1[20];                                         
> > > > >         __u8    row_height_minus1[22];    
> > > > > 
> > > > > Not something we necessarily have to tackle now.
> > > > > 
> > > > > >  ``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
> > > > > >      Specifies various slice-specific parameters, especially from
> > > > > > the 
> > NAL 
> > > > unit
> > > > > > @@ -3366,6 +3393,12 @@ enum
> > > > > > v4l2_mpeg_video_hevc_size_of_length_field -
> > > > > >      * - __u32
> > > > > >        - ``data_bit_offset``
> > > > > >        - Offset (in bits) to the video data in the current slice
> > > > > > data.
> > > > > > +    * - __u32
> > > > > > +      - ``slice_segment_addr``
> > > > > > +      - Specifies the address of the first coding tree block in the
> > slice 
> > > > segment
> > > > > 
> > > > > Not used.
> > > > > 
> > > > > > +    * - __u32
> > > > > > +      - ``num_entry_point_offsets``
> > > > > > +      - Specifies the number of entry_point_offset_minus1[ i ]
> > > > > > syntax 
> > > > elements in the slice header
> > > > > 
> > > > > Not used.
> > > > 
> > > > While above two fields may not be used in Hantro, they are for sure
> > > > useful 
> > for 
> > > > Cedrus and RPi4. I would like to keep them, otherwise with such approach
> > HEVC 
> > > > will stay in staging for a long time. I'm still baffled why scaling
> > > > matrix 
> > > > control was dropped. It would fit well in Cedrus and RPi4 driver and
> > > > after 
> > a 
> > > > quick look, it seems that it was used in driver in later patch.
> > > > 
> > > 
> > > I'd like to make sure each modification we are making to the uAPI
> > > goes in the right direction, that is in the direction of moving
> > > the API out of staging.
> > > 
> > > Since reviewing each field is quite hard, and opens some discussions,
> > > I wanted to keep this patchset specific to what's needed for Hantro G2.
> > > 
> > > The Scaling matrix control is certainly a good one, as well as the ones
> > > needed for Cedrus and RPi4. However, I feel it's better to discuss
> > > them in their own "uAPI review" series so we can review all the changes
> > > with an API hat.
> > > 
> > > This way we decouple the Hantro G2 discussion and work from the API work.
> > > 
> > > Also please feel free to submit RFC patches fo Cedrus and RPi4
> > > (API and driver changes). We can certainly start the discussion around
> > > that,
> > > with driver changes in context.
> > 
> > I don't know much about RPi4 driver, only few implementation details, so 
> > you'll have to ping developer who wrote it. Regarding HEVC on Cedrus - it
> > has 
> > one pain point - it needs entry point table which in turn needs support for 
> > variable arrays in order to be feasable AFAIK. I don't plan to develop that.
> > Patches for scaling matrix and segment address were sent a bit more than a 
> > year ago but were turned down because they change control structures (among 
> > other things). Sorry to say, but I work on other things now, so Cedrus will 
> > have to wait. Alternatively, someone can take my patches from LibreELEC, 
> > update and submit them. They are in use for a long time.
> 
> It seems to me that H.265 should be the source for what fields are
> needed in the uapi - not whatever happens to be supported by current
> h/w. The standard defines the list of fields that can occur in the
> parameter sets and headers, and everything that is needed to decode 
> any slice_seqgment_data should be in the upai.  Eventually all those
> fields are going to be needed (they aren't in the standard just to look
> pretty) and given the reluctance I've observed to change the uapi once
> released they should be there from the start.
> 
> Now some hardware requires more fields that aren't in the standard -
> those can (and should) be added in h/w specific extensions.

While this is not wrong, there is obvious bitstream parameters that are not of
the HW accelerator domain. We did the work of deciding what to omit in the H264
uAPI, and we'll do that for HEVC for sure. But now is not the time yet.

Now the point that you seem to be missing is that Benjamin is trying to stage a
driver, not a uAPI. With this third driver, we can have more folks contributing
and testing toward our goal of unstaging the uAPI.

After this, we'll be able to iteratively add features (like scaling list) to
this driver along with the required uAPI and discuss each of these needed API
separatly. Note that previous iteration, the driver implementation for the
scaling list wasn't correct, and for this reason, the uAPI wasn't being
validated.

When this is getting stable enough, there will be a final stage were someone
will have to pick the uAPI and fill in the blank with remaining field that has a
strong probability to be used by later HW WITHIN the supported HEVC profiles et
have defined.

I emphasis on that because there is a lot of HEVC (and H264) extensions we
haven't covered or mentionned, and these will be added with dedicated controls
when we have a use case for it of course. Remember that adding controls is not a
problem, we need these basic control of the basic decoding, and for sure, over
the decade to come we will have to introduce new controls for other stuff we
haven't supported yet.

All this may look like taking forever, but as Jernej states, not everyone can
afford contributing upstream. So unless someone step in, this pace is about what
we can offer, and we strongly believe that it will converge. I do personnally
expect HEVC to take more time as the codec rationales are mostly kept secret,
and our answers burried into a mountain of reference code. It's also, if you
defocus from the film industry, just one CODEC. Notably, for the web, VP9 is
more important, and this will likely puts some shadow on this work.

Meawhile, you are strongly encouraged if you can afford it to open the
discussion around the RPi4 multi-stage HEVC decoder. While doing this, you could
send a patched for staging the RPi4 driver. The benefit would be that we get to
update your drivers when we update the uAPI, saving you times, and also giving
everyone more data on what is strictly needed.

> 
> Regards
> 
> JC
> 
> > Best regards,
> > Jernej
> > 
> > > 
> > > Hope I'm making sense here :)
> > > 
> > > Thanks,
> > > Ezequiel
> > > 
> > > > Best regards,
> > > > Jernej
> > > > 
> > > > > 
> > > > > >      * - __u8
> > > > > >        - ``nal_unit_type``
> > > > > >        -
> > > > > > @@ -3422,28 +3455,20 @@ enum
> > > > > > v4l2_mpeg_video_hevc_size_of_length_field 
> > -
> > > > > >      * - __u8
> > > > > >        - ``pic_struct``
> > > > > >        -
> > > > > > -    * - __u8
> > > > > > -      - ``num_active_dpb_entries``
> > > > > > -      - The number of entries in ``dpb``.
> > > > > 
> > > > > Need to explain in the commit description why this field is moved.
> > > > > 
> > > > > >      * - __u8
> > > > > >        - ``ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > > >        - The list of L0 reference elements as indices in the DPB.
> > > > > >      * - __u8
> > > > > >        - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > > >        - The list of L1 reference elements as indices in the DPB.
> > > > > > +    * - __u16
> > > > > > +      - ``short_term_ref_pic_set_size``
> > > > > > +
> > > > > 
> > > > > Not used.
> > > > > 
> > > > > >       -
> > > > > > +    * - __u16
> > > > > > +      - ``long_term_ref_pic_set_size``
> > > > > > +      -
> > > > > 
> > > > > Not used.
> > > > > 
> > > > > >      * - __u8
> > > > > > -      - ``num_rps_poc_st_curr_before``
> > > > > > -      - The number of reference pictures in the short-term set that
> > come 
> > > > before
> > > > > > -        the current frame.
> > > > > 
> > > > > If this matches NumPocStCurrBefore from section 8.3.2 "Decoding
> > > > > process 
> > for 
> > > > reference picture set"
> > > > > then I would document that. And perhaps rename it to 
> > num_poc_st_curr_before.
> > > > > 
> > > > > > -    * - __u8
> > > > > > -      - ``num_rps_poc_st_curr_after``
> > > > > > -      - The number of reference pictures in the short-term set that
> > come 
> > > > after
> > > > > > -        the current frame.
> > > > > 
> > > > > Ditto.
> > > > > 
> > > > > > -    * - __u8
> > > > > > -      - ``num_rps_poc_lt_curr``
> > > > > > -      - The number of reference pictures in the long-term set.
> > > > > 
> > > > > Ditto.
> > > > > 
> > > > > Also, I'd like the changes that move fields from 
> > > > V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS
> > > > > to the new V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS control, to be in 
> > their
> > > > > patch.
> > > > > 
> > > > > That will allow us to put in the commit description a proper
> > > > > explanation of why are fields being moved. Nothing fancy, simply
> > > > > explaining that these variables come from section 8.3.2
> > > > > "Decoding process for reference picture set", which describes
> > > > > a process invoked once per picture, so they are not per-slice.
> > > > > 
> > > > > > -    * - __u8
> > > > > > -      - ``padding[7]``
> > > > > > +      - ``padding``
> > > > > >        - Applications and drivers must set this to zero.
> > > > > >      * - struct :c:type:`v4l2_hevc_dpb_entry`
> > > > > >        - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > > > @@ -3646,3 +3671,74 @@ enum
> > > > > > v4l2_mpeg_video_hevc_size_of_length_field -
> > > > > >      so this has to come from client.
> > > > > >      This is applicable to H264 and valid Range is from 0 to 63.
> > > > > >      Source Rec. ITU-T H.264 (06/2019); G.7.4.1.1, G.8.8.1.
> > > > > > +
> > > > > > +``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> > > > > > +    Specifies various decode parameters, especially the references 
> > picture 
> > > > order
> > > > > > +    count (POC) for all the lists (short, long, before, current, 
> > after) 
> > > > and the
> > > > > > +    number of entries for each of them.
> > > > > > +    These parameters are defined according to :ref:`hevc`.
> > > > > > +    They are described in section 8.3 "Slice decoding process" of
> > > > > > the
> > > > > > +    specification.
> > > > > > +
> > > > > > +.. c:type:: v4l2_ctrl_hevc_decode_params
> > > > > > +
> > > > > > +.. cssclass:: longtable
> > > > > > +
> > > > > > +.. flat-table:: struct v4l2_ctrl_hevc_decode_params
> > > > > > +    :header-rows:  0
> > > > > > +    :stub-columns: 0
> > > > > > +    :widths:       1 1 2
> > > > > > +
> > > > > > +    * - __s32
> > > > > > +      - ``pic_order_cnt_val``
> > > > > > +      -
> > > > > 
> > > > > Can be documented as:
> > > > > 
> > > > > """
> > > > > PicOrderCntVal as described in section 8.3.1 "Decoding process
> > > > > for picture order count" of the specification.
> > > > > """
> > > > > 
> > > > > Note that snake case is used to match the kernel style,
> > > > > but other than that we try to keep the HEVC spec variable
> > > > > names.
> > > > > 
> > > > > > +    * - __u8
> > > > > > +      - ``num_active_dpb_entries``
> > > > > > +      - The number of entries in ``dpb``.
> > > > > > +    * - struct :c:type:`v4l2_hevc_dpb_entry`
> > > > > > +      - ``dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > > > +      - The decoded picture buffer, for meta-data about reference 
> > frames.
> > > > > 
> > > > > The DPB is here, but it seems it's also in the slice control?
> > > > > 
> > > > > > +    * - __u8
> > > > > > +      - ``num_rps_poc_st_curr_before``
> > > > > > +      - The number of reference pictures in the short-term set that
> > come 
> > > > before
> > > > > > +        the current frame.
> > > > > > +    * - __u8
> > > > > > +      - ``num_rps_poc_st_curr_after``
> > > > > > +      - The number of reference pictures in the short-term set that
> > come 
> > > > after
> > > > > > +        the current frame.
> > > > > > +    * - __u8
> > > > > > +      - ``num_rps_poc_lt_curr``
> > > > > > +      - The number of reference pictures in the long-term set.
> > > > > > +    * - __u8
> > > > > > +      - ``rps_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > > > +      -
> > > > > > +    * - __u8
> > > > > > +      - ``rps_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > > > +      -
> > > > > > +    * - __u8
> > > > > > +      - ``rps_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> > > > > > +      -
> > > > > 
> > > > > Could you document these as well?
> > > > > 
> > > > > Thanks a lot,
> > > > > Ezequiel
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > 
> > > 
> > > 
> > 
> 



_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 5/9] media: hantro: Introduce G2/HEVC decoder

[Benjamin Gaignard]

Le 25/02/2021 à 18:55, Ezequiel Garcia a écrit :
> Hi Benjamin,
>
> Thanks for the good work!
> I mostly have two concerns with this implementation,
> the tiled output and the allocation path.
>
> More below.
>
> On Mon, 2021-02-22 at 13:24 +0100, Benjamin Gaignard wrote:
>> Implement all the logic to get G2 hardware decoding HEVC frames.
>> It support up level 5.1 HEVC stream.
>> It doesn't support yet 10 bits formats or scaling feature.
>>
>> Add HANTRO HEVC dedicated control to skip some bits at the beginning
>> of the slice header. That is very specific to this hardware so can't
>> go into uapi structures. Compute the needed value is complex and require
>> information from the stream that only the userland knows so let it
>> provide the correct value to the driver.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> version 2:
>> - squash multiple commits in this one.
>> - fix the comments done by Ezequiel about dma_alloc_coherent usage
>> - fix Dan's comments about control copy, reverse the test logic
>> in tile_buffer_reallocate, rework some goto and return cases.
>>
>>   drivers/staging/media/hantro/Makefile         |   2 +
>>   drivers/staging/media/hantro/hantro.h         |  19 +
>>   drivers/staging/media/hantro/hantro_drv.c     |  42 ++
>>   .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++++++++++++++++++
>>   drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++++++
>>   drivers/staging/media/hantro/hantro_hevc.c    | 321 ++++++++++
>>   drivers/staging/media/hantro/hantro_hw.h      |  47 ++
>>   7 files changed, 1216 insertions(+)
>>   create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
>>   create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
>>   create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
>>
>>
> [snip]
>> +
>> +static void set_buffers(struct hantro_ctx *ctx)
>> +{
>> +       struct vb2_v4l2_buffer *src_buf, *dst_buf;
>> +       struct hantro_dev *vpu = ctx->dev;
>> +       const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
>> +       const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
>> +       size_t cr_offset = hantro_hevc_chroma_offset(sps);
>> +       dma_addr_t src_dma, dst_dma;
>> +       u32 src_len, src_buf_len;
>> +
>> +       src_buf = hantro_get_src_buf(ctx);
>> +       dst_buf = hantro_get_dst_buf(ctx);
>> +
>> +       /* Source (stream) buffer. */
>> +       src_dma = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
>> +       src_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
>> +       src_buf_len = vb2_plane_size(&src_buf->vb2_buf, 0);
>> +
>> +       hantro_write_addr(vpu, HEVC_ADDR_STR, src_dma);
>> +       hantro_reg_write(vpu, hevc_stream_len, src_len);
>> +       hantro_reg_write(vpu, hevc_strm_buffer_len, src_buf_len);
>> +       hantro_reg_write(vpu, hevc_strm_start_offset, 0);
>> +       hantro_reg_write(vpu, hevc_write_mvs_e, 1);
>> +
>> +       /* Destination (decoded frame) buffer. */
>> +       dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf);
>> +
>> +       hantro_write_addr(vpu, HEVC_RASTER_SCAN, dst_dma);
>> +       hantro_write_addr(vpu, HEVC_RASTER_SCAN_CHR, dst_dma + cr_offset);
> The way this is done the raster-scan output is the only
> output, and the tiled ouput it kept entirely internal, in hantro_hevc_dec_hw_ctx.ref_bufs.
>
> This means there's no way to expose tiled NV12 in i.MX8M VPU tiled format
> to userspace, which could be desirable for some use cases.
>
> I'm not suggesting to actually expose tiled NV12 in this patch, but to prepare
> the driver to be able to support that easily in the future.
>
> It should be possible to consider this detiling as
> post-processing, adding some code in hantro_postproc.c
> leveraging the existing post-proc support.
>
> IOW, hantro_postproc_ctx.dec_q would hold the tiled output,
> hantro_postproc_enable() writes the raster-scan
> buffer destination address, and so on.

Well the G2 can't use the post-processor so I'm reluctant to do as if it was the case.

To support tiled NV12 for me the solution is to create a hantro_hevc_add_ref_buf() function
to add the destination buffer in hevc_dec->ref_bufs.
We can test destination format to set hevc_out_rs_e bit and HEVC_RASTER_SCAN register.

>
>> +       hantro_write_addr(vpu, HEVC_ADDR_TILE_SIZE, ctx->hevc_dec.tile_sizes.dma);
>> +       hantro_write_addr(vpu, HEVC_TILE_FILTER, ctx->hevc_dec.tile_filter.dma);
>> +       hantro_write_addr(vpu, HEVC_TILE_SAO, ctx->hevc_dec.tile_sao.dma);
>> +       hantro_write_addr(vpu, HEVC_TILE_BSD, ctx->hevc_dec.tile_bsd.dma);
>> +}
>> +
>> +void hantro_g2_check_idle(struct hantro_dev *vpu)
>> +{
>> +       int i;
>> +
>> +       for (i = 0; i < 3; i++) {
>> +               u32 status;
>> +
>> +               /* Make sure the VPU is idle */
>> +               status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
>> +               if (status & HEVC_REG_INTERRUPT_DEC_E) {
>> +                       pr_warn("%s: still enabled!!! resetting.\n", __func__);
>> +                       status |= HEVC_REG_INTERRUPT_DEC_ABORT_E | HEVC_REG_INTERRUPT_DEC_IRQ_DIS;
>> +                       vdpu_write(vpu, status, HEVC_REG_INTERRUPT);
>> +               }
>> +       }
>> +}
>> +
>> +void hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
>> +{
>> +       struct hantro_dev *vpu = ctx->dev;
>> +
>> +       hantro_g2_check_idle(vpu);
>> +
>> +       /* Prepare HEVC decoder context. */
>> +       if (hantro_hevc_dec_prepare_run(ctx))
>> +               return;
>> +
>> +       /* Configure hardware registers. */
>> +       set_params(ctx);
>> +
>> +       /* set reference pictures */
>> +       if (set_ref(ctx))
>> +               /* something goes wrong */
>> +               hantro_irq_done(vpu, VB2_BUF_STATE_ERROR);
>> +
> I don't think we want to allow the _run() operation to fail like this.
> In other words, I would avoid allocating buffers in the _run() path,
> and doing all allocation at start() time.
>
> That's why hantro_start_streaming() calls hantro_postproc_alloc() if needed.

The error could also be that the reference picture isn't found in the list so
we can't perform the decode operation.
If we do the allocation at start time we will allocate all the reference frames
buffers even if we don't know if we will use them. That could increase a lot
memory usage. Note that buffers allocated once and re-used until the end of the
session.

>
>> +       set_buffers(ctx);
>> +       prepare_tile_info_buffer(ctx);
>> +
>> +       hantro_end_prepare_run(ctx);
>> +
>> +       hantro_reg_write(vpu, hevc_mode, HEVC_DEC_MODE);
>> +       hantro_reg_write(vpu, hevc_clk_gate_e, 1);
>> +
>> +       /* Don't disable output */
>> +       hantro_reg_write(vpu, hevc_out_dis, 0);
>> +
>> +       /* Don't compress buffers */
>> +       hantro_reg_write(vpu, hevc_ref_compress_bypass, 1);
>> +
>> +       /* use NV12 as output format */
>> +       hantro_reg_write(vpu, hevc_tile_e, 0);
> Unless I'm missing something, this ^
>
>> +       hantro_reg_write(vpu, hevc_out_rs_e, 1);
>> +       hantro_reg_write(vpu, hevc_num_tile_rows, 1);
>> +       hantro_reg_write(vpu, hevc_num_tile_cols, 1);
>> +
> and this ^ these shouldn't be here.
>
> HEVC tiles are handled already. See prepare_tile_info_buffer().

You are right, I will remove that, thanks.

Benjamin

> Note that HEVC "tiles" are not related to NV12 "tiled" format.
>
> Thanks!
> Ezequiel
>
>

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

Re: [PATCH v3 5/9] media: hantro: Introduce G2/HEVC decoder

[Ezequiel Garcia]

On Fri, 2021-02-26 at 10:47 +0100, Benjamin Gaignard wrote:
> 
> Le 25/02/2021 à 18:55, Ezequiel Garcia a écrit :
> > Hi Benjamin,
> > 
> > Thanks for the good work!
> > I mostly have two concerns with this implementation,
> > the tiled output and the allocation path.
> > 
> > More below.
> > 
> > On Mon, 2021-02-22 at 13:24 +0100, Benjamin Gaignard wrote:
> > > Implement all the logic to get G2 hardware decoding HEVC frames.
> > > It support up level 5.1 HEVC stream.
> > > It doesn't support yet 10 bits formats or scaling feature.
> > > 
> > > Add HANTRO HEVC dedicated control to skip some bits at the beginning
> > > of the slice header. That is very specific to this hardware so can't
> > > go into uapi structures. Compute the needed value is complex and require
> > > information from the stream that only the userland knows so let it
> > > provide the correct value to the driver.
> > > 
> > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > ---
> > > version 2:
> > > - squash multiple commits in this one.
> > > - fix the comments done by Ezequiel about dma_alloc_coherent usage
> > > - fix Dan's comments about control copy, reverse the test logic
> > > in tile_buffer_reallocate, rework some goto and return cases.
> > > 
> > >   drivers/staging/media/hantro/Makefile         |   2 +
> > >   drivers/staging/media/hantro/hantro.h         |  19 +
> > >   drivers/staging/media/hantro/hantro_drv.c     |  42 ++
> > >   .../staging/media/hantro/hantro_g2_hevc_dec.c | 587 ++++++++++++++++++
> > >   drivers/staging/media/hantro/hantro_g2_regs.h | 198 ++++++
> > >   drivers/staging/media/hantro/hantro_hevc.c    | 321 ++++++++++
> > >   drivers/staging/media/hantro/hantro_hw.h      |  47 ++
> > >   7 files changed, 1216 insertions(+)
> > >   create mode 100644 drivers/staging/media/hantro/hantro_g2_hevc_dec.c
> > >   create mode 100644 drivers/staging/media/hantro/hantro_g2_regs.h
> > >   create mode 100644 drivers/staging/media/hantro/hantro_hevc.c
> > > 
> > > 
> > [snip]
> > > +
> > > +static void set_buffers(struct hantro_ctx *ctx)
> > > +{
> > > +       struct vb2_v4l2_buffer *src_buf, *dst_buf;
> > > +       struct hantro_dev *vpu = ctx->dev;
> > > +       const struct hantro_hevc_dec_ctrls *ctrls = &ctx->hevc_dec.ctrls;
> > > +       const struct v4l2_ctrl_hevc_sps *sps = ctrls->sps;
> > > +       size_t cr_offset = hantro_hevc_chroma_offset(sps);
> > > +       dma_addr_t src_dma, dst_dma;
> > > +       u32 src_len, src_buf_len;
> > > +
> > > +       src_buf = hantro_get_src_buf(ctx);
> > > +       dst_buf = hantro_get_dst_buf(ctx);
> > > +
> > > +       /* Source (stream) buffer. */
> > > +       src_dma = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
> > > +       src_len = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
> > > +       src_buf_len = vb2_plane_size(&src_buf->vb2_buf, 0);
> > > +
> > > +       hantro_write_addr(vpu, HEVC_ADDR_STR, src_dma);
> > > +       hantro_reg_write(vpu, hevc_stream_len, src_len);
> > > +       hantro_reg_write(vpu, hevc_strm_buffer_len, src_buf_len);
> > > +       hantro_reg_write(vpu, hevc_strm_start_offset, 0);
> > > +       hantro_reg_write(vpu, hevc_write_mvs_e, 1);
> > > +
> > > +       /* Destination (decoded frame) buffer. */
> > > +       dst_dma = hantro_get_dec_buf_addr(ctx, &dst_buf->vb2_buf);
> > > +
> > > +       hantro_write_addr(vpu, HEVC_RASTER_SCAN, dst_dma);
> > > +       hantro_write_addr(vpu, HEVC_RASTER_SCAN_CHR, dst_dma + cr_offset);
> > The way this is done the raster-scan output is the only
> > output, and the tiled ouput it kept entirely internal, in hantro_hevc_dec_hw_ctx.ref_bufs.
> > 
> > This means there's no way to expose tiled NV12 in i.MX8M VPU tiled format
> > to userspace, which could be desirable for some use cases.
> > 
> > I'm not suggesting to actually expose tiled NV12 in this patch, but to prepare
> > the driver to be able to support that easily in the future.
> > 
> > It should be possible to consider this detiling as
> > post-processing, adding some code in hantro_postproc.c
> > leveraging the existing post-proc support.
> > 
> > IOW, hantro_postproc_ctx.dec_q would hold the tiled output,
> > hantro_postproc_enable() writes the raster-scan
> > buffer destination address, and so on.
> 
> Well the G2 can't use the post-processor so I'm reluctant to do as if it was the case.
> 

I don't really see a big difference between G1 post-processing, and G2 raster-scan
detiling. As a user of the device, both features offer a post-processing.

In any case, it's not a big deal, it's fine as-is if you are inclined to this implementation.

> To support tiled NV12 for me the solution is to create a hantro_hevc_add_ref_buf() function
> to add the destination buffer in hevc_dec->ref_bufs.
> We can test destination format to set hevc_out_rs_e bit and HEVC_RASTER_SCAN register.
> 

Is it worth adding some boilerplate and/or refactoring things a bit, so it's
easier to expose tiled NV12 once that format is exposable?

> > 
> > > +       hantro_write_addr(vpu, HEVC_ADDR_TILE_SIZE, ctx->hevc_dec.tile_sizes.dma);
> > > +       hantro_write_addr(vpu, HEVC_TILE_FILTER, ctx->hevc_dec.tile_filter.dma);
> > > +       hantro_write_addr(vpu, HEVC_TILE_SAO, ctx->hevc_dec.tile_sao.dma);
> > > +       hantro_write_addr(vpu, HEVC_TILE_BSD, ctx->hevc_dec.tile_bsd.dma);
> > > +}
> > > +
> > > +void hantro_g2_check_idle(struct hantro_dev *vpu)
> > > +{
> > > +       int i;
> > > +
> > > +       for (i = 0; i < 3; i++) {
> > > +               u32 status;
> > > +
> > > +               /* Make sure the VPU is idle */
> > > +               status = vdpu_read(vpu, HEVC_REG_INTERRUPT);
> > > +               if (status & HEVC_REG_INTERRUPT_DEC_E) {
> > > +                       pr_warn("%s: still enabled!!! resetting.\n", __func__);
> > > +                       status |= HEVC_REG_INTERRUPT_DEC_ABORT_E | HEVC_REG_INTERRUPT_DEC_IRQ_DIS;
> > > +                       vdpu_write(vpu, status, HEVC_REG_INTERRUPT);
> > > +               }
> > > +       }
> > > +}
> > > +
> > > +void hantro_g2_hevc_dec_run(struct hantro_ctx *ctx)
> > > +{
> > > +       struct hantro_dev *vpu = ctx->dev;
> > > +
> > > +       hantro_g2_check_idle(vpu);
> > > +
> > > +       /* Prepare HEVC decoder context. */
> > > +       if (hantro_hevc_dec_prepare_run(ctx))
> > > +               return;
> > > +
> > > +       /* Configure hardware registers. */
> > > +       set_params(ctx);
> > > +
> > > +       /* set reference pictures */
> > > +       if (set_ref(ctx))
> > > +               /* something goes wrong */
> > > +               hantro_irq_done(vpu, VB2_BUF_STATE_ERROR);
> > > +
> > I don't think we want to allow the _run() operation to fail like this.
> > In other words, I would avoid allocating buffers in the _run() path,
> > and doing all allocation at start() time.
> > 
> > That's why hantro_start_streaming() calls hantro_postproc_alloc() if needed.
> 
> The error could also be that the reference picture isn't found in the list so
> we can't perform the decode operation.
> If we do the allocation at start time we will allocate all the reference frames
> buffers even if we don't know if we will use them. That could increase a lot
> memory usage. Note that buffers allocated once and re-used until the end of the
> session.
> 

Improving memory usage is a good point. I wonder if it makes sense to release
some buffers under some condition, instead of keeping then allocated (and unused)
until the end of the session.

(BTW, the same can be done with the post-processor, as it's basically
the same process. It would be interesting to explore.)

I'm still unsure why we'd call hantro_irq_done() (which cancels the timeout
handler and returns buffers to userspace), but we would keep programming
the operation. 

Thanks,
Ezequiel


_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip