[PATCH 0/2] media: meson: vdec: Add compliant H264 support

[PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Maxime Jourdan]

Hello,

This patch series aims to bring H.264 support as well as compliance update
to the amlogic stateful video decoder driver.

There is 1 issue that remains currently:

 - The following codepath had to be commented out from v4l2-compliance as
it led to stalling:

if (node->codec_mask & STATEFUL_DECODER) {
	struct v4l2_decoder_cmd cmd;
	buffer buf_cap(m2m_q);

	memset(&cmd, 0, sizeof(cmd));
	cmd.cmd = V4L2_DEC_CMD_STOP;

	/* No buffers are queued, call STREAMON, then STOP */
	fail_on_test(node->streamon(q.g_type()));
	fail_on_test(node->streamon(m2m_q.g_type()));
	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));

	fail_on_test(buf_cap.querybuf(node, 0));
	fail_on_test(buf_cap.qbuf(node));
	fail_on_test(buf_cap.dqbuf(node));
	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
		fail_on_test(buf_cap.g_bytesused(p));
	fail_on_test(node->streamoff(q.g_type()));
	fail_on_test(node->streamoff(m2m_q.g_type()));

	/* Call STREAMON, queue one CAPTURE buffer, then STOP */
	fail_on_test(node->streamon(q.g_type()));
	fail_on_test(node->streamon(m2m_q.g_type()));
	fail_on_test(buf_cap.querybuf(node, 0));
	fail_on_test(buf_cap.qbuf(node));
	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));

	fail_on_test(buf_cap.dqbuf(node));
	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
		fail_on_test(buf_cap.g_bytesused(p));
	fail_on_test(node->streamoff(q.g_type()));
	fail_on_test(node->streamoff(m2m_q.g_type()));
}

The reason for this is because the driver has a limitation where all
capturebuffers must be queued to the driver before STREAMON is effective.
The firmware needs to know in advance what all the buffers are before
starting to decode.
This limitation is enforced via q->min_buffers_needed.
As such, in this compliance codepath, STREAMON is never actually called
driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));


One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
by v4l2-compliance, so it was left out for the test. However, it is
present in the patch series.

The second patch has 3 "Alignment should match open parenthesis" lines
where I preferred to keep them that way.

Thanks Stanimir for sharing your HDR file creation tools, this was very
helpful :).

Maxime

# v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits

Compliance test for meson-vdec device /dev/video0:

Driver Info:
	Driver name      : meson-vdec
	Card type        : Amlogic Video Decoder
	Bus info         : platform:meson-vdec
	Driver version   : 5.4.0
	Capabilities     : 0x84204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
	Detected Stateful Decoder

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
	test read/write: OK (Not Supported)
	test blocking wait: OK
	Video Capture Multiplanar: Captured 250 buffers   
	test MMAP (select): OK
	Video Capture Multiplanar: Captured 250 buffers   
	test MMAP (epoll): OK
	test USERPTR (select): OK (Not Supported)
	test DMABUF: Cannot test, specify --expbuf-device

Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0

Maxime Jourdan (2):
  media: meson: vdec: bring up to compliance
  media: meson: vdec: add H.264 decoding support

 drivers/staging/media/meson/vdec/Makefile     |   2 +-
 drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
 drivers/staging/media/meson/vdec/codec_h264.h |  14 +
 drivers/staging/media/meson/vdec/esparser.c   |  34 +-
 drivers/staging/media/meson/vdec/vdec.c       |  70 ++-
 drivers/staging/media/meson/vdec/vdec.h       |  14 +-
 .../staging/media/meson/vdec/vdec_helpers.c   |  85 ++-
 .../staging/media/meson/vdec/vdec_helpers.h   |   6 +-
 .../staging/media/meson/vdec/vdec_platform.c  |  43 ++
 9 files changed, 654 insertions(+), 96 deletions(-)
 create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
 create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h

-- 
2.23.0

[PATCH 1/2] media: meson: vdec: bring up to compliance

[Maxime Jourdan]

Add all the necessary bits to pass v4l2-compliance in stateful decoding
mode.

Mostly includes tracking the state of the decoder, allowing the OUTPUT
queue to stream while the CAPTURE queue is inactive, handling resolution
change events, draining with V4L2_DEC_CMD_STOP, copying more metadata
from the src buffers to the dst buffers, etc.

Signed-off-by: Maxime Jourdan <mjourdan@baylibre.com>
---
 drivers/staging/media/meson/vdec/esparser.c   | 34 ++------
 drivers/staging/media/meson/vdec/vdec.c       | 70 ++++++++++-----
 drivers/staging/media/meson/vdec/vdec.h       | 14 ++-
 .../staging/media/meson/vdec/vdec_helpers.c   | 85 +++++++++----------
 .../staging/media/meson/vdec/vdec_helpers.h   |  6 +-
 .../staging/media/meson/vdec/vdec_platform.c  |  6 ++
 6 files changed, 120 insertions(+), 95 deletions(-)

diff --git a/drivers/staging/media/meson/vdec/esparser.c b/drivers/staging/media/meson/vdec/esparser.c
index 95102a4bdc62..a083d67be405 100644
--- a/drivers/staging/media/meson/vdec/esparser.c
+++ b/drivers/staging/media/meson/vdec/esparser.c
@@ -180,29 +180,25 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf)
 	int ret;
 	struct vb2_buffer *vb = &vbuf->vb2_buf;
 	struct amvdec_core *core = sess->core;
-	struct amvdec_codec_ops *codec_ops = sess->fmt_out->codec_ops;
-	u32 num_dst_bufs = 0;
 	u32 payload_size = vb2_get_plane_payload(vb, 0);
 	dma_addr_t phy = vb2_dma_contig_plane_dma_addr(vb, 0);
 	u32 offset;
 	u32 pad_size;
 
-	if (codec_ops->num_pending_bufs)
-		num_dst_bufs = codec_ops->num_pending_bufs(sess);
-
-	num_dst_bufs += v4l2_m2m_num_dst_bufs_ready(sess->m2m_ctx);
-
-	if (esparser_vififo_get_free_space(sess) < payload_size ||
-	    atomic_read(&sess->esparser_queued_bufs) >= num_dst_bufs)
+	if (esparser_vififo_get_free_space(sess) < payload_size)
 		return -EAGAIN;
 
 	v4l2_m2m_src_buf_remove_by_buf(sess->m2m_ctx, vbuf);
 
 	offset = esparser_get_offset(sess);
 
-	amvdec_add_ts_reorder(sess, vb->timestamp, offset);
-	dev_dbg(core->dev, "esparser: ts = %llu pld_size = %u offset = %08X\n",
-		vb->timestamp, payload_size, offset);
+	amvdec_add_ts(sess, vb->timestamp, vbuf->timecode, offset, vbuf->flags);
+	dev_dbg(core->dev, "esparser: ts = %llu pld_size = %u offset = %08X flags = %08X\n",
+		vb->timestamp, payload_size, offset, vbuf->flags);
+
+	vbuf->flags = 0;
+	vbuf->field = V4L2_FIELD_NONE;
+	vbuf->sequence = sess->sequence_out++;
 
 	pad_size = esparser_pad_start_code(vb);
 	ret = esparser_write_data(core, phy, payload_size + pad_size);
@@ -216,19 +212,7 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf)
 		return 0;
 	}
 
-	/* We need to wait until we parse the first keyframe.
-	 * All buffers prior to the first keyframe must be dropped.
-	 */
-	if (!sess->keyframe_found)
-		usleep_range(1000, 2000);
-
-	if (sess->keyframe_found)
-		atomic_inc(&sess->esparser_queued_bufs);
-	else
-		amvdec_remove_ts(sess, vb->timestamp);
-
-	vbuf->flags = 0;
-	vbuf->field = V4L2_FIELD_NONE;
+	atomic_inc(&sess->esparser_queued_bufs);
 	v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
 
 	return 0;
diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
index 0a1a04fd5d13..0b571b3a1e33 100644
--- a/drivers/staging/media/meson/vdec/vdec.c
+++ b/drivers/staging/media/meson/vdec/vdec.c
@@ -166,7 +166,10 @@ static void process_num_buffers(struct vb2_queue *q,
 {
 	const struct amvdec_format *fmt_out = sess->fmt_out;
 	unsigned int buffers_total = q->num_buffers + *num_buffers;
+	u32 min_buf_capture = v4l2_ctrl_g_ctrl(sess->ctrl_min_buf_capture);
 
+	if (q->num_buffers + *num_buffers < min_buf_capture)
+		*num_buffers = min_buf_capture - q->num_buffers;
 	if (is_reqbufs && buffers_total < fmt_out->min_buffers)
 		*num_buffers = fmt_out->min_buffers - q->num_buffers;
 	if (buffers_total > fmt_out->max_buffers)
@@ -191,7 +194,8 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
 	if (*num_planes) {
 		switch (q->type) {
 		case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
-			if (*num_planes != 1 || sizes[0] < output_size)
+			if (*num_planes != 1 ||
+			    sizes[0] < sess->src_buffer_size)
 				return -EINVAL;
 			break;
 		case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
@@ -222,7 +226,7 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
 
 	switch (q->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
-		sizes[0] = amvdec_get_output_size(sess);
+		sizes[0] = sess->src_buffer_size;
 		*num_planes = 1;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
@@ -248,6 +252,7 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
 		return -EINVAL;
 	}
 
+	sess->changed_format = 1;
 	return 0;
 }
 
@@ -259,10 +264,11 @@ static void vdec_vb2_buf_queue(struct vb2_buffer *vb)
 
 	v4l2_m2m_buf_queue(m2m_ctx, vbuf);
 
-	if (!sess->streamon_out || !sess->streamon_cap)
+	if (!sess->streamon_out)
 		return;
 
-	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	if (sess->streamon_cap &&
+	    vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
 	    vdec_codec_needs_recycle(sess))
 		vdec_queue_recycle(sess, vb);
 
@@ -287,16 +293,22 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 	else
 		sess->streamon_cap = 1;
 
-	if (!sess->streamon_out || !sess->streamon_cap)
+	if (!sess->streamon_out)
 		return 0;
 
 	if (sess->status == STATUS_NEEDS_RESUME &&
-	    q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
+	    q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
+	    sess->changed_format) {
 		codec_ops->resume(sess);
 		sess->status = STATUS_RUNNING;
 		return 0;
 	}
 
+	if (sess->status == STATUS_RUNNING ||
+	    sess->status == STATUS_NEEDS_RESUME ||
+	    sess->status == STATUS_INIT)
+		return 0;
+
 	sess->vififo_size = SIZE_VIFIFO;
 	sess->vififo_vaddr =
 		dma_alloc_coherent(sess->core->dev, sess->vififo_size,
@@ -321,13 +333,14 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto vififo_free;
 
 	sess->sequence_cap = 0;
+	sess->sequence_out = 0;
 	if (vdec_codec_needs_recycle(sess))
 		sess->recycle_thread = kthread_run(vdec_recycle_thread, sess,
 						   "vdec_recycle");
 
-	sess->status = STATUS_RUNNING;
+	sess->status = STATUS_INIT;
 	core->cur_sess = sess;
-
+	schedule_work(&sess->esparser_queue_work);
 	return 0;
 
 vififo_free:
@@ -384,6 +397,7 @@ static void vdec_stop_streaming(struct vb2_queue *q)
 	struct vb2_v4l2_buffer *buf;
 
 	if (sess->status == STATUS_RUNNING ||
+	    sess->status == STATUS_INIT ||
 	    (sess->status == STATUS_NEEDS_RESUME &&
 	     (!sess->streamon_out || !sess->streamon_cap))) {
 		if (vdec_codec_needs_recycle(sess))
@@ -474,20 +488,33 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,
 	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
 	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
 	const struct amvdec_format *fmts = sess->core->platform->formats;
-	const struct amvdec_format *fmt_out;
+	const struct amvdec_format *fmt_out = NULL;
+	u32 output_size = 0;
 
 	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
 	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
 
-	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
 		fmt_out = find_format(fmts, size, pixmp->pixelformat);
 		if (!fmt_out) {
 			pixmp->pixelformat = V4L2_PIX_FMT_MPEG2;
 			fmt_out = find_format(fmts, size, pixmp->pixelformat);
 		}
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		fmt_out = sess->fmt_out;
+		break;
+	default:
+		return NULL;
+	}
+
+	pixmp->width  = clamp(pixmp->width,  (u32)256, fmt_out->max_width);
+	pixmp->height = clamp(pixmp->height, (u32)144, fmt_out->max_height);
+	output_size = get_output_size(pixmp->width, pixmp->height);
 
-		pfmt[0].sizeimage =
-			get_output_size(pixmp->width, pixmp->height);
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
+		pfmt[0].sizeimage = sess->src_buffer_size;
 		pfmt[0].bytesperline = 0;
 		pixmp->num_planes = 1;
 	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
@@ -519,8 +546,6 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,
 			pfmt[2].bytesperline = ALIGN(pixmp->width, 64) / 2;
 			pixmp->num_planes = 3;
 		}
-	} else {
-		return NULL;
 	}
 
 	pixmp->width  = clamp(pixmp->width,  (u32)256, fmt_out->max_width);
@@ -584,6 +609,8 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 	orig_pixmp = *pixmp;
 
 	fmt_out = vdec_try_fmt_common(sess, num_formats, f);
+	if (!fmt_out)
+		return -EINVAL;
 
 	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
 		pixfmt_out = pixmp->pixelformat;
@@ -608,6 +635,7 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
 		sess->ycbcr_enc = pixmp->ycbcr_enc;
 		sess->quantization = pixmp->quantization;
 		sess->xfer_func = pixmp->xfer_func;
+		sess->src_buffer_size = pixmp->plane_fmt[0].sizeimage;
 	}
 
 	memset(&format, 0, sizeof(format));
@@ -699,29 +727,28 @@ vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
 	if (!(sess->streamon_out & sess->streamon_cap))
 		return 0;
 
-	/* Currently not handled since we do not support dynamic resolution
-	 * for MPEG2. We consider both queues streaming to mean that the
-	 * decoding session is started
-	 */
-	if (cmd->cmd == V4L2_DEC_CMD_START)
+	if (cmd->cmd == V4L2_DEC_CMD_START) {
+		sess->should_stop = 0;
 		return 0;
+	}
 
 	/* Should not happen */
 	if (cmd->cmd != V4L2_DEC_CMD_STOP)
 		return -EINVAL;
 
 	dev_dbg(dev, "Received V4L2_DEC_CMD_STOP\n");
-	sess->should_stop = 1;
 
-	vdec_wait_inactive(sess);
+	sess->should_stop = 1;
 
 	if (codec_ops->drain) {
+		vdec_wait_inactive(sess);
 		codec_ops->drain(sess);
 	} else if (codec_ops->eos_sequence) {
 		u32 len;
 		const u8 *data = codec_ops->eos_sequence(&len);
 
 		esparser_queue_eos(sess->core, data, len);
+		vdec_wait_inactive(sess);
 	}
 
 	return ret;
@@ -881,6 +908,7 @@ static int vdec_open(struct file *file)
 	sess->height = 720;
 	sess->pixelaspect.numerator = 1;
 	sess->pixelaspect.denominator = 1;
+	sess->src_buffer_size = SZ_1M;
 
 	INIT_LIST_HEAD(&sess->timestamps);
 	INIT_LIST_HEAD(&sess->bufs_recycle);
diff --git a/drivers/staging/media/meson/vdec/vdec.h b/drivers/staging/media/meson/vdec/vdec.h
index d811e7976519..163d6dddfad6 100644
--- a/drivers/staging/media/meson/vdec/vdec.h
+++ b/drivers/staging/media/meson/vdec/vdec.h
@@ -29,13 +29,19 @@ struct amvdec_buffer {
  * struct amvdec_timestamp - stores a src timestamp along with a VIFIFO offset
  *
  * @list: used to make lists out of this struct
- * @ts: timestamp
+ * @tc: timecode from the v4l2 buffer
+ * @ts: timestamp from the VB2 buffer
  * @offset: offset in the VIFIFO where the associated packet was written
+ * @flags: flags from the v4l2 buffer
+ * @used_count: times this timestamp was checked for a match with a dst buffer
  */
 struct amvdec_timestamp {
 	struct list_head list;
+	struct v4l2_timecode tc;
 	u64 ts;
 	u32 offset;
+	u32 flags;
+	u32 used_count;
 };
 
 struct amvdec_session;
@@ -164,6 +170,7 @@ struct amvdec_format {
 
 enum amvdec_status {
 	STATUS_STOPPED,
+	STATUS_INIT,
 	STATUS_RUNNING,
 	STATUS_NEEDS_RESUME,
 };
@@ -179,6 +186,7 @@ enum amvdec_status {
  * @ctrl_min_buf_capture: V4L2 control V4L2_CID_MIN_BUFFERS_FOR_CAPTURE
  * @fmt_out: vdec pixel format for the OUTPUT queue
  * @pixfmt_cap: V4L2 pixel format for the CAPTURE queue
+ * @src_buffer_size: size in bytes of the OUTPUT buffers' only plane
  * @width: current picture width
  * @height: current picture height
  * @colorspace: current colorspace
@@ -220,6 +228,7 @@ struct amvdec_session {
 
 	const struct amvdec_format *fmt_out;
 	u32 pixfmt_cap;
+	u32 src_buffer_size;
 
 	u32 width;
 	u32 height;
@@ -234,10 +243,11 @@ struct amvdec_session {
 	struct work_struct esparser_queue_work;
 
 	unsigned int streamon_cap, streamon_out;
-	unsigned int sequence_cap;
+	unsigned int sequence_cap, sequence_out;
 	unsigned int should_stop;
 	unsigned int keyframe_found;
 	unsigned int num_dst_bufs;
+	unsigned int changed_format;
 
 	u8 canvas_alloc[MAX_CANVAS];
 	u32 canvas_num;
diff --git a/drivers/staging/media/meson/vdec/vdec_helpers.c b/drivers/staging/media/meson/vdec/vdec_helpers.c
index f16948bdbf2f..ff4333074197 100644
--- a/drivers/staging/media/meson/vdec/vdec_helpers.c
+++ b/drivers/staging/media/meson/vdec/vdec_helpers.c
@@ -200,33 +200,23 @@ int amvdec_set_canvases(struct amvdec_session *sess,
 }
 EXPORT_SYMBOL_GPL(amvdec_set_canvases);
 
-void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
+void amvdec_add_ts(struct amvdec_session *sess, u64 ts,
+		   struct v4l2_timecode tc, u32 offset, u32 vbuf_flags)
 {
-	struct amvdec_timestamp *new_ts, *tmp;
+	struct amvdec_timestamp *new_ts;
 	unsigned long flags;
 
-	new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
+	new_ts = kzalloc(sizeof(*new_ts), GFP_KERNEL);
 	new_ts->ts = ts;
+	new_ts->tc = tc;
 	new_ts->offset = offset;
+	new_ts->flags = vbuf_flags;
 
 	spin_lock_irqsave(&sess->ts_spinlock, flags);
-
-	if (list_empty(&sess->timestamps))
-		goto add_tail;
-
-	list_for_each_entry(tmp, &sess->timestamps, list) {
-		if (ts <= tmp->ts) {
-			list_add_tail(&new_ts->list, &tmp->list);
-			goto unlock;
-		}
-	}
-
-add_tail:
 	list_add_tail(&new_ts->list, &sess->timestamps);
-unlock:
 	spin_unlock_irqrestore(&sess->ts_spinlock, flags);
 }
-EXPORT_SYMBOL_GPL(amvdec_add_ts_reorder);
+EXPORT_SYMBOL_GPL(amvdec_add_ts);
 
 void amvdec_remove_ts(struct amvdec_session *sess, u64 ts)
 {
@@ -251,8 +241,8 @@ EXPORT_SYMBOL_GPL(amvdec_remove_ts);
 
 static void dst_buf_done(struct amvdec_session *sess,
 			 struct vb2_v4l2_buffer *vbuf,
-			 u32 field,
-			 u64 timestamp)
+			 u32 field, u64 timestamp,
+			 struct v4l2_timecode timecode, u32 flags)
 {
 	struct device *dev = sess->core->dev_dec;
 	u32 output_size = amvdec_get_output_size(sess);
@@ -271,19 +261,23 @@ static void dst_buf_done(struct amvdec_session *sess,
 
 	vbuf->vb2_buf.timestamp = timestamp;
 	vbuf->sequence = sess->sequence_cap++;
+	vbuf->flags = flags;
+	vbuf->timecode = timecode;
 
 	if (sess->should_stop &&
-	    atomic_read(&sess->esparser_queued_bufs) <= 2) {
+	    atomic_read(&sess->esparser_queued_bufs) <= 1) {
 		const struct v4l2_event ev = { .type = V4L2_EVENT_EOS };
 
-		dev_dbg(dev, "Signaling EOS\n");
+		dev_dbg(dev, "Signaling EOS, sequence_cap = %u\n",
+			sess->sequence_cap - 1);
 		v4l2_event_queue_fh(&sess->fh, &ev);
 		vbuf->flags |= V4L2_BUF_FLAG_LAST;
 	} else if (sess->should_stop)
 		dev_dbg(dev, "should_stop, %u bufs remain\n",
 			atomic_read(&sess->esparser_queued_bufs));
 
-	dev_dbg(dev, "Buffer %u done\n", vbuf->vb2_buf.index);
+	dev_dbg(dev, "Buffer %u done, ts = %llu, flags = %08X\n",
+		vbuf->vb2_buf.index, timestamp, flags);
 	vbuf->field = field;
 	v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
 
@@ -297,7 +291,9 @@ void amvdec_dst_buf_done(struct amvdec_session *sess,
 	struct device *dev = sess->core->dev_dec;
 	struct amvdec_timestamp *tmp;
 	struct list_head *timestamps = &sess->timestamps;
+	struct v4l2_timecode timecode;
 	u64 timestamp;
+	u32 vbuf_flags;
 	unsigned long flags;
 
 	spin_lock_irqsave(&sess->ts_spinlock, flags);
@@ -312,11 +308,13 @@ void amvdec_dst_buf_done(struct amvdec_session *sess,
 
 	tmp = list_first_entry(timestamps, struct amvdec_timestamp, list);
 	timestamp = tmp->ts;
+	timecode = tmp->tc;
+	vbuf_flags = tmp->flags;
 	list_del(&tmp->list);
 	kfree(tmp);
 	spin_unlock_irqrestore(&sess->ts_spinlock, flags);
 
-	dst_buf_done(sess, vbuf, field, timestamp);
+	dst_buf_done(sess, vbuf, field, timestamp, timecode, vbuf_flags);
 	atomic_dec(&sess->esparser_queued_bufs);
 }
 EXPORT_SYMBOL_GPL(amvdec_dst_buf_done);
@@ -328,48 +326,43 @@ void amvdec_dst_buf_done_offset(struct amvdec_session *sess,
 	struct device *dev = sess->core->dev_dec;
 	struct amvdec_timestamp *match = NULL;
 	struct amvdec_timestamp *tmp, *n;
+	struct v4l2_timecode timecode = { 0 };
 	u64 timestamp = 0;
+	u32 vbuf_flags = 0;
 	unsigned long flags;
 
 	spin_lock_irqsave(&sess->ts_spinlock, flags);
 
 	/* Look for our vififo offset to get the corresponding timestamp. */
 	list_for_each_entry_safe(tmp, n, &sess->timestamps, list) {
-		s64 delta = (s64)offset - tmp->offset;
-
-		/* Offsets reported by codecs usually differ slightly,
-		 * so we need some wiggle room.
-		 * 4KiB being the minimum packet size, there is no risk here.
-		 */
-		if (delta > (-1 * (s32)SZ_4K) && delta < SZ_4K) {
-			match = tmp;
+		if (tmp->offset > offset) {
+			/*
+			 * Delete any record that remained unused for 32 match
+			 * checks
+			 */
+			if (tmp->used_count++ >= 32) {
+				list_del(&tmp->list);
+				kfree(tmp);
+			}
 			break;
 		}
 
-		if (!allow_drop)
-			continue;
-
-		/* Delete any timestamp entry that appears before our target
-		 * (not all src packets/timestamps lead to a frame)
-		 */
-		if (delta > 0 || delta < -1 * (s32)sess->vififo_size) {
-			atomic_dec(&sess->esparser_queued_bufs);
-			list_del(&tmp->list);
-			kfree(tmp);
-		}
+		match = tmp;
 	}
 
 	if (!match) {
-		dev_dbg(dev, "Buffer %u done but can't match offset (%08X)\n",
+		dev_err(dev, "Buffer %u done but can't match offset (%08X)\n",
 			vbuf->vb2_buf.index, offset);
 	} else {
 		timestamp = match->ts;
+		timecode = match->tc;
+		vbuf_flags = match->flags;
 		list_del(&match->list);
 		kfree(match);
 	}
 	spin_unlock_irqrestore(&sess->ts_spinlock, flags);
 
-	dst_buf_done(sess, vbuf, field, timestamp);
+	dst_buf_done(sess, vbuf, field, timestamp, timecode, vbuf_flags);
 	if (match)
 		atomic_dec(&sess->esparser_queued_bufs);
 }
@@ -420,7 +413,8 @@ void amvdec_src_change(struct amvdec_session *sess, u32 width,
 
 	v4l2_ctrl_s_ctrl(sess->ctrl_min_buf_capture, dpb_size);
 
-	/* Check if the capture queue is already configured well for our
+	/*
+	 * Check if the capture queue is already configured well for our
 	 * usecase. If so, keep decoding with it and do not send the event
 	 */
 	if (sess->width == width &&
@@ -430,6 +424,7 @@ void amvdec_src_change(struct amvdec_session *sess, u32 width,
 		return;
 	}
 
+	sess->changed_format = 0;
 	sess->width = width;
 	sess->height = height;
 	sess->status = STATUS_NEEDS_RESUME;
diff --git a/drivers/staging/media/meson/vdec/vdec_helpers.h b/drivers/staging/media/meson/vdec/vdec_helpers.h
index a455a9ee1cc2..165e6293ffba 100644
--- a/drivers/staging/media/meson/vdec/vdec_helpers.h
+++ b/drivers/staging/media/meson/vdec/vdec_helpers.h
@@ -44,13 +44,15 @@ void amvdec_dst_buf_done_offset(struct amvdec_session *sess,
 				u32 offset, u32 field, bool allow_drop);
 
 /**
- * amvdec_add_ts_reorder() - Add a timestamp to the list in chronological order
+ * amvdec_add_ts() - Add a timestamp to the list
  *
  * @sess: current session
  * @ts: timestamp to add
  * @offset: offset in the VIFIFO where the associated packet was written
+ * @flags the vb2_v4l2_buffer flags
  */
-void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset);
+void amvdec_add_ts(struct amvdec_session *sess, u64 ts,
+		   struct v4l2_timecode tc, u32 offset, u32 flags);
 void amvdec_remove_ts(struct amvdec_session *sess, u64 ts);
 
 /**
diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
index 824dbc7f46f5..accad8f8929a 100644
--- a/drivers/staging/media/meson/vdec/vdec_platform.c
+++ b/drivers/staging/media/meson/vdec/vdec_platform.c
@@ -21,6 +21,7 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
 		.codec_ops = &codec_mpeg12_ops,
 		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
 		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED,
 	}, {
 		.pixfmt = V4L2_PIX_FMT_MPEG2,
 		.min_buffers = 8,
@@ -31,6 +32,7 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
 		.codec_ops = &codec_mpeg12_ops,
 		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
 		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED,
 	},
 };
 
@@ -45,6 +47,7 @@ static const struct amvdec_format vdec_formats_gxl[] = {
 		.codec_ops = &codec_mpeg12_ops,
 		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
 		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED,
 	}, {
 		.pixfmt = V4L2_PIX_FMT_MPEG2,
 		.min_buffers = 8,
@@ -55,6 +58,7 @@ static const struct amvdec_format vdec_formats_gxl[] = {
 		.codec_ops = &codec_mpeg12_ops,
 		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
 		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED,
 	},
 };
 
@@ -69,6 +73,7 @@ static const struct amvdec_format vdec_formats_gxm[] = {
 		.codec_ops = &codec_mpeg12_ops,
 		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
 		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED,
 	}, {
 		.pixfmt = V4L2_PIX_FMT_MPEG2,
 		.min_buffers = 8,
@@ -79,6 +84,7 @@ static const struct amvdec_format vdec_formats_gxm[] = {
 		.codec_ops = &codec_mpeg12_ops,
 		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
 		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED,
 	},
 };
 
-- 
2.23.0

Re: [PATCH 1/2] media: meson: vdec: bring up to compliance

[Hans Verkuil]

On 10/7/19 4:59 PM, Maxime Jourdan wrote:
> Add all the necessary bits to pass v4l2-compliance in stateful decoding
> mode.
> 
> Mostly includes tracking the state of the decoder, allowing the OUTPUT
> queue to stream while the CAPTURE queue is inactive, handling resolution
> change events, draining with V4L2_DEC_CMD_STOP, copying more metadata
> from the src buffers to the dst buffers, etc.
> 
> Signed-off-by: Maxime Jourdan <mjourdan@baylibre.com>
> ---
>  drivers/staging/media/meson/vdec/esparser.c   | 34 ++------
>  drivers/staging/media/meson/vdec/vdec.c       | 70 ++++++++++-----
>  drivers/staging/media/meson/vdec/vdec.h       | 14 ++-
>  .../staging/media/meson/vdec/vdec_helpers.c   | 85 +++++++++----------
>  .../staging/media/meson/vdec/vdec_helpers.h   |  6 +-
>  .../staging/media/meson/vdec/vdec_platform.c  |  6 ++
>  6 files changed, 120 insertions(+), 95 deletions(-)
> 
> diff --git a/drivers/staging/media/meson/vdec/esparser.c b/drivers/staging/media/meson/vdec/esparser.c
> index 95102a4bdc62..a083d67be405 100644
> --- a/drivers/staging/media/meson/vdec/esparser.c
> +++ b/drivers/staging/media/meson/vdec/esparser.c
> @@ -180,29 +180,25 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf)
>  	int ret;
>  	struct vb2_buffer *vb = &vbuf->vb2_buf;
>  	struct amvdec_core *core = sess->core;
> -	struct amvdec_codec_ops *codec_ops = sess->fmt_out->codec_ops;
> -	u32 num_dst_bufs = 0;
>  	u32 payload_size = vb2_get_plane_payload(vb, 0);
>  	dma_addr_t phy = vb2_dma_contig_plane_dma_addr(vb, 0);
>  	u32 offset;
>  	u32 pad_size;
>  
> -	if (codec_ops->num_pending_bufs)
> -		num_dst_bufs = codec_ops->num_pending_bufs(sess);
> -
> -	num_dst_bufs += v4l2_m2m_num_dst_bufs_ready(sess->m2m_ctx);
> -
> -	if (esparser_vififo_get_free_space(sess) < payload_size ||
> -	    atomic_read(&sess->esparser_queued_bufs) >= num_dst_bufs)
> +	if (esparser_vififo_get_free_space(sess) < payload_size)
>  		return -EAGAIN;
>  
>  	v4l2_m2m_src_buf_remove_by_buf(sess->m2m_ctx, vbuf);
>  
>  	offset = esparser_get_offset(sess);
>  
> -	amvdec_add_ts_reorder(sess, vb->timestamp, offset);
> -	dev_dbg(core->dev, "esparser: ts = %llu pld_size = %u offset = %08X\n",
> -		vb->timestamp, payload_size, offset);
> +	amvdec_add_ts(sess, vb->timestamp, vbuf->timecode, offset, vbuf->flags);
> +	dev_dbg(core->dev, "esparser: ts = %llu pld_size = %u offset = %08X flags = %08X\n",
> +		vb->timestamp, payload_size, offset, vbuf->flags);
> +
> +	vbuf->flags = 0;
> +	vbuf->field = V4L2_FIELD_NONE;
> +	vbuf->sequence = sess->sequence_out++;
>  
>  	pad_size = esparser_pad_start_code(vb);
>  	ret = esparser_write_data(core, phy, payload_size + pad_size);
> @@ -216,19 +212,7 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf)
>  		return 0;
>  	}
>  
> -	/* We need to wait until we parse the first keyframe.
> -	 * All buffers prior to the first keyframe must be dropped.
> -	 */
> -	if (!sess->keyframe_found)
> -		usleep_range(1000, 2000);
> -
> -	if (sess->keyframe_found)
> -		atomic_inc(&sess->esparser_queued_bufs);
> -	else
> -		amvdec_remove_ts(sess, vb->timestamp);
> -
> -	vbuf->flags = 0;
> -	vbuf->field = V4L2_FIELD_NONE;
> +	atomic_inc(&sess->esparser_queued_bufs);
>  	v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
>  
>  	return 0;
> diff --git a/drivers/staging/media/meson/vdec/vdec.c b/drivers/staging/media/meson/vdec/vdec.c
> index 0a1a04fd5d13..0b571b3a1e33 100644
> --- a/drivers/staging/media/meson/vdec/vdec.c
> +++ b/drivers/staging/media/meson/vdec/vdec.c
> @@ -166,7 +166,10 @@ static void process_num_buffers(struct vb2_queue *q,
>  {
>  	const struct amvdec_format *fmt_out = sess->fmt_out;
>  	unsigned int buffers_total = q->num_buffers + *num_buffers;
> +	u32 min_buf_capture = v4l2_ctrl_g_ctrl(sess->ctrl_min_buf_capture);
>  
> +	if (q->num_buffers + *num_buffers < min_buf_capture)
> +		*num_buffers = min_buf_capture - q->num_buffers;
>  	if (is_reqbufs && buffers_total < fmt_out->min_buffers)
>  		*num_buffers = fmt_out->min_buffers - q->num_buffers;
>  	if (buffers_total > fmt_out->max_buffers)
> @@ -191,7 +194,8 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>  	if (*num_planes) {
>  		switch (q->type) {
>  		case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> -			if (*num_planes != 1 || sizes[0] < output_size)
> +			if (*num_planes != 1 ||
> +			    sizes[0] < sess->src_buffer_size)
>  				return -EINVAL;
>  			break;
>  		case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> @@ -222,7 +226,7 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>  
>  	switch (q->type) {
>  	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> -		sizes[0] = amvdec_get_output_size(sess);
> +		sizes[0] = sess->src_buffer_size;
>  		*num_planes = 1;
>  		break;
>  	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> @@ -248,6 +252,7 @@ static int vdec_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
>  		return -EINVAL;
>  	}
>  
> +	sess->changed_format = 1;
>  	return 0;
>  }
>  
> @@ -259,10 +264,11 @@ static void vdec_vb2_buf_queue(struct vb2_buffer *vb)
>  
>  	v4l2_m2m_buf_queue(m2m_ctx, vbuf);
>  
> -	if (!sess->streamon_out || !sess->streamon_cap)
> +	if (!sess->streamon_out)
>  		return;
>  
> -	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	if (sess->streamon_cap &&
> +	    vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
>  	    vdec_codec_needs_recycle(sess))
>  		vdec_queue_recycle(sess, vb);
>  
> @@ -287,16 +293,22 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
>  	else
>  		sess->streamon_cap = 1;
>  
> -	if (!sess->streamon_out || !sess->streamon_cap)
> +	if (!sess->streamon_out)
>  		return 0;
>  
>  	if (sess->status == STATUS_NEEDS_RESUME &&
> -	    q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> +	    q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE &&
> +	    sess->changed_format) {
>  		codec_ops->resume(sess);
>  		sess->status = STATUS_RUNNING;
>  		return 0;
>  	}
>  
> +	if (sess->status == STATUS_RUNNING ||
> +	    sess->status == STATUS_NEEDS_RESUME ||
> +	    sess->status == STATUS_INIT)
> +		return 0;
> +
>  	sess->vififo_size = SIZE_VIFIFO;
>  	sess->vififo_vaddr =
>  		dma_alloc_coherent(sess->core->dev, sess->vififo_size,
> @@ -321,13 +333,14 @@ static int vdec_start_streaming(struct vb2_queue *q, unsigned int count)
>  		goto vififo_free;
>  
>  	sess->sequence_cap = 0;
> +	sess->sequence_out = 0;
>  	if (vdec_codec_needs_recycle(sess))
>  		sess->recycle_thread = kthread_run(vdec_recycle_thread, sess,
>  						   "vdec_recycle");
>  
> -	sess->status = STATUS_RUNNING;
> +	sess->status = STATUS_INIT;
>  	core->cur_sess = sess;
> -
> +	schedule_work(&sess->esparser_queue_work);
>  	return 0;
>  
>  vififo_free:
> @@ -384,6 +397,7 @@ static void vdec_stop_streaming(struct vb2_queue *q)
>  	struct vb2_v4l2_buffer *buf;
>  
>  	if (sess->status == STATUS_RUNNING ||
> +	    sess->status == STATUS_INIT ||
>  	    (sess->status == STATUS_NEEDS_RESUME &&
>  	     (!sess->streamon_out || !sess->streamon_cap))) {
>  		if (vdec_codec_needs_recycle(sess))
> @@ -474,20 +488,33 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,
>  	struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp;
>  	struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt;
>  	const struct amvdec_format *fmts = sess->core->platform->formats;
> -	const struct amvdec_format *fmt_out;
> +	const struct amvdec_format *fmt_out = NULL;
> +	u32 output_size = 0;
>  
>  	memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved));
>  	memset(pixmp->reserved, 0, sizeof(pixmp->reserved));
>  
> -	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +	switch (f->type) {
> +	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>  		fmt_out = find_format(fmts, size, pixmp->pixelformat);
>  		if (!fmt_out) {
>  			pixmp->pixelformat = V4L2_PIX_FMT_MPEG2;
>  			fmt_out = find_format(fmts, size, pixmp->pixelformat);
>  		}
> +		break;
> +	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> +		fmt_out = sess->fmt_out;
> +		break;
> +	default:
> +		return NULL;
> +	}
> +
> +	pixmp->width  = clamp(pixmp->width,  (u32)256, fmt_out->max_width);
> +	pixmp->height = clamp(pixmp->height, (u32)144, fmt_out->max_height);
> +	output_size = get_output_size(pixmp->width, pixmp->height);

output_size is never used!

drivers/staging/media/meson/vdec/vdec.c: In function ‘vdec_try_fmt_common’:
drivers/staging/media/meson/vdec/vdec.c:492:6: warning: variable ‘output_size’ set but not used [-Wunused-but-set-variable]
  492 |  u32 output_size = 0;
      |      ^~~~~~~~~~~

Regards,

	Hans

>  
> -		pfmt[0].sizeimage =
> -			get_output_size(pixmp->width, pixmp->height);
> +	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
> +		pfmt[0].sizeimage = sess->src_buffer_size;
>  		pfmt[0].bytesperline = 0;
>  		pixmp->num_planes = 1;
>  	} else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> @@ -519,8 +546,6 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,
>  			pfmt[2].bytesperline = ALIGN(pixmp->width, 64) / 2;
>  			pixmp->num_planes = 3;
>  		}
> -	} else {
> -		return NULL;
>  	}
>  
>  	pixmp->width  = clamp(pixmp->width,  (u32)256, fmt_out->max_width);
> @@ -584,6 +609,8 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
>  	orig_pixmp = *pixmp;
>  
>  	fmt_out = vdec_try_fmt_common(sess, num_formats, f);
> +	if (!fmt_out)
> +		return -EINVAL;
>  
>  	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) {
>  		pixfmt_out = pixmp->pixelformat;
> @@ -608,6 +635,7 @@ static int vdec_s_fmt(struct file *file, void *fh, struct v4l2_format *f)
>  		sess->ycbcr_enc = pixmp->ycbcr_enc;
>  		sess->quantization = pixmp->quantization;
>  		sess->xfer_func = pixmp->xfer_func;
> +		sess->src_buffer_size = pixmp->plane_fmt[0].sizeimage;
>  	}
>  
>  	memset(&format, 0, sizeof(format));
> @@ -699,29 +727,28 @@ vdec_decoder_cmd(struct file *file, void *fh, struct v4l2_decoder_cmd *cmd)
>  	if (!(sess->streamon_out & sess->streamon_cap))
>  		return 0;
>  
> -	/* Currently not handled since we do not support dynamic resolution
> -	 * for MPEG2. We consider both queues streaming to mean that the
> -	 * decoding session is started
> -	 */
> -	if (cmd->cmd == V4L2_DEC_CMD_START)
> +	if (cmd->cmd == V4L2_DEC_CMD_START) {
> +		sess->should_stop = 0;
>  		return 0;
> +	}
>  
>  	/* Should not happen */
>  	if (cmd->cmd != V4L2_DEC_CMD_STOP)
>  		return -EINVAL;
>  
>  	dev_dbg(dev, "Received V4L2_DEC_CMD_STOP\n");
> -	sess->should_stop = 1;
>  
> -	vdec_wait_inactive(sess);
> +	sess->should_stop = 1;
>  
>  	if (codec_ops->drain) {
> +		vdec_wait_inactive(sess);
>  		codec_ops->drain(sess);
>  	} else if (codec_ops->eos_sequence) {
>  		u32 len;
>  		const u8 *data = codec_ops->eos_sequence(&len);
>  
>  		esparser_queue_eos(sess->core, data, len);
> +		vdec_wait_inactive(sess);
>  	}
>  
>  	return ret;
> @@ -881,6 +908,7 @@ static int vdec_open(struct file *file)
>  	sess->height = 720;
>  	sess->pixelaspect.numerator = 1;
>  	sess->pixelaspect.denominator = 1;
> +	sess->src_buffer_size = SZ_1M;
>  
>  	INIT_LIST_HEAD(&sess->timestamps);
>  	INIT_LIST_HEAD(&sess->bufs_recycle);
> diff --git a/drivers/staging/media/meson/vdec/vdec.h b/drivers/staging/media/meson/vdec/vdec.h
> index d811e7976519..163d6dddfad6 100644
> --- a/drivers/staging/media/meson/vdec/vdec.h
> +++ b/drivers/staging/media/meson/vdec/vdec.h
> @@ -29,13 +29,19 @@ struct amvdec_buffer {
>   * struct amvdec_timestamp - stores a src timestamp along with a VIFIFO offset
>   *
>   * @list: used to make lists out of this struct
> - * @ts: timestamp
> + * @tc: timecode from the v4l2 buffer
> + * @ts: timestamp from the VB2 buffer
>   * @offset: offset in the VIFIFO where the associated packet was written
> + * @flags: flags from the v4l2 buffer
> + * @used_count: times this timestamp was checked for a match with a dst buffer
>   */
>  struct amvdec_timestamp {
>  	struct list_head list;
> +	struct v4l2_timecode tc;
>  	u64 ts;
>  	u32 offset;
> +	u32 flags;
> +	u32 used_count;
>  };
>  
>  struct amvdec_session;
> @@ -164,6 +170,7 @@ struct amvdec_format {
>  
>  enum amvdec_status {
>  	STATUS_STOPPED,
> +	STATUS_INIT,
>  	STATUS_RUNNING,
>  	STATUS_NEEDS_RESUME,
>  };
> @@ -179,6 +186,7 @@ enum amvdec_status {
>   * @ctrl_min_buf_capture: V4L2 control V4L2_CID_MIN_BUFFERS_FOR_CAPTURE
>   * @fmt_out: vdec pixel format for the OUTPUT queue
>   * @pixfmt_cap: V4L2 pixel format for the CAPTURE queue
> + * @src_buffer_size: size in bytes of the OUTPUT buffers' only plane
>   * @width: current picture width
>   * @height: current picture height
>   * @colorspace: current colorspace
> @@ -220,6 +228,7 @@ struct amvdec_session {
>  
>  	const struct amvdec_format *fmt_out;
>  	u32 pixfmt_cap;
> +	u32 src_buffer_size;
>  
>  	u32 width;
>  	u32 height;
> @@ -234,10 +243,11 @@ struct amvdec_session {
>  	struct work_struct esparser_queue_work;
>  
>  	unsigned int streamon_cap, streamon_out;
> -	unsigned int sequence_cap;
> +	unsigned int sequence_cap, sequence_out;
>  	unsigned int should_stop;
>  	unsigned int keyframe_found;
>  	unsigned int num_dst_bufs;
> +	unsigned int changed_format;
>  
>  	u8 canvas_alloc[MAX_CANVAS];
>  	u32 canvas_num;
> diff --git a/drivers/staging/media/meson/vdec/vdec_helpers.c b/drivers/staging/media/meson/vdec/vdec_helpers.c
> index f16948bdbf2f..ff4333074197 100644
> --- a/drivers/staging/media/meson/vdec/vdec_helpers.c
> +++ b/drivers/staging/media/meson/vdec/vdec_helpers.c
> @@ -200,33 +200,23 @@ int amvdec_set_canvases(struct amvdec_session *sess,
>  }
>  EXPORT_SYMBOL_GPL(amvdec_set_canvases);
>  
> -void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset)
> +void amvdec_add_ts(struct amvdec_session *sess, u64 ts,
> +		   struct v4l2_timecode tc, u32 offset, u32 vbuf_flags)
>  {
> -	struct amvdec_timestamp *new_ts, *tmp;
> +	struct amvdec_timestamp *new_ts;
>  	unsigned long flags;
>  
> -	new_ts = kmalloc(sizeof(*new_ts), GFP_KERNEL);
> +	new_ts = kzalloc(sizeof(*new_ts), GFP_KERNEL);
>  	new_ts->ts = ts;
> +	new_ts->tc = tc;
>  	new_ts->offset = offset;
> +	new_ts->flags = vbuf_flags;
>  
>  	spin_lock_irqsave(&sess->ts_spinlock, flags);
> -
> -	if (list_empty(&sess->timestamps))
> -		goto add_tail;
> -
> -	list_for_each_entry(tmp, &sess->timestamps, list) {
> -		if (ts <= tmp->ts) {
> -			list_add_tail(&new_ts->list, &tmp->list);
> -			goto unlock;
> -		}
> -	}
> -
> -add_tail:
>  	list_add_tail(&new_ts->list, &sess->timestamps);
> -unlock:
>  	spin_unlock_irqrestore(&sess->ts_spinlock, flags);
>  }
> -EXPORT_SYMBOL_GPL(amvdec_add_ts_reorder);
> +EXPORT_SYMBOL_GPL(amvdec_add_ts);
>  
>  void amvdec_remove_ts(struct amvdec_session *sess, u64 ts)
>  {
> @@ -251,8 +241,8 @@ EXPORT_SYMBOL_GPL(amvdec_remove_ts);
>  
>  static void dst_buf_done(struct amvdec_session *sess,
>  			 struct vb2_v4l2_buffer *vbuf,
> -			 u32 field,
> -			 u64 timestamp)
> +			 u32 field, u64 timestamp,
> +			 struct v4l2_timecode timecode, u32 flags)
>  {
>  	struct device *dev = sess->core->dev_dec;
>  	u32 output_size = amvdec_get_output_size(sess);
> @@ -271,19 +261,23 @@ static void dst_buf_done(struct amvdec_session *sess,
>  
>  	vbuf->vb2_buf.timestamp = timestamp;
>  	vbuf->sequence = sess->sequence_cap++;
> +	vbuf->flags = flags;
> +	vbuf->timecode = timecode;
>  
>  	if (sess->should_stop &&
> -	    atomic_read(&sess->esparser_queued_bufs) <= 2) {
> +	    atomic_read(&sess->esparser_queued_bufs) <= 1) {
>  		const struct v4l2_event ev = { .type = V4L2_EVENT_EOS };
>  
> -		dev_dbg(dev, "Signaling EOS\n");
> +		dev_dbg(dev, "Signaling EOS, sequence_cap = %u\n",
> +			sess->sequence_cap - 1);
>  		v4l2_event_queue_fh(&sess->fh, &ev);
>  		vbuf->flags |= V4L2_BUF_FLAG_LAST;
>  	} else if (sess->should_stop)
>  		dev_dbg(dev, "should_stop, %u bufs remain\n",
>  			atomic_read(&sess->esparser_queued_bufs));
>  
> -	dev_dbg(dev, "Buffer %u done\n", vbuf->vb2_buf.index);
> +	dev_dbg(dev, "Buffer %u done, ts = %llu, flags = %08X\n",
> +		vbuf->vb2_buf.index, timestamp, flags);
>  	vbuf->field = field;
>  	v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE);
>  
> @@ -297,7 +291,9 @@ void amvdec_dst_buf_done(struct amvdec_session *sess,
>  	struct device *dev = sess->core->dev_dec;
>  	struct amvdec_timestamp *tmp;
>  	struct list_head *timestamps = &sess->timestamps;
> +	struct v4l2_timecode timecode;
>  	u64 timestamp;
> +	u32 vbuf_flags;
>  	unsigned long flags;
>  
>  	spin_lock_irqsave(&sess->ts_spinlock, flags);
> @@ -312,11 +308,13 @@ void amvdec_dst_buf_done(struct amvdec_session *sess,
>  
>  	tmp = list_first_entry(timestamps, struct amvdec_timestamp, list);
>  	timestamp = tmp->ts;
> +	timecode = tmp->tc;
> +	vbuf_flags = tmp->flags;
>  	list_del(&tmp->list);
>  	kfree(tmp);
>  	spin_unlock_irqrestore(&sess->ts_spinlock, flags);
>  
> -	dst_buf_done(sess, vbuf, field, timestamp);
> +	dst_buf_done(sess, vbuf, field, timestamp, timecode, vbuf_flags);
>  	atomic_dec(&sess->esparser_queued_bufs);
>  }
>  EXPORT_SYMBOL_GPL(amvdec_dst_buf_done);
> @@ -328,48 +326,43 @@ void amvdec_dst_buf_done_offset(struct amvdec_session *sess,
>  	struct device *dev = sess->core->dev_dec;
>  	struct amvdec_timestamp *match = NULL;
>  	struct amvdec_timestamp *tmp, *n;
> +	struct v4l2_timecode timecode = { 0 };
>  	u64 timestamp = 0;
> +	u32 vbuf_flags = 0;
>  	unsigned long flags;
>  
>  	spin_lock_irqsave(&sess->ts_spinlock, flags);
>  
>  	/* Look for our vififo offset to get the corresponding timestamp. */
>  	list_for_each_entry_safe(tmp, n, &sess->timestamps, list) {
> -		s64 delta = (s64)offset - tmp->offset;
> -
> -		/* Offsets reported by codecs usually differ slightly,
> -		 * so we need some wiggle room.
> -		 * 4KiB being the minimum packet size, there is no risk here.
> -		 */
> -		if (delta > (-1 * (s32)SZ_4K) && delta < SZ_4K) {
> -			match = tmp;
> +		if (tmp->offset > offset) {
> +			/*
> +			 * Delete any record that remained unused for 32 match
> +			 * checks
> +			 */
> +			if (tmp->used_count++ >= 32) {
> +				list_del(&tmp->list);
> +				kfree(tmp);
> +			}
>  			break;
>  		}
>  
> -		if (!allow_drop)
> -			continue;
> -
> -		/* Delete any timestamp entry that appears before our target
> -		 * (not all src packets/timestamps lead to a frame)
> -		 */
> -		if (delta > 0 || delta < -1 * (s32)sess->vififo_size) {
> -			atomic_dec(&sess->esparser_queued_bufs);
> -			list_del(&tmp->list);
> -			kfree(tmp);
> -		}
> +		match = tmp;
>  	}
>  
>  	if (!match) {
> -		dev_dbg(dev, "Buffer %u done but can't match offset (%08X)\n",
> +		dev_err(dev, "Buffer %u done but can't match offset (%08X)\n",
>  			vbuf->vb2_buf.index, offset);
>  	} else {
>  		timestamp = match->ts;
> +		timecode = match->tc;
> +		vbuf_flags = match->flags;
>  		list_del(&match->list);
>  		kfree(match);
>  	}
>  	spin_unlock_irqrestore(&sess->ts_spinlock, flags);
>  
> -	dst_buf_done(sess, vbuf, field, timestamp);
> +	dst_buf_done(sess, vbuf, field, timestamp, timecode, vbuf_flags);
>  	if (match)
>  		atomic_dec(&sess->esparser_queued_bufs);
>  }
> @@ -420,7 +413,8 @@ void amvdec_src_change(struct amvdec_session *sess, u32 width,
>  
>  	v4l2_ctrl_s_ctrl(sess->ctrl_min_buf_capture, dpb_size);
>  
> -	/* Check if the capture queue is already configured well for our
> +	/*
> +	 * Check if the capture queue is already configured well for our
>  	 * usecase. If so, keep decoding with it and do not send the event
>  	 */
>  	if (sess->width == width &&
> @@ -430,6 +424,7 @@ void amvdec_src_change(struct amvdec_session *sess, u32 width,
>  		return;
>  	}
>  
> +	sess->changed_format = 0;
>  	sess->width = width;
>  	sess->height = height;
>  	sess->status = STATUS_NEEDS_RESUME;
> diff --git a/drivers/staging/media/meson/vdec/vdec_helpers.h b/drivers/staging/media/meson/vdec/vdec_helpers.h
> index a455a9ee1cc2..165e6293ffba 100644
> --- a/drivers/staging/media/meson/vdec/vdec_helpers.h
> +++ b/drivers/staging/media/meson/vdec/vdec_helpers.h
> @@ -44,13 +44,15 @@ void amvdec_dst_buf_done_offset(struct amvdec_session *sess,
>  				u32 offset, u32 field, bool allow_drop);
>  
>  /**
> - * amvdec_add_ts_reorder() - Add a timestamp to the list in chronological order
> + * amvdec_add_ts() - Add a timestamp to the list
>   *
>   * @sess: current session
>   * @ts: timestamp to add
>   * @offset: offset in the VIFIFO where the associated packet was written
> + * @flags the vb2_v4l2_buffer flags
>   */
> -void amvdec_add_ts_reorder(struct amvdec_session *sess, u64 ts, u32 offset);
> +void amvdec_add_ts(struct amvdec_session *sess, u64 ts,
> +		   struct v4l2_timecode tc, u32 offset, u32 flags);
>  void amvdec_remove_ts(struct amvdec_session *sess, u64 ts);
>  
>  /**
> diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
> index 824dbc7f46f5..accad8f8929a 100644
> --- a/drivers/staging/media/meson/vdec/vdec_platform.c
> +++ b/drivers/staging/media/meson/vdec/vdec_platform.c
> @@ -21,6 +21,7 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
>  		.codec_ops = &codec_mpeg12_ops,
>  		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
>  		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED,
>  	}, {
>  		.pixfmt = V4L2_PIX_FMT_MPEG2,
>  		.min_buffers = 8,
> @@ -31,6 +32,7 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
>  		.codec_ops = &codec_mpeg12_ops,
>  		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
>  		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED,
>  	},
>  };
>  
> @@ -45,6 +47,7 @@ static const struct amvdec_format vdec_formats_gxl[] = {
>  		.codec_ops = &codec_mpeg12_ops,
>  		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
>  		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED,
>  	}, {
>  		.pixfmt = V4L2_PIX_FMT_MPEG2,
>  		.min_buffers = 8,
> @@ -55,6 +58,7 @@ static const struct amvdec_format vdec_formats_gxl[] = {
>  		.codec_ops = &codec_mpeg12_ops,
>  		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
>  		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED,
>  	},
>  };
>  
> @@ -69,6 +73,7 @@ static const struct amvdec_format vdec_formats_gxm[] = {
>  		.codec_ops = &codec_mpeg12_ops,
>  		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
>  		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED,
>  	}, {
>  		.pixfmt = V4L2_PIX_FMT_MPEG2,
>  		.min_buffers = 8,
> @@ -79,6 +84,7 @@ static const struct amvdec_format vdec_formats_gxm[] = {
>  		.codec_ops = &codec_mpeg12_ops,
>  		.firmware_path = "meson/vdec/gxl_mpeg12.bin",
>  		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, V4L2_PIX_FMT_YUV420M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED,
>  	},
>  };
>  
> 

[PATCH 2/2] media: meson: vdec: add H.264 decoding support

[Maxime Jourdan]

Add support for the H264 compressed format (V4L2_PIX_FMT_H264).

Signed-off-by: Maxime Jourdan <mjourdan@baylibre.com>
---
 drivers/staging/media/meson/vdec/Makefile     |   2 +-
 drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
 drivers/staging/media/meson/vdec/codec_h264.h |  14 +
 .../staging/media/meson/vdec/vdec_platform.c  |  37 ++
 4 files changed, 534 insertions(+), 1 deletion(-)
 create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
 create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h

diff --git a/drivers/staging/media/meson/vdec/Makefile b/drivers/staging/media/meson/vdec/Makefile
index 6bea129084b7..711d990c760e 100644
--- a/drivers/staging/media/meson/vdec/Makefile
+++ b/drivers/staging/media/meson/vdec/Makefile
@@ -3,6 +3,6 @@
 
 meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o
 meson-vdec-objs += vdec_1.o
-meson-vdec-objs += codec_mpeg12.o
+meson-vdec-objs += codec_mpeg12.o codec_h264.o
 
 obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
diff --git a/drivers/staging/media/meson/vdec/codec_h264.c b/drivers/staging/media/meson/vdec/codec_h264.c
new file mode 100644
index 000000000000..4528a6a01c3d
--- /dev/null
+++ b/drivers/staging/media/meson/vdec/codec_h264.c
@@ -0,0 +1,482 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Copyright (C) 2019 BayLibre, SAS
+ * Author: Maxime Jourdan <mjourdan@baylibre.com>
+ */
+
+#include <media/v4l2-mem2mem.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "vdec_helpers.h"
+#include "dos_regs.h"
+
+#define SIZE_EXT_FW	(20 * SZ_1K)
+#define SIZE_WORKSPACE	0x1ee000
+#define SIZE_SEI	(8 * SZ_1K)
+
+/*
+ * Offset added by the firmware which must be substracted
+ * from the workspace phyaddr
+ */
+#define WORKSPACE_BUF_OFFSET	0x1000000
+
+/* ISR status */
+#define CMD_MASK		GENMASK(7, 0)
+#define CMD_SRC_CHANGE		1
+#define CMD_FRAMES_READY	2
+#define CMD_FATAL_ERROR		6
+#define CMD_BAD_WIDTH		7
+#define CMD_BAD_HEIGHT		8
+
+#define SEI_DATA_READY	BIT(15)
+
+/* Picture type */
+#define PIC_TOP_BOT	5
+#define PIC_BOT_TOP	6
+
+/* Size of Motion Vector per macroblock */
+#define MB_MV_SIZE	96
+
+/* Frame status data */
+#define PIC_STRUCT_BIT	5
+#define PIC_STRUCT_MASK	GENMASK(2, 0)
+#define BUF_IDX_MASK	GENMASK(4, 0)
+#define ERROR_FLAG	BIT(9)
+#define OFFSET_BIT	16
+#define OFFSET_MASK	GENMASK(15, 0)
+
+/* Bitstream parsed data */
+#define MB_TOTAL_BIT	8
+#define MB_TOTAL_MASK	GENMASK(15, 0)
+#define MB_WIDTH_MASK	GENMASK(7, 0)
+#define MAX_REF_BIT	24
+#define MAX_REF_MASK	GENMASK(6, 0)
+#define AR_IDC_BIT	16
+#define AR_IDC_MASK	GENMASK(7, 0)
+#define AR_PRESENT_FLAG	BIT(0)
+#define AR_EXTEND	0xff
+
+/*
+ * Buffer to send to the ESPARSER to signal End Of Stream for H.264.
+ * This is a 16x16 encoded picture that will trigger drain firmware-side.
+ * There is no known alternative.
+ */
+static const u8 eos_sequence[SZ_4K] = {
+	0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 0xff, 0xe4, 0xdc, 0x45, 0xe9, 0xbd,
+	0xe6, 0xd9, 0x48, 0xb7,	0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef,
+	0x78, 0x32, 0x36, 0x34, 0x20, 0x2d, 0x20, 0x63,	0x6f, 0x72, 0x65, 0x20,
+	0x36, 0x37, 0x20, 0x72, 0x31, 0x31, 0x33, 0x30, 0x20, 0x38, 0x34, 0x37,
+	0x35, 0x39, 0x37, 0x37, 0x20, 0x2d, 0x20, 0x48, 0x2e, 0x32, 0x36, 0x34,
+	0x2f, 0x4d, 0x50, 0x45,	0x47, 0x2d, 0x34, 0x20, 0x41, 0x56, 0x43, 0x20,
+	0x63, 0x6f, 0x64, 0x65, 0x63, 0x20, 0x2d, 0x20,	0x43, 0x6f, 0x70, 0x79,
+	0x6c, 0x65, 0x66, 0x74, 0x20, 0x32, 0x30, 0x30, 0x33, 0x2d, 0x32, 0x30,
+	0x30, 0x39, 0x20, 0x2d, 0x20, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
+	0x77, 0x77, 0x77, 0x2e,	0x76, 0x69, 0x64, 0x65, 0x6f, 0x6c, 0x61, 0x6e,
+	0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x78, 0x32, 0x36,	0x34, 0x2e, 0x68, 0x74,
+	0x6d, 0x6c, 0x20, 0x2d, 0x20, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73,
+	0x3a, 0x20, 0x63, 0x61, 0x62, 0x61, 0x63, 0x3d, 0x31, 0x20, 0x72, 0x65,
+	0x66, 0x3d, 0x31, 0x20,	0x64, 0x65, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x3d,
+	0x31, 0x3a, 0x30, 0x3a, 0x30, 0x20, 0x61, 0x6e,	0x61, 0x6c, 0x79, 0x73,
+	0x65, 0x3d, 0x30, 0x78, 0x31, 0x3a, 0x30, 0x78, 0x31, 0x31, 0x31, 0x20,
+	0x6d, 0x65, 0x3d, 0x68, 0x65, 0x78, 0x20, 0x73, 0x75, 0x62, 0x6d, 0x65,
+	0x3d, 0x36, 0x20, 0x70,	0x73, 0x79, 0x5f, 0x72, 0x64, 0x3d, 0x31, 0x2e,
+	0x30, 0x3a, 0x30, 0x2e, 0x30, 0x20, 0x6d, 0x69,	0x78, 0x65, 0x64, 0x5f,
+	0x72, 0x65, 0x66, 0x3d, 0x30, 0x20, 0x6d, 0x65, 0x5f, 0x72, 0x61, 0x6e,
+	0x67, 0x65, 0x3d, 0x31, 0x36, 0x20, 0x63, 0x68, 0x72, 0x6f, 0x6d, 0x61,
+	0x5f, 0x6d, 0x65, 0x3d,	0x31, 0x20, 0x74, 0x72, 0x65, 0x6c, 0x6c, 0x69,
+	0x73, 0x3d, 0x30, 0x20, 0x38, 0x78, 0x38, 0x64,	0x63, 0x74, 0x3d, 0x30,
+	0x20, 0x63, 0x71, 0x6d, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x61, 0x64, 0x7a,
+	0x6f, 0x6e, 0x65, 0x3d, 0x32, 0x31, 0x2c, 0x31, 0x31, 0x20, 0x63, 0x68,
+	0x72, 0x6f, 0x6d, 0x61,	0x5f, 0x71, 0x70, 0x5f, 0x6f, 0x66, 0x66, 0x73,
+	0x65, 0x74, 0x3d, 0x2d, 0x32, 0x20, 0x74, 0x68,	0x72, 0x65, 0x61, 0x64,
+	0x73, 0x3d, 0x31, 0x20, 0x6e, 0x72, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x63,
+	0x69, 0x6d, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x20, 0x6d, 0x62, 0x61, 0x66,
+	0x66, 0x3d, 0x30, 0x20,	0x62, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x3d,
+	0x30, 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74,	0x3d, 0x32, 0x35, 0x30,
+	0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x5f, 0x6d, 0x69, 0x6e, 0x3d,
+	0x32, 0x35, 0x20, 0x73, 0x63, 0x65, 0x6e, 0x65, 0x63, 0x75, 0x74, 0x3d,
+	0x34, 0x30, 0x20, 0x72,	0x63, 0x3d, 0x61, 0x62, 0x72, 0x20, 0x62, 0x69,
+	0x74, 0x72, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x30,	0x20, 0x72, 0x61, 0x74,
+	0x65, 0x74, 0x6f, 0x6c, 0x3d, 0x31, 0x2e, 0x30, 0x20, 0x71, 0x63, 0x6f,
+	0x6d, 0x70, 0x3d, 0x30, 0x2e, 0x36, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x69,
+	0x6e, 0x3d, 0x31, 0x30,	0x20, 0x71, 0x70, 0x6d, 0x61, 0x78, 0x3d, 0x35,
+	0x31, 0x20, 0x71, 0x70, 0x73, 0x74, 0x65, 0x70,	0x3d, 0x34, 0x20, 0x69,
+	0x70, 0x5f, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x3d, 0x31, 0x2e, 0x34, 0x30,
+	0x20, 0x61, 0x71, 0x3d, 0x31, 0x3a, 0x31, 0x2e, 0x30, 0x30, 0x00, 0x80,
+	0x00, 0x00, 0x00, 0x01,	0x67, 0x4d, 0x40, 0x0a, 0x9a, 0x74, 0xf4, 0x20,
+	0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x06,	0x51, 0xe2, 0x44, 0xd4,
+	0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x32, 0xc8, 0x00, 0x00, 0x00, 0x01,
+	0x65, 0x88, 0x80, 0x20, 0x00, 0x08, 0x7f, 0xea, 0x6a, 0xe2, 0x99, 0xb6,
+	0x57, 0xae, 0x49, 0x30,	0xf5, 0xfe, 0x5e, 0x46, 0x0b, 0x72, 0x44, 0xc4,
+	0xe1, 0xfc, 0x62, 0xda, 0xf1, 0xfb, 0xa2, 0xdb,	0xd6, 0xbe, 0x5c, 0xd7,
+	0x24, 0xa3, 0xf5, 0xb9, 0x2f, 0x57, 0x16, 0x49, 0x75, 0x47, 0x77, 0x09,
+	0x5c, 0xa1, 0xb4, 0xc3, 0x4f, 0x60, 0x2b, 0xb0, 0x0c, 0xc8, 0xd6, 0x66,
+	0xba, 0x9b, 0x82, 0x29,	0x33, 0x92, 0x26, 0x99, 0x31, 0x1c, 0x7f, 0x9b,
+	0x00, 0x00, 0x01, 0x0ff,
+};
+
+static const u8 *codec_h264_eos_sequence(u32 *len)
+{
+	*len = ARRAY_SIZE(eos_sequence);
+	return eos_sequence;
+}
+
+struct codec_h264 {
+	/* H.264 decoder requires an extended firmware */
+	void      *ext_fw_vaddr;
+	dma_addr_t ext_fw_paddr;
+
+	/* Buffer for the H.264 Workspace */
+	void      *workspace_vaddr;
+	dma_addr_t workspace_paddr;
+
+	/* Buffer for the H.264 references MV */
+	void      *ref_vaddr;
+	dma_addr_t ref_paddr;
+	u32	   ref_size;
+
+	/* Buffer for parsed SEI data */
+	void      *sei_vaddr;
+	dma_addr_t sei_paddr;
+
+	u32 mb_width;
+	u32 mb_height;
+	u32 max_refs;
+};
+
+static int codec_h264_can_recycle(struct amvdec_core *core)
+{
+	return !amvdec_read_dos(core, AV_SCRATCH_7) ||
+	       !amvdec_read_dos(core, AV_SCRATCH_8);
+}
+
+static void codec_h264_recycle(struct amvdec_core *core, u32 buf_idx)
+{
+	/*
+	 * Tell the firmware it can recycle this buffer.
+	 * AV_SCRATCH_8 serves the same purpose.
+	 */
+	if (!amvdec_read_dos(core, AV_SCRATCH_7))
+		amvdec_write_dos(core, AV_SCRATCH_7, buf_idx + 1);
+	else
+		amvdec_write_dos(core, AV_SCRATCH_8, buf_idx + 1);
+}
+
+static int codec_h264_start(struct amvdec_session *sess)
+{
+	u32 workspace_offset;
+	struct amvdec_core *core = sess->core;
+	struct codec_h264 *h264 = sess->priv;
+
+	/* Allocate some memory for the H.264 decoder's state */
+	h264->workspace_vaddr =
+		dma_alloc_coherent(core->dev, SIZE_WORKSPACE,
+				   &h264->workspace_paddr, GFP_KERNEL);
+	if (!h264->workspace_vaddr)
+		return -ENOMEM;
+
+	/* Allocate some memory for the H.264 SEI dump */
+	h264->sei_vaddr = dma_alloc_coherent(core->dev, SIZE_SEI,
+					     &h264->sei_paddr, GFP_KERNEL);
+	if (!h264->sei_vaddr)
+		return -ENOMEM;
+
+	amvdec_write_dos_bits(core, POWER_CTL_VLD, BIT(9) | BIT(6));
+
+	workspace_offset = h264->workspace_paddr - WORKSPACE_BUF_OFFSET;
+	amvdec_write_dos(core, AV_SCRATCH_1, workspace_offset);
+	amvdec_write_dos(core, AV_SCRATCH_G, h264->ext_fw_paddr);
+	amvdec_write_dos(core, AV_SCRATCH_I, h264->sei_paddr -
+					     workspace_offset);
+
+	/* Enable "error correction" */
+	amvdec_write_dos(core, AV_SCRATCH_F,
+			 (amvdec_read_dos(core, AV_SCRATCH_F) & 0xffffffc3) |
+			 BIT(4) | BIT(7));
+
+	amvdec_write_dos(core, MDEC_PIC_DC_THRESH, 0x404038aa);
+
+	return 0;
+}
+
+static int codec_h264_stop(struct amvdec_session *sess)
+{
+	struct codec_h264 *h264 = sess->priv;
+	struct amvdec_core *core = sess->core;
+
+	if (h264->ext_fw_vaddr)
+		dma_free_coherent(core->dev, SIZE_EXT_FW,
+				  h264->ext_fw_vaddr, h264->ext_fw_paddr);
+
+	if (h264->workspace_vaddr)
+		dma_free_coherent(core->dev, SIZE_WORKSPACE,
+				 h264->workspace_vaddr, h264->workspace_paddr);
+
+	if (h264->ref_vaddr)
+		dma_free_coherent(core->dev, h264->ref_size,
+				  h264->ref_vaddr, h264->ref_paddr);
+
+	if (h264->sei_vaddr)
+		dma_free_coherent(core->dev, SIZE_SEI,
+				  h264->sei_vaddr, h264->sei_paddr);
+
+	return 0;
+}
+
+static int codec_h264_load_extended_firmware(struct amvdec_session *sess,
+					     const u8 *data, u32 len)
+{
+	struct codec_h264 *h264;
+	struct amvdec_core *core = sess->core;
+
+	if (len < SIZE_EXT_FW)
+		return -EINVAL;
+
+	h264 = kzalloc(sizeof(*h264), GFP_KERNEL);
+	if (!h264)
+		return -ENOMEM;
+
+	h264->ext_fw_vaddr = dma_alloc_coherent(core->dev, SIZE_EXT_FW,
+					      &h264->ext_fw_paddr, GFP_KERNEL);
+	if (!h264->ext_fw_vaddr) {
+		kfree(h264);
+		return -ENOMEM;
+	}
+
+	memcpy(h264->ext_fw_vaddr, data, SIZE_EXT_FW);
+	sess->priv = h264;
+
+	return 0;
+}
+
+static const struct v4l2_fract par_table[] = {
+	{ 1, 1 },   { 1, 1 },    { 12, 11 }, { 10, 11 },
+	{ 16, 11 }, { 40, 33 },  { 24, 11 }, { 20, 11 },
+	{ 32, 11 }, { 80, 33 },  { 18, 11 }, { 15, 11 },
+	{ 64, 33 }, { 160, 99 }, { 4, 3 },   { 3, 2 },
+	{ 2, 1 }
+};
+
+static void codec_h264_set_par(struct amvdec_session *sess)
+{
+	struct amvdec_core *core = sess->core;
+	u32 seq_info = amvdec_read_dos(core, AV_SCRATCH_2);
+	u32 ar_idc = (seq_info >> AR_IDC_BIT) & AR_IDC_MASK;
+
+	if (!(seq_info & AR_PRESENT_FLAG))
+		return;
+
+	if (ar_idc == AR_EXTEND) {
+		u32 ar_info = amvdec_read_dos(core, AV_SCRATCH_3);
+
+		sess->pixelaspect.numerator = ar_info & 0xffff;
+		sess->pixelaspect.denominator = (ar_info >> 16) & 0xffff;
+		return;
+	}
+
+	if (ar_idc >= ARRAY_SIZE(par_table))
+		return;
+
+	sess->pixelaspect = par_table[ar_idc];
+}
+
+static void codec_h264_resume(struct amvdec_session *sess)
+{
+	struct amvdec_core *core = sess->core;
+	struct codec_h264 *h264 = sess->priv;
+	u32 mb_width, mb_height, mb_total;
+
+	amvdec_set_canvases(sess, (u32[]){ ANC0_CANVAS_ADDR, 0 },
+				  (u32[]){ 24, 0 });
+
+	dev_dbg(core->dev, "max_refs = %u; actual_dpb_size = %u\n",
+		h264->max_refs, sess->num_dst_bufs);
+
+	/* Align to a multiple of 4 macroblocks */
+	mb_width = ALIGN(h264->mb_width, 4);
+	mb_height = ALIGN(h264->mb_height, 4);
+	mb_total = mb_width * mb_height;
+
+	h264->ref_size = mb_total * MB_MV_SIZE * h264->max_refs;
+	h264->ref_vaddr = dma_alloc_coherent(core->dev, h264->ref_size,
+					     &h264->ref_paddr, GFP_KERNEL);
+	if (!h264->ref_vaddr) {
+		amvdec_abort(sess);
+		return;
+	}
+
+	/* Address to store the references' MVs */
+	amvdec_write_dos(core, AV_SCRATCH_1, h264->ref_paddr);
+	/* End of ref MV */
+	amvdec_write_dos(core, AV_SCRATCH_4, h264->ref_paddr + h264->ref_size);
+
+	amvdec_write_dos(core, AV_SCRATCH_0, (h264->max_refs << 24) |
+					     (sess->num_dst_bufs << 16) |
+					     ((h264->max_refs - 1) << 8));
+}
+
+/**
+ * Configure the H.264 decoder when the parser detected a parameter set change
+ */
+static void codec_h264_src_change(struct amvdec_session *sess)
+{
+	struct amvdec_core *core = sess->core;
+	struct codec_h264 *h264 = sess->priv;
+	u32 parsed_info, mb_total;
+	u32 crop_infor, crop_bottom, crop_right;
+	u32 frame_width, frame_height;
+
+	sess->keyframe_found = 1;
+
+	parsed_info = amvdec_read_dos(core, AV_SCRATCH_1);
+
+	/* Total number of 16x16 macroblocks */
+	mb_total = (parsed_info >> MB_TOTAL_BIT) & MB_TOTAL_MASK;
+	/* Number of macroblocks per line */
+	h264->mb_width = parsed_info & MB_WIDTH_MASK;
+	/* Number of macroblock lines */
+	h264->mb_height = mb_total / h264->mb_width;
+
+	h264->max_refs = ((parsed_info >> MAX_REF_BIT) & MAX_REF_MASK) + 1;
+
+	crop_infor = amvdec_read_dos(core, AV_SCRATCH_6);
+	crop_bottom = (crop_infor & 0xff);
+	crop_right = (crop_infor >> 16) & 0xff;
+
+	frame_width = h264->mb_width * 16 - crop_right;
+	frame_height = h264->mb_height * 16 - crop_bottom;
+
+	dev_dbg(core->dev, "frame: %ux%u; crop: %u %u\n",
+		frame_width, frame_height, crop_right, crop_bottom);
+
+	codec_h264_set_par(sess);
+	amvdec_src_change(sess, frame_width, frame_height, h264->max_refs + 5);
+}
+
+/**
+ * The bitstream offset is split in half in 2 different registers.
+ * Fetch its MSB here, which location depends on the frame number.
+ */
+static u32 get_offset_msb(struct amvdec_core *core, int frame_num)
+{
+	int take_msb = frame_num % 2;
+	int reg_offset = (frame_num / 2) * 4;
+	u32 offset_msb = amvdec_read_dos(core, AV_SCRATCH_A + reg_offset);
+
+	if (take_msb)
+		return offset_msb & 0xffff0000;
+
+	return (offset_msb & 0x0000ffff) << 16;
+}
+
+static void codec_h264_frames_ready(struct amvdec_session *sess, u32 status)
+{
+	struct amvdec_core *core = sess->core;
+	int error_count;
+	int num_frames;
+	int i;
+
+	error_count = amvdec_read_dos(core, AV_SCRATCH_D);
+	num_frames = (status >> 8) & 0xff;
+	if (error_count) {
+		dev_warn(core->dev,
+			 "decoder error(s) happened, count %d\n", error_count);
+		amvdec_write_dos(core, AV_SCRATCH_D, 0);
+	}
+
+	for (i = 0; i < num_frames; i++) {
+		u32 frame_status = amvdec_read_dos(core, AV_SCRATCH_1 + i * 4);
+		u32 buffer_index = frame_status & BUF_IDX_MASK;
+		u32 pic_struct = (frame_status >> PIC_STRUCT_BIT) &
+				 PIC_STRUCT_MASK;
+		u32 offset = (frame_status >> OFFSET_BIT) & OFFSET_MASK;
+		u32 field = V4L2_FIELD_NONE;
+
+		/*
+		 * A buffer decode error means it was decoded,
+		 * but part of the picture will have artifacts.
+		 * Typical reason is a temporarily corrupted bitstream
+		 */
+		if (frame_status & ERROR_FLAG)
+			dev_dbg(core->dev, "Buffer %d decode error\n",
+				buffer_index);
+
+		if (pic_struct == PIC_TOP_BOT)
+			field = V4L2_FIELD_INTERLACED_TB;
+		else if (pic_struct == PIC_BOT_TOP)
+			field = V4L2_FIELD_INTERLACED_BT;
+
+		offset |= get_offset_msb(core, i);
+		amvdec_dst_buf_done_idx(sess, buffer_index, offset, field);
+	}
+}
+
+static irqreturn_t codec_h264_threaded_isr(struct amvdec_session *sess)
+{
+	struct amvdec_core *core = sess->core;
+	u32 status;
+	u32 size;
+	u8 cmd;
+
+	status = amvdec_read_dos(core, AV_SCRATCH_0);
+	cmd = status & CMD_MASK;
+
+	switch (cmd) {
+	case CMD_SRC_CHANGE:
+		codec_h264_src_change(sess);
+		break;
+	case CMD_FRAMES_READY:
+		codec_h264_frames_ready(sess, status);
+		break;
+	case CMD_FATAL_ERROR:
+		dev_err(core->dev, "H.264 decoder fatal error\n");
+		goto abort;
+	case CMD_BAD_WIDTH:
+		size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
+		dev_err(core->dev, "Unsupported video width: %u\n", size);
+		goto abort;
+	case CMD_BAD_HEIGHT:
+		size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
+		dev_err(core->dev, "Unsupported video height: %u\n", size);
+		goto abort;
+	case 0: /* Unused but not worth printing for */
+	case 9:
+		break;
+	default:
+		dev_info(core->dev, "Unexpected H264 ISR: %08X\n", cmd);
+		break;
+	}
+
+	if (cmd && cmd != CMD_SRC_CHANGE)
+		amvdec_write_dos(core, AV_SCRATCH_0, 0);
+
+	/* Decoder has some SEI data for us ; ignore */
+	if (amvdec_read_dos(core, AV_SCRATCH_J) & SEI_DATA_READY)
+		amvdec_write_dos(core, AV_SCRATCH_J, 0);
+
+	return IRQ_HANDLED;
+abort:
+	amvdec_abort(sess);
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t codec_h264_isr(struct amvdec_session *sess)
+{
+	struct amvdec_core *core = sess->core;
+
+	amvdec_write_dos(core, ASSIST_MBOX1_CLR_REG, 1);
+
+	return IRQ_WAKE_THREAD;
+}
+
+struct amvdec_codec_ops codec_h264_ops = {
+	.start = codec_h264_start,
+	.stop = codec_h264_stop,
+	.load_extended_firmware = codec_h264_load_extended_firmware,
+	.isr = codec_h264_isr,
+	.threaded_isr = codec_h264_threaded_isr,
+	.can_recycle = codec_h264_can_recycle,
+	.recycle = codec_h264_recycle,
+	.eos_sequence = codec_h264_eos_sequence,
+	.resume = codec_h264_resume,
+};
diff --git a/drivers/staging/media/meson/vdec/codec_h264.h b/drivers/staging/media/meson/vdec/codec_h264.h
new file mode 100644
index 000000000000..7cb4fb86ff36
--- /dev/null
+++ b/drivers/staging/media/meson/vdec/codec_h264.h
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * Copyright (C) 2019 BayLibre, SAS
+ * Author: Maxime Jourdan <mjourdan@baylibre.com>
+ */
+
+#ifndef __MESON_VDEC_CODEC_H264_H_
+#define __MESON_VDEC_CODEC_H264_H_
+
+#include "vdec.h"
+
+extern struct amvdec_codec_ops codec_h264_ops;
+
+#endif
diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
index accad8f8929a..025371ff8fae 100644
--- a/drivers/staging/media/meson/vdec/vdec_platform.c
+++ b/drivers/staging/media/meson/vdec/vdec_platform.c
@@ -9,9 +9,22 @@
 
 #include "vdec_1.h"
 #include "codec_mpeg12.h"
+#include "codec_h264.h"
 
 static const struct amvdec_format vdec_formats_gxbb[] = {
 	{
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.min_buffers = 2,
+		.max_buffers = 24,
+		.max_width = 1920,
+		.max_height = 1080,
+		.vdec_ops = &vdec_1_ops,
+		.codec_ops = &codec_h264_ops,
+		.firmware_path = "meson/vdec/gxbb_h264.bin",
+		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED |
+			 V4L2_FMT_FLAG_DYN_RESOLUTION,
+	}, {
 		.pixfmt = V4L2_PIX_FMT_MPEG1,
 		.min_buffers = 8,
 		.max_buffers = 8,
@@ -38,6 +51,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
 
 static const struct amvdec_format vdec_formats_gxl[] = {
 	{
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.min_buffers = 2,
+		.max_buffers = 24,
+		.max_width = 3840,
+		.max_height = 2160,
+		.vdec_ops = &vdec_1_ops,
+		.codec_ops = &codec_h264_ops,
+		.firmware_path = "meson/vdec/gxl_h264.bin",
+		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED |
+			 V4L2_FMT_FLAG_DYN_RESOLUTION,
+	}, {
 		.pixfmt = V4L2_PIX_FMT_MPEG1,
 		.min_buffers = 8,
 		.max_buffers = 8,
@@ -64,6 +89,18 @@ static const struct amvdec_format vdec_formats_gxl[] = {
 
 static const struct amvdec_format vdec_formats_gxm[] = {
 	{
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.min_buffers = 2,
+		.max_buffers = 24,
+		.max_width = 3840,
+		.max_height = 2160,
+		.vdec_ops = &vdec_1_ops,
+		.codec_ops = &codec_h264_ops,
+		.firmware_path = "meson/vdec/gxm_h264.bin",
+		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
+		.flags = V4L2_FMT_FLAG_COMPRESSED |
+			 V4L2_FMT_FLAG_DYN_RESOLUTION,
+	}, {
 		.pixfmt = V4L2_PIX_FMT_MPEG1,
 		.min_buffers = 8,
 		.max_buffers = 8,
-- 
2.23.0

Re: [PATCH 2/2] media: meson: vdec: add H.264 decoding support

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 21624 bytes --]

Le lundi 07 octobre 2019 à 16:59 +0200, Maxime Jourdan a écrit :
> Add support for the H264 compressed format (V4L2_PIX_FMT_H264).
> 
> Signed-off-by: Maxime Jourdan <mjourdan@baylibre.com>
> ---
>  drivers/staging/media/meson/vdec/Makefile     |   2 +-
>  drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
>  drivers/staging/media/meson/vdec/codec_h264.h |  14 +
>  .../staging/media/meson/vdec/vdec_platform.c  |  37 ++
>  4 files changed, 534 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
>  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
> 
> diff --git a/drivers/staging/media/meson/vdec/Makefile b/drivers/staging/media/meson/vdec/Makefile
> index 6bea129084b7..711d990c760e 100644
> --- a/drivers/staging/media/meson/vdec/Makefile
> +++ b/drivers/staging/media/meson/vdec/Makefile
> @@ -3,6 +3,6 @@
>  
>  meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o
>  meson-vdec-objs += vdec_1.o
> -meson-vdec-objs += codec_mpeg12.o
> +meson-vdec-objs += codec_mpeg12.o codec_h264.o
>  
>  obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
> diff --git a/drivers/staging/media/meson/vdec/codec_h264.c b/drivers/staging/media/meson/vdec/codec_h264.c
> new file mode 100644
> index 000000000000..4528a6a01c3d
> --- /dev/null
> +++ b/drivers/staging/media/meson/vdec/codec_h264.c
> @@ -0,0 +1,482 @@
> +// SPDX-License-Identifier: GPL-2.0+
> +/*
> + * Copyright (C) 2019 BayLibre, SAS
> + * Author: Maxime Jourdan <mjourdan@baylibre.com>
> + */
> +
> +#include <media/v4l2-mem2mem.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "vdec_helpers.h"
> +#include "dos_regs.h"
> +
> +#define SIZE_EXT_FW	(20 * SZ_1K)
> +#define SIZE_WORKSPACE	0x1ee000
> +#define SIZE_SEI	(8 * SZ_1K)
> +
> +/*
> + * Offset added by the firmware which must be substracted
> + * from the workspace phyaddr
> + */
> +#define WORKSPACE_BUF_OFFSET	0x1000000
> +
> +/* ISR status */
> +#define CMD_MASK		GENMASK(7, 0)
> +#define CMD_SRC_CHANGE		1
> +#define CMD_FRAMES_READY	2
> +#define CMD_FATAL_ERROR		6
> +#define CMD_BAD_WIDTH		7
> +#define CMD_BAD_HEIGHT		8
> +
> +#define SEI_DATA_READY	BIT(15)
> +
> +/* Picture type */
> +#define PIC_TOP_BOT	5
> +#define PIC_BOT_TOP	6
> +
> +/* Size of Motion Vector per macroblock */
> +#define MB_MV_SIZE	96
> +
> +/* Frame status data */
> +#define PIC_STRUCT_BIT	5
> +#define PIC_STRUCT_MASK	GENMASK(2, 0)
> +#define BUF_IDX_MASK	GENMASK(4, 0)
> +#define ERROR_FLAG	BIT(9)
> +#define OFFSET_BIT	16
> +#define OFFSET_MASK	GENMASK(15, 0)
> +
> +/* Bitstream parsed data */
> +#define MB_TOTAL_BIT	8
> +#define MB_TOTAL_MASK	GENMASK(15, 0)
> +#define MB_WIDTH_MASK	GENMASK(7, 0)
> +#define MAX_REF_BIT	24
> +#define MAX_REF_MASK	GENMASK(6, 0)
> +#define AR_IDC_BIT	16
> +#define AR_IDC_MASK	GENMASK(7, 0)
> +#define AR_PRESENT_FLAG	BIT(0)
> +#define AR_EXTEND	0xff
> +
> +/*
> + * Buffer to send to the ESPARSER to signal End Of Stream for H.264.
> + * This is a 16x16 encoded picture that will trigger drain firmware-side.
> + * There is no known alternative.
> + */
> +static const u8 eos_sequence[SZ_4K] = {
> +	0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 0xff, 0xe4, 0xdc, 0x45, 0xe9, 0xbd,
> +	0xe6, 0xd9, 0x48, 0xb7,	0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef,
> +	0x78, 0x32, 0x36, 0x34, 0x20, 0x2d, 0x20, 0x63,	0x6f, 0x72, 0x65, 0x20,
> +	0x36, 0x37, 0x20, 0x72, 0x31, 0x31, 0x33, 0x30, 0x20, 0x38, 0x34, 0x37,
> +	0x35, 0x39, 0x37, 0x37, 0x20, 0x2d, 0x20, 0x48, 0x2e, 0x32, 0x36, 0x34,
> +	0x2f, 0x4d, 0x50, 0x45,	0x47, 0x2d, 0x34, 0x20, 0x41, 0x56, 0x43, 0x20,
> +	0x63, 0x6f, 0x64, 0x65, 0x63, 0x20, 0x2d, 0x20,	0x43, 0x6f, 0x70, 0x79,
> +	0x6c, 0x65, 0x66, 0x74, 0x20, 0x32, 0x30, 0x30, 0x33, 0x2d, 0x32, 0x30,
> +	0x30, 0x39, 0x20, 0x2d, 0x20, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
> +	0x77, 0x77, 0x77, 0x2e,	0x76, 0x69, 0x64, 0x65, 0x6f, 0x6c, 0x61, 0x6e,
> +	0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x78, 0x32, 0x36,	0x34, 0x2e, 0x68, 0x74,
> +	0x6d, 0x6c, 0x20, 0x2d, 0x20, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73,
> +	0x3a, 0x20, 0x63, 0x61, 0x62, 0x61, 0x63, 0x3d, 0x31, 0x20, 0x72, 0x65,
> +	0x66, 0x3d, 0x31, 0x20,	0x64, 0x65, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x3d,
> +	0x31, 0x3a, 0x30, 0x3a, 0x30, 0x20, 0x61, 0x6e,	0x61, 0x6c, 0x79, 0x73,
> +	0x65, 0x3d, 0x30, 0x78, 0x31, 0x3a, 0x30, 0x78, 0x31, 0x31, 0x31, 0x20,
> +	0x6d, 0x65, 0x3d, 0x68, 0x65, 0x78, 0x20, 0x73, 0x75, 0x62, 0x6d, 0x65,
> +	0x3d, 0x36, 0x20, 0x70,	0x73, 0x79, 0x5f, 0x72, 0x64, 0x3d, 0x31, 0x2e,
> +	0x30, 0x3a, 0x30, 0x2e, 0x30, 0x20, 0x6d, 0x69,	0x78, 0x65, 0x64, 0x5f,
> +	0x72, 0x65, 0x66, 0x3d, 0x30, 0x20, 0x6d, 0x65, 0x5f, 0x72, 0x61, 0x6e,
> +	0x67, 0x65, 0x3d, 0x31, 0x36, 0x20, 0x63, 0x68, 0x72, 0x6f, 0x6d, 0x61,
> +	0x5f, 0x6d, 0x65, 0x3d,	0x31, 0x20, 0x74, 0x72, 0x65, 0x6c, 0x6c, 0x69,
> +	0x73, 0x3d, 0x30, 0x20, 0x38, 0x78, 0x38, 0x64,	0x63, 0x74, 0x3d, 0x30,
> +	0x20, 0x63, 0x71, 0x6d, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x61, 0x64, 0x7a,
> +	0x6f, 0x6e, 0x65, 0x3d, 0x32, 0x31, 0x2c, 0x31, 0x31, 0x20, 0x63, 0x68,
> +	0x72, 0x6f, 0x6d, 0x61,	0x5f, 0x71, 0x70, 0x5f, 0x6f, 0x66, 0x66, 0x73,
> +	0x65, 0x74, 0x3d, 0x2d, 0x32, 0x20, 0x74, 0x68,	0x72, 0x65, 0x61, 0x64,
> +	0x73, 0x3d, 0x31, 0x20, 0x6e, 0x72, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x63,
> +	0x69, 0x6d, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x20, 0x6d, 0x62, 0x61, 0x66,
> +	0x66, 0x3d, 0x30, 0x20,	0x62, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x3d,
> +	0x30, 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74,	0x3d, 0x32, 0x35, 0x30,
> +	0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x5f, 0x6d, 0x69, 0x6e, 0x3d,
> +	0x32, 0x35, 0x20, 0x73, 0x63, 0x65, 0x6e, 0x65, 0x63, 0x75, 0x74, 0x3d,
> +	0x34, 0x30, 0x20, 0x72,	0x63, 0x3d, 0x61, 0x62, 0x72, 0x20, 0x62, 0x69,
> +	0x74, 0x72, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x30,	0x20, 0x72, 0x61, 0x74,
> +	0x65, 0x74, 0x6f, 0x6c, 0x3d, 0x31, 0x2e, 0x30, 0x20, 0x71, 0x63, 0x6f,
> +	0x6d, 0x70, 0x3d, 0x30, 0x2e, 0x36, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x69,
> +	0x6e, 0x3d, 0x31, 0x30,	0x20, 0x71, 0x70, 0x6d, 0x61, 0x78, 0x3d, 0x35,
> +	0x31, 0x20, 0x71, 0x70, 0x73, 0x74, 0x65, 0x70,	0x3d, 0x34, 0x20, 0x69,
> +	0x70, 0x5f, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x3d, 0x31, 0x2e, 0x34, 0x30,
> +	0x20, 0x61, 0x71, 0x3d, 0x31, 0x3a, 0x31, 0x2e, 0x30, 0x30, 0x00, 0x80,
> +	0x00, 0x00, 0x00, 0x01,	0x67, 0x4d, 0x40, 0x0a, 0x9a, 0x74, 0xf4, 0x20,
> +	0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x06,	0x51, 0xe2, 0x44, 0xd4,
> +	0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x32, 0xc8, 0x00, 0x00, 0x00, 0x01,
> +	0x65, 0x88, 0x80, 0x20, 0x00, 0x08, 0x7f, 0xea, 0x6a, 0xe2, 0x99, 0xb6,
> +	0x57, 0xae, 0x49, 0x30,	0xf5, 0xfe, 0x5e, 0x46, 0x0b, 0x72, 0x44, 0xc4,
> +	0xe1, 0xfc, 0x62, 0xda, 0xf1, 0xfb, 0xa2, 0xdb,	0xd6, 0xbe, 0x5c, 0xd7,
> +	0x24, 0xa3, 0xf5, 0xb9, 0x2f, 0x57, 0x16, 0x49, 0x75, 0x47, 0x77, 0x09,
> +	0x5c, 0xa1, 0xb4, 0xc3, 0x4f, 0x60, 0x2b, 0xb0, 0x0c, 0xc8, 0xd6, 0x66,
> +	0xba, 0x9b, 0x82, 0x29,	0x33, 0x92, 0x26, 0x99, 0x31, 0x1c, 0x7f, 0x9b,
> +	0x00, 0x00, 0x01, 0x0ff,
> +};

I can read that this contains a pretty large SEI, SPS, PPS, IDR and
finally this unspecified Nal 15 with no size, but I don't expect every
can. Maybe you could format this into nals, and document whatever you
know about this blob ? If you need info, I may suggest you do put that
into a file and use YUView software to parse it.

> +
> +static const u8 *codec_h264_eos_sequence(u32 *len)
> +{
> +	*len = ARRAY_SIZE(eos_sequence);
> +	return eos_sequence;
> +}
> +
> +struct codec_h264 {
> +	/* H.264 decoder requires an extended firmware */
> +	void      *ext_fw_vaddr;
> +	dma_addr_t ext_fw_paddr;
> +
> +	/* Buffer for the H.264 Workspace */
> +	void      *workspace_vaddr;
> +	dma_addr_t workspace_paddr;
> +
> +	/* Buffer for the H.264 references MV */
> +	void      *ref_vaddr;
> +	dma_addr_t ref_paddr;
> +	u32	   ref_size;
> +
> +	/* Buffer for parsed SEI data */
> +	void      *sei_vaddr;
> +	dma_addr_t sei_paddr;
> +
> +	u32 mb_width;
> +	u32 mb_height;
> +	u32 max_refs;
> +};
> +
> +static int codec_h264_can_recycle(struct amvdec_core *core)
> +{
> +	return !amvdec_read_dos(core, AV_SCRATCH_7) ||
> +	       !amvdec_read_dos(core, AV_SCRATCH_8);
> +}
> +
> +static void codec_h264_recycle(struct amvdec_core *core, u32 buf_idx)
> +{
> +	/*
> +	 * Tell the firmware it can recycle this buffer.
> +	 * AV_SCRATCH_8 serves the same purpose.
> +	 */
> +	if (!amvdec_read_dos(core, AV_SCRATCH_7))
> +		amvdec_write_dos(core, AV_SCRATCH_7, buf_idx + 1);
> +	else
> +		amvdec_write_dos(core, AV_SCRATCH_8, buf_idx + 1);
> +}
> +
> +static int codec_h264_start(struct amvdec_session *sess)
> +{
> +	u32 workspace_offset;
> +	struct amvdec_core *core = sess->core;
> +	struct codec_h264 *h264 = sess->priv;
> +
> +	/* Allocate some memory for the H.264 decoder's state */
> +	h264->workspace_vaddr =
> +		dma_alloc_coherent(core->dev, SIZE_WORKSPACE,
> +				   &h264->workspace_paddr, GFP_KERNEL);
> +	if (!h264->workspace_vaddr)
> +		return -ENOMEM;
> +
> +	/* Allocate some memory for the H.264 SEI dump */
> +	h264->sei_vaddr = dma_alloc_coherent(core->dev, SIZE_SEI,
> +					     &h264->sei_paddr, GFP_KERNEL);
> +	if (!h264->sei_vaddr)
> +		return -ENOMEM;
> +
> +	amvdec_write_dos_bits(core, POWER_CTL_VLD, BIT(9) | BIT(6));
> +
> +	workspace_offset = h264->workspace_paddr - WORKSPACE_BUF_OFFSET;
> +	amvdec_write_dos(core, AV_SCRATCH_1, workspace_offset);
> +	amvdec_write_dos(core, AV_SCRATCH_G, h264->ext_fw_paddr);
> +	amvdec_write_dos(core, AV_SCRATCH_I, h264->sei_paddr -
> +					     workspace_offset);
> +
> +	/* Enable "error correction" */
> +	amvdec_write_dos(core, AV_SCRATCH_F,
> +			 (amvdec_read_dos(core, AV_SCRATCH_F) & 0xffffffc3) |
> +			 BIT(4) | BIT(7));
> +
> +	amvdec_write_dos(core, MDEC_PIC_DC_THRESH, 0x404038aa);
> +
> +	return 0;
> +}
> +
> +static int codec_h264_stop(struct amvdec_session *sess)
> +{
> +	struct codec_h264 *h264 = sess->priv;
> +	struct amvdec_core *core = sess->core;
> +
> +	if (h264->ext_fw_vaddr)
> +		dma_free_coherent(core->dev, SIZE_EXT_FW,
> +				  h264->ext_fw_vaddr, h264->ext_fw_paddr);
> +
> +	if (h264->workspace_vaddr)
> +		dma_free_coherent(core->dev, SIZE_WORKSPACE,
> +				 h264->workspace_vaddr, h264->workspace_paddr);
> +
> +	if (h264->ref_vaddr)
> +		dma_free_coherent(core->dev, h264->ref_size,
> +				  h264->ref_vaddr, h264->ref_paddr);
> +
> +	if (h264->sei_vaddr)
> +		dma_free_coherent(core->dev, SIZE_SEI,
> +				  h264->sei_vaddr, h264->sei_paddr);
> +
> +	return 0;
> +}
> +
> +static int codec_h264_load_extended_firmware(struct amvdec_session *sess,
> +					     const u8 *data, u32 len)
> +{
> +	struct codec_h264 *h264;
> +	struct amvdec_core *core = sess->core;
> +
> +	if (len < SIZE_EXT_FW)
> +		return -EINVAL;
> +
> +	h264 = kzalloc(sizeof(*h264), GFP_KERNEL);
> +	if (!h264)
> +		return -ENOMEM;
> +
> +	h264->ext_fw_vaddr = dma_alloc_coherent(core->dev, SIZE_EXT_FW,
> +					      &h264->ext_fw_paddr, GFP_KERNEL);
> +	if (!h264->ext_fw_vaddr) {
> +		kfree(h264);
> +		return -ENOMEM;
> +	}
> +
> +	memcpy(h264->ext_fw_vaddr, data, SIZE_EXT_FW);
> +	sess->priv = h264;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_fract par_table[] = {
> +	{ 1, 1 },   { 1, 1 },    { 12, 11 }, { 10, 11 },
> +	{ 16, 11 }, { 40, 33 },  { 24, 11 }, { 20, 11 },
> +	{ 32, 11 }, { 80, 33 },  { 18, 11 }, { 15, 11 },
> +	{ 64, 33 }, { 160, 99 }, { 4, 3 },   { 3, 2 },
> +	{ 2, 1 }
> +};
> +
> +static void codec_h264_set_par(struct amvdec_session *sess)
> +{
> +	struct amvdec_core *core = sess->core;
> +	u32 seq_info = amvdec_read_dos(core, AV_SCRATCH_2);
> +	u32 ar_idc = (seq_info >> AR_IDC_BIT) & AR_IDC_MASK;
> +
> +	if (!(seq_info & AR_PRESENT_FLAG))
> +		return;
> +
> +	if (ar_idc == AR_EXTEND) {
> +		u32 ar_info = amvdec_read_dos(core, AV_SCRATCH_3);
> +
> +		sess->pixelaspect.numerator = ar_info & 0xffff;
> +		sess->pixelaspect.denominator = (ar_info >> 16) & 0xffff;
> +		return;
> +	}
> +
> +	if (ar_idc >= ARRAY_SIZE(par_table))
> +		return;
> +
> +	sess->pixelaspect = par_table[ar_idc];
> +}
> +
> +static void codec_h264_resume(struct amvdec_session *sess)
> +{
> +	struct amvdec_core *core = sess->core;
> +	struct codec_h264 *h264 = sess->priv;
> +	u32 mb_width, mb_height, mb_total;
> +
> +	amvdec_set_canvases(sess, (u32[]){ ANC0_CANVAS_ADDR, 0 },
> +				  (u32[]){ 24, 0 });
> +
> +	dev_dbg(core->dev, "max_refs = %u; actual_dpb_size = %u\n",
> +		h264->max_refs, sess->num_dst_bufs);
> +
> +	/* Align to a multiple of 4 macroblocks */
> +	mb_width = ALIGN(h264->mb_width, 4);
> +	mb_height = ALIGN(h264->mb_height, 4);
> +	mb_total = mb_width * mb_height;
> +
> +	h264->ref_size = mb_total * MB_MV_SIZE * h264->max_refs;
> +	h264->ref_vaddr = dma_alloc_coherent(core->dev, h264->ref_size,
> +					     &h264->ref_paddr, GFP_KERNEL);
> +	if (!h264->ref_vaddr) {
> +		amvdec_abort(sess);
> +		return;
> +	}
> +
> +	/* Address to store the references' MVs */
> +	amvdec_write_dos(core, AV_SCRATCH_1, h264->ref_paddr);
> +	/* End of ref MV */
> +	amvdec_write_dos(core, AV_SCRATCH_4, h264->ref_paddr + h264->ref_size);
> +
> +	amvdec_write_dos(core, AV_SCRATCH_0, (h264->max_refs << 24) |
> +					     (sess->num_dst_bufs << 16) |
> +					     ((h264->max_refs - 1) << 8));
> +}
> +
> +/**
> + * Configure the H.264 decoder when the parser detected a parameter set change
> + */
> +static void codec_h264_src_change(struct amvdec_session *sess)
> +{
> +	struct amvdec_core *core = sess->core;
> +	struct codec_h264 *h264 = sess->priv;
> +	u32 parsed_info, mb_total;
> +	u32 crop_infor, crop_bottom, crop_right;
> +	u32 frame_width, frame_height;
> +
> +	sess->keyframe_found = 1;
> +
> +	parsed_info = amvdec_read_dos(core, AV_SCRATCH_1);
> +
> +	/* Total number of 16x16 macroblocks */
> +	mb_total = (parsed_info >> MB_TOTAL_BIT) & MB_TOTAL_MASK;
> +	/* Number of macroblocks per line */
> +	h264->mb_width = parsed_info & MB_WIDTH_MASK;
> +	/* Number of macroblock lines */
> +	h264->mb_height = mb_total / h264->mb_width;
> +
> +	h264->max_refs = ((parsed_info >> MAX_REF_BIT) & MAX_REF_MASK) + 1;
> +
> +	crop_infor = amvdec_read_dos(core, AV_SCRATCH_6);
> +	crop_bottom = (crop_infor & 0xff);
> +	crop_right = (crop_infor >> 16) & 0xff;
> +
> +	frame_width = h264->mb_width * 16 - crop_right;
> +	frame_height = h264->mb_height * 16 - crop_bottom;
> +
> +	dev_dbg(core->dev, "frame: %ux%u; crop: %u %u\n",
> +		frame_width, frame_height, crop_right, crop_bottom);
> +
> +	codec_h264_set_par(sess);
> +	amvdec_src_change(sess, frame_width, frame_height, h264->max_refs + 5);
> +}
> +
> +/**
> + * The bitstream offset is split in half in 2 different registers.
> + * Fetch its MSB here, which location depends on the frame number.
> + */
> +static u32 get_offset_msb(struct amvdec_core *core, int frame_num)
> +{
> +	int take_msb = frame_num % 2;
> +	int reg_offset = (frame_num / 2) * 4;
> +	u32 offset_msb = amvdec_read_dos(core, AV_SCRATCH_A + reg_offset);
> +
> +	if (take_msb)
> +		return offset_msb & 0xffff0000;
> +
> +	return (offset_msb & 0x0000ffff) << 16;
> +}
> +
> +static void codec_h264_frames_ready(struct amvdec_session *sess, u32 status)
> +{
> +	struct amvdec_core *core = sess->core;
> +	int error_count;
> +	int num_frames;
> +	int i;
> +
> +	error_count = amvdec_read_dos(core, AV_SCRATCH_D);
> +	num_frames = (status >> 8) & 0xff;
> +	if (error_count) {
> +		dev_warn(core->dev,
> +			 "decoder error(s) happened, count %d\n", error_count);
> +		amvdec_write_dos(core, AV_SCRATCH_D, 0);
> +	}
> +
> +	for (i = 0; i < num_frames; i++) {
> +		u32 frame_status = amvdec_read_dos(core, AV_SCRATCH_1 + i * 4);
> +		u32 buffer_index = frame_status & BUF_IDX_MASK;
> +		u32 pic_struct = (frame_status >> PIC_STRUCT_BIT) &
> +				 PIC_STRUCT_MASK;
> +		u32 offset = (frame_status >> OFFSET_BIT) & OFFSET_MASK;
> +		u32 field = V4L2_FIELD_NONE;
> +
> +		/*
> +		 * A buffer decode error means it was decoded,
> +		 * but part of the picture will have artifacts.
> +		 * Typical reason is a temporarily corrupted bitstream
> +		 */
> +		if (frame_status & ERROR_FLAG)
> +			dev_dbg(core->dev, "Buffer %d decode error\n",
> +				buffer_index);
> +
> +		if (pic_struct == PIC_TOP_BOT)
> +			field = V4L2_FIELD_INTERLACED_TB;
> +		else if (pic_struct == PIC_BOT_TOP)
> +			field = V4L2_FIELD_INTERLACED_BT;
> +
> +		offset |= get_offset_msb(core, i);
> +		amvdec_dst_buf_done_idx(sess, buffer_index, offset, field);
> +	}
> +}
> +
> +static irqreturn_t codec_h264_threaded_isr(struct amvdec_session *sess)
> +{
> +	struct amvdec_core *core = sess->core;
> +	u32 status;
> +	u32 size;
> +	u8 cmd;
> +
> +	status = amvdec_read_dos(core, AV_SCRATCH_0);
> +	cmd = status & CMD_MASK;
> +
> +	switch (cmd) {
> +	case CMD_SRC_CHANGE:
> +		codec_h264_src_change(sess);
> +		break;
> +	case CMD_FRAMES_READY:
> +		codec_h264_frames_ready(sess, status);
> +		break;
> +	case CMD_FATAL_ERROR:
> +		dev_err(core->dev, "H.264 decoder fatal error\n");
> +		goto abort;
> +	case CMD_BAD_WIDTH:
> +		size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> +		dev_err(core->dev, "Unsupported video width: %u\n", size);
> +		goto abort;
> +	case CMD_BAD_HEIGHT:
> +		size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> +		dev_err(core->dev, "Unsupported video height: %u\n", size);
> +		goto abort;
> +	case 0: /* Unused but not worth printing for */
> +	case 9:
> +		break;
> +	default:
> +		dev_info(core->dev, "Unexpected H264 ISR: %08X\n", cmd);
> +		break;
> +	}
> +
> +	if (cmd && cmd != CMD_SRC_CHANGE)
> +		amvdec_write_dos(core, AV_SCRATCH_0, 0);
> +
> +	/* Decoder has some SEI data for us ; ignore */
> +	if (amvdec_read_dos(core, AV_SCRATCH_J) & SEI_DATA_READY)
> +		amvdec_write_dos(core, AV_SCRATCH_J, 0);
> +
> +	return IRQ_HANDLED;
> +abort:
> +	amvdec_abort(sess);
> +	return IRQ_HANDLED;
> +}
> +
> +static irqreturn_t codec_h264_isr(struct amvdec_session *sess)
> +{
> +	struct amvdec_core *core = sess->core;
> +
> +	amvdec_write_dos(core, ASSIST_MBOX1_CLR_REG, 1);
> +
> +	return IRQ_WAKE_THREAD;
> +}
> +
> +struct amvdec_codec_ops codec_h264_ops = {
> +	.start = codec_h264_start,
> +	.stop = codec_h264_stop,
> +	.load_extended_firmware = codec_h264_load_extended_firmware,
> +	.isr = codec_h264_isr,
> +	.threaded_isr = codec_h264_threaded_isr,
> +	.can_recycle = codec_h264_can_recycle,
> +	.recycle = codec_h264_recycle,
> +	.eos_sequence = codec_h264_eos_sequence,
> +	.resume = codec_h264_resume,
> +};
> diff --git a/drivers/staging/media/meson/vdec/codec_h264.h b/drivers/staging/media/meson/vdec/codec_h264.h
> new file mode 100644
> index 000000000000..7cb4fb86ff36
> --- /dev/null
> +++ b/drivers/staging/media/meson/vdec/codec_h264.h
> @@ -0,0 +1,14 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * Copyright (C) 2019 BayLibre, SAS
> + * Author: Maxime Jourdan <mjourdan@baylibre.com>
> + */
> +
> +#ifndef __MESON_VDEC_CODEC_H264_H_
> +#define __MESON_VDEC_CODEC_H264_H_
> +
> +#include "vdec.h"
> +
> +extern struct amvdec_codec_ops codec_h264_ops;
> +
> +#endif
> diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
> index accad8f8929a..025371ff8fae 100644
> --- a/drivers/staging/media/meson/vdec/vdec_platform.c
> +++ b/drivers/staging/media/meson/vdec/vdec_platform.c
> @@ -9,9 +9,22 @@
>  
>  #include "vdec_1.h"
>  #include "codec_mpeg12.h"
> +#include "codec_h264.h"
>  
>  static const struct amvdec_format vdec_formats_gxbb[] = {
>  	{
> +		.pixfmt = V4L2_PIX_FMT_H264,
> +		.min_buffers = 2,
> +		.max_buffers = 24,
> +		.max_width = 1920,
> +		.max_height = 1080,
> +		.vdec_ops = &vdec_1_ops,
> +		.codec_ops = &codec_h264_ops,
> +		.firmware_path = "meson/vdec/gxbb_h264.bin",
> +		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED |
> +			 V4L2_FMT_FLAG_DYN_RESOLUTION,
> +	}, {
>  		.pixfmt = V4L2_PIX_FMT_MPEG1,
>  		.min_buffers = 8,
>  		.max_buffers = 8,
> @@ -38,6 +51,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
>  
>  static const struct amvdec_format vdec_formats_gxl[] = {
>  	{
> +		.pixfmt = V4L2_PIX_FMT_H264,
> +		.min_buffers = 2,
> +		.max_buffers = 24,
> +		.max_width = 3840,
> +		.max_height = 2160,
> +		.vdec_ops = &vdec_1_ops,
> +		.codec_ops = &codec_h264_ops,
> +		.firmware_path = "meson/vdec/gxl_h264.bin",
> +		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED |
> +			 V4L2_FMT_FLAG_DYN_RESOLUTION,
> +	}, {
>  		.pixfmt = V4L2_PIX_FMT_MPEG1,
>  		.min_buffers = 8,
>  		.max_buffers = 8,
> @@ -64,6 +89,18 @@ static const struct amvdec_format vdec_formats_gxl[] = {
>  
>  static const struct amvdec_format vdec_formats_gxm[] = {
>  	{
> +		.pixfmt = V4L2_PIX_FMT_H264,
> +		.min_buffers = 2,
> +		.max_buffers = 24,
> +		.max_width = 3840,
> +		.max_height = 2160,
> +		.vdec_ops = &vdec_1_ops,
> +		.codec_ops = &codec_h264_ops,
> +		.firmware_path = "meson/vdec/gxm_h264.bin",
> +		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> +		.flags = V4L2_FMT_FLAG_COMPRESSED |
> +			 V4L2_FMT_FLAG_DYN_RESOLUTION,
> +	}, {
>  		.pixfmt = V4L2_PIX_FMT_MPEG1,
>  		.min_buffers = 8,
>  		.max_buffers = 8,

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH 2/2] media: meson: vdec: add H.264 decoding support

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 23658 bytes --]

Le mardi 08 octobre 2019 à 16:27 -0400, Nicolas Dufresne a écrit :
> Le lundi 07 octobre 2019 à 16:59 +0200, Maxime Jourdan a écrit :
> > Add support for the H264 compressed format (V4L2_PIX_FMT_H264).
> > 
> > Signed-off-by: Maxime Jourdan <mjourdan@baylibre.com>
> > ---
> >  drivers/staging/media/meson/vdec/Makefile     |   2 +-
> >  drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
> >  drivers/staging/media/meson/vdec/codec_h264.h |  14 +
> >  .../staging/media/meson/vdec/vdec_platform.c  |  37 ++
> >  4 files changed, 534 insertions(+), 1 deletion(-)
> >  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
> >  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
> > 
> > diff --git a/drivers/staging/media/meson/vdec/Makefile b/drivers/staging/media/meson/vdec/Makefile
> > index 6bea129084b7..711d990c760e 100644
> > --- a/drivers/staging/media/meson/vdec/Makefile
> > +++ b/drivers/staging/media/meson/vdec/Makefile
> > @@ -3,6 +3,6 @@
> >  
> >  meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o
> >  meson-vdec-objs += vdec_1.o
> > -meson-vdec-objs += codec_mpeg12.o
> > +meson-vdec-objs += codec_mpeg12.o codec_h264.o
> >  
> >  obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
> > diff --git a/drivers/staging/media/meson/vdec/codec_h264.c b/drivers/staging/media/meson/vdec/codec_h264.c
> > new file mode 100644
> > index 000000000000..4528a6a01c3d
> > --- /dev/null
> > +++ b/drivers/staging/media/meson/vdec/codec_h264.c
> > @@ -0,0 +1,482 @@
> > +// SPDX-License-Identifier: GPL-2.0+
> > +/*
> > + * Copyright (C) 2019 BayLibre, SAS
> > + * Author: Maxime Jourdan <mjourdan@baylibre.com>
> > + */
> > +
> > +#include <media/v4l2-mem2mem.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "vdec_helpers.h"
> > +#include "dos_regs.h"
> > +
> > +#define SIZE_EXT_FW	(20 * SZ_1K)
> > +#define SIZE_WORKSPACE	0x1ee000
> > +#define SIZE_SEI	(8 * SZ_1K)
> > +
> > +/*
> > + * Offset added by the firmware which must be substracted
> > + * from the workspace phyaddr
> > + */
> > +#define WORKSPACE_BUF_OFFSET	0x1000000
> > +
> > +/* ISR status */
> > +#define CMD_MASK		GENMASK(7, 0)
> > +#define CMD_SRC_CHANGE		1
> > +#define CMD_FRAMES_READY	2
> > +#define CMD_FATAL_ERROR		6
> > +#define CMD_BAD_WIDTH		7
> > +#define CMD_BAD_HEIGHT		8
> > +
> > +#define SEI_DATA_READY	BIT(15)
> > +
> > +/* Picture type */
> > +#define PIC_TOP_BOT	5
> > +#define PIC_BOT_TOP	6
> > +
> > +/* Size of Motion Vector per macroblock */
> > +#define MB_MV_SIZE	96
> > +
> > +/* Frame status data */
> > +#define PIC_STRUCT_BIT	5
> > +#define PIC_STRUCT_MASK	GENMASK(2, 0)
> > +#define BUF_IDX_MASK	GENMASK(4, 0)
> > +#define ERROR_FLAG	BIT(9)
> > +#define OFFSET_BIT	16
> > +#define OFFSET_MASK	GENMASK(15, 0)
> > +
> > +/* Bitstream parsed data */
> > +#define MB_TOTAL_BIT	8
> > +#define MB_TOTAL_MASK	GENMASK(15, 0)
> > +#define MB_WIDTH_MASK	GENMASK(7, 0)
> > +#define MAX_REF_BIT	24
> > +#define MAX_REF_MASK	GENMASK(6, 0)
> > +#define AR_IDC_BIT	16
> > +#define AR_IDC_MASK	GENMASK(7, 0)
> > +#define AR_PRESENT_FLAG	BIT(0)
> > +#define AR_EXTEND	0xff
> > +
> > +/*
> > + * Buffer to send to the ESPARSER to signal End Of Stream for H.264.
> > + * This is a 16x16 encoded picture that will trigger drain firmware-side.
> > + * There is no known alternative.
> > + */
> > +static const u8 eos_sequence[SZ_4K] = {
> > +	0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 0xff, 0xe4, 0xdc, 0x45, 0xe9, 0xbd,
> > +	0xe6, 0xd9, 0x48, 0xb7,	0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef,
> > +	0x78, 0x32, 0x36, 0x34, 0x20, 0x2d, 0x20, 0x63,	0x6f, 0x72, 0x65, 0x20,
> > +	0x36, 0x37, 0x20, 0x72, 0x31, 0x31, 0x33, 0x30, 0x20, 0x38, 0x34, 0x37,
> > +	0x35, 0x39, 0x37, 0x37, 0x20, 0x2d, 0x20, 0x48, 0x2e, 0x32, 0x36, 0x34,
> > +	0x2f, 0x4d, 0x50, 0x45,	0x47, 0x2d, 0x34, 0x20, 0x41, 0x56, 0x43, 0x20,
> > +	0x63, 0x6f, 0x64, 0x65, 0x63, 0x20, 0x2d, 0x20,	0x43, 0x6f, 0x70, 0x79,
> > +	0x6c, 0x65, 0x66, 0x74, 0x20, 0x32, 0x30, 0x30, 0x33, 0x2d, 0x32, 0x30,
> > +	0x30, 0x39, 0x20, 0x2d, 0x20, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
> > +	0x77, 0x77, 0x77, 0x2e,	0x76, 0x69, 0x64, 0x65, 0x6f, 0x6c, 0x61, 0x6e,
> > +	0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x78, 0x32, 0x36,	0x34, 0x2e, 0x68, 0x74,
> > +	0x6d, 0x6c, 0x20, 0x2d, 0x20, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73,
> > +	0x3a, 0x20, 0x63, 0x61, 0x62, 0x61, 0x63, 0x3d, 0x31, 0x20, 0x72, 0x65,
> > +	0x66, 0x3d, 0x31, 0x20,	0x64, 0x65, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x3d,
> > +	0x31, 0x3a, 0x30, 0x3a, 0x30, 0x20, 0x61, 0x6e,	0x61, 0x6c, 0x79, 0x73,
> > +	0x65, 0x3d, 0x30, 0x78, 0x31, 0x3a, 0x30, 0x78, 0x31, 0x31, 0x31, 0x20,
> > +	0x6d, 0x65, 0x3d, 0x68, 0x65, 0x78, 0x20, 0x73, 0x75, 0x62, 0x6d, 0x65,
> > +	0x3d, 0x36, 0x20, 0x70,	0x73, 0x79, 0x5f, 0x72, 0x64, 0x3d, 0x31, 0x2e,
> > +	0x30, 0x3a, 0x30, 0x2e, 0x30, 0x20, 0x6d, 0x69,	0x78, 0x65, 0x64, 0x5f,
> > +	0x72, 0x65, 0x66, 0x3d, 0x30, 0x20, 0x6d, 0x65, 0x5f, 0x72, 0x61, 0x6e,
> > +	0x67, 0x65, 0x3d, 0x31, 0x36, 0x20, 0x63, 0x68, 0x72, 0x6f, 0x6d, 0x61,
> > +	0x5f, 0x6d, 0x65, 0x3d,	0x31, 0x20, 0x74, 0x72, 0x65, 0x6c, 0x6c, 0x69,
> > +	0x73, 0x3d, 0x30, 0x20, 0x38, 0x78, 0x38, 0x64,	0x63, 0x74, 0x3d, 0x30,
> > +	0x20, 0x63, 0x71, 0x6d, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x61, 0x64, 0x7a,
> > +	0x6f, 0x6e, 0x65, 0x3d, 0x32, 0x31, 0x2c, 0x31, 0x31, 0x20, 0x63, 0x68,
> > +	0x72, 0x6f, 0x6d, 0x61,	0x5f, 0x71, 0x70, 0x5f, 0x6f, 0x66, 0x66, 0x73,
> > +	0x65, 0x74, 0x3d, 0x2d, 0x32, 0x20, 0x74, 0x68,	0x72, 0x65, 0x61, 0x64,
> > +	0x73, 0x3d, 0x31, 0x20, 0x6e, 0x72, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x63,
> > +	0x69, 0x6d, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x20, 0x6d, 0x62, 0x61, 0x66,
> > +	0x66, 0x3d, 0x30, 0x20,	0x62, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x3d,
> > +	0x30, 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74,	0x3d, 0x32, 0x35, 0x30,
> > +	0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x5f, 0x6d, 0x69, 0x6e, 0x3d,
> > +	0x32, 0x35, 0x20, 0x73, 0x63, 0x65, 0x6e, 0x65, 0x63, 0x75, 0x74, 0x3d,
> > +	0x34, 0x30, 0x20, 0x72,	0x63, 0x3d, 0x61, 0x62, 0x72, 0x20, 0x62, 0x69,
> > +	0x74, 0x72, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x30,	0x20, 0x72, 0x61, 0x74,
> > +	0x65, 0x74, 0x6f, 0x6c, 0x3d, 0x31, 0x2e, 0x30, 0x20, 0x71, 0x63, 0x6f,
> > +	0x6d, 0x70, 0x3d, 0x30, 0x2e, 0x36, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x69,
> > +	0x6e, 0x3d, 0x31, 0x30,	0x20, 0x71, 0x70, 0x6d, 0x61, 0x78, 0x3d, 0x35,
> > +	0x31, 0x20, 0x71, 0x70, 0x73, 0x74, 0x65, 0x70,	0x3d, 0x34, 0x20, 0x69,
> > +	0x70, 0x5f, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x3d, 0x31, 0x2e, 0x34, 0x30,
> > +	0x20, 0x61, 0x71, 0x3d, 0x31, 0x3a, 0x31, 0x2e, 0x30, 0x30, 0x00, 0x80,
> > +	0x00, 0x00, 0x00, 0x01,	0x67, 0x4d, 0x40, 0x0a, 0x9a, 0x74, 0xf4, 0x20,
> > +	0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x06,	0x51, 0xe2, 0x44, 0xd4,
> > +	0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x32, 0xc8, 0x00, 0x00, 0x00, 0x01,
> > +	0x65, 0x88, 0x80, 0x20, 0x00, 0x08, 0x7f, 0xea, 0x6a, 0xe2, 0x99, 0xb6,
> > +	0x57, 0xae, 0x49, 0x30,	0xf5, 0xfe, 0x5e, 0x46, 0x0b, 0x72, 0x44, 0xc4,
> > +	0xe1, 0xfc, 0x62, 0xda, 0xf1, 0xfb, 0xa2, 0xdb,	0xd6, 0xbe, 0x5c, 0xd7,
> > +	0x24, 0xa3, 0xf5, 0xb9, 0x2f, 0x57, 0x16, 0x49, 0x75, 0x47, 0x77, 0x09,
> > +	0x5c, 0xa1, 0xb4, 0xc3, 0x4f, 0x60, 0x2b, 0xb0, 0x0c, 0xc8, 0xd6, 0x66,
> > +	0xba, 0x9b, 0x82, 0x29,	0x33, 0x92, 0x26, 0x99, 0x31, 0x1c, 0x7f, 0x9b,
> > +	0x00, 0x00, 0x01, 0x0ff,
> > +};
> 
> I can read that this contains a pretty large SEI, SPS, PPS, IDR and
> finally this unspecified Nal 15 with no size, but I don't expect every
> can. Maybe you could format this into nals, and document whatever you
> know about this blob ? If you need info, I may suggest you do put that
> into a file and use YUView software to parse it.

While parsing the SEI was interesting, I think you can drop it
entirely. It's a blob of user data the x264 adds so that you can get
all the parameters that was passed to it. This even includes the URI 
http://www.videolan.org/x264.html and the SVN revision number of thex264 build. 

For the SPS bit, only thing that came out is that this is Main profile,
do all supported generation of H264 decoder on Amlogic supports main ?
To be conservative, it might be nice to encode a constrained-baseline
one. You could also make it smaller by dropping the VUI parameters.
Remember that the SEI and PPS are both bigger then the black slice
being encoded.

About "unspecified" NAL at the end, not sure if it's really needed, I'd
try to drop it.

> 
> > +
> > +static const u8 *codec_h264_eos_sequence(u32 *len)
> > +{
> > +	*len = ARRAY_SIZE(eos_sequence);
> > +	return eos_sequence;
> > +}
> > +
> > +struct codec_h264 {
> > +	/* H.264 decoder requires an extended firmware */
> > +	void      *ext_fw_vaddr;
> > +	dma_addr_t ext_fw_paddr;
> > +
> > +	/* Buffer for the H.264 Workspace */
> > +	void      *workspace_vaddr;
> > +	dma_addr_t workspace_paddr;
> > +
> > +	/* Buffer for the H.264 references MV */
> > +	void      *ref_vaddr;
> > +	dma_addr_t ref_paddr;
> > +	u32	   ref_size;
> > +
> > +	/* Buffer for parsed SEI data */
> > +	void      *sei_vaddr;
> > +	dma_addr_t sei_paddr;
> > +
> > +	u32 mb_width;
> > +	u32 mb_height;
> > +	u32 max_refs;
> > +};
> > +
> > +static int codec_h264_can_recycle(struct amvdec_core *core)
> > +{
> > +	return !amvdec_read_dos(core, AV_SCRATCH_7) ||
> > +	       !amvdec_read_dos(core, AV_SCRATCH_8);
> > +}
> > +
> > +static void codec_h264_recycle(struct amvdec_core *core, u32 buf_idx)
> > +{
> > +	/*
> > +	 * Tell the firmware it can recycle this buffer.
> > +	 * AV_SCRATCH_8 serves the same purpose.
> > +	 */
> > +	if (!amvdec_read_dos(core, AV_SCRATCH_7))
> > +		amvdec_write_dos(core, AV_SCRATCH_7, buf_idx + 1);
> > +	else
> > +		amvdec_write_dos(core, AV_SCRATCH_8, buf_idx + 1);
> > +}
> > +
> > +static int codec_h264_start(struct amvdec_session *sess)
> > +{
> > +	u32 workspace_offset;
> > +	struct amvdec_core *core = sess->core;
> > +	struct codec_h264 *h264 = sess->priv;
> > +
> > +	/* Allocate some memory for the H.264 decoder's state */
> > +	h264->workspace_vaddr =
> > +		dma_alloc_coherent(core->dev, SIZE_WORKSPACE,
> > +				   &h264->workspace_paddr, GFP_KERNEL);
> > +	if (!h264->workspace_vaddr)
> > +		return -ENOMEM;
> > +
> > +	/* Allocate some memory for the H.264 SEI dump */
> > +	h264->sei_vaddr = dma_alloc_coherent(core->dev, SIZE_SEI,
> > +					     &h264->sei_paddr, GFP_KERNEL);
> > +	if (!h264->sei_vaddr)
> > +		return -ENOMEM;
> > +
> > +	amvdec_write_dos_bits(core, POWER_CTL_VLD, BIT(9) | BIT(6));
> > +
> > +	workspace_offset = h264->workspace_paddr - WORKSPACE_BUF_OFFSET;
> > +	amvdec_write_dos(core, AV_SCRATCH_1, workspace_offset);
> > +	amvdec_write_dos(core, AV_SCRATCH_G, h264->ext_fw_paddr);
> > +	amvdec_write_dos(core, AV_SCRATCH_I, h264->sei_paddr -
> > +					     workspace_offset);
> > +
> > +	/* Enable "error correction" */
> > +	amvdec_write_dos(core, AV_SCRATCH_F,
> > +			 (amvdec_read_dos(core, AV_SCRATCH_F) & 0xffffffc3) |
> > +			 BIT(4) | BIT(7));
> > +
> > +	amvdec_write_dos(core, MDEC_PIC_DC_THRESH, 0x404038aa);
> > +
> > +	return 0;
> > +}
> > +
> > +static int codec_h264_stop(struct amvdec_session *sess)
> > +{
> > +	struct codec_h264 *h264 = sess->priv;
> > +	struct amvdec_core *core = sess->core;
> > +
> > +	if (h264->ext_fw_vaddr)
> > +		dma_free_coherent(core->dev, SIZE_EXT_FW,
> > +				  h264->ext_fw_vaddr, h264->ext_fw_paddr);
> > +
> > +	if (h264->workspace_vaddr)
> > +		dma_free_coherent(core->dev, SIZE_WORKSPACE,
> > +				 h264->workspace_vaddr, h264->workspace_paddr);
> > +
> > +	if (h264->ref_vaddr)
> > +		dma_free_coherent(core->dev, h264->ref_size,
> > +				  h264->ref_vaddr, h264->ref_paddr);
> > +
> > +	if (h264->sei_vaddr)
> > +		dma_free_coherent(core->dev, SIZE_SEI,
> > +				  h264->sei_vaddr, h264->sei_paddr);
> > +
> > +	return 0;
> > +}
> > +
> > +static int codec_h264_load_extended_firmware(struct amvdec_session *sess,
> > +					     const u8 *data, u32 len)
> > +{
> > +	struct codec_h264 *h264;
> > +	struct amvdec_core *core = sess->core;
> > +
> > +	if (len < SIZE_EXT_FW)
> > +		return -EINVAL;
> > +
> > +	h264 = kzalloc(sizeof(*h264), GFP_KERNEL);
> > +	if (!h264)
> > +		return -ENOMEM;
> > +
> > +	h264->ext_fw_vaddr = dma_alloc_coherent(core->dev, SIZE_EXT_FW,
> > +					      &h264->ext_fw_paddr, GFP_KERNEL);
> > +	if (!h264->ext_fw_vaddr) {
> > +		kfree(h264);
> > +		return -ENOMEM;
> > +	}
> > +
> > +	memcpy(h264->ext_fw_vaddr, data, SIZE_EXT_FW);
> > +	sess->priv = h264;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_fract par_table[] = {
> > +	{ 1, 1 },   { 1, 1 },    { 12, 11 }, { 10, 11 },
> > +	{ 16, 11 }, { 40, 33 },  { 24, 11 }, { 20, 11 },
> > +	{ 32, 11 }, { 80, 33 },  { 18, 11 }, { 15, 11 },
> > +	{ 64, 33 }, { 160, 99 }, { 4, 3 },   { 3, 2 },
> > +	{ 2, 1 }
> > +};
> > +
> > +static void codec_h264_set_par(struct amvdec_session *sess)
> > +{
> > +	struct amvdec_core *core = sess->core;
> > +	u32 seq_info = amvdec_read_dos(core, AV_SCRATCH_2);
> > +	u32 ar_idc = (seq_info >> AR_IDC_BIT) & AR_IDC_MASK;
> > +
> > +	if (!(seq_info & AR_PRESENT_FLAG))
> > +		return;
> > +
> > +	if (ar_idc == AR_EXTEND) {
> > +		u32 ar_info = amvdec_read_dos(core, AV_SCRATCH_3);
> > +
> > +		sess->pixelaspect.numerator = ar_info & 0xffff;
> > +		sess->pixelaspect.denominator = (ar_info >> 16) & 0xffff;
> > +		return;
> > +	}
> > +
> > +	if (ar_idc >= ARRAY_SIZE(par_table))
> > +		return;
> > +
> > +	sess->pixelaspect = par_table[ar_idc];
> > +}
> > +
> > +static void codec_h264_resume(struct amvdec_session *sess)
> > +{
> > +	struct amvdec_core *core = sess->core;
> > +	struct codec_h264 *h264 = sess->priv;
> > +	u32 mb_width, mb_height, mb_total;
> > +
> > +	amvdec_set_canvases(sess, (u32[]){ ANC0_CANVAS_ADDR, 0 },
> > +				  (u32[]){ 24, 0 });
> > +
> > +	dev_dbg(core->dev, "max_refs = %u; actual_dpb_size = %u\n",
> > +		h264->max_refs, sess->num_dst_bufs);
> > +
> > +	/* Align to a multiple of 4 macroblocks */
> > +	mb_width = ALIGN(h264->mb_width, 4);
> > +	mb_height = ALIGN(h264->mb_height, 4);
> > +	mb_total = mb_width * mb_height;
> > +
> > +	h264->ref_size = mb_total * MB_MV_SIZE * h264->max_refs;
> > +	h264->ref_vaddr = dma_alloc_coherent(core->dev, h264->ref_size,
> > +					     &h264->ref_paddr, GFP_KERNEL);
> > +	if (!h264->ref_vaddr) {
> > +		amvdec_abort(sess);
> > +		return;
> > +	}
> > +
> > +	/* Address to store the references' MVs */
> > +	amvdec_write_dos(core, AV_SCRATCH_1, h264->ref_paddr);
> > +	/* End of ref MV */
> > +	amvdec_write_dos(core, AV_SCRATCH_4, h264->ref_paddr + h264->ref_size);
> > +
> > +	amvdec_write_dos(core, AV_SCRATCH_0, (h264->max_refs << 24) |
> > +					     (sess->num_dst_bufs << 16) |
> > +					     ((h264->max_refs - 1) << 8));
> > +}
> > +
> > +/**
> > + * Configure the H.264 decoder when the parser detected a parameter set change
> > + */
> > +static void codec_h264_src_change(struct amvdec_session *sess)
> > +{
> > +	struct amvdec_core *core = sess->core;
> > +	struct codec_h264 *h264 = sess->priv;
> > +	u32 parsed_info, mb_total;
> > +	u32 crop_infor, crop_bottom, crop_right;
> > +	u32 frame_width, frame_height;
> > +
> > +	sess->keyframe_found = 1;
> > +
> > +	parsed_info = amvdec_read_dos(core, AV_SCRATCH_1);
> > +
> > +	/* Total number of 16x16 macroblocks */
> > +	mb_total = (parsed_info >> MB_TOTAL_BIT) & MB_TOTAL_MASK;
> > +	/* Number of macroblocks per line */
> > +	h264->mb_width = parsed_info & MB_WIDTH_MASK;
> > +	/* Number of macroblock lines */
> > +	h264->mb_height = mb_total / h264->mb_width;
> > +
> > +	h264->max_refs = ((parsed_info >> MAX_REF_BIT) & MAX_REF_MASK) + 1;
> > +
> > +	crop_infor = amvdec_read_dos(core, AV_SCRATCH_6);
> > +	crop_bottom = (crop_infor & 0xff);
> > +	crop_right = (crop_infor >> 16) & 0xff;
> > +
> > +	frame_width = h264->mb_width * 16 - crop_right;
> > +	frame_height = h264->mb_height * 16 - crop_bottom;
> > +
> > +	dev_dbg(core->dev, "frame: %ux%u; crop: %u %u\n",
> > +		frame_width, frame_height, crop_right, crop_bottom);
> > +
> > +	codec_h264_set_par(sess);
> > +	amvdec_src_change(sess, frame_width, frame_height, h264->max_refs + 5);
> > +}
> > +
> > +/**
> > + * The bitstream offset is split in half in 2 different registers.
> > + * Fetch its MSB here, which location depends on the frame number.
> > + */
> > +static u32 get_offset_msb(struct amvdec_core *core, int frame_num)
> > +{
> > +	int take_msb = frame_num % 2;
> > +	int reg_offset = (frame_num / 2) * 4;
> > +	u32 offset_msb = amvdec_read_dos(core, AV_SCRATCH_A + reg_offset);
> > +
> > +	if (take_msb)
> > +		return offset_msb & 0xffff0000;
> > +
> > +	return (offset_msb & 0x0000ffff) << 16;
> > +}
> > +
> > +static void codec_h264_frames_ready(struct amvdec_session *sess, u32 status)
> > +{
> > +	struct amvdec_core *core = sess->core;
> > +	int error_count;
> > +	int num_frames;
> > +	int i;
> > +
> > +	error_count = amvdec_read_dos(core, AV_SCRATCH_D);
> > +	num_frames = (status >> 8) & 0xff;
> > +	if (error_count) {
> > +		dev_warn(core->dev,
> > +			 "decoder error(s) happened, count %d\n", error_count);
> > +		amvdec_write_dos(core, AV_SCRATCH_D, 0);
> > +	}
> > +
> > +	for (i = 0; i < num_frames; i++) {
> > +		u32 frame_status = amvdec_read_dos(core, AV_SCRATCH_1 + i * 4);
> > +		u32 buffer_index = frame_status & BUF_IDX_MASK;
> > +		u32 pic_struct = (frame_status >> PIC_STRUCT_BIT) &
> > +				 PIC_STRUCT_MASK;
> > +		u32 offset = (frame_status >> OFFSET_BIT) & OFFSET_MASK;
> > +		u32 field = V4L2_FIELD_NONE;
> > +
> > +		/*
> > +		 * A buffer decode error means it was decoded,
> > +		 * but part of the picture will have artifacts.
> > +		 * Typical reason is a temporarily corrupted bitstream
> > +		 */
> > +		if (frame_status & ERROR_FLAG)
> > +			dev_dbg(core->dev, "Buffer %d decode error\n",
> > +				buffer_index);
> > +
> > +		if (pic_struct == PIC_TOP_BOT)
> > +			field = V4L2_FIELD_INTERLACED_TB;
> > +		else if (pic_struct == PIC_BOT_TOP)
> > +			field = V4L2_FIELD_INTERLACED_BT;
> > +
> > +		offset |= get_offset_msb(core, i);
> > +		amvdec_dst_buf_done_idx(sess, buffer_index, offset, field);
> > +	}
> > +}
> > +
> > +static irqreturn_t codec_h264_threaded_isr(struct amvdec_session *sess)
> > +{
> > +	struct amvdec_core *core = sess->core;
> > +	u32 status;
> > +	u32 size;
> > +	u8 cmd;
> > +
> > +	status = amvdec_read_dos(core, AV_SCRATCH_0);
> > +	cmd = status & CMD_MASK;
> > +
> > +	switch (cmd) {
> > +	case CMD_SRC_CHANGE:
> > +		codec_h264_src_change(sess);
> > +		break;
> > +	case CMD_FRAMES_READY:
> > +		codec_h264_frames_ready(sess, status);
> > +		break;
> > +	case CMD_FATAL_ERROR:
> > +		dev_err(core->dev, "H.264 decoder fatal error\n");
> > +		goto abort;
> > +	case CMD_BAD_WIDTH:
> > +		size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> > +		dev_err(core->dev, "Unsupported video width: %u\n", size);
> > +		goto abort;
> > +	case CMD_BAD_HEIGHT:
> > +		size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> > +		dev_err(core->dev, "Unsupported video height: %u\n", size);
> > +		goto abort;
> > +	case 0: /* Unused but not worth printing for */
> > +	case 9:
> > +		break;
> > +	default:
> > +		dev_info(core->dev, "Unexpected H264 ISR: %08X\n", cmd);
> > +		break;
> > +	}
> > +
> > +	if (cmd && cmd != CMD_SRC_CHANGE)
> > +		amvdec_write_dos(core, AV_SCRATCH_0, 0);
> > +
> > +	/* Decoder has some SEI data for us ; ignore */
> > +	if (amvdec_read_dos(core, AV_SCRATCH_J) & SEI_DATA_READY)
> > +		amvdec_write_dos(core, AV_SCRATCH_J, 0);
> > +
> > +	return IRQ_HANDLED;
> > +abort:
> > +	amvdec_abort(sess);
> > +	return IRQ_HANDLED;
> > +}
> > +
> > +static irqreturn_t codec_h264_isr(struct amvdec_session *sess)
> > +{
> > +	struct amvdec_core *core = sess->core;
> > +
> > +	amvdec_write_dos(core, ASSIST_MBOX1_CLR_REG, 1);
> > +
> > +	return IRQ_WAKE_THREAD;
> > +}
> > +
> > +struct amvdec_codec_ops codec_h264_ops = {
> > +	.start = codec_h264_start,
> > +	.stop = codec_h264_stop,
> > +	.load_extended_firmware = codec_h264_load_extended_firmware,
> > +	.isr = codec_h264_isr,
> > +	.threaded_isr = codec_h264_threaded_isr,
> > +	.can_recycle = codec_h264_can_recycle,
> > +	.recycle = codec_h264_recycle,
> > +	.eos_sequence = codec_h264_eos_sequence,
> > +	.resume = codec_h264_resume,
> > +};
> > diff --git a/drivers/staging/media/meson/vdec/codec_h264.h b/drivers/staging/media/meson/vdec/codec_h264.h
> > new file mode 100644
> > index 000000000000..7cb4fb86ff36
> > --- /dev/null
> > +++ b/drivers/staging/media/meson/vdec/codec_h264.h
> > @@ -0,0 +1,14 @@
> > +/* SPDX-License-Identifier: GPL-2.0+ */
> > +/*
> > + * Copyright (C) 2019 BayLibre, SAS
> > + * Author: Maxime Jourdan <mjourdan@baylibre.com>
> > + */
> > +
> > +#ifndef __MESON_VDEC_CODEC_H264_H_
> > +#define __MESON_VDEC_CODEC_H264_H_
> > +
> > +#include "vdec.h"
> > +
> > +extern struct amvdec_codec_ops codec_h264_ops;
> > +
> > +#endif
> > diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
> > index accad8f8929a..025371ff8fae 100644
> > --- a/drivers/staging/media/meson/vdec/vdec_platform.c
> > +++ b/drivers/staging/media/meson/vdec/vdec_platform.c
> > @@ -9,9 +9,22 @@
> >  
> >  #include "vdec_1.h"
> >  #include "codec_mpeg12.h"
> > +#include "codec_h264.h"
> >  
> >  static const struct amvdec_format vdec_formats_gxbb[] = {
> >  	{
> > +		.pixfmt = V4L2_PIX_FMT_H264,
> > +		.min_buffers = 2,
> > +		.max_buffers = 24,
> > +		.max_width = 1920,
> > +		.max_height = 1080,
> > +		.vdec_ops = &vdec_1_ops,
> > +		.codec_ops = &codec_h264_ops,
> > +		.firmware_path = "meson/vdec/gxbb_h264.bin",
> > +		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> > +		.flags = V4L2_FMT_FLAG_COMPRESSED |
> > +			 V4L2_FMT_FLAG_DYN_RESOLUTION,
> > +	}, {
> >  		.pixfmt = V4L2_PIX_FMT_MPEG1,
> >  		.min_buffers = 8,
> >  		.max_buffers = 8,
> > @@ -38,6 +51,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
> >  
> >  static const struct amvdec_format vdec_formats_gxl[] = {
> >  	{
> > +		.pixfmt = V4L2_PIX_FMT_H264,
> > +		.min_buffers = 2,
> > +		.max_buffers = 24,
> > +		.max_width = 3840,
> > +		.max_height = 2160,
> > +		.vdec_ops = &vdec_1_ops,
> > +		.codec_ops = &codec_h264_ops,
> > +		.firmware_path = "meson/vdec/gxl_h264.bin",
> > +		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> > +		.flags = V4L2_FMT_FLAG_COMPRESSED |
> > +			 V4L2_FMT_FLAG_DYN_RESOLUTION,
> > +	}, {
> >  		.pixfmt = V4L2_PIX_FMT_MPEG1,
> >  		.min_buffers = 8,
> >  		.max_buffers = 8,
> > @@ -64,6 +89,18 @@ static const struct amvdec_format vdec_formats_gxl[] = {
> >  
> >  static const struct amvdec_format vdec_formats_gxm[] = {
> >  	{
> > +		.pixfmt = V4L2_PIX_FMT_H264,
> > +		.min_buffers = 2,
> > +		.max_buffers = 24,
> > +		.max_width = 3840,
> > +		.max_height = 2160,
> > +		.vdec_ops = &vdec_1_ops,
> > +		.codec_ops = &codec_h264_ops,
> > +		.firmware_path = "meson/vdec/gxm_h264.bin",
> > +		.pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> > +		.flags = V4L2_FMT_FLAG_COMPRESSED |
> > +			 V4L2_FMT_FLAG_DYN_RESOLUTION,
> > +	}, {
> >  		.pixfmt = V4L2_PIX_FMT_MPEG1,
> >  		.min_buffers = 8,
> >  		.max_buffers = 8,

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [PATCH 2/2] media: meson: vdec: add H.264 decoding support

[Maxime Jourdan]

On Tue, Oct 8, 2019 at 10:44 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>
> Le mardi 08 octobre 2019 à 16:27 -0400, Nicolas Dufresne a écrit :
> > Le lundi 07 octobre 2019 à 16:59 +0200, Maxime Jourdan a écrit :
> > > Add support for the H264 compressed format (V4L2_PIX_FMT_H264).
> > >
> > > Signed-off-by: Maxime Jourdan <mjourdan@baylibre.com>
> > > ---
> > >  drivers/staging/media/meson/vdec/Makefile     |   2 +-
> > >  drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
> > >  drivers/staging/media/meson/vdec/codec_h264.h |  14 +
> > >  .../staging/media/meson/vdec/vdec_platform.c  |  37 ++
> > >  4 files changed, 534 insertions(+), 1 deletion(-)
> > >  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
> > >  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
> > >
> > > diff --git a/drivers/staging/media/meson/vdec/Makefile b/drivers/staging/media/meson/vdec/Makefile
> > > index 6bea129084b7..711d990c760e 100644
> > > --- a/drivers/staging/media/meson/vdec/Makefile
> > > +++ b/drivers/staging/media/meson/vdec/Makefile
> > > @@ -3,6 +3,6 @@
> > >
> > >  meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o
> > >  meson-vdec-objs += vdec_1.o
> > > -meson-vdec-objs += codec_mpeg12.o
> > > +meson-vdec-objs += codec_mpeg12.o codec_h264.o
> > >
> > >  obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
> > > diff --git a/drivers/staging/media/meson/vdec/codec_h264.c b/drivers/staging/media/meson/vdec/codec_h264.c
> > > new file mode 100644
> > > index 000000000000..4528a6a01c3d
> > > --- /dev/null
> > > +++ b/drivers/staging/media/meson/vdec/codec_h264.c
> > > @@ -0,0 +1,482 @@
> > > +// SPDX-License-Identifier: GPL-2.0+
> > > +/*
> > > + * Copyright (C) 2019 BayLibre, SAS
> > > + * Author: Maxime Jourdan <mjourdan@baylibre.com>
> > > + */
> > > +
> > > +#include <media/v4l2-mem2mem.h>
> > > +#include <media/videobuf2-dma-contig.h>
> > > +
> > > +#include "vdec_helpers.h"
> > > +#include "dos_regs.h"
> > > +
> > > +#define SIZE_EXT_FW        (20 * SZ_1K)
> > > +#define SIZE_WORKSPACE     0x1ee000
> > > +#define SIZE_SEI   (8 * SZ_1K)
> > > +
> > > +/*
> > > + * Offset added by the firmware which must be substracted
> > > + * from the workspace phyaddr
> > > + */
> > > +#define WORKSPACE_BUF_OFFSET       0x1000000
> > > +
> > > +/* ISR status */
> > > +#define CMD_MASK           GENMASK(7, 0)
> > > +#define CMD_SRC_CHANGE             1
> > > +#define CMD_FRAMES_READY   2
> > > +#define CMD_FATAL_ERROR            6
> > > +#define CMD_BAD_WIDTH              7
> > > +#define CMD_BAD_HEIGHT             8
> > > +
> > > +#define SEI_DATA_READY     BIT(15)
> > > +
> > > +/* Picture type */
> > > +#define PIC_TOP_BOT        5
> > > +#define PIC_BOT_TOP        6
> > > +
> > > +/* Size of Motion Vector per macroblock */
> > > +#define MB_MV_SIZE 96
> > > +
> > > +/* Frame status data */
> > > +#define PIC_STRUCT_BIT     5
> > > +#define PIC_STRUCT_MASK    GENMASK(2, 0)
> > > +#define BUF_IDX_MASK       GENMASK(4, 0)
> > > +#define ERROR_FLAG BIT(9)
> > > +#define OFFSET_BIT 16
> > > +#define OFFSET_MASK        GENMASK(15, 0)
> > > +
> > > +/* Bitstream parsed data */
> > > +#define MB_TOTAL_BIT       8
> > > +#define MB_TOTAL_MASK      GENMASK(15, 0)
> > > +#define MB_WIDTH_MASK      GENMASK(7, 0)
> > > +#define MAX_REF_BIT        24
> > > +#define MAX_REF_MASK       GENMASK(6, 0)
> > > +#define AR_IDC_BIT 16
> > > +#define AR_IDC_MASK        GENMASK(7, 0)
> > > +#define AR_PRESENT_FLAG    BIT(0)
> > > +#define AR_EXTEND  0xff
> > > +
> > > +/*
> > > + * Buffer to send to the ESPARSER to signal End Of Stream for H.264.
> > > + * This is a 16x16 encoded picture that will trigger drain firmware-side.
> > > + * There is no known alternative.
> > > + */
> > > +static const u8 eos_sequence[SZ_4K] = {
> > > +   0x00, 0x00, 0x00, 0x01, 0x06, 0x05, 0xff, 0xe4, 0xdc, 0x45, 0xe9, 0xbd,
> > > +   0xe6, 0xd9, 0x48, 0xb7, 0x96, 0x2c, 0xd8, 0x20, 0xd9, 0x23, 0xee, 0xef,
> > > +   0x78, 0x32, 0x36, 0x34, 0x20, 0x2d, 0x20, 0x63, 0x6f, 0x72, 0x65, 0x20,
> > > +   0x36, 0x37, 0x20, 0x72, 0x31, 0x31, 0x33, 0x30, 0x20, 0x38, 0x34, 0x37,
> > > +   0x35, 0x39, 0x37, 0x37, 0x20, 0x2d, 0x20, 0x48, 0x2e, 0x32, 0x36, 0x34,
> > > +   0x2f, 0x4d, 0x50, 0x45, 0x47, 0x2d, 0x34, 0x20, 0x41, 0x56, 0x43, 0x20,
> > > +   0x63, 0x6f, 0x64, 0x65, 0x63, 0x20, 0x2d, 0x20, 0x43, 0x6f, 0x70, 0x79,
> > > +   0x6c, 0x65, 0x66, 0x74, 0x20, 0x32, 0x30, 0x30, 0x33, 0x2d, 0x32, 0x30,
> > > +   0x30, 0x39, 0x20, 0x2d, 0x20, 0x68, 0x74, 0x74, 0x70, 0x3a, 0x2f, 0x2f,
> > > +   0x77, 0x77, 0x77, 0x2e, 0x76, 0x69, 0x64, 0x65, 0x6f, 0x6c, 0x61, 0x6e,
> > > +   0x2e, 0x6f, 0x72, 0x67, 0x2f, 0x78, 0x32, 0x36, 0x34, 0x2e, 0x68, 0x74,
> > > +   0x6d, 0x6c, 0x20, 0x2d, 0x20, 0x6f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73,
> > > +   0x3a, 0x20, 0x63, 0x61, 0x62, 0x61, 0x63, 0x3d, 0x31, 0x20, 0x72, 0x65,
> > > +   0x66, 0x3d, 0x31, 0x20, 0x64, 0x65, 0x62, 0x6c, 0x6f, 0x63, 0x6b, 0x3d,
> > > +   0x31, 0x3a, 0x30, 0x3a, 0x30, 0x20, 0x61, 0x6e, 0x61, 0x6c, 0x79, 0x73,
> > > +   0x65, 0x3d, 0x30, 0x78, 0x31, 0x3a, 0x30, 0x78, 0x31, 0x31, 0x31, 0x20,
> > > +   0x6d, 0x65, 0x3d, 0x68, 0x65, 0x78, 0x20, 0x73, 0x75, 0x62, 0x6d, 0x65,
> > > +   0x3d, 0x36, 0x20, 0x70, 0x73, 0x79, 0x5f, 0x72, 0x64, 0x3d, 0x31, 0x2e,
> > > +   0x30, 0x3a, 0x30, 0x2e, 0x30, 0x20, 0x6d, 0x69, 0x78, 0x65, 0x64, 0x5f,
> > > +   0x72, 0x65, 0x66, 0x3d, 0x30, 0x20, 0x6d, 0x65, 0x5f, 0x72, 0x61, 0x6e,
> > > +   0x67, 0x65, 0x3d, 0x31, 0x36, 0x20, 0x63, 0x68, 0x72, 0x6f, 0x6d, 0x61,
> > > +   0x5f, 0x6d, 0x65, 0x3d, 0x31, 0x20, 0x74, 0x72, 0x65, 0x6c, 0x6c, 0x69,
> > > +   0x73, 0x3d, 0x30, 0x20, 0x38, 0x78, 0x38, 0x64, 0x63, 0x74, 0x3d, 0x30,
> > > +   0x20, 0x63, 0x71, 0x6d, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x61, 0x64, 0x7a,
> > > +   0x6f, 0x6e, 0x65, 0x3d, 0x32, 0x31, 0x2c, 0x31, 0x31, 0x20, 0x63, 0x68,
> > > +   0x72, 0x6f, 0x6d, 0x61, 0x5f, 0x71, 0x70, 0x5f, 0x6f, 0x66, 0x66, 0x73,
> > > +   0x65, 0x74, 0x3d, 0x2d, 0x32, 0x20, 0x74, 0x68, 0x72, 0x65, 0x61, 0x64,
> > > +   0x73, 0x3d, 0x31, 0x20, 0x6e, 0x72, 0x3d, 0x30, 0x20, 0x64, 0x65, 0x63,
> > > +   0x69, 0x6d, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x20, 0x6d, 0x62, 0x61, 0x66,
> > > +   0x66, 0x3d, 0x30, 0x20, 0x62, 0x66, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x3d,
> > > +   0x30, 0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x3d, 0x32, 0x35, 0x30,
> > > +   0x20, 0x6b, 0x65, 0x79, 0x69, 0x6e, 0x74, 0x5f, 0x6d, 0x69, 0x6e, 0x3d,
> > > +   0x32, 0x35, 0x20, 0x73, 0x63, 0x65, 0x6e, 0x65, 0x63, 0x75, 0x74, 0x3d,
> > > +   0x34, 0x30, 0x20, 0x72, 0x63, 0x3d, 0x61, 0x62, 0x72, 0x20, 0x62, 0x69,
> > > +   0x74, 0x72, 0x61, 0x74, 0x65, 0x3d, 0x31, 0x30, 0x20, 0x72, 0x61, 0x74,
> > > +   0x65, 0x74, 0x6f, 0x6c, 0x3d, 0x31, 0x2e, 0x30, 0x20, 0x71, 0x63, 0x6f,
> > > +   0x6d, 0x70, 0x3d, 0x30, 0x2e, 0x36, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x69,
> > > +   0x6e, 0x3d, 0x31, 0x30, 0x20, 0x71, 0x70, 0x6d, 0x61, 0x78, 0x3d, 0x35,
> > > +   0x31, 0x20, 0x71, 0x70, 0x73, 0x74, 0x65, 0x70, 0x3d, 0x34, 0x20, 0x69,
> > > +   0x70, 0x5f, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x3d, 0x31, 0x2e, 0x34, 0x30,
> > > +   0x20, 0x61, 0x71, 0x3d, 0x31, 0x3a, 0x31, 0x2e, 0x30, 0x30, 0x00, 0x80,
> > > +   0x00, 0x00, 0x00, 0x01, 0x67, 0x4d, 0x40, 0x0a, 0x9a, 0x74, 0xf4, 0x20,
> > > +   0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x06, 0x51, 0xe2, 0x44, 0xd4,
> > > +   0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x32, 0xc8, 0x00, 0x00, 0x00, 0x01,
> > > +   0x65, 0x88, 0x80, 0x20, 0x00, 0x08, 0x7f, 0xea, 0x6a, 0xe2, 0x99, 0xb6,
> > > +   0x57, 0xae, 0x49, 0x30, 0xf5, 0xfe, 0x5e, 0x46, 0x0b, 0x72, 0x44, 0xc4,
> > > +   0xe1, 0xfc, 0x62, 0xda, 0xf1, 0xfb, 0xa2, 0xdb, 0xd6, 0xbe, 0x5c, 0xd7,
> > > +   0x24, 0xa3, 0xf5, 0xb9, 0x2f, 0x57, 0x16, 0x49, 0x75, 0x47, 0x77, 0x09,
> > > +   0x5c, 0xa1, 0xb4, 0xc3, 0x4f, 0x60, 0x2b, 0xb0, 0x0c, 0xc8, 0xd6, 0x66,
> > > +   0xba, 0x9b, 0x82, 0x29, 0x33, 0x92, 0x26, 0x99, 0x31, 0x1c, 0x7f, 0x9b,
> > > +   0x00, 0x00, 0x01, 0x0ff,
> > > +};
> >
> > I can read that this contains a pretty large SEI, SPS, PPS, IDR and
> > finally this unspecified Nal 15 with no size, but I don't expect every
> > can. Maybe you could format this into nals, and document whatever you
> > know about this blob ? If you need info, I may suggest you do put that
> > into a file and use YUView software to parse it.
>
> While parsing the SEI was interesting, I think you can drop it
> entirely. It's a blob of user data the x264 adds so that you can get
> all the parameters that was passed to it. This even includes the URI
> http://www.videolan.org/x264.html and the SVN revision number of thex264 build.
>
> For the SPS bit, only thing that came out is that this is Main profile,
> do all supported generation of H264 decoder on Amlogic supports main ?
> To be conservative, it might be nice to encode a constrained-baseline
> one. You could also make it smaller by dropping the VUI parameters.
> Remember that the SEI and PPS are both bigger then the black slice
> being encoded.
>
> About "unspecified" NAL at the end, not sure if it's really needed, I'd
> try to drop it.
>

I actually had a talk with amlogic about this a while ago, their
summary was that this is a 16x16 encoded picture (so no wonder you
found a SPS & cie in it) that will trigger drain firmware side. It's a
complete hack but last time I checked this was their only way (in
vendor SDK) of draining the H.264 decoder. This blob is actually the
same that is sent by vendor userspace when trying to drain the
decoder, hence why I kept it.

I will see if we can find a way around this, or at least shorten this
packet. Thanks for all the insights!

> >
> > > +
> > > +static const u8 *codec_h264_eos_sequence(u32 *len)
> > > +{
> > > +   *len = ARRAY_SIZE(eos_sequence);
> > > +   return eos_sequence;
> > > +}
> > > +
> > > +struct codec_h264 {
> > > +   /* H.264 decoder requires an extended firmware */
> > > +   void      *ext_fw_vaddr;
> > > +   dma_addr_t ext_fw_paddr;
> > > +
> > > +   /* Buffer for the H.264 Workspace */
> > > +   void      *workspace_vaddr;
> > > +   dma_addr_t workspace_paddr;
> > > +
> > > +   /* Buffer for the H.264 references MV */
> > > +   void      *ref_vaddr;
> > > +   dma_addr_t ref_paddr;
> > > +   u32        ref_size;
> > > +
> > > +   /* Buffer for parsed SEI data */
> > > +   void      *sei_vaddr;
> > > +   dma_addr_t sei_paddr;
> > > +
> > > +   u32 mb_width;
> > > +   u32 mb_height;
> > > +   u32 max_refs;
> > > +};
> > > +
> > > +static int codec_h264_can_recycle(struct amvdec_core *core)
> > > +{
> > > +   return !amvdec_read_dos(core, AV_SCRATCH_7) ||
> > > +          !amvdec_read_dos(core, AV_SCRATCH_8);
> > > +}
> > > +
> > > +static void codec_h264_recycle(struct amvdec_core *core, u32 buf_idx)
> > > +{
> > > +   /*
> > > +    * Tell the firmware it can recycle this buffer.
> > > +    * AV_SCRATCH_8 serves the same purpose.
> > > +    */
> > > +   if (!amvdec_read_dos(core, AV_SCRATCH_7))
> > > +           amvdec_write_dos(core, AV_SCRATCH_7, buf_idx + 1);
> > > +   else
> > > +           amvdec_write_dos(core, AV_SCRATCH_8, buf_idx + 1);
> > > +}
> > > +
> > > +static int codec_h264_start(struct amvdec_session *sess)
> > > +{
> > > +   u32 workspace_offset;
> > > +   struct amvdec_core *core = sess->core;
> > > +   struct codec_h264 *h264 = sess->priv;
> > > +
> > > +   /* Allocate some memory for the H.264 decoder's state */
> > > +   h264->workspace_vaddr =
> > > +           dma_alloc_coherent(core->dev, SIZE_WORKSPACE,
> > > +                              &h264->workspace_paddr, GFP_KERNEL);
> > > +   if (!h264->workspace_vaddr)
> > > +           return -ENOMEM;
> > > +
> > > +   /* Allocate some memory for the H.264 SEI dump */
> > > +   h264->sei_vaddr = dma_alloc_coherent(core->dev, SIZE_SEI,
> > > +                                        &h264->sei_paddr, GFP_KERNEL);
> > > +   if (!h264->sei_vaddr)
> > > +           return -ENOMEM;
> > > +
> > > +   amvdec_write_dos_bits(core, POWER_CTL_VLD, BIT(9) | BIT(6));
> > > +
> > > +   workspace_offset = h264->workspace_paddr - WORKSPACE_BUF_OFFSET;
> > > +   amvdec_write_dos(core, AV_SCRATCH_1, workspace_offset);
> > > +   amvdec_write_dos(core, AV_SCRATCH_G, h264->ext_fw_paddr);
> > > +   amvdec_write_dos(core, AV_SCRATCH_I, h264->sei_paddr -
> > > +                                        workspace_offset);
> > > +
> > > +   /* Enable "error correction" */
> > > +   amvdec_write_dos(core, AV_SCRATCH_F,
> > > +                    (amvdec_read_dos(core, AV_SCRATCH_F) & 0xffffffc3) |
> > > +                    BIT(4) | BIT(7));
> > > +
> > > +   amvdec_write_dos(core, MDEC_PIC_DC_THRESH, 0x404038aa);
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int codec_h264_stop(struct amvdec_session *sess)
> > > +{
> > > +   struct codec_h264 *h264 = sess->priv;
> > > +   struct amvdec_core *core = sess->core;
> > > +
> > > +   if (h264->ext_fw_vaddr)
> > > +           dma_free_coherent(core->dev, SIZE_EXT_FW,
> > > +                             h264->ext_fw_vaddr, h264->ext_fw_paddr);
> > > +
> > > +   if (h264->workspace_vaddr)
> > > +           dma_free_coherent(core->dev, SIZE_WORKSPACE,
> > > +                            h264->workspace_vaddr, h264->workspace_paddr);
> > > +
> > > +   if (h264->ref_vaddr)
> > > +           dma_free_coherent(core->dev, h264->ref_size,
> > > +                             h264->ref_vaddr, h264->ref_paddr);
> > > +
> > > +   if (h264->sei_vaddr)
> > > +           dma_free_coherent(core->dev, SIZE_SEI,
> > > +                             h264->sei_vaddr, h264->sei_paddr);
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static int codec_h264_load_extended_firmware(struct amvdec_session *sess,
> > > +                                        const u8 *data, u32 len)
> > > +{
> > > +   struct codec_h264 *h264;
> > > +   struct amvdec_core *core = sess->core;
> > > +
> > > +   if (len < SIZE_EXT_FW)
> > > +           return -EINVAL;
> > > +
> > > +   h264 = kzalloc(sizeof(*h264), GFP_KERNEL);
> > > +   if (!h264)
> > > +           return -ENOMEM;
> > > +
> > > +   h264->ext_fw_vaddr = dma_alloc_coherent(core->dev, SIZE_EXT_FW,
> > > +                                         &h264->ext_fw_paddr, GFP_KERNEL);
> > > +   if (!h264->ext_fw_vaddr) {
> > > +           kfree(h264);
> > > +           return -ENOMEM;
> > > +   }
> > > +
> > > +   memcpy(h264->ext_fw_vaddr, data, SIZE_EXT_FW);
> > > +   sess->priv = h264;
> > > +
> > > +   return 0;
> > > +}
> > > +
> > > +static const struct v4l2_fract par_table[] = {
> > > +   { 1, 1 },   { 1, 1 },    { 12, 11 }, { 10, 11 },
> > > +   { 16, 11 }, { 40, 33 },  { 24, 11 }, { 20, 11 },
> > > +   { 32, 11 }, { 80, 33 },  { 18, 11 }, { 15, 11 },
> > > +   { 64, 33 }, { 160, 99 }, { 4, 3 },   { 3, 2 },
> > > +   { 2, 1 }
> > > +};
> > > +
> > > +static void codec_h264_set_par(struct amvdec_session *sess)
> > > +{
> > > +   struct amvdec_core *core = sess->core;
> > > +   u32 seq_info = amvdec_read_dos(core, AV_SCRATCH_2);
> > > +   u32 ar_idc = (seq_info >> AR_IDC_BIT) & AR_IDC_MASK;
> > > +
> > > +   if (!(seq_info & AR_PRESENT_FLAG))
> > > +           return;
> > > +
> > > +   if (ar_idc == AR_EXTEND) {
> > > +           u32 ar_info = amvdec_read_dos(core, AV_SCRATCH_3);
> > > +
> > > +           sess->pixelaspect.numerator = ar_info & 0xffff;
> > > +           sess->pixelaspect.denominator = (ar_info >> 16) & 0xffff;
> > > +           return;
> > > +   }
> > > +
> > > +   if (ar_idc >= ARRAY_SIZE(par_table))
> > > +           return;
> > > +
> > > +   sess->pixelaspect = par_table[ar_idc];
> > > +}
> > > +
> > > +static void codec_h264_resume(struct amvdec_session *sess)
> > > +{
> > > +   struct amvdec_core *core = sess->core;
> > > +   struct codec_h264 *h264 = sess->priv;
> > > +   u32 mb_width, mb_height, mb_total;
> > > +
> > > +   amvdec_set_canvases(sess, (u32[]){ ANC0_CANVAS_ADDR, 0 },
> > > +                             (u32[]){ 24, 0 });
> > > +
> > > +   dev_dbg(core->dev, "max_refs = %u; actual_dpb_size = %u\n",
> > > +           h264->max_refs, sess->num_dst_bufs);
> > > +
> > > +   /* Align to a multiple of 4 macroblocks */
> > > +   mb_width = ALIGN(h264->mb_width, 4);
> > > +   mb_height = ALIGN(h264->mb_height, 4);
> > > +   mb_total = mb_width * mb_height;
> > > +
> > > +   h264->ref_size = mb_total * MB_MV_SIZE * h264->max_refs;
> > > +   h264->ref_vaddr = dma_alloc_coherent(core->dev, h264->ref_size,
> > > +                                        &h264->ref_paddr, GFP_KERNEL);
> > > +   if (!h264->ref_vaddr) {
> > > +           amvdec_abort(sess);
> > > +           return;
> > > +   }
> > > +
> > > +   /* Address to store the references' MVs */
> > > +   amvdec_write_dos(core, AV_SCRATCH_1, h264->ref_paddr);
> > > +   /* End of ref MV */
> > > +   amvdec_write_dos(core, AV_SCRATCH_4, h264->ref_paddr + h264->ref_size);
> > > +
> > > +   amvdec_write_dos(core, AV_SCRATCH_0, (h264->max_refs << 24) |
> > > +                                        (sess->num_dst_bufs << 16) |
> > > +                                        ((h264->max_refs - 1) << 8));
> > > +}
> > > +
> > > +/**
> > > + * Configure the H.264 decoder when the parser detected a parameter set change
> > > + */
> > > +static void codec_h264_src_change(struct amvdec_session *sess)
> > > +{
> > > +   struct amvdec_core *core = sess->core;
> > > +   struct codec_h264 *h264 = sess->priv;
> > > +   u32 parsed_info, mb_total;
> > > +   u32 crop_infor, crop_bottom, crop_right;
> > > +   u32 frame_width, frame_height;
> > > +
> > > +   sess->keyframe_found = 1;
> > > +
> > > +   parsed_info = amvdec_read_dos(core, AV_SCRATCH_1);
> > > +
> > > +   /* Total number of 16x16 macroblocks */
> > > +   mb_total = (parsed_info >> MB_TOTAL_BIT) & MB_TOTAL_MASK;
> > > +   /* Number of macroblocks per line */
> > > +   h264->mb_width = parsed_info & MB_WIDTH_MASK;
> > > +   /* Number of macroblock lines */
> > > +   h264->mb_height = mb_total / h264->mb_width;
> > > +
> > > +   h264->max_refs = ((parsed_info >> MAX_REF_BIT) & MAX_REF_MASK) + 1;
> > > +
> > > +   crop_infor = amvdec_read_dos(core, AV_SCRATCH_6);
> > > +   crop_bottom = (crop_infor & 0xff);
> > > +   crop_right = (crop_infor >> 16) & 0xff;
> > > +
> > > +   frame_width = h264->mb_width * 16 - crop_right;
> > > +   frame_height = h264->mb_height * 16 - crop_bottom;
> > > +
> > > +   dev_dbg(core->dev, "frame: %ux%u; crop: %u %u\n",
> > > +           frame_width, frame_height, crop_right, crop_bottom);
> > > +
> > > +   codec_h264_set_par(sess);
> > > +   amvdec_src_change(sess, frame_width, frame_height, h264->max_refs + 5);
> > > +}
> > > +
> > > +/**
> > > + * The bitstream offset is split in half in 2 different registers.
> > > + * Fetch its MSB here, which location depends on the frame number.
> > > + */
> > > +static u32 get_offset_msb(struct amvdec_core *core, int frame_num)
> > > +{
> > > +   int take_msb = frame_num % 2;
> > > +   int reg_offset = (frame_num / 2) * 4;
> > > +   u32 offset_msb = amvdec_read_dos(core, AV_SCRATCH_A + reg_offset);
> > > +
> > > +   if (take_msb)
> > > +           return offset_msb & 0xffff0000;
> > > +
> > > +   return (offset_msb & 0x0000ffff) << 16;
> > > +}
> > > +
> > > +static void codec_h264_frames_ready(struct amvdec_session *sess, u32 status)
> > > +{
> > > +   struct amvdec_core *core = sess->core;
> > > +   int error_count;
> > > +   int num_frames;
> > > +   int i;
> > > +
> > > +   error_count = amvdec_read_dos(core, AV_SCRATCH_D);
> > > +   num_frames = (status >> 8) & 0xff;
> > > +   if (error_count) {
> > > +           dev_warn(core->dev,
> > > +                    "decoder error(s) happened, count %d\n", error_count);
> > > +           amvdec_write_dos(core, AV_SCRATCH_D, 0);
> > > +   }
> > > +
> > > +   for (i = 0; i < num_frames; i++) {
> > > +           u32 frame_status = amvdec_read_dos(core, AV_SCRATCH_1 + i * 4);
> > > +           u32 buffer_index = frame_status & BUF_IDX_MASK;
> > > +           u32 pic_struct = (frame_status >> PIC_STRUCT_BIT) &
> > > +                            PIC_STRUCT_MASK;
> > > +           u32 offset = (frame_status >> OFFSET_BIT) & OFFSET_MASK;
> > > +           u32 field = V4L2_FIELD_NONE;
> > > +
> > > +           /*
> > > +            * A buffer decode error means it was decoded,
> > > +            * but part of the picture will have artifacts.
> > > +            * Typical reason is a temporarily corrupted bitstream
> > > +            */
> > > +           if (frame_status & ERROR_FLAG)
> > > +                   dev_dbg(core->dev, "Buffer %d decode error\n",
> > > +                           buffer_index);
> > > +
> > > +           if (pic_struct == PIC_TOP_BOT)
> > > +                   field = V4L2_FIELD_INTERLACED_TB;
> > > +           else if (pic_struct == PIC_BOT_TOP)
> > > +                   field = V4L2_FIELD_INTERLACED_BT;
> > > +
> > > +           offset |= get_offset_msb(core, i);
> > > +           amvdec_dst_buf_done_idx(sess, buffer_index, offset, field);
> > > +   }
> > > +}
> > > +
> > > +static irqreturn_t codec_h264_threaded_isr(struct amvdec_session *sess)
> > > +{
> > > +   struct amvdec_core *core = sess->core;
> > > +   u32 status;
> > > +   u32 size;
> > > +   u8 cmd;
> > > +
> > > +   status = amvdec_read_dos(core, AV_SCRATCH_0);
> > > +   cmd = status & CMD_MASK;
> > > +
> > > +   switch (cmd) {
> > > +   case CMD_SRC_CHANGE:
> > > +           codec_h264_src_change(sess);
> > > +           break;
> > > +   case CMD_FRAMES_READY:
> > > +           codec_h264_frames_ready(sess, status);
> > > +           break;
> > > +   case CMD_FATAL_ERROR:
> > > +           dev_err(core->dev, "H.264 decoder fatal error\n");
> > > +           goto abort;
> > > +   case CMD_BAD_WIDTH:
> > > +           size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> > > +           dev_err(core->dev, "Unsupported video width: %u\n", size);
> > > +           goto abort;
> > > +   case CMD_BAD_HEIGHT:
> > > +           size = (amvdec_read_dos(core, AV_SCRATCH_1) + 1) * 16;
> > > +           dev_err(core->dev, "Unsupported video height: %u\n", size);
> > > +           goto abort;
> > > +   case 0: /* Unused but not worth printing for */
> > > +   case 9:
> > > +           break;
> > > +   default:
> > > +           dev_info(core->dev, "Unexpected H264 ISR: %08X\n", cmd);
> > > +           break;
> > > +   }
> > > +
> > > +   if (cmd && cmd != CMD_SRC_CHANGE)
> > > +           amvdec_write_dos(core, AV_SCRATCH_0, 0);
> > > +
> > > +   /* Decoder has some SEI data for us ; ignore */
> > > +   if (amvdec_read_dos(core, AV_SCRATCH_J) & SEI_DATA_READY)
> > > +           amvdec_write_dos(core, AV_SCRATCH_J, 0);
> > > +
> > > +   return IRQ_HANDLED;
> > > +abort:
> > > +   amvdec_abort(sess);
> > > +   return IRQ_HANDLED;
> > > +}
> > > +
> > > +static irqreturn_t codec_h264_isr(struct amvdec_session *sess)
> > > +{
> > > +   struct amvdec_core *core = sess->core;
> > > +
> > > +   amvdec_write_dos(core, ASSIST_MBOX1_CLR_REG, 1);
> > > +
> > > +   return IRQ_WAKE_THREAD;
> > > +}
> > > +
> > > +struct amvdec_codec_ops codec_h264_ops = {
> > > +   .start = codec_h264_start,
> > > +   .stop = codec_h264_stop,
> > > +   .load_extended_firmware = codec_h264_load_extended_firmware,
> > > +   .isr = codec_h264_isr,
> > > +   .threaded_isr = codec_h264_threaded_isr,
> > > +   .can_recycle = codec_h264_can_recycle,
> > > +   .recycle = codec_h264_recycle,
> > > +   .eos_sequence = codec_h264_eos_sequence,
> > > +   .resume = codec_h264_resume,
> > > +};
> > > diff --git a/drivers/staging/media/meson/vdec/codec_h264.h b/drivers/staging/media/meson/vdec/codec_h264.h
> > > new file mode 100644
> > > index 000000000000..7cb4fb86ff36
> > > --- /dev/null
> > > +++ b/drivers/staging/media/meson/vdec/codec_h264.h
> > > @@ -0,0 +1,14 @@
> > > +/* SPDX-License-Identifier: GPL-2.0+ */
> > > +/*
> > > + * Copyright (C) 2019 BayLibre, SAS
> > > + * Author: Maxime Jourdan <mjourdan@baylibre.com>
> > > + */
> > > +
> > > +#ifndef __MESON_VDEC_CODEC_H264_H_
> > > +#define __MESON_VDEC_CODEC_H264_H_
> > > +
> > > +#include "vdec.h"
> > > +
> > > +extern struct amvdec_codec_ops codec_h264_ops;
> > > +
> > > +#endif
> > > diff --git a/drivers/staging/media/meson/vdec/vdec_platform.c b/drivers/staging/media/meson/vdec/vdec_platform.c
> > > index accad8f8929a..025371ff8fae 100644
> > > --- a/drivers/staging/media/meson/vdec/vdec_platform.c
> > > +++ b/drivers/staging/media/meson/vdec/vdec_platform.c
> > > @@ -9,9 +9,22 @@
> > >
> > >  #include "vdec_1.h"
> > >  #include "codec_mpeg12.h"
> > > +#include "codec_h264.h"
> > >
> > >  static const struct amvdec_format vdec_formats_gxbb[] = {
> > >     {
> > > +           .pixfmt = V4L2_PIX_FMT_H264,
> > > +           .min_buffers = 2,
> > > +           .max_buffers = 24,
> > > +           .max_width = 1920,
> > > +           .max_height = 1080,
> > > +           .vdec_ops = &vdec_1_ops,
> > > +           .codec_ops = &codec_h264_ops,
> > > +           .firmware_path = "meson/vdec/gxbb_h264.bin",
> > > +           .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> > > +           .flags = V4L2_FMT_FLAG_COMPRESSED |
> > > +                    V4L2_FMT_FLAG_DYN_RESOLUTION,
> > > +   }, {
> > >             .pixfmt = V4L2_PIX_FMT_MPEG1,
> > >             .min_buffers = 8,
> > >             .max_buffers = 8,
> > > @@ -38,6 +51,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {
> > >
> > >  static const struct amvdec_format vdec_formats_gxl[] = {
> > >     {
> > > +           .pixfmt = V4L2_PIX_FMT_H264,
> > > +           .min_buffers = 2,
> > > +           .max_buffers = 24,
> > > +           .max_width = 3840,
> > > +           .max_height = 2160,
> > > +           .vdec_ops = &vdec_1_ops,
> > > +           .codec_ops = &codec_h264_ops,
> > > +           .firmware_path = "meson/vdec/gxl_h264.bin",
> > > +           .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> > > +           .flags = V4L2_FMT_FLAG_COMPRESSED |
> > > +                    V4L2_FMT_FLAG_DYN_RESOLUTION,
> > > +   }, {
> > >             .pixfmt = V4L2_PIX_FMT_MPEG1,
> > >             .min_buffers = 8,
> > >             .max_buffers = 8,
> > > @@ -64,6 +89,18 @@ static const struct amvdec_format vdec_formats_gxl[] = {
> > >
> > >  static const struct amvdec_format vdec_formats_gxm[] = {
> > >     {
> > > +           .pixfmt = V4L2_PIX_FMT_H264,
> > > +           .min_buffers = 2,
> > > +           .max_buffers = 24,
> > > +           .max_width = 3840,
> > > +           .max_height = 2160,
> > > +           .vdec_ops = &vdec_1_ops,
> > > +           .codec_ops = &codec_h264_ops,
> > > +           .firmware_path = "meson/vdec/gxm_h264.bin",
> > > +           .pixfmts_cap = { V4L2_PIX_FMT_NV12M, 0 },
> > > +           .flags = V4L2_FMT_FLAG_COMPRESSED |
> > > +                    V4L2_FMT_FLAG_DYN_RESOLUTION,
> > > +   }, {
> > >             .pixfmt = V4L2_PIX_FMT_MPEG1,
> > >             .min_buffers = 8,
> > >             .max_buffers = 8,

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Hans Verkuil]

On 10/7/19 4:59 PM, Maxime Jourdan wrote:
> Hello,
> 
> This patch series aims to bring H.264 support as well as compliance update
> to the amlogic stateful video decoder driver.
> 
> There is 1 issue that remains currently:
> 
>  - The following codepath had to be commented out from v4l2-compliance as
> it led to stalling:
> 
> if (node->codec_mask & STATEFUL_DECODER) {
> 	struct v4l2_decoder_cmd cmd;
> 	buffer buf_cap(m2m_q);
> 
> 	memset(&cmd, 0, sizeof(cmd));
> 	cmd.cmd = V4L2_DEC_CMD_STOP;
> 
> 	/* No buffers are queued, call STREAMON, then STOP */
> 	fail_on_test(node->streamon(q.g_type()));
> 	fail_on_test(node->streamon(m2m_q.g_type()));
> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> 
> 	fail_on_test(buf_cap.querybuf(node, 0));
> 	fail_on_test(buf_cap.qbuf(node));
> 	fail_on_test(buf_cap.dqbuf(node));
> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
> 		fail_on_test(buf_cap.g_bytesused(p));
> 	fail_on_test(node->streamoff(q.g_type()));
> 	fail_on_test(node->streamoff(m2m_q.g_type()));
> 
> 	/* Call STREAMON, queue one CAPTURE buffer, then STOP */
> 	fail_on_test(node->streamon(q.g_type()));
> 	fail_on_test(node->streamon(m2m_q.g_type()));
> 	fail_on_test(buf_cap.querybuf(node, 0));
> 	fail_on_test(buf_cap.qbuf(node));
> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> 
> 	fail_on_test(buf_cap.dqbuf(node));
> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
> 		fail_on_test(buf_cap.g_bytesused(p));
> 	fail_on_test(node->streamoff(q.g_type()));
> 	fail_on_test(node->streamoff(m2m_q.g_type()));
> }
> 
> The reason for this is because the driver has a limitation where all
> capturebuffers must be queued to the driver before STREAMON is effective.
> The firmware needs to know in advance what all the buffers are before
> starting to decode.
> This limitation is enforced via q->min_buffers_needed.
> As such, in this compliance codepath, STREAMON is never actually called
> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));

That's interesting. I will have to look more closely at this.

> 
> 
> One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
> by v4l2-compliance, so it was left out for the test. However, it is
> present in the patch series.

It is definitely recognized by v4l2-compliance.

> 
> The second patch has 3 "Alignment should match open parenthesis" lines
> where I preferred to keep them that way.
> 
> Thanks Stanimir for sharing your HDR file creation tools, this was very
> helpful :).
> 
> Maxime
> 
> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
> v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits

But this SHA isn't in the v4l-utils repo, so this makes me wonder where you
got this repo from.

Regards,

	Hans

> 
> Compliance test for meson-vdec device /dev/video0:
> 
> Driver Info:
> 	Driver name      : meson-vdec
> 	Card type        : Amlogic Video Decoder
> 	Bus info         : platform:meson-vdec
> 	Driver version   : 5.4.0
> 	Capabilities     : 0x84204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 	Detected Stateful Decoder
> 
> Required ioctls:
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video0 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test VIDIOC_EXPBUF: OK
> 	test Requests: OK (Not Supported)
> 
> Test input 0:
> 
> Streaming ioctls:
> 	test read/write: OK (Not Supported)
> 	test blocking wait: OK
> 	Video Capture Multiplanar: Captured 250 buffers   
> 	test MMAP (select): OK
> 	Video Capture Multiplanar: Captured 250 buffers   
> 	test MMAP (epoll): OK
> 	test USERPTR (select): OK (Not Supported)
> 	test DMABUF: Cannot test, specify --expbuf-device
> 
> Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
> 
> Maxime Jourdan (2):
>   media: meson: vdec: bring up to compliance
>   media: meson: vdec: add H.264 decoding support
> 
>  drivers/staging/media/meson/vdec/Makefile     |   2 +-
>  drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
>  drivers/staging/media/meson/vdec/codec_h264.h |  14 +
>  drivers/staging/media/meson/vdec/esparser.c   |  34 +-
>  drivers/staging/media/meson/vdec/vdec.c       |  70 ++-
>  drivers/staging/media/meson/vdec/vdec.h       |  14 +-
>  .../staging/media/meson/vdec/vdec_helpers.c   |  85 ++-
>  .../staging/media/meson/vdec/vdec_helpers.h   |   6 +-
>  .../staging/media/meson/vdec/vdec_platform.c  |  43 ++
>  9 files changed, 654 insertions(+), 96 deletions(-)
>  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
>  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
> 

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Maxime Jourdan]

On 07/10/2019 17:12, Hans Verkuil wrote:
> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
>> Hello,
>>
>> This patch series aims to bring H.264 support as well as compliance update
>> to the amlogic stateful video decoder driver.
>>
>> There is 1 issue that remains currently:
>>
>>   - The following codepath had to be commented out from v4l2-compliance as
>> it led to stalling:
>>
>> if (node->codec_mask & STATEFUL_DECODER) {
>> 	struct v4l2_decoder_cmd cmd;
>> 	buffer buf_cap(m2m_q);
>>
>> 	memset(&cmd, 0, sizeof(cmd));
>> 	cmd.cmd = V4L2_DEC_CMD_STOP;
>>
>> 	/* No buffers are queued, call STREAMON, then STOP */
>> 	fail_on_test(node->streamon(q.g_type()));
>> 	fail_on_test(node->streamon(m2m_q.g_type()));
>> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>
>> 	fail_on_test(buf_cap.querybuf(node, 0));
>> 	fail_on_test(buf_cap.qbuf(node));
>> 	fail_on_test(buf_cap.dqbuf(node));
>> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>> 		fail_on_test(buf_cap.g_bytesused(p));
>> 	fail_on_test(node->streamoff(q.g_type()));
>> 	fail_on_test(node->streamoff(m2m_q.g_type()));
>>
>> 	/* Call STREAMON, queue one CAPTURE buffer, then STOP */
>> 	fail_on_test(node->streamon(q.g_type()));
>> 	fail_on_test(node->streamon(m2m_q.g_type()));
>> 	fail_on_test(buf_cap.querybuf(node, 0));
>> 	fail_on_test(buf_cap.qbuf(node));
>> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>
>> 	fail_on_test(buf_cap.dqbuf(node));
>> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>> 		fail_on_test(buf_cap.g_bytesused(p));
>> 	fail_on_test(node->streamoff(q.g_type()));
>> 	fail_on_test(node->streamoff(m2m_q.g_type()));
>> }
>>
>> The reason for this is because the driver has a limitation where all
>> capturebuffers must be queued to the driver before STREAMON is effective.
>> The firmware needs to know in advance what all the buffers are before
>> starting to decode.
>> This limitation is enforced via q->min_buffers_needed.
>> As such, in this compliance codepath, STREAMON is never actually called
>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
> 
> That's interesting. I will have to look more closely at this.
> 
>>
>>
>> One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
>> by v4l2-compliance, so it was left out for the test. However, it is
>> present in the patch series.
> 
> It is definitely recognized by v4l2-compliance.
> 
>>
>> The second patch has 3 "Alignment should match open parenthesis" lines
>> where I preferred to keep them that way.
>>
>> Thanks Stanimir for sharing your HDR file creation tools, this was very
>> helpful :).
>>
>> Maxime
>>
>> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
>> v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits
> 
> But this SHA isn't in the v4l-utils repo, so this makes me wonder where you
> got this repo from.
> 

I am based off the hverkuil/vicodec branch. The SHA I am on is actually 
05387265053bc6f9 ("test-media: add vicodec tests"), but it wasn't 
updated as I found out it requires a new bootstrap to refresh the SHA. 
Maybe some rebasing at some point got rid of a162244d.

I started fresh and ran it again. As you can see, 
V4L2_FMT_FLAG_DYN_RESOLUTION is still problematic (removing it makes all 
the checks pass):

-------------------------------
# v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
v4l2-compliance SHA: 05387265053bc6f9c8c98e112543adb28ae39cfa, 64 bits

Compliance test for meson-vdec device /dev/video0:

Driver Info:
	Driver name      : meson-vdec
	Card type        : Amlogic Video Decoder
	Bus info         : platform:meson-vdec
	Driver version   : 5.4.0
	Capabilities     : 0x84204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
	Detected Stateful Decoder

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
		fail: v4l2-test-formats.cpp(263): unknown flag 00000009 returned
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
		fail: v4l2-test-formats.cpp(457): pixelformat 34363248 (H264) for 
buftype 10 not reported by ENUM_FMT
	test VIDIOC_G_FMT: FAIL
		fail: v4l2-test-formats.cpp(457): pixelformat 34363248 (H264) for 
buftype 10 not reported by ENUM_FMT
	test VIDIOC_TRY_FMT: FAIL
		fail: v4l2-test-formats.cpp(457): pixelformat 3247504d (MPG2) for 
buftype 10 not reported by ENUM_FMT
	test VIDIOC_S_FMT: FAIL
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
	test read/write: OK (Not Supported)
	test blocking wait: OK
	Video Capture Multiplanar: Captured 250 buffers
	test MMAP (select): OK
	Video Capture Multiplanar: Captured 250 buffers
	test MMAP (epoll): OK
	test USERPTR (select): OK (Not Supported)
	test DMABUF: Cannot test, specify --expbuf-device

Total for meson-vdec device /dev/video0: 49, Succeeded: 45, Failed: 4, 
Warnings: 0

-------------------------------

Should I be using another branch than vicodec ?


> Regards,
> 
> 	Hans
> 

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Hans Verkuil]

On 10/7/19 6:24 PM, Maxime Jourdan wrote:
> On 07/10/2019 17:12, Hans Verkuil wrote:
>> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
>>> Hello,
>>>
>>> This patch series aims to bring H.264 support as well as compliance update
>>> to the amlogic stateful video decoder driver.
>>>
>>> There is 1 issue that remains currently:
>>>
>>>   - The following codepath had to be commented out from v4l2-compliance as
>>> it led to stalling:
>>>
>>> if (node->codec_mask & STATEFUL_DECODER) {
>>>     struct v4l2_decoder_cmd cmd;
>>>     buffer buf_cap(m2m_q);
>>>
>>>     memset(&cmd, 0, sizeof(cmd));
>>>     cmd.cmd = V4L2_DEC_CMD_STOP;
>>>
>>>     /* No buffers are queued, call STREAMON, then STOP */
>>>     fail_on_test(node->streamon(q.g_type()));
>>>     fail_on_test(node->streamon(m2m_q.g_type()));
>>>     fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>
>>>     fail_on_test(buf_cap.querybuf(node, 0));
>>>     fail_on_test(buf_cap.qbuf(node));
>>>     fail_on_test(buf_cap.dqbuf(node));
>>>     fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>     for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>         fail_on_test(buf_cap.g_bytesused(p));
>>>     fail_on_test(node->streamoff(q.g_type()));
>>>     fail_on_test(node->streamoff(m2m_q.g_type()));
>>>
>>>     /* Call STREAMON, queue one CAPTURE buffer, then STOP */
>>>     fail_on_test(node->streamon(q.g_type()));
>>>     fail_on_test(node->streamon(m2m_q.g_type()));
>>>     fail_on_test(buf_cap.querybuf(node, 0));
>>>     fail_on_test(buf_cap.qbuf(node));
>>>     fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>
>>>     fail_on_test(buf_cap.dqbuf(node));
>>>     fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>     for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>         fail_on_test(buf_cap.g_bytesused(p));
>>>     fail_on_test(node->streamoff(q.g_type()));
>>>     fail_on_test(node->streamoff(m2m_q.g_type()));
>>> }
>>>
>>> The reason for this is because the driver has a limitation where all
>>> capturebuffers must be queued to the driver before STREAMON is effective.
>>> The firmware needs to know in advance what all the buffers are before
>>> starting to decode.
>>> This limitation is enforced via q->min_buffers_needed.
>>> As such, in this compliance codepath, STREAMON is never actually called
>>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
>>
>> That's interesting. I will have to look more closely at this.
>>
>>>
>>>
>>> One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
>>> by v4l2-compliance, so it was left out for the test. However, it is
>>> present in the patch series.
>>
>> It is definitely recognized by v4l2-compliance.
>>
>>>
>>> The second patch has 3 "Alignment should match open parenthesis" lines
>>> where I preferred to keep them that way.
>>>
>>> Thanks Stanimir for sharing your HDR file creation tools, this was very
>>> helpful :).
>>>
>>> Maxime
>>>
>>> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
>>> v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits
>>
>> But this SHA isn't in the v4l-utils repo, so this makes me wonder where you
>> got this repo from.
>>
> 
> I am based off the hverkuil/vicodec branch. The SHA I am on is actually 05387265053bc6f9 ("test-media: add vicodec tests"), but it wasn't updated as I found out it requires a new bootstrap to refresh
> the SHA. Maybe some rebasing at some point got rid of a162244d.

Don't use the hverkuil/vicodec branch. Everything there has been merged into the
regular v4l-utils repo some time ago. So just clone git://linuxtv.org/v4l-utils.git
and use that.

Regards,

	Hans

> 
> I started fresh and ran it again. As you can see, V4L2_FMT_FLAG_DYN_RESOLUTION is still problematic (removing it makes all the checks pass):
> 
> -------------------------------
> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
> v4l2-compliance SHA: 05387265053bc6f9c8c98e112543adb28ae39cfa, 64 bits
> 
> Compliance test for meson-vdec device /dev/video0:
> 
> Driver Info:
>     Driver name      : meson-vdec
>     Card type        : Amlogic Video Decoder
>     Bus info         : platform:meson-vdec
>     Driver version   : 5.4.0
>     Capabilities     : 0x84204000
>         Video Memory-to-Memory Multiplanar
>         Streaming
>         Extended Pix Format
>         Device Capabilities
>     Device Caps      : 0x04204000
>         Video Memory-to-Memory Multiplanar
>         Streaming
>         Extended Pix Format
>     Detected Stateful Decoder
> 
> Required ioctls:
>     test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>     test second /dev/video0 open: OK
>     test VIDIOC_QUERYCAP: OK
>     test VIDIOC_G/S_PRIORITY: OK
>     test for unlimited opens: OK
> 
> Debug ioctls:
>     test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>     test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>     test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>     test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>     test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>     test VIDIOC_ENUMAUDIO: OK (Not Supported)
>     test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>     test VIDIOC_G/S_AUDIO: OK (Not Supported)
>     Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>     test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>     test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>     test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>     test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>     test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>     Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>     test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>     test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>     test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>     test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
>     test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>     test VIDIOC_QUERYCTRL: OK
>     test VIDIOC_G/S_CTRL: OK
>     test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>     test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>     test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>     Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
>         fail: v4l2-test-formats.cpp(263): unknown flag 00000009 returned
>     test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL
>     test VIDIOC_G/S_PARM: OK (Not Supported)
>     test VIDIOC_G_FBUF: OK (Not Supported)
>         fail: v4l2-test-formats.cpp(457): pixelformat 34363248 (H264) for buftype 10 not reported by ENUM_FMT
>     test VIDIOC_G_FMT: FAIL
>         fail: v4l2-test-formats.cpp(457): pixelformat 34363248 (H264) for buftype 10 not reported by ENUM_FMT
>     test VIDIOC_TRY_FMT: FAIL
>         fail: v4l2-test-formats.cpp(457): pixelformat 3247504d (MPG2) for buftype 10 not reported by ENUM_FMT
>     test VIDIOC_S_FMT: FAIL
>     test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>     test Cropping: OK (Not Supported)
>     test Composing: OK (Not Supported)
>     test Scaling: OK
> 
> Codec ioctls:
>     test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>     test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>     test VIDIOC_(TRY_)DECODER_CMD: OK
> 
> Buffer ioctls:
>     test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>     test VIDIOC_EXPBUF: OK
>     test Requests: OK (Not Supported)
> 
> Test input 0:
> 
> Streaming ioctls:
>     test read/write: OK (Not Supported)
>     test blocking wait: OK
>     Video Capture Multiplanar: Captured 250 buffers
>     test MMAP (select): OK
>     Video Capture Multiplanar: Captured 250 buffers
>     test MMAP (epoll): OK
>     test USERPTR (select): OK (Not Supported)
>     test DMABUF: Cannot test, specify --expbuf-device
> 
> Total for meson-vdec device /dev/video0: 49, Succeeded: 45, Failed: 4, Warnings: 0
> 
> -------------------------------
> 
> Should I be using another branch than vicodec ?
> 
> 
>> Regards,
>>
>>     Hans
>>

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Maxime Jourdan]

On 07/10/2019 18:39, Hans Verkuil wrote:
> On 10/7/19 6:24 PM, Maxime Jourdan wrote:
>> On 07/10/2019 17:12, Hans Verkuil wrote:
>>> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
>>>> Hello,
>>>>
>>>> This patch series aims to bring H.264 support as well as compliance update
>>>> to the amlogic stateful video decoder driver.
>>>>
>>>> There is 1 issue that remains currently:
>>>>
>>>>    - The following codepath had to be commented out from v4l2-compliance as
>>>> it led to stalling:
>>>>
>>>> if (node->codec_mask & STATEFUL_DECODER) {
>>>>      struct v4l2_decoder_cmd cmd;
>>>>      buffer buf_cap(m2m_q);
>>>>
>>>>      memset(&cmd, 0, sizeof(cmd));
>>>>      cmd.cmd = V4L2_DEC_CMD_STOP;
>>>>
>>>>      /* No buffers are queued, call STREAMON, then STOP */
>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>
>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>
>>>>      /* Call STREAMON, queue one CAPTURE buffer, then STOP */
>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>
>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>> }
>>>>
>>>> The reason for this is because the driver has a limitation where all
>>>> capturebuffers must be queued to the driver before STREAMON is effective.
>>>> The firmware needs to know in advance what all the buffers are before
>>>> starting to decode.
>>>> This limitation is enforced via q->min_buffers_needed.
>>>> As such, in this compliance codepath, STREAMON is never actually called
>>>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
>>>
>>> That's interesting. I will have to look more closely at this.
>>>
>>>>
>>>>
>>>> One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
>>>> by v4l2-compliance, so it was left out for the test. However, it is
>>>> present in the patch series.
>>>
>>> It is definitely recognized by v4l2-compliance.
>>>
>>>>
>>>> The second patch has 3 "Alignment should match open parenthesis" lines
>>>> where I preferred to keep them that way.
>>>>
>>>> Thanks Stanimir for sharing your HDR file creation tools, this was very
>>>> helpful :).
>>>>
>>>> Maxime
>>>>
>>>> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
>>>> v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits
>>>
>>> But this SHA isn't in the v4l-utils repo, so this makes me wonder where you
>>> got this repo from.
>>>
>>
>> I am based off the hverkuil/vicodec branch. The SHA I am on is actually 05387265053bc6f9 ("test-media: add vicodec tests"), but it wasn't updated as I found out it requires a new bootstrap to refresh
>> the SHA. Maybe some rebasing at some point got rid of a162244d.
> 
> Don't use the hverkuil/vicodec branch. Everything there has been merged into the
> regular v4l-utils repo some time ago. So just clone git://linuxtv.org/v4l-utils.git
> and use that.
> 

Here is v4l2-compliance master without removing the flag from the driver 
this time. I however had to keep the codepath mentionned earlier commented.

# v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
v4l2-compliance SHA: fd74ecee9020fcf80b3b9628f277d9311b443395, 64 bits

Compliance test for meson-vdec device /dev/video0:

Driver Info:
	Driver name      : meson-vdec
	Card type        : Amlogic Video Decoder
	Bus info         : platform:meson-vdec
	Driver version   : 5.4.0
	Capabilities     : 0x84204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
	Detected Stateful Decoder

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
	test read/write: OK (Not Supported)
	test blocking wait: OK
	Video Capture Multiplanar: Captured 250 buffers
	test MMAP (select): OK
	Video Capture Multiplanar: Captured 250 buffers
	test MMAP (epoll): OK
	test USERPTR (select): OK (Not Supported)
	test DMABUF: Cannot test, specify --expbuf-device

Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, 
Warnings: 0

Maxime

 >
> Regards,
> 
> 	Hans
> 

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Hans Verkuil]

On 10/8/19 3:40 PM, Maxime Jourdan wrote:
> On 07/10/2019 18:39, Hans Verkuil wrote:
>> On 10/7/19 6:24 PM, Maxime Jourdan wrote:
>>> On 07/10/2019 17:12, Hans Verkuil wrote:
>>>> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
>>>>> Hello,
>>>>>
>>>>> This patch series aims to bring H.264 support as well as compliance update
>>>>> to the amlogic stateful video decoder driver.
>>>>>
>>>>> There is 1 issue that remains currently:
>>>>>
>>>>>    - The following codepath had to be commented out from v4l2-compliance as
>>>>> it led to stalling:
>>>>>
>>>>> if (node->codec_mask & STATEFUL_DECODER) {
>>>>>      struct v4l2_decoder_cmd cmd;
>>>>>      buffer buf_cap(m2m_q);
>>>>>
>>>>>      memset(&cmd, 0, sizeof(cmd));
>>>>>      cmd.cmd = V4L2_DEC_CMD_STOP;
>>>>>
>>>>>      /* No buffers are queued, call STREAMON, then STOP */
>>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>
>>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>>
>>>>>      /* Call STREAMON, queue one CAPTURE buffer, then STOP */
>>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>
>>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>> }
>>>>>
>>>>> The reason for this is because the driver has a limitation where all
>>>>> capturebuffers must be queued to the driver before STREAMON is effective.
>>>>> The firmware needs to know in advance what all the buffers are before
>>>>> starting to decode.
>>>>> This limitation is enforced via q->min_buffers_needed.
>>>>> As such, in this compliance codepath, STREAMON is never actually called
>>>>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
>>>>
>>>> That's interesting. I will have to look more closely at this.
>>>>
>>>>>
>>>>>
>>>>> One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
>>>>> by v4l2-compliance, so it was left out for the test. However, it is
>>>>> present in the patch series.
>>>>
>>>> It is definitely recognized by v4l2-compliance.
>>>>
>>>>>
>>>>> The second patch has 3 "Alignment should match open parenthesis" lines
>>>>> where I preferred to keep them that way.
>>>>>
>>>>> Thanks Stanimir for sharing your HDR file creation tools, this was very
>>>>> helpful :).
>>>>>
>>>>> Maxime
>>>>>
>>>>> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
>>>>> v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits
>>>>
>>>> But this SHA isn't in the v4l-utils repo, so this makes me wonder where you
>>>> got this repo from.
>>>>
>>>
>>> I am based off the hverkuil/vicodec branch. The SHA I am on is actually 05387265053bc6f9 ("test-media: add vicodec tests"), but it wasn't updated as I found out it requires a new bootstrap to refresh
>>> the SHA. Maybe some rebasing at some point got rid of a162244d.
>>
>> Don't use the hverkuil/vicodec branch. Everything there has been merged into the
>> regular v4l-utils repo some time ago. So just clone git://linuxtv.org/v4l-utils.git
>> and use that.
>>
> 
> Here is v4l2-compliance master without removing the flag from the driver this time. I however had to keep the codepath mentionned earlier commented.
> 
> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
> v4l2-compliance SHA: fd74ecee9020fcf80b3b9628f277d9311b443395, 64 bits

Ah, that looks much better :-)

The stall on 'fail_on_test(buf_cap.dqbuf(node));' I understand, and is something I need to take a closer look at.

Regards,

	Hans

> 
> Compliance test for meson-vdec device /dev/video0:
> 
> Driver Info:
>     Driver name      : meson-vdec
>     Card type        : Amlogic Video Decoder
>     Bus info         : platform:meson-vdec
>     Driver version   : 5.4.0
>     Capabilities     : 0x84204000
>         Video Memory-to-Memory Multiplanar
>         Streaming
>         Extended Pix Format
>         Device Capabilities
>     Device Caps      : 0x04204000
>         Video Memory-to-Memory Multiplanar
>         Streaming
>         Extended Pix Format
>     Detected Stateful Decoder
> 
> Required ioctls:
>     test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
>     test second /dev/video0 open: OK
>     test VIDIOC_QUERYCAP: OK
>     test VIDIOC_G/S_PRIORITY: OK
>     test for unlimited opens: OK
> 
> Debug ioctls:
>     test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>     test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
>     test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>     test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>     test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>     test VIDIOC_ENUMAUDIO: OK (Not Supported)
>     test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>     test VIDIOC_G/S_AUDIO: OK (Not Supported)
>     Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
>     test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>     test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>     test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>     test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>     test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>     Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
>     test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>     test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>     test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>     test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
>     test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>     test VIDIOC_QUERYCTRL: OK
>     test VIDIOC_G/S_CTRL: OK
>     test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>     test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>     test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>     Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
>     test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>     test VIDIOC_G/S_PARM: OK (Not Supported)
>     test VIDIOC_G_FBUF: OK (Not Supported)
>     test VIDIOC_G_FMT: OK
>     test VIDIOC_TRY_FMT: OK
>     test VIDIOC_S_FMT: OK
>     test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>     test Cropping: OK (Not Supported)
>     test Composing: OK (Not Supported)
>     test Scaling: OK
> 
> Codec ioctls:
>     test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>     test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>     test VIDIOC_(TRY_)DECODER_CMD: OK
> 
> Buffer ioctls:
>     test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>     test VIDIOC_EXPBUF: OK
>     test Requests: OK (Not Supported)
> 
> Test input 0:
> 
> Streaming ioctls:
>     test read/write: OK (Not Supported)
>     test blocking wait: OK
>     Video Capture Multiplanar: Captured 250 buffers
>     test MMAP (select): OK
>     Video Capture Multiplanar: Captured 250 buffers
>     test MMAP (epoll): OK
>     test USERPTR (select): OK (Not Supported)
>     test DMABUF: Cannot test, specify --expbuf-device
> 
> Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
> 
> Maxime
> 
>>
>> Regards,
>>
>>     Hans
>>
> 

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Hans Verkuil]

On 10/8/19 3:40 PM, Maxime Jourdan wrote:
> On 07/10/2019 18:39, Hans Verkuil wrote:
>> On 10/7/19 6:24 PM, Maxime Jourdan wrote:
>>> On 07/10/2019 17:12, Hans Verkuil wrote:
>>>> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
>>>>> Hello,
>>>>>
>>>>> This patch series aims to bring H.264 support as well as compliance update
>>>>> to the amlogic stateful video decoder driver.
>>>>>
>>>>> There is 1 issue that remains currently:
>>>>>
>>>>>    - The following codepath had to be commented out from v4l2-compliance as
>>>>> it led to stalling:
>>>>>
>>>>> if (node->codec_mask & STATEFUL_DECODER) {
>>>>>      struct v4l2_decoder_cmd cmd;
>>>>>      buffer buf_cap(m2m_q);
>>>>>
>>>>>      memset(&cmd, 0, sizeof(cmd));
>>>>>      cmd.cmd = V4L2_DEC_CMD_STOP;
>>>>>
>>>>>      /* No buffers are queued, call STREAMON, then STOP */
>>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>
>>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>>
>>>>>      /* Call STREAMON, queue one CAPTURE buffer, then STOP */
>>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>
>>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>> }
>>>>>
>>>>> The reason for this is because the driver has a limitation where all
>>>>> capturebuffers must be queued to the driver before STREAMON is effective.
>>>>> The firmware needs to know in advance what all the buffers are before
>>>>> starting to decode.
>>>>> This limitation is enforced via q->min_buffers_needed.
>>>>> As such, in this compliance codepath, STREAMON is never actually called
>>>>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
>>>>
>>>> That's interesting. I will have to look more closely at this.

This requires a helper function in videobuf2-v4l2.c.

In vdec_decoder_cmd you would need code like this:

	if (!vb2_start_streaming_called(&capture_queue)) {
		vb2_dequeue_empty_last_buf(&capture_queue);
		return 0;
	}

The vb2_dequeue_empty_last_buf (function name can probably be improved upon!)
does nothing if no capture buffers were queued, otherwise it takes the first
buffer, sets the LAST flag and sets bytesused to 0 and marks it as DONE.

The driver cannot do this directly, since the buffers were never queued to the
driver and are owned by vb2.

This is something that needs to be done for any codec driver that sets
min_buffers_needed to a value > 1.

The vb2 function would look something like this:

void vb2_dqbuf_empty_last_buf(struct vb2_queue *q)
{
        struct vb2_buffer *vb;
        struct vb2_v4l2_buffer *vbuf;
        unsigned int i;

        if (WARN_ON(q->is_output))
                return;
        if (list_empty(&q->queued_list))
                return;
        vb = list_first_entry(&q->queued_list, struct vb2_buffer, queued_entry);
        list_del(&vb->queued_entry);
        for (i = 0; i < vb->num_planes; i++)
                vb2_set_plane_payload(vb, i, 0)
        vbuf = to_vb2_v4l2_buffer(vb);
        vbuf->flags |= V4L2_BUF_FLAG_LAST;
        vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
}
EXPORT_SYMBOL_GPL(vb2_dqbuf_empty_last_buf);

Neither compiled, nor tested, and I think this should be in v4l2-mem2mem.c instead of
in videobuf2-v4l2.c since this is very m2m specific.

So see this as a suggestion :-)

Anyway, the key take-away from this is that userspace does not know if your driver
behaves the way it does, so STOP should still produce a sane expected result.

Which in this is just a single empty capture buffer marked LAST.

Regards,

	Hans

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Maxime Jourdan]

On Wed, Oct 9, 2019 at 2:01 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 10/8/19 3:40 PM, Maxime Jourdan wrote:
> > On 07/10/2019 18:39, Hans Verkuil wrote:
> >> On 10/7/19 6:24 PM, Maxime Jourdan wrote:
> >>> On 07/10/2019 17:12, Hans Verkuil wrote:
> >>>> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
> >>>>> Hello,
> >>>>>
> >>>>> This patch series aims to bring H.264 support as well as compliance update
> >>>>> to the amlogic stateful video decoder driver.
> >>>>>
> >>>>> There is 1 issue that remains currently:
> >>>>>
> >>>>>    - The following codepath had to be commented out from v4l2-compliance as
> >>>>> it led to stalling:
> >>>>>
> >>>>> if (node->codec_mask & STATEFUL_DECODER) {
> >>>>>      struct v4l2_decoder_cmd cmd;
> >>>>>      buffer buf_cap(m2m_q);
> >>>>>
> >>>>>      memset(&cmd, 0, sizeof(cmd));
> >>>>>      cmd.cmd = V4L2_DEC_CMD_STOP;
> >>>>>
> >>>>>      /* No buffers are queued, call STREAMON, then STOP */
> >>>>>      fail_on_test(node->streamon(q.g_type()));
> >>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
> >>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> >>>>>
> >>>>>      fail_on_test(buf_cap.querybuf(node, 0));
> >>>>>      fail_on_test(buf_cap.qbuf(node));
> >>>>>      fail_on_test(buf_cap.dqbuf(node));
> >>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> >>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
> >>>>>          fail_on_test(buf_cap.g_bytesused(p));
> >>>>>      fail_on_test(node->streamoff(q.g_type()));
> >>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
> >>>>>
> >>>>>      /* Call STREAMON, queue one CAPTURE buffer, then STOP */
> >>>>>      fail_on_test(node->streamon(q.g_type()));
> >>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
> >>>>>      fail_on_test(buf_cap.querybuf(node, 0));
> >>>>>      fail_on_test(buf_cap.qbuf(node));
> >>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> >>>>>
> >>>>>      fail_on_test(buf_cap.dqbuf(node));
> >>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> >>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
> >>>>>          fail_on_test(buf_cap.g_bytesused(p));
> >>>>>      fail_on_test(node->streamoff(q.g_type()));
> >>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
> >>>>> }
> >>>>>
> >>>>> The reason for this is because the driver has a limitation where all
> >>>>> capturebuffers must be queued to the driver before STREAMON is effective.
> >>>>> The firmware needs to know in advance what all the buffers are before
> >>>>> starting to decode.
> >>>>> This limitation is enforced via q->min_buffers_needed.
> >>>>> As such, in this compliance codepath, STREAMON is never actually called
> >>>>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
> >>>>
> >>>> That's interesting. I will have to look more closely at this.
>
> This requires a helper function in videobuf2-v4l2.c.
>
> In vdec_decoder_cmd you would need code like this:
>
>         if (!vb2_start_streaming_called(&capture_queue)) {
>                 vb2_dequeue_empty_last_buf(&capture_queue);
>                 return 0;
>         }
>
> The vb2_dequeue_empty_last_buf (function name can probably be improved upon!)
> does nothing if no capture buffers were queued, otherwise it takes the first
> buffer, sets the LAST flag and sets bytesused to 0 and marks it as DONE.
>
> The driver cannot do this directly, since the buffers were never queued to the
> driver and are owned by vb2.
>
> This is something that needs to be done for any codec driver that sets
> min_buffers_needed to a value > 1.
>
> The vb2 function would look something like this:
>
> void vb2_dqbuf_empty_last_buf(struct vb2_queue *q)
> {
>         struct vb2_buffer *vb;
>         struct vb2_v4l2_buffer *vbuf;
>         unsigned int i;
>
>         if (WARN_ON(q->is_output))
>                 return;
>         if (list_empty(&q->queued_list))
>                 return;
>         vb = list_first_entry(&q->queued_list, struct vb2_buffer, queued_entry);
>         list_del(&vb->queued_entry);
>         for (i = 0; i < vb->num_planes; i++)
>                 vb2_set_plane_payload(vb, i, 0)
>         vbuf = to_vb2_v4l2_buffer(vb);
>         vbuf->flags |= V4L2_BUF_FLAG_LAST;
>         vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
> }
> EXPORT_SYMBOL_GPL(vb2_dqbuf_empty_last_buf);
>
> Neither compiled, nor tested, and I think this should be in v4l2-mem2mem.c instead of
> in videobuf2-v4l2.c since this is very m2m specific.
>
> So see this as a suggestion :-)
>
> Anyway, the key take-away from this is that userspace does not know if your driver
> behaves the way it does, so STOP should still produce a sane expected result.
>
> Which in this is just a single empty capture buffer marked LAST.

Thanks, this makes sense. It doesn't quite fit the current usage
unfortunately as the test in v4l2-compliance goes like this:

fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
fail_on_test(buf_cap.querybuf(node, 0));
fail_on_test(buf_cap.qbuf(node));
fail_on_test(buf_cap.dqbuf(node));
fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));

Since the buffer is queued after issuing the stop cmd, it is not
possible to flag it as DONE in vdec_decoder_cmd.

A solution would be to hijack vidioc_qbuf and flag the buffer if a
stop has been issued previously and the capture queue is not
streaming. Would that be okay ?

Maxime

>
> Regards,
>
>         Hans

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Maxime Jourdan]

On 18/10/2019 09:50, Maxime Jourdan wrote:
> On Wed, Oct 9, 2019 at 2:01 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 10/8/19 3:40 PM, Maxime Jourdan wrote:
>>> On 07/10/2019 18:39, Hans Verkuil wrote:
>>>> On 10/7/19 6:24 PM, Maxime Jourdan wrote:
>>>>> On 07/10/2019 17:12, Hans Verkuil wrote:
>>>>>> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> This patch series aims to bring H.264 support as well as compliance update
>>>>>>> to the amlogic stateful video decoder driver.
>>>>>>>
>>>>>>> There is 1 issue that remains currently:
>>>>>>>
>>>>>>>     - The following codepath had to be commented out from v4l2-compliance as
>>>>>>> it led to stalling:
>>>>>>>
>>>>>>> if (node->codec_mask & STATEFUL_DECODER) {
>>>>>>>       struct v4l2_decoder_cmd cmd;
>>>>>>>       buffer buf_cap(m2m_q);
>>>>>>>
>>>>>>>       memset(&cmd, 0, sizeof(cmd));
>>>>>>>       cmd.cmd = V4L2_DEC_CMD_STOP;
>>>>>>>
>>>>>>>       /* No buffers are queued, call STREAMON, then STOP */
>>>>>>>       fail_on_test(node->streamon(q.g_type()));
>>>>>>>       fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>>>       fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>>>
>>>>>>>       fail_on_test(buf_cap.querybuf(node, 0));
>>>>>>>       fail_on_test(buf_cap.qbuf(node));
>>>>>>>       fail_on_test(buf_cap.dqbuf(node));
>>>>>>>       fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>>>       for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>>>           fail_on_test(buf_cap.g_bytesused(p));
>>>>>>>       fail_on_test(node->streamoff(q.g_type()));
>>>>>>>       fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>>>>
>>>>>>>       /* Call STREAMON, queue one CAPTURE buffer, then STOP */
>>>>>>>       fail_on_test(node->streamon(q.g_type()));
>>>>>>>       fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>>>       fail_on_test(buf_cap.querybuf(node, 0));
>>>>>>>       fail_on_test(buf_cap.qbuf(node));
>>>>>>>       fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>>>
>>>>>>>       fail_on_test(buf_cap.dqbuf(node));
>>>>>>>       fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>>>       for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>>>           fail_on_test(buf_cap.g_bytesused(p));
>>>>>>>       fail_on_test(node->streamoff(q.g_type()));
>>>>>>>       fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>>>> }
>>>>>>>
>>>>>>> The reason for this is because the driver has a limitation where all
>>>>>>> capturebuffers must be queued to the driver before STREAMON is effective.
>>>>>>> The firmware needs to know in advance what all the buffers are before
>>>>>>> starting to decode.
>>>>>>> This limitation is enforced via q->min_buffers_needed.
>>>>>>> As such, in this compliance codepath, STREAMON is never actually called
>>>>>>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
>>>>>>
>>>>>> That's interesting. I will have to look more closely at this.
>>
>> This requires a helper function in videobuf2-v4l2.c.
>>
>> In vdec_decoder_cmd you would need code like this:
>>
>>          if (!vb2_start_streaming_called(&capture_queue)) {
>>                  vb2_dequeue_empty_last_buf(&capture_queue);
>>                  return 0;
>>          }
>>
>> The vb2_dequeue_empty_last_buf (function name can probably be improved upon!)
>> does nothing if no capture buffers were queued, otherwise it takes the first
>> buffer, sets the LAST flag and sets bytesused to 0 and marks it as DONE.
>>
>> The driver cannot do this directly, since the buffers were never queued to the
>> driver and are owned by vb2.
>>
>> This is something that needs to be done for any codec driver that sets
>> min_buffers_needed to a value > 1.
>>
>> The vb2 function would look something like this:
>>
>> void vb2_dqbuf_empty_last_buf(struct vb2_queue *q)
>> {
>>          struct vb2_buffer *vb;
>>          struct vb2_v4l2_buffer *vbuf;
>>          unsigned int i;
>>
>>          if (WARN_ON(q->is_output))
>>                  return;
>>          if (list_empty(&q->queued_list))
>>                  return;
>>          vb = list_first_entry(&q->queued_list, struct vb2_buffer, queued_entry);
>>          list_del(&vb->queued_entry);
>>          for (i = 0; i < vb->num_planes; i++)
>>                  vb2_set_plane_payload(vb, i, 0)
>>          vbuf = to_vb2_v4l2_buffer(vb);
>>          vbuf->flags |= V4L2_BUF_FLAG_LAST;
>>          vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
>> }
>> EXPORT_SYMBOL_GPL(vb2_dqbuf_empty_last_buf);
>>
>> Neither compiled, nor tested, and I think this should be in v4l2-mem2mem.c instead of
>> in videobuf2-v4l2.c since this is very m2m specific.
>>
>> So see this as a suggestion :-)
>>
>> Anyway, the key take-away from this is that userspace does not know if your driver
>> behaves the way it does, so STOP should still produce a sane expected result.
>>
>> Which in this is just a single empty capture buffer marked LAST.
> 
> Thanks, this makes sense. It doesn't quite fit the current usage
> unfortunately as the test in v4l2-compliance goes like this:
> 
> fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> fail_on_test(buf_cap.querybuf(node, 0));
> fail_on_test(buf_cap.qbuf(node));
> fail_on_test(buf_cap.dqbuf(node));
> fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> 
> Since the buffer is queued after issuing the stop cmd, it is not
> possible to flag it as DONE in vdec_decoder_cmd.
> 
> A solution would be to hijack vidioc_qbuf and flag the buffer if a
> stop has been issued previously and the capture queue is not
> streaming. Would that be okay ?
> 

I was able to pass the vanilla v4l2-compliance tests by hacking in the 
following 2 things:

1) Adding your function that I modified a bit (buffer had to be tagged 
as VB2_BUF_STATE_ACTIVE, it shouldn't be removed from the list, and 
q->owned_by_drv_count needs to be atomic_inc'd)

void vb2_dqbuf_empty_last_buf(struct vb2_queue *q)
{
         struct vb2_buffer *vb;
         struct vb2_v4l2_buffer *vbuf;
         unsigned int i;

         if (WARN_ON(q->is_output))
                 return;
         if (list_empty(&q->queued_list))
                 return;

         vb = list_first_entry(&q->queued_list, struct vb2_buffer, 
queued_entry);
         for (i = 0; i < vb->num_planes; i++)
                 vb2_set_plane_payload(vb, i, 0);

	vb->state = VB2_BUF_STATE_ACTIVE;
	atomic_inc(&q->owned_by_drv_count);
         vbuf = to_vb2_v4l2_buffer(vb);
         vbuf->flags |= V4L2_BUF_FLAG_LAST;
         vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
}

2) Hijacking vidioc_qbuf to DONE the buffer if a stop was previously 
asked for, and if the capture queue isn't streaming:

static int vdec_qbuf(struct file *file, void *priv,
		     struct v4l2_buffer *buf)
{
	struct v4l2_fh *fh = file->private_data;
	struct amvdec_session *sess;
	struct vb2_queue *vq;
	int ret;

	ret = v4l2_m2m_ioctl_qbuf(file, priv, buf);
	if (ret)
		return ret;

	sess = container_of(file->private_data, struct amvdec_session, fh);
	vq = v4l2_m2m_get_vq(fh->m2m_ctx, buf->type);
	/*
	 * If the capture queue isn't streaming and we were asked to
	 * stop, DONE the buffer instantly
	 */
	if (!V4L2_TYPE_IS_OUTPUT(vq->type) &&
	    !sess->streamon_cap && sess->should_stop)
		vb2_dqbuf_empty_last_buf(vq);

	return 0;
}

Overall it feels quite messy :( . It doesn't look like a buffer is 
supposed to be dequeued if streaming hasn't started (they are tagged as 
VB2_BUF_STATE_ACTIVE only when streaming starts, and this flag is a 
requirement for vb2_buffer_done).

All this could be built in more properly into v4l2-mem2mem.c, though it 
would require the same hacks around VB2_BUF_STATE_ACTIVE and 
q->owned_by_drv_count. Or it would need a vb2 function specifically for 
this case, which would be very similar to vb2_buffer_done but allowing 
the state being VB2_BUF_STATE_QUEUED and not changing q->owned_by_drv_count.

>>
>> Regards,
>>
>>          Hans

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Hans Verkuil]

On 10/18/19 9:50 AM, Maxime Jourdan wrote:
> On Wed, Oct 9, 2019 at 2:01 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 10/8/19 3:40 PM, Maxime Jourdan wrote:
>>> On 07/10/2019 18:39, Hans Verkuil wrote:
>>>> On 10/7/19 6:24 PM, Maxime Jourdan wrote:
>>>>> On 07/10/2019 17:12, Hans Verkuil wrote:
>>>>>> On 10/7/19 4:59 PM, Maxime Jourdan wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> This patch series aims to bring H.264 support as well as compliance update
>>>>>>> to the amlogic stateful video decoder driver.
>>>>>>>
>>>>>>> There is 1 issue that remains currently:
>>>>>>>
>>>>>>>    - The following codepath had to be commented out from v4l2-compliance as
>>>>>>> it led to stalling:
>>>>>>>
>>>>>>> if (node->codec_mask & STATEFUL_DECODER) {
>>>>>>>      struct v4l2_decoder_cmd cmd;
>>>>>>>      buffer buf_cap(m2m_q);
>>>>>>>
>>>>>>>      memset(&cmd, 0, sizeof(cmd));
>>>>>>>      cmd.cmd = V4L2_DEC_CMD_STOP;
>>>>>>>
>>>>>>>      /* No buffers are queued, call STREAMON, then STOP */
>>>>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>>>
>>>>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>>>>
>>>>>>>      /* Call STREAMON, queue one CAPTURE buffer, then STOP */
>>>>>>>      fail_on_test(node->streamon(q.g_type()));
>>>>>>>      fail_on_test(node->streamon(m2m_q.g_type()));
>>>>>>>      fail_on_test(buf_cap.querybuf(node, 0));
>>>>>>>      fail_on_test(buf_cap.qbuf(node));
>>>>>>>      fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>>>>>>
>>>>>>>      fail_on_test(buf_cap.dqbuf(node));
>>>>>>>      fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>>>>>>>      for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>>>>>>>          fail_on_test(buf_cap.g_bytesused(p));
>>>>>>>      fail_on_test(node->streamoff(q.g_type()));
>>>>>>>      fail_on_test(node->streamoff(m2m_q.g_type()));
>>>>>>> }
>>>>>>>
>>>>>>> The reason for this is because the driver has a limitation where all
>>>>>>> capturebuffers must be queued to the driver before STREAMON is effective.
>>>>>>> The firmware needs to know in advance what all the buffers are before
>>>>>>> starting to decode.
>>>>>>> This limitation is enforced via q->min_buffers_needed.
>>>>>>> As such, in this compliance codepath, STREAMON is never actually called
>>>>>>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
>>>>>>
>>>>>> That's interesting. I will have to look more closely at this.
>>
>> This requires a helper function in videobuf2-v4l2.c.
>>
>> In vdec_decoder_cmd you would need code like this:
>>
>>         if (!vb2_start_streaming_called(&capture_queue)) {
>>                 vb2_dequeue_empty_last_buf(&capture_queue);
>>                 return 0;
>>         }
>>
>> The vb2_dequeue_empty_last_buf (function name can probably be improved upon!)
>> does nothing if no capture buffers were queued, otherwise it takes the first
>> buffer, sets the LAST flag and sets bytesused to 0 and marks it as DONE.
>>
>> The driver cannot do this directly, since the buffers were never queued to the
>> driver and are owned by vb2.
>>
>> This is something that needs to be done for any codec driver that sets
>> min_buffers_needed to a value > 1.
>>
>> The vb2 function would look something like this:
>>
>> void vb2_dqbuf_empty_last_buf(struct vb2_queue *q)
>> {
>>         struct vb2_buffer *vb;
>>         struct vb2_v4l2_buffer *vbuf;
>>         unsigned int i;
>>
>>         if (WARN_ON(q->is_output))
>>                 return;
>>         if (list_empty(&q->queued_list))
>>                 return;
>>         vb = list_first_entry(&q->queued_list, struct vb2_buffer, queued_entry);
>>         list_del(&vb->queued_entry);
>>         for (i = 0; i < vb->num_planes; i++)
>>                 vb2_set_plane_payload(vb, i, 0)
>>         vbuf = to_vb2_v4l2_buffer(vb);
>>         vbuf->flags |= V4L2_BUF_FLAG_LAST;
>>         vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
>> }
>> EXPORT_SYMBOL_GPL(vb2_dqbuf_empty_last_buf);
>>
>> Neither compiled, nor tested, and I think this should be in v4l2-mem2mem.c instead of
>> in videobuf2-v4l2.c since this is very m2m specific.
>>
>> So see this as a suggestion :-)
>>
>> Anyway, the key take-away from this is that userspace does not know if your driver
>> behaves the way it does, so STOP should still produce a sane expected result.
>>
>> Which in this is just a single empty capture buffer marked LAST.
> 
> Thanks, this makes sense. It doesn't quite fit the current usage
> unfortunately as the test in v4l2-compliance goes like this:
> 
> fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> fail_on_test(buf_cap.querybuf(node, 0));
> fail_on_test(buf_cap.qbuf(node));
> fail_on_test(buf_cap.dqbuf(node));
> fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> 
> Since the buffer is queued after issuing the stop cmd, it is not
> possible to flag it as DONE in vdec_decoder_cmd.
> 
> A solution would be to hijack vidioc_qbuf and flag the buffer if a
> stop has been issued previously and the capture queue is not
> streaming. Would that be okay ?

Actually, I am wondering if this shouldn't be integrated into
v4l2-mem2mem.c. The corner case where you need to use an empty
last buffer is really awkward for drivers. So perhaps this should
be integrated into v4l2-mem2mem.c where you can mark that the
next queued buffer shall be immediately returned as an empty buffer
with the LAST flag set.

Since v4l2-mem2mem already has its own vidioc_qbuf function it can
easily be added there, and I think that's a much better place than
having to touch vb2 itself.

Regards,

	Hans

> 
> Maxime
> 
>>
>> Regards,
>>
>>         Hans

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Nicolas Dufresne]

Le lundi 07 octobre 2019 à 16:59 +0200, Maxime Jourdan a écrit :
> Hello,
> 
> This patch series aims to bring H.264 support as well as compliance update
> to the amlogic stateful video decoder driver.
> 
> There is 1 issue that remains currently:
> 
>  - The following codepath had to be commented out from v4l2-compliance as
> it led to stalling:
> 
> if (node->codec_mask & STATEFUL_DECODER) {
> 	struct v4l2_decoder_cmd cmd;
> 	buffer buf_cap(m2m_q);
> 
> 	memset(&cmd, 0, sizeof(cmd));
> 	cmd.cmd = V4L2_DEC_CMD_STOP;
> 
> 	/* No buffers are queued, call STREAMON, then STOP */
> 	fail_on_test(node->streamon(q.g_type()));
> 	fail_on_test(node->streamon(m2m_q.g_type()));
> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> 
> 	fail_on_test(buf_cap.querybuf(node, 0));
> 	fail_on_test(buf_cap.qbuf(node));
> 	fail_on_test(buf_cap.dqbuf(node));
> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
> 		fail_on_test(buf_cap.g_bytesused(p));
> 	fail_on_test(node->streamoff(q.g_type()));
> 	fail_on_test(node->streamoff(m2m_q.g_type()));
> 
> 	/* Call STREAMON, queue one CAPTURE buffer, then STOP */
> 	fail_on_test(node->streamon(q.g_type()));
> 	fail_on_test(node->streamon(m2m_q.g_type()));
> 	fail_on_test(buf_cap.querybuf(node, 0));
> 	fail_on_test(buf_cap.qbuf(node));
> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
> 
> 	fail_on_test(buf_cap.dqbuf(node));
> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
> 		fail_on_test(buf_cap.g_bytesused(p));
> 	fail_on_test(node->streamoff(q.g_type()));
> 	fail_on_test(node->streamoff(m2m_q.g_type()));
> }
> 
> The reason for this is because the driver has a limitation where all
> capturebuffers must be queued to the driver before STREAMON is effective.
> The firmware needs to know in advance what all the buffers are before
> starting to decode.
> This limitation is enforced via q->min_buffers_needed.
> As such, in this compliance codepath, STREAMON is never actually called
> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
> 
> 
> One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
> by v4l2-compliance, so it was left out for the test. However, it is
> present in the patch series.
> 
> The second patch has 3 "Alignment should match open parenthesis" lines
> where I preferred to keep them that way.
> 
> Thanks Stanimir for sharing your HDR file creation tools, this was very
> helpful :).

I tried to test this with a pending branch of GStreamer supporting
dynamic resolution changes. The even driver mechanism does not seem to
work with this driver. I've grepped the code, and don't see any places
were the event would be emitted.

Then I grepped, and it seems the driver accept source_change
subscription but does not set V4L2_FMT_FLAG_DYN_RESOLUTION. I believe
these two things are bit redundant and confusing, I'll fix the proposed
patch never the less, and see if that makes it work.

> 
> Maxime
> 
> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
> v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits
> 
> Compliance test for meson-vdec device /dev/video0:
> 
> Driver Info:
> 	Driver name      : meson-vdec
> 	Card type        : Amlogic Video Decoder
> 	Bus info         : platform:meson-vdec
> 	Driver version   : 5.4.0
> 	Capabilities     : 0x84204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 		Device Capabilities
> 	Device Caps      : 0x04204000
> 		Video Memory-to-Memory Multiplanar
> 		Streaming
> 		Extended Pix Format
> 	Detected Stateful Decoder
> 
> Required ioctls:
> 	test VIDIOC_QUERYCAP: OK
> 
> Allow for multiple opens:
> 	test second /dev/video0 open: OK
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
> 	test for unlimited opens: OK
> 
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
> 
> Input ioctls:
> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
> 
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
> 
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
> 
> Control ioctls:
> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> 	test VIDIOC_QUERYCTRL: OK
> 	test VIDIOC_G/S_CTRL: OK
> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 	Standard Controls: 2 Private Controls: 0
> 
> Format ioctls:
> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 	test VIDIOC_G/S_PARM: OK (Not Supported)
> 	test VIDIOC_G_FBUF: OK (Not Supported)
> 	test VIDIOC_G_FMT: OK
> 	test VIDIOC_TRY_FMT: OK
> 	test VIDIOC_S_FMT: OK
> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> 	test Cropping: OK (Not Supported)
> 	test Composing: OK (Not Supported)
> 	test Scaling: OK
> 
> Codec ioctls:
> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 	test VIDIOC_(TRY_)DECODER_CMD: OK
> 
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test VIDIOC_EXPBUF: OK
> 	test Requests: OK (Not Supported)
> 
> Test input 0:
> 
> Streaming ioctls:
> 	test read/write: OK (Not Supported)
> 	test blocking wait: OK
> 	Video Capture Multiplanar: Captured 250 buffers   
> 	test MMAP (select): OK
> 	Video Capture Multiplanar: Captured 250 buffers   
> 	test MMAP (epoll): OK
> 	test USERPTR (select): OK (Not Supported)
> 	test DMABUF: Cannot test, specify --expbuf-device
> 
> Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
> 
> Maxime Jourdan (2):
>   media: meson: vdec: bring up to compliance
>   media: meson: vdec: add H.264 decoding support
> 
>  drivers/staging/media/meson/vdec/Makefile     |   2 +-
>  drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
>  drivers/staging/media/meson/vdec/codec_h264.h |  14 +
>  drivers/staging/media/meson/vdec/esparser.c   |  34 +-
>  drivers/staging/media/meson/vdec/vdec.c       |  70 ++-
>  drivers/staging/media/meson/vdec/vdec.h       |  14 +-
>  .../staging/media/meson/vdec/vdec_helpers.c   |  85 ++-
>  .../staging/media/meson/vdec/vdec_helpers.h   |   6 +-
>  .../staging/media/meson/vdec/vdec_platform.c  |  43 ++
>  9 files changed, 654 insertions(+), 96 deletions(-)
>  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
>  create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
> 

Re: [PATCH 0/2] media: meson: vdec: Add compliant H264 support

[Maxime Jourdan]

Hi Nicolas,
On 13/10/2019 03:08, Nicolas Dufresne wrote:
> Le lundi 07 octobre 2019 à 16:59 +0200, Maxime Jourdan a écrit :
>> Hello,
>>
>> This patch series aims to bring H.264 support as well as compliance update
>> to the amlogic stateful video decoder driver.
>>
>> There is 1 issue that remains currently:
>>
>>   - The following codepath had to be commented out from v4l2-compliance as
>> it led to stalling:
>>
>> if (node->codec_mask & STATEFUL_DECODER) {
>> 	struct v4l2_decoder_cmd cmd;
>> 	buffer buf_cap(m2m_q);
>>
>> 	memset(&cmd, 0, sizeof(cmd));
>> 	cmd.cmd = V4L2_DEC_CMD_STOP;
>>
>> 	/* No buffers are queued, call STREAMON, then STOP */
>> 	fail_on_test(node->streamon(q.g_type()));
>> 	fail_on_test(node->streamon(m2m_q.g_type()));
>> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>
>> 	fail_on_test(buf_cap.querybuf(node, 0));
>> 	fail_on_test(buf_cap.qbuf(node));
>> 	fail_on_test(buf_cap.dqbuf(node));
>> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>> 		fail_on_test(buf_cap.g_bytesused(p));
>> 	fail_on_test(node->streamoff(q.g_type()));
>> 	fail_on_test(node->streamoff(m2m_q.g_type()));
>>
>> 	/* Call STREAMON, queue one CAPTURE buffer, then STOP */
>> 	fail_on_test(node->streamon(q.g_type()));
>> 	fail_on_test(node->streamon(m2m_q.g_type()));
>> 	fail_on_test(buf_cap.querybuf(node, 0));
>> 	fail_on_test(buf_cap.qbuf(node));
>> 	fail_on_test(doioctl(node, VIDIOC_DECODER_CMD, &cmd));
>>
>> 	fail_on_test(buf_cap.dqbuf(node));
>> 	fail_on_test(!(buf_cap.g_flags() & V4L2_BUF_FLAG_LAST));
>> 	for (unsigned p = 0; p < buf_cap.g_num_planes(); p++)
>> 		fail_on_test(buf_cap.g_bytesused(p));
>> 	fail_on_test(node->streamoff(q.g_type()));
>> 	fail_on_test(node->streamoff(m2m_q.g_type()));
>> }
>>
>> The reason for this is because the driver has a limitation where all
>> capturebuffers must be queued to the driver before STREAMON is effective.
>> The firmware needs to know in advance what all the buffers are before
>> starting to decode.
>> This limitation is enforced via q->min_buffers_needed.
>> As such, in this compliance codepath, STREAMON is never actually called
>> driver-side and there is a stall on fail_on_test(buf_cap.dqbuf(node));
>>
>>
>> One last detail: V4L2_FMT_FLAG_DYN_RESOLUTION is currently not recognized
>> by v4l2-compliance, so it was left out for the test. However, it is
>> present in the patch series.
>>
>> The second patch has 3 "Alignment should match open parenthesis" lines
>> where I preferred to keep them that way.
>>
>> Thanks Stanimir for sharing your HDR file creation tools, this was very
>> helpful :).
> 
> I tried to test this with a pending branch of GStreamer supporting
> dynamic resolution changes. The even driver mechanism does not seem to
> work with this driver. I've grepped the code, and don't see any places
> were the event would be emitted.

Thanks for taking the time to test!

The event is sent in vdec_helpers.c:434

void amvdec_src_change(struct amvdec_session *sess, u32 width,
		       u32 height, u32 dpb_size)
{
	static const struct v4l2_event ev = {
		.type = V4L2_EVENT_SOURCE_CHANGE,
		.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION };

[..]

	dev_dbg(sess->core->dev, "Res. changed (%ux%u), DPB size %u\n",
		width, height, dpb_size);
	v4l2_event_queue_fh(&sess->fh, &ev);
}

> 
> Then I grepped, and it seems the driver accept source_change
> subscription but does not set V4L2_FMT_FLAG_DYN_RESOLUTION. I believe
> these two things are bit redundant and confusing, I'll fix the proposed
> patch never the less, and see if that makes it work.
It is set for H.264 if you look at the second patch of the series adding 
support for it.

{
	.pixfmt = V4L2_PIX_FMT_H264,
[..]
	.flags = V4L2_FMT_FLAG_COMPRESSED |
		 V4L2_FMT_FLAG_DYN_RESOLUTION,
},

The reason for this flag is because not all formats within a decoder 
driver may support dynamic resolution.

Your 2 points make me wonder if you used the staging driver + this patch 
series, or if you used something else ? The various branches I have on 
github are not up to date with all the compliance work.

> 
>>
>> Maxime
>>
>> # v4l2-compliance --stream-from-hdr test-25fps.h264.hdr -s250
>> v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits
>>
>> Compliance test for meson-vdec device /dev/video0:
>>
>> Driver Info:
>> 	Driver name      : meson-vdec
>> 	Card type        : Amlogic Video Decoder
>> 	Bus info         : platform:meson-vdec
>> 	Driver version   : 5.4.0
>> 	Capabilities     : 0x84204000
>> 		Video Memory-to-Memory Multiplanar
>> 		Streaming
>> 		Extended Pix Format
>> 		Device Capabilities
>> 	Device Caps      : 0x04204000
>> 		Video Memory-to-Memory Multiplanar
>> 		Streaming
>> 		Extended Pix Format
>> 	Detected Stateful Decoder
>>
>> Required ioctls:
>> 	test VIDIOC_QUERYCAP: OK
>>
>> Allow for multiple opens:
>> 	test second /dev/video0 open: OK
>> 	test VIDIOC_QUERYCAP: OK
>> 	test VIDIOC_G/S_PRIORITY: OK
>> 	test for unlimited opens: OK
>>
>> Debug ioctls:
>> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>>
>> Input ioctls:
>> 	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
>> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
>> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>>
>> Output ioctls:
>> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>>
>> Input/Output configuration ioctls:
>> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>>
>> Control ioctls:
>> 	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>> 	test VIDIOC_QUERYCTRL: OK
>> 	test VIDIOC_G/S_CTRL: OK
>> 	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>> 	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>> 	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>> 	Standard Controls: 2 Private Controls: 0
>>
>> Format ioctls:
>> 	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>> 	test VIDIOC_G/S_PARM: OK (Not Supported)
>> 	test VIDIOC_G_FBUF: OK (Not Supported)
>> 	test VIDIOC_G_FMT: OK
>> 	test VIDIOC_TRY_FMT: OK
>> 	test VIDIOC_S_FMT: OK
>> 	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>> 	test Cropping: OK (Not Supported)
>> 	test Composing: OK (Not Supported)
>> 	test Scaling: OK
>>
>> Codec ioctls:
>> 	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>> 	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>> 	test VIDIOC_(TRY_)DECODER_CMD: OK
>>
>> Buffer ioctls:
>> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
>> 	test VIDIOC_EXPBUF: OK
>> 	test Requests: OK (Not Supported)
>>
>> Test input 0:
>>
>> Streaming ioctls:
>> 	test read/write: OK (Not Supported)
>> 	test blocking wait: OK
>> 	Video Capture Multiplanar: Captured 250 buffers
>> 	test MMAP (select): OK
>> 	Video Capture Multiplanar: Captured 250 buffers
>> 	test MMAP (epoll): OK
>> 	test USERPTR (select): OK (Not Supported)
>> 	test DMABUF: Cannot test, specify --expbuf-device
>>
>> Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
>>
>> Maxime Jourdan (2):
>>    media: meson: vdec: bring up to compliance
>>    media: meson: vdec: add H.264 decoding support
>>
>>   drivers/staging/media/meson/vdec/Makefile     |   2 +-
>>   drivers/staging/media/meson/vdec/codec_h264.c | 482 ++++++++++++++++++
>>   drivers/staging/media/meson/vdec/codec_h264.h |  14 +
>>   drivers/staging/media/meson/vdec/esparser.c   |  34 +-
>>   drivers/staging/media/meson/vdec/vdec.c       |  70 ++-
>>   drivers/staging/media/meson/vdec/vdec.h       |  14 +-
>>   .../staging/media/meson/vdec/vdec_helpers.c   |  85 ++-
>>   .../staging/media/meson/vdec/vdec_helpers.h   |   6 +-
>>   .../staging/media/meson/vdec/vdec_platform.c  |  43 ++
>>   9 files changed, 654 insertions(+), 96 deletions(-)
>>   create mode 100644 drivers/staging/media/meson/vdec/codec_h264.c
>>   create mode 100644 drivers/staging/media/meson/vdec/codec_h264.h
>>
>