[PATCH 00/28] media: coda: fixes and improvements

[PATCH 00/28] media: coda: fixes and improvements

[Philipp Zabel]

Hi,

this series contains a few fixes for MPEG-2 sequence number counting,
decoder/encoder stop command race conditions and some more work to
further move towards V4L2 stateful decoder API compliance:

Sequence initialization, which lets the firmware parse the bitstream
headers, is reworked to run before the capture queue is streaming,
with some padding acrobatics to work around issues when only headers
or very small buffers are queued initially. A SOURCE_CHANGE event
is issued after sequence initialization completes.

The sequence counting mechanism used to determine the last processed
buffer is replaced with a last buffer flag that is set on the last
queued output buffer and carried over to the corresponding bitstream
metadata, to the decoded internal tiled frame, and finally to the
returned linear capture buffer.

Dynamic parameter change support is added for encoder rate control
or quantization parameters, slice mode, and cyclic intra refresh
interval.

Inbetween there are a few code cleanup patches that introduce, change,
or make use of helper functions, and improve driver structures.

regards
Philipp

Marco Felsch (2):
  media: coda: fix last buffer handling in V4L2_ENC_CMD_STOP
  media: coda: fix V4L2_DEC_CMD_STOP when all buffers are already
    consumed

Michael Tretter (1):
  media: coda: implement CMD_START to restart decoding

Philipp Zabel (25):
  media: coda: use mem2mem try_en/decoder_cmd helpers
  media: coda: fix mpeg2 sequence number handling
  media: coda: add coda_wake_up_capture_queue
  media: coda: split decoder sequence initialization out of start
    decoding
  media: coda: add sequence initialization work
  media: coda: implement decoder source change event
  media: coda: integrate internal frame metadata into a structure
  media: coda: make coda_bitstream_queue more versatile
  media: coda: pad first buffer with repeated MPEG headers to fix
    sequence init
  media: coda: do not enforce 512-byte initial bitstream payload on
    CODA960
  media: coda: flush bitstream ring buffer on decoder restart
  media: coda: increment sequence offset for the last returned frame
  media: coda: allow flagging last output buffer internally
  media: coda: mark the last output buffer on decoder stop command
  media: coda: only set the stream end flags if there are no more
    pending output buffers
  media: coda: mark the last output buffer on encoder stop command
  media: coda: retire coda_buf_is_end_of_stream
  media: coda: only wake up capture queue if no pending buffers to
    encode
  media: coda: flag the last encoded buffer
  media: coda: lock capture queue wakeup against encoder stop command
  media: coda: mark last pending buffer or last meta on decoder stop
    command
  media: coda: mark last returned frame
  media: coda: store device pointer in driver structure instead of pdev
  media: coda: add coda_slice_mode() function
  media: coda: encoder parameter change support

 drivers/media/platform/coda/coda-bit.c    | 403 +++++++++++++++++-----
 drivers/media/platform/coda/coda-common.c | 244 ++++++++++---
 drivers/media/platform/coda/coda-mpeg2.c  |  43 +++
 drivers/media/platform/coda/coda-mpeg4.c  |  38 ++
 drivers/media/platform/coda/coda.h        |  33 +-
 drivers/media/platform/coda/coda_regs.h   |  18 +
 6 files changed, 637 insertions(+), 142 deletions(-)

-- 
2.20.1

[PATCH 01/28] media: coda: implement CMD_START to restart decoding

[Philipp Zabel]

From: Michael Tretter <m.tretter@pengutronix.de>

The CMD_START shall be used to start the processing after a drain that
was initiated with CMD_STOP.

Up until now, a drain on coda could only be finished with a
STREAMOFF-STREAMON, which resulted in a reset of the device.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 751b0be1c2ea..a5e0d5c1528e 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1059,16 +1059,29 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 			    struct v4l2_decoder_cmd *dc)
 {
 	struct coda_ctx *ctx = fh_to_ctx(fh);
+	struct vb2_queue *dst_vq;
 	int ret;
 
 	ret = coda_try_decoder_cmd(file, fh, dc);
 	if (ret < 0)
 		return ret;
 
-	/* Set the stream-end flag on this context */
-	coda_bit_stream_end_flag(ctx);
-	ctx->hold = false;
-	v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+	switch (dc->cmd) {
+	case V4L2_DEC_CMD_START:
+		dst_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
+					 V4L2_BUF_TYPE_VIDEO_CAPTURE);
+		vb2_clear_last_buffer_dequeued(dst_vq);
+		ctx->bit_stream_param &= ~CODA_BIT_STREAM_END_FLAG;
+		break;
+	case V4L2_DEC_CMD_STOP:
+		/* Set the stream-end flag on this context */
+		coda_bit_stream_end_flag(ctx);
+		ctx->hold = false;
+		v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+		break;
+	default:
+		return -EINVAL;
+	}
 
 	return 0;
 }
-- 
2.20.1

[PATCH 02/28] media: coda: use mem2mem try_en/decoder_cmd helpers

[Philipp Zabel]

Use mem2mem try_en/decoder_cmd helpers to ensure consistent behaviour
with other video codec drivers.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 19 ++-----------------
 1 file changed, 2 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index a5e0d5c1528e..e09d48170e5c 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1001,13 +1001,7 @@ static int coda_try_encoder_cmd(struct file *file, void *fh,
 	if (ctx->inst_type != CODA_INST_ENCODER)
 		return -ENOTTY;
 
-	if (ec->cmd != V4L2_ENC_CMD_STOP)
-		return -EINVAL;
-
-	if (ec->flags & V4L2_ENC_CMD_STOP_AT_GOP_END)
-		return -EINVAL;
-
-	return 0;
+	return v4l2_m2m_ioctl_try_encoder_cmd(file, fh, ec);
 }
 
 static int coda_encoder_cmd(struct file *file, void *fh,
@@ -1043,16 +1037,7 @@ static int coda_try_decoder_cmd(struct file *file, void *fh,
 	if (ctx->inst_type != CODA_INST_DECODER)
 		return -ENOTTY;
 
-	if (dc->cmd != V4L2_DEC_CMD_STOP)
-		return -EINVAL;
-
-	if (dc->flags & V4L2_DEC_CMD_STOP_TO_BLACK)
-		return -EINVAL;
-
-	if (!(dc->flags & V4L2_DEC_CMD_STOP_IMMEDIATELY) && (dc->stop.pts != 0))
-		return -EINVAL;
-
-	return 0;
+	return v4l2_m2m_ioctl_try_decoder_cmd(file, fh, dc);
 }
 
 static int coda_decoder_cmd(struct file *file, void *fh,
-- 
2.20.1

[PATCH 03/28] media: coda: fix mpeg2 sequence number handling

[Philipp Zabel]

Sequence number handling assumed that the BIT processor frame number
starts counting at 1, but this is not true for the MPEG-2 decoder,
which starts at 0. Fix the sequence counter offset detection to handle
this.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 06a352659bae..45ffe2e87e0a 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -1746,6 +1746,7 @@ static int __coda_start_decoding(struct coda_ctx *ctx)
 		v4l2_err(&dev->v4l2_dev, "CODA_COMMAND_SEQ_INIT timeout\n");
 		return ret;
 	}
+	ctx->sequence_offset = ~0U;
 	ctx->initialized = 1;
 
 	/* Update kfifo out pointer from coda bitstream read pointer */
@@ -2169,7 +2170,9 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		v4l2_err(&dev->v4l2_dev,
 			 "decoded frame index out of range: %d\n", decoded_idx);
 	} else {
-		val = coda_read(dev, CODA_RET_DEC_PIC_FRAME_NUM) - 1;
+		val = coda_read(dev, CODA_RET_DEC_PIC_FRAME_NUM);
+		if (ctx->sequence_offset == -1)
+			ctx->sequence_offset = val;
 		val -= ctx->sequence_offset;
 		spin_lock(&ctx->buffer_meta_lock);
 		if (!list_empty(&ctx->buffer_meta_list)) {
-- 
2.20.1

[PATCH 04/28] media: coda: fix last buffer handling in V4L2_ENC_CMD_STOP

[Philipp Zabel]

From: Marco Felsch <m.felsch@pengutronix.de>

coda_encoder_cmd() is racy, as the last scheduled picture run worker can
still be in-flight while the ENC_CMD_STOP command is issued. Depending
on the exact timing the sequence numbers might already be changed, but
the last buffer might not have been put on the destination queue yet.

In this case the current implementation would prematurely wake the
destination queue with last_buffer_dequeued=true, causing userspace to
call streamoff before the last buffer is handled.

Close this race window by synchronizing with the pic_run_worker before
doing the sequence check.

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
[l.stach@pengutronix.de: switch to flush_work, reword commit message]
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index e09d48170e5c..d3b1e7b87a8a 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1018,6 +1018,8 @@ static int coda_encoder_cmd(struct file *file, void *fh,
 	/* Set the stream-end flag on this context */
 	ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
 
+	flush_work(&ctx->pic_run_work);
+
 	/* If there is no buffer in flight, wake up */
 	if (!ctx->streamon_out || ctx->qsequence == ctx->osequence) {
 		dst_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
-- 
2.20.1

[PATCH 05/28] media: coda: add coda_wake_up_capture_queue

[Philipp Zabel]

Combine setting the last_buffer_dequeued flag on the capture video
queue and waking up its done workqueue into a helper function.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index d3b1e7b87a8a..7dbeb80d40f4 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1004,11 +1004,21 @@ static int coda_try_encoder_cmd(struct file *file, void *fh,
 	return v4l2_m2m_ioctl_try_encoder_cmd(file, fh, ec);
 }
 
+static void coda_wake_up_capture_queue(struct coda_ctx *ctx)
+{
+	struct vb2_queue *dst_vq;
+
+	coda_dbg(1, ctx, "waking up capture queue\n");
+
+	dst_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+	dst_vq->last_buffer_dequeued = true;
+	wake_up(&dst_vq->done_wq);
+}
+
 static int coda_encoder_cmd(struct file *file, void *fh,
 			    struct v4l2_encoder_cmd *ec)
 {
 	struct coda_ctx *ctx = fh_to_ctx(fh);
-	struct vb2_queue *dst_vq;
 	int ret;
 
 	ret = coda_try_encoder_cmd(file, fh, ec);
@@ -1021,12 +1031,8 @@ static int coda_encoder_cmd(struct file *file, void *fh,
 	flush_work(&ctx->pic_run_work);
 
 	/* If there is no buffer in flight, wake up */
-	if (!ctx->streamon_out || ctx->qsequence == ctx->osequence) {
-		dst_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
-					 V4L2_BUF_TYPE_VIDEO_CAPTURE);
-		dst_vq->last_buffer_dequeued = true;
-		wake_up(&dst_vq->done_wq);
-	}
+	if (!ctx->streamon_out || ctx->qsequence == ctx->osequence)
+		coda_wake_up_capture_queue(ctx);
 
 	return 0;
 }
-- 
2.20.1

[PATCH 06/28] media: coda: fix V4L2_DEC_CMD_STOP when all buffers are already consumed

[Philipp Zabel]

From: Marco Felsch <m.felsch@pengutronix.de>

When the DEC_CMD_STOP command is issued after the context has already
consumed all the queued buffers, we need to make sure to wake the
destination queue with last_buffer_dequeued set, to allow userspace to
make progress in its EOS handling.

As there might still be picture run workers pending at that point, we
need to synchronize with them, so the sequence number comparison reads
stable values.

Signed-off-by: Marco Felsch <m.felsch@pengutronix.de>
[l.stach@pengutronix.de: rewrite to fix multi-context use-cases,
 reword commit message]
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 7dbeb80d40f4..232bda4b7016 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1071,6 +1071,12 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 		coda_bit_stream_end_flag(ctx);
 		ctx->hold = false;
 		v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+
+		flush_work(&ctx->pic_run_work);
+
+		/* If there is no buffer in flight, wake up */
+		if (!ctx->streamon_out || ctx->qsequence == ctx->osequence)
+			coda_wake_up_capture_queue(ctx);
 		break;
 	default:
 		return -EINVAL;
-- 
2.20.1

[PATCH 07/28] media: coda: split decoder sequence initialization out of start decoding

[Philipp Zabel]

The sequence initialization already has to happen during the
initialization phase, after headers have been queued on the OUTPUT
queue. This means that sequence initialization has to be queued as
a work item from QBUF on the OUTPUT queue. The internal framebuffer
setup should be done later during VIDIOC_REQBUFS() on the CAPTURE
queue.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 30 ++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 45ffe2e87e0a..cecfd51e3838 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -1661,7 +1661,7 @@ static bool coda_reorder_enable(struct coda_ctx *ctx)
 	return profile > V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE;
 }
 
-static int __coda_start_decoding(struct coda_ctx *ctx)
+static int __coda_decoder_seq_init(struct coda_ctx *ctx)
 {
 	struct coda_q_data *q_data_src, *q_data_dst;
 	u32 bitstream_buf, bitstream_size;
@@ -1684,8 +1684,6 @@ static int __coda_start_decoding(struct coda_ctx *ctx)
 	src_fourcc = q_data_src->fourcc;
 	dst_fourcc = q_data_dst->fourcc;
 
-	coda_write(dev, ctx->parabuf.paddr, CODA_REG_BIT_PARA_BUF_ADDR);
-
 	/* Update coda bitstream read and write pointers from kfifo */
 	coda_kfifo_sync_to_device_full(ctx);
 
@@ -1823,6 +1821,29 @@ static int __coda_start_decoding(struct coda_ctx *ctx)
 			coda_update_profile_level_ctrls(ctx, profile, level);
 	}
 
+	return 0;
+}
+
+static int __coda_start_decoding(struct coda_ctx *ctx)
+{
+	struct coda_q_data *q_data_src, *q_data_dst;
+	struct coda_dev *dev = ctx->dev;
+	u32 src_fourcc, dst_fourcc;
+	int ret;
+
+	if (!ctx->initialized) {
+		ret = __coda_decoder_seq_init(ctx);
+		if (ret < 0)
+			return ret;
+	}
+
+	q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+	src_fourcc = q_data_src->fourcc;
+	dst_fourcc = q_data_dst->fourcc;
+
+	coda_write(dev, ctx->parabuf.paddr, CODA_REG_BIT_PARA_BUF_ADDR);
+
 	ret = coda_alloc_framebuffers(ctx, q_data_dst, src_fourcc);
 	if (ret < 0) {
 		v4l2_err(&dev->v4l2_dev, "failed to allocate framebuffers\n");
@@ -1831,7 +1852,8 @@ static int __coda_start_decoding(struct coda_ctx *ctx)
 
 	/* Tell the decoder how many frame buffers we allocated. */
 	coda_write(dev, ctx->num_internal_frames, CODA_CMD_SET_FRAME_BUF_NUM);
-	coda_write(dev, width, CODA_CMD_SET_FRAME_BUF_STRIDE);
+	coda_write(dev, round_up(q_data_dst->rect.width, 16),
+		   CODA_CMD_SET_FRAME_BUF_STRIDE);
 
 	if (dev->devtype->product != CODA_DX6) {
 		/* Set secondary AXI IRAM */
-- 
2.20.1

[PATCH 08/28] media: coda: add sequence initialization work

[Philipp Zabel]

Add a sequence initialization work item to be run when OUTPUT buffers
are queued in the initialization state.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c    | 25 +++++++++++++++++++++++
 drivers/media/platform/coda/coda-common.c | 24 ++++++++++++++++++++++
 drivers/media/platform/coda/coda.h        |  2 ++
 3 files changed, 51 insertions(+)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index cecfd51e3838..9f8e2342d175 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -1824,6 +1824,30 @@ static int __coda_decoder_seq_init(struct coda_ctx *ctx)
 	return 0;
 }
 
+static void coda_dec_seq_init_work(struct work_struct *work)
+{
+	struct coda_ctx *ctx = container_of(work,
+					    struct coda_ctx, seq_init_work);
+	struct coda_dev *dev = ctx->dev;
+	int ret;
+
+	mutex_lock(&ctx->buffer_mutex);
+	mutex_lock(&dev->coda_mutex);
+
+	if (ctx->initialized == 1)
+		goto out;
+
+	ret = __coda_decoder_seq_init(ctx);
+	if (ret < 0)
+		goto out;
+
+	ctx->initialized = 1;
+
+out:
+	mutex_unlock(&dev->coda_mutex);
+	mutex_unlock(&ctx->buffer_mutex);
+}
+
 static int __coda_start_decoding(struct coda_ctx *ctx)
 {
 	struct coda_q_data *q_data_src, *q_data_dst;
@@ -2352,6 +2376,7 @@ const struct coda_context_ops coda_bit_decode_ops = {
 	.prepare_run = coda_prepare_decode,
 	.finish_run = coda_finish_decode,
 	.run_timeout = coda_decode_timeout,
+	.seq_init_work = coda_dec_seq_init_work,
 	.seq_end_work = coda_seq_end_work,
 	.release = coda_bit_release,
 };
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 232bda4b7016..f8cfafa2ce42 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1684,6 +1684,19 @@ static void coda_buf_queue(struct vb2_buffer *vb)
 			/* This set buf->sequence = ctx->qsequence++ */
 			coda_fill_bitstream(ctx, NULL);
 		mutex_unlock(&ctx->bitstream_mutex);
+
+		if (!ctx->initialized) {
+			/*
+			 * Run sequence initialization in case the queued
+			 * buffer contained headers.
+			 */
+			if (vb2_is_streaming(vb->vb2_queue) &&
+			    ctx->ops->seq_init_work) {
+				queue_work(ctx->dev->workqueue,
+					   &ctx->seq_init_work);
+				flush_work(&ctx->seq_init_work);
+			}
+		}
 	} else {
 		if (ctx->inst_type == CODA_INST_ENCODER &&
 		    vq->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
@@ -1761,6 +1774,15 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 				ret = -EINVAL;
 				goto err;
 			}
+
+			if (!ctx->initialized) {
+				/* Run sequence initialization */
+				if (ctx->ops->seq_init_work) {
+					queue_work(ctx->dev->workqueue,
+						   &ctx->seq_init_work);
+					flush_work(&ctx->seq_init_work);
+				}
+			}
 		}
 
 		ctx->streamon_out = 1;
@@ -2317,6 +2339,8 @@ static int coda_open(struct file *file)
 	ctx->use_bit = !ctx->cvd->direct;
 	init_completion(&ctx->completion);
 	INIT_WORK(&ctx->pic_run_work, coda_pic_run_work);
+	if (ctx->ops->seq_init_work)
+		INIT_WORK(&ctx->seq_init_work, ctx->ops->seq_init_work);
 	if (ctx->ops->seq_end_work)
 		INIT_WORK(&ctx->seq_end_work, ctx->ops->seq_end_work);
 	v4l2_fh_init(&ctx->fh, video_devdata(file));
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index 0c2cace53ce8..c97ea3e24b80 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -185,6 +185,7 @@ struct coda_context_ops {
 	int (*prepare_run)(struct coda_ctx *ctx);
 	void (*finish_run)(struct coda_ctx *ctx);
 	void (*run_timeout)(struct coda_ctx *ctx);
+	void (*seq_init_work)(struct work_struct *work);
 	void (*seq_end_work)(struct work_struct *work);
 	void (*release)(struct coda_ctx *ctx);
 };
@@ -193,6 +194,7 @@ struct coda_ctx {
 	struct coda_dev			*dev;
 	struct mutex			buffer_mutex;
 	struct work_struct		pic_run_work;
+	struct work_struct		seq_init_work;
 	struct work_struct		seq_end_work;
 	struct completion		completion;
 	const struct coda_video_device	*cvd;
-- 
2.20.1

[PATCH 09/28] media: coda: implement decoder source change event

[Philipp Zabel]

The stateful decoder API requires decoders to signal detection
of stream dimensions via the V4L2_EVENT_SOURCE_CHANGE event.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index f8cfafa2ce42..b30945fa0a70 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1255,9 +1255,16 @@ static int coda_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
 static int coda_subscribe_event(struct v4l2_fh *fh,
 				const struct v4l2_event_subscription *sub)
 {
+	struct coda_ctx *ctx = fh_to_ctx(fh);
+
 	switch (sub->type) {
 	case V4L2_EVENT_EOS:
 		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	case V4L2_EVENT_SOURCE_CHANGE:
+		if (ctx->inst_type == CODA_INST_DECODER)
+			return v4l2_event_subscribe(fh, sub, 0, NULL);
+		else
+			return -EINVAL;
 	default:
 		return v4l2_ctrl_subscribe_event(fh, sub);
 	}
@@ -1642,6 +1649,16 @@ void coda_update_profile_level_ctrls(struct coda_ctx *ctx, u8 profile_idc,
 	}
 }
 
+static void coda_queue_source_change_event(struct coda_ctx *ctx)
+{
+	static const struct v4l2_event source_change_event = {
+		.type = V4L2_EVENT_SOURCE_CHANGE,
+		.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
+	};
+
+	v4l2_event_queue_fh(&ctx->fh, &source_change_event);
+}
+
 static void coda_buf_queue(struct vb2_buffer *vb)
 {
 	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
@@ -1696,6 +1713,9 @@ static void coda_buf_queue(struct vb2_buffer *vb)
 					   &ctx->seq_init_work);
 				flush_work(&ctx->seq_init_work);
 			}
+
+			if (ctx->initialized)
+				coda_queue_source_change_event(ctx);
 		}
 	} else {
 		if (ctx->inst_type == CODA_INST_ENCODER &&
-- 
2.20.1

[PATCH 10/28] media: coda: integrate internal frame metadata into a structure

[Philipp Zabel]

Combine the separate auxiliary buffer, buffer meta, frame type, and
decode error arrays into an array of struct coda_internal_frame.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 62 +++++++++++++-------------
 drivers/media/platform/coda/coda.h     | 12 +++--
 2 files changed, 40 insertions(+), 34 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 9f8e2342d175..494bc130c7af 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -395,7 +395,7 @@ static void coda_free_framebuffers(struct coda_ctx *ctx)
 	int i;
 
 	for (i = 0; i < CODA_MAX_FRAMEBUFFERS; i++)
-		coda_free_aux_buf(ctx->dev, &ctx->internal_frames[i]);
+		coda_free_aux_buf(ctx->dev, &ctx->internal_frames[i].buf);
 }
 
 static int coda_alloc_framebuffers(struct coda_ctx *ctx,
@@ -435,7 +435,7 @@ static int coda_alloc_framebuffers(struct coda_ctx *ctx,
 			coda_free_framebuffers(ctx);
 			return -ENOMEM;
 		}
-		ret = coda_alloc_context_buf(ctx, &ctx->internal_frames[i],
+		ret = coda_alloc_context_buf(ctx, &ctx->internal_frames[i].buf,
 					     size, name);
 		kfree(name);
 		if (ret < 0) {
@@ -449,7 +449,7 @@ static int coda_alloc_framebuffers(struct coda_ctx *ctx,
 		u32 y, cb, cr, mvcol;
 
 		/* Start addresses of Y, Cb, Cr planes */
-		y = ctx->internal_frames[i].paddr;
+		y = ctx->internal_frames[i].buf.paddr;
 		cb = y + ysize;
 		cr = y + ysize + ysize/4;
 		mvcol = y + ysize + ysize/4 + ysize/4;
@@ -1202,9 +1202,9 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 			coda9_set_frame_cache(ctx, q_data_src->fourcc);
 
 			/* FIXME */
-			coda_write(dev, ctx->internal_frames[2].paddr,
+			coda_write(dev, ctx->internal_frames[2].buf.paddr,
 				   CODA9_CMD_SET_FRAME_SUBSAMP_A);
-			coda_write(dev, ctx->internal_frames[3].paddr,
+			coda_write(dev, ctx->internal_frames[3].buf.paddr,
 				   CODA9_CMD_SET_FRAME_SUBSAMP_B);
 		}
 	}
@@ -1993,7 +1993,7 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 	    ctx->display_idx < ctx->num_internal_frames) {
 		vdoa_device_run(ctx->vdoa,
 				vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0),
-				ctx->internal_frames[ctx->display_idx].paddr);
+				ctx->internal_frames[ctx->display_idx].buf.paddr);
 	} else {
 		if (dev->devtype->product == CODA_960) {
 			/*
@@ -2091,6 +2091,7 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 	int width, height;
 	int decoded_idx;
 	int display_idx;
+	struct coda_internal_frame *decoded_frame = NULL;
 	u32 src_fourcc;
 	int success;
 	u32 err_mb;
@@ -2216,6 +2217,8 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		v4l2_err(&dev->v4l2_dev,
 			 "decoded frame index out of range: %d\n", decoded_idx);
 	} else {
+		decoded_frame = &ctx->internal_frames[decoded_idx];
+
 		val = coda_read(dev, CODA_RET_DEC_PIC_FRAME_NUM);
 		if (ctx->sequence_offset == -1)
 			ctx->sequence_offset = val;
@@ -2240,28 +2243,25 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 					 val, ctx->sequence_offset,
 					 meta->sequence);
 			}
-			ctx->frame_metas[decoded_idx] = *meta;
+			decoded_frame->meta = *meta;
 			kfree(meta);
 		} else {
 			spin_unlock(&ctx->buffer_meta_lock);
 			v4l2_err(&dev->v4l2_dev, "empty timestamp list!\n");
-			memset(&ctx->frame_metas[decoded_idx], 0,
+			memset(&decoded_frame->meta, 0,
 			       sizeof(struct coda_buffer_meta));
-			ctx->frame_metas[decoded_idx].sequence = val;
+			decoded_frame->meta.sequence = val;
 			ctx->sequence_offset++;
 		}
 
-		trace_coda_dec_pic_done(ctx, &ctx->frame_metas[decoded_idx]);
+		trace_coda_dec_pic_done(ctx, &decoded_frame->meta);
 
 		val = coda_read(dev, CODA_RET_DEC_PIC_TYPE) & 0x7;
-		if (val == 0)
-			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_KEYFRAME;
-		else if (val == 1)
-			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_PFRAME;
-		else
-			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_BFRAME;
+		decoded_frame->type = (val == 0) ? V4L2_BUF_FLAG_KEYFRAME :
+				      (val == 1) ? V4L2_BUF_FLAG_PFRAME :
+						   V4L2_BUF_FLAG_BFRAME;
 
-		ctx->frame_errors[decoded_idx] = err_mb;
+		decoded_frame->error = err_mb;
 	}
 
 	if (display_idx == -1) {
@@ -2281,6 +2281,10 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 	/* If a frame was copied out, return it */
 	if (ctx->display_idx >= 0 &&
 	    ctx->display_idx < ctx->num_internal_frames) {
+		struct coda_internal_frame *ready_frame;
+
+		ready_frame = &ctx->internal_frames[ctx->display_idx];
+
 		dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 		dst_buf->sequence = ctx->osequence++;
 
@@ -2288,8 +2292,8 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		dst_buf->flags &= ~(V4L2_BUF_FLAG_KEYFRAME |
 					     V4L2_BUF_FLAG_PFRAME |
 					     V4L2_BUF_FLAG_BFRAME);
-		dst_buf->flags |= ctx->frame_types[ctx->display_idx];
-		meta = &ctx->frame_metas[ctx->display_idx];
+		dst_buf->flags |= ready_frame->type;
+		meta = &ready_frame->meta;
 		dst_buf->timecode = meta->timecode;
 		dst_buf->vb2_buf.timestamp = meta->timestamp;
 
@@ -2298,18 +2302,17 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
 				      q_data_dst->sizeimage);
 
-		if (ctx->frame_errors[ctx->display_idx] || err_vdoa)
+		if (ready_frame->error || err_vdoa)
 			coda_m2m_buf_done(ctx, dst_buf, VB2_BUF_STATE_ERROR);
 		else
 			coda_m2m_buf_done(ctx, dst_buf, VB2_BUF_STATE_DONE);
 
-		if (decoded_idx >= 0 &&
-		    decoded_idx < ctx->num_internal_frames) {
+		if (decoded_frame) {
 			coda_dbg(1, ctx, "job finished: decoded %c frame %u, returned %c frame %u (%u/%u)%s\n",
-				 coda_frame_type_char(ctx->frame_types[decoded_idx]),
-				 ctx->frame_metas[decoded_idx].sequence,
+				 coda_frame_type_char(decoded_frame->type),
+				 decoded_frame->meta.sequence,
 				 coda_frame_type_char(dst_buf->flags),
-				 ctx->frame_metas[ctx->display_idx].sequence,
+				 ready_frame->meta.sequence,
 				 dst_buf->sequence, ctx->qsequence,
 				 (dst_buf->flags & V4L2_BUF_FLAG_LAST) ?
 				 " (last)" : "");
@@ -2317,17 +2320,16 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 			coda_dbg(1, ctx, "job finished: no frame decoded (%d), returned %c frame %u (%u/%u)%s\n",
 				 decoded_idx,
 				 coda_frame_type_char(dst_buf->flags),
-				 ctx->frame_metas[ctx->display_idx].sequence,
+				 ready_frame->meta.sequence,
 				 dst_buf->sequence, ctx->qsequence,
 				 (dst_buf->flags & V4L2_BUF_FLAG_LAST) ?
 				 " (last)" : "");
 		}
 	} else {
-		if (decoded_idx >= 0 &&
-		    decoded_idx < ctx->num_internal_frames) {
+		if (decoded_frame) {
 			coda_dbg(1, ctx, "job finished: decoded %c frame %u, no frame returned (%d)\n",
-				 coda_frame_type_char(ctx->frame_types[decoded_idx]),
-				 ctx->frame_metas[decoded_idx].sequence,
+				 coda_frame_type_char(decoded_frame->type),
+				 decoded_frame->meta.sequence,
 				 ctx->display_idx);
 		} else {
 			coda_dbg(1, ctx, "job finished: no frame decoded (%d) or returned (%d)\n",
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index c97ea3e24b80..10207e9534c2 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -190,6 +190,13 @@ struct coda_context_ops {
 	void (*release)(struct coda_ctx *ctx);
 };
 
+struct coda_internal_frame {
+	struct coda_aux_buf		buf;
+	struct coda_buffer_meta		meta;
+	u32				type;
+	u32				error;
+};
+
 struct coda_ctx {
 	struct coda_dev			*dev;
 	struct mutex			buffer_mutex;
@@ -233,10 +240,7 @@ struct coda_ctx {
 	struct coda_aux_buf		parabuf;
 	struct coda_aux_buf		psbuf;
 	struct coda_aux_buf		slicebuf;
-	struct coda_aux_buf		internal_frames[CODA_MAX_FRAMEBUFFERS];
-	u32				frame_types[CODA_MAX_FRAMEBUFFERS];
-	struct coda_buffer_meta		frame_metas[CODA_MAX_FRAMEBUFFERS];
-	u32				frame_errors[CODA_MAX_FRAMEBUFFERS];
+	struct coda_internal_frame	internal_frames[CODA_MAX_FRAMEBUFFERS];
 	struct list_head		buffer_meta_list;
 	spinlock_t			buffer_meta_lock;
 	int				num_metas;
-- 
2.20.1

[PATCH 11/28] media: coda: make coda_bitstream_queue more versatile

[Philipp Zabel]

Pass vaddr and size to coda_bitstream_queue instead of a struct
vb2_v4l2_buffer to make it reusable for queueing data that is
not exactly a whole v4l2 buffer into the bitstream ringbuffer.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 23 +++++++++--------------
 1 file changed, 9 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 494bc130c7af..4f96f808b4dd 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -199,33 +199,25 @@ static int coda_bitstream_pad(struct coda_ctx *ctx, u32 size)
 	return (n < size) ? -ENOSPC : 0;
 }
 
-static int coda_bitstream_queue(struct coda_ctx *ctx,
-				struct vb2_v4l2_buffer *src_buf)
+static int coda_bitstream_queue(struct coda_ctx *ctx, const u8 *buf, u32 size)
 {
-	u32 src_size = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
-	u32 n;
-
-	n = kfifo_in(&ctx->bitstream_fifo,
-			vb2_plane_vaddr(&src_buf->vb2_buf, 0), src_size);
-	if (n < src_size)
-		return -ENOSPC;
+	u32 n = kfifo_in(&ctx->bitstream_fifo, buf, size);
 
-	src_buf->sequence = ctx->qsequence++;
-
-	return 0;
+	return (n < size) ? -ENOSPC : 0;
 }
 
 static bool coda_bitstream_try_queue(struct coda_ctx *ctx,
 				     struct vb2_v4l2_buffer *src_buf)
 {
 	unsigned long payload = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
+	u8 *vaddr = vb2_plane_vaddr(&src_buf->vb2_buf, 0);
 	int ret;
 
 	if (coda_get_bitstream_payload(ctx) + payload + 512 >=
 	    ctx->bitstream.size)
 		return false;
 
-	if (vb2_plane_vaddr(&src_buf->vb2_buf, 0) == NULL) {
+	if (!vaddr) {
 		v4l2_err(&ctx->dev->v4l2_dev, "trying to queue empty buffer\n");
 		return true;
 	}
@@ -235,11 +227,14 @@ static bool coda_bitstream_try_queue(struct coda_ctx *ctx,
 	    ctx->codec->src_fourcc == V4L2_PIX_FMT_H264)
 		coda_bitstream_pad(ctx, 512 - payload);
 
-	ret = coda_bitstream_queue(ctx, src_buf);
+	ret = coda_bitstream_queue(ctx, vaddr, payload);
 	if (ret < 0) {
 		v4l2_err(&ctx->dev->v4l2_dev, "bitstream buffer overflow\n");
 		return false;
 	}
+
+	src_buf->sequence = ctx->qsequence++;
+
 	/* Sync read pointer to device */
 	if (ctx == v4l2_m2m_get_curr_priv(ctx->dev->m2m_dev))
 		coda_kfifo_sync_to_device_write(ctx);
-- 
2.20.1

[PATCH 12/28] media: coda: pad first buffer with repeated MPEG headers to fix sequence init

[Philipp Zabel]

If the first buffer contains only headers, the sequence initialization
command fails. On CodaHx4 the buffer must be padded to at least 512
bytes, on CODA960 it seems to be enough to just repeat the sequence and
extension headers (MPEG-2) or the VOS and VO headers (MPEG-4) once for
for sequence initialization to succeed without further bitstream data.
On CodaHx4 the headers can be repeated multiple times until the 512 byte
mark is reached.

A similar issue was solved for h.264 by padding with a filler NAL in
commit 0eef89403ece ("[media] coda: pad first h.264 buffer to 512
bytes").

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c   | 59 ++++++++++++++++++++++--
 drivers/media/platform/coda/coda-mpeg2.c | 43 +++++++++++++++++
 drivers/media/platform/coda/coda-mpeg4.c | 38 +++++++++++++++
 drivers/media/platform/coda/coda.h       |  2 +
 4 files changed, 139 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 4f96f808b4dd..5a1016243032 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -180,7 +180,7 @@ static void coda_kfifo_sync_to_device_write(struct coda_ctx *ctx)
 	coda_write(dev, wr_ptr, CODA_REG_BIT_WR_PTR(ctx->reg_idx));
 }
 
-static int coda_bitstream_pad(struct coda_ctx *ctx, u32 size)
+static int coda_h264_bitstream_pad(struct coda_ctx *ctx, u32 size)
 {
 	unsigned char *buf;
 	u32 n;
@@ -206,12 +206,34 @@ static int coda_bitstream_queue(struct coda_ctx *ctx, const u8 *buf, u32 size)
 	return (n < size) ? -ENOSPC : 0;
 }
 
+static u32 coda_buffer_parse_headers(struct coda_ctx *ctx,
+				     struct vb2_v4l2_buffer *src_buf,
+				     u32 payload)
+{
+	u8 *vaddr = vb2_plane_vaddr(&src_buf->vb2_buf, 0);
+	u32 size = 0;
+
+	switch (ctx->codec->src_fourcc) {
+	case V4L2_PIX_FMT_MPEG2:
+		size = coda_mpeg2_parse_headers(ctx, vaddr, payload);
+		break;
+	case V4L2_PIX_FMT_MPEG4:
+		size = coda_mpeg4_parse_headers(ctx, vaddr, payload);
+		break;
+	default:
+		break;
+	}
+
+	return size;
+}
+
 static bool coda_bitstream_try_queue(struct coda_ctx *ctx,
 				     struct vb2_v4l2_buffer *src_buf)
 {
 	unsigned long payload = vb2_get_plane_payload(&src_buf->vb2_buf, 0);
 	u8 *vaddr = vb2_plane_vaddr(&src_buf->vb2_buf, 0);
 	int ret;
+	int i;
 
 	if (coda_get_bitstream_payload(ctx) + payload + 512 >=
 	    ctx->bitstream.size)
@@ -222,10 +244,41 @@ static bool coda_bitstream_try_queue(struct coda_ctx *ctx,
 		return true;
 	}
 
-	/* Add zero padding before the first H.264 buffer, if it is too small */
+	if (ctx->qsequence == 0 && payload < 512) {
+		/*
+		 * Add padding after the first buffer, if it is too small to be
+		 * fetched by the CODA, by repeating the headers. Without
+		 * repeated headers, or the first frame already queued, decoder
+		 * sequence initialization fails with error code 0x2000 on i.MX6
+		 * or error code 0x1 on i.MX51.
+		 */
+		u32 header_size = coda_buffer_parse_headers(ctx, src_buf,
+							    payload);
+
+		if (header_size) {
+			coda_dbg(1, ctx, "pad with %u-byte header\n",
+				 header_size);
+			for (i = payload; i < 512; i += header_size) {
+				ret = coda_bitstream_queue(ctx, vaddr,
+							   header_size);
+				if (ret < 0) {
+					v4l2_err(&ctx->dev->v4l2_dev,
+						 "bitstream buffer overflow\n");
+					return false;
+				}
+				if (ctx->dev->devtype->product == CODA_960)
+					break;
+			}
+		} else {
+			coda_dbg(1, ctx,
+				 "could not parse header, sequence initialization might fail\n");
+		}
+	}
+
+	/* Add padding before the first buffer, if it is too small */
 	if (ctx->qsequence == 0 && payload < 512 &&
 	    ctx->codec->src_fourcc == V4L2_PIX_FMT_H264)
-		coda_bitstream_pad(ctx, 512 - payload);
+		coda_h264_bitstream_pad(ctx, 512 - payload);
 
 	ret = coda_bitstream_queue(ctx, vaddr, payload);
 	if (ret < 0) {
diff --git a/drivers/media/platform/coda/coda-mpeg2.c b/drivers/media/platform/coda/coda-mpeg2.c
index 73e50dabce19..6f3f6721d286 100644
--- a/drivers/media/platform/coda/coda-mpeg2.c
+++ b/drivers/media/platform/coda/coda-mpeg2.c
@@ -42,3 +42,46 @@ int coda_mpeg2_level(int level_idc)
 		return -EINVAL;
 	}
 }
+
+/*
+ * Check if the buffer starts with the MPEG-2 sequence header (with or without
+ * quantization matrix) and extension header, for example:
+ *
+ *   00 00 01 b3 2d 01 e0 34 08 8b a3 81
+ *               10 11 11 12 12 12 13 13 13 13 14 14 14 14 14 15
+ *               15 15 15 15 15 16 16 16 16 16 16 16 17 17 17 17
+ *               17 17 17 17 18 18 18 19 18 18 18 19 1a 1a 1a 1a
+ *               19 1b 1b 1b 1b 1b 1c 1c 1c 1c 1e 1e 1e 1f 1f 21
+ *   00 00 01 b5 14 8a 00 01 00 00
+ *
+ * or:
+ *
+ *   00 00 01 b3 08 00 40 15 ff ff e0 28
+ *   00 00 01 b5 14 8a 00 01 00 00
+ *
+ * Returns the detected header size in bytes or 0.
+ */
+u32 coda_mpeg2_parse_headers(struct coda_ctx *ctx, u8 *buf, u32 size)
+{
+	static const u8 sequence_header_start[4] = { 0x00, 0x00, 0x01, 0xb3 };
+	static const union {
+		u8 extension_start[4];
+		u8 start_code_prefix[3];
+	} u = { { 0x00, 0x00, 0x01, 0xb5 } };
+
+	if (size < 22 ||
+	    memcmp(buf, sequence_header_start, 4) != 0)
+		return 0;
+
+	if ((size == 22 ||
+	     (size >= 25 && memcmp(buf + 22, u.start_code_prefix, 3) == 0)) &&
+	    memcmp(buf + 12, u.extension_start, 4) == 0)
+		return 22;
+
+	if ((size == 86 ||
+	     (size > 89 && memcmp(buf + 86, u.start_code_prefix, 3) == 0)) &&
+	    memcmp(buf + 76, u.extension_start, 4) == 0)
+		return 86;
+
+	return 0;
+}
diff --git a/drivers/media/platform/coda/coda-mpeg4.c b/drivers/media/platform/coda/coda-mpeg4.c
index c3aca763c320..483a4fba1b4f 100644
--- a/drivers/media/platform/coda/coda-mpeg4.c
+++ b/drivers/media/platform/coda/coda-mpeg4.c
@@ -47,3 +47,41 @@ int coda_mpeg4_level(int level_idc)
 		return -EINVAL;
 	}
 }
+
+/*
+ * Check if the buffer starts with the MPEG-4 visual object sequence and visual
+ * object headers, for example:
+ *
+ *   00 00 01 b0 f1
+ *   00 00 01 b5 a9 13 00 00 01 00 00 00 01 20 08
+ *               d4 8d 88 00 f5 04 04 08 14 30 3f
+ *
+ * Returns the detected header size in bytes or 0.
+ */
+u32 coda_mpeg4_parse_headers(struct coda_ctx *ctx, u8 *buf, u32 size)
+{
+	static const u8 vos_start[4] = { 0x00, 0x00, 0x01, 0xb0 };
+	static const union {
+		u8 vo_start[4];
+		u8 start_code_prefix[3];
+	} u = { { 0x00, 0x00, 0x01, 0xb5 } };
+
+	if (size < 30 ||
+	    memcmp(buf, vos_start, 4) != 0 ||
+	    memcmp(buf + 5, u.vo_start, 4) != 0)
+		return 0;
+
+	if (size == 30 ||
+	    (size >= 33 && memcmp(buf + 30, u.start_code_prefix, 3) == 0))
+		return 30;
+
+	if (size == 31 ||
+	    (size >= 34 && memcmp(buf + 31, u.start_code_prefix, 3) == 0))
+		return 31;
+
+	if (size == 32 ||
+	    (size >= 35 && memcmp(buf + 32, u.start_code_prefix, 3) == 0))
+		return 32;
+
+	return 0;
+}
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index 10207e9534c2..12bbd3129269 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -338,8 +338,10 @@ int coda_h264_sps_fixup(struct coda_ctx *ctx, int width, int height, char *buf,
 
 int coda_mpeg2_profile(int profile_idc);
 int coda_mpeg2_level(int level_idc);
+u32 coda_mpeg2_parse_headers(struct coda_ctx *ctx, u8 *buf, u32 size);
 int coda_mpeg4_profile(int profile_idc);
 int coda_mpeg4_level(int level_idc);
+u32 coda_mpeg4_parse_headers(struct coda_ctx *ctx, u8 *buf, u32 size);
 
 void coda_update_profile_level_ctrls(struct coda_ctx *ctx, u8 profile_idc,
 				     u8 level_idc);
-- 
2.20.1

[PATCH 13/28] media: coda: do not enforce 512-byte initial bitstream payload on CODA960

[Philipp Zabel]

On CODA960, sequence initialization can succeed if less than 512 bytes
are ready in the bitstream ring buffer.
On other variants, warn about too small payload in start_streaming.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index b30945fa0a70..dc9bce896003 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1790,7 +1790,9 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 			coda_fill_bitstream(ctx, &list);
 			mutex_unlock(&ctx->bitstream_mutex);
 
-			if (coda_get_bitstream_payload(ctx) < 512) {
+			if (ctx->dev->devtype->product != CODA_960 &&
+			    coda_get_bitstream_payload(ctx) < 512) {
+				v4l2_err(v4l2_dev, "start payload < 512\n");
 				ret = -EINVAL;
 				goto err;
 			}
-- 
2.20.1

[PATCH 14/28] media: coda: flush bitstream ring buffer on decoder restart

[Philipp Zabel]

The bitstream ringbuffer might be in an underrun state after draining,
or it might still contain unread data if the previous decoder stop
command was flagged as immediate. Flush the bitstream ring buffer
during V4L2_DEC_CMD_START to get into a well defined state. Also fill
the bitstream with buffers that have been queued during draining,
to resume decoding immediately.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c    | 20 ++++++++++++++++++++
 drivers/media/platform/coda/coda-common.c |  7 +++++++
 drivers/media/platform/coda/coda.h        |  1 +
 3 files changed, 28 insertions(+)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 5a1016243032..843f92312f47 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -199,6 +199,26 @@ static int coda_h264_bitstream_pad(struct coda_ctx *ctx, u32 size)
 	return (n < size) ? -ENOSPC : 0;
 }
 
+int coda_bitstream_flush(struct coda_ctx *ctx)
+{
+	int ret;
+
+	if (ctx->inst_type != CODA_INST_DECODER || !ctx->use_bit)
+		return 0;
+
+	ret = coda_command_sync(ctx, CODA_COMMAND_DEC_BUF_FLUSH);
+	if (ret < 0) {
+		v4l2_err(&ctx->dev->v4l2_dev, "failed to flush bitstream\n");
+		return ret;
+	}
+
+	kfifo_init(&ctx->bitstream_fifo, ctx->bitstream.vaddr,
+		   ctx->bitstream.size);
+	coda_kfifo_sync_to_device_full(ctx);
+
+	return 0;
+}
+
 static int coda_bitstream_queue(struct coda_ctx *ctx, const u8 *buf, u32 size)
 {
 	u32 n = kfifo_in(&ctx->bitstream_fifo, buf, size);
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index dc9bce896003..ddd819ea13f2 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1052,6 +1052,7 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 			    struct v4l2_decoder_cmd *dc)
 {
 	struct coda_ctx *ctx = fh_to_ctx(fh);
+	struct coda_dev *dev = ctx->dev;
 	struct vb2_queue *dst_vq;
 	int ret;
 
@@ -1061,10 +1062,16 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 
 	switch (dc->cmd) {
 	case V4L2_DEC_CMD_START:
+		mutex_lock(&ctx->bitstream_mutex);
+		mutex_lock(&dev->coda_mutex);
+		coda_bitstream_flush(ctx);
+		mutex_unlock(&dev->coda_mutex);
 		dst_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx,
 					 V4L2_BUF_TYPE_VIDEO_CAPTURE);
 		vb2_clear_last_buffer_dequeued(dst_vq);
 		ctx->bit_stream_param &= ~CODA_BIT_STREAM_END_FLAG;
+		coda_fill_bitstream(ctx, NULL);
+		mutex_unlock(&ctx->bitstream_mutex);
 		break;
 	case V4L2_DEC_CMD_STOP:
 		/* Set the stream-end flag on this context */
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index 12bbd3129269..6911c1c811ce 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -322,6 +322,7 @@ static inline bool coda_bitstream_can_fetch_past(struct coda_ctx *ctx,
 }
 
 bool coda_bitstream_can_fetch_past(struct coda_ctx *ctx, unsigned int pos);
+int coda_bitstream_flush(struct coda_ctx *ctx);
 
 void coda_bit_stream_end_flag(struct coda_ctx *ctx);
 
-- 
2.20.1

[PATCH 15/28] media: coda: increment sequence offset for the last returned frame

[Philipp Zabel]

If no more frames are decoded in bitstream end mode, and a previously
decoded frame has been returned, the firmware still increments the frame
number. To avoid a sequence number mismatch after decoder restart,
increment the sequence_offset correction parameter.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 843f92312f47..bfe6019e68a8 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -2280,6 +2280,9 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		else if (ctx->display_idx < 0)
 			ctx->hold = true;
 	} else if (decoded_idx == -2) {
+		if (ctx->display_idx >= 0 &&
+		    ctx->display_idx < ctx->num_internal_frames)
+			ctx->sequence_offset++;
 		/* no frame was decoded, we still return remaining buffers */
 	} else if (decoded_idx < 0 || decoded_idx >= ctx->num_internal_frames) {
 		v4l2_err(&dev->v4l2_dev,
-- 
2.20.1

[PATCH 16/28] media: coda: allow flagging last output buffer internally

[Philipp Zabel]

Since V4L2_BUF_FLAG_LAST is a CAPTURE only flag, clear it from OUTPUT
buffers in QBUF and DQBUF. This allows to use the flag internally to
signal the last buffer to decode after a decoder stop command was
issued.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index ddd819ea13f2..29d050fbb899 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -879,9 +879,27 @@ static int coda_qbuf(struct file *file, void *priv,
 {
 	struct coda_ctx *ctx = fh_to_ctx(priv);
 
+	if (ctx->inst_type == CODA_INST_DECODER &&
+	    buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		buf->flags &= ~V4L2_BUF_FLAG_LAST;
+
 	return v4l2_m2m_qbuf(file, ctx->fh.m2m_ctx, buf);
 }
 
+static int coda_dqbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
+{
+	struct coda_ctx *ctx = fh_to_ctx(priv);
+	int ret;
+
+	ret = v4l2_m2m_dqbuf(file, ctx->fh.m2m_ctx, buf);
+
+	if (ctx->inst_type == CODA_INST_DECODER &&
+	    buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
+		buf->flags &= ~V4L2_BUF_FLAG_LAST;
+
+	return ret;
+}
+
 static bool coda_buf_is_end_of_stream(struct coda_ctx *ctx,
 				      struct vb2_v4l2_buffer *buf)
 {
@@ -1295,7 +1313,7 @@ static const struct v4l2_ioctl_ops coda_ioctl_ops = {
 
 	.vidioc_qbuf		= coda_qbuf,
 	.vidioc_expbuf		= v4l2_m2m_ioctl_expbuf,
-	.vidioc_dqbuf		= v4l2_m2m_ioctl_dqbuf,
+	.vidioc_dqbuf		= coda_dqbuf,
 	.vidioc_create_bufs	= v4l2_m2m_ioctl_create_bufs,
 	.vidioc_prepare_buf	= v4l2_m2m_ioctl_prepare_buf,
 
-- 
2.20.1

[PATCH 17/28] media: coda: mark the last output buffer on decoder stop command

[Philipp Zabel]

Mark the last output buffer to be decoded and only copy pending queued
output buffers into the bitstream ring buffer in the BIT processor
decoder case.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c    | 3 +++
 drivers/media/platform/coda/coda-common.c | 6 ++++++
 2 files changed, 9 insertions(+)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index bfe6019e68a8..cbcec571a014 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -312,6 +312,9 @@ static bool coda_bitstream_try_queue(struct coda_ctx *ctx,
 	if (ctx == v4l2_m2m_get_curr_priv(ctx->dev->m2m_dev))
 		coda_kfifo_sync_to_device_write(ctx);
 
+	/* Set the stream-end flag after the last buffer is queued */
+	if (src_buf->flags & V4L2_BUF_FLAG_LAST)
+		coda_bit_stream_end_flag(ctx);
 	ctx->hold = false;
 
 	return true;
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 29d050fbb899..9b32b5862aa8 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1071,6 +1071,7 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 {
 	struct coda_ctx *ctx = fh_to_ctx(fh);
 	struct coda_dev *dev = ctx->dev;
+	struct vb2_v4l2_buffer *buf;
 	struct vb2_queue *dst_vq;
 	int ret;
 
@@ -1092,6 +1093,11 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 		mutex_unlock(&ctx->bitstream_mutex);
 		break;
 	case V4L2_DEC_CMD_STOP:
+		buf = v4l2_m2m_last_src_buf(ctx->fh.m2m_ctx);
+		if (buf)
+			/* Mark last buffer */
+			buf->flags |= V4L2_BUF_FLAG_LAST;
+
 		/* Set the stream-end flag on this context */
 		coda_bit_stream_end_flag(ctx);
 		ctx->hold = false;
-- 
2.20.1

[PATCH 18/28] media: coda: only set the stream end flags if there are no more pending output buffers

[Philipp Zabel]

If there are still queued output buffers pending to be copied into the
bitstream ring buffer, setting the stream end flag should be deferred
until the marked last output buffer is written into the bitstream ring
buffer.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 9b32b5862aa8..4002a5b8c9ea 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1098,16 +1098,20 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 			/* Mark last buffer */
 			buf->flags |= V4L2_BUF_FLAG_LAST;
 
-		/* Set the stream-end flag on this context */
-		coda_bit_stream_end_flag(ctx);
-		ctx->hold = false;
-		v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+		if (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) == 0) {
+			/* Set the stream-end flag on this context */
+			coda_bit_stream_end_flag(ctx);
+			ctx->hold = false;
+			v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
 
-		flush_work(&ctx->pic_run_work);
+			flush_work(&ctx->pic_run_work);
+
+			/* If there is no buffer in flight, wake up */
+			if (!ctx->streamon_out ||
+			    ctx->qsequence == ctx->osequence)
+				coda_wake_up_capture_queue(ctx);
+		}
 
-		/* If there is no buffer in flight, wake up */
-		if (!ctx->streamon_out || ctx->qsequence == ctx->osequence)
-			coda_wake_up_capture_queue(ctx);
 		break;
 	default:
 		return -EINVAL;
-- 
2.20.1

[PATCH 19/28] media: coda: mark the last output buffer on encoder stop command

[Philipp Zabel]

Mark the last output buffer to be encoded.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 4002a5b8c9ea..c55124e8b4c8 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1037,12 +1037,17 @@ static int coda_encoder_cmd(struct file *file, void *fh,
 			    struct v4l2_encoder_cmd *ec)
 {
 	struct coda_ctx *ctx = fh_to_ctx(fh);
+	struct vb2_v4l2_buffer *buf;
 	int ret;
 
 	ret = coda_try_encoder_cmd(file, fh, ec);
 	if (ret < 0)
 		return ret;
 
+	buf = v4l2_m2m_last_src_buf(ctx->fh.m2m_ctx);
+	if (buf)
+		buf->flags |= V4L2_BUF_FLAG_LAST;
+
 	/* Set the stream-end flag on this context */
 	ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
 
-- 
2.20.1

[PATCH 20/28] media: coda: retire coda_buf_is_end_of_stream

[Philipp Zabel]

Using the output queue sequence counter to determine the last buffer to
be encoded or decoded always was fragile at best. Now that we have the
last buffer flag propagating from the output queue to the capture queue
correctly, this is not needed anymore.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index c55124e8b4c8..9349df773077 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -900,13 +900,6 @@ static int coda_dqbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 	return ret;
 }
 
-static bool coda_buf_is_end_of_stream(struct coda_ctx *ctx,
-				      struct vb2_v4l2_buffer *buf)
-{
-	return ((ctx->bit_stream_param & CODA_BIT_STREAM_END_FLAG) &&
-		(buf->sequence == (ctx->qsequence - 1)));
-}
-
 void coda_m2m_buf_done(struct coda_ctx *ctx, struct vb2_v4l2_buffer *buf,
 		       enum vb2_buffer_state state)
 {
@@ -914,11 +907,8 @@ void coda_m2m_buf_done(struct coda_ctx *ctx, struct vb2_v4l2_buffer *buf,
 		.type = V4L2_EVENT_EOS
 	};
 
-	if (coda_buf_is_end_of_stream(ctx, buf)) {
-		buf->flags |= V4L2_BUF_FLAG_LAST;
-
+	if (buf->flags & V4L2_BUF_FLAG_LAST)
 		v4l2_event_queue_fh(&ctx->fh, &eos_event);
-	}
 
 	v4l2_m2m_buf_done(buf, state);
 }
-- 
2.20.1

[PATCH 21/28] media: coda: only wake up capture queue if no pending buffers to encode

[Philipp Zabel]

If there are no pending queued output buffers to be encoded, waking up
the capture queue with -EPIPE signals end of stream. If there are
pending buffers on the other hand, setting the V4L2_BUF_FLAG_LAST on
the resulting encoded capture buffers is all that is needed.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-common.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 9349df773077..0d64ddc49494 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1035,17 +1035,18 @@ static int coda_encoder_cmd(struct file *file, void *fh,
 		return ret;
 
 	buf = v4l2_m2m_last_src_buf(ctx->fh.m2m_ctx);
-	if (buf)
+	if (buf) {
 		buf->flags |= V4L2_BUF_FLAG_LAST;
+	} else {
+		/* Set the stream-end flag on this context */
+		ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
 
-	/* Set the stream-end flag on this context */
-	ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
-
-	flush_work(&ctx->pic_run_work);
+		flush_work(&ctx->pic_run_work);
 
-	/* If there is no buffer in flight, wake up */
-	if (!ctx->streamon_out || ctx->qsequence == ctx->osequence)
-		coda_wake_up_capture_queue(ctx);
+		/* If there is no buffer in flight, wake up */
+		if (!ctx->streamon_out || ctx->qsequence == ctx->osequence)
+			coda_wake_up_capture_queue(ctx);
+	}
 
 	return 0;
 }
-- 
2.20.1

[PATCH 22/28] media: coda: flag the last encoded buffer

[Philipp Zabel]

Use the flagged last output buffer to also flag the corresponding
capture buffer after encoding. This causes the end of stream event
to be issued and the buffer to be dequeued with the last flag set.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index cbcec571a014..7bcdfe8dcf3d 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -1565,13 +1565,14 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 	coda_read(dev, CODA_RET_ENC_PIC_SLICE_NUM);
 	coda_read(dev, CODA_RET_ENC_PIC_FLAG);
 
-	if (coda_read(dev, CODA_RET_ENC_PIC_TYPE) == 0) {
+	dst_buf->flags &= ~(V4L2_BUF_FLAG_KEYFRAME |
+			    V4L2_BUF_FLAG_PFRAME |
+			    V4L2_BUF_FLAG_LAST);
+	if (coda_read(dev, CODA_RET_ENC_PIC_TYPE) == 0)
 		dst_buf->flags |= V4L2_BUF_FLAG_KEYFRAME;
-		dst_buf->flags &= ~V4L2_BUF_FLAG_PFRAME;
-	} else {
+	else
 		dst_buf->flags |= V4L2_BUF_FLAG_PFRAME;
-		dst_buf->flags &= ~V4L2_BUF_FLAG_KEYFRAME;
-	}
+	dst_buf->flags |= src_buf->flags & V4L2_BUF_FLAG_LAST;
 
 	v4l2_m2m_buf_copy_metadata(src_buf, dst_buf, false);
 
@@ -1584,8 +1585,9 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 	if (ctx->gopcounter < 0)
 		ctx->gopcounter = ctx->params.gop_size - 1;
 
-	coda_dbg(1, ctx, "job finished: encoded %c frame (%d)\n",
-		 coda_frame_type_char(dst_buf->flags), dst_buf->sequence);
+	coda_dbg(1, ctx, "job finished: encoded %c frame (%d)%s\n",
+		 coda_frame_type_char(dst_buf->flags), dst_buf->sequence,
+		 (dst_buf->flags & V4L2_BUF_FLAG_LAST) ? " (last)" : "");
 }
 
 static void coda_seq_end_work(struct work_struct *work)
-- 
2.20.1

[PATCH 23/28] media: coda: lock capture queue wakeup against encoder stop command

[Philipp Zabel]

Make sure that an encoder stop command running concurrently with an
encoder finish_run always either flags the last returned buffer or wakes
up the capture queue to signal the end of stream condition afterwards.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c    |  7 +++++++
 drivers/media/platform/coda/coda-common.c | 19 ++++++++++++++-----
 drivers/media/platform/coda/coda.h        |  6 ++++++
 3 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 7bcdfe8dcf3d..44085e3d43d5 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -1540,6 +1540,12 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 	struct coda_dev *dev = ctx->dev;
 	u32 wr_ptr, start_ptr;
 
+	/*
+	 * Lock to make sure that an encoder stop command running in parallel
+	 * will either already have marked src_buf as last, or it will wake up
+	 * the capture queue after the buffers are returned.
+	 */
+	mutex_lock(&ctx->wakeup_mutex);
 	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 
@@ -1580,6 +1586,7 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 
 	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	coda_m2m_buf_done(ctx, dst_buf, VB2_BUF_STATE_DONE);
+	mutex_unlock(&ctx->wakeup_mutex);
 
 	ctx->gopcounter--;
 	if (ctx->gopcounter < 0)
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 0d64ddc49494..829d1e565911 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1034,19 +1034,27 @@ static int coda_encoder_cmd(struct file *file, void *fh,
 	if (ret < 0)
 		return ret;
 
+	mutex_lock(&ctx->wakeup_mutex);
 	buf = v4l2_m2m_last_src_buf(ctx->fh.m2m_ctx);
 	if (buf) {
+		/*
+		 * If the last output buffer is still on the queue, make sure
+		 * that decoder finish_run will see the last flag and report it
+		 * to userspace.
+		 */
 		buf->flags |= V4L2_BUF_FLAG_LAST;
 	} else {
 		/* Set the stream-end flag on this context */
 		ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
 
-		flush_work(&ctx->pic_run_work);
-
-		/* If there is no buffer in flight, wake up */
-		if (!ctx->streamon_out || ctx->qsequence == ctx->osequence)
-			coda_wake_up_capture_queue(ctx);
+		/*
+		 * If the last output buffer has already been taken from the
+		 * queue, wake up the capture queue and signal end of stream
+		 * via the -EPIPE mechanism.
+		 */
+		coda_wake_up_capture_queue(ctx);
 	}
+	mutex_unlock(&ctx->wakeup_mutex);
 
 	return 0;
 }
@@ -2466,6 +2474,7 @@ static int coda_open(struct file *file)
 
 	mutex_init(&ctx->bitstream_mutex);
 	mutex_init(&ctx->buffer_mutex);
+	mutex_init(&ctx->wakeup_mutex);
 	INIT_LIST_HEAD(&ctx->buffer_meta_list);
 	spin_lock_init(&ctx->buffer_meta_lock);
 
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index 6911c1c811ce..97845e58fb8b 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -258,6 +258,12 @@ struct coda_ctx {
 	bool				use_bit;
 	bool				use_vdoa;
 	struct vdoa_ctx			*vdoa;
+	/*
+	 * wakeup mutex used to serialize encoder stop command and finish_run,
+	 * ensures that finish_run always either flags the last returned buffer
+	 * or wakes up the capture queue to signal EOS afterwards.
+	 */
+	struct mutex			wakeup_mutex;
 };
 
 extern int coda_debug;
-- 
2.20.1

[PATCH 24/28] media: coda: mark last pending buffer or last meta on decoder stop command

[Philipp Zabel]

If there is still a buffer pending, mark it as the last buffer. It will
create a meta that is flagged as last when the buffer is copied into the
bitstream ring buffer. If there are no more buffers pending, find the
last bitstream meta and mark it as last. If there is no bitstream meta
either, wake up the capture queue as there will be no more decoded
frames.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c    |  4 +++
 drivers/media/platform/coda/coda-common.c | 44 +++++++++++++++++++----
 drivers/media/platform/coda/coda.h        |  1 +
 3 files changed, 42 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 44085e3d43d5..1157454e3bc8 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -402,6 +402,9 @@ void coda_fill_bitstream(struct coda_ctx *ctx, struct list_head *buffer_list)
 				meta->timestamp = src_buf->vb2_buf.timestamp;
 				meta->start = start;
 				meta->end = ctx->bitstream_fifo.kfifo.in;
+				meta->last = src_buf->flags & V4L2_BUF_FLAG_LAST;
+				if (meta->last)
+					coda_dbg(1, ctx, "marking last meta");
 				spin_lock(&ctx->buffer_meta_lock);
 				list_add_tail(&meta->list,
 					      &ctx->buffer_meta_list);
@@ -2334,6 +2337,7 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 			memset(&decoded_frame->meta, 0,
 			       sizeof(struct coda_buffer_meta));
 			decoded_frame->meta.sequence = val;
+			decoded_frame->meta.last = false;
 			ctx->sequence_offset++;
 		}
 
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 829d1e565911..cd33c260409e 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1077,6 +1077,8 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 	struct coda_dev *dev = ctx->dev;
 	struct vb2_v4l2_buffer *buf;
 	struct vb2_queue *dst_vq;
+	bool stream_end;
+	bool wakeup;
 	int ret;
 
 	ret = coda_try_decoder_cmd(file, fh, dc);
@@ -1097,23 +1099,51 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 		mutex_unlock(&ctx->bitstream_mutex);
 		break;
 	case V4L2_DEC_CMD_STOP:
+		stream_end = false;
+		wakeup = false;
+
 		buf = v4l2_m2m_last_src_buf(ctx->fh.m2m_ctx);
-		if (buf)
+		if (buf) {
+			coda_dbg(1, ctx, "marking last pending buffer\n");
+
 			/* Mark last buffer */
 			buf->flags |= V4L2_BUF_FLAG_LAST;
 
-		if (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) == 0) {
+			if (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) == 0) {
+				coda_dbg(1, ctx, "all remaining buffers queued\n");
+				stream_end = true;
+			}
+		} else {
+			coda_dbg(1, ctx, "marking last meta\n");
+
+			/* Mark last meta */
+			spin_lock(&ctx->buffer_meta_lock);
+			if (!list_empty(&ctx->buffer_meta_list)) {
+				struct coda_buffer_meta *meta;
+
+				meta = list_last_entry(&ctx->buffer_meta_list,
+						       struct coda_buffer_meta,
+						       list);
+				meta->last = true;
+				stream_end = true;
+			} else {
+				wakeup = true;
+			}
+			spin_unlock(&ctx->buffer_meta_lock);
+		}
+
+		if (stream_end) {
+			coda_dbg(1, ctx, "all remaining buffers queued\n");
+
 			/* Set the stream-end flag on this context */
 			coda_bit_stream_end_flag(ctx);
 			ctx->hold = false;
 			v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+		}
 
-			flush_work(&ctx->pic_run_work);
-
+		if (wakeup) {
 			/* If there is no buffer in flight, wake up */
-			if (!ctx->streamon_out ||
-			    ctx->qsequence == ctx->osequence)
-				coda_wake_up_capture_queue(ctx);
+			coda_wake_up_capture_queue(ctx);
 		}
 
 		break;
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index 97845e58fb8b..5c183c1944fe 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -146,6 +146,7 @@ struct coda_buffer_meta {
 	u64			timestamp;
 	unsigned int		start;
 	unsigned int		end;
+	bool			last;
 };
 
 /* Per-queue, driver-specific private data */
-- 
2.20.1

[PATCH 25/28] media: coda: mark last returned frame

[Philipp Zabel]

If reordering is not enabled, the last decoded frame has to be the last
returned buffer. Otherwise wait for the firmware to report no more
frame to display. In that case the return buffer is the last one as
well, and can be reported as such.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 1157454e3bc8..167a92772c84 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -2381,6 +2381,23 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 					     V4L2_BUF_FLAG_BFRAME);
 		dst_buf->flags |= ready_frame->type;
 		meta = &ready_frame->meta;
+		if (meta->last && !coda_reorder_enable(ctx)) {
+			/*
+			 * If this was the last decoded frame, and reordering
+			 * is disabled, this will be the last display frame.
+			 */
+			coda_dbg(1, ctx, "last meta, marking as last frame\n");
+			dst_buf->flags |= V4L2_BUF_FLAG_LAST;
+		} else if (ctx->bit_stream_param & CODA_BIT_STREAM_END_FLAG &&
+			   display_idx == -1) {
+			/*
+			 * If there is no designated presentation frame anymore,
+			 * this frame has to be the last one.
+			 */
+			coda_dbg(1, ctx,
+				 "no more frames to return, marking as last frame\n");
+			dst_buf->flags |= V4L2_BUF_FLAG_LAST;
+		}
 		dst_buf->timecode = meta->timecode;
 		dst_buf->vb2_buf.timestamp = meta->timestamp;
 
-- 
2.20.1

[PATCH 26/28] media: coda: store device pointer in driver structure instead of pdev

[Philipp Zabel]

From: Philipp Zabel <philipp.zabel@gmail.com>

Currently the platform device pointer is stored in struct coda_dev,
only to convert it into a device pointer wherever it is used. Just
store the device pointer directly.

Signed-off-by: Philipp Zabel <philipp.zabel@gmail.com>
---
 drivers/media/platform/coda/coda-bit.c    |  7 +++--
 drivers/media/platform/coda/coda-common.c | 33 +++++++++++------------
 drivers/media/platform/coda/coda.h        |  2 +-
 3 files changed, 19 insertions(+), 23 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index 167a92772c84..de6a4216a182 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -1667,8 +1667,7 @@ static int coda_alloc_bitstream_buffer(struct coda_ctx *ctx,
 		return 0;
 
 	ctx->bitstream.size = roundup_pow_of_two(q_data->sizeimage * 2);
-	ctx->bitstream.vaddr = dma_alloc_wc(&ctx->dev->plat_dev->dev,
-					    ctx->bitstream.size,
+	ctx->bitstream.vaddr = dma_alloc_wc(ctx->dev->dev, ctx->bitstream.size,
 					    &ctx->bitstream.paddr, GFP_KERNEL);
 	if (!ctx->bitstream.vaddr) {
 		v4l2_err(&ctx->dev->v4l2_dev,
@@ -1686,8 +1685,8 @@ static void coda_free_bitstream_buffer(struct coda_ctx *ctx)
 	if (ctx->bitstream.vaddr == NULL)
 		return;
 
-	dma_free_wc(&ctx->dev->plat_dev->dev, ctx->bitstream.size,
-		    ctx->bitstream.vaddr, ctx->bitstream.paddr);
+	dma_free_wc(ctx->dev->dev, ctx->bitstream.size, ctx->bitstream.vaddr,
+		    ctx->bitstream.paddr);
 	ctx->bitstream.vaddr = NULL;
 	kfifo_init(&ctx->bitstream_fifo, NULL, 0);
 }
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index cd33c260409e..8060a3425325 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1413,7 +1413,7 @@ static void coda_pic_run_work(struct work_struct *work)
 
 	if (!wait_for_completion_timeout(&ctx->completion,
 					 msecs_to_jiffies(1000))) {
-		dev_err(&dev->plat_dev->dev, "CODA PIC_RUN timeout\n");
+		dev_err(dev->dev, "CODA PIC_RUN timeout\n");
 
 		ctx->hold = true;
 
@@ -1797,7 +1797,7 @@ static void coda_buf_queue(struct vb2_buffer *vb)
 int coda_alloc_aux_buf(struct coda_dev *dev, struct coda_aux_buf *buf,
 		       size_t size, const char *name, struct dentry *parent)
 {
-	buf->vaddr = dma_alloc_coherent(&dev->plat_dev->dev, size, &buf->paddr,
+	buf->vaddr = dma_alloc_coherent(dev->dev, size, &buf->paddr,
 					GFP_KERNEL);
 	if (!buf->vaddr) {
 		v4l2_err(&dev->v4l2_dev,
@@ -1814,7 +1814,7 @@ int coda_alloc_aux_buf(struct coda_dev *dev, struct coda_aux_buf *buf,
 		buf->dentry = debugfs_create_blob(name, 0644, parent,
 						  &buf->blob);
 		if (!buf->dentry)
-			dev_warn(&dev->plat_dev->dev,
+			dev_warn(dev->dev,
 				 "failed to create debugfs entry %s\n", name);
 	}
 
@@ -1825,8 +1825,7 @@ void coda_free_aux_buf(struct coda_dev *dev,
 		       struct coda_aux_buf *buf)
 {
 	if (buf->vaddr) {
-		dma_free_coherent(&dev->plat_dev->dev, buf->size,
-				  buf->vaddr, buf->paddr);
+		dma_free_coherent(dev->dev, buf->size, buf->vaddr, buf->paddr);
 		buf->vaddr = NULL;
 		buf->size = 0;
 		debugfs_remove(buf->dentry);
@@ -2344,7 +2343,7 @@ static int coda_queue_init(struct coda_ctx *ctx, struct vb2_queue *vq)
 	 * queues to have at least one buffer queued.
 	 */
 	vq->min_buffers_needed = 1;
-	vq->dev = &ctx->dev->plat_dev->dev;
+	vq->dev = ctx->dev->dev;
 
 	return vb2_queue_init(vq);
 }
@@ -2469,7 +2468,7 @@ static int coda_open(struct file *file)
 	ctx->use_vdoa = false;
 
 	/* Power up and upload firmware if necessary */
-	ret = pm_runtime_get_sync(&dev->plat_dev->dev);
+	ret = pm_runtime_get_sync(dev->dev);
 	if (ret < 0) {
 		v4l2_err(&dev->v4l2_dev, "failed to power up: %d\n", ret);
 		goto err_pm_get;
@@ -2517,7 +2516,7 @@ static int coda_open(struct file *file)
 err_clk_ahb:
 	clk_disable_unprepare(dev->clk_per);
 err_clk_per:
-	pm_runtime_put_sync(&dev->plat_dev->dev);
+	pm_runtime_put_sync(dev->dev);
 err_pm_get:
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
@@ -2556,7 +2555,7 @@ static int coda_release(struct file *file)
 	v4l2_ctrl_handler_free(&ctx->ctrls);
 	clk_disable_unprepare(dev->clk_ahb);
 	clk_disable_unprepare(dev->clk_per);
-	pm_runtime_put_sync(&dev->plat_dev->dev);
+	pm_runtime_put_sync(dev->dev);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	ida_free(&dev->ida, ctx->idx);
@@ -2751,18 +2750,16 @@ static int coda_firmware_request(struct coda_dev *dev)
 
 	fw = dev->devtype->firmware[dev->firmware];
 
-	dev_dbg(&dev->plat_dev->dev, "requesting firmware '%s' for %s\n", fw,
+	dev_dbg(dev->dev, "requesting firmware '%s' for %s\n", fw,
 		coda_product_name(dev->devtype->product));
 
-	return request_firmware_nowait(THIS_MODULE, true, fw,
-				       &dev->plat_dev->dev, GFP_KERNEL, dev,
-				       coda_fw_callback);
+	return request_firmware_nowait(THIS_MODULE, true, fw, dev->dev,
+				       GFP_KERNEL, dev, coda_fw_callback);
 }
 
 static void coda_fw_callback(const struct firmware *fw, void *context)
 {
 	struct coda_dev *dev = context;
-	struct platform_device *pdev = dev->plat_dev;
 	int i, ret;
 
 	if (!fw) {
@@ -2780,7 +2777,7 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
 		 * firmware requests, report that the fallback firmware was
 		 * found.
 		 */
-		dev_info(&pdev->dev, "Using fallback firmware %s\n",
+		dev_info(dev->dev, "Using fallback firmware %s\n",
 			 dev->devtype->firmware[dev->firmware]);
 	}
 
@@ -2819,7 +2816,7 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
 		}
 	}
 
-	pm_runtime_put_sync(&pdev->dev);
+	pm_runtime_put_sync(dev->dev);
 	return;
 
 rel_vfd:
@@ -2827,7 +2824,7 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
 		video_unregister_device(&dev->vfd[i]);
 	v4l2_m2m_release(dev->m2m_dev);
 put_pm:
-	pm_runtime_put_sync(&pdev->dev);
+	pm_runtime_put_sync(dev->dev);
 }
 
 enum coda_platform {
@@ -2960,7 +2957,7 @@ static int coda_probe(struct platform_device *pdev)
 
 	spin_lock_init(&dev->irqlock);
 
-	dev->plat_dev = pdev;
+	dev->dev = &pdev->dev;
 	dev->clk_per = devm_clk_get(&pdev->dev, "per");
 	if (IS_ERR(dev->clk_per)) {
 		dev_err(&pdev->dev, "Could not get per clock\n");
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index 5c183c1944fe..502a6272629a 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -70,7 +70,7 @@ struct coda_aux_buf {
 struct coda_dev {
 	struct v4l2_device	v4l2_dev;
 	struct video_device	vfd[5];
-	struct platform_device	*plat_dev;
+	struct device		*dev;
 	const struct coda_devtype *devtype;
 	int			firmware;
 	struct vdoa_data	*vdoa;
-- 
2.20.1

[PATCH 27/28] media: coda: add coda_slice_mode() function

[Philipp Zabel]

Changing slice mode dynamically while encoding will require to calculate
the register value again, so split it out into a separate function.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c | 45 ++++++++++++++------------
 1 file changed, 24 insertions(+), 21 deletions(-)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index de6a4216a182..b59cb16f75a1 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -675,6 +675,29 @@ static int coda_encode_header(struct coda_ctx *ctx, struct vb2_v4l2_buffer *buf,
 	return 0;
 }
 
+static u32 coda_slice_mode(struct coda_ctx *ctx)
+{
+	int size, unit;
+
+	switch (ctx->params.slice_mode) {
+	case V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE:
+	default:
+		return 0;
+	case V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB:
+		size = ctx->params.slice_max_mb;
+		unit = 1;
+		break;
+	case V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES:
+		size = ctx->params.slice_max_bits;
+		unit = 0;
+		break;
+	}
+
+	return ((size & CODA_SLICING_SIZE_MASK) << CODA_SLICING_SIZE_OFFSET) |
+	       ((unit & CODA_SLICING_UNIT_MASK) << CODA_SLICING_UNIT_OFFSET) |
+	       ((1 & CODA_SLICING_MODE_MASK) << CODA_SLICING_MODE_OFFSET);
+}
+
 static phys_addr_t coda_iram_alloc(struct coda_iram_info *iram, size_t size)
 {
 	phys_addr_t ret;
@@ -1113,27 +1136,7 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 	 * in JPEG mode
 	 */
 	if (dst_fourcc != V4L2_PIX_FMT_JPEG) {
-		switch (ctx->params.slice_mode) {
-		case V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE:
-			value = 0;
-			break;
-		case V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB:
-			value  = (ctx->params.slice_max_mb &
-				  CODA_SLICING_SIZE_MASK)
-				 << CODA_SLICING_SIZE_OFFSET;
-			value |= (1 & CODA_SLICING_UNIT_MASK)
-				 << CODA_SLICING_UNIT_OFFSET;
-			value |=  1 & CODA_SLICING_MODE_MASK;
-			break;
-		case V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES:
-			value  = (ctx->params.slice_max_bits &
-				  CODA_SLICING_SIZE_MASK)
-				 << CODA_SLICING_SIZE_OFFSET;
-			value |= (0 & CODA_SLICING_UNIT_MASK)
-				 << CODA_SLICING_UNIT_OFFSET;
-			value |=  1 & CODA_SLICING_MODE_MASK;
-			break;
-		}
+		value = coda_slice_mode(ctx);
 		coda_write(dev, value, CODA_CMD_ENC_SEQ_SLICE_MODE);
 		value = ctx->params.gop_size;
 		coda_write(dev, value, CODA_CMD_ENC_SEQ_GOP_SIZE);
-- 
2.20.1

[PATCH 28/28] media: coda: encoder parameter change support

[Philipp Zabel]

Add support for dynamically changing the GOP size, bitrate, frame rate,
constant intra quantization parameter, number of intra refresh macro
blocks and slice mode parameters.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda/coda-bit.c    | 83 +++++++++++++++++++++++
 drivers/media/platform/coda/coda-common.c |  7 ++
 drivers/media/platform/coda/coda.h        |  7 ++
 drivers/media/platform/coda/coda_regs.h   | 18 +++++
 4 files changed, 115 insertions(+)

diff --git a/drivers/media/platform/coda/coda-bit.c b/drivers/media/platform/coda/coda-bit.c
index b59cb16f75a1..00c7bed3dd57 100644
--- a/drivers/media/platform/coda/coda-bit.c
+++ b/drivers/media/platform/coda/coda-bit.c
@@ -698,6 +698,79 @@ static u32 coda_slice_mode(struct coda_ctx *ctx)
 	       ((1 & CODA_SLICING_MODE_MASK) << CODA_SLICING_MODE_OFFSET);
 }
 
+static int coda_enc_param_change(struct coda_ctx *ctx)
+{
+	struct coda_dev *dev = ctx->dev;
+	u32 change_enable = 0;
+	u32 success;
+	int ret;
+
+	if (ctx->params.gop_size_changed) {
+		change_enable |= CODA_PARAM_CHANGE_RC_GOP;
+		coda_write(dev, ctx->params.gop_size,
+			   CODA_CMD_ENC_PARAM_RC_GOP);
+		ctx->gopcounter = ctx->params.gop_size - 1;
+		ctx->params.gop_size_changed = false;
+	}
+	if (ctx->params.h264_intra_qp_changed) {
+		coda_dbg(1, ctx, "parameter change: intra Qp %u\n",
+			 ctx->params.h264_intra_qp);
+
+		if (ctx->params.bitrate) {
+			change_enable |= CODA_PARAM_CHANGE_RC_INTRA_QP;
+			coda_write(dev, ctx->params.h264_intra_qp,
+				   CODA_CMD_ENC_PARAM_RC_INTRA_QP);
+		}
+		ctx->params.h264_intra_qp_changed = false;
+	}
+	if (ctx->params.bitrate_changed) {
+		coda_dbg(1, ctx, "parameter change: bitrate %u kbit/s\n",
+			 ctx->params.bitrate);
+		change_enable |= CODA_PARAM_CHANGE_RC_BITRATE;
+		coda_write(dev, ctx->params.bitrate,
+			   CODA_CMD_ENC_PARAM_RC_BITRATE);
+		ctx->params.bitrate_changed = false;
+	}
+	if (ctx->params.framerate_changed) {
+		coda_dbg(1, ctx, "parameter change: frame rate %u/%u Hz\n",
+			 ctx->params.framerate & 0xffff,
+			 (ctx->params.framerate >> 16) + 1);
+		change_enable |= CODA_PARAM_CHANGE_RC_FRAME_RATE;
+		coda_write(dev, ctx->params.framerate,
+			   CODA_CMD_ENC_PARAM_RC_FRAME_RATE);
+		ctx->params.framerate_changed = false;
+	}
+	if (ctx->params.intra_refresh_changed) {
+		coda_dbg(1, ctx, "parameter change: intra refresh MBs %u\n",
+			 ctx->params.intra_refresh);
+		change_enable |= CODA_PARAM_CHANGE_INTRA_MB_NUM;
+		coda_write(dev, ctx->params.intra_refresh,
+			   CODA_CMD_ENC_PARAM_INTRA_MB_NUM);
+		ctx->params.intra_refresh_changed = false;
+	}
+	if (ctx->params.slice_mode_changed) {
+		change_enable |= CODA_PARAM_CHANGE_SLICE_MODE;
+		coda_write(dev, coda_slice_mode(ctx),
+			   CODA_CMD_ENC_PARAM_SLICE_MODE);
+		ctx->params.slice_mode_changed = false;
+	}
+
+	if (!change_enable)
+		return 0;
+
+	coda_write(dev, change_enable, CODA_CMD_ENC_PARAM_CHANGE_ENABLE);
+
+	ret = coda_command_sync(ctx, CODA_COMMAND_RC_CHANGE_PARAMETER);
+	if (ret < 0)
+		return ret;
+
+	success = coda_read(dev, CODA_RET_ENC_PARAM_CHANGE_SUCCESS);
+	if (success != 1)
+		coda_dbg(1, ctx, "parameter change failed: %u\n", success);
+
+	return 0;
+}
+
 static phys_addr_t coda_iram_alloc(struct coda_iram_info *iram, size_t size)
 {
 	phys_addr_t ret;
@@ -1143,6 +1216,9 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 	}
 
 	if (ctx->params.bitrate) {
+		ctx->params.bitrate_changed = false;
+		ctx->params.h264_intra_qp_changed = false;
+
 		/* Rate control enabled */
 		value = (ctx->params.bitrate & CODA_RATECONTROL_BITRATE_MASK)
 			<< CODA_RATECONTROL_BITRATE_OFFSET;
@@ -1397,6 +1473,13 @@ static int coda_prepare_encode(struct coda_ctx *ctx)
 	u32 rot_mode = 0;
 	u32 dst_fourcc;
 	u32 reg;
+	int ret;
+
+	ret = coda_enc_param_change(ctx);
+	if (ret < 0) {
+		v4l2_warn(&ctx->dev->v4l2_dev, "parameter change failed: %d\n",
+			  ret);
+	}
 
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
diff --git a/drivers/media/platform/coda/coda-common.c b/drivers/media/platform/coda/coda-common.c
index 8060a3425325..e6bbe1f6cd51 100644
--- a/drivers/media/platform/coda/coda-common.c
+++ b/drivers/media/platform/coda/coda-common.c
@@ -1317,6 +1317,7 @@ static int coda_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
 	tpf = &a->parm.output.timeperframe;
 	coda_approximate_timeperframe(tpf);
 	ctx->params.framerate = coda_timeperframe_to_frate(tpf);
+	ctx->params.framerate_changed = true;
 
 	return 0;
 }
@@ -2033,12 +2034,14 @@ static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
 		break;
 	case V4L2_CID_MPEG_VIDEO_BITRATE:
 		ctx->params.bitrate = ctrl->val / 1000;
+		ctx->params.bitrate_changed = true;
 		break;
 	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
 		ctx->params.gop_size = ctrl->val;
 		break;
 	case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP:
 		ctx->params.h264_intra_qp = ctrl->val;
+		ctx->params.h264_intra_qp_changed = true;
 		break;
 	case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP:
 		ctx->params.h264_inter_qp = ctrl->val;
@@ -2086,17 +2089,21 @@ static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
 		break;
 	case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE:
 		ctx->params.slice_mode = ctrl->val;
+		ctx->params.slice_mode_changed = true;
 		break;
 	case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB:
 		ctx->params.slice_max_mb = ctrl->val;
+		ctx->params.slice_mode_changed = true;
 		break;
 	case V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES:
 		ctx->params.slice_max_bits = ctrl->val * 8;
+		ctx->params.slice_mode_changed = true;
 		break;
 	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
 		break;
 	case V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB:
 		ctx->params.intra_refresh = ctrl->val;
+		ctx->params.intra_refresh_changed = true;
 		break;
 	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:
 		ctx->params.force_ipicture = true;
diff --git a/drivers/media/platform/coda/coda.h b/drivers/media/platform/coda/coda.h
index 502a6272629a..848bf1da401e 100644
--- a/drivers/media/platform/coda/coda.h
+++ b/drivers/media/platform/coda/coda.h
@@ -137,6 +137,12 @@ struct coda_params {
 	u32			slice_max_bits;
 	u32			slice_max_mb;
 	bool			force_ipicture;
+	bool			gop_size_changed;
+	bool			bitrate_changed;
+	bool			framerate_changed;
+	bool			h264_intra_qp_changed;
+	bool			intra_refresh_changed;
+	bool			slice_mode_changed;
 };
 
 struct coda_buffer_meta {
@@ -254,6 +260,7 @@ struct coda_ctx {
 	u32				bit_stream_param;
 	u32				frm_dis_flg;
 	u32				frame_mem_ctrl;
+	u32				para_change;
 	int				display_idx;
 	struct dentry			*debugfs_entry;
 	bool				use_bit;
diff --git a/drivers/media/platform/coda/coda_regs.h b/drivers/media/platform/coda/coda_regs.h
index 4d503f472397..b17464b56d3d 100644
--- a/drivers/media/platform/coda/coda_regs.h
+++ b/drivers/media/platform/coda/coda_regs.h
@@ -342,6 +342,24 @@
 #define CODA_CMD_ENC_SEQ_JPG_THUMB_SIZE				0x1a4
 #define CODA_CMD_ENC_SEQ_JPG_THUMB_OFFSET			0x1a8
 
+/* Encoder Parameter Change */
+#define CODA_CMD_ENC_PARAM_CHANGE_ENABLE	0x180
+#define		CODA_PARAM_CHANGE_RC_GOP			BIT(0)
+#define		CODA_PARAM_CHANGE_RC_INTRA_QP			BIT(1)
+#define		CODA_PARAM_CHANGE_RC_BITRATE			BIT(2)
+#define		CODA_PARAM_CHANGE_RC_FRAME_RATE			BIT(3)
+#define		CODA_PARAM_CHANGE_INTRA_MB_NUM			BIT(4)
+#define		CODA_PARAM_CHANGE_SLICE_MODE			BIT(5)
+#define		CODA_PARAM_CHANGE_HEC_MODE			BIT(6)
+#define CODA_CMD_ENC_PARAM_RC_GOP		0x184
+#define CODA_CMD_ENC_PARAM_RC_INTRA_QP		0x188
+#define CODA_CMD_ENC_PARAM_RC_BITRATE		0x18c
+#define CODA_CMD_ENC_PARAM_RC_FRAME_RATE	0x190
+#define CODA_CMD_ENC_PARAM_INTRA_MB_NUM		0x194
+#define CODA_CMD_ENC_PARAM_SLICE_MODE		0x198
+#define CODA_CMD_ENC_PARAM_HEC_MODE		0x19c
+#define CODA_RET_ENC_PARAM_CHANGE_SUCCESS	0x1c0
+
 /* Encoder Picture Run */
 #define CODA9_CMD_ENC_PIC_SRC_INDEX		0x180
 #define CODA9_CMD_ENC_PIC_SRC_STRIDE		0x184
-- 
2.20.1