[PATCH 00/30] Initial CODA960 (i.MX6 VPU) support

[PATCH 00/30] Initial CODA960 (i.MX6 VPU) support

[Philipp Zabel]

Hi,

the following series adds initial support for the CODA960 Video
Processing Unit on i.MX6Q/D/DL/S SoCs to the coda driver.

This series contains a few fixes and preparations, the CODA960
support patch, a rework of the hardware access serialization
into a single threaded workqueue, some cleanups to use more
infrastructure that is available in the meantime, runtime PM
support, a few h.264 related v4l2 controls and fixes, support
for hard resets via the i.MX system reset controller, and a
patch that exports internal buffers to debugfs.

regards
Philipp

Michael Olbrich (2):
  [media] v4l2-mem2mem: export v4l2_m2m_try_schedule
  [media] coda: try to schedule a decode run after a stop command

Philipp Zabel (28):
  [media] coda: fix decoder I/P/B frame detection
  [media] coda: fix readback of CODA_RET_DEC_SEQ_FRAME_NEED
  [media] coda: fix h.264 quantization parameter range
  [media] coda: fix internal framebuffer allocation size
  [media] coda: simplify IRAM setup
  [media] coda: Add encoder/decoder support for CODA960
  [media] coda: add selection API support for h.264 decoder
  [media] coda: add support for frame size enumeration
  [media] coda: add workqueue to serialize hardware commands
  [media] coda: Use mem-to-mem ioctl helpers
  [media] coda: use ctx->fh.m2m_ctx instead of ctx->m2m_ctx
  [media] coda: Add runtime pm support
  [media] coda: split firmware version check out of coda_hw_init
  [media] coda: select GENERIC_ALLOCATOR
  [media] coda: add h.264 min/max qp controls
  [media] coda: add h.264 deblocking filter controls
  [media] coda: add cyclic intra refresh control
  [media] coda: let userspace force IDR frames by enabling the keyframe
    flag in the source buffer
  [media] coda: add decoder timestamp queue
  [media] coda: alert userspace about macroblock errors
  [media] coda: add sequence counter offset
  [media] coda: use prescan_failed variable to stop stream after a
    timeout
  [media] coda: add reset control support
  [media] coda: add bytesperline to queue data
  [media] coda: allow odd width, but still round up bytesperline
  [media] coda: round up internal frames to multiples of macroblock size
    for h.264
  [media] coda: increase frame stride to 16 for h.264
  [media] coda: export auxiliary buffers via debugfs

 drivers/media/platform/Kconfig         |    1 +
 drivers/media/platform/coda.c          | 1505 +++++++++++++++++++++++---------
 drivers/media/platform/coda.h          |  115 ++-
 drivers/media/v4l2-core/v4l2-mem2mem.c |    3 +-
 include/media/v4l2-mem2mem.h           |    2 +
 5 files changed, 1197 insertions(+), 429 deletions(-)

-- 
2.0.0.rc2

[PATCH 01/30] [media] coda: fix decoder I/P/B frame detection

[Philipp Zabel]

Currently the rotator unit is used to copy decoded frames out into buffers
provided by videobuf2. Since the CODA reports the I/P/B frame type of the
last decoded frame, and this frame will be copied out in a later device_run,
depending on display order, we have to store the frame type until such time.
This patch also adds the B-frame type.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index b178379..a69fa3b 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -209,6 +209,7 @@ struct coda_ctx {
 	struct coda_aux_buf		psbuf;
 	struct coda_aux_buf		slicebuf;
 	struct coda_aux_buf		internal_frames[CODA_MAX_FRAMEBUFFERS];
+	u32				frame_types[CODA_MAX_FRAMEBUFFERS];
 	struct coda_aux_buf		workbuf;
 	int				num_internal_frames;
 	int				idx;
@@ -2693,15 +2694,6 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 
 	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
 
-	val = coda_read(dev, CODA_RET_DEC_PIC_TYPE);
-	if ((val & 0x7) == 0) {
-		dst_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_KEYFRAME;
-		dst_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_PFRAME;
-	} else {
-		dst_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_PFRAME;
-		dst_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_KEYFRAME;
-	}
-
 	val = coda_read(dev, CODA_RET_DEC_PIC_ERR_MB);
 	if (val > 0)
 		v4l2_err(&dev->v4l2_dev,
@@ -2748,6 +2740,14 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 	} else if (decoded_idx < 0 || decoded_idx >= ctx->num_internal_frames) {
 		v4l2_err(&dev->v4l2_dev,
 			 "decoded frame index out of range: %d\n", decoded_idx);
+	} else {
+		val = coda_read(dev, CODA_RET_DEC_PIC_TYPE) & 0x7;
+		if (val == 0)
+			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_KEYFRAME;
+		else if (val == 1)
+			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_PFRAME;
+		else
+			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_BFRAME;
 	}
 
 	if (display_idx == -1) {
@@ -2770,6 +2770,10 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
 		dst_buf->v4l2_buf.sequence = ctx->osequence++;
 
+		dst_buf->v4l2_buf.flags &= ~(V4L2_BUF_FLAG_KEYFRAME |
+					     V4L2_BUF_FLAG_PFRAME);
+		dst_buf->v4l2_buf.flags |= ctx->frame_types[ctx->display_idx];
+
 		vb2_set_plane_payload(dst_buf, 0, width * height * 3 / 2);
 
 		v4l2_m2m_buf_done(dst_buf, success ? VB2_BUF_STATE_DONE :
-- 
2.0.0.rc2

Re: [PATCH 01/30] [media] coda: fix decoder I/P/B frame detection

[Hans Verkuil]

On 06/13/2014 06:08 PM, Philipp Zabel wrote:
> Currently the rotator unit is used to copy decoded frames out into buffers
> provided by videobuf2. Since the CODA reports the I/P/B frame type of the
> last decoded frame, and this frame will be copied out in a later device_run,
> depending on display order, we have to store the frame type until such time.
> This patch also adds the B-frame type.
> 
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
>  drivers/media/platform/coda.c | 22 +++++++++++++---------
>  1 file changed, 13 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
> index b178379..a69fa3b 100644
> --- a/drivers/media/platform/coda.c
> +++ b/drivers/media/platform/coda.c
> @@ -209,6 +209,7 @@ struct coda_ctx {
>  	struct coda_aux_buf		psbuf;
>  	struct coda_aux_buf		slicebuf;
>  	struct coda_aux_buf		internal_frames[CODA_MAX_FRAMEBUFFERS];
> +	u32				frame_types[CODA_MAX_FRAMEBUFFERS];
>  	struct coda_aux_buf		workbuf;
>  	int				num_internal_frames;
>  	int				idx;
> @@ -2693,15 +2694,6 @@ static void coda_finish_decode(struct coda_ctx *ctx)
>  
>  	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
>  
> -	val = coda_read(dev, CODA_RET_DEC_PIC_TYPE);
> -	if ((val & 0x7) == 0) {
> -		dst_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_KEYFRAME;
> -		dst_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_PFRAME;
> -	} else {
> -		dst_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_PFRAME;
> -		dst_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_KEYFRAME;
> -	}
> -
>  	val = coda_read(dev, CODA_RET_DEC_PIC_ERR_MB);
>  	if (val > 0)
>  		v4l2_err(&dev->v4l2_dev,
> @@ -2748,6 +2740,14 @@ static void coda_finish_decode(struct coda_ctx *ctx)
>  	} else if (decoded_idx < 0 || decoded_idx >= ctx->num_internal_frames) {
>  		v4l2_err(&dev->v4l2_dev,
>  			 "decoded frame index out of range: %d\n", decoded_idx);
> +	} else {
> +		val = coda_read(dev, CODA_RET_DEC_PIC_TYPE) & 0x7;
> +		if (val == 0)
> +			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_KEYFRAME;
> +		else if (val == 1)
> +			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_PFRAME;
> +		else
> +			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_BFRAME;
>  	}
>  
>  	if (display_idx == -1) {
> @@ -2770,6 +2770,10 @@ static void coda_finish_decode(struct coda_ctx *ctx)
>  		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
>  		dst_buf->v4l2_buf.sequence = ctx->osequence++;
>  
> +		dst_buf->v4l2_buf.flags &= ~(V4L2_BUF_FLAG_KEYFRAME |
> +					     V4L2_BUF_FLAG_PFRAME);

Shouldn't this include '| V4L2_BUF_FLAG_BFRAME' as well?

	Hans

> +		dst_buf->v4l2_buf.flags |= ctx->frame_types[ctx->display_idx];
> +
>  		vb2_set_plane_payload(dst_buf, 0, width * height * 3 / 2);
>  
>  		v4l2_m2m_buf_done(dst_buf, success ? VB2_BUF_STATE_DONE :
> 

[PATCH 02/30] [media] coda: fix readback of CODA_RET_DEC_SEQ_FRAME_NEED

[Philipp Zabel]

Previously we'd add one to this value, allocating one additional, superfluous
internal buffer.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index a69fa3b..453ac4b 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -1889,7 +1889,7 @@ static int coda_start_decoding(struct coda_ctx *ctx)
 	v4l2_dbg(1, coda_debug, &dev->v4l2_dev, "%s instance %d now: %dx%d\n",
 		 __func__, ctx->idx, width, height);
 
-	ctx->num_internal_frames = coda_read(dev, CODA_RET_DEC_SEQ_FRAME_NEED) + 1;
+	ctx->num_internal_frames = coda_read(dev, CODA_RET_DEC_SEQ_FRAME_NEED);
 	if (ctx->num_internal_frames > CODA_MAX_FRAMEBUFFERS) {
 		v4l2_err(&dev->v4l2_dev,
 			 "not enough framebuffers to decode (%d < %d)\n",
-- 
2.0.0.rc2

[PATCH 03/30] [media] coda: fix h.264 quantization parameter range

[Philipp Zabel]

If bitrate is not set, the encoder is running in VBR mode, with the
I- and P-frame quantization parameters configured from userspace.
For the quantization parameters, 0 is a valid value.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 453ac4b..544360e 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -2385,9 +2385,9 @@ static int coda_ctrls_setup(struct coda_ctx *ctx)
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
 		V4L2_CID_MPEG_VIDEO_GOP_SIZE, 1, 60, 1, 16);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
-		V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, 1, 51, 1, 25);
+		V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, 0, 51, 1, 25);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
-		V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, 1, 51, 1, 25);
+		V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, 0, 51, 1, 25);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
 		V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP, 1, 31, 1, 2);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
-- 
2.0.0.rc2

[PATCH 04/30] [media] coda: fix internal framebuffer allocation size

[Philipp Zabel]

This error was introduced by 5677e3b04d3b3961200aa2bb9cc715e709eafeb9
"[media] coda: update CODA7541 to firmware 1.4.50".

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 544360e..0384c9b 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -1527,10 +1527,10 @@ static int coda_alloc_framebuffers(struct coda_ctx *ctx, struct coda_q_data *q_d
 	for (i = 0; i < ctx->num_internal_frames; i++) {
 		size_t size;
 
-		size = q_data->sizeimage;
+		size = ysize + ysize / 2;
 		if (ctx->codec->src_fourcc == V4L2_PIX_FMT_H264 &&
 		    dev->devtype->product != CODA_DX6)
-			ctx->internal_frames[i].size += ysize/4;
+			size += ysize / 4;
 		ret = coda_alloc_context_buf(ctx, &ctx->internal_frames[i], size);
 		if (ret < 0) {
 			coda_free_framebuffers(ctx);
-- 
2.0.0.rc2

[PATCH 05/30] [media] coda: simplify IRAM setup

[Philipp Zabel]

OVL and BTP IRAM buffers are never used, setup the bits for
for DBK/BIT/IP usage depending on CODA version in one place.
Also, use a simple allocator function and group IRAM addresses
and size in a coda_aux_buf structure.
This is done in preparation for CODA960 support.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 175 ++++++++++++++++++------------------------
 1 file changed, 74 insertions(+), 101 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 0384c9b..2b27998 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -135,9 +135,7 @@ struct coda_dev {
 	struct coda_aux_buf	tempbuf;
 	struct coda_aux_buf	workbuf;
 	struct gen_pool		*iram_pool;
-	long unsigned int	iram_vaddr;
-	long unsigned int	iram_paddr;
-	unsigned long		iram_size;
+	struct coda_aux_buf	iram;
 
 	spinlock_t		irqlock;
 	struct mutex		dev_mutex;
@@ -175,6 +173,8 @@ struct coda_iram_info {
 	phys_addr_t	buf_btp_use;
 	phys_addr_t	search_ram_paddr;
 	int		search_ram_size;
+	int		remaining;
+	phys_addr_t	next_paddr;
 };
 
 struct coda_ctx {
@@ -1580,23 +1580,43 @@ static int coda_h264_padding(int size, char *p)
 	return nal_size;
 }
 
+static phys_addr_t coda_iram_alloc(struct coda_iram_info *iram, size_t size)
+{
+	phys_addr_t ret;
+
+	size = round_up(size, 1024);
+	if (size > iram->remaining)
+		return 0;
+	iram->remaining -= size;
+
+	ret = iram->next_paddr;
+	iram->next_paddr += size;
+
+	return ret;
+}
+
 static void coda_setup_iram(struct coda_ctx *ctx)
 {
 	struct coda_iram_info *iram_info = &ctx->iram_info;
 	struct coda_dev *dev = ctx->dev;
-	int ipacdc_size;
-	int bitram_size;
-	int dbk_size;
-	int ovl_size;
 	int mb_width;
-	int me_size;
-	int size;
+	int dbk_bits;
+	int bit_bits;
+	int ip_bits;
 
 	memset(iram_info, 0, sizeof(*iram_info));
-	size = dev->iram_size;
+	iram_info->next_paddr = dev->iram.paddr;
+	iram_info->remaining = dev->iram.size;
 
-	if (dev->devtype->product == CODA_DX6)
+	switch (dev->devtype->product) {
+	case CODA_7541:
+		dbk_bits = CODA7_USE_HOST_DBK_ENABLE | CODA7_USE_DBK_ENABLE;
+		bit_bits = CODA7_USE_HOST_BIT_ENABLE | CODA7_USE_BIT_ENABLE;
+		ip_bits = CODA7_USE_HOST_IP_ENABLE | CODA7_USE_IP_ENABLE;
+		break;
+	default: /* CODA_DX6 */
 		return;
+	}
 
 	if (ctx->inst_type == CODA_INST_ENCODER) {
 		struct coda_q_data *q_data_src;
@@ -1605,111 +1625,63 @@ static void coda_setup_iram(struct coda_ctx *ctx)
 		mb_width = DIV_ROUND_UP(q_data_src->width, 16);
 
 		/* Prioritize in case IRAM is too small for everything */
-		me_size = round_up(round_up(q_data_src->width, 16) * 36 + 2048,
-				   1024);
-		iram_info->search_ram_size = me_size;
-		if (size >= iram_info->search_ram_size) {
-			if (dev->devtype->product == CODA_7541)
-				iram_info->axi_sram_use |= CODA7_USE_HOST_ME_ENABLE;
-			iram_info->search_ram_paddr = dev->iram_paddr;
-			size -= iram_info->search_ram_size;
-		} else {
-			pr_err("IRAM is smaller than the search ram size\n");
-			goto out;
+		if (dev->devtype->product == CODA_7541) {
+			iram_info->search_ram_size = round_up(mb_width * 16 *
+							      36 + 2048, 1024);
+			iram_info->search_ram_paddr = coda_iram_alloc(iram_info,
+							iram_info->search_ram_size);
+			if (!iram_info->search_ram_paddr) {
+				pr_err("IRAM is smaller than the search ram size\n");
+				goto out;
+			}
+			iram_info->axi_sram_use |= CODA7_USE_HOST_ME_ENABLE |
+						   CODA7_USE_ME_ENABLE;
 		}
 
 		/* Only H.264BP and H.263P3 are considered */
-		dbk_size = round_up(128 * mb_width, 1024);
-		if (size >= dbk_size) {
-			iram_info->axi_sram_use |= CODA7_USE_HOST_DBK_ENABLE;
-			iram_info->buf_dbk_y_use = dev->iram_paddr +
-						   iram_info->search_ram_size;
-			iram_info->buf_dbk_c_use = iram_info->buf_dbk_y_use +
-						   dbk_size / 2;
-			size -= dbk_size;
-		} else {
+		iram_info->buf_dbk_y_use = coda_iram_alloc(iram_info, 64 * mb_width);
+		iram_info->buf_dbk_c_use = coda_iram_alloc(iram_info, 64 * mb_width);
+		if (!iram_info->buf_dbk_c_use)
 			goto out;
-		}
+		iram_info->axi_sram_use |= dbk_bits;
 
-		bitram_size = round_up(128 * mb_width, 1024);
-		if (size >= bitram_size) {
-			iram_info->axi_sram_use |= CODA7_USE_HOST_BIT_ENABLE;
-			iram_info->buf_bit_use = iram_info->buf_dbk_c_use +
-						 dbk_size / 2;
-			size -= bitram_size;
-		} else {
+		iram_info->buf_bit_use = coda_iram_alloc(iram_info, 128 * mb_width);
+		if (!iram_info->buf_bit_use)
 			goto out;
-		}
+		iram_info->axi_sram_use |= bit_bits;
 
-		ipacdc_size = round_up(128 * mb_width, 1024);
-		if (size >= ipacdc_size) {
-			iram_info->axi_sram_use |= CODA7_USE_HOST_IP_ENABLE;
-			iram_info->buf_ip_ac_dc_use = iram_info->buf_bit_use +
-						      bitram_size;
-			size -= ipacdc_size;
-		}
+		iram_info->buf_ip_ac_dc_use = coda_iram_alloc(iram_info, 128 * mb_width);
+		if (!iram_info->buf_ip_ac_dc_use)
+			goto out;
+		iram_info->axi_sram_use |= ip_bits;
 
 		/* OVL and BTP disabled for encoder */
 	} else if (ctx->inst_type == CODA_INST_DECODER) {
 		struct coda_q_data *q_data_dst;
-		int mb_height;
 
 		q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
 		mb_width = DIV_ROUND_UP(q_data_dst->width, 16);
-		mb_height = DIV_ROUND_UP(q_data_dst->height, 16);
-
-		dbk_size = round_up(256 * mb_width, 1024);
-		if (size >= dbk_size) {
-			iram_info->axi_sram_use |= CODA7_USE_HOST_DBK_ENABLE;
-			iram_info->buf_dbk_y_use = dev->iram_paddr;
-			iram_info->buf_dbk_c_use = dev->iram_paddr +
-						   dbk_size / 2;
-			size -= dbk_size;
-		} else {
+
+		iram_info->buf_dbk_y_use = coda_iram_alloc(iram_info, 128 * mb_width);
+		iram_info->buf_dbk_c_use = coda_iram_alloc(iram_info, 128 * mb_width);
+		if (!iram_info->buf_dbk_c_use)
 			goto out;
-		}
+		iram_info->axi_sram_use |= dbk_bits;
 
-		bitram_size = round_up(128 * mb_width, 1024);
-		if (size >= bitram_size) {
-			iram_info->axi_sram_use |= CODA7_USE_HOST_BIT_ENABLE;
-			iram_info->buf_bit_use = iram_info->buf_dbk_c_use +
-						 dbk_size / 2;
-			size -= bitram_size;
-		} else {
+		iram_info->buf_bit_use = coda_iram_alloc(iram_info, 128 * mb_width);
+		if (!iram_info->buf_bit_use)
 			goto out;
-		}
+		iram_info->axi_sram_use |= bit_bits;
 
-		ipacdc_size = round_up(128 * mb_width, 1024);
-		if (size >= ipacdc_size) {
-			iram_info->axi_sram_use |= CODA7_USE_HOST_IP_ENABLE;
-			iram_info->buf_ip_ac_dc_use = iram_info->buf_bit_use +
-						      bitram_size;
-			size -= ipacdc_size;
-		} else {
+		iram_info->buf_ip_ac_dc_use = coda_iram_alloc(iram_info, 128 * mb_width);
+		if (!iram_info->buf_ip_ac_dc_use)
 			goto out;
-		}
+		iram_info->axi_sram_use |= ip_bits;
 
-		ovl_size = round_up(80 * mb_width, 1024);
+		/* OVL and BTP unused as there is no VC1 support yet */
 	}
 
 out:
-	switch (dev->devtype->product) {
-	case CODA_DX6:
-		break;
-	case CODA_7541:
-		/* i.MX53 uses secondary AXI for IRAM access */
-		if (iram_info->axi_sram_use & CODA7_USE_HOST_BIT_ENABLE)
-			iram_info->axi_sram_use |= CODA7_USE_BIT_ENABLE;
-		if (iram_info->axi_sram_use & CODA7_USE_HOST_IP_ENABLE)
-			iram_info->axi_sram_use |= CODA7_USE_IP_ENABLE;
-		if (iram_info->axi_sram_use & CODA7_USE_HOST_DBK_ENABLE)
-			iram_info->axi_sram_use |= CODA7_USE_DBK_ENABLE;
-		if (iram_info->axi_sram_use & CODA7_USE_HOST_OVL_ENABLE)
-			iram_info->axi_sram_use |= CODA7_USE_OVL_ENABLE;
-		if (iram_info->axi_sram_use & CODA7_USE_HOST_ME_ENABLE)
-			iram_info->axi_sram_use |= CODA7_USE_ME_ENABLE;
-	}
-
 	if (!(iram_info->axi_sram_use & CODA7_USE_HOST_IP_ENABLE))
 		v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev,
 			 "IRAM smaller than needed\n");
@@ -2065,7 +2037,7 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 
 	if (dev->devtype->product == CODA_DX6) {
 		/* Configure the coda */
-		coda_write(dev, dev->iram_paddr, CODADX6_REG_BIT_SEARCH_RAM_BASE_ADDR);
+		coda_write(dev, dev->iram.paddr, CODADX6_REG_BIT_SEARCH_RAM_BASE_ADDR);
 	}
 
 	/* Could set rotation here if needed */
@@ -3297,15 +3269,15 @@ static int coda_probe(struct platform_device *pdev)
 
 	switch (dev->devtype->product) {
 	case CODA_DX6:
-		dev->iram_size = CODADX6_IRAM_SIZE;
+		dev->iram.size = CODADX6_IRAM_SIZE;
 		break;
 	case CODA_7541:
-		dev->iram_size = CODA7_IRAM_SIZE;
+		dev->iram.size = CODA7_IRAM_SIZE;
 		break;
 	}
-	dev->iram_vaddr = (unsigned long)gen_pool_dma_alloc(dev->iram_pool,
-			dev->iram_size, (dma_addr_t *)&dev->iram_paddr);
-	if (!dev->iram_vaddr) {
+	dev->iram.vaddr = gen_pool_dma_alloc(dev->iram_pool, dev->iram.size,
+					     &dev->iram.paddr);
+	if (!dev->iram.vaddr) {
 		dev_err(&pdev->dev, "unable to alloc iram\n");
 		return -ENOMEM;
 	}
@@ -3325,8 +3297,9 @@ static int coda_remove(struct platform_device *pdev)
 	if (dev->alloc_ctx)
 		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
 	v4l2_device_unregister(&dev->v4l2_dev);
-	if (dev->iram_vaddr)
-		gen_pool_free(dev->iram_pool, dev->iram_vaddr, dev->iram_size);
+	if (dev->iram.vaddr)
+		gen_pool_free(dev->iram_pool, (unsigned long)dev->iram.vaddr,
+			      dev->iram.size);
 	coda_free_aux_buf(dev, &dev->codebuf);
 	coda_free_aux_buf(dev, &dev->tempbuf);
 	coda_free_aux_buf(dev, &dev->workbuf);
-- 
2.0.0.rc2

[PATCH 06/30] [media] coda: Add encoder/decoder support for CODA960

[Philipp Zabel]

This patch adds support for the CODA960 VPU in Freescale i.MX6 SoCs.

It enables h.264 and MPEG4 encoding and decoding support. Besides the usual
register shifting, the CODA960 gains frame memory control and GDI registers
that are set up for linear mapping right now, needs ENC_PIC_SRC_INDEX to be
set beyond the number of internal buffers for some reason, and has subsampling
buffers that need to be set up. Also, the work buffer size is increased to
80 KiB.

The CODA960 firmware spins if there is not enough input data in the bitstream
buffer. To make it continue, buffers need to be copied into the bitstream as
soon as they are queued. As the bitstream fifo is written into from two places,
it must be protected with a mutex. For that, using a threaded interrupt handler
is necessary.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 397 +++++++++++++++++++++++++++++++++++++-----
 drivers/media/platform/coda.h | 115 +++++++++++-
 2 files changed, 464 insertions(+), 48 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 2b27998..10cc031 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -44,19 +44,24 @@
 #define CODA_FMO_BUF_SIZE	32
 #define CODADX6_WORK_BUF_SIZE	(288 * 1024 + CODA_FMO_BUF_SIZE * 8 * 1024)
 #define CODA7_WORK_BUF_SIZE	(128 * 1024)
+#define CODA9_WORK_BUF_SIZE	(80 * 1024)
 #define CODA7_TEMP_BUF_SIZE	(304 * 1024)
+#define CODA9_TEMP_BUF_SIZE	(204 * 1024)
 #define CODA_PARA_BUF_SIZE	(10 * 1024)
 #define CODA_ISRAM_SIZE	(2048 * 2)
 #define CODADX6_IRAM_SIZE	0xb000
 #define CODA7_IRAM_SIZE		0x14000
+#define CODA9_IRAM_SIZE		0x21000
 
 #define CODA7_PS_BUF_SIZE	0x28000
+#define CODA9_PS_SAVE_SIZE	(512 * 1024)
 
 #define CODA_MAX_FRAMEBUFFERS	8
 
 #define CODA_MAX_FRAME_SIZE	0x100000
 #define FMO_SLICE_SAVE_BUF_SIZE         (32)
 #define CODA_DEFAULT_GAMMA		4096
+#define CODA9_DEFAULT_GAMMA		24576	/* 0.75 * 32768 */
 
 #define MIN_W 176
 #define MIN_H 144
@@ -84,6 +89,7 @@ enum coda_inst_type {
 enum coda_product {
 	CODA_DX6 = 0xf001,
 	CODA_7541 = 0xf012,
+	CODA_960 = 0xf020,
 };
 
 struct coda_fmt {
@@ -177,6 +183,16 @@ struct coda_iram_info {
 	phys_addr_t	next_paddr;
 };
 
+struct gdi_tiled_map {
+	int xy2ca_map[16];
+	int xy2ba_map[16];
+	int xy2ra_map[16];
+	int rbc2axi_map[32];
+	int xy2rbc_config;
+	int map_type;
+#define GDI_LINEAR_FRAME_MAP 0
+};
+
 struct coda_ctx {
 	struct coda_dev			*dev;
 	struct mutex			buffer_mutex;
@@ -215,8 +231,10 @@ struct coda_ctx {
 	int				idx;
 	int				reg_idx;
 	struct coda_iram_info		iram_info;
+	struct gdi_tiled_map		tiled_map;
 	u32				bit_stream_param;
 	u32				frm_dis_flg;
+	u32				frame_mem_ctrl;
 	int				display_idx;
 };
 
@@ -265,15 +283,23 @@ static void coda_command_async(struct coda_ctx *ctx, int cmd)
 {
 	struct coda_dev *dev = ctx->dev;
 
-	if (dev->devtype->product == CODA_7541) {
+	if (dev->devtype->product == CODA_960 ||
+	    dev->devtype->product == CODA_7541) {
 		/* Restore context related registers to CODA */
 		coda_write(dev, ctx->bit_stream_param,
 				CODA_REG_BIT_BIT_STREAM_PARAM);
 		coda_write(dev, ctx->frm_dis_flg,
 				CODA_REG_BIT_FRM_DIS_FLG(ctx->reg_idx));
+		coda_write(dev, ctx->frame_mem_ctrl,
+				CODA_REG_BIT_FRAME_MEM_CTRL);
 		coda_write(dev, ctx->workbuf.paddr, CODA_REG_BIT_WORK_BUF_ADDR);
 	}
 
+	if (dev->devtype->product == CODA_960) {
+		coda_write(dev, 1, CODA9_GDI_WPROT_ERR_CLR);
+		coda_write(dev, 0, CODA9_GDI_WPROT_RGN_EN);
+	}
+
 	coda_write(dev, CODA_REG_BIT_BUSY_FLAG, CODA_REG_BIT_BUSY);
 
 	coda_write(dev, ctx->idx, CODA_REG_BIT_RUN_INDEX);
@@ -349,6 +375,13 @@ static struct coda_codec coda7_codecs[] = {
 	CODA_CODEC(CODA7_MODE_DECODE_MP4,  V4L2_PIX_FMT_MPEG4,  V4L2_PIX_FMT_YUV420, 1920, 1080),
 };
 
+static struct coda_codec coda9_codecs[] = {
+	CODA_CODEC(CODA9_MODE_ENCODE_H264, V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_H264,   1920, 1080),
+	CODA_CODEC(CODA9_MODE_ENCODE_MP4,  V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_MPEG4,  1920, 1080),
+	CODA_CODEC(CODA9_MODE_DECODE_H264, V4L2_PIX_FMT_H264,   V4L2_PIX_FMT_YUV420, 1920, 1080),
+	CODA_CODEC(CODA9_MODE_DECODE_MP4,  V4L2_PIX_FMT_MPEG4,  V4L2_PIX_FMT_YUV420, 1920, 1080),
+};
+
 static bool coda_format_is_yuv(u32 fourcc)
 {
 	switch (fourcc) {
@@ -427,6 +460,8 @@ static char *coda_product_name(int product)
 		return "CodaDx6";
 	case CODA_7541:
 		return "CODA7541";
+	case CODA_960:
+		return "CODA960";
 	default:
 		snprintf(buf, sizeof(buf), "(0x%04x)", product);
 		return buf;
@@ -857,6 +892,7 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 			    struct v4l2_decoder_cmd *dc)
 {
 	struct coda_ctx *ctx = fh_to_ctx(fh);
+	struct coda_dev *dev = ctx->dev;
 	int ret;
 
 	ret = coda_try_decoder_cmd(file, fh, dc);
@@ -870,6 +906,13 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 	/* Set the strem-end flag on this context */
 	ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
 
+	if ((dev->devtype->product == CODA_960) &&
+	    coda_isbusy(dev) &&
+	    (ctx->idx == coda_read(dev, CODA_REG_BIT_RUN_INDEX))) {
+		/* If this context is currently running, update the hardware flag */
+		coda_write(dev, ctx->bit_stream_param, CODA_REG_BIT_BIT_STREAM_PARAM);
+	}
+
 	return 0;
 }
 
@@ -1025,6 +1068,27 @@ static void coda_fill_bitstream(struct coda_ctx *ctx)
 	}
 }
 
+static void coda_set_gdi_regs(struct coda_ctx *ctx)
+{
+	struct gdi_tiled_map *tiled_map = &ctx->tiled_map;
+	struct coda_dev *dev = ctx->dev;
+	int i;
+
+	for (i = 0; i < 16; i++)
+		coda_write(dev, tiled_map->xy2ca_map[i],
+				CODA9_GDI_XY2_CAS_0 + 4 * i);
+	for (i = 0; i < 4; i++)
+		coda_write(dev, tiled_map->xy2ba_map[i],
+				CODA9_GDI_XY2_BA_0 + 4 * i);
+	for (i = 0; i < 16; i++)
+		coda_write(dev, tiled_map->xy2ra_map[i],
+				CODA9_GDI_XY2_RAS_0 + 4 * i);
+	coda_write(dev, tiled_map->xy2rbc_config, CODA9_GDI_XY2_RBC_CONFIG);
+	for (i = 0; i < 32; i++)
+		coda_write(dev, tiled_map->rbc2axi_map[i],
+				CODA9_GDI_RBC2_AXI_0 + 4 * i);
+}
+
 /*
  * Mem-to-mem operations.
  */
@@ -1073,6 +1137,9 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 		}
 	}
 
+	if (dev->devtype->product == CODA_960)
+		coda_set_gdi_regs(ctx);
+
 	/* Set rotator output */
 	picture_y = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
 	if (q_data_dst->fourcc == V4L2_PIX_FMT_YVU420) {
@@ -1083,10 +1150,26 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 		picture_cb = picture_y + stridey * height;
 		picture_cr = picture_cb + stridey / 2 * height / 2;
 	}
-	coda_write(dev, picture_y, CODA_CMD_DEC_PIC_ROT_ADDR_Y);
-	coda_write(dev, picture_cb, CODA_CMD_DEC_PIC_ROT_ADDR_CB);
-	coda_write(dev, picture_cr, CODA_CMD_DEC_PIC_ROT_ADDR_CR);
-	coda_write(dev, stridey, CODA_CMD_DEC_PIC_ROT_STRIDE);
+
+	if (dev->devtype->product == CODA_960) {
+		/*
+		 * The CODA960 seems to have an internal list of buffers with
+		 * 64 entries that includes the registered frame buffers as
+		 * well as the rotator buffer output.
+		 * ROT_INDEX needs to be < 0x40, but > ctx->num_internal_frames.
+		 */
+		coda_write(dev, CODA_MAX_FRAMEBUFFERS + dst_buf->v4l2_buf.index,
+				CODA9_CMD_DEC_PIC_ROT_INDEX);
+		coda_write(dev, picture_y, CODA9_CMD_DEC_PIC_ROT_ADDR_Y);
+		coda_write(dev, picture_cb, CODA9_CMD_DEC_PIC_ROT_ADDR_CB);
+		coda_write(dev, picture_cr, CODA9_CMD_DEC_PIC_ROT_ADDR_CR);
+		coda_write(dev, stridey, CODA9_CMD_DEC_PIC_ROT_STRIDE);
+	} else {
+		coda_write(dev, picture_y, CODA_CMD_DEC_PIC_ROT_ADDR_Y);
+		coda_write(dev, picture_cb, CODA_CMD_DEC_PIC_ROT_ADDR_CB);
+		coda_write(dev, picture_cr, CODA_CMD_DEC_PIC_ROT_ADDR_CR);
+		coda_write(dev, stridey, CODA_CMD_DEC_PIC_ROT_STRIDE);
+	}
 	coda_write(dev, CODA_ROT_MIR_ENABLE | ctx->params.rot_mode,
 			CODA_CMD_DEC_PIC_ROT_MODE);
 
@@ -1096,6 +1179,9 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 	case CODA_7541:
 		coda_write(dev, CODA_PRE_SCAN_EN, CODA_CMD_DEC_PIC_OPTION);
 		break;
+	case CODA_960:
+		coda_write(dev, (1 << 10), CODA_CMD_DEC_PIC_OPTION); /* 'hardcode to use interrupt disable mode'? */
+		break;
 	}
 
 	coda_write(dev, 0, CODA_CMD_DEC_PIC_SKIP_NUM);
@@ -1140,6 +1226,9 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
 		src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_PFRAME;
 	}
 
+	if (dev->devtype->product == CODA_960)
+		coda_set_gdi_regs(ctx);
+
 	/*
 	 * Copy headers at the beginning of the first frame for H.264 only.
 	 * In MPEG4 they are already copied by the coda.
@@ -1218,15 +1307,31 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
 		break;
 	}
 
-	coda_write(dev, picture_y, CODA_CMD_ENC_PIC_SRC_ADDR_Y);
-	coda_write(dev, picture_cb, CODA_CMD_ENC_PIC_SRC_ADDR_CB);
-	coda_write(dev, picture_cr, CODA_CMD_ENC_PIC_SRC_ADDR_CR);
+	if (dev->devtype->product == CODA_960) {
+		coda_write(dev, 4/*FIXME: 0*/, CODA9_CMD_ENC_PIC_SRC_INDEX);
+		coda_write(dev, q_data_src->width, CODA9_CMD_ENC_PIC_SRC_STRIDE);
+		coda_write(dev, 0, CODA9_CMD_ENC_PIC_SUB_FRAME_SYNC);
+
+		coda_write(dev, picture_y, CODA9_CMD_ENC_PIC_SRC_ADDR_Y);
+		coda_write(dev, picture_cb, CODA9_CMD_ENC_PIC_SRC_ADDR_CB);
+		coda_write(dev, picture_cr, CODA9_CMD_ENC_PIC_SRC_ADDR_CR);
+	} else {
+		coda_write(dev, picture_y, CODA_CMD_ENC_PIC_SRC_ADDR_Y);
+		coda_write(dev, picture_cb, CODA_CMD_ENC_PIC_SRC_ADDR_CB);
+		coda_write(dev, picture_cr, CODA_CMD_ENC_PIC_SRC_ADDR_CR);
+	}
 	coda_write(dev, force_ipicture << 1 & 0x2,
 		   CODA_CMD_ENC_PIC_OPTION);
 
 	coda_write(dev, pic_stream_buffer_addr, CODA_CMD_ENC_PIC_BB_START);
 	coda_write(dev, pic_stream_buffer_size / 1024,
 		   CODA_CMD_ENC_PIC_BB_SIZE);
+
+	if (!ctx->streamon_out) {
+		/* After streamoff on the output side, set the stream end flag */
+		ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
+		coda_write(dev, ctx->bit_stream_param, CODA_REG_BIT_BIT_STREAM_PARAM);
+	}
 }
 
 static void coda_device_run(void *m2m_priv)
@@ -1352,6 +1457,32 @@ static struct v4l2_m2m_ops coda_m2m_ops = {
 	.unlock		= coda_unlock,
 };
 
+static void coda_set_tiled_map_type(struct coda_ctx *ctx, int tiled_map_type)
+{
+	struct gdi_tiled_map *tiled_map = &ctx->tiled_map;
+	int luma_map, chro_map, i;
+
+	memset(tiled_map, 0, sizeof(*tiled_map));
+
+	luma_map = 64;
+	chro_map = 64;
+	tiled_map->map_type = tiled_map_type;
+	for (i = 0; i < 16; i++)
+		tiled_map->xy2ca_map[i] = luma_map << 8 | chro_map;
+	for (i = 0; i < 4; i++)
+		tiled_map->xy2ba_map[i] = luma_map << 8 | chro_map;
+	for (i = 0; i < 16; i++)
+		tiled_map->xy2ra_map[i] = luma_map << 8 | chro_map;
+
+	if (tiled_map_type == GDI_LINEAR_FRAME_MAP) {
+		tiled_map->xy2rbc_config = 0;
+	} else {
+		dev_err(&ctx->dev->plat_dev->dev, "invalid map type: %d\n",
+			tiled_map_type);
+		return;
+	}
+}
+
 static void set_default_params(struct coda_ctx *ctx)
 {
 	int max_w;
@@ -1375,6 +1506,9 @@ static void set_default_params(struct coda_ctx *ctx)
 	ctx->q_data[V4L2_M2M_DST].width = max_w;
 	ctx->q_data[V4L2_M2M_DST].height = max_h;
 	ctx->q_data[V4L2_M2M_DST].sizeimage = CODA_MAX_FRAME_SIZE;
+
+	if (ctx->dev->devtype->product == CODA_960)
+		coda_set_tiled_map_type(ctx, GDI_LINEAR_FRAME_MAP);
 }
 
 /*
@@ -1424,6 +1558,7 @@ static int coda_buf_prepare(struct vb2_buffer *vb)
 static void coda_buf_queue(struct vb2_buffer *vb)
 {
 	struct coda_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct coda_dev *dev = ctx->dev;
 	struct coda_q_data *q_data;
 
 	q_data = get_q_data(ctx, vb->vb2_queue->type);
@@ -1438,8 +1573,15 @@ static void coda_buf_queue(struct vb2_buffer *vb)
 		 * For backwards compatibility, queuing an empty buffer marks
 		 * the stream end
 		 */
-		if (vb2_get_plane_payload(vb, 0) == 0)
+		if (vb2_get_plane_payload(vb, 0) == 0) {
 			ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
+			if ((dev->devtype->product == CODA_960) &&
+			    coda_isbusy(dev) &&
+			    (ctx->idx == coda_read(dev, CODA_REG_BIT_RUN_INDEX))) {
+				/* if this decoder instance is running, set the stream end flag */
+				coda_write(dev, ctx->bit_stream_param, CODA_REG_BIT_BIT_STREAM_PARAM);
+			}
+		}
 		mutex_lock(&ctx->bitstream_mutex);
 		v4l2_m2m_buf_queue(ctx->m2m_ctx, vb);
 		coda_fill_bitstream(ctx);
@@ -1614,6 +1756,11 @@ static void coda_setup_iram(struct coda_ctx *ctx)
 		bit_bits = CODA7_USE_HOST_BIT_ENABLE | CODA7_USE_BIT_ENABLE;
 		ip_bits = CODA7_USE_HOST_IP_ENABLE | CODA7_USE_IP_ENABLE;
 		break;
+	case CODA_960:
+		dbk_bits = CODA9_USE_HOST_DBK_ENABLE | CODA9_USE_DBK_ENABLE;
+		bit_bits = CODA9_USE_HOST_BIT_ENABLE | CODA7_USE_BIT_ENABLE;
+		ip_bits = CODA9_USE_HOST_IP_ENABLE | CODA7_USE_IP_ENABLE;
+		break;
 	default: /* CODA_DX6 */
 		return;
 	}
@@ -1723,6 +1870,11 @@ static int coda_alloc_context_buffers(struct coda_ctx *ctx,
 	case CODA_7541:
 		size = CODA7_WORK_BUF_SIZE;
 		break;
+	case CODA_960:
+		size = CODA9_WORK_BUF_SIZE;
+		if (q_data->fourcc == V4L2_PIX_FMT_H264)
+			size += CODA9_PS_SAVE_SIZE;
+		break;
 	default:
 		return 0;
 	}
@@ -1807,12 +1959,17 @@ static int coda_start_decoding(struct coda_ctx *ctx)
 	coda_write(dev, bitstream_buf, CODA_CMD_DEC_SEQ_BB_START);
 	coda_write(dev, bitstream_size / 1024, CODA_CMD_DEC_SEQ_BB_SIZE);
 	val = 0;
-	if (dev->devtype->product == CODA_7541)
+	if ((dev->devtype->product == CODA_7541) ||
+	    (dev->devtype->product == CODA_960))
 		val |= CODA_REORDER_ENABLE;
 	coda_write(dev, val, CODA_CMD_DEC_SEQ_OPTION);
 
 	ctx->params.codec_mode = ctx->codec->mode;
-	ctx->params.codec_mode_aux = 0;
+	if (dev->devtype->product == CODA_960 &&
+	    src_fourcc == V4L2_PIX_FMT_MPEG4)
+		ctx->params.codec_mode_aux = CODA_MP4_AUX_MPEG4;
+	else
+		ctx->params.codec_mode_aux = 0;
 	if (src_fourcc == V4L2_PIX_FMT_H264) {
 		if (dev->devtype->product == CODA_7541) {
 			coda_write(dev, ctx->psbuf.paddr,
@@ -1820,6 +1977,13 @@ static int coda_start_decoding(struct coda_ctx *ctx)
 			coda_write(dev, (CODA7_PS_BUF_SIZE / 1024),
 					CODA_CMD_DEC_SEQ_PS_BB_SIZE);
 		}
+		if (dev->devtype->product == CODA_960) {
+			coda_write(dev, 0, CODA_CMD_DEC_SEQ_X264_MV_EN);
+			coda_write(dev, 512, CODA_CMD_DEC_SEQ_SPP_CHUNK_SIZE);
+		}
+	}
+	if (dev->devtype->product != CODA_960) {
+		coda_write(dev, 0, CODA_CMD_DEC_SEQ_SRC_SIZE);
 	}
 
 	if (coda_command_sync(ctx, CODA_COMMAND_SEQ_INIT)) {
@@ -1891,6 +2055,20 @@ static int coda_start_decoding(struct coda_ctx *ctx)
 				CODA7_CMD_SET_FRAME_AXI_DBKC_ADDR);
 		coda_write(dev, ctx->iram_info.buf_ovl_use,
 				CODA7_CMD_SET_FRAME_AXI_OVL_ADDR);
+		if (dev->devtype->product == CODA_960)
+			coda_write(dev, ctx->iram_info.buf_btp_use,
+					CODA9_CMD_SET_FRAME_AXI_BTP_ADDR);
+	}
+
+	if (dev->devtype->product == CODA_960) {
+		coda_write(dev, -1, CODA9_CMD_SET_FRAME_DELAY);
+
+		coda_write(dev, 0x20262024, CODA9_CMD_SET_FRAME_CACHE_SIZE);
+		coda_write(dev, 2 << CODA9_CACHE_PAGEMERGE_OFFSET |
+				32 << CODA9_CACHE_LUMA_BUFFER_SIZE_OFFSET |
+				8 << CODA9_CACHE_CB_BUFFER_SIZE_OFFSET |
+				8 << CODA9_CACHE_CR_BUFFER_SIZE_OFFSET,
+				CODA9_CMD_SET_FRAME_CACHE_CONFIG);
 	}
 
 	if (src_fourcc == V4L2_PIX_FMT_H264) {
@@ -1900,7 +2078,13 @@ static int coda_start_decoding(struct coda_ctx *ctx)
 				CODA_CMD_SET_FRAME_SLICE_BB_SIZE);
 	}
 
-	if (dev->devtype->product == CODA_7541) {
+	if (dev->devtype->product == CODA_960) {
+		int max_mb_x = 1920 / 16;
+		int max_mb_y = 1088 / 16;
+		int max_mb_num = max_mb_x * max_mb_y;
+		coda_write(dev, max_mb_num << 16 | max_mb_x << 8 | max_mb_y,
+				CODA9_CMD_SET_FRAME_MAX_DEC_SIZE);
+	} else {
 		int max_mb_x = 1920 / 16;
 		int max_mb_y = 1088 / 16;
 		int max_mb_num = max_mb_x * max_mb_y;
@@ -1921,34 +2105,49 @@ static int coda_encode_header(struct coda_ctx *ctx, struct vb2_buffer *buf,
 			      int header_code, u8 *header, int *size)
 {
 	struct coda_dev *dev = ctx->dev;
+	size_t bufsize;
 	int ret;
+	int i;
+
+	if (dev->devtype->product == CODA_960)
+		memset(vb2_plane_vaddr(buf, 0), 0, 64);
 
 	coda_write(dev, vb2_dma_contig_plane_dma_addr(buf, 0),
 		   CODA_CMD_ENC_HEADER_BB_START);
-	coda_write(dev, vb2_plane_size(buf, 0), CODA_CMD_ENC_HEADER_BB_SIZE);
+	bufsize = vb2_plane_size(buf, 0);
+	if (dev->devtype->product == CODA_960)
+		bufsize /= 1024;
+	coda_write(dev, bufsize, CODA_CMD_ENC_HEADER_BB_SIZE);
 	coda_write(dev, header_code, CODA_CMD_ENC_HEADER_CODE);
 	ret = coda_command_sync(ctx, CODA_COMMAND_ENCODE_HEADER);
 	if (ret < 0) {
 		v4l2_err(&dev->v4l2_dev, "CODA_COMMAND_ENCODE_HEADER timeout\n");
 		return ret;
 	}
-	*size = coda_read(dev, CODA_REG_BIT_WR_PTR(ctx->reg_idx)) -
-		coda_read(dev, CODA_CMD_ENC_HEADER_BB_START);
+
+	if (dev->devtype->product == CODA_960) {
+		for (i = 63; i > 0; i--)
+			if (((char *)vb2_plane_vaddr(buf, 0))[i] != 0)
+				break;
+		*size = i + 1;
+	} else {
+		*size = coda_read(dev, CODA_REG_BIT_WR_PTR(ctx->reg_idx)) -
+			coda_read(dev, CODA_CMD_ENC_HEADER_BB_START);
+	}
 	memcpy(header, vb2_plane_vaddr(buf, 0), *size);
 
 	return 0;
 }
 
+static int coda_start_encoding(struct coda_ctx *ctx);
+
 static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct coda_ctx *ctx = vb2_get_drv_priv(q);
 	struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
-	u32 bitstream_buf, bitstream_size;
 	struct coda_dev *dev = ctx->dev;
 	struct coda_q_data *q_data_src, *q_data_dst;
-	struct vb2_buffer *buf;
 	u32 dst_fourcc;
-	u32 value;
 	int ret = 0;
 
 	q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
@@ -1983,10 +2182,7 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 		v4l2_m2m_set_src_buffered(ctx->m2m_ctx, true);
 
 	ctx->gopcounter = ctx->params.gop_size - 1;
-	buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
-	bitstream_buf = vb2_dma_contig_plane_dma_addr(buf, 0);
 	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
-	bitstream_size = q_data_dst->sizeimage;
 	dst_fourcc = q_data_dst->fourcc;
 
 	ctx->codec = coda_find_codec(ctx->dev, q_data_src->fourcc,
@@ -2005,16 +2201,36 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 		mutex_lock(&dev->coda_mutex);
 		ret = coda_start_decoding(ctx);
 		mutex_unlock(&dev->coda_mutex);
-		if (ret == -EAGAIN) {
+		if (ret == -EAGAIN)
 			return 0;
-		} else if (ret < 0) {
+		else if (ret < 0)
 			return ret;
-		} else {
-			ctx->initialized = 1;
-			return 0;
-		}
+	} else {
+		ret = coda_start_encoding(ctx);
 	}
 
+	ctx->initialized = 1;
+	return ret;
+}
+
+static int coda_start_encoding(struct coda_ctx *ctx)
+{
+	struct coda_dev *dev = ctx->dev;
+	struct v4l2_device *v4l2_dev = &dev->v4l2_dev;
+	struct coda_q_data *q_data_src, *q_data_dst;
+	u32 bitstream_buf, bitstream_size;
+	struct vb2_buffer *buf;
+	int gamma, ret, value;
+	u32 dst_fourcc;
+
+	q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+	dst_fourcc = q_data_dst->fourcc;
+
+	buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	bitstream_buf = vb2_dma_contig_plane_dma_addr(buf, 0);
+	bitstream_size = q_data_dst->sizeimage;
+
 	if (!coda_is_initialized(dev)) {
 		v4l2_err(v4l2_dev, "coda is not initialized.\n");
 		return -EFAULT;
@@ -2030,11 +2246,20 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 		coda_write(dev, CODADX6_STREAM_BUF_DYNALLOC_EN |
 			CODADX6_STREAM_BUF_PIC_RESET, CODA_REG_BIT_STREAM_CTRL);
 		break;
-	default:
+	case CODA_960:
+		coda_write(dev, 0, CODA9_GDI_WPROT_RGN_EN);
+		/* fallthrough */
+	case CODA_7541:
 		coda_write(dev, CODA7_STREAM_BUF_DYNALLOC_EN |
 			CODA7_STREAM_BUF_PIC_RESET, CODA_REG_BIT_STREAM_CTRL);
+		break;
 	}
 
+	value = coda_read(dev, CODA_REG_BIT_FRAME_MEM_CTRL);
+	value &= ~(1 << 2 | 0x7 << 9);
+	ctx->frame_mem_ctrl = value;
+	coda_write(dev, value, CODA_REG_BIT_FRAME_MEM_CTRL);
+
 	if (dev->devtype->product == CODA_DX6) {
 		/* Configure the coda */
 		coda_write(dev, dev->iram.paddr, CODADX6_REG_BIT_SEARCH_RAM_BASE_ADDR);
@@ -2057,11 +2282,17 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 	ctx->params.codec_mode = ctx->codec->mode;
 	switch (dst_fourcc) {
 	case V4L2_PIX_FMT_MPEG4:
-		coda_write(dev, CODA_STD_MPEG4, CODA_CMD_ENC_SEQ_COD_STD);
+		if (dev->devtype->product == CODA_960)
+			coda_write(dev, CODA9_STD_MPEG4, CODA_CMD_ENC_SEQ_COD_STD);
+		else
+			coda_write(dev, CODA_STD_MPEG4, CODA_CMD_ENC_SEQ_COD_STD);
 		coda_write(dev, 0, CODA_CMD_ENC_SEQ_MP4_PARA);
 		break;
 	case V4L2_PIX_FMT_H264:
-		coda_write(dev, CODA_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
+		if (dev->devtype->product == CODA_960)
+			coda_write(dev, CODA9_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
+		else
+			coda_write(dev, CODA_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
 		coda_write(dev, 0, CODA_CMD_ENC_SEQ_264_PARA);
 		break;
 	default:
@@ -2094,6 +2325,8 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 		/* Rate control enabled */
 		value = (ctx->params.bitrate & CODA_RATECONTROL_BITRATE_MASK) << CODA_RATECONTROL_BITRATE_OFFSET;
 		value |=  1 & CODA_RATECONTROL_ENABLE_MASK;
+		if (dev->devtype->product == CODA_960)
+			value |= BIT(31); /* disable autoskip */
 	} else {
 		value = 0;
 	}
@@ -2105,31 +2338,48 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 	coda_write(dev, bitstream_buf, CODA_CMD_ENC_SEQ_BB_START);
 	coda_write(dev, bitstream_size / 1024, CODA_CMD_ENC_SEQ_BB_SIZE);
 
-	/* set default gamma */
-	value = (CODA_DEFAULT_GAMMA & CODA_GAMMA_MASK) << CODA_GAMMA_OFFSET;
-	coda_write(dev, value, CODA_CMD_ENC_SEQ_RC_GAMMA);
 
-	if (CODA_DEFAULT_GAMMA > 0) {
-		if (dev->devtype->product == CODA_DX6)
-			value  = 1 << CODADX6_OPTION_GAMMA_OFFSET;
-		else
-			value  = 1 << CODA7_OPTION_GAMMA_OFFSET;
+	value = 0;
+	if (dev->devtype->product == CODA_960)
+		gamma = CODA9_DEFAULT_GAMMA;
+	else
+		gamma = CODA_DEFAULT_GAMMA;
+	if (gamma > 0) {
+		coda_write(dev, (gamma & CODA_GAMMA_MASK) << CODA_GAMMA_OFFSET,
+			   CODA_CMD_ENC_SEQ_RC_GAMMA);
+	}
+	if (dev->devtype->product == CODA_960) {
+		if (CODA_DEFAULT_GAMMA > 0)
+			value |= 1 << CODA9_OPTION_GAMMA_OFFSET;
 	} else {
-		value = 0;
+		if (CODA_DEFAULT_GAMMA > 0) {
+			if (dev->devtype->product == CODA_DX6)
+				value |= 1 << CODADX6_OPTION_GAMMA_OFFSET;
+			else
+				value |= 1 << CODA7_OPTION_GAMMA_OFFSET;
+		}
 	}
 	coda_write(dev, value, CODA_CMD_ENC_SEQ_OPTION);
 
+	coda_write(dev, 0, CODA_CMD_ENC_SEQ_RC_INTERVAL_MODE);
+
 	coda_setup_iram(ctx);
 
 	if (dst_fourcc == V4L2_PIX_FMT_H264) {
-		if (dev->devtype->product == CODA_DX6) {
+		switch (dev->devtype->product) {
+		case CODA_DX6:
 			value = FMO_SLICE_SAVE_BUF_SIZE << 7;
 			coda_write(dev, value, CODADX6_CMD_ENC_SEQ_FMO);
-		} else {
+			break;
+		case CODA_7541:
 			coda_write(dev, ctx->iram_info.search_ram_paddr,
 					CODA7_CMD_ENC_SEQ_SEARCH_BASE);
 			coda_write(dev, ctx->iram_info.search_ram_size,
 					CODA7_CMD_ENC_SEQ_SEARCH_SIZE);
+			break;
+		case CODA_960:
+			coda_write(dev, 0, CODA9_CMD_ENC_SEQ_ME_OPTION);
+			coda_write(dev, 0, CODA9_CMD_ENC_SEQ_INTRA_WEIGHT);
 		}
 	}
 
@@ -2145,7 +2395,10 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 		goto out;
 	}
 
-	ctx->num_internal_frames = 2;
+	if (dev->devtype->product == CODA_960)
+		ctx->num_internal_frames = 4;
+	else
+		ctx->num_internal_frames = 2;
 	ret = coda_alloc_framebuffers(ctx, q_data_src, dst_fourcc);
 	if (ret < 0) {
 		v4l2_err(v4l2_dev, "failed to allocate framebuffers\n");
@@ -2168,7 +2421,16 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 				CODA7_CMD_SET_FRAME_AXI_DBKC_ADDR);
 		coda_write(dev, ctx->iram_info.buf_ovl_use,
 				CODA7_CMD_SET_FRAME_AXI_OVL_ADDR);
+		if (dev->devtype->product == CODA_960) {
+			coda_write(dev, ctx->iram_info.buf_btp_use,
+					CODA9_CMD_SET_FRAME_AXI_BTP_ADDR);
+
+			/* FIXME */
+			coda_write(dev, ctx->internal_frames[2].paddr, CODA9_CMD_SET_FRAME_SUBSAMP_A);
+			coda_write(dev, ctx->internal_frames[3].paddr, CODA9_CMD_SET_FRAME_SUBSAMP_B);
+		}
 	}
+
 	ret = coda_command_sync(ctx, CODA_COMMAND_SET_FRAME_BUF);
 	if (ret < 0) {
 		v4l2_err(v4l2_dev, "CODA_COMMAND_SET_FRAME_BUF timeout\n");
@@ -2252,6 +2514,16 @@ static void coda_stop_streaming(struct vb2_queue *q)
 			 "%s: output\n", __func__);
 		ctx->streamon_out = 0;
 
+		if (ctx->inst_type == CODA_INST_DECODER &&
+		    coda_isbusy(dev) && ctx->idx == coda_read(dev, CODA_REG_BIT_RUN_INDEX)) {
+			/* if this decoder instance is running, set the stream end flag */
+			if (dev->devtype->product == CODA_960) {
+				u32 val = coda_read(dev, CODA_REG_BIT_BIT_STREAM_PARAM);
+				val |= CODA_BIT_STREAM_END_FLAG;
+				coda_write(dev, val, CODA_REG_BIT_BIT_STREAM_PARAM);
+				ctx->bit_stream_param = val;
+			}
+		}
 		ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
 
 		ctx->isequence = 0;
@@ -2453,6 +2725,7 @@ static int coda_open(struct file *file)
 	ctx->idx = idx;
 	switch (dev->devtype->product) {
 	case CODA_7541:
+	case CODA_960:
 		ctx->reg_idx = 0;
 		break;
 	default:
@@ -2772,7 +3045,7 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 	u32 wr_ptr, start_ptr;
 
 	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
-	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
 
 	/* Get results from the coda */
 	start_ptr = coda_read(dev, CODA_CMD_ENC_PIC_BB_START);
@@ -2809,6 +3082,8 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 	dst_buf->v4l2_buf.timecode = src_buf->v4l2_buf.timecode;
 
 	v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
 	v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
 
 	ctx->gopcounter--;
@@ -2907,6 +3182,7 @@ static void coda_timeout(struct work_struct *work)
 static u32 coda_supported_firmwares[] = {
 	CODA_FIRMWARE_VERNUM(CODA_DX6, 2, 2, 5),
 	CODA_FIRMWARE_VERNUM(CODA_7541, 1, 4, 50),
+	CODA_FIRMWARE_VERNUM(CODA_960, 2, 1, 5),
 };
 
 static bool coda_firmware_supported(u32 vernum)
@@ -2961,7 +3237,8 @@ static int coda_hw_init(struct coda_dev *dev)
 		coda_write(dev, 0, CODA_REG_BIT_CODE_BUF_ADDR + i * 4);
 
 	/* Tell the BIT where to find everything it needs */
-	if (dev->devtype->product == CODA_7541) {
+	if (dev->devtype->product == CODA_960 ||
+	    dev->devtype->product == CODA_7541) {
 		coda_write(dev, dev->tempbuf.paddr,
 				CODA_REG_BIT_TEMP_BUF_ADDR);
 		coda_write(dev, 0, CODA_REG_BIT_BIT_STREAM_PARAM);
@@ -2981,7 +3258,10 @@ static int coda_hw_init(struct coda_dev *dev)
 	default:
 		coda_write(dev, CODA7_STREAM_BUF_PIC_FLUSH, CODA_REG_BIT_STREAM_CTRL);
 	}
-	coda_write(dev, 0, CODA_REG_BIT_FRAME_MEM_CTRL);
+	if (dev->devtype->product == CODA_960)
+		coda_write(dev, 1 << 12, CODA_REG_BIT_FRAME_MEM_CTRL);
+	else
+		coda_write(dev, 0, CODA_REG_BIT_FRAME_MEM_CTRL);
 
 	if (dev->devtype->product != CODA_DX6)
 		coda_write(dev, 0, CODA7_REG_BIT_AXI_SRAM_USE);
@@ -3011,6 +3291,12 @@ static int coda_hw_init(struct coda_dev *dev)
 		return -EIO;
 	}
 
+	if (dev->devtype->product == CODA_960) {
+		data = coda_read(dev, CODA9_CMD_FIRMWARE_CODE_REV);
+		v4l2_info(&dev->v4l2_dev, "Firmware code revision: %d\n",
+			  data);
+	}
+
 	/* Check we are compatible with the loaded firmware */
 	data = coda_read(dev, CODA_CMD_FIRMWARE_VERNUM);
 	product = CODA_FIRMWARE_PRODUCT(data);
@@ -3126,6 +3412,8 @@ static int coda_firmware_request(struct coda_dev *dev)
 enum coda_platform {
 	CODA_IMX27,
 	CODA_IMX53,
+	CODA_IMX6Q,
+	CODA_IMX6DL,
 };
 
 static const struct coda_devtype coda_devdata[] = {
@@ -3141,6 +3429,18 @@ static const struct coda_devtype coda_devdata[] = {
 		.codecs     = coda7_codecs,
 		.num_codecs = ARRAY_SIZE(coda7_codecs),
 	},
+	[CODA_IMX6Q] = {
+		.firmware   = "v4l-coda960-imx6q.bin",
+		.product    = CODA_960,
+		.codecs     = coda9_codecs,
+		.num_codecs = ARRAY_SIZE(coda9_codecs),
+	},
+	[CODA_IMX6DL] = {
+		.firmware   = "v4l-coda960-imx6dl.bin",
+		.product    = CODA_960,
+		.codecs     = coda9_codecs,
+		.num_codecs = ARRAY_SIZE(coda9_codecs),
+	},
 };
 
 static struct platform_device_id coda_platform_ids[] = {
@@ -3154,6 +3454,8 @@ MODULE_DEVICE_TABLE(platform, coda_platform_ids);
 static const struct of_device_id coda_dt_ids[] = {
 	{ .compatible = "fsl,imx27-vpu", .data = &coda_devdata[CODA_IMX27] },
 	{ .compatible = "fsl,imx53-vpu", .data = &coda_devdata[CODA_IMX53] },
+	{ .compatible = "fsl,imx6q-vpu", .data = &coda_devdata[CODA_IMX6Q] },
+	{ .compatible = "fsl,imx6dl-vpu", .data = &coda_devdata[CODA_IMX6DL] },
 	{ /* sentinel */ }
 };
 MODULE_DEVICE_TABLE(of, coda_dt_ids);
@@ -3256,6 +3558,9 @@ static int coda_probe(struct platform_device *pdev)
 	case CODA_7541:
 		dev->tempbuf.size = CODA7_TEMP_BUF_SIZE;
 		break;
+	case CODA_960:
+		dev->tempbuf.size = CODA9_TEMP_BUF_SIZE;
+		break;
 	}
 	if (dev->tempbuf.size) {
 		ret = coda_alloc_aux_buf(dev, &dev->tempbuf,
@@ -3274,6 +3579,8 @@ static int coda_probe(struct platform_device *pdev)
 	case CODA_7541:
 		dev->iram.size = CODA7_IRAM_SIZE;
 		break;
+	case CODA_960:
+		dev->iram.size = CODA9_IRAM_SIZE;
 	}
 	dev->iram.vaddr = gen_pool_dma_alloc(dev->iram_pool, dev->iram.size,
 					     &dev->iram.paddr);
diff --git a/drivers/media/platform/coda.h b/drivers/media/platform/coda.h
index 4e32e2e..c791275 100644
--- a/drivers/media/platform/coda.h
+++ b/drivers/media/platform/coda.h
@@ -27,6 +27,14 @@
 #define CODA_REG_BIT_CODE_RESET		0x014
 #define		CODA_REG_RESET_ENABLE		(1 << 0)
 #define CODA_REG_BIT_CUR_PC			0x018
+#define CODA9_REG_BIT_SW_RESET			0x024
+#define		CODA9_SW_RESET_BPU_CORE   0x008
+#define		CODA9_SW_RESET_BPU_BUS    0x010
+#define		CODA9_SW_RESET_VCE_CORE   0x020
+#define		CODA9_SW_RESET_VCE_BUS    0x040
+#define		CODA9_SW_RESET_GDI_CORE   0x080
+#define		CODA9_SW_RESET_GDI_BUS    0x100
+#define CODA9_REG_BIT_SW_RESET_STATUS		0x034
 
 /* Static SW registers */
 #define CODA_REG_BIT_CODE_BUF_ADDR		0x100
@@ -39,9 +47,11 @@
 #define		CODADX6_STREAM_BUF_PIC_FLUSH	(1 << 2)
 #define		CODA7_STREAM_BUF_DYNALLOC_EN	(1 << 5)
 #define		CODADX6_STREAM_BUF_DYNALLOC_EN	(1 << 4)
-#define 	CODA_STREAM_CHKDIS_OFFSET	(1 << 1)
+#define		CODADX6_STREAM_CHKDIS_OFFSET	(1 << 1)
+#define		CODA7_STREAM_SEL_64BITS_ENDIAN	(1 << 1)
 #define		CODA_STREAM_ENDIAN_SELECT	(1 << 0)
 #define CODA_REG_BIT_FRAME_MEM_CTRL		0x110
+#define		CODA_FRAME_CHROMA_INTERLEAVE	(1 << 2)
 #define		CODA_IMAGE_ENDIAN_SELECT	(1 << 0)
 #define CODA_REG_BIT_BIT_STREAM_PARAM		0x114
 #define		CODA_BIT_STREAM_END_FLAG	(1 << 2)
@@ -52,13 +62,21 @@
 #define CODA_REG_BIT_FRM_DIS_FLG(x)		(0x150 + 4 * (x))
 #define CODADX6_REG_BIT_SEARCH_RAM_BASE_ADDR	0x140
 #define CODA7_REG_BIT_AXI_SRAM_USE		0x140
+#define		CODA9_USE_HOST_BTP_ENABLE	(1 << 13)
+#define		CODA9_USE_HOST_OVL_ENABLE	(1 << 12)
 #define		CODA7_USE_HOST_ME_ENABLE	(1 << 11)
+#define		CODA9_USE_HOST_DBK_ENABLE	(3 << 10)
 #define		CODA7_USE_HOST_OVL_ENABLE	(1 << 10)
 #define		CODA7_USE_HOST_DBK_ENABLE	(1 << 9)
+#define		CODA9_USE_HOST_IP_ENABLE	(1 << 9)
 #define		CODA7_USE_HOST_IP_ENABLE	(1 << 8)
+#define		CODA9_USE_HOST_BIT_ENABLE	(1 << 8)
 #define		CODA7_USE_HOST_BIT_ENABLE	(1 << 7)
+#define		CODA9_USE_BTP_ENABLE		(1 << 5)
 #define		CODA7_USE_ME_ENABLE		(1 << 4)
+#define		CODA9_USE_OVL_ENABLE		(1 << 4)
 #define		CODA7_USE_OVL_ENABLE		(1 << 3)
+#define		CODA9_USE_DBK_ENABLE		(3 << 2)
 #define		CODA7_USE_DBK_ENABLE		(1 << 2)
 #define		CODA7_USE_IP_ENABLE		(1 << 1)
 #define		CODA7_USE_BIT_ENABLE		(1 << 0)
@@ -93,6 +111,18 @@
 #define		CODA7_MODE_ENCODE_H264		8
 #define		CODA7_MODE_ENCODE_MP4		11
 #define		CODA7_MODE_ENCODE_MJPG		13
+#define		CODA9_MODE_DECODE_H264		0
+#define		CODA9_MODE_DECODE_VC1		1
+#define		CODA9_MODE_DECODE_MP2		2
+#define		CODA9_MODE_DECODE_MP4		3
+#define		CODA9_MODE_DECODE_DV3		3
+#define		CODA9_MODE_DECODE_RV		4
+#define		CODA9_MODE_DECODE_AVS		5
+#define		CODA9_MODE_DECODE_MJPG		6
+#define		CODA9_MODE_DECODE_VPX		7
+#define		CODA9_MODE_ENCODE_H264		8
+#define		CODA9_MODE_ENCODE_MP4		11
+#define		CODA9_MODE_ENCODE_MJPG		13
 #define 	CODA_MODE_INVALID		0xffff
 #define CODA_REG_BIT_INT_ENABLE		0x170
 #define		CODA_INT_INTERRUPT_ENABLE	(1 << 3)
@@ -129,6 +159,7 @@
 #define CODA_CMD_DEC_SEQ_SPP_CHUNK_SIZE		0x1a0
 
 #define CODA7_RET_DEC_SEQ_ASPECT		0x1b0
+#define CODA9_RET_DEC_SEQ_BITRATE		0x1b4
 #define CODA_RET_DEC_SEQ_SUCCESS		0x1c0
 #define CODA_RET_DEC_SEQ_SRC_FMT		0x1c4 /* SRC_SIZE on CODA7 */
 #define CODA_RET_DEC_SEQ_SRC_SIZE		0x1c4
@@ -145,13 +176,19 @@
 #define CODA_RET_DEC_SEQ_FRATE_DR		0x1e8
 #define CODA_RET_DEC_SEQ_JPG_PARA		0x1e4
 #define CODA_RET_DEC_SEQ_JPG_THUMB_IND		0x1e8
+#define CODA9_RET_DEC_SEQ_HEADER_REPORT		0x1ec
 
 /* Decoder Picture Run */
 #define CODA_CMD_DEC_PIC_ROT_MODE		0x180
 #define CODA_CMD_DEC_PIC_ROT_ADDR_Y		0x184
+#define CODA9_CMD_DEC_PIC_ROT_INDEX		0x184
 #define CODA_CMD_DEC_PIC_ROT_ADDR_CB		0x188
+#define CODA9_CMD_DEC_PIC_ROT_ADDR_Y		0x188
 #define CODA_CMD_DEC_PIC_ROT_ADDR_CR		0x18c
+#define CODA9_CMD_DEC_PIC_ROT_ADDR_CB		0x18c
 #define CODA_CMD_DEC_PIC_ROT_STRIDE		0x190
+#define CODA9_CMD_DEC_PIC_ROT_ADDR_CR		0x190
+#define CODA9_CMD_DEC_PIC_ROT_STRIDE		0x1b8
 
 #define CODA_CMD_DEC_PIC_OPTION			0x194
 #define		CODA_PRE_SCAN_EN			(1 << 0)
@@ -183,25 +220,39 @@
 #define CODA_RET_DEC_PIC_CROP_TOP_BOTTOM	0x1e4
 #define CODA_RET_DEC_PIC_FRAME_NEED		0x1ec
 
+#define CODA9_RET_DEC_PIC_VP8_PIC_REPORT	0x1e8
+#define CODA9_RET_DEC_PIC_ASPECT		0x1f0
+#define CODA9_RET_DEC_PIC_VP8_SCALE_INFO	0x1f0
+#define CODA9_RET_DEC_PIC_FRATE_NR		0x1f4
+#define CODA9_RET_DEC_PIC_FRATE_DR		0x1f8
+
 /* Encoder Sequence Initialization */
 #define CODA_CMD_ENC_SEQ_BB_START				0x180
 #define CODA_CMD_ENC_SEQ_BB_SIZE				0x184
 #define CODA_CMD_ENC_SEQ_OPTION				0x188
 #define		CODA7_OPTION_AVCINTRA16X16ONLY_OFFSET		9
+#define		CODA9_OPTION_MVC_PREFIX_NAL_OFFSET		9
 #define		CODA7_OPTION_GAMMA_OFFSET			8
+#define		CODA9_OPTION_MVC_PARASET_REFRESH_OFFSET		8
 #define		CODA7_OPTION_RCQPMAX_OFFSET			7
+#define		CODA9_OPTION_GAMMA_OFFSET			7
 #define		CODADX6_OPTION_GAMMA_OFFSET			7
 #define		CODA7_OPTION_RCQPMIN_OFFSET			6
+#define		CODA9_OPTION_RCQPMAX_OFFSET			6
 #define		CODA_OPTION_LIMITQP_OFFSET			6
 #define		CODA_OPTION_RCINTRAQP_OFFSET			5
 #define		CODA_OPTION_FMO_OFFSET				4
+#define		CODA9_OPTION_MVC_INTERVIEW_OFFSET		4
 #define		CODA_OPTION_AVC_AUD_OFFSET			2
 #define		CODA_OPTION_SLICEREPORT_OFFSET			1
 #define CODA_CMD_ENC_SEQ_COD_STD				0x18c
 #define		CODA_STD_MPEG4					0
+#define		CODA9_STD_H264					0
 #define		CODA_STD_H263					1
 #define		CODA_STD_H264					2
 #define		CODA_STD_MJPG					3
+#define		CODA9_STD_MPEG4					3
+
 #define CODA_CMD_ENC_SEQ_SRC_SIZE				0x190
 #define		CODA7_PICWIDTH_OFFSET				16
 #define		CODA7_PICWIDTH_MASK				0xffff
@@ -268,15 +319,26 @@
 #define CODA7_CMD_ENC_SEQ_SEARCH_BASE				0x1b8
 #define CODA7_CMD_ENC_SEQ_SEARCH_SIZE				0x1bc
 #define CODA7_CMD_ENC_SEQ_INTRA_QP				0x1c4
-#define CODA_CMD_ENC_SEQ_RC_QP_MAX				0x1c8
+#define CODA_CMD_ENC_SEQ_RC_QP_MIN_MAX				0x1c8
+#define		CODA_QPMIN_OFFSET				8
+#define		CODA_QPMIN_MASK					0x3f
 #define		CODA_QPMAX_OFFSET				0
 #define		CODA_QPMAX_MASK					0x3f
 #define CODA_CMD_ENC_SEQ_RC_GAMMA				0x1cc
 #define		CODA_GAMMA_OFFSET				0
 #define		CODA_GAMMA_MASK					0xffff
+#define CODA_CMD_ENC_SEQ_RC_INTERVAL_MODE			0x1d0
+#define CODA9_CMD_ENC_SEQ_INTRA_WEIGHT				0x1d4
+#define CODA9_CMD_ENC_SEQ_ME_OPTION				0x1d8
 #define CODA_RET_ENC_SEQ_SUCCESS				0x1c0
 
 /* Encoder Picture Run */
+#define CODA9_CMD_ENC_PIC_SRC_INDEX		0x180
+#define CODA9_CMD_ENC_PIC_SRC_STRIDE		0x184
+#define CODA9_CMD_ENC_PIC_SUB_FRAME_SYNC	0x1a4
+#define CODA9_CMD_ENC_PIC_SRC_ADDR_Y		0x1a8
+#define CODA9_CMD_ENC_PIC_SRC_ADDR_CB		0x1ac
+#define CODA9_CMD_ENC_PIC_SRC_ADDR_CR		0x1b0
 #define CODA_CMD_ENC_PIC_SRC_ADDR_Y	0x180
 #define CODA_CMD_ENC_PIC_SRC_ADDR_CB	0x184
 #define CODA_CMD_ENC_PIC_SRC_ADDR_CR	0x188
@@ -291,7 +353,11 @@
 #define		CODA_MIR_VER					(0x1 << 2)
 #define		CODA_MIR_HOR					(0x2 << 2)
 #define		CODA_MIR_VER_HOR				(0x3 << 2)
-#define CODA_CMD_ENC_PIC_OPTION	0x194
+#define CODA_CMD_ENC_PIC_OPTION		0x194
+#define		CODA_FORCE_IPICTURE				BIT(1)
+#define		CODA_REPORT_MB_INFO				BIT(3)
+#define		CODA_REPORT_MV_INFO				BIT(4)
+#define		CODA_REPORT_SLICE_INFO				BIT(5)
 #define CODA_CMD_ENC_PIC_BB_START	0x198
 #define CODA_CMD_ENC_PIC_BB_SIZE	0x19c
 #define CODA_RET_ENC_FRAME_NUM		0x1c0
@@ -306,13 +372,30 @@
 #define CODA_CMD_SET_FRAME_BUF_STRIDE		0x184
 #define CODA_CMD_SET_FRAME_SLICE_BB_START	0x188
 #define CODA_CMD_SET_FRAME_SLICE_BB_SIZE	0x18c
+#define CODA9_CMD_SET_FRAME_SUBSAMP_A		0x188
+#define CODA9_CMD_SET_FRAME_SUBSAMP_B		0x18c
 #define CODA7_CMD_SET_FRAME_AXI_BIT_ADDR	0x190
 #define CODA7_CMD_SET_FRAME_AXI_IPACDC_ADDR	0x194
 #define CODA7_CMD_SET_FRAME_AXI_DBKY_ADDR	0x198
 #define CODA7_CMD_SET_FRAME_AXI_DBKC_ADDR	0x19c
 #define CODA7_CMD_SET_FRAME_AXI_OVL_ADDR	0x1a0
 #define CODA7_CMD_SET_FRAME_MAX_DEC_SIZE	0x1a4
+#define CODA9_CMD_SET_FRAME_AXI_BTP_ADDR	0x1a4
 #define CODA7_CMD_SET_FRAME_SOURCE_BUF_STRIDE	0x1a8
+#define CODA9_CMD_SET_FRAME_CACHE_SIZE		0x1a8
+#define CODA9_CMD_SET_FRAME_CACHE_CONFIG	0x1ac
+#define		CODA9_CACHE_BYPASS_OFFSET		28
+#define		CODA9_CACHE_DUALCONF_OFFSET		26
+#define		CODA9_CACHE_PAGEMERGE_OFFSET		24
+#define		CODA9_CACHE_LUMA_BUFFER_SIZE_OFFSET	16
+#define		CODA9_CACHE_CB_BUFFER_SIZE_OFFSET	8
+#define		CODA9_CACHE_CR_BUFFER_SIZE_OFFSET	0
+#define CODA9_CMD_SET_FRAME_SUBSAMP_A_MVC	0x1b0
+#define CODA9_CMD_SET_FRAME_SUBSAMP_B_MVC	0x1b4
+#define CODA9_CMD_SET_FRAME_DP_BUF_BASE		0x1b0
+#define CODA9_CMD_SET_FRAME_DP_BUF_SIZE		0x1b4
+#define CODA9_CMD_SET_FRAME_MAX_DEC_SIZE	0x1b8
+#define CODA9_CMD_SET_FRAME_DELAY		0x1bc
 
 /* Encoder Header */
 #define CODA_CMD_ENC_HEADER_CODE	0x180
@@ -322,8 +405,11 @@
 #define		CODA_HEADER_MP4V_VOL	0
 #define		CODA_HEADER_MP4V_VOS	1
 #define		CODA_HEADER_MP4V_VIS	2
+#define		CODA9_HEADER_FRAME_CROP	(1 << 3)
 #define CODA_CMD_ENC_HEADER_BB_START	0x184
 #define CODA_CMD_ENC_HEADER_BB_SIZE	0x188
+#define CODA9_CMD_ENC_HEADER_FRAME_CROP_H	0x18c
+#define CODA9_CMD_ENC_HEADER_FRAME_CROP_V	0x190
 
 /* Get Version */
 #define CODA_CMD_FIRMWARE_VERNUM		0x1c0
@@ -334,5 +420,28 @@
 #define		CODA_FIRMWARE_VERNUM(product, major, minor, release)	\
 			((product) << 16 | ((major) << 12) |		\
 			((minor) << 8) | (release))
+#define CODA9_CMD_FIRMWARE_CODE_REV		0x1c4
+
+#define CODA9_GDMA_BASE				0x1000
+#define CODA9_GDI_WPROT_ERR_CLR			(CODA9_GDMA_BASE + 0x0a0)
+#define CODA9_GDI_WPROT_RGN_EN			(CODA9_GDMA_BASE + 0x0ac)
+
+#define CODA9_GDI_BUS_CTRL			(CODA9_GDMA_BASE + 0x0f0)
+#define CODA9_GDI_BUS_STATUS			(CODA9_GDMA_BASE + 0x0f4)
+
+#define CODA9_GDI_XY2_CAS_0			(CODA9_GDMA_BASE + 0x800)
+#define CODA9_GDI_XY2_CAS_F			(CODA9_GDMA_BASE + 0x83c)
+
+#define CODA9_GDI_XY2_BA_0			(CODA9_GDMA_BASE + 0x840)
+#define CODA9_GDI_XY2_BA_1			(CODA9_GDMA_BASE + 0x844)
+#define CODA9_GDI_XY2_BA_2			(CODA9_GDMA_BASE + 0x848)
+#define CODA9_GDI_XY2_BA_3			(CODA9_GDMA_BASE + 0x84c)
+
+#define CODA9_GDI_XY2_RAS_0			(CODA9_GDMA_BASE + 0x850)
+#define CODA9_GDI_XY2_RAS_F			(CODA9_GDMA_BASE + 0x88c)
+
+#define CODA9_GDI_XY2_RBC_CONFIG		(CODA9_GDMA_BASE + 0x890)
+#define CODA9_GDI_RBC2_AXI_0			(CODA9_GDMA_BASE + 0x8a0)
+#define CODA9_GDI_RBC2_AXI_1F			(CODA9_GDMA_BASE + 0x91c)
 
 #endif
-- 
2.0.0.rc2

Re: [PATCH 06/30] [media] coda: Add encoder/decoder support for CODA960

[Nicolas Dufresne]

Le vendredi 13 juin 2014 à 18:08 +0200, Philipp Zabel a écrit :
> This patch adds support for the CODA960 VPU in Freescale i.MX6 SoCs.

I might be confused, but is this driver sharing the same device node for
the encoder and the decoder ? If so why ? I know the spec might not be
preventing it, but I don't know how in userspace I'm supposed to figure
the type of m2m node this is ? Other drivers have decided to split
encoding, decoding and transformation into their own node, which made it
easier to use generically.

> 
> It enables h.264 and MPEG4 encoding and decoding support. Besides the usual
> register shifting, the CODA960 gains frame memory control and GDI registers
> that are set up for linear mapping right now, needs ENC_PIC_SRC_INDEX to be
> set beyond the number of internal buffers for some reason, and has subsampling
> buffers that need to be set up. Also, the work buffer size is increased to
> 80 KiB.
> 
> The CODA960 firmware spins if there is not enough input data in the bitstream
> buffer. To make it continue, buffers need to be copied into the bitstream as
> soon as they are queued. As the bitstream fifo is written into from two places,
> it must be protected with a mutex. For that, using a threaded interrupt handler
> is necessary.
> 
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
>  drivers/media/platform/coda.c | 397 +++++++++++++++++++++++++++++++++++++-----
>  drivers/media/platform/coda.h | 115 +++++++++++-
>  2 files changed, 464 insertions(+), 48 deletions(-)
> 
> diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
> index 2b27998..10cc031 100644
> --- a/drivers/media/platform/coda.c
> +++ b/drivers/media/platform/coda.c
> @@ -44,19 +44,24 @@
>  #define CODA_FMO_BUF_SIZE	32
>  #define CODADX6_WORK_BUF_SIZE	(288 * 1024 + CODA_FMO_BUF_SIZE * 8 * 1024)
>  #define CODA7_WORK_BUF_SIZE	(128 * 1024)
> +#define CODA9_WORK_BUF_SIZE	(80 * 1024)
>  #define CODA7_TEMP_BUF_SIZE	(304 * 1024)
> +#define CODA9_TEMP_BUF_SIZE	(204 * 1024)
>  #define CODA_PARA_BUF_SIZE	(10 * 1024)
>  #define CODA_ISRAM_SIZE	(2048 * 2)
>  #define CODADX6_IRAM_SIZE	0xb000
>  #define CODA7_IRAM_SIZE		0x14000
> +#define CODA9_IRAM_SIZE		0x21000
>  
>  #define CODA7_PS_BUF_SIZE	0x28000
> +#define CODA9_PS_SAVE_SIZE	(512 * 1024)
>  
>  #define CODA_MAX_FRAMEBUFFERS	8
>  
>  #define CODA_MAX_FRAME_SIZE	0x100000
>  #define FMO_SLICE_SAVE_BUF_SIZE         (32)
>  #define CODA_DEFAULT_GAMMA		4096
> +#define CODA9_DEFAULT_GAMMA		24576	/* 0.75 * 32768 */
>  
>  #define MIN_W 176
>  #define MIN_H 144
> @@ -84,6 +89,7 @@ enum coda_inst_type {
>  enum coda_product {
>  	CODA_DX6 = 0xf001,
>  	CODA_7541 = 0xf012,
> +	CODA_960 = 0xf020,
>  };
>  
>  struct coda_fmt {
> @@ -177,6 +183,16 @@ struct coda_iram_info {
>  	phys_addr_t	next_paddr;
>  };
>  
> +struct gdi_tiled_map {
> +	int xy2ca_map[16];
> +	int xy2ba_map[16];
> +	int xy2ra_map[16];
> +	int rbc2axi_map[32];
> +	int xy2rbc_config;
> +	int map_type;
> +#define GDI_LINEAR_FRAME_MAP 0
> +};
> +
>  struct coda_ctx {
>  	struct coda_dev			*dev;
>  	struct mutex			buffer_mutex;
> @@ -215,8 +231,10 @@ struct coda_ctx {
>  	int				idx;
>  	int				reg_idx;
>  	struct coda_iram_info		iram_info;
> +	struct gdi_tiled_map		tiled_map;
>  	u32				bit_stream_param;
>  	u32				frm_dis_flg;
> +	u32				frame_mem_ctrl;
>  	int				display_idx;
>  };
>  
> @@ -265,15 +283,23 @@ static void coda_command_async(struct coda_ctx *ctx, int cmd)
>  {
>  	struct coda_dev *dev = ctx->dev;
>  
> -	if (dev->devtype->product == CODA_7541) {
> +	if (dev->devtype->product == CODA_960 ||
> +	    dev->devtype->product == CODA_7541) {
>  		/* Restore context related registers to CODA */
>  		coda_write(dev, ctx->bit_stream_param,
>  				CODA_REG_BIT_BIT_STREAM_PARAM);
>  		coda_write(dev, ctx->frm_dis_flg,
>  				CODA_REG_BIT_FRM_DIS_FLG(ctx->reg_idx));
> +		coda_write(dev, ctx->frame_mem_ctrl,
> +				CODA_REG_BIT_FRAME_MEM_CTRL);
>  		coda_write(dev, ctx->workbuf.paddr, CODA_REG_BIT_WORK_BUF_ADDR);
>  	}
>  
> +	if (dev->devtype->product == CODA_960) {
> +		coda_write(dev, 1, CODA9_GDI_WPROT_ERR_CLR);
> +		coda_write(dev, 0, CODA9_GDI_WPROT_RGN_EN);
> +	}
> +
>  	coda_write(dev, CODA_REG_BIT_BUSY_FLAG, CODA_REG_BIT_BUSY);
>  
>  	coda_write(dev, ctx->idx, CODA_REG_BIT_RUN_INDEX);
> @@ -349,6 +375,13 @@ static struct coda_codec coda7_codecs[] = {
>  	CODA_CODEC(CODA7_MODE_DECODE_MP4,  V4L2_PIX_FMT_MPEG4,  V4L2_PIX_FMT_YUV420, 1920, 1080),
>  };
>  
> +static struct coda_codec coda9_codecs[] = {
> +	CODA_CODEC(CODA9_MODE_ENCODE_H264, V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_H264,   1920, 1080),
> +	CODA_CODEC(CODA9_MODE_ENCODE_MP4,  V4L2_PIX_FMT_YUV420, V4L2_PIX_FMT_MPEG4,  1920, 1080),
> +	CODA_CODEC(CODA9_MODE_DECODE_H264, V4L2_PIX_FMT_H264,   V4L2_PIX_FMT_YUV420, 1920, 1080),
> +	CODA_CODEC(CODA9_MODE_DECODE_MP4,  V4L2_PIX_FMT_MPEG4,  V4L2_PIX_FMT_YUV420, 1920, 1080),
> +};
> +
>  static bool coda_format_is_yuv(u32 fourcc)
>  {
>  	switch (fourcc) {
> @@ -427,6 +460,8 @@ static char *coda_product_name(int product)
>  		return "CodaDx6";
>  	case CODA_7541:
>  		return "CODA7541";
> +	case CODA_960:
> +		return "CODA960";
>  	default:
>  		snprintf(buf, sizeof(buf), "(0x%04x)", product);
>  		return buf;
> @@ -857,6 +892,7 @@ static int coda_decoder_cmd(struct file *file, void *fh,
>  			    struct v4l2_decoder_cmd *dc)
>  {
>  	struct coda_ctx *ctx = fh_to_ctx(fh);
> +	struct coda_dev *dev = ctx->dev;
>  	int ret;
>  
>  	ret = coda_try_decoder_cmd(file, fh, dc);
> @@ -870,6 +906,13 @@ static int coda_decoder_cmd(struct file *file, void *fh,
>  	/* Set the strem-end flag on this context */
>  	ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
>  
> +	if ((dev->devtype->product == CODA_960) &&
> +	    coda_isbusy(dev) &&
> +	    (ctx->idx == coda_read(dev, CODA_REG_BIT_RUN_INDEX))) {
> +		/* If this context is currently running, update the hardware flag */
> +		coda_write(dev, ctx->bit_stream_param, CODA_REG_BIT_BIT_STREAM_PARAM);
> +	}
> +
>  	return 0;
>  }
>  
> @@ -1025,6 +1068,27 @@ static void coda_fill_bitstream(struct coda_ctx *ctx)
>  	}
>  }
>  
> +static void coda_set_gdi_regs(struct coda_ctx *ctx)
> +{
> +	struct gdi_tiled_map *tiled_map = &ctx->tiled_map;
> +	struct coda_dev *dev = ctx->dev;
> +	int i;
> +
> +	for (i = 0; i < 16; i++)
> +		coda_write(dev, tiled_map->xy2ca_map[i],
> +				CODA9_GDI_XY2_CAS_0 + 4 * i);
> +	for (i = 0; i < 4; i++)
> +		coda_write(dev, tiled_map->xy2ba_map[i],
> +				CODA9_GDI_XY2_BA_0 + 4 * i);
> +	for (i = 0; i < 16; i++)
> +		coda_write(dev, tiled_map->xy2ra_map[i],
> +				CODA9_GDI_XY2_RAS_0 + 4 * i);
> +	coda_write(dev, tiled_map->xy2rbc_config, CODA9_GDI_XY2_RBC_CONFIG);
> +	for (i = 0; i < 32; i++)
> +		coda_write(dev, tiled_map->rbc2axi_map[i],
> +				CODA9_GDI_RBC2_AXI_0 + 4 * i);
> +}
> +
>  /*
>   * Mem-to-mem operations.
>   */
> @@ -1073,6 +1137,9 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
>  		}
>  	}
>  
> +	if (dev->devtype->product == CODA_960)
> +		coda_set_gdi_regs(ctx);
> +
>  	/* Set rotator output */
>  	picture_y = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
>  	if (q_data_dst->fourcc == V4L2_PIX_FMT_YVU420) {
> @@ -1083,10 +1150,26 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
>  		picture_cb = picture_y + stridey * height;
>  		picture_cr = picture_cb + stridey / 2 * height / 2;
>  	}
> -	coda_write(dev, picture_y, CODA_CMD_DEC_PIC_ROT_ADDR_Y);
> -	coda_write(dev, picture_cb, CODA_CMD_DEC_PIC_ROT_ADDR_CB);
> -	coda_write(dev, picture_cr, CODA_CMD_DEC_PIC_ROT_ADDR_CR);
> -	coda_write(dev, stridey, CODA_CMD_DEC_PIC_ROT_STRIDE);
> +
> +	if (dev->devtype->product == CODA_960) {
> +		/*
> +		 * The CODA960 seems to have an internal list of buffers with
> +		 * 64 entries that includes the registered frame buffers as
> +		 * well as the rotator buffer output.
> +		 * ROT_INDEX needs to be < 0x40, but > ctx->num_internal_frames.
> +		 */
> +		coda_write(dev, CODA_MAX_FRAMEBUFFERS + dst_buf->v4l2_buf.index,
> +				CODA9_CMD_DEC_PIC_ROT_INDEX);
> +		coda_write(dev, picture_y, CODA9_CMD_DEC_PIC_ROT_ADDR_Y);
> +		coda_write(dev, picture_cb, CODA9_CMD_DEC_PIC_ROT_ADDR_CB);
> +		coda_write(dev, picture_cr, CODA9_CMD_DEC_PIC_ROT_ADDR_CR);
> +		coda_write(dev, stridey, CODA9_CMD_DEC_PIC_ROT_STRIDE);
> +	} else {
> +		coda_write(dev, picture_y, CODA_CMD_DEC_PIC_ROT_ADDR_Y);
> +		coda_write(dev, picture_cb, CODA_CMD_DEC_PIC_ROT_ADDR_CB);
> +		coda_write(dev, picture_cr, CODA_CMD_DEC_PIC_ROT_ADDR_CR);
> +		coda_write(dev, stridey, CODA_CMD_DEC_PIC_ROT_STRIDE);
> +	}
>  	coda_write(dev, CODA_ROT_MIR_ENABLE | ctx->params.rot_mode,
>  			CODA_CMD_DEC_PIC_ROT_MODE);
>  
> @@ -1096,6 +1179,9 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
>  	case CODA_7541:
>  		coda_write(dev, CODA_PRE_SCAN_EN, CODA_CMD_DEC_PIC_OPTION);
>  		break;
> +	case CODA_960:
> +		coda_write(dev, (1 << 10), CODA_CMD_DEC_PIC_OPTION); /* 'hardcode to use interrupt disable mode'? */
> +		break;
>  	}
>  
>  	coda_write(dev, 0, CODA_CMD_DEC_PIC_SKIP_NUM);
> @@ -1140,6 +1226,9 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
>  		src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_PFRAME;
>  	}
>  
> +	if (dev->devtype->product == CODA_960)
> +		coda_set_gdi_regs(ctx);
> +
>  	/*
>  	 * Copy headers at the beginning of the first frame for H.264 only.
>  	 * In MPEG4 they are already copied by the coda.
> @@ -1218,15 +1307,31 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
>  		break;
>  	}
>  
> -	coda_write(dev, picture_y, CODA_CMD_ENC_PIC_SRC_ADDR_Y);
> -	coda_write(dev, picture_cb, CODA_CMD_ENC_PIC_SRC_ADDR_CB);
> -	coda_write(dev, picture_cr, CODA_CMD_ENC_PIC_SRC_ADDR_CR);
> +	if (dev->devtype->product == CODA_960) {
> +		coda_write(dev, 4/*FIXME: 0*/, CODA9_CMD_ENC_PIC_SRC_INDEX);
> +		coda_write(dev, q_data_src->width, CODA9_CMD_ENC_PIC_SRC_STRIDE);
> +		coda_write(dev, 0, CODA9_CMD_ENC_PIC_SUB_FRAME_SYNC);
> +
> +		coda_write(dev, picture_y, CODA9_CMD_ENC_PIC_SRC_ADDR_Y);
> +		coda_write(dev, picture_cb, CODA9_CMD_ENC_PIC_SRC_ADDR_CB);
> +		coda_write(dev, picture_cr, CODA9_CMD_ENC_PIC_SRC_ADDR_CR);
> +	} else {
> +		coda_write(dev, picture_y, CODA_CMD_ENC_PIC_SRC_ADDR_Y);
> +		coda_write(dev, picture_cb, CODA_CMD_ENC_PIC_SRC_ADDR_CB);
> +		coda_write(dev, picture_cr, CODA_CMD_ENC_PIC_SRC_ADDR_CR);
> +	}
>  	coda_write(dev, force_ipicture << 1 & 0x2,
>  		   CODA_CMD_ENC_PIC_OPTION);
>  
>  	coda_write(dev, pic_stream_buffer_addr, CODA_CMD_ENC_PIC_BB_START);
>  	coda_write(dev, pic_stream_buffer_size / 1024,
>  		   CODA_CMD_ENC_PIC_BB_SIZE);
> +
> +	if (!ctx->streamon_out) {
> +		/* After streamoff on the output side, set the stream end flag */
> +		ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
> +		coda_write(dev, ctx->bit_stream_param, CODA_REG_BIT_BIT_STREAM_PARAM);
> +	}
>  }
>  
>  static void coda_device_run(void *m2m_priv)
> @@ -1352,6 +1457,32 @@ static struct v4l2_m2m_ops coda_m2m_ops = {
>  	.unlock		= coda_unlock,
>  };
>  
> +static void coda_set_tiled_map_type(struct coda_ctx *ctx, int tiled_map_type)
> +{
> +	struct gdi_tiled_map *tiled_map = &ctx->tiled_map;
> +	int luma_map, chro_map, i;
> +
> +	memset(tiled_map, 0, sizeof(*tiled_map));
> +
> +	luma_map = 64;
> +	chro_map = 64;
> +	tiled_map->map_type = tiled_map_type;
> +	for (i = 0; i < 16; i++)
> +		tiled_map->xy2ca_map[i] = luma_map << 8 | chro_map;
> +	for (i = 0; i < 4; i++)
> +		tiled_map->xy2ba_map[i] = luma_map << 8 | chro_map;
> +	for (i = 0; i < 16; i++)
> +		tiled_map->xy2ra_map[i] = luma_map << 8 | chro_map;
> +
> +	if (tiled_map_type == GDI_LINEAR_FRAME_MAP) {
> +		tiled_map->xy2rbc_config = 0;
> +	} else {
> +		dev_err(&ctx->dev->plat_dev->dev, "invalid map type: %d\n",
> +			tiled_map_type);
> +		return;
> +	}
> +}
> +
>  static void set_default_params(struct coda_ctx *ctx)
>  {
>  	int max_w;
> @@ -1375,6 +1506,9 @@ static void set_default_params(struct coda_ctx *ctx)
>  	ctx->q_data[V4L2_M2M_DST].width = max_w;
>  	ctx->q_data[V4L2_M2M_DST].height = max_h;
>  	ctx->q_data[V4L2_M2M_DST].sizeimage = CODA_MAX_FRAME_SIZE;
> +
> +	if (ctx->dev->devtype->product == CODA_960)
> +		coda_set_tiled_map_type(ctx, GDI_LINEAR_FRAME_MAP);
>  }
>  
>  /*
> @@ -1424,6 +1558,7 @@ static int coda_buf_prepare(struct vb2_buffer *vb)
>  static void coda_buf_queue(struct vb2_buffer *vb)
>  {
>  	struct coda_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
> +	struct coda_dev *dev = ctx->dev;
>  	struct coda_q_data *q_data;
>  
>  	q_data = get_q_data(ctx, vb->vb2_queue->type);
> @@ -1438,8 +1573,15 @@ static void coda_buf_queue(struct vb2_buffer *vb)
>  		 * For backwards compatibility, queuing an empty buffer marks
>  		 * the stream end
>  		 */
> -		if (vb2_get_plane_payload(vb, 0) == 0)
> +		if (vb2_get_plane_payload(vb, 0) == 0) {
>  			ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
> +			if ((dev->devtype->product == CODA_960) &&
> +			    coda_isbusy(dev) &&
> +			    (ctx->idx == coda_read(dev, CODA_REG_BIT_RUN_INDEX))) {
> +				/* if this decoder instance is running, set the stream end flag */
> +				coda_write(dev, ctx->bit_stream_param, CODA_REG_BIT_BIT_STREAM_PARAM);
> +			}
> +		}
>  		mutex_lock(&ctx->bitstream_mutex);
>  		v4l2_m2m_buf_queue(ctx->m2m_ctx, vb);
>  		coda_fill_bitstream(ctx);
> @@ -1614,6 +1756,11 @@ static void coda_setup_iram(struct coda_ctx *ctx)
>  		bit_bits = CODA7_USE_HOST_BIT_ENABLE | CODA7_USE_BIT_ENABLE;
>  		ip_bits = CODA7_USE_HOST_IP_ENABLE | CODA7_USE_IP_ENABLE;
>  		break;
> +	case CODA_960:
> +		dbk_bits = CODA9_USE_HOST_DBK_ENABLE | CODA9_USE_DBK_ENABLE;
> +		bit_bits = CODA9_USE_HOST_BIT_ENABLE | CODA7_USE_BIT_ENABLE;
> +		ip_bits = CODA9_USE_HOST_IP_ENABLE | CODA7_USE_IP_ENABLE;
> +		break;
>  	default: /* CODA_DX6 */
>  		return;
>  	}
> @@ -1723,6 +1870,11 @@ static int coda_alloc_context_buffers(struct coda_ctx *ctx,
>  	case CODA_7541:
>  		size = CODA7_WORK_BUF_SIZE;
>  		break;
> +	case CODA_960:
> +		size = CODA9_WORK_BUF_SIZE;
> +		if (q_data->fourcc == V4L2_PIX_FMT_H264)
> +			size += CODA9_PS_SAVE_SIZE;
> +		break;
>  	default:
>  		return 0;
>  	}
> @@ -1807,12 +1959,17 @@ static int coda_start_decoding(struct coda_ctx *ctx)
>  	coda_write(dev, bitstream_buf, CODA_CMD_DEC_SEQ_BB_START);
>  	coda_write(dev, bitstream_size / 1024, CODA_CMD_DEC_SEQ_BB_SIZE);
>  	val = 0;
> -	if (dev->devtype->product == CODA_7541)
> +	if ((dev->devtype->product == CODA_7541) ||
> +	    (dev->devtype->product == CODA_960))
>  		val |= CODA_REORDER_ENABLE;
>  	coda_write(dev, val, CODA_CMD_DEC_SEQ_OPTION);
>  
>  	ctx->params.codec_mode = ctx->codec->mode;
> -	ctx->params.codec_mode_aux = 0;
> +	if (dev->devtype->product == CODA_960 &&
> +	    src_fourcc == V4L2_PIX_FMT_MPEG4)
> +		ctx->params.codec_mode_aux = CODA_MP4_AUX_MPEG4;
> +	else
> +		ctx->params.codec_mode_aux = 0;
>  	if (src_fourcc == V4L2_PIX_FMT_H264) {
>  		if (dev->devtype->product == CODA_7541) {
>  			coda_write(dev, ctx->psbuf.paddr,
> @@ -1820,6 +1977,13 @@ static int coda_start_decoding(struct coda_ctx *ctx)
>  			coda_write(dev, (CODA7_PS_BUF_SIZE / 1024),
>  					CODA_CMD_DEC_SEQ_PS_BB_SIZE);
>  		}
> +		if (dev->devtype->product == CODA_960) {
> +			coda_write(dev, 0, CODA_CMD_DEC_SEQ_X264_MV_EN);
> +			coda_write(dev, 512, CODA_CMD_DEC_SEQ_SPP_CHUNK_SIZE);
> +		}
> +	}
> +	if (dev->devtype->product != CODA_960) {
> +		coda_write(dev, 0, CODA_CMD_DEC_SEQ_SRC_SIZE);
>  	}
>  
>  	if (coda_command_sync(ctx, CODA_COMMAND_SEQ_INIT)) {
> @@ -1891,6 +2055,20 @@ static int coda_start_decoding(struct coda_ctx *ctx)
>  				CODA7_CMD_SET_FRAME_AXI_DBKC_ADDR);
>  		coda_write(dev, ctx->iram_info.buf_ovl_use,
>  				CODA7_CMD_SET_FRAME_AXI_OVL_ADDR);
> +		if (dev->devtype->product == CODA_960)
> +			coda_write(dev, ctx->iram_info.buf_btp_use,
> +					CODA9_CMD_SET_FRAME_AXI_BTP_ADDR);
> +	}
> +
> +	if (dev->devtype->product == CODA_960) {
> +		coda_write(dev, -1, CODA9_CMD_SET_FRAME_DELAY);
> +
> +		coda_write(dev, 0x20262024, CODA9_CMD_SET_FRAME_CACHE_SIZE);
> +		coda_write(dev, 2 << CODA9_CACHE_PAGEMERGE_OFFSET |
> +				32 << CODA9_CACHE_LUMA_BUFFER_SIZE_OFFSET |
> +				8 << CODA9_CACHE_CB_BUFFER_SIZE_OFFSET |
> +				8 << CODA9_CACHE_CR_BUFFER_SIZE_OFFSET,
> +				CODA9_CMD_SET_FRAME_CACHE_CONFIG);
>  	}
>  
>  	if (src_fourcc == V4L2_PIX_FMT_H264) {
> @@ -1900,7 +2078,13 @@ static int coda_start_decoding(struct coda_ctx *ctx)
>  				CODA_CMD_SET_FRAME_SLICE_BB_SIZE);
>  	}
>  
> -	if (dev->devtype->product == CODA_7541) {
> +	if (dev->devtype->product == CODA_960) {
> +		int max_mb_x = 1920 / 16;
> +		int max_mb_y = 1088 / 16;
> +		int max_mb_num = max_mb_x * max_mb_y;
> +		coda_write(dev, max_mb_num << 16 | max_mb_x << 8 | max_mb_y,
> +				CODA9_CMD_SET_FRAME_MAX_DEC_SIZE);
> +	} else {
>  		int max_mb_x = 1920 / 16;
>  		int max_mb_y = 1088 / 16;
>  		int max_mb_num = max_mb_x * max_mb_y;
> @@ -1921,34 +2105,49 @@ static int coda_encode_header(struct coda_ctx *ctx, struct vb2_buffer *buf,
>  			      int header_code, u8 *header, int *size)
>  {
>  	struct coda_dev *dev = ctx->dev;
> +	size_t bufsize;
>  	int ret;
> +	int i;
> +
> +	if (dev->devtype->product == CODA_960)
> +		memset(vb2_plane_vaddr(buf, 0), 0, 64);
>  
>  	coda_write(dev, vb2_dma_contig_plane_dma_addr(buf, 0),
>  		   CODA_CMD_ENC_HEADER_BB_START);
> -	coda_write(dev, vb2_plane_size(buf, 0), CODA_CMD_ENC_HEADER_BB_SIZE);
> +	bufsize = vb2_plane_size(buf, 0);
> +	if (dev->devtype->product == CODA_960)
> +		bufsize /= 1024;
> +	coda_write(dev, bufsize, CODA_CMD_ENC_HEADER_BB_SIZE);
>  	coda_write(dev, header_code, CODA_CMD_ENC_HEADER_CODE);
>  	ret = coda_command_sync(ctx, CODA_COMMAND_ENCODE_HEADER);
>  	if (ret < 0) {
>  		v4l2_err(&dev->v4l2_dev, "CODA_COMMAND_ENCODE_HEADER timeout\n");
>  		return ret;
>  	}
> -	*size = coda_read(dev, CODA_REG_BIT_WR_PTR(ctx->reg_idx)) -
> -		coda_read(dev, CODA_CMD_ENC_HEADER_BB_START);
> +
> +	if (dev->devtype->product == CODA_960) {
> +		for (i = 63; i > 0; i--)
> +			if (((char *)vb2_plane_vaddr(buf, 0))[i] != 0)
> +				break;
> +		*size = i + 1;
> +	} else {
> +		*size = coda_read(dev, CODA_REG_BIT_WR_PTR(ctx->reg_idx)) -
> +			coda_read(dev, CODA_CMD_ENC_HEADER_BB_START);
> +	}
>  	memcpy(header, vb2_plane_vaddr(buf, 0), *size);
>  
>  	return 0;
>  }
>  
> +static int coda_start_encoding(struct coda_ctx *ctx);
> +
>  static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  {
>  	struct coda_ctx *ctx = vb2_get_drv_priv(q);
>  	struct v4l2_device *v4l2_dev = &ctx->dev->v4l2_dev;
> -	u32 bitstream_buf, bitstream_size;
>  	struct coda_dev *dev = ctx->dev;
>  	struct coda_q_data *q_data_src, *q_data_dst;
> -	struct vb2_buffer *buf;
>  	u32 dst_fourcc;
> -	u32 value;
>  	int ret = 0;
>  
>  	q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
> @@ -1983,10 +2182,7 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  		v4l2_m2m_set_src_buffered(ctx->m2m_ctx, true);
>  
>  	ctx->gopcounter = ctx->params.gop_size - 1;
> -	buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
> -	bitstream_buf = vb2_dma_contig_plane_dma_addr(buf, 0);
>  	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
> -	bitstream_size = q_data_dst->sizeimage;
>  	dst_fourcc = q_data_dst->fourcc;
>  
>  	ctx->codec = coda_find_codec(ctx->dev, q_data_src->fourcc,
> @@ -2005,16 +2201,36 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  		mutex_lock(&dev->coda_mutex);
>  		ret = coda_start_decoding(ctx);
>  		mutex_unlock(&dev->coda_mutex);
> -		if (ret == -EAGAIN) {
> +		if (ret == -EAGAIN)
>  			return 0;
> -		} else if (ret < 0) {
> +		else if (ret < 0)
>  			return ret;
> -		} else {
> -			ctx->initialized = 1;
> -			return 0;
> -		}
> +	} else {
> +		ret = coda_start_encoding(ctx);
>  	}
>  
> +	ctx->initialized = 1;
> +	return ret;
> +}
> +
> +static int coda_start_encoding(struct coda_ctx *ctx)
> +{
> +	struct coda_dev *dev = ctx->dev;
> +	struct v4l2_device *v4l2_dev = &dev->v4l2_dev;
> +	struct coda_q_data *q_data_src, *q_data_dst;
> +	u32 bitstream_buf, bitstream_size;
> +	struct vb2_buffer *buf;
> +	int gamma, ret, value;
> +	u32 dst_fourcc;
> +
> +	q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
> +	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
> +	dst_fourcc = q_data_dst->fourcc;
> +
> +	buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
> +	bitstream_buf = vb2_dma_contig_plane_dma_addr(buf, 0);
> +	bitstream_size = q_data_dst->sizeimage;
> +
>  	if (!coda_is_initialized(dev)) {
>  		v4l2_err(v4l2_dev, "coda is not initialized.\n");
>  		return -EFAULT;
> @@ -2030,11 +2246,20 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  		coda_write(dev, CODADX6_STREAM_BUF_DYNALLOC_EN |
>  			CODADX6_STREAM_BUF_PIC_RESET, CODA_REG_BIT_STREAM_CTRL);
>  		break;
> -	default:
> +	case CODA_960:
> +		coda_write(dev, 0, CODA9_GDI_WPROT_RGN_EN);
> +		/* fallthrough */
> +	case CODA_7541:
>  		coda_write(dev, CODA7_STREAM_BUF_DYNALLOC_EN |
>  			CODA7_STREAM_BUF_PIC_RESET, CODA_REG_BIT_STREAM_CTRL);
> +		break;
>  	}
>  
> +	value = coda_read(dev, CODA_REG_BIT_FRAME_MEM_CTRL);
> +	value &= ~(1 << 2 | 0x7 << 9);
> +	ctx->frame_mem_ctrl = value;
> +	coda_write(dev, value, CODA_REG_BIT_FRAME_MEM_CTRL);
> +
>  	if (dev->devtype->product == CODA_DX6) {
>  		/* Configure the coda */
>  		coda_write(dev, dev->iram.paddr, CODADX6_REG_BIT_SEARCH_RAM_BASE_ADDR);
> @@ -2057,11 +2282,17 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  	ctx->params.codec_mode = ctx->codec->mode;
>  	switch (dst_fourcc) {
>  	case V4L2_PIX_FMT_MPEG4:
> -		coda_write(dev, CODA_STD_MPEG4, CODA_CMD_ENC_SEQ_COD_STD);
> +		if (dev->devtype->product == CODA_960)
> +			coda_write(dev, CODA9_STD_MPEG4, CODA_CMD_ENC_SEQ_COD_STD);
> +		else
> +			coda_write(dev, CODA_STD_MPEG4, CODA_CMD_ENC_SEQ_COD_STD);
>  		coda_write(dev, 0, CODA_CMD_ENC_SEQ_MP4_PARA);
>  		break;
>  	case V4L2_PIX_FMT_H264:
> -		coda_write(dev, CODA_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
> +		if (dev->devtype->product == CODA_960)
> +			coda_write(dev, CODA9_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
> +		else
> +			coda_write(dev, CODA_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
>  		coda_write(dev, 0, CODA_CMD_ENC_SEQ_264_PARA);
>  		break;
>  	default:
> @@ -2094,6 +2325,8 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  		/* Rate control enabled */
>  		value = (ctx->params.bitrate & CODA_RATECONTROL_BITRATE_MASK) << CODA_RATECONTROL_BITRATE_OFFSET;
>  		value |=  1 & CODA_RATECONTROL_ENABLE_MASK;
> +		if (dev->devtype->product == CODA_960)
> +			value |= BIT(31); /* disable autoskip */
>  	} else {
>  		value = 0;
>  	}
> @@ -2105,31 +2338,48 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  	coda_write(dev, bitstream_buf, CODA_CMD_ENC_SEQ_BB_START);
>  	coda_write(dev, bitstream_size / 1024, CODA_CMD_ENC_SEQ_BB_SIZE);
>  
> -	/* set default gamma */
> -	value = (CODA_DEFAULT_GAMMA & CODA_GAMMA_MASK) << CODA_GAMMA_OFFSET;
> -	coda_write(dev, value, CODA_CMD_ENC_SEQ_RC_GAMMA);
>  
> -	if (CODA_DEFAULT_GAMMA > 0) {
> -		if (dev->devtype->product == CODA_DX6)
> -			value  = 1 << CODADX6_OPTION_GAMMA_OFFSET;
> -		else
> -			value  = 1 << CODA7_OPTION_GAMMA_OFFSET;
> +	value = 0;
> +	if (dev->devtype->product == CODA_960)
> +		gamma = CODA9_DEFAULT_GAMMA;
> +	else
> +		gamma = CODA_DEFAULT_GAMMA;
> +	if (gamma > 0) {
> +		coda_write(dev, (gamma & CODA_GAMMA_MASK) << CODA_GAMMA_OFFSET,
> +			   CODA_CMD_ENC_SEQ_RC_GAMMA);
> +	}
> +	if (dev->devtype->product == CODA_960) {
> +		if (CODA_DEFAULT_GAMMA > 0)
> +			value |= 1 << CODA9_OPTION_GAMMA_OFFSET;
>  	} else {
> -		value = 0;
> +		if (CODA_DEFAULT_GAMMA > 0) {
> +			if (dev->devtype->product == CODA_DX6)
> +				value |= 1 << CODADX6_OPTION_GAMMA_OFFSET;
> +			else
> +				value |= 1 << CODA7_OPTION_GAMMA_OFFSET;
> +		}
>  	}
>  	coda_write(dev, value, CODA_CMD_ENC_SEQ_OPTION);
>  
> +	coda_write(dev, 0, CODA_CMD_ENC_SEQ_RC_INTERVAL_MODE);
> +
>  	coda_setup_iram(ctx);
>  
>  	if (dst_fourcc == V4L2_PIX_FMT_H264) {
> -		if (dev->devtype->product == CODA_DX6) {
> +		switch (dev->devtype->product) {
> +		case CODA_DX6:
>  			value = FMO_SLICE_SAVE_BUF_SIZE << 7;
>  			coda_write(dev, value, CODADX6_CMD_ENC_SEQ_FMO);
> -		} else {
> +			break;
> +		case CODA_7541:
>  			coda_write(dev, ctx->iram_info.search_ram_paddr,
>  					CODA7_CMD_ENC_SEQ_SEARCH_BASE);
>  			coda_write(dev, ctx->iram_info.search_ram_size,
>  					CODA7_CMD_ENC_SEQ_SEARCH_SIZE);
> +			break;
> +		case CODA_960:
> +			coda_write(dev, 0, CODA9_CMD_ENC_SEQ_ME_OPTION);
> +			coda_write(dev, 0, CODA9_CMD_ENC_SEQ_INTRA_WEIGHT);
>  		}
>  	}
>  
> @@ -2145,7 +2395,10 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  		goto out;
>  	}
>  
> -	ctx->num_internal_frames = 2;
> +	if (dev->devtype->product == CODA_960)
> +		ctx->num_internal_frames = 4;
> +	else
> +		ctx->num_internal_frames = 2;
>  	ret = coda_alloc_framebuffers(ctx, q_data_src, dst_fourcc);
>  	if (ret < 0) {
>  		v4l2_err(v4l2_dev, "failed to allocate framebuffers\n");
> @@ -2168,7 +2421,16 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
>  				CODA7_CMD_SET_FRAME_AXI_DBKC_ADDR);
>  		coda_write(dev, ctx->iram_info.buf_ovl_use,
>  				CODA7_CMD_SET_FRAME_AXI_OVL_ADDR);
> +		if (dev->devtype->product == CODA_960) {
> +			coda_write(dev, ctx->iram_info.buf_btp_use,
> +					CODA9_CMD_SET_FRAME_AXI_BTP_ADDR);
> +
> +			/* FIXME */
> +			coda_write(dev, ctx->internal_frames[2].paddr, CODA9_CMD_SET_FRAME_SUBSAMP_A);
> +			coda_write(dev, ctx->internal_frames[3].paddr, CODA9_CMD_SET_FRAME_SUBSAMP_B);
> +		}
>  	}
> +
>  	ret = coda_command_sync(ctx, CODA_COMMAND_SET_FRAME_BUF);
>  	if (ret < 0) {
>  		v4l2_err(v4l2_dev, "CODA_COMMAND_SET_FRAME_BUF timeout\n");
> @@ -2252,6 +2514,16 @@ static void coda_stop_streaming(struct vb2_queue *q)
>  			 "%s: output\n", __func__);
>  		ctx->streamon_out = 0;
>  
> +		if (ctx->inst_type == CODA_INST_DECODER &&
> +		    coda_isbusy(dev) && ctx->idx == coda_read(dev, CODA_REG_BIT_RUN_INDEX)) {
> +			/* if this decoder instance is running, set the stream end flag */
> +			if (dev->devtype->product == CODA_960) {
> +				u32 val = coda_read(dev, CODA_REG_BIT_BIT_STREAM_PARAM);
> +				val |= CODA_BIT_STREAM_END_FLAG;
> +				coda_write(dev, val, CODA_REG_BIT_BIT_STREAM_PARAM);
> +				ctx->bit_stream_param = val;
> +			}
> +		}
>  		ctx->bit_stream_param |= CODA_BIT_STREAM_END_FLAG;
>  
>  		ctx->isequence = 0;
> @@ -2453,6 +2725,7 @@ static int coda_open(struct file *file)
>  	ctx->idx = idx;
>  	switch (dev->devtype->product) {
>  	case CODA_7541:
> +	case CODA_960:
>  		ctx->reg_idx = 0;
>  		break;
>  	default:
> @@ -2772,7 +3045,7 @@ static void coda_finish_encode(struct coda_ctx *ctx)
>  	u32 wr_ptr, start_ptr;
>  
>  	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
> -	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
> +	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
>  
>  	/* Get results from the coda */
>  	start_ptr = coda_read(dev, CODA_CMD_ENC_PIC_BB_START);
> @@ -2809,6 +3082,8 @@ static void coda_finish_encode(struct coda_ctx *ctx)
>  	dst_buf->v4l2_buf.timecode = src_buf->v4l2_buf.timecode;
>  
>  	v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
> +
> +	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
>  	v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
>  
>  	ctx->gopcounter--;
> @@ -2907,6 +3182,7 @@ static void coda_timeout(struct work_struct *work)
>  static u32 coda_supported_firmwares[] = {
>  	CODA_FIRMWARE_VERNUM(CODA_DX6, 2, 2, 5),
>  	CODA_FIRMWARE_VERNUM(CODA_7541, 1, 4, 50),
> +	CODA_FIRMWARE_VERNUM(CODA_960, 2, 1, 5),
>  };
>  
>  static bool coda_firmware_supported(u32 vernum)
> @@ -2961,7 +3237,8 @@ static int coda_hw_init(struct coda_dev *dev)
>  		coda_write(dev, 0, CODA_REG_BIT_CODE_BUF_ADDR + i * 4);
>  
>  	/* Tell the BIT where to find everything it needs */
> -	if (dev->devtype->product == CODA_7541) {
> +	if (dev->devtype->product == CODA_960 ||
> +	    dev->devtype->product == CODA_7541) {
>  		coda_write(dev, dev->tempbuf.paddr,
>  				CODA_REG_BIT_TEMP_BUF_ADDR);
>  		coda_write(dev, 0, CODA_REG_BIT_BIT_STREAM_PARAM);
> @@ -2981,7 +3258,10 @@ static int coda_hw_init(struct coda_dev *dev)
>  	default:
>  		coda_write(dev, CODA7_STREAM_BUF_PIC_FLUSH, CODA_REG_BIT_STREAM_CTRL);
>  	}
> -	coda_write(dev, 0, CODA_REG_BIT_FRAME_MEM_CTRL);
> +	if (dev->devtype->product == CODA_960)
> +		coda_write(dev, 1 << 12, CODA_REG_BIT_FRAME_MEM_CTRL);
> +	else
> +		coda_write(dev, 0, CODA_REG_BIT_FRAME_MEM_CTRL);
>  
>  	if (dev->devtype->product != CODA_DX6)
>  		coda_write(dev, 0, CODA7_REG_BIT_AXI_SRAM_USE);
> @@ -3011,6 +3291,12 @@ static int coda_hw_init(struct coda_dev *dev)
>  		return -EIO;
>  	}
>  
> +	if (dev->devtype->product == CODA_960) {
> +		data = coda_read(dev, CODA9_CMD_FIRMWARE_CODE_REV);
> +		v4l2_info(&dev->v4l2_dev, "Firmware code revision: %d\n",
> +			  data);
> +	}
> +
>  	/* Check we are compatible with the loaded firmware */
>  	data = coda_read(dev, CODA_CMD_FIRMWARE_VERNUM);
>  	product = CODA_FIRMWARE_PRODUCT(data);
> @@ -3126,6 +3412,8 @@ static int coda_firmware_request(struct coda_dev *dev)
>  enum coda_platform {
>  	CODA_IMX27,
>  	CODA_IMX53,
> +	CODA_IMX6Q,
> +	CODA_IMX6DL,
>  };
>  
>  static const struct coda_devtype coda_devdata[] = {
> @@ -3141,6 +3429,18 @@ static const struct coda_devtype coda_devdata[] = {
>  		.codecs     = coda7_codecs,
>  		.num_codecs = ARRAY_SIZE(coda7_codecs),
>  	},
> +	[CODA_IMX6Q] = {
> +		.firmware   = "v4l-coda960-imx6q.bin",
> +		.product    = CODA_960,
> +		.codecs     = coda9_codecs,
> +		.num_codecs = ARRAY_SIZE(coda9_codecs),
> +	},
> +	[CODA_IMX6DL] = {
> +		.firmware   = "v4l-coda960-imx6dl.bin",
> +		.product    = CODA_960,
> +		.codecs     = coda9_codecs,
> +		.num_codecs = ARRAY_SIZE(coda9_codecs),
> +	},
>  };
>  
>  static struct platform_device_id coda_platform_ids[] = {
> @@ -3154,6 +3454,8 @@ MODULE_DEVICE_TABLE(platform, coda_platform_ids);
>  static const struct of_device_id coda_dt_ids[] = {
>  	{ .compatible = "fsl,imx27-vpu", .data = &coda_devdata[CODA_IMX27] },
>  	{ .compatible = "fsl,imx53-vpu", .data = &coda_devdata[CODA_IMX53] },
> +	{ .compatible = "fsl,imx6q-vpu", .data = &coda_devdata[CODA_IMX6Q] },
> +	{ .compatible = "fsl,imx6dl-vpu", .data = &coda_devdata[CODA_IMX6DL] },
>  	{ /* sentinel */ }
>  };
>  MODULE_DEVICE_TABLE(of, coda_dt_ids);
> @@ -3256,6 +3558,9 @@ static int coda_probe(struct platform_device *pdev)
>  	case CODA_7541:
>  		dev->tempbuf.size = CODA7_TEMP_BUF_SIZE;
>  		break;
> +	case CODA_960:
> +		dev->tempbuf.size = CODA9_TEMP_BUF_SIZE;
> +		break;
>  	}
>  	if (dev->tempbuf.size) {
>  		ret = coda_alloc_aux_buf(dev, &dev->tempbuf,
> @@ -3274,6 +3579,8 @@ static int coda_probe(struct platform_device *pdev)
>  	case CODA_7541:
>  		dev->iram.size = CODA7_IRAM_SIZE;
>  		break;
> +	case CODA_960:
> +		dev->iram.size = CODA9_IRAM_SIZE;
>  	}
>  	dev->iram.vaddr = gen_pool_dma_alloc(dev->iram_pool, dev->iram.size,
>  					     &dev->iram.paddr);
> diff --git a/drivers/media/platform/coda.h b/drivers/media/platform/coda.h
> index 4e32e2e..c791275 100644
> --- a/drivers/media/platform/coda.h
> +++ b/drivers/media/platform/coda.h
> @@ -27,6 +27,14 @@
>  #define CODA_REG_BIT_CODE_RESET		0x014
>  #define		CODA_REG_RESET_ENABLE		(1 << 0)
>  #define CODA_REG_BIT_CUR_PC			0x018
> +#define CODA9_REG_BIT_SW_RESET			0x024
> +#define		CODA9_SW_RESET_BPU_CORE   0x008
> +#define		CODA9_SW_RESET_BPU_BUS    0x010
> +#define		CODA9_SW_RESET_VCE_CORE   0x020
> +#define		CODA9_SW_RESET_VCE_BUS    0x040
> +#define		CODA9_SW_RESET_GDI_CORE   0x080
> +#define		CODA9_SW_RESET_GDI_BUS    0x100
> +#define CODA9_REG_BIT_SW_RESET_STATUS		0x034
>  
>  /* Static SW registers */
>  #define CODA_REG_BIT_CODE_BUF_ADDR		0x100
> @@ -39,9 +47,11 @@
>  #define		CODADX6_STREAM_BUF_PIC_FLUSH	(1 << 2)
>  #define		CODA7_STREAM_BUF_DYNALLOC_EN	(1 << 5)
>  #define		CODADX6_STREAM_BUF_DYNALLOC_EN	(1 << 4)
> -#define 	CODA_STREAM_CHKDIS_OFFSET	(1 << 1)
> +#define		CODADX6_STREAM_CHKDIS_OFFSET	(1 << 1)
> +#define		CODA7_STREAM_SEL_64BITS_ENDIAN	(1 << 1)
>  #define		CODA_STREAM_ENDIAN_SELECT	(1 << 0)
>  #define CODA_REG_BIT_FRAME_MEM_CTRL		0x110
> +#define		CODA_FRAME_CHROMA_INTERLEAVE	(1 << 2)
>  #define		CODA_IMAGE_ENDIAN_SELECT	(1 << 0)
>  #define CODA_REG_BIT_BIT_STREAM_PARAM		0x114
>  #define		CODA_BIT_STREAM_END_FLAG	(1 << 2)
> @@ -52,13 +62,21 @@
>  #define CODA_REG_BIT_FRM_DIS_FLG(x)		(0x150 + 4 * (x))
>  #define CODADX6_REG_BIT_SEARCH_RAM_BASE_ADDR	0x140
>  #define CODA7_REG_BIT_AXI_SRAM_USE		0x140
> +#define		CODA9_USE_HOST_BTP_ENABLE	(1 << 13)
> +#define		CODA9_USE_HOST_OVL_ENABLE	(1 << 12)
>  #define		CODA7_USE_HOST_ME_ENABLE	(1 << 11)
> +#define		CODA9_USE_HOST_DBK_ENABLE	(3 << 10)
>  #define		CODA7_USE_HOST_OVL_ENABLE	(1 << 10)
>  #define		CODA7_USE_HOST_DBK_ENABLE	(1 << 9)
> +#define		CODA9_USE_HOST_IP_ENABLE	(1 << 9)
>  #define		CODA7_USE_HOST_IP_ENABLE	(1 << 8)
> +#define		CODA9_USE_HOST_BIT_ENABLE	(1 << 8)
>  #define		CODA7_USE_HOST_BIT_ENABLE	(1 << 7)
> +#define		CODA9_USE_BTP_ENABLE		(1 << 5)
>  #define		CODA7_USE_ME_ENABLE		(1 << 4)
> +#define		CODA9_USE_OVL_ENABLE		(1 << 4)
>  #define		CODA7_USE_OVL_ENABLE		(1 << 3)
> +#define		CODA9_USE_DBK_ENABLE		(3 << 2)
>  #define		CODA7_USE_DBK_ENABLE		(1 << 2)
>  #define		CODA7_USE_IP_ENABLE		(1 << 1)
>  #define		CODA7_USE_BIT_ENABLE		(1 << 0)
> @@ -93,6 +111,18 @@
>  #define		CODA7_MODE_ENCODE_H264		8
>  #define		CODA7_MODE_ENCODE_MP4		11
>  #define		CODA7_MODE_ENCODE_MJPG		13
> +#define		CODA9_MODE_DECODE_H264		0
> +#define		CODA9_MODE_DECODE_VC1		1
> +#define		CODA9_MODE_DECODE_MP2		2
> +#define		CODA9_MODE_DECODE_MP4		3
> +#define		CODA9_MODE_DECODE_DV3		3
> +#define		CODA9_MODE_DECODE_RV		4
> +#define		CODA9_MODE_DECODE_AVS		5
> +#define		CODA9_MODE_DECODE_MJPG		6
> +#define		CODA9_MODE_DECODE_VPX		7
> +#define		CODA9_MODE_ENCODE_H264		8
> +#define		CODA9_MODE_ENCODE_MP4		11
> +#define		CODA9_MODE_ENCODE_MJPG		13
>  #define 	CODA_MODE_INVALID		0xffff
>  #define CODA_REG_BIT_INT_ENABLE		0x170
>  #define		CODA_INT_INTERRUPT_ENABLE	(1 << 3)
> @@ -129,6 +159,7 @@
>  #define CODA_CMD_DEC_SEQ_SPP_CHUNK_SIZE		0x1a0
>  
>  #define CODA7_RET_DEC_SEQ_ASPECT		0x1b0
> +#define CODA9_RET_DEC_SEQ_BITRATE		0x1b4
>  #define CODA_RET_DEC_SEQ_SUCCESS		0x1c0
>  #define CODA_RET_DEC_SEQ_SRC_FMT		0x1c4 /* SRC_SIZE on CODA7 */
>  #define CODA_RET_DEC_SEQ_SRC_SIZE		0x1c4
> @@ -145,13 +176,19 @@
>  #define CODA_RET_DEC_SEQ_FRATE_DR		0x1e8
>  #define CODA_RET_DEC_SEQ_JPG_PARA		0x1e4
>  #define CODA_RET_DEC_SEQ_JPG_THUMB_IND		0x1e8
> +#define CODA9_RET_DEC_SEQ_HEADER_REPORT		0x1ec
>  
>  /* Decoder Picture Run */
>  #define CODA_CMD_DEC_PIC_ROT_MODE		0x180
>  #define CODA_CMD_DEC_PIC_ROT_ADDR_Y		0x184
> +#define CODA9_CMD_DEC_PIC_ROT_INDEX		0x184
>  #define CODA_CMD_DEC_PIC_ROT_ADDR_CB		0x188
> +#define CODA9_CMD_DEC_PIC_ROT_ADDR_Y		0x188
>  #define CODA_CMD_DEC_PIC_ROT_ADDR_CR		0x18c
> +#define CODA9_CMD_DEC_PIC_ROT_ADDR_CB		0x18c
>  #define CODA_CMD_DEC_PIC_ROT_STRIDE		0x190
> +#define CODA9_CMD_DEC_PIC_ROT_ADDR_CR		0x190
> +#define CODA9_CMD_DEC_PIC_ROT_STRIDE		0x1b8
>  
>  #define CODA_CMD_DEC_PIC_OPTION			0x194
>  #define		CODA_PRE_SCAN_EN			(1 << 0)
> @@ -183,25 +220,39 @@
>  #define CODA_RET_DEC_PIC_CROP_TOP_BOTTOM	0x1e4
>  #define CODA_RET_DEC_PIC_FRAME_NEED		0x1ec
>  
> +#define CODA9_RET_DEC_PIC_VP8_PIC_REPORT	0x1e8
> +#define CODA9_RET_DEC_PIC_ASPECT		0x1f0
> +#define CODA9_RET_DEC_PIC_VP8_SCALE_INFO	0x1f0
> +#define CODA9_RET_DEC_PIC_FRATE_NR		0x1f4
> +#define CODA9_RET_DEC_PIC_FRATE_DR		0x1f8
> +
>  /* Encoder Sequence Initialization */
>  #define CODA_CMD_ENC_SEQ_BB_START				0x180
>  #define CODA_CMD_ENC_SEQ_BB_SIZE				0x184
>  #define CODA_CMD_ENC_SEQ_OPTION				0x188
>  #define		CODA7_OPTION_AVCINTRA16X16ONLY_OFFSET		9
> +#define		CODA9_OPTION_MVC_PREFIX_NAL_OFFSET		9
>  #define		CODA7_OPTION_GAMMA_OFFSET			8
> +#define		CODA9_OPTION_MVC_PARASET_REFRESH_OFFSET		8
>  #define		CODA7_OPTION_RCQPMAX_OFFSET			7
> +#define		CODA9_OPTION_GAMMA_OFFSET			7
>  #define		CODADX6_OPTION_GAMMA_OFFSET			7
>  #define		CODA7_OPTION_RCQPMIN_OFFSET			6
> +#define		CODA9_OPTION_RCQPMAX_OFFSET			6
>  #define		CODA_OPTION_LIMITQP_OFFSET			6
>  #define		CODA_OPTION_RCINTRAQP_OFFSET			5
>  #define		CODA_OPTION_FMO_OFFSET				4
> +#define		CODA9_OPTION_MVC_INTERVIEW_OFFSET		4
>  #define		CODA_OPTION_AVC_AUD_OFFSET			2
>  #define		CODA_OPTION_SLICEREPORT_OFFSET			1
>  #define CODA_CMD_ENC_SEQ_COD_STD				0x18c
>  #define		CODA_STD_MPEG4					0
> +#define		CODA9_STD_H264					0
>  #define		CODA_STD_H263					1
>  #define		CODA_STD_H264					2
>  #define		CODA_STD_MJPG					3
> +#define		CODA9_STD_MPEG4					3
> +
>  #define CODA_CMD_ENC_SEQ_SRC_SIZE				0x190
>  #define		CODA7_PICWIDTH_OFFSET				16
>  #define		CODA7_PICWIDTH_MASK				0xffff
> @@ -268,15 +319,26 @@
>  #define CODA7_CMD_ENC_SEQ_SEARCH_BASE				0x1b8
>  #define CODA7_CMD_ENC_SEQ_SEARCH_SIZE				0x1bc
>  #define CODA7_CMD_ENC_SEQ_INTRA_QP				0x1c4
> -#define CODA_CMD_ENC_SEQ_RC_QP_MAX				0x1c8
> +#define CODA_CMD_ENC_SEQ_RC_QP_MIN_MAX				0x1c8
> +#define		CODA_QPMIN_OFFSET				8
> +#define		CODA_QPMIN_MASK					0x3f
>  #define		CODA_QPMAX_OFFSET				0
>  #define		CODA_QPMAX_MASK					0x3f
>  #define CODA_CMD_ENC_SEQ_RC_GAMMA				0x1cc
>  #define		CODA_GAMMA_OFFSET				0
>  #define		CODA_GAMMA_MASK					0xffff
> +#define CODA_CMD_ENC_SEQ_RC_INTERVAL_MODE			0x1d0
> +#define CODA9_CMD_ENC_SEQ_INTRA_WEIGHT				0x1d4
> +#define CODA9_CMD_ENC_SEQ_ME_OPTION				0x1d8
>  #define CODA_RET_ENC_SEQ_SUCCESS				0x1c0
>  
>  /* Encoder Picture Run */
> +#define CODA9_CMD_ENC_PIC_SRC_INDEX		0x180
> +#define CODA9_CMD_ENC_PIC_SRC_STRIDE		0x184
> +#define CODA9_CMD_ENC_PIC_SUB_FRAME_SYNC	0x1a4
> +#define CODA9_CMD_ENC_PIC_SRC_ADDR_Y		0x1a8
> +#define CODA9_CMD_ENC_PIC_SRC_ADDR_CB		0x1ac
> +#define CODA9_CMD_ENC_PIC_SRC_ADDR_CR		0x1b0
>  #define CODA_CMD_ENC_PIC_SRC_ADDR_Y	0x180
>  #define CODA_CMD_ENC_PIC_SRC_ADDR_CB	0x184
>  #define CODA_CMD_ENC_PIC_SRC_ADDR_CR	0x188
> @@ -291,7 +353,11 @@
>  #define		CODA_MIR_VER					(0x1 << 2)
>  #define		CODA_MIR_HOR					(0x2 << 2)
>  #define		CODA_MIR_VER_HOR				(0x3 << 2)
> -#define CODA_CMD_ENC_PIC_OPTION	0x194
> +#define CODA_CMD_ENC_PIC_OPTION		0x194
> +#define		CODA_FORCE_IPICTURE				BIT(1)
> +#define		CODA_REPORT_MB_INFO				BIT(3)
> +#define		CODA_REPORT_MV_INFO				BIT(4)
> +#define		CODA_REPORT_SLICE_INFO				BIT(5)
>  #define CODA_CMD_ENC_PIC_BB_START	0x198
>  #define CODA_CMD_ENC_PIC_BB_SIZE	0x19c
>  #define CODA_RET_ENC_FRAME_NUM		0x1c0
> @@ -306,13 +372,30 @@
>  #define CODA_CMD_SET_FRAME_BUF_STRIDE		0x184
>  #define CODA_CMD_SET_FRAME_SLICE_BB_START	0x188
>  #define CODA_CMD_SET_FRAME_SLICE_BB_SIZE	0x18c
> +#define CODA9_CMD_SET_FRAME_SUBSAMP_A		0x188
> +#define CODA9_CMD_SET_FRAME_SUBSAMP_B		0x18c
>  #define CODA7_CMD_SET_FRAME_AXI_BIT_ADDR	0x190
>  #define CODA7_CMD_SET_FRAME_AXI_IPACDC_ADDR	0x194
>  #define CODA7_CMD_SET_FRAME_AXI_DBKY_ADDR	0x198
>  #define CODA7_CMD_SET_FRAME_AXI_DBKC_ADDR	0x19c
>  #define CODA7_CMD_SET_FRAME_AXI_OVL_ADDR	0x1a0
>  #define CODA7_CMD_SET_FRAME_MAX_DEC_SIZE	0x1a4
> +#define CODA9_CMD_SET_FRAME_AXI_BTP_ADDR	0x1a4
>  #define CODA7_CMD_SET_FRAME_SOURCE_BUF_STRIDE	0x1a8
> +#define CODA9_CMD_SET_FRAME_CACHE_SIZE		0x1a8
> +#define CODA9_CMD_SET_FRAME_CACHE_CONFIG	0x1ac
> +#define		CODA9_CACHE_BYPASS_OFFSET		28
> +#define		CODA9_CACHE_DUALCONF_OFFSET		26
> +#define		CODA9_CACHE_PAGEMERGE_OFFSET		24
> +#define		CODA9_CACHE_LUMA_BUFFER_SIZE_OFFSET	16
> +#define		CODA9_CACHE_CB_BUFFER_SIZE_OFFSET	8
> +#define		CODA9_CACHE_CR_BUFFER_SIZE_OFFSET	0
> +#define CODA9_CMD_SET_FRAME_SUBSAMP_A_MVC	0x1b0
> +#define CODA9_CMD_SET_FRAME_SUBSAMP_B_MVC	0x1b4
> +#define CODA9_CMD_SET_FRAME_DP_BUF_BASE		0x1b0
> +#define CODA9_CMD_SET_FRAME_DP_BUF_SIZE		0x1b4
> +#define CODA9_CMD_SET_FRAME_MAX_DEC_SIZE	0x1b8
> +#define CODA9_CMD_SET_FRAME_DELAY		0x1bc
>  
>  /* Encoder Header */
>  #define CODA_CMD_ENC_HEADER_CODE	0x180
> @@ -322,8 +405,11 @@
>  #define		CODA_HEADER_MP4V_VOL	0
>  #define		CODA_HEADER_MP4V_VOS	1
>  #define		CODA_HEADER_MP4V_VIS	2
> +#define		CODA9_HEADER_FRAME_CROP	(1 << 3)
>  #define CODA_CMD_ENC_HEADER_BB_START	0x184
>  #define CODA_CMD_ENC_HEADER_BB_SIZE	0x188
> +#define CODA9_CMD_ENC_HEADER_FRAME_CROP_H	0x18c
> +#define CODA9_CMD_ENC_HEADER_FRAME_CROP_V	0x190
>  
>  /* Get Version */
>  #define CODA_CMD_FIRMWARE_VERNUM		0x1c0
> @@ -334,5 +420,28 @@
>  #define		CODA_FIRMWARE_VERNUM(product, major, minor, release)	\
>  			((product) << 16 | ((major) << 12) |		\
>  			((minor) << 8) | (release))
> +#define CODA9_CMD_FIRMWARE_CODE_REV		0x1c4
> +
> +#define CODA9_GDMA_BASE				0x1000
> +#define CODA9_GDI_WPROT_ERR_CLR			(CODA9_GDMA_BASE + 0x0a0)
> +#define CODA9_GDI_WPROT_RGN_EN			(CODA9_GDMA_BASE + 0x0ac)
> +
> +#define CODA9_GDI_BUS_CTRL			(CODA9_GDMA_BASE + 0x0f0)
> +#define CODA9_GDI_BUS_STATUS			(CODA9_GDMA_BASE + 0x0f4)
> +
> +#define CODA9_GDI_XY2_CAS_0			(CODA9_GDMA_BASE + 0x800)
> +#define CODA9_GDI_XY2_CAS_F			(CODA9_GDMA_BASE + 0x83c)
> +
> +#define CODA9_GDI_XY2_BA_0			(CODA9_GDMA_BASE + 0x840)
> +#define CODA9_GDI_XY2_BA_1			(CODA9_GDMA_BASE + 0x844)
> +#define CODA9_GDI_XY2_BA_2			(CODA9_GDMA_BASE + 0x848)
> +#define CODA9_GDI_XY2_BA_3			(CODA9_GDMA_BASE + 0x84c)
> +
> +#define CODA9_GDI_XY2_RAS_0			(CODA9_GDMA_BASE + 0x850)
> +#define CODA9_GDI_XY2_RAS_F			(CODA9_GDMA_BASE + 0x88c)
> +
> +#define CODA9_GDI_XY2_RBC_CONFIG		(CODA9_GDMA_BASE + 0x890)
> +#define CODA9_GDI_RBC2_AXI_0			(CODA9_GDMA_BASE + 0x8a0)
> +#define CODA9_GDI_RBC2_AXI_1F			(CODA9_GDMA_BASE + 0x91c)
>  
>  #endif

Re: [PATCH 06/30] [media] coda: Add encoder/decoder support for CODA960

[Philipp Zabel]

Hi Nicolas,

Am Dienstag, den 24.06.2014, 11:53 -0400 schrieb Nicolas Dufresne:
> Le vendredi 13 juin 2014 à 18:08 +0200, Philipp Zabel a écrit :
> > This patch adds support for the CODA960 VPU in Freescale i.MX6 SoCs.
> 
> I might be confused, but is this driver sharing the same device node for
> the encoder and the decoder ? If so why ? I know the spec might not be
> preventing it, but I don't know how in userspace I'm supposed to figure
> the type of m2m node this is ? Other drivers have decided to split
> encoding, decoding and transformation into their own node, which made it
> easier to use generically.

you are right. I'm planning to split this into at least encoder and
decoder device, and probably even later add the JPEG codec as separate
devices because of different hardware constraints.
I just didn't manage to find enough time for this yet.

regards
Philipp

[PATCH 07/30] [media] coda: add selection API support for h.264 decoder

[Philipp Zabel]

The h.264 decoder produces capture frames that are a multiple of the macroblock
size (16 pixels). To inform userspace about invalid pixel data at the edges,
use the active and padded composing rectangles on the capture queue.
The cropping information is obtained from the h.264 sequence parameter set.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 87 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 10cc031..7e4df82 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -119,6 +119,7 @@ struct coda_q_data {
 	unsigned int		height;
 	unsigned int		sizeimage;
 	unsigned int		fourcc;
+	struct v4l2_rect	rect;
 };
 
 struct coda_aux_buf {
@@ -737,6 +738,10 @@ static int coda_s_fmt(struct coda_ctx *ctx, struct v4l2_format *f)
 	q_data->width = f->fmt.pix.width;
 	q_data->height = f->fmt.pix.height;
 	q_data->sizeimage = f->fmt.pix.sizeimage;
+	q_data->rect.left = 0;
+	q_data->rect.top = 0;
+	q_data->rect.width = f->fmt.pix.width;
+	q_data->rect.height = f->fmt.pix.height;
 
 	v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev,
 		"Setting format for type %d, wxh: %dx%d, fmt: %d\n",
@@ -873,6 +878,43 @@ static int coda_streamoff(struct file *file, void *priv,
 	return ret;
 }
 
+static int coda_g_selection(struct file *file, void *fh,
+			    struct v4l2_selection *s)
+{
+	struct coda_ctx *ctx = fh_to_ctx(fh);
+	struct coda_q_data *q_data;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+		s->r = q_data->rect;
+		break;
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = q_data->width;
+		s->r.height = q_data->height;
+		break;
+	case V4L2_SEL_TGT_COMPOSE:
+	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+		s->r = q_data->rect;
+		break;
+	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+	case V4L2_SEL_TGT_COMPOSE_PADDED:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = q_data->width;
+		s->r.height = q_data->height;
+		break;
+	}
+
+	return 0;
+}
+
 static int coda_try_decoder_cmd(struct file *file, void *fh,
 				struct v4l2_decoder_cmd *dc)
 {
@@ -951,6 +993,8 @@ static const struct v4l2_ioctl_ops coda_ioctl_ops = {
 	.vidioc_streamon	= coda_streamon,
 	.vidioc_streamoff	= coda_streamoff,
 
+	.vidioc_g_selection	= coda_g_selection,
+
 	.vidioc_try_decoder_cmd	= coda_try_decoder_cmd,
 	.vidioc_decoder_cmd	= coda_decoder_cmd,
 
@@ -1506,6 +1550,10 @@ static void set_default_params(struct coda_ctx *ctx)
 	ctx->q_data[V4L2_M2M_DST].width = max_w;
 	ctx->q_data[V4L2_M2M_DST].height = max_h;
 	ctx->q_data[V4L2_M2M_DST].sizeimage = CODA_MAX_FRAME_SIZE;
+	ctx->q_data[V4L2_M2M_SRC].rect.width = max_w;
+	ctx->q_data[V4L2_M2M_SRC].rect.height = max_h;
+	ctx->q_data[V4L2_M2M_DST].rect.width = max_w;
+	ctx->q_data[V4L2_M2M_DST].rect.height = max_h;
 
 	if (ctx->dev->devtype->product == CODA_960)
 		coda_set_tiled_map_type(ctx, GDI_LINEAR_FRAME_MAP);
@@ -2033,6 +2081,21 @@ static int coda_start_decoding(struct coda_ctx *ctx)
 		return -EINVAL;
 	}
 
+	if (src_fourcc == V4L2_PIX_FMT_H264) {
+		u32 left_right;
+		u32 top_bottom;
+
+		left_right = coda_read(dev, CODA_RET_DEC_SEQ_CROP_LEFT_RIGHT);
+		top_bottom = coda_read(dev, CODA_RET_DEC_SEQ_CROP_TOP_BOTTOM);
+
+		q_data_dst->rect.left = (left_right >> 10) & 0x3ff;
+		q_data_dst->rect.top = (top_bottom >> 10) & 0x3ff;
+		q_data_dst->rect.width = width - q_data_dst->rect.left -
+					 (left_right & 0x3ff);
+		q_data_dst->rect.height = height - q_data_dst->rect.top -
+					  (top_bottom & 0x3ff);
+	}
+
 	ret = coda_alloc_framebuffers(ctx, q_data_dst, src_fourcc);
 	if (ret < 0)
 		return ret;
@@ -2939,6 +3002,30 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 
 	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
 
+	/* frame crop information */
+	if (src_fourcc == V4L2_PIX_FMT_H264) {
+		u32 left_right;
+		u32 top_bottom;
+
+		left_right = coda_read(dev, CODA_RET_DEC_PIC_CROP_LEFT_RIGHT);
+		top_bottom = coda_read(dev, CODA_RET_DEC_PIC_CROP_TOP_BOTTOM);
+
+		if (left_right == 0xffffffff && top_bottom == 0xffffffff) {
+			/* Keep current crop information */
+		} else {
+			struct v4l2_rect *rect = &q_data_dst->rect;
+
+			rect->left = left_right >> 16 & 0xffff;
+			rect->top = top_bottom >> 16 & 0xffff;
+			rect->width = width - rect->left -
+				      (left_right & 0xffff);
+			rect->height = height - rect->top -
+				       (top_bottom & 0xffff);
+		}
+	} else {
+		/* no cropping */
+	}
+
 	val = coda_read(dev, CODA_RET_DEC_PIC_ERR_MB);
 	if (val > 0)
 		v4l2_err(&dev->v4l2_dev,
-- 
2.0.0.rc2

Re: [PATCH 07/30] [media] coda: add selection API support for h.264 decoder

[Hans Verkuil]

On 06/13/2014 06:08 PM, Philipp Zabel wrote:
> The h.264 decoder produces capture frames that are a multiple of the macroblock
> size (16 pixels). To inform userspace about invalid pixel data at the edges,
> use the active and padded composing rectangles on the capture queue.
> The cropping information is obtained from the h.264 sequence parameter set.
> 
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
>  drivers/media/platform/coda.c | 87 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 87 insertions(+)
> 
> diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
> index 10cc031..7e4df82 100644
> --- a/drivers/media/platform/coda.c
> +++ b/drivers/media/platform/coda.c
> @@ -119,6 +119,7 @@ struct coda_q_data {
>  	unsigned int		height;
>  	unsigned int		sizeimage;
>  	unsigned int		fourcc;
> +	struct v4l2_rect	rect;
>  };
>  
>  struct coda_aux_buf {
> @@ -737,6 +738,10 @@ static int coda_s_fmt(struct coda_ctx *ctx, struct v4l2_format *f)
>  	q_data->width = f->fmt.pix.width;
>  	q_data->height = f->fmt.pix.height;
>  	q_data->sizeimage = f->fmt.pix.sizeimage;
> +	q_data->rect.left = 0;
> +	q_data->rect.top = 0;
> +	q_data->rect.width = f->fmt.pix.width;
> +	q_data->rect.height = f->fmt.pix.height;
>  
>  	v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev,
>  		"Setting format for type %d, wxh: %dx%d, fmt: %d\n",
> @@ -873,6 +878,43 @@ static int coda_streamoff(struct file *file, void *priv,
>  	return ret;
>  }
>  
> +static int coda_g_selection(struct file *file, void *fh,
> +			    struct v4l2_selection *s)
> +{
> +	struct coda_ctx *ctx = fh_to_ctx(fh);
> +	struct coda_q_data *q_data;
> +
> +	switch (s->target) {
> +	case V4L2_SEL_TGT_CROP:
> +		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);

There is no check against s->type: CROP is only supported for output
buffers and should return -EINVAL for capture buffers and the reverse
for COMPOSE.

> +		s->r = q_data->rect;
> +		break;
> +	case V4L2_SEL_TGT_CROP_DEFAULT:
> +	case V4L2_SEL_TGT_CROP_BOUNDS:
> +		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
> +		s->r.left = 0;
> +		s->r.top = 0;
> +		s->r.width = q_data->width;
> +		s->r.height = q_data->height;
> +		break;
> +	case V4L2_SEL_TGT_COMPOSE:
> +	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
> +		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
> +		s->r = q_data->rect;
> +		break;
> +	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
> +	case V4L2_SEL_TGT_COMPOSE_PADDED:
> +		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
> +		s->r.left = 0;
> +		s->r.top = 0;
> +		s->r.width = q_data->width;
> +		s->r.height = q_data->height;
> +		break;

Add:

	default:
		return -EINVAL;

Regards,

	Hans

> +	}
> +
> +	return 0;
> +}
> +
>  static int coda_try_decoder_cmd(struct file *file, void *fh,
>  				struct v4l2_decoder_cmd *dc)
>  {
> @@ -951,6 +993,8 @@ static const struct v4l2_ioctl_ops coda_ioctl_ops = {
>  	.vidioc_streamon	= coda_streamon,
>  	.vidioc_streamoff	= coda_streamoff,
>  
> +	.vidioc_g_selection	= coda_g_selection,
> +
>  	.vidioc_try_decoder_cmd	= coda_try_decoder_cmd,
>  	.vidioc_decoder_cmd	= coda_decoder_cmd,
>  
> @@ -1506,6 +1550,10 @@ static void set_default_params(struct coda_ctx *ctx)
>  	ctx->q_data[V4L2_M2M_DST].width = max_w;
>  	ctx->q_data[V4L2_M2M_DST].height = max_h;
>  	ctx->q_data[V4L2_M2M_DST].sizeimage = CODA_MAX_FRAME_SIZE;
> +	ctx->q_data[V4L2_M2M_SRC].rect.width = max_w;
> +	ctx->q_data[V4L2_M2M_SRC].rect.height = max_h;
> +	ctx->q_data[V4L2_M2M_DST].rect.width = max_w;
> +	ctx->q_data[V4L2_M2M_DST].rect.height = max_h;
>  
>  	if (ctx->dev->devtype->product == CODA_960)
>  		coda_set_tiled_map_type(ctx, GDI_LINEAR_FRAME_MAP);
> @@ -2033,6 +2081,21 @@ static int coda_start_decoding(struct coda_ctx *ctx)
>  		return -EINVAL;
>  	}
>  
> +	if (src_fourcc == V4L2_PIX_FMT_H264) {
> +		u32 left_right;
> +		u32 top_bottom;
> +
> +		left_right = coda_read(dev, CODA_RET_DEC_SEQ_CROP_LEFT_RIGHT);
> +		top_bottom = coda_read(dev, CODA_RET_DEC_SEQ_CROP_TOP_BOTTOM);
> +
> +		q_data_dst->rect.left = (left_right >> 10) & 0x3ff;
> +		q_data_dst->rect.top = (top_bottom >> 10) & 0x3ff;
> +		q_data_dst->rect.width = width - q_data_dst->rect.left -
> +					 (left_right & 0x3ff);
> +		q_data_dst->rect.height = height - q_data_dst->rect.top -
> +					  (top_bottom & 0x3ff);
> +	}
> +
>  	ret = coda_alloc_framebuffers(ctx, q_data_dst, src_fourcc);
>  	if (ret < 0)
>  		return ret;
> @@ -2939,6 +3002,30 @@ static void coda_finish_decode(struct coda_ctx *ctx)
>  
>  	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
>  
> +	/* frame crop information */
> +	if (src_fourcc == V4L2_PIX_FMT_H264) {
> +		u32 left_right;
> +		u32 top_bottom;
> +
> +		left_right = coda_read(dev, CODA_RET_DEC_PIC_CROP_LEFT_RIGHT);
> +		top_bottom = coda_read(dev, CODA_RET_DEC_PIC_CROP_TOP_BOTTOM);
> +
> +		if (left_right == 0xffffffff && top_bottom == 0xffffffff) {
> +			/* Keep current crop information */
> +		} else {
> +			struct v4l2_rect *rect = &q_data_dst->rect;
> +
> +			rect->left = left_right >> 16 & 0xffff;
> +			rect->top = top_bottom >> 16 & 0xffff;
> +			rect->width = width - rect->left -
> +				      (left_right & 0xffff);
> +			rect->height = height - rect->top -
> +				       (top_bottom & 0xffff);
> +		}
> +	} else {
> +		/* no cropping */
> +	}
> +
>  	val = coda_read(dev, CODA_RET_DEC_PIC_ERR_MB);
>  	if (val > 0)
>  		v4l2_err(&dev->v4l2_dev,
> 

[PATCH 08/30] [media] coda: add support for frame size enumeration

[Philipp Zabel]

This patch adds support for the VIDIOC_ENUM_FRAMESIZES ioctl.
When decoding H.264, the output frame size is rounded up to the
next multiple of the macroblock size (16 pixels).

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 59 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 7e4df82..b5e5983 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -958,6 +958,63 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 	return 0;
 }
 
+static int coda_enum_framesizes(struct file *file, void *fh,
+				struct v4l2_frmsizeenum *fs)
+{
+	struct coda_ctx *ctx = fh_to_ctx(fh);
+	struct coda_q_data *q_data_src;
+	struct coda_codec *codec;
+	struct vb2_queue *src_vq;
+	int max_w;
+	int max_h;
+	int i;
+
+	if (fs->index > 0)
+		return -EINVAL;
+
+	/*
+	 * If the source format is already fixed, try to find a codec that
+	 * converts to the given destination format
+	 */
+	src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	if (vb2_is_streaming(src_vq)) {
+		q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+
+		codec = coda_find_codec(ctx->dev, q_data_src->fourcc,
+					fs->pixel_format);
+		if (!codec)
+			return -EINVAL;
+
+		fs->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+		if (codec->src_fourcc == V4L2_PIX_FMT_H264) {
+			fs->discrete.width = round_up(q_data_src->width, 16);
+			fs->discrete.height = round_up(q_data_src->height, 16);
+		} else {
+			fs->discrete.width = q_data_src->width;
+			fs->discrete.height = q_data_src->height;
+		}
+	} else {
+		/* We don't know if input or output frame sizes are requested */
+		coda_get_max_dimensions(ctx->dev, NULL, &max_w, &max_h);
+		fs->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		fs->stepwise.min_width = MIN_W;
+		fs->stepwise.max_width = max_w;
+		fs->stepwise.step_width = 1;
+		fs->stepwise.min_height = MIN_H;
+		fs->stepwise.max_height = max_h;
+		fs->stepwise.step_height = 1;
+
+		for (i = 0; i < ARRAY_SIZE(coda_formats); i++) {
+			if (coda_formats[i].fourcc == fs->pixel_format)
+				break;
+		}
+		if (i == ARRAY_SIZE(coda_formats))
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int coda_subscribe_event(struct v4l2_fh *fh,
 				const struct v4l2_event_subscription *sub)
 {
@@ -998,6 +1055,8 @@ static const struct v4l2_ioctl_ops coda_ioctl_ops = {
 	.vidioc_try_decoder_cmd	= coda_try_decoder_cmd,
 	.vidioc_decoder_cmd	= coda_decoder_cmd,
 
+	.vidioc_enum_framesizes	= coda_enum_framesizes,
+
 	.vidioc_subscribe_event = coda_subscribe_event,
 	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
 };
-- 
2.0.0.rc2

Re: [PATCH 08/30] [media] coda: add support for frame size enumeration

[Hans Verkuil]

On 06/13/2014 06:08 PM, Philipp Zabel wrote:
> This patch adds support for the VIDIOC_ENUM_FRAMESIZES ioctl.
> When decoding H.264, the output frame size is rounded up to the
> next multiple of the macroblock size (16 pixels).

Why do you need this? Implementing VIDIOC_ENUM_FRAMESIZES for a m2m device
seems odd.

Regards,

	Hans

> 
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
>  drivers/media/platform/coda.c | 59 +++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 59 insertions(+)
> 
> diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
> index 7e4df82..b5e5983 100644
> --- a/drivers/media/platform/coda.c
> +++ b/drivers/media/platform/coda.c
> @@ -958,6 +958,63 @@ static int coda_decoder_cmd(struct file *file, void *fh,
>  	return 0;
>  }
>  
> +static int coda_enum_framesizes(struct file *file, void *fh,
> +				struct v4l2_frmsizeenum *fs)
> +{
> +	struct coda_ctx *ctx = fh_to_ctx(fh);
> +	struct coda_q_data *q_data_src;
> +	struct coda_codec *codec;
> +	struct vb2_queue *src_vq;
> +	int max_w;
> +	int max_h;
> +	int i;
> +
> +	if (fs->index > 0)
> +		return -EINVAL;
> +
> +	/*
> +	 * If the source format is already fixed, try to find a codec that
> +	 * converts to the given destination format
> +	 */
> +	src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
> +	if (vb2_is_streaming(src_vq)) {
> +		q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
> +
> +		codec = coda_find_codec(ctx->dev, q_data_src->fourcc,
> +					fs->pixel_format);
> +		if (!codec)
> +			return -EINVAL;
> +
> +		fs->type = V4L2_FRMSIZE_TYPE_DISCRETE;
> +		if (codec->src_fourcc == V4L2_PIX_FMT_H264) {
> +			fs->discrete.width = round_up(q_data_src->width, 16);
> +			fs->discrete.height = round_up(q_data_src->height, 16);
> +		} else {
> +			fs->discrete.width = q_data_src->width;
> +			fs->discrete.height = q_data_src->height;
> +		}
> +	} else {
> +		/* We don't know if input or output frame sizes are requested */
> +		coda_get_max_dimensions(ctx->dev, NULL, &max_w, &max_h);
> +		fs->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> +		fs->stepwise.min_width = MIN_W;
> +		fs->stepwise.max_width = max_w;
> +		fs->stepwise.step_width = 1;
> +		fs->stepwise.min_height = MIN_H;
> +		fs->stepwise.max_height = max_h;
> +		fs->stepwise.step_height = 1;
> +
> +		for (i = 0; i < ARRAY_SIZE(coda_formats); i++) {
> +			if (coda_formats[i].fourcc == fs->pixel_format)
> +				break;
> +		}
> +		if (i == ARRAY_SIZE(coda_formats))
> +			return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
>  static int coda_subscribe_event(struct v4l2_fh *fh,
>  				const struct v4l2_event_subscription *sub)
>  {
> @@ -998,6 +1055,8 @@ static const struct v4l2_ioctl_ops coda_ioctl_ops = {
>  	.vidioc_try_decoder_cmd	= coda_try_decoder_cmd,
>  	.vidioc_decoder_cmd	= coda_decoder_cmd,
>  
> +	.vidioc_enum_framesizes	= coda_enum_framesizes,
> +
>  	.vidioc_subscribe_event = coda_subscribe_event,
>  	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>  };
> 

Re: [PATCH 08/30] [media] coda: add support for frame size enumeration

[Philipp Zabel]

Am Montag, den 16.06.2014, 10:08 +0200 schrieb Hans Verkuil:
> On 06/13/2014 06:08 PM, Philipp Zabel wrote:
> > This patch adds support for the VIDIOC_ENUM_FRAMESIZES ioctl.
> > When decoding H.264, the output frame size is rounded up to the
> > next multiple of the macroblock size (16 pixels).
> 
> Why do you need this? Implementing VIDIOC_ENUM_FRAMESIZES for a m2m device
> seems odd.

As soon as the OUTPUT side starts streaming, the possible buffer
dimensions on the CAPTURE side are fixed, but not necessarily the same
as on the OUTPUT side. Since GStreamer checks ENUM_FRAMESIZES before
trying to find the possible min/max dimensions via TRY_FMT, I just
implemented that.
Right now I could (and probably should) also implement this using just
TRY_FMT, but the JPEG decoder will also support decoding to half, 1/4
and 1/8 size, and I'd like to report that to userspace.

I'll postpone this patch for now.

regards
Philipp

[PATCH 09/30] [media] coda: add workqueue to serialize hardware commands

[Philipp Zabel]

Using the coda_mutex lock to serialize hardware access would cause
"INFO: possible circular locking dependency detected" lockdep warnings.
Since the possible locking paths are hard to follow, serialize hardware
access with a single workqueue thread. Ultimately the workqueue could
be converted to only do register setup and readout for per-command work
items.
Using the initialized context property, SEQ_END is only queued in
coda_release when needed.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 162 +++++++++++++++++++-----------------------
 1 file changed, 74 insertions(+), 88 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index b5e5983..fe94acd 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -147,11 +147,11 @@ struct coda_dev {
 	spinlock_t		irqlock;
 	struct mutex		dev_mutex;
 	struct mutex		coda_mutex;
+	struct workqueue_struct	*workqueue;
 	struct v4l2_m2m_dev	*m2m_dev;
 	struct vb2_alloc_ctx	*alloc_ctx;
 	struct list_head	instances;
 	unsigned long		instance_mask;
-	struct delayed_work	timeout;
 };
 
 struct coda_params {
@@ -198,7 +198,9 @@ struct coda_ctx {
 	struct coda_dev			*dev;
 	struct mutex			buffer_mutex;
 	struct list_head		list;
-	struct work_struct		skip_run;
+	struct work_struct		pic_run_work;
+	struct work_struct		seq_end_work;
+	struct completion		completion;
 	int				aborting;
 	int				initialized;
 	int				streamon_out;
@@ -1063,13 +1065,6 @@ static const struct v4l2_ioctl_ops coda_ioctl_ops = {
 
 static int coda_start_decoding(struct coda_ctx *ctx);
 
-static void coda_skip_run(struct work_struct *work)
-{
-	struct coda_ctx *ctx = container_of(work, struct coda_ctx, skip_run);
-
-	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
-}
-
 static inline int coda_get_bitstream_payload(struct coda_ctx *ctx)
 {
 	return kfifo_len(&ctx->bitstream_fifo);
@@ -1224,7 +1219,7 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 		v4l2_dbg(1, coda_debug, &dev->v4l2_dev,
 			 "bitstream payload: %d, skipping\n",
 			 coda_get_bitstream_payload(ctx));
-		schedule_work(&ctx->skip_run);
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
 		return -EAGAIN;
 	}
 
@@ -1233,7 +1228,7 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 		int ret = coda_start_decoding(ctx);
 		if (ret < 0) {
 			v4l2_err(&dev->v4l2_dev, "failed to start decoding\n");
-			schedule_work(&ctx->skip_run);
+			v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
 			return -EAGAIN;
 		} else {
 			ctx->initialized = 1;
@@ -1441,24 +1436,48 @@ static void coda_device_run(void *m2m_priv)
 {
 	struct coda_ctx *ctx = m2m_priv;
 	struct coda_dev *dev = ctx->dev;
-	int ret;
+
+	queue_work(dev->workqueue, &ctx->pic_run_work);
+}
+
+static void coda_free_framebuffers(struct coda_ctx *ctx);
+static void coda_free_context_buffers(struct coda_ctx *ctx);
+
+static void coda_seq_end_work(struct work_struct *work)
+{
+	struct coda_ctx *ctx = container_of(work, struct coda_ctx, seq_end_work);
+	struct coda_dev *dev = ctx->dev;
 
 	mutex_lock(&ctx->buffer_mutex);
+	mutex_lock(&dev->coda_mutex);
 
-	/*
-	 * If streamoff dequeued all buffers before we could get the lock,
-	 * just bail out immediately.
-	 */
-	if ((!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx) &&
-	    ctx->inst_type != CODA_INST_DECODER) ||
-		!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
-		v4l2_dbg(1, coda_debug, &dev->v4l2_dev,
-			"%d: device_run without buffers\n", ctx->idx);
-		mutex_unlock(&ctx->buffer_mutex);
-		schedule_work(&ctx->skip_run);
-		return;
+	v4l2_dbg(1, coda_debug, &dev->v4l2_dev,
+		 "%d: %s: sent command 'SEQ_END' to coda\n", ctx->idx, __func__);
+	if (coda_command_sync(ctx, CODA_COMMAND_SEQ_END)) {
+		v4l2_err(&dev->v4l2_dev,
+			 "CODA_COMMAND_SEQ_END failed\n");
 	}
 
+	kfifo_init(&ctx->bitstream_fifo,
+		ctx->bitstream.vaddr, ctx->bitstream.size);
+
+	coda_free_framebuffers(ctx);
+	coda_free_context_buffers(ctx);
+
+	mutex_unlock(&dev->coda_mutex);
+	mutex_unlock(&ctx->buffer_mutex);
+}
+
+static void coda_finish_decode(struct coda_ctx *ctx);
+static void coda_finish_encode(struct coda_ctx *ctx);
+
+static void coda_pic_run_work(struct work_struct *work)
+{
+	struct coda_ctx *ctx = container_of(work, struct coda_ctx, pic_run_work);
+	struct coda_dev *dev = ctx->dev;
+	int ret;
+
+	mutex_lock(&ctx->buffer_mutex);
 	mutex_lock(&dev->coda_mutex);
 
 	if (ctx->inst_type == CODA_INST_DECODER) {
@@ -1477,12 +1496,26 @@ static void coda_device_run(void *m2m_priv)
 		coda_write(dev, ctx->iram_info.axi_sram_use,
 				CODA7_REG_BIT_AXI_SRAM_USE);
 
-	/* 1 second timeout in case CODA locks up */
-	schedule_delayed_work(&dev->timeout, HZ);
-
 	if (ctx->inst_type == CODA_INST_DECODER)
 		coda_kfifo_sync_to_device_full(ctx);
 	coda_command_async(ctx, CODA_COMMAND_PIC_RUN);
+
+	if (!wait_for_completion_timeout(&ctx->completion, msecs_to_jiffies(1000))) {
+		dev_err(&dev->plat_dev->dev, "CODA PIC_RUN timeout\n");
+	} else if (!ctx->aborting) {
+		if (ctx->inst_type == CODA_INST_DECODER)
+			coda_finish_decode(ctx);
+		else
+			coda_finish_encode(ctx);
+	}
+
+	if (ctx->aborting || (!ctx->streamon_cap && !ctx->streamon_out))
+		queue_work(dev->workqueue, &ctx->seq_end_work);
+
+	mutex_unlock(&dev->coda_mutex);
+	mutex_unlock(&ctx->buffer_mutex);
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
 }
 
 static int coda_job_ready(void *m2m_priv)
@@ -2839,7 +2872,9 @@ static int coda_open(struct file *file)
 	}
 	set_bit(idx, &dev->instance_mask);
 
-	INIT_WORK(&ctx->skip_run, coda_skip_run);
+	init_completion(&ctx->completion);
+	INIT_WORK(&ctx->pic_run_work, coda_pic_run_work);
+	INIT_WORK(&ctx->seq_end_work, coda_seq_end_work);
 	v4l2_fh_init(&ctx->fh, video_devdata(file));
 	file->private_data = &ctx->fh;
 	v4l2_fh_add(&ctx->fh);
@@ -2942,16 +2977,10 @@ static int coda_release(struct file *file)
 	v4l2_m2m_ctx_release(ctx->m2m_ctx);
 
 	/* In case the instance was not running, we still need to call SEQ_END */
-	mutex_lock(&dev->coda_mutex);
-	v4l2_dbg(1, coda_debug, &dev->v4l2_dev,
-		 "%s: sent command 'SEQ_END' to coda\n", __func__);
-	if (coda_command_sync(ctx, CODA_COMMAND_SEQ_END)) {
-		v4l2_err(&dev->v4l2_dev,
-			 "CODA_COMMAND_SEQ_END failed\n");
-		mutex_unlock(&dev->coda_mutex);
-		return -ETIMEDOUT;
+	if (ctx->initialized) {
+		queue_work(dev->workqueue, &ctx->seq_end_work);
+		flush_work(&ctx->seq_end_work);
 	}
-	mutex_unlock(&dev->coda_mutex);
 
 	coda_free_framebuffers(ctx);
 
@@ -3248,8 +3277,6 @@ static irqreturn_t coda_irq_handler(int irq, void *data)
 	struct coda_dev *dev = data;
 	struct coda_ctx *ctx;
 
-	cancel_delayed_work(&dev->timeout);
-
 	/* read status register to attend the IRQ */
 	coda_read(dev, CODA_REG_BIT_INT_STATUS);
 	coda_write(dev, CODA_REG_BIT_INT_CLEAR_SET,
@@ -3265,7 +3292,6 @@ static irqreturn_t coda_irq_handler(int irq, void *data)
 	if (ctx->aborting) {
 		v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev,
 			 "task has been aborted\n");
-		goto out;
 	}
 
 	if (coda_isbusy(ctx->dev)) {
@@ -3274,57 +3300,11 @@ static irqreturn_t coda_irq_handler(int irq, void *data)
 		return IRQ_NONE;
 	}
 
-	if (ctx->inst_type == CODA_INST_DECODER)
-		coda_finish_decode(ctx);
-	else
-		coda_finish_encode(ctx);
-
-out:
-	if (ctx->aborting || (!ctx->streamon_cap && !ctx->streamon_out)) {
-		v4l2_dbg(1, coda_debug, &dev->v4l2_dev,
-			 "%s: sent command 'SEQ_END' to coda\n", __func__);
-		if (coda_command_sync(ctx, CODA_COMMAND_SEQ_END)) {
-			v4l2_err(&dev->v4l2_dev,
-				 "CODA_COMMAND_SEQ_END failed\n");
-		}
-
-		kfifo_init(&ctx->bitstream_fifo,
-			ctx->bitstream.vaddr, ctx->bitstream.size);
-
-		coda_free_framebuffers(ctx);
-		coda_free_context_buffers(ctx);
-	}
-
-	mutex_unlock(&dev->coda_mutex);
-	mutex_unlock(&ctx->buffer_mutex);
-
-	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
+	complete(&ctx->completion);
 
 	return IRQ_HANDLED;
 }
 
-static void coda_timeout(struct work_struct *work)
-{
-	struct coda_ctx *ctx;
-	struct coda_dev *dev = container_of(to_delayed_work(work),
-					    struct coda_dev, timeout);
-
-	dev_err(&dev->plat_dev->dev, "CODA PIC_RUN timeout, stopping all streams\n");
-
-	mutex_lock(&dev->dev_mutex);
-	list_for_each_entry(ctx, &dev->instances, list) {
-		if (mutex_is_locked(&ctx->buffer_mutex))
-			mutex_unlock(&ctx->buffer_mutex);
-		v4l2_m2m_streamoff(NULL, ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
-		v4l2_m2m_streamoff(NULL, ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
-	}
-	mutex_unlock(&dev->dev_mutex);
-
-	mutex_unlock(&dev->coda_mutex);
-	ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
-	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
-}
-
 static u32 coda_supported_firmwares[] = {
 	CODA_FIRMWARE_VERNUM(CODA_DX6, 2, 2, 5),
 	CODA_FIRMWARE_VERNUM(CODA_7541, 1, 4, 50),
@@ -3628,7 +3608,6 @@ static int coda_probe(struct platform_device *pdev)
 
 	spin_lock_init(&dev->irqlock);
 	INIT_LIST_HEAD(&dev->instances);
-	INIT_DELAYED_WORK(&dev->timeout, coda_timeout);
 
 	dev->plat_dev = pdev;
 	dev->clk_per = devm_clk_get(&pdev->dev, "per");
@@ -3735,6 +3714,12 @@ static int coda_probe(struct platform_device *pdev)
 		return -ENOMEM;
 	}
 
+	dev->workqueue = alloc_workqueue("coda", WQ_UNBOUND | WQ_MEM_RECLAIM, 1);
+	if (!dev->workqueue) {
+		dev_err(&pdev->dev, "unable to alloc workqueue\n");
+		return -ENOMEM;
+	}
+
 	platform_set_drvdata(pdev, dev);
 
 	return coda_firmware_request(dev);
@@ -3750,6 +3735,7 @@ static int coda_remove(struct platform_device *pdev)
 	if (dev->alloc_ctx)
 		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
 	v4l2_device_unregister(&dev->v4l2_dev);
+	destroy_workqueue(dev->workqueue);
 	if (dev->iram.vaddr)
 		gen_pool_free(dev->iram_pool, (unsigned long)dev->iram.vaddr,
 			      dev->iram.size);
-- 
2.0.0.rc2

[PATCH 10/30] [media] coda: Use mem-to-mem ioctl helpers

[Philipp Zabel]

Use the mem2mem helpers introduced to get rid of some duplicated code.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 113 ++++++------------------------------------
 1 file changed, 14 insertions(+), 99 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index fe94acd..52a429f 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -782,22 +782,6 @@ static int coda_s_fmt_vid_out(struct file *file, void *priv,
 	return ret;
 }
 
-static int coda_reqbufs(struct file *file, void *priv,
-			struct v4l2_requestbuffers *reqbufs)
-{
-	struct coda_ctx *ctx = fh_to_ctx(priv);
-
-	return v4l2_m2m_reqbufs(file, ctx->m2m_ctx, reqbufs);
-}
-
-static int coda_querybuf(struct file *file, void *priv,
-			 struct v4l2_buffer *buf)
-{
-	struct coda_ctx *ctx = fh_to_ctx(priv);
-
-	return v4l2_m2m_querybuf(file, ctx->m2m_ctx, buf);
-}
-
 static int coda_qbuf(struct file *file, void *priv,
 		     struct v4l2_buffer *buf)
 {
@@ -806,14 +790,6 @@ static int coda_qbuf(struct file *file, void *priv,
 	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
 }
 
-static int coda_expbuf(struct file *file, void *priv,
-		       struct v4l2_exportbuffer *eb)
-{
-	struct coda_ctx *ctx = fh_to_ctx(priv);
-
-	return v4l2_m2m_expbuf(file, ctx->m2m_ctx, eb);
-}
-
 static bool coda_buf_is_end_of_stream(struct coda_ctx *ctx,
 				      struct v4l2_buffer *buf)
 {
@@ -846,40 +822,6 @@ static int coda_dqbuf(struct file *file, void *priv,
 	return ret;
 }
 
-static int coda_create_bufs(struct file *file, void *priv,
-			    struct v4l2_create_buffers *create)
-{
-	struct coda_ctx *ctx = fh_to_ctx(priv);
-
-	return v4l2_m2m_create_bufs(file, ctx->m2m_ctx, create);
-}
-
-static int coda_streamon(struct file *file, void *priv,
-			 enum v4l2_buf_type type)
-{
-	struct coda_ctx *ctx = fh_to_ctx(priv);
-
-	return v4l2_m2m_streamon(file, ctx->m2m_ctx, type);
-}
-
-static int coda_streamoff(struct file *file, void *priv,
-			  enum v4l2_buf_type type)
-{
-	struct coda_ctx *ctx = fh_to_ctx(priv);
-	int ret;
-
-	/*
-	 * This indirectly calls __vb2_queue_cancel, which dequeues all buffers.
-	 * We therefore have to lock it against running hardware in this context,
-	 * which still needs the buffers.
-	 */
-	mutex_lock(&ctx->buffer_mutex);
-	ret = v4l2_m2m_streamoff(file, ctx->m2m_ctx, type);
-	mutex_unlock(&ctx->buffer_mutex);
-
-	return ret;
-}
-
 static int coda_g_selection(struct file *file, void *fh,
 			    struct v4l2_selection *s)
 {
@@ -1041,16 +983,16 @@ static const struct v4l2_ioctl_ops coda_ioctl_ops = {
 	.vidioc_try_fmt_vid_out	= coda_try_fmt_vid_out,
 	.vidioc_s_fmt_vid_out	= coda_s_fmt_vid_out,
 
-	.vidioc_reqbufs		= coda_reqbufs,
-	.vidioc_querybuf	= coda_querybuf,
+	.vidioc_reqbufs		= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf	= v4l2_m2m_ioctl_querybuf,
 
 	.vidioc_qbuf		= coda_qbuf,
-	.vidioc_expbuf		= coda_expbuf,
+	.vidioc_expbuf		= v4l2_m2m_ioctl_expbuf,
 	.vidioc_dqbuf		= coda_dqbuf,
-	.vidioc_create_bufs	= coda_create_bufs,
+	.vidioc_create_bufs	= v4l2_m2m_ioctl_create_bufs,
 
-	.vidioc_streamon	= coda_streamon,
-	.vidioc_streamoff	= coda_streamoff,
+	.vidioc_streamon	= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff	= v4l2_m2m_ioctl_streamoff,
 
 	.vidioc_g_selection	= coda_g_selection,
 
@@ -1731,18 +1673,6 @@ static void coda_buf_queue(struct vb2_buffer *vb)
 	}
 }
 
-static void coda_wait_prepare(struct vb2_queue *q)
-{
-	struct coda_ctx *ctx = vb2_get_drv_priv(q);
-	coda_unlock(ctx);
-}
-
-static void coda_wait_finish(struct vb2_queue *q)
-{
-	struct coda_ctx *ctx = vb2_get_drv_priv(q);
-	coda_lock(ctx);
-}
-
 static void coda_parabuf_write(struct coda_ctx *ctx, int index, u32 value)
 {
 	struct coda_dev *dev = ctx->dev;
@@ -2701,10 +2631,10 @@ static struct vb2_ops coda_qops = {
 	.queue_setup		= coda_queue_setup,
 	.buf_prepare		= coda_buf_prepare,
 	.buf_queue		= coda_buf_queue,
-	.wait_prepare		= coda_wait_prepare,
-	.wait_finish		= coda_wait_finish,
 	.start_streaming	= coda_start_streaming,
 	.stop_streaming		= coda_stop_streaming,
+	.wait_prepare		= vb2_ops_wait_prepare,
+	.wait_finish		= vb2_ops_wait_finish,
 };
 
 static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
@@ -2827,6 +2757,7 @@ static int coda_queue_init(void *priv, struct vb2_queue *src_vq,
 	src_vq->ops = &coda_qops;
 	src_vq->mem_ops = &vb2_dma_contig_memops;
 	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->dev->dev_mutex;
 
 	ret = vb2_queue_init(src_vq);
 	if (ret)
@@ -2839,6 +2770,7 @@ static int coda_queue_init(void *priv, struct vb2_queue *src_vq,
 	dst_vq->ops = &coda_qops;
 	dst_vq->mem_ops = &vb2_dma_contig_memops;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->dev->dev_mutex;
 
 	return vb2_queue_init(dst_vq);
 }
@@ -2907,6 +2839,8 @@ static int coda_open(struct file *file)
 			 __func__, ret);
 		goto err_ctx_init;
 	}
+	ctx->fh.m2m_ctx = ctx->m2m_ctx;
+
 	ret = coda_ctrls_setup(ctx);
 	if (ret) {
 		v4l2_err(&dev->v4l2_dev, "failed to setup coda controls\n");
@@ -3006,32 +2940,13 @@ static int coda_release(struct file *file)
 	return 0;
 }
 
-static unsigned int coda_poll(struct file *file,
-				 struct poll_table_struct *wait)
-{
-	struct coda_ctx *ctx = fh_to_ctx(file->private_data);
-	int ret;
-
-	coda_lock(ctx);
-	ret = v4l2_m2m_poll(file, ctx->m2m_ctx, wait);
-	coda_unlock(ctx);
-	return ret;
-}
-
-static int coda_mmap(struct file *file, struct vm_area_struct *vma)
-{
-	struct coda_ctx *ctx = fh_to_ctx(file->private_data);
-
-	return v4l2_m2m_mmap(file, ctx->m2m_ctx, vma);
-}
-
 static const struct v4l2_file_operations coda_fops = {
 	.owner		= THIS_MODULE,
 	.open		= coda_open,
 	.release	= coda_release,
-	.poll		= coda_poll,
+	.poll		= v4l2_m2m_fop_poll,
 	.unlocked_ioctl	= video_ioctl2,
-	.mmap		= coda_mmap,
+	.mmap		= v4l2_m2m_fop_mmap,
 };
 
 static void coda_finish_decode(struct coda_ctx *ctx)
-- 
2.0.0.rc2

[PATCH 11/30] [media] coda: use ctx->fh.m2m_ctx instead of ctx->m2m_ctx

[Philipp Zabel]

v4l2_fh already contains a mem2mem context pointer. Use it.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 70 +++++++++++++++++++++----------------------
 1 file changed, 34 insertions(+), 36 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 52a429f..8321243 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -213,7 +213,6 @@ struct coda_ctx {
 	struct coda_codec		*codec;
 	enum v4l2_colorspace		colorspace;
 	struct coda_params		params;
-	struct v4l2_m2m_ctx		*m2m_ctx;
 	struct v4l2_ctrl_handler	ctrls;
 	struct v4l2_fh			fh;
 	int				gopcounter;
@@ -554,7 +553,7 @@ static int coda_enum_fmt_vid_cap(struct file *file, void *priv,
 	struct coda_q_data *q_data_src;
 
 	/* If the source format is already fixed, only list matching formats */
-	src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	src_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
 	if (vb2_is_streaming(src_vq)) {
 		q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
 
@@ -578,7 +577,7 @@ static int coda_g_fmt(struct file *file, void *priv,
 	struct coda_q_data *q_data;
 	struct coda_ctx *ctx = fh_to_ctx(priv);
 
-	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
 	if (!vq)
 		return -EINVAL;
 
@@ -669,7 +668,7 @@ static int coda_try_fmt_vid_cap(struct file *file, void *priv,
 	 * If the source format is already fixed, try to find a codec that
 	 * converts to the given destination format
 	 */
-	src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	src_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
 	if (vb2_is_streaming(src_vq)) {
 		struct coda_q_data *q_data_src;
 
@@ -723,7 +722,7 @@ static int coda_s_fmt(struct coda_ctx *ctx, struct v4l2_format *f)
 	struct coda_q_data *q_data;
 	struct vb2_queue *vq;
 
-	vq = v4l2_m2m_get_vq(ctx->m2m_ctx, f->type);
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
 	if (!vq)
 		return -EINVAL;
 
@@ -787,7 +786,7 @@ static int coda_qbuf(struct file *file, void *priv,
 {
 	struct coda_ctx *ctx = fh_to_ctx(priv);
 
-	return v4l2_m2m_qbuf(file, ctx->m2m_ctx, buf);
+	return v4l2_m2m_qbuf(file, ctx->fh.m2m_ctx, buf);
 }
 
 static bool coda_buf_is_end_of_stream(struct coda_ctx *ctx,
@@ -795,7 +794,7 @@ static bool coda_buf_is_end_of_stream(struct coda_ctx *ctx,
 {
 	struct vb2_queue *src_vq;
 
-	src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	src_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
 
 	return ((ctx->bit_stream_param & CODA_BIT_STREAM_END_FLAG) &&
 		(buf->sequence == (ctx->qsequence - 1)));
@@ -807,7 +806,7 @@ static int coda_dqbuf(struct file *file, void *priv,
 	struct coda_ctx *ctx = fh_to_ctx(priv);
 	int ret;
 
-	ret = v4l2_m2m_dqbuf(file, ctx->m2m_ctx, buf);
+	ret = v4l2_m2m_dqbuf(file, ctx->fh.m2m_ctx, buf);
 
 	/* If this is the last capture buffer, emit an end-of-stream event */
 	if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE &&
@@ -920,7 +919,7 @@ static int coda_enum_framesizes(struct file *file, void *fh,
 	 * If the source format is already fixed, try to find a codec that
 	 * converts to the given destination format
 	 */
-	src_vq = v4l2_m2m_get_vq(ctx->m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	src_vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
 	if (vb2_is_streaming(src_vq)) {
 		q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
 
@@ -1096,11 +1095,11 @@ static void coda_fill_bitstream(struct coda_ctx *ctx)
 {
 	struct vb2_buffer *src_buf;
 
-	while (v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx) > 0) {
-		src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
+	while (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) > 0) {
+		src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 
 		if (coda_bitstream_try_queue(ctx, src_buf)) {
-			src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
+			src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
 			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
 		} else {
 			break;
@@ -1140,7 +1139,7 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 	u32 stridey, height;
 	u32 picture_y, picture_cb, picture_cr;
 
-	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
 
 	if (ctx->params.rot_mode & CODA_ROT_90) {
@@ -1161,7 +1160,7 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 		v4l2_dbg(1, coda_debug, &dev->v4l2_dev,
 			 "bitstream payload: %d, skipping\n",
 			 coda_get_bitstream_payload(ctx));
-		v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
 		return -EAGAIN;
 	}
 
@@ -1170,7 +1169,7 @@ static int coda_prepare_decode(struct coda_ctx *ctx)
 		int ret = coda_start_decoding(ctx);
 		if (ret < 0) {
 			v4l2_err(&dev->v4l2_dev, "failed to start decoding\n");
-			v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
+			v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
 			return -EAGAIN;
 		} else {
 			ctx->initialized = 1;
@@ -1243,8 +1242,8 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
 	u32 pic_stream_buffer_addr, pic_stream_buffer_size;
 	u32 dst_fourcc;
 
-	src_buf = v4l2_m2m_next_src_buf(ctx->m2m_ctx);
-	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 	q_data_src = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
 	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
 	dst_fourcc = q_data_dst->fourcc;
@@ -1457,7 +1456,7 @@ static void coda_pic_run_work(struct work_struct *work)
 	mutex_unlock(&dev->coda_mutex);
 	mutex_unlock(&ctx->buffer_mutex);
 
-	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->m2m_ctx);
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
 }
 
 static int coda_job_ready(void *m2m_priv)
@@ -1469,14 +1468,14 @@ static int coda_job_ready(void *m2m_priv)
 	 * and 1 frame are needed. In the decoder case,
 	 * the compressed frame can be in the bitstream.
 	 */
-	if (!v4l2_m2m_num_src_bufs_ready(ctx->m2m_ctx) &&
+	if (!v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) &&
 	    ctx->inst_type != CODA_INST_DECODER) {
 		v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev,
 			 "not ready: not enough video buffers.\n");
 		return 0;
 	}
 
-	if (!v4l2_m2m_num_dst_bufs_ready(ctx->m2m_ctx)) {
+	if (!v4l2_m2m_num_dst_bufs_ready(ctx->fh.m2m_ctx)) {
 		v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev,
 			 "not ready: not enough video capture buffers.\n");
 		return 0;
@@ -1665,11 +1664,11 @@ static void coda_buf_queue(struct vb2_buffer *vb)
 			}
 		}
 		mutex_lock(&ctx->bitstream_mutex);
-		v4l2_m2m_buf_queue(ctx->m2m_ctx, vb);
+		v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vb);
 		coda_fill_bitstream(ctx);
 		mutex_unlock(&ctx->bitstream_mutex);
 	} else {
-		v4l2_m2m_buf_queue(ctx->m2m_ctx, vb);
+		v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vb);
 	}
 }
 
@@ -2264,7 +2263,7 @@ static int coda_start_streaming(struct vb2_queue *q, unsigned int count)
 
 	/* Allow decoder device_run with no new buffers queued */
 	if (ctx->inst_type == CODA_INST_DECODER)
-		v4l2_m2m_set_src_buffered(ctx->m2m_ctx, true);
+		v4l2_m2m_set_src_buffered(ctx->fh.m2m_ctx, true);
 
 	ctx->gopcounter = ctx->params.gop_size - 1;
 	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
@@ -2312,7 +2311,7 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 	q_data_dst = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
 	dst_fourcc = q_data_dst->fourcc;
 
-	buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 	bitstream_buf = vb2_dma_contig_plane_dma_addr(buf, 0);
 	bitstream_size = q_data_dst->sizeimage;
 
@@ -2523,7 +2522,7 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 	}
 
 	/* Save stream headers */
-	buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 	switch (dst_fourcc) {
 	case V4L2_PIX_FMT_H264:
 		/*
@@ -2830,16 +2829,15 @@ static int coda_open(struct file *file)
 		goto err_clk_ahb;
 
 	set_default_params(ctx);
-	ctx->m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx,
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx,
 					 &coda_queue_init);
-	if (IS_ERR(ctx->m2m_ctx)) {
-		ret = PTR_ERR(ctx->m2m_ctx);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		ret = PTR_ERR(ctx->fh.m2m_ctx);
 
 		v4l2_err(&dev->v4l2_dev, "%s return error (%d)\n",
 			 __func__, ret);
 		goto err_ctx_init;
 	}
-	ctx->fh.m2m_ctx = ctx->m2m_ctx;
 
 	ret = coda_ctrls_setup(ctx);
 	if (ret) {
@@ -2885,7 +2883,7 @@ err_dma_writecombine:
 err_dma_alloc:
 	v4l2_ctrl_handler_free(&ctx->ctrls);
 err_ctrls_setup:
-	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
 err_ctx_init:
 	clk_disable_unprepare(dev->clk_ahb);
 err_clk_ahb:
@@ -2908,7 +2906,7 @@ static int coda_release(struct file *file)
 		 ctx);
 
 	/* If this instance is running, call .job_abort and wait for it to end */
-	v4l2_m2m_ctx_release(ctx->m2m_ctx);
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
 
 	/* In case the instance was not running, we still need to call SEQ_END */
 	if (ctx->initialized) {
@@ -2962,7 +2960,7 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 	int success;
 	u32 val;
 
-	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 
 	/* Update kfifo out pointer from coda bitstream read pointer */
 	coda_kfifo_sync_from_device(ctx);
@@ -3102,7 +3100,7 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 	/* If a frame was copied out, return it */
 	if (ctx->display_idx >= 0 &&
 	    ctx->display_idx < ctx->num_internal_frames) {
-		dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+		dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 		dst_buf->v4l2_buf.sequence = ctx->osequence++;
 
 		dst_buf->v4l2_buf.flags &= ~(V4L2_BUF_FLAG_KEYFRAME |
@@ -3134,8 +3132,8 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 	struct coda_dev *dev = ctx->dev;
 	u32 wr_ptr, start_ptr;
 
-	src_buf = v4l2_m2m_src_buf_remove(ctx->m2m_ctx);
-	dst_buf = v4l2_m2m_next_dst_buf(ctx->m2m_ctx);
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 
 	/* Get results from the coda */
 	start_ptr = coda_read(dev, CODA_CMD_ENC_PIC_BB_START);
@@ -3173,7 +3171,7 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 
 	v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
 
-	dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx);
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 	v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
 
 	ctx->gopcounter--;
-- 
2.0.0.rc2

[PATCH 12/30] [media] coda: Add runtime pm support

[Philipp Zabel]

This patch allows to use the runtime pm and generic pm domain frameworks
to completely gate power to the VPU if it is unused. This functionality
is available on i.MX6.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 65 +++++++++++++++++++++++++++++++++++++++----
 1 file changed, 60 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 8321243..f39f693 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -22,6 +22,7 @@
 #include <linux/module.h>
 #include <linux/of_device.h>
 #include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
 #include <linux/slab.h>
 #include <linux/videodev2.h>
 #include <linux/of.h>
@@ -2820,6 +2821,13 @@ static int coda_open(struct file *file)
 		ctx->reg_idx = idx;
 	}
 
+	/* Power up and upload firmware if necessary */
+	ret = pm_runtime_get_sync(&dev->plat_dev->dev);
+	if (ret < 0) {
+		v4l2_err(&dev->v4l2_dev, "failed to power up: %d\n", ret);
+		goto err_pm_get;
+	}
+
 	ret = clk_prepare_enable(dev->clk_per);
 	if (ret)
 		goto err_clk_per;
@@ -2889,6 +2897,8 @@ err_ctx_init:
 err_clk_ahb:
 	clk_disable_unprepare(dev->clk_per);
 err_clk_per:
+	pm_runtime_put_sync(&dev->plat_dev->dev);
+err_pm_get:
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	clear_bit(ctx->idx, &dev->instance_mask);
@@ -2930,6 +2940,7 @@ static int coda_release(struct file *file)
 	v4l2_ctrl_handler_free(&ctx->ctrls);
 	clk_disable_unprepare(dev->clk_ahb);
 	clk_disable_unprepare(dev->clk_per);
+	pm_runtime_put_sync(&dev->plat_dev->dev);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	clear_bit(ctx->idx, &dev->instance_mask);
@@ -3243,7 +3254,7 @@ static int coda_hw_init(struct coda_dev *dev)
 
 	ret = clk_prepare_enable(dev->clk_per);
 	if (ret)
-		return ret;
+		goto err_clk_per;
 
 	ret = clk_prepare_enable(dev->clk_ahb);
 	if (ret)
@@ -3369,6 +3380,7 @@ static int coda_hw_init(struct coda_dev *dev)
 
 err_clk_ahb:
 	clk_disable_unprepare(dev->clk_per);
+err_clk_per:
 	return ret;
 }
 
@@ -3394,10 +3406,29 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
 	memcpy(dev->codebuf.vaddr, fw->data, fw->size);
 	release_firmware(fw);
 
-	ret = coda_hw_init(dev);
-	if (ret) {
-		v4l2_err(&dev->v4l2_dev, "HW initialization failed\n");
-		return;
+	if (IS_ENABLED(CONFIG_PM_RUNTIME) && pdev->dev.pm_domain) {
+		/*
+		 * Enabling power temporarily will cause coda_hw_init to be
+		 * called via coda_runtime_resume by the pm domain.
+		 */
+		ret = pm_runtime_get_sync(&dev->plat_dev->dev);
+		if (ret < 0) {
+			v4l2_err(&dev->v4l2_dev, "failed to power on: %d\n",
+				 ret);
+			return;
+		}
+
+		pm_runtime_put_sync(&dev->plat_dev->dev);
+	} else {
+		/*
+		 * If runtime pm is disabled or pm_domain is not set,
+		 * initialize once manually.
+		 */
+		ret = coda_hw_init(dev);
+		if (ret < 0) {
+			v4l2_err(&dev->v4l2_dev, "HW initialization failed\n");
+			return;
+		}
 	}
 
 	dev->vfd.fops	= &coda_fops,
@@ -3635,6 +3666,8 @@ static int coda_probe(struct platform_device *pdev)
 
 	platform_set_drvdata(pdev, dev);
 
+	pm_runtime_enable(&pdev->dev);
+
 	return coda_firmware_request(dev);
 }
 
@@ -3645,6 +3678,7 @@ static int coda_remove(struct platform_device *pdev)
 	video_unregister_device(&dev->vfd);
 	if (dev->m2m_dev)
 		v4l2_m2m_release(dev->m2m_dev);
+	pm_runtime_disable(&pdev->dev);
 	if (dev->alloc_ctx)
 		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
 	v4l2_device_unregister(&dev->v4l2_dev);
@@ -3658,6 +3692,26 @@ static int coda_remove(struct platform_device *pdev)
 	return 0;
 }
 
+#ifdef CONFIG_PM_RUNTIME
+static int coda_runtime_resume(struct device *dev)
+{
+	struct coda_dev *cdev = dev_get_drvdata(dev);
+	int ret = 0;
+
+	if (dev->pm_domain) {
+		ret = coda_hw_init(cdev);
+		if (ret)
+			v4l2_err(&cdev->v4l2_dev, "HW initialization failed\n");
+	}
+
+	return ret;
+}
+#endif
+
+static const struct dev_pm_ops coda_pm_ops = {
+	SET_RUNTIME_PM_OPS(NULL, coda_runtime_resume, NULL)
+};
+
 static struct platform_driver coda_driver = {
 	.probe	= coda_probe,
 	.remove	= coda_remove,
@@ -3665,6 +3719,7 @@ static struct platform_driver coda_driver = {
 		.name	= CODA_NAME,
 		.owner	= THIS_MODULE,
 		.of_match_table = of_match_ptr(coda_dt_ids),
+		.pm	= &coda_pm_ops,
 	},
 	.id_table = coda_platform_ids,
 };
-- 
2.0.0.rc2

Re: [PATCH 12/30] [media] coda: Add runtime pm support

[Sylwester Nawrocki]

Hello Philipp,

On 13/06/14 18:08, Philipp Zabel wrote:
> This patch allows to use the runtime pm and generic pm domain frameworks
> to completely gate power to the VPU if it is unused. This functionality
> is available on i.MX6.
> 
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
>  drivers/media/platform/coda.c | 65 +++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 60 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
> index 8321243..f39f693 100644
> --- a/drivers/media/platform/coda.c
> +++ b/drivers/media/platform/coda.c
> @@ -22,6 +22,7 @@
>  #include <linux/module.h>
>  #include <linux/of_device.h>
>  #include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
>  #include <linux/slab.h>
>  #include <linux/videodev2.h>
>  #include <linux/of.h>
> @@ -2820,6 +2821,13 @@ static int coda_open(struct file *file)
>  		ctx->reg_idx = idx;
>  	}
>  
> +	/* Power up and upload firmware if necessary */
> +	ret = pm_runtime_get_sync(&dev->plat_dev->dev);
> +	if (ret < 0) {
> +		v4l2_err(&dev->v4l2_dev, "failed to power up: %d\n", ret);
> +		goto err_pm_get;
> +	}
> +
>  	ret = clk_prepare_enable(dev->clk_per);
>  	if (ret)
>  		goto err_clk_per;
> @@ -2889,6 +2897,8 @@ err_ctx_init:
>  err_clk_ahb:
>  	clk_disable_unprepare(dev->clk_per);
>  err_clk_per:
> +	pm_runtime_put_sync(&dev->plat_dev->dev);
> +err_pm_get:
>  	v4l2_fh_del(&ctx->fh);
>  	v4l2_fh_exit(&ctx->fh);
>  	clear_bit(ctx->idx, &dev->instance_mask);
> @@ -2930,6 +2940,7 @@ static int coda_release(struct file *file)
>  	v4l2_ctrl_handler_free(&ctx->ctrls);
>  	clk_disable_unprepare(dev->clk_ahb);
>  	clk_disable_unprepare(dev->clk_per);
> +	pm_runtime_put_sync(&dev->plat_dev->dev);
>  	v4l2_fh_del(&ctx->fh);
>  	v4l2_fh_exit(&ctx->fh);
>  	clear_bit(ctx->idx, &dev->instance_mask);
> @@ -3243,7 +3254,7 @@ static int coda_hw_init(struct coda_dev *dev)
>  
>  	ret = clk_prepare_enable(dev->clk_per);
>  	if (ret)
> -		return ret;
> +		goto err_clk_per;
>  
>  	ret = clk_prepare_enable(dev->clk_ahb);
>  	if (ret)
> @@ -3369,6 +3380,7 @@ static int coda_hw_init(struct coda_dev *dev)
>  
>  err_clk_ahb:
>  	clk_disable_unprepare(dev->clk_per);
> +err_clk_per:
>  	return ret;
>  }
>  
> @@ -3394,10 +3406,29 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
>  	memcpy(dev->codebuf.vaddr, fw->data, fw->size);
>  	release_firmware(fw);
>  
> -	ret = coda_hw_init(dev);
> -	if (ret) {
> -		v4l2_err(&dev->v4l2_dev, "HW initialization failed\n");
> -		return;
> +	if (IS_ENABLED(CONFIG_PM_RUNTIME) && pdev->dev.pm_domain) {

How about using the pm_runtime_enabled() helper ? Also why do you need to
be checking dev.pm_domain here and in the resume() callback ? Couldn't it 
be done unconditionally ? Why the driver needs to care about the PM domain
existence ?

> +		/*
> +		 * Enabling power temporarily will cause coda_hw_init to be
> +		 * called via coda_runtime_resume by the pm domain.
> +		 */
> +		ret = pm_runtime_get_sync(&dev->plat_dev->dev);
> +		if (ret < 0) {
> +			v4l2_err(&dev->v4l2_dev, "failed to power on: %d\n",
> +				 ret);
> +			return;
> +		}
> +
> +		pm_runtime_put_sync(&dev->plat_dev->dev);
> +	} else {
> +		/*
> +		 * If runtime pm is disabled or pm_domain is not set,
> +		 * initialize once manually.
> +		 */
> +		ret = coda_hw_init(dev);
> +		if (ret < 0) {
> +			v4l2_err(&dev->v4l2_dev, "HW initialization failed\n");
> +			return;
> +		}
>  	}
>  
>  	dev->vfd.fops	= &coda_fops,
> @@ -3635,6 +3666,8 @@ static int coda_probe(struct platform_device *pdev)
>  
>  	platform_set_drvdata(pdev, dev);
>  
> +	pm_runtime_enable(&pdev->dev);
> +
>  	return coda_firmware_request(dev);
>  }
>  
> @@ -3645,6 +3678,7 @@ static int coda_remove(struct platform_device *pdev)
>  	video_unregister_device(&dev->vfd);
>  	if (dev->m2m_dev)
>  		v4l2_m2m_release(dev->m2m_dev);
> +	pm_runtime_disable(&pdev->dev);
>  	if (dev->alloc_ctx)
>  		vb2_dma_contig_cleanup_ctx(dev->alloc_ctx);
>  	v4l2_device_unregister(&dev->v4l2_dev);
> @@ -3658,6 +3692,26 @@ static int coda_remove(struct platform_device *pdev)
>  	return 0;
>  }
>  
> +#ifdef CONFIG_PM_RUNTIME
> +static int coda_runtime_resume(struct device *dev)
> +{
> +	struct coda_dev *cdev = dev_get_drvdata(dev);
> +	int ret = 0;
> +
> +	if (dev->pm_domain) {
> +		ret = coda_hw_init(cdev);
> +		if (ret)
> +			v4l2_err(&cdev->v4l2_dev, "HW initialization failed\n");
> +	}
> +
> +	return ret;
> +}
> +#endif

--
Regards,
Sylwester

Re: [PATCH 12/30] [media] coda: Add runtime pm support

[Philipp Zabel]

Hi Sylwester,

On Fri, Jun 13, 2014 at 06:56:16PM +0200, Sylwester Nawrocki wrote:
[...]
> > @@ -3394,10 +3406,29 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
> >  	memcpy(dev->codebuf.vaddr, fw->data, fw->size);
> >  	release_firmware(fw);
> >  
> > -	ret = coda_hw_init(dev);
> > -	if (ret) {
> > -		v4l2_err(&dev->v4l2_dev, "HW initialization failed\n");
> > -		return;
> > +	if (IS_ENABLED(CONFIG_PM_RUNTIME) && pdev->dev.pm_domain) {
> 
> How about using the pm_runtime_enabled() helper ? Also why do you need to
> be checking dev.pm_domain here and in the resume() callback ? Couldn't it 
> be done unconditionally ? Why the driver needs to care about the PM domain
> existence ?

Thank you for the hint, pm_runtime_enabled() is what I want here.

The idea with the pm_domain check was that without an associated pm_domain
there is no need to do the hardware initialization over and over again.
So if PM_RUNTIME is enabled, but no pm_domain is associated with the device,
we call hw_init only once, and not on every runtime_resume.
The hardware initialization on coda mostly consists of a 4KiB firmware upload
into the code SRAM via an upload register, and a reset of the DSP processor.

regards
Philipp

[PATCH 13/30] [media] coda: split firmware version check out of coda_hw_init

[Philipp Zabel]

This adds a new function coda_check_firmware that does the firmware
version checks so that this can be done only once from coda_probe
instead of every time the runtime pm framework resumes the coda.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 42 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index f39f693..b2e8e0e 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -3247,7 +3247,6 @@ static bool coda_firmware_supported(u32 vernum)
 
 static int coda_hw_init(struct coda_dev *dev)
 {
-	u16 product, major, minor, release;
 	u32 data;
 	u16 *p;
 	int i, ret;
@@ -3328,17 +3327,40 @@ static int coda_hw_init(struct coda_dev *dev)
 	coda_write(dev, data, CODA_REG_BIT_CODE_RESET);
 	coda_write(dev, CODA_REG_RUN_ENABLE, CODA_REG_BIT_CODE_RUN);
 
-	/* Load firmware */
+	clk_disable_unprepare(dev->clk_ahb);
+	clk_disable_unprepare(dev->clk_per);
+
+	return 0;
+
+err_clk_ahb:
+	clk_disable_unprepare(dev->clk_per);
+err_clk_per:
+	return ret;
+}
+
+static int coda_check_firmware(struct coda_dev *dev)
+{
+	u16 product, major, minor, release;
+	u32 data;
+	int ret;
+
+	ret = clk_prepare_enable(dev->clk_per);
+	if (ret)
+		goto err_clk_per;
+
+	ret = clk_prepare_enable(dev->clk_ahb);
+	if (ret)
+		goto err_clk_ahb;
+
 	coda_write(dev, 0, CODA_CMD_FIRMWARE_VERNUM);
 	coda_write(dev, CODA_REG_BIT_BUSY_FLAG, CODA_REG_BIT_BUSY);
 	coda_write(dev, 0, CODA_REG_BIT_RUN_INDEX);
 	coda_write(dev, 0, CODA_REG_BIT_RUN_COD_STD);
 	coda_write(dev, CODA_COMMAND_FIRMWARE_GET, CODA_REG_BIT_RUN_COMMAND);
 	if (coda_wait_timeout(dev)) {
-		clk_disable_unprepare(dev->clk_per);
-		clk_disable_unprepare(dev->clk_ahb);
 		v4l2_err(&dev->v4l2_dev, "firmware get command error\n");
-		return -EIO;
+		ret = -EIO;
+		goto err_run_cmd;
 	}
 
 	if (dev->devtype->product == CODA_960) {
@@ -3378,6 +3400,8 @@ static int coda_hw_init(struct coda_dev *dev)
 
 	return 0;
 
+err_run_cmd:
+	clk_disable_unprepare(dev->clk_ahb);
 err_clk_ahb:
 	clk_disable_unprepare(dev->clk_per);
 err_clk_per:
@@ -3418,6 +3442,10 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
 			return;
 		}
 
+		ret = coda_check_firmware(dev);
+		if (ret < 0)
+			return;
+
 		pm_runtime_put_sync(&dev->plat_dev->dev);
 	} else {
 		/*
@@ -3429,6 +3457,10 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
 			v4l2_err(&dev->v4l2_dev, "HW initialization failed\n");
 			return;
 		}
+
+		ret = coda_check_firmware(dev);
+		if (ret < 0)
+			return;
 	}
 
 	dev->vfd.fops	= &coda_fops,
-- 
2.0.0.rc2

[PATCH 14/30] [media] coda: select GENERIC_ALLOCATOR

[Philipp Zabel]

The driver uses the genalloc API, which doesn't have stubs in
case GENERIC_ALLOCATOR is disabled.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/Kconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 20f1655..1d2ac9d 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -140,6 +140,7 @@ config VIDEO_CODA
 	select SRAM
 	select VIDEOBUF2_DMA_CONTIG
 	select V4L2_MEM2MEM_DEV
+	select GENERIC_ALLOCATOR
 	---help---
 	   Coda is a range of video codec IPs that supports
 	   H.264, MPEG-4, and other video formats.
-- 
2.0.0.rc2

[PATCH 15/30] [media] coda: add h.264 min/max qp controls

[Philipp Zabel]

If the bitrate control is set, the encoder works in CBR mode, dynamically
changing the quantization parameters to achieve a constant bitrate.
With the min/max QP controls the quantization parameters can be limited
to a given range.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index b2e8e0e..fa7eafb 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -159,6 +159,8 @@ struct coda_params {
 	u8			rot_mode;
 	u8			h264_intra_qp;
 	u8			h264_inter_qp;
+	u8			h264_min_qp;
+	u8			h264_max_qp;
 	u8			mpeg4_intra_qp;
 	u8			mpeg4_inter_qp;
 	u8			gop_size;
@@ -2433,7 +2435,16 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 		coda_write(dev, (gamma & CODA_GAMMA_MASK) << CODA_GAMMA_OFFSET,
 			   CODA_CMD_ENC_SEQ_RC_GAMMA);
 	}
+
+	if (ctx->params.h264_min_qp || ctx->params.h264_max_qp) {
+		coda_write(dev,
+			   ctx->params.h264_min_qp << CODA_QPMIN_OFFSET |
+			   ctx->params.h264_max_qp << CODA_QPMAX_OFFSET,
+			   CODA_CMD_ENC_SEQ_RC_QP_MIN_MAX);
+	}
 	if (dev->devtype->product == CODA_960) {
+		if (ctx->params.h264_max_qp)
+			value |= 1 << CODA9_OPTION_RCQPMAX_OFFSET;
 		if (CODA_DEFAULT_GAMMA > 0)
 			value |= 1 << CODA9_OPTION_GAMMA_OFFSET;
 	} else {
@@ -2443,6 +2454,10 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 			else
 				value |= 1 << CODA7_OPTION_GAMMA_OFFSET;
 		}
+		if (ctx->params.h264_min_qp)
+			value |= 1 << CODA7_OPTION_RCQPMIN_OFFSET;
+		if (ctx->params.h264_max_qp)
+			value |= 1 << CODA7_OPTION_RCQPMAX_OFFSET;
 	}
 	coda_write(dev, value, CODA_CMD_ENC_SEQ_OPTION);
 
@@ -2670,6 +2685,12 @@ static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
 	case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP:
 		ctx->params.h264_inter_qp = ctrl->val;
 		break;
+	case V4L2_CID_MPEG_VIDEO_H264_MIN_QP:
+		ctx->params.h264_min_qp = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
+		ctx->params.h264_max_qp = ctrl->val;
+		break;
 	case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP:
 		ctx->params.mpeg4_intra_qp = ctrl->val;
 		break;
@@ -2717,6 +2738,12 @@ static int coda_ctrls_setup(struct coda_ctx *ctx)
 		V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, 0, 51, 1, 25);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
 		V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, 0, 51, 1, 25);
+	if (ctx->dev->devtype->product != CODA_960) {
+		v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
+			V4L2_CID_MPEG_VIDEO_H264_MIN_QP, 0, 51, 1, 12);
+	}
+	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
+		V4L2_CID_MPEG_VIDEO_H264_MAX_QP, 0, 51, 1, 51);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
 		V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP, 1, 31, 1, 2);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
-- 
2.0.0.rc2

[PATCH 16/30] [media] coda: add h.264 deblocking filter controls

[Philipp Zabel]

This adds controls for the h.264 deblocking loop filter.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index fa7eafb..4b84d16 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -161,6 +161,9 @@ struct coda_params {
 	u8			h264_inter_qp;
 	u8			h264_min_qp;
 	u8			h264_max_qp;
+	u8			h264_deblk_enabled;
+	u8			h264_deblk_alpha;
+	u8			h264_deblk_beta;
 	u8			mpeg4_intra_qp;
 	u8			mpeg4_inter_qp;
 	u8			gop_size;
@@ -2380,7 +2383,17 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 			coda_write(dev, CODA9_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
 		else
 			coda_write(dev, CODA_STD_H264, CODA_CMD_ENC_SEQ_COD_STD);
-		coda_write(dev, 0, CODA_CMD_ENC_SEQ_264_PARA);
+		if (ctx->params.h264_deblk_enabled) {
+			value = ((ctx->params.h264_deblk_alpha &
+				  CODA_264PARAM_DEBLKFILTEROFFSETALPHA_MASK) <<
+				 CODA_264PARAM_DEBLKFILTEROFFSETALPHA_OFFSET) |
+				((ctx->params.h264_deblk_beta &
+				  CODA_264PARAM_DEBLKFILTEROFFSETBETA_MASK) <<
+				 CODA_264PARAM_DEBLKFILTEROFFSETBETA_OFFSET);
+		} else {
+			value = 1 << CODA_264PARAM_DISABLEDEBLK_OFFSET;
+		}
+		coda_write(dev, value, CODA_CMD_ENC_SEQ_264_PARA);
 		break;
 	default:
 		v4l2_err(v4l2_dev,
@@ -2691,6 +2704,16 @@ static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
 	case V4L2_CID_MPEG_VIDEO_H264_MAX_QP:
 		ctx->params.h264_max_qp = ctrl->val;
 		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA:
+		ctx->params.h264_deblk_alpha = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA:
+		ctx->params.h264_deblk_beta = ctrl->val;
+		break;
+	case V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE:
+		ctx->params.h264_deblk_enabled = (ctrl->val ==
+				V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_ENABLED);
+		break;
 	case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP:
 		ctx->params.mpeg4_intra_qp = ctrl->val;
 		break;
@@ -2745,6 +2768,14 @@ static int coda_ctrls_setup(struct coda_ctx *ctx)
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
 		V4L2_CID_MPEG_VIDEO_H264_MAX_QP, 0, 51, 1, 51);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
+		V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_ALPHA, 0, 15, 1, 0);
+	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
+		V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_BETA, 0, 15, 1, 0);
+	v4l2_ctrl_new_std_menu(&ctx->ctrls, &coda_ctrl_ops,
+		V4L2_CID_MPEG_VIDEO_H264_LOOP_FILTER_MODE,
+		V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_DISABLED, 0x0,
+		V4L2_MPEG_VIDEO_H264_LOOP_FILTER_MODE_ENABLED);
+	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
 		V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP, 1, 31, 1, 2);
 	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
 		V4L2_CID_MPEG_VIDEO_MPEG4_P_FRAME_QP, 1, 31, 1, 2);
-- 
2.0.0.rc2

[PATCH 17/30] [media] coda: add cyclic intra refresh control

[Philipp Zabel]

Allow userspace to enable cyclic intra refresh by setting the number of
intra macroblocks per frame to a non-zero value.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 4b84d16..11e059d 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -167,6 +167,7 @@ struct coda_params {
 	u8			mpeg4_intra_qp;
 	u8			mpeg4_inter_qp;
 	u8			gop_size;
+	int			intra_refresh;
 	int			codec_mode;
 	int			codec_mode_aux;
 	enum v4l2_mpeg_video_multi_slice_mode slice_mode;
@@ -2433,7 +2434,8 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 	coda_write(dev, value, CODA_CMD_ENC_SEQ_RC_PARA);
 
 	coda_write(dev, 0, CODA_CMD_ENC_SEQ_RC_BUF_SIZE);
-	coda_write(dev, 0, CODA_CMD_ENC_SEQ_INTRA_REFRESH);
+	coda_write(dev, ctx->params.intra_refresh,
+		   CODA_CMD_ENC_SEQ_INTRA_REFRESH);
 
 	coda_write(dev, bitstream_buf, CODA_CMD_ENC_SEQ_BB_START);
 	coda_write(dev, bitstream_size / 1024, CODA_CMD_ENC_SEQ_BB_SIZE);
@@ -2731,6 +2733,9 @@ static int coda_s_ctrl(struct v4l2_ctrl *ctrl)
 		break;
 	case V4L2_CID_MPEG_VIDEO_HEADER_MODE:
 		break;
+	case V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB:
+		ctx->params.intra_refresh = ctrl->val;
+		break;
 	default:
 		v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev,
 			"Invalid control, id=%d, val=%d\n",
@@ -2792,6 +2797,8 @@ static int coda_ctrls_setup(struct coda_ctx *ctx)
 		V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME,
 		(1 << V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE),
 		V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME);
+	v4l2_ctrl_new_std(&ctx->ctrls, &coda_ctrl_ops,
+		V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB, 0, 1920 * 1088 / 256, 1, 0);
 
 	if (ctx->ctrls.error) {
 		v4l2_err(&ctx->dev->v4l2_dev, "control initialization error (%d)",
-- 
2.0.0.rc2

[PATCH 18/30] [media] coda: let userspace force IDR frames by enabling the keyframe flag in the source buffer

[Philipp Zabel]

This disables forcing IDR frames at GOP size intervals on CODA7541 and CODA960,
which is only needed to work around a firmware bug on CodaDx6.
Instead, the V4L2_BUF_FLAG_KEYFRAME v4l2 buffer flag is cleared before marking
the source buffer done for dequeueing. Userspace can set it before queueing a
frame to force an IDR frame, to implement VFU (Video Fast Update).

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 11e059d..cf75112 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -1264,22 +1264,22 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
 	 * frame as IDR. This is a problem for some decoders that can't
 	 * recover when a frame is lost.
 	 */
-	if (src_buf->v4l2_buf.sequence % ctx->params.gop_size) {
-		src_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_PFRAME;
-		src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_KEYFRAME;
-	} else {
+	if ((src_buf->v4l2_buf.sequence % ctx->params.gop_size) == 0)
 		src_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_KEYFRAME;
+	if (src_buf->v4l2_buf.flags & V4L2_BUF_FLAG_KEYFRAME)
 		src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_PFRAME;
-	}
+	else
+		src_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_PFRAME;
 
 	if (dev->devtype->product == CODA_960)
 		coda_set_gdi_regs(ctx);
 
 	/*
-	 * Copy headers at the beginning of the first frame for H.264 only.
-	 * In MPEG4 they are already copied by the coda.
+	 * Copy headers in front of the first frame and forced I frames for
+	 * H.264 only. In MPEG4 they are already copied by the CODA.
 	 */
-	if (src_buf->v4l2_buf.sequence == 0) {
+	if (src_buf->v4l2_buf.sequence == 0 ||
+	    src_buf->v4l2_buf.flags & V4L2_BUF_FLAG_KEYFRAME) {
 		pic_stream_buffer_addr =
 			vb2_dma_contig_plane_dma_addr(dst_buf, 0) +
 			ctx->vpu_header_size[0] +
@@ -3245,6 +3245,8 @@ static void coda_finish_encode(struct coda_ctx *ctx)
 		src_buf->v4l2_buf.flags & V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
 	dst_buf->v4l2_buf.timecode = src_buf->v4l2_buf.timecode;
 
+	/* Clear keyframe flag so userspace can misuse it to force an IDR frame */
+	src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_KEYFRAME;
 	v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
 
 	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
-- 
2.0.0.rc2

Re: [PATCH 18/30] [media] coda: let userspace force IDR frames by enabling the keyframe flag in the source buffer

[Hans Verkuil]

On 06/13/2014 06:08 PM, Philipp Zabel wrote:
> This disables forcing IDR frames at GOP size intervals on CODA7541 and CODA960,
> which is only needed to work around a firmware bug on CodaDx6.
> Instead, the V4L2_BUF_FLAG_KEYFRAME v4l2 buffer flag is cleared before marking
> the source buffer done for dequeueing. Userspace can set it before queueing a
> frame to force an IDR frame, to implement VFU (Video Fast Update).

I'd like to see an RFC for this feature. Rather than 'misuse' it, I think this
should be standardized. I have nothing against using KEYFRAME in order to
implement VFU (in fact, I like it!), but it should be documented and well-defined.

Regards,

	Hans

> 
> Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> ---
>  drivers/media/platform/coda.c | 18 ++++++++++--------
>  1 file changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
> index 11e059d..cf75112 100644
> --- a/drivers/media/platform/coda.c
> +++ b/drivers/media/platform/coda.c
> @@ -1264,22 +1264,22 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
>  	 * frame as IDR. This is a problem for some decoders that can't
>  	 * recover when a frame is lost.
>  	 */
> -	if (src_buf->v4l2_buf.sequence % ctx->params.gop_size) {
> -		src_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_PFRAME;
> -		src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_KEYFRAME;
> -	} else {
> +	if ((src_buf->v4l2_buf.sequence % ctx->params.gop_size) == 0)
>  		src_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_KEYFRAME;
> +	if (src_buf->v4l2_buf.flags & V4L2_BUF_FLAG_KEYFRAME)
>  		src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_PFRAME;
> -	}
> +	else
> +		src_buf->v4l2_buf.flags |= V4L2_BUF_FLAG_PFRAME;
>  
>  	if (dev->devtype->product == CODA_960)
>  		coda_set_gdi_regs(ctx);
>  
>  	/*
> -	 * Copy headers at the beginning of the first frame for H.264 only.
> -	 * In MPEG4 they are already copied by the coda.
> +	 * Copy headers in front of the first frame and forced I frames for
> +	 * H.264 only. In MPEG4 they are already copied by the CODA.
>  	 */
> -	if (src_buf->v4l2_buf.sequence == 0) {
> +	if (src_buf->v4l2_buf.sequence == 0 ||
> +	    src_buf->v4l2_buf.flags & V4L2_BUF_FLAG_KEYFRAME) {
>  		pic_stream_buffer_addr =
>  			vb2_dma_contig_plane_dma_addr(dst_buf, 0) +
>  			ctx->vpu_header_size[0] +
> @@ -3245,6 +3245,8 @@ static void coda_finish_encode(struct coda_ctx *ctx)
>  		src_buf->v4l2_buf.flags & V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
>  	dst_buf->v4l2_buf.timecode = src_buf->v4l2_buf.timecode;
>  
> +	/* Clear keyframe flag so userspace can misuse it to force an IDR frame */
> +	src_buf->v4l2_buf.flags &= ~V4L2_BUF_FLAG_KEYFRAME;
>  	v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
>  
>  	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
> 

Re: [PATCH 18/30] [media] coda: let userspace force IDR frames by enabling the keyframe flag in the source buffer

[Philipp Zabel]

Am Montag, den 16.06.2014, 10:24 +0200 schrieb Hans Verkuil:
> On 06/13/2014 06:08 PM, Philipp Zabel wrote:
> > This disables forcing IDR frames at GOP size intervals on CODA7541 and CODA960,
> > which is only needed to work around a firmware bug on CodaDx6.
> > Instead, the V4L2_BUF_FLAG_KEYFRAME v4l2 buffer flag is cleared before marking
> > the source buffer done for dequeueing. Userspace can set it before queueing a
> > frame to force an IDR frame, to implement VFU (Video Fast Update).
> 
> I'd like to see an RFC for this feature. Rather than 'misuse' it, I think this
> should be standardized. I have nothing against using KEYFRAME in order to
> implement VFU (in fact, I like it!), but it should be documented and well-defined.

Thanks, I'll prepare a separate RFC for this.
The other possibility would be to use a V4L2_ENC_CMD for this feature.

One thing I'm not sure about is how to signal to userspace that this
feature is available when using the KEYFRAME clearing. With encoder
commands, TRY_COMMAND could be used. But when using the buffer KEYFRAME
bits, I suppose a flag would have to be introduced somewhere?

regards
Philipp

[PATCH 19/30] [media] v4l2-mem2mem: export v4l2_m2m_try_schedule

[Philipp Zabel]

From: Michael Olbrich <m.olbrich@pengutronix.de>

Some drivers might allow to decode remaining frames from an internal ringbuffer
after a decoder stop command. Allow those to call v4l2_m2m_try_schedule
directly.

Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/v4l2-core/v4l2-mem2mem.c | 3 ++-
 include/media/v4l2-mem2mem.h           | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 178ce96..5f5c175 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -208,7 +208,7 @@ static void v4l2_m2m_try_run(struct v4l2_m2m_dev *m2m_dev)
  * An example of the above could be an instance that requires more than one
  * src/dst buffer per transaction.
  */
-static void v4l2_m2m_try_schedule(struct v4l2_m2m_ctx *m2m_ctx)
+void v4l2_m2m_try_schedule(struct v4l2_m2m_ctx *m2m_ctx)
 {
 	struct v4l2_m2m_dev *m2m_dev;
 	unsigned long flags_job, flags_out, flags_cap;
@@ -274,6 +274,7 @@ static void v4l2_m2m_try_schedule(struct v4l2_m2m_ctx *m2m_ctx)
 
 	v4l2_m2m_try_run(m2m_dev);
 }
+EXPORT_SYMBOL(v4l2_m2m_try_schedule);
 
 /**
  * v4l2_m2m_cancel_job() - cancel pending jobs for the context
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index 12ea5a6..c5f3914 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -95,6 +95,8 @@ void *v4l2_m2m_get_curr_priv(struct v4l2_m2m_dev *m2m_dev);
 struct vb2_queue *v4l2_m2m_get_vq(struct v4l2_m2m_ctx *m2m_ctx,
 				       enum v4l2_buf_type type);
 
+void v4l2_m2m_try_schedule(struct v4l2_m2m_ctx *m2m_ctx);
+
 void v4l2_m2m_job_finish(struct v4l2_m2m_dev *m2m_dev,
 			 struct v4l2_m2m_ctx *m2m_ctx);
 
-- 
2.0.0.rc2

[PATCH 20/30] [media] coda: try to schedule a decode run after a stop command

[Philipp Zabel]

From: Michael Olbrich <m.olbrich@pengutronix.de>

In case no further buffers are queued after the stop command, restart
job scheduling explicitly.

Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index cf75112..a00eaaf 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -904,6 +904,8 @@ static int coda_decoder_cmd(struct file *file, void *fh,
 		/* If this context is currently running, update the hardware flag */
 		coda_write(dev, ctx->bit_stream_param, CODA_REG_BIT_BIT_STREAM_PARAM);
 	}
+	ctx->prescan_failed = false;
+	v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
 
 	return 0;
 }
-- 
2.0.0.rc2

[PATCH 21/30] [media] coda: add decoder timestamp queue

[Philipp Zabel]

The coda driver advertises timestamp_type V4L2_BUF_FLAG_TIMESTAMP_COPY on
both queues, so we have to copy timestamps from input v4l2 buffers to the
corresponding destination v4l2 buffers. Since the h.264 decoder can reorder
frames, a timestamp queue is needed to keep track of and assign the correct
timestamp to destination buffers.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 50 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index a00eaaf..9de0af0 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -201,6 +201,13 @@ struct gdi_tiled_map {
 #define GDI_LINEAR_FRAME_MAP 0
 };
 
+struct coda_timestamp {
+	struct list_head	list;
+	u32			sequence;
+	struct v4l2_timecode	timecode;
+	struct timeval		timestamp;
+};
+
 struct coda_ctx {
 	struct coda_dev			*dev;
 	struct mutex			buffer_mutex;
@@ -235,6 +242,8 @@ struct coda_ctx {
 	struct coda_aux_buf		slicebuf;
 	struct coda_aux_buf		internal_frames[CODA_MAX_FRAMEBUFFERS];
 	u32				frame_types[CODA_MAX_FRAMEBUFFERS];
+	struct coda_timestamp		frame_timestamps[CODA_MAX_FRAMEBUFFERS];
+	struct list_head		timestamp_list;
 	struct coda_aux_buf		workbuf;
 	int				num_internal_frames;
 	int				idx;
@@ -1067,7 +1076,7 @@ static int coda_bitstream_queue(struct coda_ctx *ctx, struct vb2_buffer *src_buf
 	dma_sync_single_for_device(&ctx->dev->plat_dev->dev, ctx->bitstream.paddr,
 				   ctx->bitstream.size, DMA_TO_DEVICE);
 
-	ctx->qsequence++;
+	src_buf->v4l2_buf.sequence = ctx->qsequence++;
 
 	return 0;
 }
@@ -1103,12 +1112,26 @@ static bool coda_bitstream_try_queue(struct coda_ctx *ctx,
 static void coda_fill_bitstream(struct coda_ctx *ctx)
 {
 	struct vb2_buffer *src_buf;
+	struct coda_timestamp *ts;
 
 	while (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) > 0) {
 		src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 
 		if (coda_bitstream_try_queue(ctx, src_buf)) {
+			/*
+			 * Source buffer is queued in the bitstream ringbuffer;
+			 * queue the timestamp and mark source buffer as done
+			 */
 			src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+
+			ts = kmalloc(sizeof(*ts), GFP_KERNEL);
+			if (ts) {
+				ts->sequence = src_buf->v4l2_buf.sequence;
+				ts->timecode = src_buf->v4l2_buf.timecode;
+				ts->timestamp = src_buf->v4l2_buf.timestamp;
+				list_add_tail(&ts->list, &ctx->timestamp_list);
+			}
+
 			v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
 		} else {
 			break;
@@ -2653,6 +2676,14 @@ static void coda_stop_streaming(struct vb2_queue *q)
 	}
 
 	if (!ctx->streamon_out && !ctx->streamon_cap) {
+		struct coda_timestamp *ts;
+
+		while (!list_empty(&ctx->timestamp_list)) {
+			ts = list_first_entry(&ctx->timestamp_list,
+					      struct coda_timestamp, list);
+			list_del(&ts->list);
+			kfree(ts);
+		}
 		kfifo_init(&ctx->bitstream_fifo,
 			ctx->bitstream.vaddr, ctx->bitstream.size);
 		ctx->runcounter = 0;
@@ -2940,6 +2971,7 @@ static int coda_open(struct file *file)
 		ctx->bitstream.vaddr, ctx->bitstream.size);
 	mutex_init(&ctx->bitstream_mutex);
 	mutex_init(&ctx->buffer_mutex);
+	INIT_LIST_HEAD(&ctx->timestamp_list);
 
 	coda_lock(ctx);
 	list_add(&ctx->list, &dev->instances);
@@ -3031,6 +3063,7 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 	struct coda_q_data *q_data_src;
 	struct coda_q_data *q_data_dst;
 	struct vb2_buffer *dst_buf;
+	struct coda_timestamp *ts;
 	int width, height;
 	int decoded_idx;
 	int display_idx;
@@ -3152,6 +3185,18 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		v4l2_err(&dev->v4l2_dev,
 			 "decoded frame index out of range: %d\n", decoded_idx);
 	} else {
+		ts = list_first_entry(&ctx->timestamp_list,
+				      struct coda_timestamp, list);
+		list_del(&ts->list);
+		val = coda_read(dev, CODA_RET_DEC_PIC_FRAME_NUM) - 1;
+		if (val != ts->sequence) {
+			v4l2_err(&dev->v4l2_dev,
+				 "sequence number mismatch (%d != %d)\n",
+				 val, ts->sequence);
+		}
+		ctx->frame_timestamps[decoded_idx] = *ts;
+		kfree(ts);
+
 		val = coda_read(dev, CODA_RET_DEC_PIC_TYPE) & 0x7;
 		if (val == 0)
 			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_KEYFRAME;
@@ -3184,6 +3229,9 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		dst_buf->v4l2_buf.flags &= ~(V4L2_BUF_FLAG_KEYFRAME |
 					     V4L2_BUF_FLAG_PFRAME);
 		dst_buf->v4l2_buf.flags |= ctx->frame_types[ctx->display_idx];
+		ts = &ctx->frame_timestamps[ctx->display_idx];
+		dst_buf->v4l2_buf.timecode = ts->timecode;
+		dst_buf->v4l2_buf.timestamp = ts->timestamp;
 
 		vb2_set_plane_payload(dst_buf, 0, width * height * 3 / 2);
 
-- 
2.0.0.rc2

[PATCH 22/30] [media] coda: alert userspace about macroblock errors

[Philipp Zabel]

If the CODA reports macroblock errors, also set the VB2_BUF_STATE_ERROR flag
to alert userspace.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 9de0af0..93836c8 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -243,6 +243,7 @@ struct coda_ctx {
 	struct coda_aux_buf		internal_frames[CODA_MAX_FRAMEBUFFERS];
 	u32				frame_types[CODA_MAX_FRAMEBUFFERS];
 	struct coda_timestamp		frame_timestamps[CODA_MAX_FRAMEBUFFERS];
+	u32				frame_errors[CODA_MAX_FRAMEBUFFERS];
 	struct list_head		timestamp_list;
 	struct coda_aux_buf		workbuf;
 	int				num_internal_frames;
@@ -3069,6 +3070,7 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 	int display_idx;
 	u32 src_fourcc;
 	int success;
+	u32 err_mb;
 	u32 val;
 
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
@@ -3138,10 +3140,10 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 		/* no cropping */
 	}
 
-	val = coda_read(dev, CODA_RET_DEC_PIC_ERR_MB);
-	if (val > 0)
+	err_mb = coda_read(dev, CODA_RET_DEC_PIC_ERR_MB);
+	if (err_mb > 0)
 		v4l2_err(&dev->v4l2_dev,
-			 "errors in %d macroblocks\n", val);
+			 "errors in %d macroblocks\n", err_mb);
 
 	if (dev->devtype->product == CODA_7541) {
 		val = coda_read(dev, CODA_RET_DEC_PIC_OPTION);
@@ -3204,6 +3206,8 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_PFRAME;
 		else
 			ctx->frame_types[decoded_idx] = V4L2_BUF_FLAG_BFRAME;
+
+		ctx->frame_errors[decoded_idx] = err_mb;
 	}
 
 	if (display_idx == -1) {
@@ -3235,8 +3239,8 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 
 		vb2_set_plane_payload(dst_buf, 0, width * height * 3 / 2);
 
-		v4l2_m2m_buf_done(dst_buf, success ? VB2_BUF_STATE_DONE :
-						     VB2_BUF_STATE_ERROR);
+		v4l2_m2m_buf_done(dst_buf, ctx->frame_errors[display_idx] ?
+				  VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
 
 		v4l2_dbg(1, coda_debug, &dev->v4l2_dev,
 			"job finished: decoding frame (%d) (%s)\n",
-- 
2.0.0.rc2

[PATCH 23/30] [media] coda: add sequence counter offset

[Philipp Zabel]

The coda h.264 decoder also counts PIC_RUNs where no frame was decoded but
a frame was rotated out / marked as ready to be displayed. This causes an
offset between the incoming encoded frame's sequence number and the decode
sequence number returned by the coda. This patch introduces a sequence
counter offset variable to keep track of the difference.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 93836c8..4641667dbc 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -222,6 +222,7 @@ struct coda_ctx {
 	u32				isequence;
 	u32				qsequence;
 	u32				osequence;
+	u32				sequence_offset;
 	struct coda_q_data		q_data[2];
 	enum coda_inst_type		inst_type;
 	struct coda_codec		*codec;
@@ -2674,6 +2675,7 @@ static void coda_stop_streaming(struct vb2_queue *q)
 		ctx->streamon_cap = 0;
 
 		ctx->osequence = 0;
+		ctx->sequence_offset = 0;
 	}
 
 	if (!ctx->streamon_out && !ctx->streamon_cap) {
@@ -3179,7 +3181,9 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 
 	if (decoded_idx == -1) {
 		/* no frame was decoded, but we might have a display frame */
-		if (display_idx < 0 && ctx->display_idx < 0)
+		if (display_idx >= 0 && display_idx < ctx->num_internal_frames)
+			ctx->sequence_offset++;
+		else if (ctx->display_idx < 0)
 			ctx->prescan_failed = true;
 	} else if (decoded_idx == -2) {
 		/* no frame was decoded, we still return the remaining buffers */
@@ -3191,10 +3195,11 @@ static void coda_finish_decode(struct coda_ctx *ctx)
 				      struct coda_timestamp, list);
 		list_del(&ts->list);
 		val = coda_read(dev, CODA_RET_DEC_PIC_FRAME_NUM) - 1;
+		val -= ctx->sequence_offset;
 		if (val != ts->sequence) {
 			v4l2_err(&dev->v4l2_dev,
-				 "sequence number mismatch (%d != %d)\n",
-				 val, ts->sequence);
+				 "sequence number mismatch (%d(%d) != %d)\n",
+				 val, ctx->sequence_offset, ts->sequence);
 		}
 		ctx->frame_timestamps[decoded_idx] = *ts;
 		kfree(ts);
-- 
2.0.0.rc2

[PATCH 24/30] [media] coda: use prescan_failed variable to stop stream after a timeout

[Philipp Zabel]

This variable should be renamed to hold instead (temporarily stopping streaming
until new data is fed into the bitstream buffer).

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 4641667dbc..a3fda5e 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -1477,6 +1477,8 @@ static void coda_pic_run_work(struct work_struct *work)
 
 	if (!wait_for_completion_timeout(&ctx->completion, msecs_to_jiffies(1000))) {
 		dev_err(&dev->plat_dev->dev, "CODA PIC_RUN timeout\n");
+
+		ctx->prescan_failed = true;
 	} else if (!ctx->aborting) {
 		if (ctx->inst_type == CODA_INST_DECODER)
 			coda_finish_decode(ctx);
-- 
2.0.0.rc2

[PATCH 25/30] [media] coda: add reset control support

[Philipp Zabel]

On i.MX53 and i.MX6, the CODA VPU can be reset by the System Reset Controller.
We can use this to get out of dire situations, for example after a PIC_RUN
timeout.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 51 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index a3fda5e..e71898e 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -27,6 +27,7 @@
 #include <linux/videodev2.h>
 #include <linux/of.h>
 #include <linux/platform_data/coda.h>
+#include <linux/reset.h>
 
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-device.h>
@@ -138,6 +139,7 @@ struct coda_dev {
 	void __iomem		*regs_base;
 	struct clk		*clk_per;
 	struct clk		*clk_ahb;
+	struct reset_control	*rstc;
 
 	struct coda_aux_buf	codebuf;
 	struct coda_aux_buf	tempbuf;
@@ -337,6 +339,39 @@ static int coda_command_sync(struct coda_ctx *ctx, int cmd)
 	return coda_wait_timeout(dev);
 }
 
+static int coda_hw_reset(struct coda_ctx *ctx)
+{
+	struct coda_dev *dev = ctx->dev;
+	unsigned long timeout;
+	unsigned int idx;
+	int ret;
+
+	if (!dev->rstc)
+		return -ENOENT;
+
+	idx = coda_read(dev, CODA_REG_BIT_RUN_INDEX);
+
+	timeout = jiffies + msecs_to_jiffies(100);
+	coda_write(dev, 0x11, CODA9_GDI_BUS_CTRL);
+	while (coda_read(dev, CODA9_GDI_BUS_STATUS) != 0x77) {
+		if (time_after(jiffies, timeout))
+			return -ETIME;
+		cpu_relax();
+	}
+
+	ret = reset_control_reset(dev->rstc);
+	if (ret < 0)
+		return ret;
+
+	coda_write(dev, 0x00, CODA9_GDI_BUS_CTRL);
+	coda_write(dev, CODA_REG_BIT_BUSY_FLAG, CODA_REG_BIT_BUSY);
+	coda_write(dev, CODA_REG_RUN_ENABLE, CODA_REG_BIT_CODE_RUN);
+	ret = coda_wait_timeout(dev);
+	coda_write(dev, idx, CODA_REG_BIT_RUN_INDEX);
+
+	return ret;
+}
+
 static struct coda_q_data *get_q_data(struct coda_ctx *ctx,
 					 enum v4l2_buf_type type)
 {
@@ -1479,6 +1514,8 @@ static void coda_pic_run_work(struct work_struct *work)
 		dev_err(&dev->plat_dev->dev, "CODA PIC_RUN timeout\n");
 
 		ctx->prescan_failed = true;
+
+		coda_hw_reset(ctx);
 	} else if (!ctx->aborting) {
 		if (ctx->inst_type == CODA_INST_DECODER)
 			coda_finish_decode(ctx);
@@ -3387,6 +3424,9 @@ static int coda_hw_init(struct coda_dev *dev)
 	if (ret)
 		goto err_clk_ahb;
 
+	if (dev->rstc)
+		reset_control_reset(dev->rstc);
+
 	/*
 	 * Copy the first CODA_ISRAM_SIZE in the internal SRAM.
 	 * The 16-bit chars in the code buffer are in memory access
@@ -3745,6 +3785,17 @@ static int coda_probe(struct platform_device *pdev)
 		return -ENOENT;
 	}
 
+	dev->rstc = devm_reset_control_get(&pdev->dev, NULL);
+	if (IS_ERR(dev->rstc)) {
+		ret = PTR_ERR(dev->rstc);
+		if (ret == -ENOENT) {
+			dev->rstc = NULL;
+		} else {
+			dev_err(&pdev->dev, "failed get reset control: %d\n", ret);
+			return ret;
+		}
+	}
+
 	/* Get IRAM pool from device tree or platform data */
 	pool = of_get_named_gen_pool(np, "iram", 0);
 	if (!pool && pdata)
-- 
2.0.0.rc2

[PATCH 26/30] [media] coda: add bytesperline to queue data

[Philipp Zabel]

bytesperline is calculated in multiple places, store it in the coda_q_data
structure. This will be more useful later when adding JPEG support.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index e71898e..0697436 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -119,6 +119,7 @@ struct coda_devtype {
 struct coda_q_data {
 	unsigned int		width;
 	unsigned int		height;
+	unsigned int		bytesperline;
 	unsigned int		sizeimage;
 	unsigned int		fourcc;
 	struct v4l2_rect	rect;
@@ -640,10 +641,7 @@ static int coda_g_fmt(struct file *file, void *priv,
 	f->fmt.pix.pixelformat	= q_data->fourcc;
 	f->fmt.pix.width	= q_data->width;
 	f->fmt.pix.height	= q_data->height;
-	if (coda_format_is_yuv(f->fmt.pix.pixelformat))
-		f->fmt.pix.bytesperline = round_up(f->fmt.pix.width, 2);
-	else /* encoded formats h.264/mpeg4 */
-		f->fmt.pix.bytesperline = 0;
+	f->fmt.pix.bytesperline = q_data->bytesperline;
 
 	f->fmt.pix.sizeimage	= q_data->sizeimage;
 	f->fmt.pix.colorspace	= ctx->colorspace;
@@ -791,6 +789,7 @@ static int coda_s_fmt(struct coda_ctx *ctx, struct v4l2_format *f)
 	q_data->fourcc = f->fmt.pix.pixelformat;
 	q_data->width = f->fmt.pix.width;
 	q_data->height = f->fmt.pix.height;
+	q_data->bytesperline = f->fmt.pix.bytesperline;
 	q_data->sizeimage = f->fmt.pix.sizeimage;
 	q_data->rect.left = 0;
 	q_data->rect.top = 0;
@@ -1403,14 +1402,16 @@ static void coda_prepare_encode(struct coda_ctx *ctx)
 	switch (q_data_src->fourcc) {
 	case V4L2_PIX_FMT_YVU420:
 		/* Switch Cb and Cr for YVU420 format */
-		picture_cr = picture_y + q_data_src->width * q_data_src->height;
-		picture_cb = picture_cr + q_data_src->width / 2 *
+		picture_cr = picture_y + q_data_src->bytesperline *
+				q_data_src->height;
+		picture_cb = picture_cr + q_data_src->bytesperline / 2 *
 				q_data_src->height / 2;
 		break;
 	case V4L2_PIX_FMT_YUV420:
 	default:
-		picture_cb = picture_y + q_data_src->width * q_data_src->height;
-		picture_cr = picture_cb + q_data_src->width / 2 *
+		picture_cb = picture_y + q_data_src->bytesperline *
+				q_data_src->height;
+		picture_cr = picture_cb + q_data_src->bytesperline / 2 *
 				q_data_src->height / 2;
 		break;
 	}
@@ -2587,10 +2588,12 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 	}
 
 	coda_write(dev, ctx->num_internal_frames, CODA_CMD_SET_FRAME_BUF_NUM);
-	coda_write(dev, round_up(q_data_src->width, 8), CODA_CMD_SET_FRAME_BUF_STRIDE);
-	if (dev->devtype->product == CODA_7541)
-		coda_write(dev, round_up(q_data_src->width, 8),
+	coda_write(dev, q_data_src->bytesperline,
+			CODA_CMD_SET_FRAME_BUF_STRIDE);
+	if (dev->devtype->product == CODA_7541) {
+		coda_write(dev, q_data_src->bytesperline,
 				CODA7_CMD_SET_FRAME_SOURCE_BUF_STRIDE);
+	}
 	if (dev->devtype->product != CODA_DX6) {
 		coda_write(dev, ctx->iram_info.buf_bit_use,
 				CODA7_CMD_SET_FRAME_AXI_BIT_ADDR);
-- 
2.0.0.rc2

[PATCH 27/30] [media] coda: allow odd width, but still round up bytesperline

[Philipp Zabel]

Even though the CODA h.264 decoder always decodes complete macroblocks, we can
set the stride to the corresponding multiple of 16 and use a value smaller than
that as real width. Unfortunately the same doesn't work for height, as there
is no vertical linesperframe stride for discontiguous planar YUV frames.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 0697436..2b997bd 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -742,9 +742,9 @@ static int coda_try_fmt_vid_cap(struct file *file, void *priv,
 
 	/* The h.264 decoder only returns complete 16x16 macroblocks */
 	if (codec && codec->src_fourcc == V4L2_PIX_FMT_H264) {
-		f->fmt.pix.width = round_up(f->fmt.pix.width, 16);
+		f->fmt.pix.width = f->fmt.pix.width;
 		f->fmt.pix.height = round_up(f->fmt.pix.height, 16);
-		f->fmt.pix.bytesperline = f->fmt.pix.width;
+		f->fmt.pix.bytesperline = round_up(f->fmt.pix.width, 16);
 		f->fmt.pix.sizeimage = f->fmt.pix.bytesperline *
 				       f->fmt.pix.height * 3 / 2;
 	}
-- 
2.0.0.rc2

[PATCH 28/30] [media] coda: round up internal frames to multiples of macroblock size for h.264

[Philipp Zabel]

CODA7541 only supports encoding h.264 frames with width and height that are
multiples of the macroblock size.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index 2b997bd..c65047f 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -1798,15 +1798,21 @@ static void coda_free_framebuffers(struct coda_ctx *ctx)
 static int coda_alloc_framebuffers(struct coda_ctx *ctx, struct coda_q_data *q_data, u32 fourcc)
 {
 	struct coda_dev *dev = ctx->dev;
-	int height = q_data->height;
+	int width, height;
 	dma_addr_t paddr;
 	int ysize;
 	int ret;
 	int i;
 
-	if (ctx->codec && ctx->codec->src_fourcc == V4L2_PIX_FMT_H264)
-		height = round_up(height, 16);
-	ysize = round_up(q_data->width, 8) * height;
+	if (ctx->codec && (ctx->codec->src_fourcc == V4L2_PIX_FMT_H264 ||
+	     ctx->codec->dst_fourcc == V4L2_PIX_FMT_H264)) {
+		width = round_up(q_data->width, 16);
+		height = round_up(q_data->height, 16);
+	} else {
+		width = round_up(q_data->width, 8);
+		height = q_data->height;
+	}
+	ysize = width * height;
 
 	/* Allocate frame buffers */
 	for (i = 0; i < ctx->num_internal_frames; i++) {
@@ -2429,7 +2435,16 @@ static int coda_start_encoding(struct coda_ctx *ctx)
 		value = (q_data_src->width & CODADX6_PICWIDTH_MASK) << CODADX6_PICWIDTH_OFFSET;
 		value |= (q_data_src->height & CODADX6_PICHEIGHT_MASK) << CODA_PICHEIGHT_OFFSET;
 		break;
-	default:
+	case CODA_7541:
+		if (dst_fourcc == V4L2_PIX_FMT_H264) {
+			value = (round_up(q_data_src->width, 16) &
+				 CODA7_PICWIDTH_MASK) << CODA7_PICWIDTH_OFFSET;
+			value |= (round_up(q_data_src->height, 16) &
+				  CODA7_PICHEIGHT_MASK) << CODA_PICHEIGHT_OFFSET;
+			break;
+		}
+		/* fallthrough */
+	case CODA_960:
 		value = (q_data_src->width & CODA7_PICWIDTH_MASK) << CODA7_PICWIDTH_OFFSET;
 		value |= (q_data_src->height & CODA7_PICHEIGHT_MASK) << CODA_PICHEIGHT_OFFSET;
 	}
-- 
2.0.0.rc2

[PATCH 29/30] [media] coda: increase frame stride to 16 for h.264

[Philipp Zabel]

When encoding into h.264, the input frame stride needs to be a multiple of 16.
During allocation of the input buffers, it may not be known yet whether the
encoder should create h.264 or not. Assume the worst and always use a frame
stride that is a multiple of 16.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index c65047f..aabd639d 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -687,8 +687,8 @@ static int coda_try_fmt(struct coda_ctx *ctx, struct coda_codec *codec,
 	switch (f->fmt.pix.pixelformat) {
 	case V4L2_PIX_FMT_YUV420:
 	case V4L2_PIX_FMT_YVU420:
-		/* Frame stride must be multiple of 8 */
-		f->fmt.pix.bytesperline = round_up(f->fmt.pix.width, 8);
+		/* Frame stride must be multiple of 8, but 16 for h.264 */
+		f->fmt.pix.bytesperline = round_up(f->fmt.pix.width, 16);
 		f->fmt.pix.sizeimage = f->fmt.pix.bytesperline *
 					f->fmt.pix.height * 3 / 2;
 		break;
-- 
2.0.0.rc2

[PATCH 30/30] [media] coda: export auxiliary buffers via debugfs

[Philipp Zabel]

This patch exports all auxiliary buffers, including SRAM, as debugfs binary
blobs for debugging purposes. It shows, for example, that psbuf currently
doesn't seem to be used at all on CODA7541, and that slicebuf and workbuf
usage is far from the maximum. It can also be used to validate SRAM size
allocation.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/coda.c | 64 +++++++++++++++++++++++++++++++++++--------
 1 file changed, 53 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/coda.c b/drivers/media/platform/coda.c
index aabd639d..0b90087 100644
--- a/drivers/media/platform/coda.c
+++ b/drivers/media/platform/coda.c
@@ -12,6 +12,7 @@
  */
 
 #include <linux/clk.h>
+#include <linux/debugfs.h>
 #include <linux/delay.h>
 #include <linux/firmware.h>
 #include <linux/genalloc.h>
@@ -129,6 +130,8 @@ struct coda_aux_buf {
 	void			*vaddr;
 	dma_addr_t		paddr;
 	u32			size;
+	struct debugfs_blob_wrapper blob;
+	struct dentry		*dentry;
 };
 
 struct coda_dev {
@@ -156,6 +159,7 @@ struct coda_dev {
 	struct vb2_alloc_ctx	*alloc_ctx;
 	struct list_head	instances;
 	unsigned long		instance_mask;
+	struct dentry		*debugfs_root;
 };
 
 struct coda_params {
@@ -259,6 +263,7 @@ struct coda_ctx {
 	u32				frm_dis_flg;
 	u32				frame_mem_ctrl;
 	int				display_idx;
+	struct dentry			*debugfs_entry;
 };
 
 static const u8 coda_filler_nal[14] = { 0x00, 0x00, 0x00, 0x01, 0x0c, 0xff,
@@ -1758,7 +1763,8 @@ static void coda_parabuf_write(struct coda_ctx *ctx, int index, u32 value)
 }
 
 static int coda_alloc_aux_buf(struct coda_dev *dev,
-			      struct coda_aux_buf *buf, size_t size)
+			      struct coda_aux_buf *buf, size_t size,
+			      const char *name, struct dentry *parent)
 {
 	buf->vaddr = dma_alloc_coherent(&dev->plat_dev->dev, size, &buf->paddr,
 					GFP_KERNEL);
@@ -1767,13 +1773,23 @@ static int coda_alloc_aux_buf(struct coda_dev *dev,
 
 	buf->size = size;
 
+	if (name && parent) {
+		buf->blob.data = buf->vaddr;
+		buf->blob.size = size;
+		buf->dentry = debugfs_create_blob(name, 0644, parent, &buf->blob);
+		if (!buf->dentry)
+			dev_warn(&dev->plat_dev->dev,
+				 "failed to create debugfs entry %s\n", name);
+	}
+
 	return 0;
 }
 
 static inline int coda_alloc_context_buf(struct coda_ctx *ctx,
-					 struct coda_aux_buf *buf, size_t size)
+					 struct coda_aux_buf *buf, size_t size,
+					 const char *name)
 {
-	return coda_alloc_aux_buf(ctx->dev, buf, size);
+	return coda_alloc_aux_buf(ctx->dev, buf, size, name, ctx->debugfs_entry);
 }
 
 static void coda_free_aux_buf(struct coda_dev *dev,
@@ -1785,6 +1801,7 @@ static void coda_free_aux_buf(struct coda_dev *dev,
 		buf->vaddr = NULL;
 		buf->size = 0;
 	}
+	debugfs_remove(buf->dentry);
 }
 
 static void coda_free_framebuffers(struct coda_ctx *ctx)
@@ -1817,12 +1834,16 @@ static int coda_alloc_framebuffers(struct coda_ctx *ctx, struct coda_q_data *q_d
 	/* Allocate frame buffers */
 	for (i = 0; i < ctx->num_internal_frames; i++) {
 		size_t size;
+		char *name;
 
 		size = ysize + ysize / 2;
 		if (ctx->codec->src_fourcc == V4L2_PIX_FMT_H264 &&
 		    dev->devtype->product != CODA_DX6)
 			size += ysize / 4;
-		ret = coda_alloc_context_buf(ctx, &ctx->internal_frames[i], size);
+		name = kasprintf(GFP_KERNEL, "fb%d", i);
+		ret = coda_alloc_context_buf(ctx, &ctx->internal_frames[i],
+					     size, name);
+		kfree(name);
 		if (ret < 0) {
 			coda_free_framebuffers(ctx);
 			return ret;
@@ -2046,7 +2067,7 @@ static int coda_alloc_context_buffers(struct coda_ctx *ctx,
 		/* worst case slice size */
 		size = (DIV_ROUND_UP(q_data->width, 16) *
 			DIV_ROUND_UP(q_data->height, 16)) * 3200 / 8 + 512;
-		ret = coda_alloc_context_buf(ctx, &ctx->slicebuf, size);
+		ret = coda_alloc_context_buf(ctx, &ctx->slicebuf, size, "slicebuf");
 		if (ret < 0) {
 			v4l2_err(&dev->v4l2_dev, "failed to allocate %d byte slice buffer",
 				 ctx->slicebuf.size);
@@ -2055,14 +2076,14 @@ static int coda_alloc_context_buffers(struct coda_ctx *ctx,
 	}
 
 	if (dev->devtype->product == CODA_7541) {
-		ret = coda_alloc_context_buf(ctx, &ctx->psbuf, CODA7_PS_BUF_SIZE);
+		ret = coda_alloc_context_buf(ctx, &ctx->psbuf, CODA7_PS_BUF_SIZE, "psbuf");
 		if (ret < 0) {
 			v4l2_err(&dev->v4l2_dev, "failed to allocate psmem buffer");
 			goto err;
 		}
 	}
 
-	ret = coda_alloc_context_buf(ctx, &ctx->workbuf, size);
+	ret = coda_alloc_context_buf(ctx, &ctx->workbuf, size, "workbuf");
 	if (ret < 0) {
 		v4l2_err(&dev->v4l2_dev, "failed to allocate %d byte context buffer",
 			 ctx->workbuf.size);
@@ -2948,6 +2969,7 @@ static int coda_open(struct file *file)
 {
 	struct coda_dev *dev = video_drvdata(file);
 	struct coda_ctx *ctx = NULL;
+	char *name;
 	int ret;
 	int idx;
 
@@ -2962,6 +2984,10 @@ static int coda_open(struct file *file)
 	}
 	set_bit(idx, &dev->instance_mask);
 
+	name = kasprintf(GFP_KERNEL, "context%d", idx);
+	ctx->debugfs_entry = debugfs_create_dir(name, dev->debugfs_root);
+	kfree(name);
+
 	init_completion(&ctx->completion);
 	INIT_WORK(&ctx->pic_run_work, coda_pic_run_work);
 	INIT_WORK(&ctx->seq_end_work, coda_seq_end_work);
@@ -3013,7 +3039,8 @@ static int coda_open(struct file *file)
 
 	ctx->fh.ctrl_handler = &ctx->ctrls;
 
-	ret = coda_alloc_context_buf(ctx, &ctx->parabuf, CODA_PARA_BUF_SIZE);
+	ret = coda_alloc_context_buf(ctx, &ctx->parabuf, CODA_PARA_BUF_SIZE,
+				     "parabuf");
 	if (ret < 0) {
 		v4l2_err(&dev->v4l2_dev, "failed to allocate parabuf");
 		goto err_dma_alloc;
@@ -3074,6 +3101,8 @@ static int coda_release(struct file *file)
 	v4l2_dbg(1, coda_debug, &dev->v4l2_dev, "Releasing instance %p\n",
 		 ctx);
 
+	debugfs_remove_recursive(ctx->debugfs_entry);
+
 	/* If this instance is running, call .job_abort and wait for it to end */
 	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
 
@@ -3606,7 +3635,8 @@ static void coda_fw_callback(const struct firmware *fw, void *context)
 	}
 
 	/* allocate auxiliary per-device code buffer for the BIT processor */
-	ret = coda_alloc_aux_buf(dev, &dev->codebuf, fw->size);
+	ret = coda_alloc_aux_buf(dev, &dev->codebuf, fw->size, "codebuf",
+				 dev->debugfs_root);
 	if (ret < 0) {
 		dev_err(&pdev->dev, "failed to allocate code buffer\n");
 		return;
@@ -3842,11 +3872,16 @@ static int coda_probe(struct platform_device *pdev)
 		return -EINVAL;
 	}
 
+	dev->debugfs_root = debugfs_create_dir("coda", NULL);
+	if (!dev->debugfs_root)
+		dev_warn(&pdev->dev, "failed to create debugfs root\n");
+
 	/* allocate auxiliary per-device buffers for the BIT processor */
 	switch (dev->devtype->product) {
 	case CODA_DX6:
 		ret = coda_alloc_aux_buf(dev, &dev->workbuf,
-					 CODADX6_WORK_BUF_SIZE);
+					 CODADX6_WORK_BUF_SIZE, "workbuf",
+					 dev->debugfs_root);
 		if (ret < 0) {
 			dev_err(&pdev->dev, "failed to allocate work buffer\n");
 			v4l2_device_unregister(&dev->v4l2_dev);
@@ -3862,7 +3897,8 @@ static int coda_probe(struct platform_device *pdev)
 	}
 	if (dev->tempbuf.size) {
 		ret = coda_alloc_aux_buf(dev, &dev->tempbuf,
-					 dev->tempbuf.size);
+					 dev->tempbuf.size, "tempbuf",
+					 dev->debugfs_root);
 		if (ret < 0) {
 			dev_err(&pdev->dev, "failed to allocate temp buffer\n");
 			v4l2_device_unregister(&dev->v4l2_dev);
@@ -3887,6 +3923,11 @@ static int coda_probe(struct platform_device *pdev)
 		return -ENOMEM;
 	}
 
+	dev->iram.blob.data = dev->iram.vaddr;
+	dev->iram.blob.size = dev->iram.size;
+	dev->iram.dentry = debugfs_create_blob("iram", 0644, dev->debugfs_root,
+					       &dev->iram.blob);
+
 	dev->workqueue = alloc_workqueue("coda", WQ_UNBOUND | WQ_MEM_RECLAIM, 1);
 	if (!dev->workqueue) {
 		dev_err(&pdev->dev, "unable to alloc workqueue\n");
@@ -3918,6 +3959,7 @@ static int coda_remove(struct platform_device *pdev)
 	coda_free_aux_buf(dev, &dev->codebuf);
 	coda_free_aux_buf(dev, &dev->tempbuf);
 	coda_free_aux_buf(dev, &dev->workbuf);
+	debugfs_remove_recursive(dev->debugfs_root);
 	return 0;
 }
 
-- 
2.0.0.rc2

Re: [PATCH 00/30] Initial CODA960 (i.MX6 VPU) support

[Hans Verkuil]

Hi Philipp,

I went through this patch series and replied with some comments.

I have two more general questions:

1) can you post the output of 'v4l2-compliance'?
2) what would be needed for 'v4l2-compliance -s' to work?

For the encoder 'v4l2-compliance -s' will probably work OK, but for
the decoder you need to feed v4l2-compliance -s some compressed
stream. I assume each buffer should contain a single P/B/I frame?

The v4l2-ctl utility has already support for writing captured data
to a file, but it has no support to store the image sizes as well.
So if the captured buffers do not all have the same size you cannot
'index' the captured file. If I would add support for that, then I
can add support for it to v4l2-compliance as well, allowing you to
playback an earlier captured compressed video stream and use that
as the compliance test input.

Does this makes sense?

Regards,

	Hans

On 06/13/2014 06:08 PM, Philipp Zabel wrote:
> Hi,
> 
> the following series adds initial support for the CODA960 Video
> Processing Unit on i.MX6Q/D/DL/S SoCs to the coda driver.
> 
> This series contains a few fixes and preparations, the CODA960
> support patch, a rework of the hardware access serialization
> into a single threaded workqueue, some cleanups to use more
> infrastructure that is available in the meantime, runtime PM
> support, a few h.264 related v4l2 controls and fixes, support
> for hard resets via the i.MX system reset controller, and a
> patch that exports internal buffers to debugfs.
> 
> regards
> Philipp
> 
> Michael Olbrich (2):
>   [media] v4l2-mem2mem: export v4l2_m2m_try_schedule
>   [media] coda: try to schedule a decode run after a stop command
> 
> Philipp Zabel (28):
>   [media] coda: fix decoder I/P/B frame detection
>   [media] coda: fix readback of CODA_RET_DEC_SEQ_FRAME_NEED
>   [media] coda: fix h.264 quantization parameter range
>   [media] coda: fix internal framebuffer allocation size
>   [media] coda: simplify IRAM setup
>   [media] coda: Add encoder/decoder support for CODA960
>   [media] coda: add selection API support for h.264 decoder
>   [media] coda: add support for frame size enumeration
>   [media] coda: add workqueue to serialize hardware commands
>   [media] coda: Use mem-to-mem ioctl helpers
>   [media] coda: use ctx->fh.m2m_ctx instead of ctx->m2m_ctx
>   [media] coda: Add runtime pm support
>   [media] coda: split firmware version check out of coda_hw_init
>   [media] coda: select GENERIC_ALLOCATOR
>   [media] coda: add h.264 min/max qp controls
>   [media] coda: add h.264 deblocking filter controls
>   [media] coda: add cyclic intra refresh control
>   [media] coda: let userspace force IDR frames by enabling the keyframe
>     flag in the source buffer
>   [media] coda: add decoder timestamp queue
>   [media] coda: alert userspace about macroblock errors
>   [media] coda: add sequence counter offset
>   [media] coda: use prescan_failed variable to stop stream after a
>     timeout
>   [media] coda: add reset control support
>   [media] coda: add bytesperline to queue data
>   [media] coda: allow odd width, but still round up bytesperline
>   [media] coda: round up internal frames to multiples of macroblock size
>     for h.264
>   [media] coda: increase frame stride to 16 for h.264
>   [media] coda: export auxiliary buffers via debugfs
> 
>  drivers/media/platform/Kconfig         |    1 +
>  drivers/media/platform/coda.c          | 1505 +++++++++++++++++++++++---------
>  drivers/media/platform/coda.h          |  115 ++-
>  drivers/media/v4l2-core/v4l2-mem2mem.c |    3 +-
>  include/media/v4l2-mem2mem.h           |    2 +
>  5 files changed, 1197 insertions(+), 429 deletions(-)
> 

Re: [PATCH 00/30] Initial CODA960 (i.MX6 VPU) support

[Philipp Zabel]

Hi Hans,

Am Montag, den 16.06.2014, 10:35 +0200 schrieb Hans Verkuil:
> Hi Philipp,
> 
> I went through this patch series and replied with some comments.

thank you for the comments. I have dropped the force IDR patch in
v2 and will send a separate RFC for the VFU / forced keyframe
support.
I have also dropped the enum_framesizes patch for now.

> I have two more general questions:
> 
> 1) can you post the output of 'v4l2-compliance'?

This is for the v2 series, the previously posted patches still had
one TRY_FMT(G_FMT) != G_FMT error introduced by the "[media] coda:
add bytesperline to queue data" patch:

$ v4l2-compliance -d /dev/video8
Driver Info:
	Driver name   : coda
	Card type     : CODA960
	Bus info      : platform:coda
	Driver version: 3.16.0
	Capabilities  : 0x84008003
		Video Capture
		Video Output
		Video Memory-to-Memory
		Streaming
		Device Capabilities
	Device Caps   : 0x04008003
		Video Capture
		Video Output
		Video Memory-to-Memory
		Streaming

Compliance test for device /dev/video8 (not using libv4l2):

Required ioctls:
		warn: v4l2-compliance.cpp(366): VIDIOC_QUERYCAP: m2m with video input and output caps
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second video open: OK
		warn: v4l2-compliance.cpp(366): VIDIOC_QUERYCAP: m2m with video input and output caps
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

	Control ioctls:
		test VIDIOC_QUERYCTRL/MENU: OK
		test VIDIOC_G/S_CTRL: OK
		test VIDIOC_G/S/TRY_EXT_CTRLS: OK
		test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
		test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
		Standard Controls: 19 Private Controls: 0

	Format ioctls:
		test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
		test VIDIOC_G/S_PARM: OK (Not Supported)
		test VIDIOC_G_FBUF: OK (Not Supported)
		test VIDIOC_G_FMT: OK
		test VIDIOC_TRY_FMT: OK
		test VIDIOC_S_FMT: OK
		test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)

	Codec ioctls:
		test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
		test VIDIOC_G_ENC_INDEX: OK (Not Supported)
		test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK

Total: 38, Succeeded: 38, Failed: 0, Warnings: 2

> 2) what would be needed for 'v4l2-compliance -s' to work?

I haven't looked at this in detail yet. v4l2-compliance -s curently fails:

Buffer ioctls:
		info: test buftype Video Capture
		info: test buftype Video Output
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test read/write: OK (Not Supported)
		fail: v4l2-test-buffers.cpp(859): ret != EINVAL
	test MMAP: FAIL
		fail: v4l2-test-buffers.cpp(936): buf.qbuf(q)
		fail: v4l2-test-buffers.cpp(976): setupUserPtr(node, q)
	test USERPTR: FAIL
	test DMABUF: Cannot test, specify --expbuf-device

In principle the h.264 encoder should work, as you can just feed it
one frame at a time and then pick up the encoded result on the capture
side.

> For the encoder 'v4l2-compliance -s' will probably work OK, but for
> the decoder you need to feed v4l2-compliance -s some compressed
> stream. I assume each buffer should contain a single P/B/I frame?

Yes, for h.264 we currently expect all NAL units for a complete frame
in the source buffers.

> The v4l2-ctl utility has already support for writing captured data
> to a file, but it has no support to store the image sizes as well.
> So if the captured buffers do not all have the same size you cannot
> 'index' the captured file. If I would add support for that, then I
> can add support for it to v4l2-compliance as well, allowing you to
> playback an earlier captured compressed video stream and use that
> as the compliance test input.
> 
> Does this makes sense?

Wouldn't that mean that you had to add a stream parser for every
supported compressed format? Or are you planning to store an index
separately?

regards
Philipp

Re: [PATCH 00/30] Initial CODA960 (i.MX6 VPU) support

[Hans Verkuil]

On 06/24/2014 05:07 PM, Philipp Zabel wrote:
> Hi Hans,
>
> Am Montag, den 16.06.2014, 10:35 +0200 schrieb Hans Verkuil:
>> Hi Philipp,
>>
>> I went through this patch series and replied with some comments.
>
> thank you for the comments. I have dropped the force IDR patch in
> v2 and will send a separate RFC for the VFU / forced keyframe
> support.
> I have also dropped the enum_framesizes patch for now.
>
>> I have two more general questions:
>>
>> 1) can you post the output of 'v4l2-compliance'?
>
> This is for the v2 series, the previously posted patches still had
> one TRY_FMT(G_FMT) != G_FMT error introduced by the "[media] coda:
> add bytesperline to queue data" patch:
>
> $ v4l2-compliance -d /dev/video8
> Driver Info:
> 	Driver name   : coda
> 	Card type     : CODA960
> 	Bus info      : platform:coda
> 	Driver version: 3.16.0
> 	Capabilities  : 0x84008003
> 		Video Capture
> 		Video Output
> 		Video Memory-to-Memory

This is wrong, m2m devices should only set the VIDEO_M2M capability, it shouldn't be combined with
CAPTURE and OUTPUT.

> 		Streaming
> 		Device Capabilities
> 	Device Caps   : 0x04008003
> 		Video Capture
> 		Video Output
> 		Video Memory-to-Memory
> 		Streaming
>
> Compliance test for device /dev/video8 (not using libv4l2):
>
> Required ioctls:
> 		warn: v4l2-compliance.cpp(366): VIDIOC_QUERYCAP: m2m with video input and output caps

This should be an error, not a warning. I'll update that in v4l2-compliance.

In the very beginning when m2m devices were introduced they were marked as capture+output
devices, but some applications scan video devices for those that have the CAPTURE cap set
(quite reasonable), and they would also match such m2m devices. Quite soon we realized
that this was a problem and we introduced the m2m cap.

> 	test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> 	test second video open: OK
> 		warn: v4l2-compliance.cpp(366): VIDIOC_QUERYCAP: m2m with video input and output caps
> 	test VIDIOC_QUERYCAP: OK
> 	test VIDIOC_G/S_PRIORITY: OK
>
> Debug ioctls:
> 	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> 	test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> 	test VIDIOC_G/S_TUNER: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> 	test VIDIOC_ENUMAUDIO: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDIO: OK (Not Supported)
> 	Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> 	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> 	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> 	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> 	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> 	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> 	Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> 	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> 	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> 	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> 	test VIDIOC_G/S_EDID: OK (Not Supported)
>
> 	Control ioctls:
> 		test VIDIOC_QUERYCTRL/MENU: OK
> 		test VIDIOC_G/S_CTRL: OK
> 		test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> 		test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> 		test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> 		Standard Controls: 19 Private Controls: 0
>
> 	Format ioctls:
> 		test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> 		test VIDIOC_G/S_PARM: OK (Not Supported)
> 		test VIDIOC_G_FBUF: OK (Not Supported)
> 		test VIDIOC_G_FMT: OK
> 		test VIDIOC_TRY_FMT: OK
> 		test VIDIOC_S_FMT: OK
> 		test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>
> 	Codec ioctls:
> 		test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> 		test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> 		test VIDIOC_(TRY_)DECODER_CMD: OK
>
> Buffer ioctls:
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test VIDIOC_EXPBUF: OK
>
> Total: 38, Succeeded: 38, Failed: 0, Warnings: 2
>
>> 2) what would be needed for 'v4l2-compliance -s' to work?
>
> I haven't looked at this in detail yet. v4l2-compliance -s curently fails:
>
> Buffer ioctls:
> 		info: test buftype Video Capture
> 		info: test buftype Video Output
> 	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> 	test VIDIOC_EXPBUF: OK
> 	test read/write: OK (Not Supported)
> 		fail: v4l2-test-buffers.cpp(859): ret != EINVAL

This test tries to create a buffer with a sizeimage that is only half of
what the current format is. It expects an error based on the assumption
that this driver cannot change format mid-stream. If this is the case
for your driver, then you need to put a sanity check in queue_setup, if
this is allowed for your driver, then try commenting out this check.

> 	test MMAP: FAIL
> 		fail: v4l2-test-buffers.cpp(936): buf.qbuf(q)
> 		fail: v4l2-test-buffers.cpp(976): setupUserPtr(node, q)
> 	test USERPTR: FAIL

This is a real bug: you added VB2_USERPTR to the io_modes field of the
vb2 queues, but you are using videobuf2-dma-contig.h, which make userptr
support impossible since that requires scatter-gather DMA. Just drop the
VB2_USERPTR from io_modes.

> 	test DMABUF: Cannot test, specify --expbuf-device
>
> In principle the h.264 encoder should work, as you can just feed it
> one frame at a time and then pick up the encoded result on the capture
> side.
>
>> For the encoder 'v4l2-compliance -s' will probably work OK, but for
>> the decoder you need to feed v4l2-compliance -s some compressed
>> stream. I assume each buffer should contain a single P/B/I frame?
>
> Yes, for h.264 we currently expect all NAL units for a complete frame
> in the source buffers.
>
>> The v4l2-ctl utility has already support for writing captured data
>> to a file, but it has no support to store the image sizes as well.
>> So if the captured buffers do not all have the same size you cannot
>> 'index' the captured file. If I would add support for that, then I
>> can add support for it to v4l2-compliance as well, allowing you to
>> playback an earlier captured compressed video stream and use that
>> as the compliance test input.
>>
>> Does this makes sense?
>
> Wouldn't that mean that you had to add a stream parser for every
> supported compressed format? Or are you planning to store an index
> separately?

Most likely I would store a separate index file.

Regards,

	Hans