[RFC PATCH 00/26] i.MX5/6 IPUv3 CSI/IC

[RFC PATCH 00/26] i.MX5/6 IPUv3 CSI/IC

[Philipp Zabel]

Hi,

attached is a series of our work in progress i.MX6 capture drivers.
I'm posting this now in reaction to Steve's i.MX6 Video capture series,
as a reference for further discussion.
Of the Image Converter (IC) we only use the postprocessor task, with
tiling for larger frames, to implement v4l2 mem2mem scaler/colorspace
converter and deinterlacer devices.
The capture code capture code already uses the media controller framework
and creates a subdevice representing the CSI, but the path to memory is
fixed to IDMAC via SMFC, which is the only possible path for grayscale
and  and anything with multiple output ports connected
to the CSIs (such as the CSI2IPU gasket on i.MX6) doesn't work yet. Also,
I think the CSI subdevice driver should be completely separate from the
capture driver.

regards
Philipp

Philipp Zabel (16):
  gpu: ipu-v3: Add function to setup CP channel as interlaced
  gpu: ipu-v3: Add ipu_cpmem_get_buffer function
  gpu: ipu-v3: Add support for partial interleaved YCbCr 4:2:0 (NV12)
    format
  gpu: ipu-v3: Add support for planar YUV 4:2:2 (YUV422P) format
  imx-drm: currently only IPUv3 is supported, make it mandatory
  [media] Add i.MX SoC wide media device driver
  [media] imx-ipu: Add i.MX IPUv3 capture driver
  [media] ipuv3-csi: Skip 3 lines for NTSC BT.656
  [media] imx-ipuv3-csi: Add support for temporarily stopping the stream
    on sync loss
  [media] imx-ipuv3-csi: Export sync lock event to userspace
  [media] v4l2-subdev.h: Add lock status notification
  [media] v4l2-subdev: Export v4l2_subdev_fops
  mfd: syscon: add child device support
  [media] imx: Add video switch
  ARM: dts: Add IPU aliases on i.MX6
  ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
    connections

Sascha Hauer (10):
  gpu: ipu-v3: Add IC support
  gpu: ipu-v3: Register IC with IPUv3
  [media] imx-ipu: add ipu media common code
  [media] imx-ipu: Add i.MX IPUv3 scaler driver
  [media] imx-ipu: Add i.MX IPUv3 deinterlacer driver
  [media] v4l2: subdev: Add v4l2_device_register_subdev_node function
  [media] v4l2: Fix V4L2_CID_PIXEL_RATE
  [media] v4l2 async: remove from notifier list
  [media] ipuv3-csi: Pass ipucsi to v4l2_media_subdev_s_power
  [media] ipuv3-csi: make subdev controls available on video device

 Documentation/devicetree/bindings/mfd/syscon.txt |   11 +
 arch/arm/boot/dts/imx6dl.dtsi                    |  182 +++
 arch/arm/boot/dts/imx6q.dtsi                     |  119 ++
 arch/arm/boot/dts/imx6qdl.dtsi                   |    9 +
 drivers/gpu/ipu-v3/Makefile                      |    2 +-
 drivers/gpu/ipu-v3/ipu-common.c                  |  119 ++
 drivers/gpu/ipu-v3/ipu-ic.c                      | 1227 +++++++++++++++
 drivers/gpu/ipu-v3/ipu-prv.h                     |    6 +
 drivers/media/platform/Kconfig                   |    4 +
 drivers/media/platform/Makefile                  |    1 +
 drivers/media/platform/imx/Kconfig               |   50 +
 drivers/media/platform/imx/Makefile              |    6 +
 drivers/media/platform/imx/imx-ipu-scaler.c      |  825 +++++++++++
 drivers/media/platform/imx/imx-ipu-vdic.c        |  716 +++++++++
 drivers/media/platform/imx/imx-ipu.c             |  313 ++++
 drivers/media/platform/imx/imx-ipu.h             |   36 +
 drivers/media/platform/imx/imx-ipuv3-csi.c       | 1729 ++++++++++++++++++++++
 drivers/media/platform/imx/imx-media.c           |  174 +++
 drivers/media/platform/imx/imx-video-switch.c    |  347 +++++
 drivers/media/v4l2-core/v4l2-async.c             |    1 +
 drivers/media/v4l2-core/v4l2-ctrls.c             |    8 +-
 drivers/media/v4l2-core/v4l2-device.c            |   63 +-
 drivers/media/v4l2-core/v4l2-subdev.c            |    1 +
 drivers/mfd/syscon.c                             |    3 +
 drivers/staging/imx-drm/Kconfig                  |    7 +-
 include/media/imx.h                              |   25 +
 include/media/v4l2-device.h                      |    5 +
 include/media/v4l2-subdev.h                      |    3 +
 include/video/imx-ipu-v3.h                       |   16 +
 29 files changed, 5976 insertions(+), 32 deletions(-)
 create mode 100644 drivers/gpu/ipu-v3/ipu-ic.c
 create mode 100644 drivers/media/platform/imx/Kconfig
 create mode 100644 drivers/media/platform/imx/Makefile
 create mode 100644 drivers/media/platform/imx/imx-ipu-scaler.c
 create mode 100644 drivers/media/platform/imx/imx-ipu-vdic.c
 create mode 100644 drivers/media/platform/imx/imx-ipu.c
 create mode 100644 drivers/media/platform/imx/imx-ipu.h
 create mode 100644 drivers/media/platform/imx/imx-ipuv3-csi.c
 create mode 100644 drivers/media/platform/imx/imx-media.c
 create mode 100644 drivers/media/platform/imx/imx-video-switch.c
 create mode 100644 include/media/imx.h

-- 
2.0.0.rc2

[RFC PATCH 01/26] gpu: ipu-v3: Add IC support

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

This patch adds support for the IC module used for scaling, rotation,
and colorspace conversions.
Scaling images larger than 1024x1024 is supported by tiling over multiple
IC scaling tasks. Instead of scaling tiles separately, which causes an ugly
seam at the edge, use a single scaling coefficient and write them with
overlap due to the 8-pixel burst size. Due to overlap, tiles have to be
rendered right to left and bottom to top. Up to 7 pixels (depending on
the scaling factor) have to be available to be written after the end of
the frame.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/gpu/ipu-v3/Makefile |    2 +-
 drivers/gpu/ipu-v3/ipu-ic.c | 1227 +++++++++++++++++++++++++++++++++++++++++++
 include/video/imx-ipu-v3.h  |    6 +
 3 files changed, 1234 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/ipu-v3/ipu-ic.c

diff --git a/drivers/gpu/ipu-v3/Makefile b/drivers/gpu/ipu-v3/Makefile
index 1887972b..2c4d1ef 100644
--- a/drivers/gpu/ipu-v3/Makefile
+++ b/drivers/gpu/ipu-v3/Makefile
@@ -1,3 +1,3 @@
 obj-$(CONFIG_IMX_IPUV3_CORE) += imx-ipu-v3.o
 
-imx-ipu-v3-objs := ipu-common.o ipu-dc.o ipu-di.o ipu-dp.o ipu-dmfc.o ipu-smfc.o
+imx-ipu-v3-objs := ipu-common.o ipu-dc.o ipu-di.o ipu-dp.o ipu-dmfc.o ipu-smfc.o ipu-ic.o
diff --git a/drivers/gpu/ipu-v3/ipu-ic.c b/drivers/gpu/ipu-v3/ipu-ic.c
new file mode 100644
index 0000000..778751f
--- /dev/null
+++ b/drivers/gpu/ipu-v3/ipu-ic.c
@@ -0,0 +1,1227 @@
+/*
+ * Copyright (c) 2012 Sascha Hauer <s.hauer@pengutronix.de>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * for more details.
+ */
+#include <linux/types.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/errno.h>
+#include <linux/export.h>
+#include <linux/spinlock.h>
+#include <linux/io.h>
+#include <linux/bitrev.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <video/imx-ipu-v3.h>
+
+#include "ipu-prv.h"
+
+enum {
+	IC_TASK_VIEWFINDER,
+	IC_TASK_ENCODER,
+	IC_TASK_POST_PROCESSOR,
+	IC_TASK_MAX,
+};
+
+struct image_convert_ctx {
+	void (*complete)(void *ctx, int err);
+	void *complete_context;
+
+	struct list_head list;
+	struct ipu_image in;
+	struct ipu_image in_n;
+	struct ipu_image in_p;
+	struct ipu_image out;
+
+	void *freep;
+
+	unsigned int rotate:1;
+	unsigned int deinterlace:1;
+
+	u32 rsc;
+};
+
+struct ipu_ic_task {
+	int id;
+	struct ipu_ic_priv *priv;
+	void __iomem *reg_rsc;
+	u32 csc1_mask;
+	u32 csc2_mask;
+	u32 cmb_mask;
+	void __iomem *tpm_base_csc1;
+	void __iomem *tpm_base_csc2;
+	u32 ic_conf_rot_en;
+	u32 ic_conf_en;
+	int first_deinterlace_frame;
+	struct ipuv3_channel *input_channel_p;
+	struct ipuv3_channel *input_channel;
+	struct ipuv3_channel *input_channel_n;
+	struct ipuv3_channel *output_channel;
+	struct ipuv3_channel *rotation_input_channel;
+	struct ipuv3_channel *rotation_output_channel;
+
+	struct list_head image_list;
+
+	struct workqueue_struct *workqueue;
+	struct work_struct work;
+	struct completion complete;
+};
+
+struct ipu_ic_priv {
+	void __iomem *ic_base;
+	void __iomem *tpm_base;
+	void __iomem *vdi_base;
+	struct device *dev;
+	struct ipu_soc *ipu;
+	struct mutex mutex;
+	struct ipu_ic_task task[IC_TASK_MAX];
+	spinlock_t lock;
+	int ic_usecount;
+};
+
+#define IC_CONF			0x0
+#define IC_PRP_ENC_RSC		0x4
+#define IC_PRP_VF_RSC		0x8
+#define IC_PP_RSC		0xC
+#define IC_CMBP_1		0x10
+#define IC_CMBP_2		0x14
+#define IC_IDMAC_1		0x18
+#define IC_IDMAC_2		0x1C
+#define IC_IDMAC_3		0x20
+#define IC_IDMAC_4		0x24
+
+/* Image Converter Register bits */
+#define	IC_CONF_PRPENC_EN	0x00000001
+#define	IC_CONF_PRPENC_CSC1	0x00000002
+#define	IC_CONF_PRPENC_ROT_EN	0x00000004
+#define	IC_CONF_PRPVF_EN	0x00000100
+#define	IC_CONF_PRPVF_CSC1	0x00000200
+#define	IC_CONF_PRPVF_CSC2	0x00000400
+#define	IC_CONF_PRPVF_CMB	0x00000800
+#define	IC_CONF_PRPVF_ROT_EN	0x00001000
+#define	IC_CONF_PP_EN		0x00010000
+#define	IC_CONF_PP_CSC1		0x00020000
+#define	IC_CONF_PP_CSC2		0x00040000
+#define	IC_CONF_PP_CMB		0x00080000
+#define	IC_CONF_PP_ROT_EN	0x00100000
+#define	IC_CONF_IC_GLB_LOC_A	0x10000000
+#define	IC_CONF_KEY_COLOR_EN	0x20000000
+#define	IC_CONF_RWS_EN		0x40000000
+#define	IC_CONF_CSI_MEM_WR_EN	0x80000000
+
+struct ipu_ic_task_template {
+	unsigned long ofs_rsc;
+	u32 csc1_mask;
+	u32 csc2_mask;
+	u32 cmb_mask;
+	u32 tpm_ofs_csc1;
+	u32 tpm_ofs_csc2;
+	int rotation_input_channel;
+	int rotation_output_channel;
+	int input_channel_p;
+	int input_channel;
+	int input_channel_n;
+	int output_channel;
+	u32 ic_conf_rot_en;
+	u32 ic_conf_en;
+};
+
+struct ipu_ic_task_template templates[] = {
+	{
+		.ofs_rsc = IC_PRP_VF_RSC,
+		.csc1_mask = IC_CONF_PRPVF_CSC1,
+		.csc2_mask = IC_CONF_PRPVF_CSC2,
+		.cmb_mask = IC_CONF_PRPVF_CMB,
+		.tpm_ofs_csc1 = 0x4028,
+		.tpm_ofs_csc2 = 0x4040,
+		.rotation_input_channel = 46,
+		.rotation_output_channel = 49,
+		.input_channel_p = 8,
+		.input_channel = 9,
+		.input_channel_n = 10,
+		.output_channel = 21,
+		.ic_conf_rot_en = IC_CONF_PRPVF_ROT_EN,
+		.ic_conf_en = IC_CONF_PRPVF_EN,
+	}, {
+		.ofs_rsc = IC_PRP_ENC_RSC,
+		.csc1_mask = IC_CONF_PRPENC_CSC1,
+		.csc2_mask = 0,
+		.cmb_mask = 0,
+		.tpm_ofs_csc1 = 0x2008,
+		.tpm_ofs_csc2 = 0x0,
+		.rotation_input_channel = 45,
+		.rotation_output_channel = 48,
+		.input_channel = 12,
+		.output_channel = 20,
+		.ic_conf_rot_en = IC_CONF_PRPENC_ROT_EN,
+		.ic_conf_en = IC_CONF_PRPENC_EN,
+	}, {
+		.ofs_rsc = IC_PP_RSC,
+		.csc1_mask = IC_CONF_PP_CSC1,
+		.csc2_mask = IC_CONF_PP_CSC2,
+		.cmb_mask = IC_CONF_PP_CMB,
+		.tpm_ofs_csc1 = 0x6060,
+		.tpm_ofs_csc2 = 0x6078,
+		.rotation_input_channel = 47,
+		.rotation_output_channel = 50,
+		.input_channel = 11,
+		.output_channel = 22,
+		.ic_conf_rot_en = IC_CONF_PP_ROT_EN,
+		.ic_conf_en = IC_CONF_PP_EN,
+	},
+};
+
+#define	IC_IDMAC_1_CB0_BURST_16 0x00000001
+#define	IC_IDMAC_1_CB1_BURST_16 0x00000002
+#define	IC_IDMAC_1_CB2_BURST_16 0x00000004
+#define	IC_IDMAC_1_CB3_BURST_16 0x00000008
+#define	IC_IDMAC_1_CB4_BURST_16 0x00000010
+#define	IC_IDMAC_1_CB5_BURST_16 0x00000020
+#define	IC_IDMAC_1_CB6_BURST_16 0x00000040
+#define	IC_IDMAC_1_CB7_BURST_16 0x00000080
+#define	IC_IDMAC_1_PRPENC_ROT_MASK 0x00003800
+#define	IC_IDMAC_1_PRPENC_ROT_OFFSET 11
+#define	IC_IDMAC_1_PRPVF_ROT_MASK 0x0001C000
+#define	IC_IDMAC_1_PRPVF_ROT_OFFSET 14
+#define	IC_IDMAC_1_PP_ROT_MASK 0x000E0000
+#define	IC_IDMAC_1_PP_ROT_OFFSET 17
+#define	IC_IDMAC_1_PP_FLIP_RS 0x00400000
+#define	IC_IDMAC_1_PRPVF_FLIP_RS 0x00200000
+#define	IC_IDMAC_1_PRPENC_FLIP_RS 0x00100000
+
+#define	IC_IDMAC_2_PRPENC_HEIGHT_MASK 0x000003FF
+#define	IC_IDMAC_2_PRPENC_HEIGHT_OFFSET 0
+#define	IC_IDMAC_2_PRPVF_HEIGHT_MASK 0x000FFC00
+#define	IC_IDMAC_2_PRPVF_HEIGHT_OFFSET 10
+#define	IC_IDMAC_2_PP_HEIGHT_MASK 0x3FF00000
+#define	IC_IDMAC_2_PP_HEIGHT_OFFSET 20
+
+#define	IC_IDMAC_3_PRPENC_WIDTH_MASK 0x000003FF
+#define	IC_IDMAC_3_PRPENC_WIDTH_OFFSET 0
+#define	IC_IDMAC_3_PRPVF_WIDTH_MASK 0x000FFC00
+#define	IC_IDMAC_3_PRPVF_WIDTH_OFFSET 10
+#define	IC_IDMAC_3_PP_WIDTH_MASK 0x3FF00000
+#define	IC_IDMAC_3_PP_WIDTH_OFFSET 20
+
+enum ipu_rotate_mode {
+	/* Note the enum values correspond to BAM value */
+	IPU_ROTATE_NONE = 0,
+	IPU_ROTATE_VERT_FLIP = 1,
+	IPU_ROTATE_HORIZ_FLIP = 2,
+	IPU_ROTATE_180 = 3,
+	IPU_ROTATE_90_RIGHT = 4,
+	IPU_ROTATE_90_RIGHT_VFLIP = 5,
+	IPU_ROTATE_90_RIGHT_HFLIP = 6,
+	IPU_ROTATE_90_LEFT = 7,
+};
+
+struct ic_setup {
+	u32 in_width;
+	u32 in_height;
+	enum ipu_color_space in_fmt;
+	u32 out_width;
+	u32 out_height;
+	enum ipu_color_space out_fmt;
+	bool graphics_combine_en;
+	enum ipu_color_space in_g_fmt;
+	u32 rsc;
+};
+
+static u32 ipu_vdi_read(struct ipu_ic_priv *priv, unsigned offset)
+{
+	return readl(priv->vdi_base + offset);
+}
+
+static void ipu_vdi_write(struct ipu_ic_priv *priv, u32 value, unsigned offset)
+{
+	writel(value, priv->vdi_base + offset);
+}
+
+static void ipu_ic_enable(struct ipu_ic_priv *priv)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&priv->lock, flags);
+
+	if (!priv->ic_usecount)
+		ipu_module_enable(priv->ipu, IPU_CONF_IC_EN);
+
+	priv->ic_usecount++;
+
+	spin_unlock_irqrestore(&priv->lock, flags);
+}
+
+static void ipu_ic_disable(struct ipu_ic_priv *priv)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&priv->lock, flags);
+
+	priv->ic_usecount--;
+
+	BUG_ON(priv->ic_usecount < 0);
+
+	if (!priv->ic_usecount)
+		ipu_module_disable(priv->ipu, IPU_CONF_IC_EN);
+
+	spin_unlock_irqrestore(&priv->lock, flags);
+}
+
+static int ipu_ic_enable_task(struct ipu_ic_task *task,
+			      struct image_convert_ctx *ctx)
+{
+	struct ipu_ic_priv *priv = task->priv;
+	u32 ic_conf;
+
+	ipu_idmac_enable_channel(task->input_channel);
+	if (ctx->deinterlace) {
+		ipu_idmac_enable_channel(task->input_channel_n);
+		ipu_idmac_enable_channel(task->input_channel_p);
+	}
+	ipu_idmac_enable_channel(task->output_channel);
+
+	ic_conf = readl(priv->ic_base + IC_CONF);
+
+	if (ctx->rotate)
+		ic_conf |= task->ic_conf_rot_en;
+	ic_conf |= task->ic_conf_en;
+
+	writel(ic_conf, priv->ic_base + IC_CONF);
+
+	ipu_idmac_select_buffer(task->input_channel, 0);
+	if (ctx->deinterlace) {
+		ipu_idmac_select_buffer(task->input_channel_n, 0);
+		ipu_idmac_select_buffer(task->input_channel_p, 0);
+	}
+	ipu_idmac_select_buffer(task->output_channel, 0);
+
+	if (ctx->deinterlace && task->first_deinterlace_frame) {
+		/*
+		 * With the very first frame we have to trigger the vdic input
+		 * channel twice. I have no idea why. This may be a bug in the
+		 * IPU or in the code.
+		 */
+		mdelay(5);
+		ipu_idmac_select_buffer(task->input_channel, 0);
+		task->first_deinterlace_frame = 0;
+	}
+
+	return 0;
+}
+
+static int ipu_ic_disable_task(struct ipu_ic_task *task,
+			       struct image_convert_ctx *ctx)
+{
+	struct ipu_ic_priv *priv = task->priv;
+	u32 ic_conf;
+
+	ic_conf = readl(priv->ic_base + IC_CONF);
+
+	if (ctx->rotate)
+		ic_conf &= ~task->ic_conf_rot_en;
+	ic_conf &= ~task->ic_conf_en;
+
+	writel(ic_conf, priv->ic_base + IC_CONF);
+
+	ipu_idmac_disable_channel(task->input_channel);
+	if (ctx->deinterlace) {
+		ipu_idmac_disable_channel(task->input_channel_n);
+		ipu_idmac_disable_channel(task->input_channel_p);
+	}
+	ipu_idmac_disable_channel(task->output_channel);
+
+	return 0;
+}
+
+static void ipu_ic_set_csc_matrix(u32 __iomem *base, const u32 (*coeff)[3],
+				  int scale)
+{
+	writel((coeff[3][0] << 27) | (coeff[0][0] << 18) | (coeff[1][1] << 9) |
+	       coeff[2][2], base++);
+	/* sat = 0 */
+	writel((coeff[3][0] >> 5) | (scale << 8), base++);
+
+	writel((coeff[3][1] << 27) | (coeff[0][1] << 18) | (coeff[1][0] << 9) |
+	       coeff[2][0], base++);
+	writel(coeff[3][1] >> 5, base++);
+
+	writel((coeff[3][2] << 27) | (coeff[0][2] << 18) | (coeff[1][2] << 9) |
+	       coeff[2][1], base++);
+	writel(coeff[3][2] >> 5, base++);
+}
+
+static int ipu_ic_init_csc(struct ipu_ic_task *task,
+			   enum ipu_color_space in_format,
+			   enum ipu_color_space out_format, int csc_index)
+{
+	struct ipu_ic_priv *priv = task->priv;
+	u32 __iomem *base;
+
+	/*
+	 * Y = R *  .299 + G *  .587 + B *  .114
+	 * U = R * -.169 + G * -.332 + B *  .500 + 128.
+	 * V = R *  .500 + G * -.419 + B * -.0813 + 128.
+	 */
+	static const u32 rgb2ycbcr_coeff[4][3] = {
+		{0x004D, 0x0096, 0x001D},
+		{0x01D5, 0x01AB, 0x0080},
+		{0x0080, 0x0195, 0x01EB},
+		{0x0000, 0x0200, 0x0200},	/* A0, A1, A2 */
+	};
+
+	/*
+	 * transparent IPUV3_COLORSPACE_RGB -> IPUV3_COLORSPACE_RGB matrix
+	 * for combining
+	 */
+	static const u32 rgb2rgb_coeff[4][3] = {
+		{0x0080, 0x0000, 0x0000},
+		{0x0000, 0x0080, 0x0000},
+		{0x0000, 0x0000, 0x0080},
+		{0x0000, 0x0000, 0x0000},	/* A0, A1, A2 */
+	};
+
+	/*
+	 * R = (1.164 * (Y - 16)) + (1.596 * (Cr - 128))
+	 * G = (1.164 * (Y - 16)) - (0.392 * (Cb - 128)) - (0.813 * (Cr - 128))
+	 * B = (1.164 * (Y - 16)) + (2.017 * (Cb - 128)
+	 */
+	static const u32 ycbcr2rgb_coeff[4][3] = {
+		{149, 0, 204},
+		{149, 462, 408},
+		{149, 255, 0},
+		{8192 - 446, 266, 8192 - 554},	/* A0, A1, A2 */
+	};
+
+	if (csc_index == 1)
+		base = task->tpm_base_csc1;
+	else
+		base = task->tpm_base_csc2;
+
+	/* Init CSC */
+	if ((in_format == IPUV3_COLORSPACE_YUV) &&
+	    (out_format == IPUV3_COLORSPACE_RGB)) {
+		ipu_ic_set_csc_matrix(base, ycbcr2rgb_coeff, 2);
+	} else if ((in_format == IPUV3_COLORSPACE_RGB) &&
+		   (out_format == IPUV3_COLORSPACE_YUV)) {
+		ipu_ic_set_csc_matrix(base, rgb2ycbcr_coeff, 1);
+	} else if ((in_format == IPUV3_COLORSPACE_RGB) &&
+		   (out_format == IPUV3_COLORSPACE_RGB)) {
+		ipu_ic_set_csc_matrix(base, rgb2rgb_coeff, 2);
+	} else {
+		dev_err(priv->dev, "Unsupported color space conversion\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int ipu_ic_calc_resize_coeffs(struct ipu_ic_priv *priv, u32 in_size,
+				     u32 out_size, u32 *resize_coeff,
+				     u32 *downsize_coeff, int n_tiles)
+{
+	u32 temp_size;
+	u32 temp_downsize;
+
+	/* Input size cannot be more than 4096 */
+	/* Output size cannot be more than 1024 */
+	if ((in_size > 4096 * n_tiles) || (out_size > 1024 * n_tiles))
+		return -EINVAL;
+
+	/* Cannot downsize more than 8:1 */
+	if ((out_size << 3) < in_size)
+		return -EINVAL;
+
+	/* Compute downsizing coefficient */
+	/* Output of downsizing unit cannot be more than 1024 */
+	temp_downsize = 0;
+	temp_size = in_size;
+	while (((temp_size > 1024 * n_tiles) || (temp_size >= out_size * 2)) &&
+	       (temp_downsize < 2)) {
+		temp_size >>= 1;
+		temp_downsize++;
+	}
+	*downsize_coeff = temp_downsize;
+
+	/* compute resizing coefficient using the following equation:
+	   resizeCoeff = M*(SI -1)/(SO - 1)
+	   where M = 2^13, SI - input size, SO - output size    */
+	*resize_coeff = (8192L * (temp_size - 1)) / (out_size - 1);
+	if (*resize_coeff >= 16384L) {
+		dev_err(priv->dev, "Warning! Overflow on resize coeff.\n");
+		*resize_coeff = 0x3FFF;
+	}
+
+	return 0;
+}
+
+static int ipu_ic_init_task(struct ipu_ic_task *task, struct ic_setup *p,
+			    bool src_is_csi)
+{
+	struct ipu_ic_priv *priv = task->priv;
+	u32 ic_conf;
+	int ret;
+
+	writel(p->rsc, task->reg_rsc);
+
+	ic_conf = readl(priv->ic_base + IC_CONF);
+	ic_conf &= ~task->cmb_mask;
+
+	/* Setup color space conversion */
+	if (p->in_fmt != p->out_fmt) {
+		ret = ipu_ic_init_csc(task, p->in_fmt, p->out_fmt, 1);
+		if (ret)
+			return ret;
+		ic_conf |= task->csc1_mask;
+	} else {
+		ic_conf &= ~task->csc1_mask;
+	}
+
+	if (p->graphics_combine_en) {
+		ic_conf |= task->cmb_mask;
+
+		if (!(ic_conf & task->csc1_mask)) {
+			/* need transparent CSC1 conversion */
+			ipu_ic_init_csc(task, IPUV3_COLORSPACE_RGB,
+					IPUV3_COLORSPACE_RGB, 1);
+			/* Enable RGB -> RGB CSC */
+			ic_conf |= task->csc1_mask;
+		}
+		if (p->in_g_fmt != p->out_fmt) {
+			ret = ipu_ic_init_csc(task, p->in_g_fmt, p->out_fmt, 2);
+			if (ret)
+				return ret;
+			ic_conf |= task->csc2_mask;
+		}
+	}
+
+	writel(ic_conf, priv->ic_base + IC_CONF);
+
+	return 0;
+}
+
+static int ipu_ic_idma_init(struct ipu_ic_task *task, int dma_chan, u16 width,
+			    u16 height, int burst_size,
+			    enum ipu_rotate_mode rot)
+{
+	struct ipu_ic_priv *priv = task->priv;
+	u32 ic_idmac_1, ic_idmac_2, ic_idmac_3;
+	u32 temp_rot = bitrev8(rot) >> 5;
+	bool need_hor_flip = false;
+
+	if ((burst_size != 8) && (burst_size != 16)) {
+		dev_dbg(priv->dev, "Illegal burst length for IC\n");
+		return -EINVAL;
+	}
+
+	width--;
+	height--;
+
+	if (temp_rot & 0x2)	/* Need horizontal flip */
+		need_hor_flip = true;
+
+	ic_idmac_1 = readl(priv->ic_base + IC_IDMAC_1);
+	ic_idmac_2 = readl(priv->ic_base + IC_IDMAC_2);
+	ic_idmac_3 = readl(priv->ic_base + IC_IDMAC_3);
+	switch (dma_chan) {
+	case 22:	/* PP output - CB2 */
+		if (burst_size == 16)
+			ic_idmac_1 |= IC_IDMAC_1_CB2_BURST_16;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_CB2_BURST_16;
+
+		if (need_hor_flip)
+			ic_idmac_1 |= IC_IDMAC_1_PP_FLIP_RS;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_PP_FLIP_RS;
+
+		ic_idmac_2 &= ~IC_IDMAC_2_PP_HEIGHT_MASK;
+		ic_idmac_2 |= height << IC_IDMAC_2_PP_HEIGHT_OFFSET;
+
+		ic_idmac_3 &= ~IC_IDMAC_3_PP_WIDTH_MASK;
+		ic_idmac_3 |= width << IC_IDMAC_3_PP_WIDTH_OFFSET;
+		break;
+	case 11:	/* PP Input - CB5 */
+		if (burst_size == 16)
+			ic_idmac_1 |= IC_IDMAC_1_CB5_BURST_16;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_CB5_BURST_16;
+		break;
+	case 47:	/* PP Rot input */
+		ic_idmac_1 &= ~IC_IDMAC_1_PP_ROT_MASK;
+		ic_idmac_1 |= temp_rot << IC_IDMAC_1_PP_ROT_OFFSET;
+		break;
+	case 12: /* PRP Input - CB6 */
+		if (burst_size == 16)
+			ic_idmac_1 |= IC_IDMAC_1_CB6_BURST_16;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_CB6_BURST_16;
+		break;
+	case 20:	/* PRP ENC output - CB0 */
+		if (burst_size == 16)
+			ic_idmac_1 |= IC_IDMAC_1_CB0_BURST_16;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_CB0_BURST_16;
+
+		if (need_hor_flip)
+			ic_idmac_1 |= IC_IDMAC_1_PRPENC_FLIP_RS;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_PRPENC_FLIP_RS;
+
+		ic_idmac_2 &= ~IC_IDMAC_2_PRPENC_HEIGHT_MASK;
+		ic_idmac_2 |= height << IC_IDMAC_2_PRPENC_HEIGHT_OFFSET;
+
+		ic_idmac_3 &= ~IC_IDMAC_3_PRPENC_WIDTH_MASK;
+		ic_idmac_3 |= width << IC_IDMAC_3_PRPENC_WIDTH_OFFSET;
+		break;
+	case 45:	/* PRP ENC Rot input */
+		ic_idmac_1 &= ~IC_IDMAC_1_PRPENC_ROT_MASK;
+		ic_idmac_1 |= temp_rot << IC_IDMAC_1_PRPENC_ROT_OFFSET;
+		break;
+
+	case 21:	/* PRP VF output - CB1 */
+		if (burst_size == 16)
+			ic_idmac_1 |= IC_IDMAC_1_CB1_BURST_16;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_CB1_BURST_16;
+
+		if (need_hor_flip)
+			ic_idmac_1 |= IC_IDMAC_1_PRPVF_FLIP_RS;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_PRPVF_FLIP_RS;
+
+		ic_idmac_2 &= ~IC_IDMAC_2_PRPVF_HEIGHT_MASK;
+		ic_idmac_2 |= height << IC_IDMAC_2_PRPVF_HEIGHT_OFFSET;
+
+		ic_idmac_3 &= ~IC_IDMAC_3_PRPVF_WIDTH_MASK;
+		ic_idmac_3 |= width << IC_IDMAC_3_PRPVF_WIDTH_OFFSET;
+		break;
+
+	case 46:	/* PRP VF Rot input */
+		ic_idmac_1 &= ~IC_IDMAC_1_PRPVF_ROT_MASK;
+		ic_idmac_1 |= temp_rot << IC_IDMAC_1_PRPVF_ROT_OFFSET;
+		break;
+	case 14:	/* PRP VF graphics combining input - CB3 */
+		if (burst_size == 16)
+			ic_idmac_1 |= IC_IDMAC_1_CB3_BURST_16;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_CB3_BURST_16;
+		break;
+	case 15:	/* PP graphics combining input - CB4 */
+		if (burst_size == 16)
+			ic_idmac_1 |= IC_IDMAC_1_CB4_BURST_16;
+		else
+			ic_idmac_1 &= ~IC_IDMAC_1_CB4_BURST_16;
+		break;
+	}
+
+	writel(ic_idmac_1, priv->ic_base + IC_IDMAC_1);
+	writel(ic_idmac_2, priv->ic_base + IC_IDMAC_2);
+	writel(ic_idmac_3, priv->ic_base + IC_IDMAC_3);
+
+	return 0;
+}
+
+static struct image_convert_ctx *ipu_image_convert_next(struct ipu_ic_task *task)
+{
+	struct ipu_ic_priv *priv = task->priv;
+	struct ipu_ch_param *cpmem_in = ipu_get_cpmem(task->input_channel);
+	struct ipu_ch_param *cpmem_in_p, *cpmem_in_n;
+	struct ipu_ch_param *cpmem_out = ipu_get_cpmem(task->output_channel);
+	struct image_convert_ctx *ctx;
+	struct ipu_image *in_p, *in, *in_n;
+	struct ipu_image *out;
+	struct ic_setup setup;
+	int ret;
+	unsigned long flags;
+	unsigned int inburst, outburst;
+
+	spin_lock_irqsave(&priv->lock, flags);
+
+	if (list_empty(&task->image_list)) {
+		spin_unlock_irqrestore(&priv->lock, flags);
+		return NULL;
+	}
+
+	ctx = list_first_entry(&task->image_list, struct image_convert_ctx, list);
+
+	list_del(&ctx->list);
+
+	spin_unlock_irqrestore(&priv->lock, flags);
+
+	in_p = &ctx->in_p;
+	in = &ctx->in;
+	in_n = &ctx->in_n;
+	out = &ctx->out;
+
+	memset(cpmem_in, 0, sizeof(*cpmem_in));
+	memset(cpmem_out, 0, sizeof(*cpmem_out));
+
+	inburst = in->rect.width & 0xf ? 8 : 16;
+	outburst = out->rect.width & 0xf ? 8 : 16;
+
+	ipu_ic_enable(priv);
+
+	ipu_ic_idma_init(task, task->input_channel->num, in->rect.width,
+			 in->rect.height, inburst, 0);
+	ipu_ic_idma_init(task, task->output_channel->num, out->rect.width,
+			 out->rect.height, outburst, 0);
+
+	ipu_cpmem_set_image(cpmem_in, &ctx->in);
+	ipu_cpmem_set_image(cpmem_out, &ctx->out);
+
+	ipu_cpmem_set_burstsize(cpmem_in, inburst);
+	ipu_cpmem_set_burstsize(cpmem_out, outburst);
+
+	if (ctx->deinterlace) {
+		cpmem_in_p = ipu_get_cpmem(task->input_channel_p);
+		cpmem_in_n = ipu_get_cpmem(task->input_channel_n);
+
+		memset(cpmem_in_p, 0, sizeof(*cpmem_in_p));
+		memset(cpmem_in_n, 0, sizeof(*cpmem_in_n));
+
+		ipu_ic_idma_init(task, task->input_channel_p->num,
+				 in_p->rect.width, in_p->rect.height, inburst, 0);
+		ipu_ic_idma_init(task, task->input_channel_n->num,
+				 in_n->rect.width, in_n->rect.height, inburst, 0);
+
+		ipu_cpmem_set_image(cpmem_in_p, &ctx->in_p);
+		ipu_cpmem_set_image(cpmem_in_n, &ctx->in_n);
+
+		ipu_cpmem_set_burstsize(cpmem_in_p, inburst);
+		ipu_cpmem_set_burstsize(cpmem_in_n, inburst);
+	}
+
+	memset(&setup, 0, sizeof(setup));
+	setup.in_width = in->rect.width;
+	if (ctx->deinterlace)
+		setup.in_height = in->rect.height * 2;
+	else
+		setup.in_height = in->rect.height;
+
+	setup.in_fmt = ipu_pixelformat_to_colorspace(in->pix.pixelformat);
+	setup.out_width = out->rect.width;
+	setup.out_height = out->rect.height;
+	setup.out_fmt = ipu_pixelformat_to_colorspace(out->pix.pixelformat);
+
+	setup.rsc = ctx->rsc;
+
+	dev_dbg(priv->dev, "%s: %dx%d(%dx%d@%d,%d) -> %dx%d(%dx%d@%d,%d)\n",
+		__func__, in->pix.width, in->pix.height,
+		in->rect.width, in->rect.height, in->rect.left, in->rect.top,
+		out->pix.width, out->pix.height,
+		out->rect.width, out->rect.height, out->rect.left, out->rect.top);
+
+	dev_dbg(priv->dev, "%s: hscale: >>%d *8192/%d vscale: >>%d *8192/%d\n",
+			__func__, (ctx->rsc >> 14) & 0x3, (ctx->rsc & 0x3fff),
+			ctx->rsc >> 30, (ctx->rsc >> 16) & 0x3fff);
+
+	ret = ipu_ic_init_task(task, &setup, 0);
+	if (ret) {
+		ipu_ic_disable(priv);
+		return ERR_PTR(ret);
+	}
+
+	ipu_ic_enable_task(task, ctx);
+
+	return ctx;
+}
+
+static void ipu_image_convert_work(struct work_struct *work)
+{
+	struct ipu_ic_task *task = container_of(work, struct ipu_ic_task, work);
+	struct ipu_ic_priv *priv = task->priv;
+	struct image_convert_ctx *ctx;
+	int ret;
+
+	while (1) {
+		int task_error = 0;
+
+		ctx = ipu_image_convert_next(task);
+		if (!ctx)
+			return;
+
+		if (IS_ERR(ctx)) {
+			task_error = PTR_ERR(ctx);
+		} else {
+			ret = wait_for_completion_interruptible_timeout(
+						&task->complete, 100 * HZ);
+			if (!ret)
+				task_error = -ETIMEDOUT;
+		}
+
+		ipu_ic_disable_task(task, ctx);
+		ipu_ic_disable(priv);
+
+		if (ctx->complete)
+			ctx->complete(ctx->complete_context, task_error);
+		kfree(ctx->freep);
+	}
+}
+
+static irqreturn_t ipu_image_convert_handler(int irq, void *context)
+{
+	struct ipu_ic_task *task = context;
+
+	complete(&task->complete);
+
+	return IRQ_HANDLED;
+}
+
+#define VDI_FSIZE	0
+#define VDI_C		0x4
+
+#define VDI_C_CH_420		0x00000000
+#define VDI_C_CH_422		0x00000002
+#define VDI_C_MOT_SEL_FULL	0x00000008
+#define VDI_C_MOT_SEL_LOW	0x00000004
+#define VDI_C_MOT_SEL_MED	0x00000000
+#define VDI_C_BURST_SIZE1_4	0x00000030
+#define VDI_C_BURST_SIZE2_4	0x00000300
+#define VDI_C_BURST_SIZE3_4	0x00003000
+#define VDI_C_VWM1_CLR_2	0x00080000
+#define VDI_C_VWM3_CLR_2	0x02000000
+#define VDI_C_TOP_FIELD_MAN_1	0x40000000
+#define VDI_C_TOP_FIELD_AUTO_1	0x80000000
+
+static void ipu_vdi_init(struct ipu_ic_priv *priv, u32 pixelfmt,
+			 int xres, int yres, u32 field)
+{
+	uint32_t reg;
+	uint32_t pixel_fmt;
+
+	reg = ((yres - 1) << 16) | (xres - 1);
+	ipu_vdi_write(priv, reg, VDI_FSIZE);
+
+	/*
+	 * Full motion, only vertical filter is used
+	 * Burst size is 4 accesses
+	 */
+	if (pixelfmt == V4L2_PIX_FMT_UYVY || pixelfmt == V4L2_PIX_FMT_YUYV)
+		pixel_fmt = VDI_C_CH_422;
+	else
+		pixel_fmt = VDI_C_CH_420;
+
+	reg = ipu_vdi_read(priv, VDI_C);
+
+	reg |= pixel_fmt;
+	reg |= VDI_C_BURST_SIZE2_4;
+	reg |= VDI_C_BURST_SIZE1_4 | VDI_C_VWM1_CLR_2;
+	reg |= VDI_C_BURST_SIZE3_4 | VDI_C_VWM3_CLR_2;
+
+	if (field == V4L2_FIELD_INTERLACED_TB)
+		reg |= VDI_C_TOP_FIELD_MAN_1;
+
+	reg |= VDI_C_MOT_SEL_LOW;
+
+	ipu_vdi_write(priv, reg, VDI_C);
+}
+
+int ipu_image_deinterlace_convert(struct ipu_soc *ipu, struct ipu_image *in_p,
+		struct ipu_image *in, struct ipu_image *in_n,
+		struct ipu_image *out, void (*complete)(void *ctx, int err),
+		void *complete_context)
+{
+	struct ipu_ic_priv *priv = ipu->ic_priv;
+	struct ipu_ic_task *task = &priv->task[IC_TASK_VIEWFINDER];
+	u32 v_downsize_coeff, v_resize_coeff;
+	u32 h_downsize_coeff, h_resize_coeff;
+	struct image_convert_ctx *ctx;
+	unsigned long flags;
+	int ret;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	ipu_module_enable(ipu, 1<<30);
+	ipu_module_enable(ipu, IPU_CONF_VDI_EN);
+	ipu_vdi_init(priv, V4L2_PIX_FMT_UYVY, in->pix.width, in->pix.height * 2,
+		     V4L2_FIELD_INTERLACED_TB);
+
+	memcpy(&ctx->in_p, in_p, sizeof(*in_p));
+	memcpy(&ctx->in, in, sizeof(*in));
+	memcpy(&ctx->in_n, in_n, sizeof(*in_n));
+	memcpy(&ctx->out, out, sizeof(*out));
+
+	ctx->complete = complete;
+	ctx->complete_context = complete_context;
+	ctx->freep = ctx;
+	ctx->deinterlace = 1;
+
+	if (out->rect.width > 1024 || out->rect.height > 1024)
+		return -EINVAL;
+
+	/* Setup vertical resizing */
+	ret = ipu_ic_calc_resize_coeffs(priv, in->rect.height * 2,
+					out->rect.height, &v_resize_coeff,
+					&v_downsize_coeff, 1);
+	if (ret)
+		return ret;
+
+	/* Setup horizontal resizing */
+	ret = ipu_ic_calc_resize_coeffs(priv, in->rect.width, out->rect.width,
+			&h_resize_coeff, &h_downsize_coeff, 1);
+	if (ret)
+		return ret;
+
+	ctx->rsc = (v_downsize_coeff << 30) | (v_resize_coeff << 16) |
+		   (h_downsize_coeff << 14) | h_resize_coeff;
+
+	spin_lock_irqsave(&priv->lock, flags);
+
+	list_add_tail(&ctx->list, &task->image_list);
+
+	queue_work(task->workqueue, &task->work);
+
+	spin_unlock_irqrestore(&priv->lock, flags);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_image_deinterlace_convert);
+
+int ipu_image_convert(struct ipu_soc *ipu, struct ipu_image *in,
+		      struct ipu_image *out,
+		      void (*complete)(void *ctx, int err),
+		      void *complete_context)
+{
+	struct ipu_ic_priv *priv = ipu->ic_priv;
+	struct ipu_ic_task *task = &priv->task[IC_TASK_POST_PROCESSOR];
+	struct image_convert_ctx *ctx, *c;
+	int htiles = 1, vtiles = 1;
+	int x, y, i, numtiles, bppin, bppout;
+	int dec_width, dec_height;
+	unsigned long flags;
+	int in_rect_width;
+	u32 v_downsize_coeff, v_resize_coeff;
+	u32 h_downsize_coeff, h_resize_coeff;
+	int seam_x, seam_y;
+	int ret;
+
+	/* Force input width to a multiple of 8 pixels, hardware limitation */
+	in_rect_width = in->rect.width & ~0x7;
+	if (in_rect_width == 0) {
+		dev_dbg(priv->dev, "%s: invalid input width: %d\n", __func__,
+			in->rect.width);
+		return -EINVAL;
+	}
+
+	if (out->rect.width > 1024) {
+		/*
+		 * With horizontal tiling force input width to a multiple of 16
+		 * pixels, we could do better with the cost of a better tiling
+		 * logic below.
+		 */
+		in_rect_width = in->rect.width & ~0x7;
+		htiles = 2;
+	}
+
+	if (out->rect.height > 1024) {
+		if (in->rect.height < 3)
+			return -EINVAL;
+		vtiles = 2;
+	}
+
+	/* Setup vertical resizing coefficients */
+	ret = ipu_ic_calc_resize_coeffs(priv, in->rect.height, out->rect.height,
+			&v_resize_coeff, &v_downsize_coeff, vtiles);
+	if (ret)
+		return ret;
+
+	/* Setup horizontal resizing coefficients */
+	ret = ipu_ic_calc_resize_coeffs(priv, in_rect_width, out->rect.width,
+			&h_resize_coeff, &h_downsize_coeff, htiles);
+	if (ret)
+		return ret;
+
+	bppin = in->pix.bytesperline / in->pix.width;
+	bppout = out->pix.bytesperline / out->pix.width;
+
+	dev_dbg(priv->dev, "%s: in: %dx%d(%dx%d@%d,%d) -> %dx%d(%dx%d@%d,%d) phys 0x%08x -> 0x%08x\n",
+		__func__, in->pix.width, in->pix.height,
+		in->rect.width, in->rect.height, in->rect.left, in->rect.top,
+		out->pix.width, out->pix.height,
+		out->rect.width, out->rect.height, out->rect.left, out->rect.top,
+		in->phys, out->phys);
+
+	dev_dbg(priv->dev, "%s: hscale: >>%d *8192/%d vscale: >>%d *8192/%d, %dx%d tiles\n",
+			__func__, h_downsize_coeff, h_resize_coeff,
+			v_downsize_coeff, v_resize_coeff, htiles, vtiles);
+
+	numtiles = htiles * vtiles;
+
+	ctx = kzalloc(sizeof(*ctx) * numtiles, GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	c = ctx;
+
+	/* Decimator output to the bilinear scaling unit */
+	dec_width = (in_rect_width / htiles) >> h_downsize_coeff;
+	dec_height = (in->rect.height / vtiles) >> v_downsize_coeff;
+
+	seam_x = dec_width * 8192 / h_resize_coeff;
+	seam_y = dec_height * 8192 / v_resize_coeff;
+
+	/*
+	 * Move the horizontal seam a little bit according to IDMAC limitations.
+	 * A slight distortion should be less noticeable than a sharp seam, so
+	 * the horizontal resizing coefficients will be recalculated below.
+	 */
+	switch (out->pix.pixelformat) {
+	case V4L2_PIX_FMT_RGB32:
+	case V4L2_PIX_FMT_BGR32:
+		seam_x = round_down(seam_x, 2);
+		break;
+	case V4L2_PIX_FMT_RGB565:
+	case V4L2_PIX_FMT_UYVY:
+	case V4L2_PIX_FMT_YUYV:
+	case V4L2_PIX_FMT_NV12:
+		seam_x = round_down(seam_x, 4);
+		break;
+	case V4L2_PIX_FMT_YUV420:
+	case V4L2_PIX_FMT_YVU420:
+	case V4L2_PIX_FMT_YUV422P:
+	default:
+		seam_x = round_down(seam_x, 8);
+		break;
+	};
+
+	switch (out->pix.pixelformat) {
+	case V4L2_PIX_FMT_YUV420:
+	case V4L2_PIX_FMT_YVU420:
+	case V4L2_PIX_FMT_NV12:
+		seam_y = round_down(seam_y, 2);
+		break;
+	};
+
+	for (x = htiles - 1; x >= 0; x--) {
+		for (y = vtiles - 1; y >= 0; y--) {
+			c->in.rect.width = in_rect_width / htiles;
+			c->in.rect.left = x * c->in.rect.width + in->rect.left;
+			/*
+			 * For all but the rightmost tile, we need to include
+			 * the first pixel of the following tile.
+			 */
+			if (x < htiles - 1)
+				c->in.rect.width++;
+			c->in.rect.width = round_up(c->in.rect.width, 8);
+
+			c->in.rect.height = in->rect.height / vtiles;
+			c->in.rect.top = y * c->in.rect.height + in->rect.top;
+			/*
+			 * For all but the bottom tiles, we need to include the
+			 * first line of the tile below
+			 */
+			if (y < vtiles - 1) {
+				c->in.rect.height++;
+				if (out->pix.pixelformat == V4L2_PIX_FMT_YUV420 ||
+				    out->pix.pixelformat == V4L2_PIX_FMT_YVU420 ||
+				    out->pix.pixelformat == V4L2_PIX_FMT_NV12)
+					c->in.rect.height = round_up(c->in.rect.height, 2);
+			}
+
+			c->in.phys = in->phys;
+
+			memcpy(&c->in.pix, &in->pix, sizeof(struct v4l2_pix_format));
+
+			c->out.rect.left = out->rect.left;
+			if (htiles == 1) {
+				c->out.rect.width = out->rect.width;
+			} else {
+				if (x) {
+					h_resize_coeff = 8192 * (dec_width - 1) / (out->rect.width - seam_x - 1);
+					c->out.rect.left += seam_x;
+					c->out.rect.width = out->rect.width - c->out.rect.left;
+				} else {
+					h_resize_coeff = DIV_ROUND_CLOSEST(8192 * dec_width, seam_x);
+					c->out.rect.width = seam_x;
+				}
+			}
+
+			/*
+			 * The IC can only process bursts of 8 or 16 pixels at
+			 * a time. Rounding up here will overwrite pixels in
+			 * the next tile. This is why tiles have to be processed
+			 * from right to left and from bottom to top.
+			 */
+			c->out.rect.width = round_up(c->out.rect.width, 8);
+
+			c->out.rect.top = out->rect.top;
+			if (vtiles == 1) {
+				c->out.rect.height = out->rect.height;
+			} else {
+				if (y) {
+					v_resize_coeff = 8192 * (dec_height - 1) / (out->rect.height - seam_y - 1);
+					c->out.rect.top += seam_y;
+					c->out.rect.height = out->rect.height - c->out.rect.top;
+
+					/* Rounding errors can shift the right pixel over the edge */
+					if ((((c->out.rect.height - 1) * v_resize_coeff) + 8191 / 8192) > (c->in.rect.height - 1))
+						c->in.rect.height++;
+				} else {
+					v_resize_coeff = DIV_ROUND_CLOSEST(8192 * dec_height, seam_y);
+					c->out.rect.height = seam_y;
+				}
+			}
+			c->out.phys = out->phys;
+
+			memcpy(&c->out.pix, &out->pix, sizeof(struct v4l2_pix_format));
+
+			c->rsc = (v_downsize_coeff << 30) | (v_resize_coeff << 16) |
+				 (h_downsize_coeff << 14) | h_resize_coeff;
+
+			c++;
+		}
+	}
+
+	ctx[numtiles - 1].complete = complete;
+	ctx[numtiles - 1].complete_context = complete_context;
+	ctx[numtiles - 1].freep = ctx;
+
+	spin_lock_irqsave(&priv->lock, flags);
+
+	for (i = 0; i < numtiles; i++)
+		list_add_tail(&ctx[i].list, &task->image_list);
+
+	queue_work(task->workqueue, &task->work);
+
+	spin_unlock_irqrestore(&priv->lock, flags);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_image_convert);
+
+static int ipu_image_convert_init(struct device *dev, struct ipu_soc *ipu,
+		struct ipu_ic_priv *priv)
+{
+	int i, ret;
+
+	for (i = 0; i < IC_TASK_MAX; i++) {
+		struct ipu_ic_task *task = &priv->task[i];
+		int irq = ipu_idmac_channel_irq(ipu, task->output_channel,
+						IPU_IRQ_EOF);
+
+		task->workqueue = create_singlethread_workqueue(
+				dev_name(ipu->dev));
+		if (!task->workqueue) {
+			ret = -ENOMEM;
+			goto err;
+		}
+
+		INIT_WORK(&task->work, ipu_image_convert_work);
+		init_completion(&task->complete);
+
+		ret = devm_request_threaded_irq(dev, irq, NULL,
+					ipu_image_convert_handler,
+					IRQF_ONESHOT, "IC", task);
+		if (ret)
+			goto err;
+	}
+
+	return 0;
+err:
+	while (i-- > 0) {
+		struct ipu_ic_task *task = &priv->task[i];
+
+		destroy_workqueue(task->workqueue);
+	}
+
+	return ret;
+}
+
+int ipu_ic_init(struct ipu_soc *ipu, struct device *dev, unsigned long ic_base,
+		unsigned long tpm_base, unsigned long vdi_base)
+{
+	struct ipu_ic_priv *priv;
+	int i;
+
+	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+	priv->dev = dev;
+	priv->ipu = ipu;
+
+	ipu->ic_priv = priv;
+
+	priv->ic_base = devm_ioremap(dev, ic_base, PAGE_SIZE);
+	if (!priv->ic_base)
+		return -ENOMEM;
+
+	priv->vdi_base = devm_ioremap(dev, vdi_base, PAGE_SIZE);
+	if (!priv->vdi_base)
+		return -ENOMEM;
+
+	priv->tpm_base = devm_ioremap(dev, tpm_base, SZ_64K);
+	if (!priv->tpm_base)
+		return -ENOMEM;
+
+	spin_lock_init(&priv->lock);
+
+	for (i = 0; i < IC_TASK_MAX; i++) {
+		struct ipu_ic_task_template *templ = &templates[i];
+		struct ipu_ic_task *task = &priv->task[i];
+		INIT_LIST_HEAD(&task->image_list);
+		task->id = i;
+		task->priv = priv;
+		task->reg_rsc = priv->ic_base + templ->ofs_rsc;
+		task->csc1_mask = templ->csc1_mask;
+		task->csc2_mask = templ->csc2_mask;
+		task->cmb_mask = templ->cmb_mask;
+		task->tpm_base_csc1 = priv->tpm_base + templ->tpm_ofs_csc1;
+		task->tpm_base_csc2 = priv->tpm_base + templ->tpm_ofs_csc2;
+		task->ic_conf_rot_en = templ->ic_conf_rot_en;
+		task->ic_conf_en = templ->ic_conf_en;
+		task->first_deinterlace_frame = 1;
+		task->input_channel = ipu_idmac_get(ipu, templ->input_channel);
+		if (IS_ERR(task->input_channel))
+			goto out;
+		if (templ->input_channel_n) {
+			task->input_channel_n = ipu_idmac_get(ipu,
+							templ->input_channel_n);
+			if (IS_ERR(task->input_channel_n))
+				goto out;
+		}
+		if (templ->input_channel_p) {
+			task->input_channel_p = ipu_idmac_get(ipu,
+							templ->input_channel_p);
+			if (IS_ERR(task->input_channel_p))
+				goto out;
+		}
+
+		task->output_channel = ipu_idmac_get(ipu, templ->output_channel);
+		if (IS_ERR(task->output_channel))
+			goto out;
+
+		task->rotation_input_channel = ipu_idmac_get(ipu,
+				templ->rotation_input_channel);
+		if (IS_ERR(task->rotation_input_channel))
+			goto out;
+		task->rotation_output_channel = ipu_idmac_get(ipu,
+				templ->rotation_output_channel);
+		if (IS_ERR(task->rotation_output_channel))
+			goto out;
+	}
+
+	ipu_image_convert_init(dev, ipu, priv);
+
+	mutex_init(&priv->mutex);
+
+	return 0;
+out:
+	dev_err(priv->dev, "failed to initialize IC\n");
+
+	/* FIXME: Cleanup */
+
+	return -EINVAL;
+}
+
+void ipu_ic_exit(struct ipu_soc *ipu)
+{
+}
diff --git a/include/video/imx-ipu-v3.h b/include/video/imx-ipu-v3.h
index 61d6d25..2d14425 100644
--- a/include/video/imx-ipu-v3.h
+++ b/include/video/imx-ipu-v3.h
@@ -339,4 +339,10 @@ struct ipu_client_platformdata {
 	int dma[2];
 };
 
+int ipu_image_convert(struct ipu_soc *ipu, struct ipu_image *in, struct ipu_image *out,
+		void (*complete)(void *ctx, int err), void *ctx);
+int ipu_image_deinterlace_convert(struct ipu_soc *ipu, struct ipu_image *in_p,
+		struct ipu_image *in, struct ipu_image *in_n, struct ipu_image *out,
+		void (*complete)(void *ctx, int err), void *complete_context);
+
 #endif /* __DRM_IPU_H__ */
-- 
2.0.0.rc2

[RFC PATCH 02/26] gpu: ipu-v3: Register IC with IPUv3

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/gpu/ipu-v3/ipu-common.c | 35 +++++++++++++++++++++++++++++++++++
 drivers/gpu/ipu-v3/ipu-prv.h    |  6 ++++++
 2 files changed, 41 insertions(+)

diff --git a/drivers/gpu/ipu-v3/ipu-common.c b/drivers/gpu/ipu-v3/ipu-common.c
index 719788c..a4910d8 100644
--- a/drivers/gpu/ipu-v3/ipu-common.c
+++ b/drivers/gpu/ipu-v3/ipu-common.c
@@ -914,8 +914,18 @@ static int ipu_submodules_init(struct ipu_soc *ipu,
 		goto err_smfc;
 	}
 
+	ret = ipu_ic_init(ipu, dev, ipu_base + devtype->cm_ofs +
+			IPU_CM_IC_REG_OFS, ipu_base + devtype->tpm_ofs,
+			ipu_base + devtype->vdi_ofs);
+	if (ret) {
+		unit = "ic";
+		goto err_ic;
+	}
+
 	return 0;
 
+err_ic:
+	ipu_smfc_exit(ipu);
 err_smfc:
 	ipu_dp_exit(ipu);
 err_dp:
@@ -995,6 +1005,7 @@ static void ipu_submodules_exit(struct ipu_soc *ipu)
 	ipu_dc_exit(ipu);
 	ipu_di_exit(ipu, 1);
 	ipu_di_exit(ipu, 0);
+	ipu_ic_exit(ipu);
 }
 
 static int platform_remove_devices_fn(struct device *dev, void *unused)
@@ -1052,6 +1063,18 @@ static const struct ipu_platform_reg client_reg[] = {
 		},
 		.reg_offset = IPU_CM_CSI1_REG_OFS,
 		.name = "imx-ipuv3-camera",
+	}, {
+		.pdata = {
+			.dma[0] = IPUV3_CHANNEL_MEM_FG_SYNC,
+		},
+		.name = "imx-ipuv3-scaler",
+	}, {
+		.pdata = {
+			.dma[0] = IPUV3_CHANNEL_MEM_FG_SYNC,
+		},
+		.name = "imx-ipuv3-ovl",
+	}, {
+		.name = "imx-ipuv3-vdic",
 	},
 };
 
@@ -1179,6 +1202,7 @@ static int ipu_probe(struct platform_device *pdev)
 	unsigned long ipu_base;
 	int i, ret, irq_sync, irq_err;
 	const struct ipu_devtype *devtype;
+	u32 reg;
 
 	devtype = of_id->data;
 
@@ -1272,6 +1296,9 @@ static int ipu_probe(struct platform_device *pdev)
 	if (ret)
 		goto out_failed_reset;
 
+	/* Set sync refresh channels as high priority */
+	ipu_idmac_write(ipu, 0x18800000, IDMAC_CHA_PRI(0));
+
 	/* Set MCU_T to divide MCU access window into 2 */
 	ipu_cm_write(ipu, 0x00400000L | (IPU_MCU_T_DEFAULT << 18),
 			IPU_DISP_GEN);
@@ -1280,6 +1307,14 @@ static int ipu_probe(struct platform_device *pdev)
 	if (ret)
 		goto failed_submodules_init;
 
+	reg = ipu_cm_read(ipu, IPU_FS_PROC_FLOW1);
+	reg |= (5 << 24);
+	ipu_cm_write(ipu, reg, IPU_FS_PROC_FLOW1);
+
+	reg = ipu_cm_read(ipu, IPU_CONF);
+	reg |= IPU_CONF_IC_INPUT;
+	ipu_cm_write(ipu, reg, IPU_CONF);
+
 	ret = ipu_add_client_devices(ipu, ipu_base);
 	if (ret) {
 		dev_err(&pdev->dev, "adding client devices failed with %d\n",
diff --git a/drivers/gpu/ipu-v3/ipu-prv.h b/drivers/gpu/ipu-v3/ipu-prv.h
index acf1811..e869ac1 100644
--- a/drivers/gpu/ipu-v3/ipu-prv.h
+++ b/drivers/gpu/ipu-v3/ipu-prv.h
@@ -152,6 +152,7 @@ struct ipu_dc_priv;
 struct ipu_dmfc_priv;
 struct ipu_di;
 struct ipu_smfc_priv;
+struct ipu_ic_priv;
 
 struct ipu_devtype;
 
@@ -181,6 +182,7 @@ struct ipu_soc {
 	struct ipu_dmfc_priv	*dmfc_priv;
 	struct ipu_di		*di_priv[2];
 	struct ipu_smfc_priv	*smfc_priv;
+	struct ipu_ic_priv	*ic_priv;
 };
 
 void ipu_srm_dp_sync_update(struct ipu_soc *ipu);
@@ -209,4 +211,8 @@ void ipu_cpmem_exit(struct ipu_soc *ipu);
 int ipu_smfc_init(struct ipu_soc *ipu, struct device *dev, unsigned long base);
 void ipu_smfc_exit(struct ipu_soc *ipu);
 
+int ipu_ic_init(struct ipu_soc *ipu, struct device *dev, unsigned long ic_base,
+		unsigned long tpm_base, unsigned long vdi_base);
+void ipu_ic_exit(struct ipu_soc *ipu);
+
 #endif				/* __IPU_PRV_H__ */
-- 
2.0.0.rc2

[RFC PATCH 03/26] gpu: ipu-v3: Add function to setup CP channel as interlaced

[Philipp Zabel]

This patch adds a function to enable line interlaced buffer scanout
and writing.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/gpu/ipu-v3/ipu-common.c | 19 +++++++++++++++++++
 include/video/imx-ipu-v3.h      |  1 +
 2 files changed, 20 insertions(+)

diff --git a/drivers/gpu/ipu-v3/ipu-common.c b/drivers/gpu/ipu-v3/ipu-common.c
index a4910d8..94b9e8e 100644
--- a/drivers/gpu/ipu-v3/ipu-common.c
+++ b/drivers/gpu/ipu-v3/ipu-common.c
@@ -525,6 +525,25 @@ int ipu_cpmem_set_image(struct ipu_ch_param __iomem *cpmem,
 }
 EXPORT_SYMBOL_GPL(ipu_cpmem_set_image);
 
+int ipu_ch_cpmem_set_interlaced_scan(struct ipuv3_channel *channel)
+{
+	u32 stride;
+	struct ipu_soc *ipu = channel->ipu;
+	struct ipu_ch_param __iomem *p = ipu_get_cpmem(channel);
+
+	stride = ipu_ch_param_read_field(p, IPU_FIELD_SL) + 1;
+	if (stride % 8)
+		dev_warn(ipu->dev,
+			 "IDMAC%d's ILO is not 8-byte aligned\n", channel->num);
+
+	ipu_ch_param_write_field(p, IPU_FIELD_SO, 1);
+	ipu_ch_param_write_field(p, IPU_FIELD_ILO, stride / 8);
+	ipu_ch_param_write_field(p, IPU_FIELD_SL, 2 * stride - 1);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_ch_cpmem_set_interlaced_scan);
+
 enum ipu_color_space ipu_pixelformat_to_colorspace(u32 pixelformat)
 {
 	switch (pixelformat) {
diff --git a/include/video/imx-ipu-v3.h b/include/video/imx-ipu-v3.h
index 2d14425..77a82f5 100644
--- a/include/video/imx-ipu-v3.h
+++ b/include/video/imx-ipu-v3.h
@@ -320,6 +320,7 @@ void ipu_cpmem_set_yuv_planar_full(struct ipu_ch_param __iomem *p,
 int ipu_cpmem_set_fmt(struct ipu_ch_param __iomem *cpmem, u32 pixelformat);
 int ipu_cpmem_set_image(struct ipu_ch_param __iomem *cpmem,
 		struct ipu_image *image);
+int ipu_ch_cpmem_set_interlaced_scan(struct ipuv3_channel *channel);
 
 enum ipu_color_space ipu_drm_fourcc_to_colorspace(u32 drm_fourcc);
 enum ipu_color_space ipu_pixelformat_to_colorspace(u32 pixelformat);
-- 
2.0.0.rc2

[RFC PATCH 04/26] gpu: ipu-v3: Add ipu_cpmem_get_buffer function

[Philipp Zabel]

This is needed for imx-ipu-vout to extract the buffer address from a
saved CPMEM block.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 include/video/imx-ipu-v3.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/include/video/imx-ipu-v3.h b/include/video/imx-ipu-v3.h
index 77a82f5..e8764dc 100644
--- a/include/video/imx-ipu-v3.h
+++ b/include/video/imx-ipu-v3.h
@@ -268,6 +268,15 @@ static inline void ipu_cpmem_set_buffer(struct ipu_ch_param __iomem *p,
 		ipu_ch_param_write_field(p, IPU_FIELD_EBA0, buf >> 3);
 }
 
+static inline dma_addr_t ipu_cpmem_get_buffer(struct ipu_ch_param __iomem *p,
+		int bufnum)
+{
+	if (bufnum)
+		return ipu_ch_param_read_field(p, IPU_FIELD_EBA1) << 3;
+	else
+		return ipu_ch_param_read_field(p, IPU_FIELD_EBA0) << 3;
+}
+
 static inline void ipu_cpmem_set_resolution(struct ipu_ch_param __iomem *p,
 		int xres, int yres)
 {
-- 
2.0.0.rc2

[RFC PATCH 05/26] gpu: ipu-v3: Add support for partial interleaved YCbCr 4:2:0 (NV12) format

[Philipp Zabel]

From: Philipp Zabel <philipp.zabel@gmail.com>

The partial interleaved format consists of two planes, one with 8-bit
luma (Y) values, and one with alternating 8-bit chroma (CbCr) values.
This format can be produced by CODA960 VPU and VDOA.

Signed-off-by: Philipp Zabel <philipp.zabel@gmail.com>
---
 drivers/gpu/ipu-v3/ipu-common.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/drivers/gpu/ipu-v3/ipu-common.c b/drivers/gpu/ipu-v3/ipu-common.c
index 94b9e8e..7a0b377 100644
--- a/drivers/gpu/ipu-v3/ipu-common.c
+++ b/drivers/gpu/ipu-v3/ipu-common.c
@@ -260,6 +260,11 @@ void ipu_cpmem_set_yuv_planar_full(struct ipu_ch_param __iomem *p,
 		ipu_ch_param_write_field(p, IPU_FIELD_UBO, v_offset / 8);
 		ipu_ch_param_write_field(p, IPU_FIELD_VBO, u_offset / 8);
 		break;
+	case V4L2_PIX_FMT_NV12:
+		ipu_ch_param_write_field(p, IPU_FIELD_SLUV, stride - 1);
+		ipu_ch_param_write_field(p, IPU_FIELD_UBO, u_offset / 8);
+		ipu_ch_param_write_field(p, IPU_FIELD_VBO, u_offset / 8);
+		break;
 	}
 }
 EXPORT_SYMBOL_GPL(ipu_cpmem_set_yuv_planar_full);
@@ -279,6 +284,11 @@ void ipu_cpmem_set_yuv_planar(struct ipu_ch_param __iomem *p, u32 pixel_format,
 		ipu_cpmem_set_yuv_planar_full(p, pixel_format, stride,
 				u_offset, v_offset);
 		break;
+	case V4L2_PIX_FMT_NV12:
+		u_offset = v_offset = stride * height;
+		ipu_cpmem_set_yuv_planar_full(p, pixel_format, stride,
+				u_offset, v_offset);
+		break;
 	}
 }
 EXPORT_SYMBOL_GPL(ipu_cpmem_set_yuv_planar);
@@ -348,6 +358,10 @@ int ipu_cpmem_set_fmt(struct ipu_ch_param __iomem *cpmem, u32 drm_fourcc)
 		/* burst size */
 		ipu_ch_param_write_field(cpmem, IPU_FIELD_NPB, 63);
 		break;
+	case DRM_FORMAT_NV12:
+		ipu_ch_param_write_field(cpmem, IPU_FIELD_PFS, 4);
+		ipu_ch_param_write_field(cpmem, IPU_FIELD_NPB, 63);
+		break;
 	case DRM_FORMAT_UYVY:
 		/* bits/pixel */
 		ipu_ch_param_write_field(cpmem, IPU_FIELD_BPP, 3);
@@ -433,6 +447,8 @@ static int v4l2_pix_fmt_to_drm_fourcc(u32 pixelformat)
 		return DRM_FORMAT_YUV420;
 	case V4L2_PIX_FMT_YVU420:
 		return DRM_FORMAT_YVU420;
+	case V4L2_PIX_FMT_NV12:
+		return DRM_FORMAT_NV12;
 	}
 
 	return -EINVAL;
@@ -458,6 +474,7 @@ enum ipu_color_space ipu_drm_fourcc_to_colorspace(u32 drm_fourcc)
 	case DRM_FORMAT_UYVY:
 	case DRM_FORMAT_YUV420:
 	case DRM_FORMAT_YVU420:
+	case DRM_FORMAT_NV12:
 		return IPUV3_COLORSPACE_YUV;
 	default:
 		return IPUV3_COLORSPACE_UNKNOWN;
@@ -494,6 +511,19 @@ int ipu_cpmem_set_image(struct ipu_ch_param __iomem *cpmem,
 				pix->bytesperline, u_offset, v_offset);
 		ipu_cpmem_set_buffer(cpmem, 0, image->phys + y_offset);
 		break;
+	case V4L2_PIX_FMT_NV12:
+		y_offset = Y_OFFSET(pix, image->rect.left, image->rect.top);
+		u_offset = U_OFFSET(pix, image->rect.left,
+				image->rect.top) - y_offset;
+
+		ipu_cpmem_set_yuv_planar_full(cpmem, pix->pixelformat,
+				pix->bytesperline, u_offset, u_offset);
+		ipu_cpmem_set_buffer(cpmem, 0, image->phys + y_offset);
+		break;
+		ipu_cpmem_set_yuv_planar_full(cpmem, pix->pixelformat,
+				pix->bytesperline, u_offset, v_offset);
+		ipu_cpmem_set_buffer(cpmem, 0, image->phys + y_offset);
+		break;
 	case V4L2_PIX_FMT_UYVY:
 	case V4L2_PIX_FMT_YUYV:
 		ipu_cpmem_set_buffer(cpmem, 0, image->phys +
@@ -551,6 +581,7 @@ enum ipu_color_space ipu_pixelformat_to_colorspace(u32 pixelformat)
 	case V4L2_PIX_FMT_YVU420:
 	case V4L2_PIX_FMT_UYVY:
 	case V4L2_PIX_FMT_YUYV:
+	case V4L2_PIX_FMT_NV12:
 		return IPUV3_COLORSPACE_YUV;
 	case V4L2_PIX_FMT_RGB32:
 	case V4L2_PIX_FMT_BGR32:
-- 
2.0.0.rc2

[RFC PATCH 06/26] gpu: ipu-v3: Add support for planar YUV 4:2:2 (YUV422P) format

[Philipp Zabel]

From: Philipp Zabel <philipp.zabel@gmail.com>

Signed-off-by: Philipp Zabel <philipp.zabel@gmail.com>
---
 drivers/gpu/ipu-v3/ipu-common.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/drivers/gpu/ipu-v3/ipu-common.c b/drivers/gpu/ipu-v3/ipu-common.c
index 7a0b377..cd4f584 100644
--- a/drivers/gpu/ipu-v3/ipu-common.c
+++ b/drivers/gpu/ipu-v3/ipu-common.c
@@ -251,6 +251,7 @@ void ipu_cpmem_set_yuv_planar_full(struct ipu_ch_param __iomem *p,
 {
 	switch (pixel_format) {
 	case V4L2_PIX_FMT_YUV420:
+	case V4L2_PIX_FMT_YUV422P:
 		ipu_ch_param_write_field(p, IPU_FIELD_SLUV, (stride / 2) - 1);
 		ipu_ch_param_write_field(p, IPU_FIELD_UBO, u_offset / 8);
 		ipu_ch_param_write_field(p, IPU_FIELD_VBO, v_offset / 8);
@@ -284,6 +285,13 @@ void ipu_cpmem_set_yuv_planar(struct ipu_ch_param __iomem *p, u32 pixel_format,
 		ipu_cpmem_set_yuv_planar_full(p, pixel_format, stride,
 				u_offset, v_offset);
 		break;
+	case V4L2_PIX_FMT_YUV422P:
+		uv_stride = stride / 2;
+		u_offset = stride * height;
+		v_offset = u_offset + (uv_stride * height);
+		ipu_cpmem_set_yuv_planar_full(p, pixel_format, stride,
+				u_offset, v_offset);
+		break;
 	case V4L2_PIX_FMT_NV12:
 		u_offset = v_offset = stride * height;
 		ipu_cpmem_set_yuv_planar_full(p, pixel_format, stride,
@@ -347,6 +355,11 @@ static const struct ipu_rgb def_bgr_16 = {
 #define V_OFFSET(pix, x, y)	((pix->width * pix->height) + \
 					(pix->width * pix->height / 4) + \
 					(pix->width * (y) / 4) + (x) / 2)
+#define U_OFFSET2(pix, x, y)	((pix->width * pix->height) + \
+					(pix->width * (y) / 2) + (x) / 2)
+#define V_OFFSET2(pix, x, y)	((pix->width * pix->height) + \
+					(pix->width * pix->height / 2) + \
+					(pix->width * (y) / 2) + (x) / 2)
 
 int ipu_cpmem_set_fmt(struct ipu_ch_param __iomem *cpmem, u32 drm_fourcc)
 {
@@ -358,6 +371,11 @@ int ipu_cpmem_set_fmt(struct ipu_ch_param __iomem *cpmem, u32 drm_fourcc)
 		/* burst size */
 		ipu_ch_param_write_field(cpmem, IPU_FIELD_NPB, 63);
 		break;
+	case DRM_FORMAT_YUV422:
+	case DRM_FORMAT_YVU422:
+		ipu_ch_param_write_field(cpmem, IPU_FIELD_PFS, 1);
+		ipu_ch_param_write_field(cpmem, IPU_FIELD_NPB, 63);
+		break;
 	case DRM_FORMAT_NV12:
 		ipu_ch_param_write_field(cpmem, IPU_FIELD_PFS, 4);
 		ipu_ch_param_write_field(cpmem, IPU_FIELD_NPB, 63);
@@ -447,6 +465,8 @@ static int v4l2_pix_fmt_to_drm_fourcc(u32 pixelformat)
 		return DRM_FORMAT_YUV420;
 	case V4L2_PIX_FMT_YVU420:
 		return DRM_FORMAT_YVU420;
+	case V4L2_PIX_FMT_YUV422P:
+		return DRM_FORMAT_YUV422;
 	case V4L2_PIX_FMT_NV12:
 		return DRM_FORMAT_NV12;
 	}
@@ -474,6 +494,8 @@ enum ipu_color_space ipu_drm_fourcc_to_colorspace(u32 drm_fourcc)
 	case DRM_FORMAT_UYVY:
 	case DRM_FORMAT_YUV420:
 	case DRM_FORMAT_YVU420:
+	case DRM_FORMAT_YUV422:
+	case DRM_FORMAT_YVU422:
 	case DRM_FORMAT_NV12:
 		return IPUV3_COLORSPACE_YUV;
 	default:
@@ -511,6 +533,17 @@ int ipu_cpmem_set_image(struct ipu_ch_param __iomem *cpmem,
 				pix->bytesperline, u_offset, v_offset);
 		ipu_cpmem_set_buffer(cpmem, 0, image->phys + y_offset);
 		break;
+	case V4L2_PIX_FMT_YUV422P:
+		y_offset = Y_OFFSET(pix, image->rect.left, image->rect.top);
+		u_offset = U_OFFSET2(pix, image->rect.left,
+				image->rect.top) - y_offset;
+		v_offset = V_OFFSET2(pix, image->rect.left,
+				image->rect.top) - y_offset;
+
+		ipu_cpmem_set_yuv_planar_full(cpmem, pix->pixelformat,
+				pix->bytesperline, u_offset, v_offset);
+		ipu_cpmem_set_buffer(cpmem, 0, image->phys + y_offset);
+		break;
 	case V4L2_PIX_FMT_NV12:
 		y_offset = Y_OFFSET(pix, image->rect.left, image->rect.top);
 		u_offset = U_OFFSET(pix, image->rect.left,
@@ -579,6 +612,7 @@ enum ipu_color_space ipu_pixelformat_to_colorspace(u32 pixelformat)
 	switch (pixelformat) {
 	case V4L2_PIX_FMT_YUV420:
 	case V4L2_PIX_FMT_YVU420:
+	case V4L2_PIX_FMT_YUV422P:
 	case V4L2_PIX_FMT_UYVY:
 	case V4L2_PIX_FMT_YUYV:
 	case V4L2_PIX_FMT_NV12:
-- 
2.0.0.rc2

[RFC PATCH 07/26] imx-drm: currently only IPUv3 is supported, make it mandatory

[Philipp Zabel]

As long as only IPUv3 is supported in imx-drm, hide the separate
DRM_IMX_IPUV3 option and make DRM_IMX depend on IMX_IPUV3_CORE.

Reported-by: Michael Olbrich <m.olbrich@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/staging/imx-drm/Kconfig | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/imx-drm/Kconfig b/drivers/staging/imx-drm/Kconfig
index 82fb758..ab31848 100644
--- a/drivers/staging/imx-drm/Kconfig
+++ b/drivers/staging/imx-drm/Kconfig
@@ -6,6 +6,7 @@ config DRM_IMX
 	select DRM_GEM_CMA_HELPER
 	select DRM_KMS_CMA_HELPER
 	depends on DRM && (ARCH_MXC || ARCH_MULTIPLATFORM)
+	depends on IMX_IPUV3_CORE
 	help
 	  enable i.MX graphics support
 
@@ -40,11 +41,11 @@ config DRM_IMX_LDB
 	  found on i.MX53 and i.MX6 processors.
 
 config DRM_IMX_IPUV3
-	tristate "DRM Support for i.MX IPUv3"
+	tristate
 	depends on DRM_IMX
 	depends on IMX_IPUV3_CORE
-	help
-	  Choose this if you have a i.MX5 or i.MX6 processor.
+	default y if DRM_IMX=y
+	default m if DRM_IMX=m
 
 config DRM_IMX_HDMI
 	tristate "Freescale i.MX DRM HDMI"
-- 
2.0.0.rc2

[RFC PATCH 08/26] [media] imx-ipu: add ipu media common code

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

Add video4linux API routines common to IPU scaler, overlay, and deinterlacer
drivers.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Lucas Stach <l.stach@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/Kconfig       |   2 +
 drivers/media/platform/Makefile      |   1 +
 drivers/media/platform/imx/Kconfig   |   2 +
 drivers/media/platform/imx/Makefile  |   1 +
 drivers/media/platform/imx/imx-ipu.c | 313 +++++++++++++++++++++++++++++++++++
 drivers/media/platform/imx/imx-ipu.h |  35 ++++
 6 files changed, 354 insertions(+)
 create mode 100644 drivers/media/platform/imx/Kconfig
 create mode 100644 drivers/media/platform/imx/Makefile
 create mode 100644 drivers/media/platform/imx/imx-ipu.c
 create mode 100644 drivers/media/platform/imx/imx-ipu.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 20f1655..8e9c26c 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -29,6 +29,8 @@ config VIDEO_VIA_CAMERA
 
 source "drivers/media/platform/davinci/Kconfig"
 
+source "drivers/media/platform/imx/Kconfig"
+
 source "drivers/media/platform/omap/Kconfig"
 
 source "drivers/media/platform/blackfin/Kconfig"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index e5269da..ba114c0 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -48,6 +48,7 @@ obj-$(CONFIG_SOC_CAMERA)		+= soc_camera/
 obj-$(CONFIG_VIDEO_RENESAS_VSP1)	+= vsp1/
 
 obj-y	+= davinci/
+obj-y	+= imx/
 
 obj-$(CONFIG_ARCH_OMAP)	+= omap/
 
diff --git a/drivers/media/platform/imx/Kconfig b/drivers/media/platform/imx/Kconfig
new file mode 100644
index 0000000..a90c973
--- /dev/null
+++ b/drivers/media/platform/imx/Kconfig
@@ -0,0 +1,2 @@
+config VIDEO_IMX_IPU_COMMON
+	tristate
diff --git a/drivers/media/platform/imx/Makefile b/drivers/media/platform/imx/Makefile
new file mode 100644
index 0000000..5de119c
--- /dev/null
+++ b/drivers/media/platform/imx/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_VIDEO_IMX_IPU_COMMON)	+= imx-ipu.o
diff --git a/drivers/media/platform/imx/imx-ipu.c b/drivers/media/platform/imx/imx-ipu.c
new file mode 100644
index 0000000..c1b8637
--- /dev/null
+++ b/drivers/media/platform/imx/imx-ipu.c
@@ -0,0 +1,313 @@
+/*
+ * i.MX IPUv3 common v4l2 support
+ *
+ * Copyright (C) 2011 Sascha Hauer, Pengutronix
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+#include <linux/module.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-ioctl.h>
+
+#include "imx-ipu.h"
+
+static struct ipu_fmt ipu_fmt_yuv[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_YUV420,
+		.name = "YUV 4:2:0 planar, YCbCr",
+		.bytes_per_pixel = 1,
+	}, {
+		.fourcc = V4L2_PIX_FMT_YVU420,
+		.name = "YUV 4:2:0 planar, YCrCb",
+		.bytes_per_pixel = 1,
+	}, {
+		.fourcc = V4L2_PIX_FMT_YUV422P,
+		.name = "YUV 4:2:2 planar, YCbCr",
+		.bytes_per_pixel = 1,
+	}, {
+		.fourcc = V4L2_PIX_FMT_NV12,
+		.name = "YUV 4:2:0 partial interleaved, YCbCr",
+		.bytes_per_pixel = 1,
+	}, {
+		.fourcc = V4L2_PIX_FMT_UYVY,
+		.name = "4:2:2, packed, UYVY",
+		.bytes_per_pixel = 2,
+	}, {
+		.fourcc = V4L2_PIX_FMT_YUYV,
+		.name = "4:2:2, packed, YUYV",
+		.bytes_per_pixel = 2,
+	},
+};
+
+static struct ipu_fmt ipu_fmt_rgb[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_RGB32,
+		.name = "RGB888",
+		.bytes_per_pixel = 4,
+	}, {
+		.fourcc = V4L2_PIX_FMT_RGB24,
+		.name = "RGB24",
+		.bytes_per_pixel = 3,
+	}, {
+		.fourcc = V4L2_PIX_FMT_BGR24,
+		.name = "BGR24",
+		.bytes_per_pixel = 3,
+	}, {
+		.fourcc = V4L2_PIX_FMT_RGB565,
+		.name = "RGB565",
+		.bytes_per_pixel = 2,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_BGR32,
+		.name = "BGR888",
+		.bytes_per_pixel = 4,
+	},
+};
+
+struct ipu_fmt *ipu_find_fmt_yuv(unsigned int pixelformat)
+{
+	struct ipu_fmt *fmt;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(ipu_fmt_yuv); i++) {
+		fmt = &ipu_fmt_yuv[i];
+		if (fmt->fourcc == pixelformat)
+			return fmt;
+	}
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(ipu_find_fmt_yuv);
+
+struct ipu_fmt *ipu_find_fmt_rgb(unsigned int pixelformat)
+{
+	struct ipu_fmt *fmt;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(ipu_fmt_rgb); i++) {
+		fmt = &ipu_fmt_rgb[i];
+		if (fmt->fourcc == pixelformat)
+			return fmt;
+	}
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(ipu_find_fmt_rgb);
+
+static struct ipu_fmt *ipu_find_fmt(unsigned long pixelformat)
+{
+	struct ipu_fmt *fmt;
+
+	fmt = ipu_find_fmt_yuv(pixelformat);
+	if (fmt)
+		return fmt;
+	fmt = ipu_find_fmt_rgb(pixelformat);
+
+	return fmt;
+}
+EXPORT_SYMBOL_GPL(ipu_find_fmt);
+
+int ipu_try_fmt(struct file *file, void *fh,
+		struct v4l2_format *f)
+{
+	struct ipu_fmt *fmt;
+
+	v4l_bound_align_image(&f->fmt.pix.width, 8, 4096, 2,
+			      &f->fmt.pix.height, 2, 4096, 1, 0);
+
+	f->fmt.pix.field = V4L2_FIELD_NONE;
+
+	fmt = ipu_find_fmt(f->fmt.pix.pixelformat);
+	if (!fmt)
+		return -EINVAL;
+
+	f->fmt.pix.bytesperline = f->fmt.pix.width * fmt->bytes_per_pixel;
+	f->fmt.pix.sizeimage = f->fmt.pix.bytesperline * f->fmt.pix.height;
+	if (fmt->fourcc == V4L2_PIX_FMT_YUV420 ||
+	    fmt->fourcc == V4L2_PIX_FMT_YVU420 ||
+	    fmt->fourcc == V4L2_PIX_FMT_NV12)
+		f->fmt.pix.sizeimage = f->fmt.pix.sizeimage * 3 / 2;
+	else if (fmt->fourcc == V4L2_PIX_FMT_YUV422P)
+		f->fmt.pix.sizeimage *= 2;
+
+	f->fmt.pix.priv = 0;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_try_fmt);
+
+int ipu_try_fmt_rgb(struct file *file, void *fh,
+		struct v4l2_format *f)
+{
+	struct ipu_fmt *fmt;
+
+	fmt = ipu_find_fmt_rgb(f->fmt.pix.pixelformat);
+	if (!fmt)
+		return -EINVAL;
+
+	return ipu_try_fmt(file, fh, f);
+}
+EXPORT_SYMBOL_GPL(ipu_try_fmt_rgb);
+
+int ipu_try_fmt_yuv(struct file *file, void *fh,
+		struct v4l2_format *f)
+{
+	struct ipu_fmt *fmt;
+
+	fmt = ipu_find_fmt_yuv(f->fmt.pix.pixelformat);
+	if (!fmt)
+		return -EINVAL;
+
+	return ipu_try_fmt(file, fh, f);
+}
+EXPORT_SYMBOL_GPL(ipu_try_fmt_yuv);
+
+int ipu_enum_fmt_rgb(struct file *file, void *fh,
+		struct v4l2_fmtdesc *f)
+{
+	struct ipu_fmt *fmt;
+
+	if (f->index >= ARRAY_SIZE(ipu_fmt_rgb))
+		return -EINVAL;
+
+	fmt = &ipu_fmt_rgb[f->index];
+
+	strlcpy(f->description, fmt->name, sizeof(f->description));
+	f->pixelformat = fmt->fourcc;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_enum_fmt_rgb);
+
+int ipu_enum_fmt_yuv(struct file *file, void *fh,
+		struct v4l2_fmtdesc *f)
+{
+	struct ipu_fmt *fmt;
+
+	if (f->index >= ARRAY_SIZE(ipu_fmt_yuv))
+		return -EINVAL;
+
+	fmt = &ipu_fmt_yuv[f->index];
+
+	strlcpy(f->description, fmt->name, sizeof(f->description));
+	f->pixelformat = fmt->fourcc;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_enum_fmt_yuv);
+
+int ipu_enum_fmt(struct file *file, void *fh,
+		struct v4l2_fmtdesc *f)
+{
+	struct ipu_fmt *fmt;
+	int index = f->index;
+
+	if (index >= ARRAY_SIZE(ipu_fmt_yuv)) {
+		index -= ARRAY_SIZE(ipu_fmt_yuv);
+		if (index >= ARRAY_SIZE(ipu_fmt_rgb))
+			return -EINVAL;
+		fmt = &ipu_fmt_rgb[index];
+	} else {
+		fmt = &ipu_fmt_yuv[index];
+	}
+
+	strlcpy(f->description, fmt->name, sizeof(f->description));
+	f->pixelformat = fmt->fourcc;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_enum_fmt);
+
+int ipu_s_fmt(struct file *file, void *fh,
+		struct v4l2_format *f, struct v4l2_pix_format *pix)
+{
+	int ret;
+
+	ret = ipu_try_fmt(file, fh, f);
+	if (ret)
+		return ret;
+
+	pix->width = f->fmt.pix.width;
+	pix->height = f->fmt.pix.height;
+	pix->pixelformat = f->fmt.pix.pixelformat;
+	pix->bytesperline = f->fmt.pix.bytesperline;
+	pix->sizeimage = f->fmt.pix.sizeimage;
+	pix->colorspace = f->fmt.pix.colorspace;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_s_fmt);
+
+int ipu_s_fmt_rgb(struct file *file, void *fh,
+		struct v4l2_format *f, struct v4l2_pix_format *pix)
+{
+	struct ipu_fmt *fmt;
+
+	fmt = ipu_find_fmt_rgb(f->fmt.pix.pixelformat);
+	if (!fmt)
+		return -EINVAL;
+
+	return ipu_s_fmt(file, fh, f, pix);
+}
+EXPORT_SYMBOL_GPL(ipu_s_fmt_rgb);
+
+int ipu_s_fmt_yuv(struct file *file, void *fh,
+		struct v4l2_format *f, struct v4l2_pix_format *pix)
+{
+	struct ipu_fmt *fmt;
+
+	fmt = ipu_find_fmt_yuv(f->fmt.pix.pixelformat);
+	if (!fmt)
+		return -EINVAL;
+
+	return ipu_s_fmt(file, fh, f, pix);
+}
+EXPORT_SYMBOL_GPL(ipu_s_fmt_yuv);
+
+int ipu_g_fmt(struct v4l2_format *f, struct v4l2_pix_format *pix)
+{
+	f->fmt.pix.field = V4L2_FIELD_NONE;
+	f->fmt.pix.pixelformat = pix->pixelformat;
+	f->fmt.pix.bytesperline = pix->bytesperline;
+	f->fmt.pix.width = pix->width;
+	f->fmt.pix.height = pix->height;
+	f->fmt.pix.sizeimage = pix->sizeimage;
+	f->fmt.pix.colorspace = pix->colorspace;
+	f->fmt.pix.priv = 0;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_g_fmt);
+
+int ipu_enum_framesizes(struct file *file, void *fh,
+			struct v4l2_frmsizeenum *fsize)
+{
+	struct ipu_fmt *fmt;
+
+	if (fsize->index != 0)
+		return -EINVAL;
+
+	fmt = ipu_find_fmt(fsize->pixel_format);
+	if (!fmt)
+		return -EINVAL;
+
+	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+	fsize->stepwise.min_width = 1;
+	fsize->stepwise.min_height = 1;
+	fsize->stepwise.max_width = 4096;
+	fsize->stepwise.max_height = 4096;
+	fsize->stepwise.step_width = fsize->stepwise.step_height = 1;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_enum_framesizes);
+
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/imx/imx-ipu.h b/drivers/media/platform/imx/imx-ipu.h
new file mode 100644
index 0000000..51c0982
--- /dev/null
+++ b/drivers/media/platform/imx/imx-ipu.h
@@ -0,0 +1,35 @@
+#ifndef __MEDIA_IMX_IPU_H
+#define __MEDIA_IMX_IPU_H
+#include <linux/videodev2.h>
+
+struct ipu_fmt {
+	u32 fourcc;
+	const char *name;
+	int bytes_per_pixel;
+};
+
+int ipu_enum_fmt(struct file *file, void *fh,
+		struct v4l2_fmtdesc *f);
+int ipu_enum_fmt_rgb(struct file *file, void *fh,
+		struct v4l2_fmtdesc *f);
+int ipu_enum_fmt_yuv(struct file *file, void *fh,
+		struct v4l2_fmtdesc *f);
+struct ipu_fmt *ipu_find_fmt_rgb(unsigned int pixelformat);
+struct ipu_fmt *ipu_find_fmt_yuv(unsigned int pixelformat);
+int ipu_try_fmt(struct file *file, void *fh,
+		struct v4l2_format *f);
+int ipu_try_fmt_rgb(struct file *file, void *fh,
+		struct v4l2_format *f);
+int ipu_try_fmt_yuv(struct file *file, void *fh,
+		struct v4l2_format *f);
+int ipu_s_fmt(struct file *file, void *fh,
+		struct v4l2_format *f, struct v4l2_pix_format *pix);
+int ipu_s_fmt_rgb(struct file *file, void *fh,
+		struct v4l2_format *f, struct v4l2_pix_format *pix);
+int ipu_s_fmt_yuv(struct file *file, void *fh,
+		struct v4l2_format *f, struct v4l2_pix_format *pix);
+int ipu_g_fmt(struct v4l2_format *f, struct v4l2_pix_format *pix);
+int ipu_enum_framesizes(struct file *file, void *fh,
+			struct v4l2_frmsizeenum *fsize);
+
+#endif /* __MEDIA_IMX_IPU_H */
-- 
2.0.0.rc2

[RFC PATCH 09/26] [media] imx-ipu: Add i.MX IPUv3 scaler driver

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

This patch adds support for hardware accelerated scaling and color
space conversion between memory buffers using the IPUv3 IC.
Since the maximum output size of the IC unit is 1024x1024 pixels, multiple
IC tasks with overlapping tiles are used to scale and convert larger frames.

The IC operates with a burst size of at least 8 pixels. Depending on the
scaling factor, up to 7 junk pixels may be written after the end of the
frame. The sizeimage is increased accordingly.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/imx/Kconfig          |   9 +
 drivers/media/platform/imx/Makefile         |   1 +
 drivers/media/platform/imx/imx-ipu-scaler.c | 825 ++++++++++++++++++++++++++++
 drivers/media/platform/imx/imx-ipu.c        |   2 +-
 drivers/media/platform/imx/imx-ipu.h        |   1 +
 5 files changed, 837 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/imx/imx-ipu-scaler.c

diff --git a/drivers/media/platform/imx/Kconfig b/drivers/media/platform/imx/Kconfig
index a90c973..4694367 100644
--- a/drivers/media/platform/imx/Kconfig
+++ b/drivers/media/platform/imx/Kconfig
@@ -1,2 +1,11 @@
 config VIDEO_IMX_IPU_COMMON
 	tristate
+
+config VIDEO_IMX_IPU_SCALER
+	tristate "i.MX5/6 IPUv3 based image scaler driver"
+	depends on VIDEO_DEV && IMX_IPUV3_CORE
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEO_IMX_IPU_COMMON
+	select V4L2_MEM2MEM_DEV
+	---help---
+	  This is a v4l2 scaler video driver for the IPUv3 on i.MX5/6.
diff --git a/drivers/media/platform/imx/Makefile b/drivers/media/platform/imx/Makefile
index 5de119c..f20aa0b 100644
--- a/drivers/media/platform/imx/Makefile
+++ b/drivers/media/platform/imx/Makefile
@@ -1 +1,2 @@
 obj-$(CONFIG_VIDEO_IMX_IPU_COMMON)	+= imx-ipu.o
+obj-$(CONFIG_VIDEO_IMX_IPU_SCALER)	+= imx-ipu-scaler.o
diff --git a/drivers/media/platform/imx/imx-ipu-scaler.c b/drivers/media/platform/imx/imx-ipu-scaler.c
new file mode 100644
index 0000000..ced209a
--- /dev/null
+++ b/drivers/media/platform/imx/imx-ipu-scaler.c
@@ -0,0 +1,825 @@
+/*
+ * i.MX IPUv3 scaler driver
+ *
+ * Copyright (C) 2011 Sascha Hauer, Pengutronix
+ *
+ * based on the mem2mem test driver
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+#include <linux/module.h>
+#include <linux/delay.h>
+#include <linux/fs.h>
+#include <linux/version.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <video/imx-ipu-v3.h>
+
+#include <linux/platform_device.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "imx-ipu.h"
+
+#define MIN_W 32
+#define MIN_H 32
+#define MAX_W 4096
+#define MAX_H 4096
+#define DIM_ALIGN_MASK 0x08 /* 8-alignment for dimensions */
+
+/* Flags that indicate a format can be used for capture/output */
+#define MEM2MEM_CAPTURE	(1 << 0)
+#define MEM2MEM_OUTPUT	(1 << 1)
+
+#define MEM2MEM_NAME		"imx-ipuv3-scale"
+
+/* Per queue */
+#define MEM2MEM_DEF_NUM_BUFS	VIDEO_MAX_FRAME
+/* In bytes, per queue */
+#define MEM2MEM_VID_MEM_LIMIT	(64 * 1024 * 1024)
+
+#define fh_to_ctx(__fh)	container_of(__fh, struct ipu_scale_ctx, fh)
+
+enum {
+	V4L2_M2M_SRC = 0,
+	V4L2_M2M_DST = 1,
+};
+
+struct ipu_scale_dev {
+	struct v4l2_device	v4l2_dev;
+	struct video_device	*vfd;
+	struct device		*dev;
+	struct ipu_soc		*ipu;
+
+	atomic_t		num_inst;
+	spinlock_t		irqlock;
+
+	struct v4l2_m2m_dev	*m2m_dev;
+	struct mutex		dev_mutex;
+};
+
+/* Per-queue, driver-specific private data */
+struct ipu_scale_q_data {
+	struct v4l2_pix_format	cur_fmt;
+	struct v4l2_rect	rect;
+};
+
+struct ipu_scale_ctx {
+	struct ipu_scale_dev	*ipu_scaler;
+
+	struct v4l2_fh		fh;
+	struct vb2_alloc_ctx	*alloc_ctx;
+	struct ipu_scale_q_data	q_data[2];
+	struct work_struct	work;
+	struct completion	completion;
+	struct work_struct	skip_run;
+	int			error;
+	int			aborting;
+};
+
+static struct ipu_scale_q_data *get_q_data(struct ipu_scale_ctx *ctx,
+					   enum v4l2_buf_type type)
+{
+	switch (type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+		return &ctx->q_data[V4L2_M2M_SRC];
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+		return &ctx->q_data[V4L2_M2M_DST];
+	default:
+		BUG();
+	}
+	return NULL;
+}
+
+/*
+ * mem2mem callbacks
+ */
+
+/**
+ * job_ready() - check whether an instance is ready to be scheduled to run
+ */
+static int job_ready(void *priv)
+{
+	struct ipu_scale_ctx *ctx = priv;
+
+	if (ctx->aborting)
+		return 0;
+
+	if (v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) < 1
+	    || v4l2_m2m_num_dst_bufs_ready(ctx->fh.m2m_ctx) < 1) {
+		dev_dbg(ctx->ipu_scaler->dev, "Not enough buffers available\n");
+		return 0;
+	}
+
+	return 1;
+}
+
+static void job_abort(void *priv)
+{
+	struct ipu_scale_ctx *ctx = priv;
+
+	ctx->aborting = 1;
+}
+
+static void ipu_complete(void *priv, int err)
+{
+	struct ipu_scale_dev *ipu_scaler = priv;
+	struct ipu_scale_ctx *curr_ctx;
+
+	curr_ctx = v4l2_m2m_get_curr_priv(ipu_scaler->m2m_dev);
+
+	if (NULL == curr_ctx) {
+		dev_dbg(ipu_scaler->dev,
+			"Instance released before the end of transaction\n");
+		return;
+	}
+
+	curr_ctx->error = err;
+	complete(&curr_ctx->completion);
+}
+
+static void device_run(void *priv)
+{
+	struct ipu_scale_ctx *ctx = priv;
+
+	schedule_work(&ctx->work);
+}
+
+static void ipu_scaler_work(struct work_struct *work)
+{
+	struct ipu_scale_ctx *ctx = container_of(work, struct ipu_scale_ctx,
+						 work);
+	struct ipu_scale_dev *ipu_scaler = ctx->ipu_scaler;
+	struct vb2_buffer *src_buf, *dst_buf;
+	struct ipu_scale_q_data *q_data;
+	struct v4l2_pix_format *pix;
+	struct ipu_image in, out;
+	int err = -ETIMEDOUT;
+	unsigned long flags;
+
+	/*
+	 * If streamoff dequeued all buffers before we could get the lock,
+	 * just bail out immediately.
+	 */
+	if (!v4l2_m2m_num_src_bufs_ready(ctx->fh.m2m_ctx) ||
+		!v4l2_m2m_num_dst_bufs_ready(ctx->fh.m2m_ctx)) {
+		WARN_ON(1);
+		schedule_work(&ctx->skip_run);
+		return;
+	}
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+
+	q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	pix = &q_data->cur_fmt;
+
+	in.pix.width = pix->width;
+	in.pix.height = pix->height;
+	in.pix.bytesperline = pix->bytesperline;
+	in.pix.pixelformat = pix->pixelformat;
+	in.rect = q_data->rect;
+	in.phys = vb2_dma_contig_plane_dma_addr(src_buf, 0);
+
+	q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+	pix = &q_data->cur_fmt;
+
+	out.pix.width = pix->width;
+	out.pix.height = pix->height;
+	out.pix.bytesperline = pix->bytesperline;
+	out.pix.pixelformat = pix->pixelformat;
+	out.rect = q_data->rect;
+	out.phys = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+
+	ipu_image_convert(ipu_scaler->ipu, &in, &out, ipu_complete, ipu_scaler);
+
+	if (!wait_for_completion_timeout(&ctx->completion,
+					 msecs_to_jiffies(300))) {
+		dev_err(ipu_scaler->dev,
+			"Timeout waiting for scaling result\n");
+		err = -ETIMEDOUT;
+	} else {
+		err = ctx->error;
+	}
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+	dst_buf->v4l2_buf.timestamp = src_buf->v4l2_buf.timestamp;
+	dst_buf->v4l2_buf.timecode = src_buf->v4l2_buf.timecode;
+
+	spin_lock_irqsave(&ipu_scaler->irqlock, flags);
+	v4l2_m2m_buf_done(src_buf, err ? VB2_BUF_STATE_ERROR :
+					 VB2_BUF_STATE_DONE);
+	v4l2_m2m_buf_done(dst_buf, err ? VB2_BUF_STATE_ERROR :
+					 VB2_BUF_STATE_DONE);
+	spin_unlock_irqrestore(&ipu_scaler->irqlock, flags);
+
+	v4l2_m2m_job_finish(ipu_scaler->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+/*
+ * video ioctls
+ */
+static int vidioc_querycap(struct file *file, void *priv,
+			   struct v4l2_capability *cap)
+{
+	strncpy(cap->driver, MEM2MEM_NAME, sizeof(cap->driver) - 1);
+	strncpy(cap->card, MEM2MEM_NAME, sizeof(cap->card) - 1);
+	strncpy(cap->bus_info, "platform:" MEM2MEM_NAME,
+		sizeof(cap->bus_info) - 1);
+	/*
+	 * This is only a mem-to-mem video device. The capture and output
+	 * device capability flags are left for backward compatibility and
+	 * are scheduled for removal.
+	 */
+	cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT |
+			   V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+
+	return 0;
+}
+
+static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return ipu_enum_fmt(file, priv, f);
+}
+
+static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return ipu_enum_fmt(file, priv, f);
+}
+
+static int vidioc_g_fmt(struct ipu_scale_ctx *ctx, struct v4l2_format *f)
+{
+	struct vb2_queue *vq;
+	struct ipu_scale_q_data *q_data;
+	struct v4l2_pix_format *pix;
+
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
+	if (!vq)
+		return -EINVAL;
+
+	q_data = get_q_data(ctx, f->type);
+	pix = &q_data->cur_fmt;
+
+	return ipu_g_fmt(f, pix);
+}
+
+static int vidioc_g_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(priv, f);
+}
+
+static int vidioc_g_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(priv, f);
+}
+
+static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	int ret;
+
+	ret = ipu_try_fmt(file, priv, f);
+
+	/*
+	 * Leave enough output space for worst-case overhead caused by 8 pixel
+	 * burst size: 7 RGBA pixels.
+	 */
+	f->fmt.pix.sizeimage += 7 * 4;
+
+	return ret;
+}
+
+static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	f->fmt.pix.width &= ~0x7;
+	return ipu_try_fmt(file, priv, f);
+}
+
+static int vidioc_s_fmt(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	struct ipu_scale_q_data *q_data;
+	struct vb2_queue *vq;
+	struct ipu_scale_ctx *ctx = fh_to_ctx(priv);
+	int ret;
+
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
+	if (!vq)
+		return -EINVAL;
+
+	q_data = get_q_data(ctx, f->type);
+	if (!q_data)
+		return -EINVAL;
+
+	if (vb2_is_busy(vq)) {
+		v4l2_err(&ctx->ipu_scaler->v4l2_dev, "%s queue busy\n",
+			 __func__);
+		return -EBUSY;
+	}
+
+	ret = ipu_s_fmt(file, priv, f, &q_data->cur_fmt);
+	if (ret < 0)
+		return ret;
+
+	/*
+	 * Leave enough output space for worst-case overhead caused by 8 pixel
+	 * burst size: 7 RGBA pixels.
+	 */
+	f->fmt.pix.sizeimage += 7 * 4;
+	q_data->cur_fmt.sizeimage = f->fmt.pix.sizeimage;
+
+	/* Reset cropping/composing rectangle */
+	q_data->rect.left = 0;
+	q_data->rect.top = 0;
+	q_data->rect.width = q_data->cur_fmt.width;
+	q_data->rect.height = q_data->cur_fmt.height;
+
+	return 0;
+}
+
+static int vidioc_g_selection(struct file *file, void *priv,
+			      struct v4l2_selection *s)
+{
+	struct ipu_scale_ctx *ctx = fh_to_ctx(priv);
+	struct ipu_scale_q_data *q_data;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = q_data->cur_fmt.width;
+		s->r.height = q_data->cur_fmt.height;
+		break;
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = q_data->cur_fmt.width;
+		s->r.height = q_data->cur_fmt.height;
+		break;
+	case V4L2_SEL_TGT_COMPOSE:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+		s->r = q_data->rect;
+		break;
+	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+	case V4L2_SEL_TGT_COMPOSE_PADDED:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = q_data->cur_fmt.width;
+		s->r.height = q_data->cur_fmt.height;
+		break;
+	}
+
+	return 0;
+}
+
+static int vidioc_s_selection(struct file *file, void *priv,
+			      struct v4l2_selection *s)
+{
+	struct ipu_scale_ctx *ctx = fh_to_ctx(priv);
+	struct ipu_scale_q_data *q_data;
+	struct vb2_queue *vq;
+
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, s->type);
+	if (!vq)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_CROP:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+		break;
+	case V4L2_SEL_TGT_COMPOSE:
+		q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	/* The input's frame width to the IC must be a multiple of 8 pixels
+	 * When performing resizing the frame width must be multiple of burst
+	 * size - 8 or 16 pixels as defined by CB#_BURST_16 parameter.
+	 */
+	if (s->flags & V4L2_SEL_FLAG_GE)
+		s->r.width = round_up(s->r.width, 8);
+	if (s->flags & V4L2_SEL_FLAG_LE)
+		s->r.width = round_down(s->r.width, 8);
+	s->r.width = clamp_t(unsigned int, s->r.width, 8,
+			     round_down(q_data->cur_fmt.width, 8));
+	s->r.height = clamp_t(unsigned int, s->r.height, 1,
+			      q_data->cur_fmt.height);
+	s->r.left = clamp_t(unsigned int, s->r.left, 0,
+			    q_data->cur_fmt.width - s->r.width);
+	s->r.top = clamp_t(unsigned int, s->r.top, 0,
+			   q_data->cur_fmt.height - s->r.height);
+
+	/* V4L2_SEL_FLAG_KEEP_CONFIG is only valid for subdevices */
+	q_data->rect = s->r;
+
+	return 0;
+}
+
+static int vidioc_enum_framesizes(struct file *file, void *fh,
+				  struct v4l2_frmsizeenum *fsize)
+{
+	return ipu_enum_framesizes(file, fh, fsize);
+}
+
+static const struct v4l2_ioctl_ops ipu_scale_ioctl_ops = {
+	.vidioc_querycap	= vidioc_querycap,
+
+	.vidioc_enum_fmt_vid_cap = vidioc_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap	= vidioc_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap	= vidioc_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap	= vidioc_s_fmt,
+
+	.vidioc_enum_fmt_vid_out = vidioc_enum_fmt_vid_out,
+	.vidioc_g_fmt_vid_out	= vidioc_g_fmt_vid_out,
+	.vidioc_try_fmt_vid_out	= vidioc_try_fmt_vid_out,
+	.vidioc_s_fmt_vid_out	= vidioc_s_fmt,
+
+	.vidioc_g_selection	= vidioc_g_selection,
+	.vidioc_s_selection	= vidioc_s_selection,
+
+	.vidioc_reqbufs		= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf	= v4l2_m2m_ioctl_querybuf,
+
+	.vidioc_qbuf		= v4l2_m2m_ioctl_qbuf,
+	.vidioc_expbuf		= v4l2_m2m_ioctl_expbuf,
+	.vidioc_dqbuf		= v4l2_m2m_ioctl_dqbuf,
+	.vidioc_create_bufs	= v4l2_m2m_ioctl_create_bufs,
+
+	.vidioc_streamon	= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff	= v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_enum_framesizes = vidioc_enum_framesizes,
+};
+
+static void ipu_scale_skip_run(struct work_struct *work)
+{
+	struct ipu_scale_ctx *ctx = container_of(work, struct ipu_scale_ctx,
+						 skip_run);
+
+	v4l2_m2m_job_finish(ctx->ipu_scaler->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+
+/*
+ * Queue operations
+ */
+
+static int ipu_scale_queue_setup(struct vb2_queue *vq,
+		const struct v4l2_format *fmt,
+		unsigned int *nbuffers,
+		unsigned int *nplanes, unsigned int sizes[],
+		void *alloc_ctxs[])
+{
+	struct ipu_scale_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ipu_scale_q_data *q_data;
+	unsigned int size, count = *nbuffers;
+	struct v4l2_pix_format *pix;
+
+	q_data = get_q_data(ctx, vq->type);
+	pix = &q_data->cur_fmt;
+
+	size = pix->sizeimage;
+
+	while (size * count > MEM2MEM_VID_MEM_LIMIT)
+		(count)--;
+
+	*nplanes = 1;
+	*nbuffers = count;
+	sizes[0] = size;
+
+	ctx->alloc_ctx = vb2_dma_contig_init_ctx(ctx->ipu_scaler->dev);
+	if (IS_ERR(ctx->alloc_ctx))
+		return PTR_ERR(ctx->alloc_ctx);
+
+	alloc_ctxs[0] = ctx->alloc_ctx;
+
+	dev_dbg(ctx->ipu_scaler->dev, "get %d buffer(s) of size %d each.\n",
+		count, size);
+
+	return 0;
+}
+
+static int ipu_scale_buf_prepare(struct vb2_buffer *vb)
+{
+	struct ipu_scale_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct ipu_scale_q_data *q_data;
+	struct v4l2_pix_format *pix;
+
+	dev_dbg(ctx->ipu_scaler->dev, "type: %d\n", vb->vb2_queue->type);
+
+	q_data = get_q_data(ctx, vb->vb2_queue->type);
+	pix = &q_data->cur_fmt;
+
+	if (vb2_plane_size(vb, 0) < pix->sizeimage) {
+		dev_dbg(ctx->ipu_scaler->dev,
+				"%s data will not fit into plane (%lu < %lu)\n",
+				__func__, vb2_plane_size(vb, 0),
+				(long)pix->sizeimage);
+		return -EINVAL;
+	}
+
+	vb2_set_plane_payload(vb, 0, pix->sizeimage);
+
+	return 0;
+}
+
+static void ipu_scale_buf_queue(struct vb2_buffer *vb)
+{
+	struct ipu_scale_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vb);
+}
+
+static int ipu_scale_stop_streaming(struct vb2_queue *q)
+{
+	struct ipu_scale_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_buffer *buf;
+
+	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) {
+		while ((buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx)))
+			v4l2_m2m_buf_done(buf, VB2_BUF_STATE_ERROR);
+	} else {
+		while ((buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx)))
+			v4l2_m2m_buf_done(buf, VB2_BUF_STATE_ERROR);
+	}
+
+	return 0;
+}
+
+static struct vb2_ops ipu_scale_qops = {
+	.queue_setup	= ipu_scale_queue_setup,
+	.buf_prepare	= ipu_scale_buf_prepare,
+	.buf_queue	= ipu_scale_buf_queue,
+	.wait_prepare	= vb2_ops_wait_prepare,
+	.wait_finish	= vb2_ops_wait_finish,
+	.stop_streaming = ipu_scale_stop_streaming,
+};
+
+static int queue_init(void *priv, struct vb2_queue *src_vq,
+		      struct vb2_queue *dst_vq)
+{
+	struct ipu_scale_ctx *ctx = priv;
+	int ret;
+
+	memset(src_vq, 0, sizeof(*src_vq));
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
+	src_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	src_vq->ops = &ipu_scale_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->ipu_scaler->dev_mutex;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	memset(dst_vq, 0, sizeof(*dst_vq));
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	dst_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops = &ipu_scale_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->ipu_scaler->dev_mutex;
+
+	return vb2_queue_init(dst_vq);
+}
+
+/*
+ * File operations
+ */
+static int ipu_scale_open(struct file *file)
+{
+	struct ipu_scale_dev *ipu_scaler = video_drvdata(file);
+	struct ipu_scale_ctx *ctx = NULL;
+	const int width = 720;
+	const int height = 576;
+	int i;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	INIT_WORK(&ctx->skip_run, ipu_scale_skip_run);
+	INIT_WORK(&ctx->work, ipu_scaler_work);
+	init_completion(&ctx->completion);
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+	ctx->ipu_scaler = ipu_scaler;
+
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(ipu_scaler->m2m_dev, ctx,
+					    &queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		int ret = PTR_ERR(ctx->fh.m2m_ctx);
+
+		kfree(ctx);
+		return ret;
+	}
+
+	for (i = 0; i < 2; i++) {
+		ctx->q_data[i].cur_fmt.width = width;
+		ctx->q_data[i].cur_fmt.height = height;
+		ctx->q_data[i].cur_fmt.bytesperline = width;
+		ctx->q_data[i].cur_fmt.pixelformat = V4L2_PIX_FMT_YUV420;
+		ctx->q_data[i].cur_fmt.sizeimage = width * height * 3 / 2;
+		ctx->q_data[i].cur_fmt.colorspace = V4L2_COLORSPACE_REC709;
+		ctx->q_data[i].rect.left = 0;
+		ctx->q_data[i].rect.top = 0;
+		ctx->q_data[i].rect.width = width;
+		ctx->q_data[i].rect.height = height;
+	}
+
+	atomic_inc(&ipu_scaler->num_inst);
+
+	dev_dbg(ipu_scaler->dev, "Created instance %p, m2m_ctx: %p\n",
+			ctx, ctx->fh.m2m_ctx);
+
+	return 0;
+}
+
+static int ipu_scale_release(struct file *file)
+{
+	struct ipu_scale_dev *ipu_scaler = video_drvdata(file);
+	struct ipu_scale_ctx *ctx = fh_to_ctx(file->private_data);
+
+	dev_dbg(ipu_scaler->dev, "Releasing instance %p\n", ctx);
+
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	kfree(ctx);
+
+	atomic_dec(&ipu_scaler->num_inst);
+
+	return 0;
+}
+
+static const struct v4l2_file_operations ipu_scale_fops = {
+	.owner		= THIS_MODULE,
+	.open		= ipu_scale_open,
+	.release	= ipu_scale_release,
+	.poll		= v4l2_m2m_fop_poll,
+	.unlocked_ioctl	= video_ioctl2,
+	.mmap		= v4l2_m2m_fop_mmap,
+};
+
+static struct video_device ipu_scale_videodev = {
+	.name		= MEM2MEM_NAME,
+	.fops		= &ipu_scale_fops,
+	.ioctl_ops	= &ipu_scale_ioctl_ops,
+	.minor		= -1,
+	.release	= video_device_release,
+	.vfl_dir	= VFL_DIR_M2M,
+};
+
+static struct v4l2_m2m_ops m2m_ops = {
+	.device_run	= device_run,
+	.job_ready	= job_ready,
+	.job_abort	= job_abort,
+};
+
+static u64 vout_dmamask = ~(u32)0;
+
+static int ipu_scale_probe(struct platform_device *pdev)
+{
+	struct ipu_scale_dev *ipu_scaler;
+	struct video_device *vfd;
+	struct ipu_soc *ipu = dev_get_drvdata(pdev->dev.parent);
+	int ret;
+
+	pdev->dev.dma_mask = &vout_dmamask;
+	pdev->dev.coherent_dma_mask = 0xffffffff;
+
+	ipu_scaler = devm_kzalloc(&pdev->dev, sizeof(*ipu_scaler), GFP_KERNEL);
+	if (!ipu_scaler)
+		return -ENOMEM;
+
+	ipu_scaler->ipu = ipu;
+	ipu_scaler->dev = &pdev->dev;
+
+	spin_lock_init(&ipu_scaler->irqlock);
+	mutex_init(&ipu_scaler->dev_mutex);
+
+	ret = v4l2_device_register(&pdev->dev, &ipu_scaler->v4l2_dev);
+	if (ret)
+		return ret;
+
+	atomic_set(&ipu_scaler->num_inst, 0);
+
+	vfd = video_device_alloc();
+	if (!vfd) {
+		dev_err(ipu_scaler->dev, "Failed to allocate video device\n");
+		ret = -ENOMEM;
+		goto unreg_dev;
+	}
+
+	*vfd = ipu_scale_videodev;
+	vfd->lock = &ipu_scaler->dev_mutex;
+	vfd->v4l2_dev = &ipu_scaler->v4l2_dev;
+
+	ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
+	if (ret) {
+		dev_err(ipu_scaler->dev, "Failed to register video device\n");
+		goto rel_vdev;
+	}
+
+	video_set_drvdata(vfd, ipu_scaler);
+	snprintf(vfd->name, sizeof(vfd->name), "%s", ipu_scale_videodev.name);
+	ipu_scaler->vfd = vfd;
+	dev_dbg(ipu_scaler->dev, "Device registered as /dev/video%d\n",
+		vfd->num);
+
+	platform_set_drvdata(pdev, ipu_scaler);
+
+	ipu_scaler->m2m_dev = v4l2_m2m_init(&m2m_ops);
+	if (IS_ERR(ipu_scaler->m2m_dev)) {
+		dev_err(ipu_scaler->dev, "Failed to init mem2mem device\n");
+		ret = PTR_ERR(ipu_scaler->m2m_dev);
+		goto err_m2m;
+	}
+
+	return 0;
+
+	v4l2_m2m_release(ipu_scaler->m2m_dev);
+err_m2m:
+	video_unregister_device(ipu_scaler->vfd);
+rel_vdev:
+	video_device_release(vfd);
+unreg_dev:
+	v4l2_device_unregister(&ipu_scaler->v4l2_dev);
+
+	return ret;
+}
+
+static int ipu_scale_remove(struct platform_device *pdev)
+{
+	struct ipu_scale_dev *ipu_scaler =
+		(struct ipu_scale_dev *)platform_get_drvdata(pdev);
+
+	v4l2_m2m_release(ipu_scaler->m2m_dev);
+	video_unregister_device(ipu_scaler->vfd);
+	v4l2_device_unregister(&ipu_scaler->v4l2_dev);
+	kfree(ipu_scaler);
+
+	return 0;
+}
+
+static const struct platform_device_id ipu_scale_id[] = {
+	{ "imx-ipuv3-scaler" },
+	{}
+};
+MODULE_DEVICE_TABLE(platform, ipu_scale_id);
+
+static struct platform_driver ipu_scale_pdrv = {
+	.probe		= ipu_scale_probe,
+	.remove		= ipu_scale_remove,
+	.driver		= {
+		.name	= "imx-ipuv3-scaler",
+		.owner	= THIS_MODULE,
+	},
+};
+
+static void __exit ipu_scale_exit(void)
+{
+	platform_driver_unregister(&ipu_scale_pdrv);
+}
+
+static int __init ipu_scale_init(void)
+{
+	return  platform_driver_register(&ipu_scale_pdrv);
+}
+
+module_init(ipu_scale_init);
+module_exit(ipu_scale_exit);
+
+MODULE_DESCRIPTION("Virtual device for mem2mem framework testing");
+MODULE_AUTHOR("Sascha Hauer <s.hauer@pengutronix.de>");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/imx/imx-ipu.c b/drivers/media/platform/imx/imx-ipu.c
index c1b8637..402cc8b 100644
--- a/drivers/media/platform/imx/imx-ipu.c
+++ b/drivers/media/platform/imx/imx-ipu.c
@@ -102,7 +102,7 @@ struct ipu_fmt *ipu_find_fmt_rgb(unsigned int pixelformat)
 }
 EXPORT_SYMBOL_GPL(ipu_find_fmt_rgb);
 
-static struct ipu_fmt *ipu_find_fmt(unsigned long pixelformat)
+struct ipu_fmt *ipu_find_fmt(unsigned long pixelformat)
 {
 	struct ipu_fmt *fmt;
 
diff --git a/drivers/media/platform/imx/imx-ipu.h b/drivers/media/platform/imx/imx-ipu.h
index 51c0982..288ee79 100644
--- a/drivers/media/platform/imx/imx-ipu.h
+++ b/drivers/media/platform/imx/imx-ipu.h
@@ -16,6 +16,7 @@ int ipu_enum_fmt_yuv(struct file *file, void *fh,
 		struct v4l2_fmtdesc *f);
 struct ipu_fmt *ipu_find_fmt_rgb(unsigned int pixelformat);
 struct ipu_fmt *ipu_find_fmt_yuv(unsigned int pixelformat);
+struct ipu_fmt *ipu_find_fmt(unsigned long pixelformat);
 int ipu_try_fmt(struct file *file, void *fh,
 		struct v4l2_format *f);
 int ipu_try_fmt_rgb(struct file *file, void *fh,
-- 
2.0.0.rc2

[RFC PATCH 10/26] [media] imx-ipu: Add i.MX IPUv3 deinterlacer driver

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

This adds a video4linux mem2mem driver that deinterlaces input
frames using the IPUv3 VDIC module.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/imx/Kconfig        |   9 +
 drivers/media/platform/imx/Makefile       |   1 +
 drivers/media/platform/imx/imx-ipu-vdic.c | 716 ++++++++++++++++++++++++++++++
 3 files changed, 726 insertions(+)
 create mode 100644 drivers/media/platform/imx/imx-ipu-vdic.c

diff --git a/drivers/media/platform/imx/Kconfig b/drivers/media/platform/imx/Kconfig
index 4694367..506326a 100644
--- a/drivers/media/platform/imx/Kconfig
+++ b/drivers/media/platform/imx/Kconfig
@@ -9,3 +9,12 @@ config VIDEO_IMX_IPU_SCALER
 	select V4L2_MEM2MEM_DEV
 	---help---
 	  This is a v4l2 scaler video driver for the IPUv3 on i.MX5/6.
+
+config VIDEO_IMX_IPU_VDIC
+	tristate "i.MX5/6 IPUv3 based deinterlacer driver"
+	depends on VIDEO_DEV && IMX_IPUV3_CORE
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEO_IMX_IPU_COMMON
+	select V4L2_MEM2MEM_DEV
+	---help---
+	  This is a v4l2 deinterlacer video driver for the IPUv3 on i.MX5/6.
diff --git a/drivers/media/platform/imx/Makefile b/drivers/media/platform/imx/Makefile
index f20aa0b..5c9da82 100644
--- a/drivers/media/platform/imx/Makefile
+++ b/drivers/media/platform/imx/Makefile
@@ -1,2 +1,3 @@
 obj-$(CONFIG_VIDEO_IMX_IPU_COMMON)	+= imx-ipu.o
 obj-$(CONFIG_VIDEO_IMX_IPU_SCALER)	+= imx-ipu-scaler.o
+obj-$(CONFIG_VIDEO_IMX_IPU_VDIC)	+= imx-ipu-vdic.o
diff --git a/drivers/media/platform/imx/imx-ipu-vdic.c b/drivers/media/platform/imx/imx-ipu-vdic.c
new file mode 100644
index 0000000..f4c7a31
--- /dev/null
+++ b/drivers/media/platform/imx/imx-ipu-vdic.c
@@ -0,0 +1,716 @@
+/*
+ * i.MX IPUv3 vdic driver
+ *
+ * Copyright (C) 2011 Sascha Hauer, Pengutronix
+ *
+ * based on the mem2mem test driver
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+#include <linux/module.h>
+#include <linux/delay.h>
+#include <linux/fs.h>
+#include <linux/version.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <video/imx-ipu-v3.h>
+
+#include <linux/platform_device.h>
+#include <media/v4l2-mem2mem.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "imx-ipu.h"
+
+#define MIN_W 32
+#define MIN_H 32
+#define MAX_W 4096
+#define MAX_H 4096
+#define DIM_ALIGN_MASK 0x08 /* 8-alignment for dimensions */
+
+/* Flags that indicate a format can be used for capture/output */
+#define MEM2MEM_CAPTURE	(1 << 0)
+#define MEM2MEM_OUTPUT	(1 << 1)
+
+#define MEM2MEM_NAME		"imx-ipuv3-vdic"
+
+/* Per queue */
+#define MEM2MEM_DEF_NUM_BUFS	VIDEO_MAX_FRAME
+/* In bytes, per queue */
+#define MEM2MEM_VID_MEM_LIMIT	(64 * 1024 * 1024)
+
+#define fh_to_ctx(__fh)	container_of(__fh, struct ipu_vdic_ctx, fh)
+
+enum {
+	V4L2_M2M_SRC = 0,
+	V4L2_M2M_DST = 1,
+};
+
+struct ipu_vdic_dev {
+	struct v4l2_device	v4l2_dev;
+	struct video_device	*vfd;
+	struct device		*dev;
+	struct ipu_soc		*ipu;
+
+	atomic_t		num_inst;
+	spinlock_t		irqlock;
+
+	struct v4l2_m2m_dev	*m2m_dev;
+	struct mutex		dev_mutex;
+	struct ipuv3_channel	*ipu_ch[4];
+};
+
+/* Per-queue, driver-specific private data */
+struct ipu_vdic_q_data {
+	struct v4l2_pix_format	cur_fmt;
+};
+
+struct ipu_vdic_ctx {
+	struct ipu_vdic_dev	*ipu_vdic;
+
+	struct v4l2_fh		fh;
+	struct vb2_alloc_ctx	*alloc_ctx;
+	struct ipu_vdic_q_data	q_data[2];
+
+	struct vb2_buffer *in_p, *in;
+};
+
+static struct ipu_vdic_q_data *get_q_data(struct ipu_vdic_ctx *ctx,
+					  enum v4l2_buf_type type)
+{
+	switch (type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+		return &ctx->q_data[V4L2_M2M_SRC];
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+		return &ctx->q_data[V4L2_M2M_DST];
+	default:
+		BUG();
+	}
+	return NULL;
+}
+
+/*
+ * mem2mem callbacks
+ */
+
+static void job_abort(void *priv)
+{
+}
+
+static void ipu_complete(void *priv, int err)
+{
+	struct ipu_vdic_dev *ipu_vdic = priv;
+	struct ipu_vdic_ctx *ctx;
+	struct vb2_buffer *dst_vb;
+	unsigned long flags;
+
+	ctx = v4l2_m2m_get_curr_priv(ipu_vdic->m2m_dev);
+
+	if (!ctx) {
+		dev_dbg(ipu_vdic->dev,
+			"Instance released before the end of transaction\n");
+		return;
+	}
+
+	spin_lock_irqsave(&ipu_vdic->irqlock, flags);
+
+	if (ctx->in_p != ctx->in)
+		v4l2_m2m_buf_done(ctx->in_p, err ? VB2_BUF_STATE_ERROR :
+						   VB2_BUF_STATE_DONE);
+	ctx->in_p = ctx->in;
+
+	dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	dst_vb->v4l2_buf.timestamp = ctx->in->v4l2_buf.timestamp;
+	dst_vb->v4l2_buf.timecode = ctx->in->v4l2_buf.timecode;
+	v4l2_m2m_buf_done(dst_vb, err ? VB2_BUF_STATE_ERROR :
+					VB2_BUF_STATE_DONE);
+
+	spin_unlock_irqrestore(&ipu_vdic->irqlock, flags);
+
+	v4l2_m2m_job_finish(ipu_vdic->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+static void device_run(void *priv)
+{
+	struct ipu_vdic_ctx *ctx = priv;
+	struct ipu_vdic_dev *ipu_vdic = ctx->ipu_vdic;
+	struct vb2_buffer *dst_buf;
+	struct ipu_vdic_q_data *q_data;
+	struct v4l2_pix_format *pix;
+	struct ipu_image image_in[3];
+	struct ipu_image image_out = {};
+	int i;
+
+	ctx->in = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+
+	if (!ctx->in_p)
+		ctx->in_p = ctx->in;
+
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+
+	q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT);
+	pix = &q_data->cur_fmt;
+
+	for (i = 0; i < 3; i++) {
+		struct ipu_image *image = &image_in[i];
+
+		memset(image, 0, sizeof(*image));
+		image->pix.width = pix->width;
+		image->pix.height = pix->height / 2;
+		image->pix.bytesperline = pix->bytesperline;
+		image->pix.pixelformat = pix->pixelformat;
+		image->rect.left = 0;
+		image->rect.top = 0;
+		image->rect.width = pix->width;
+		image->rect.height = pix->height / 2;
+	}
+	if (pix->field == V4L2_FIELD_INTERLACED_BT) {
+		for (i = 0; i < 3; i++)
+			image_in[i].pix.bytesperline = pix->bytesperline * 2;
+
+		image_in[0].phys = vb2_dma_contig_plane_dma_addr(ctx->in_p, 0);
+		image_in[1].phys = vb2_dma_contig_plane_dma_addr(ctx->in, 0) +
+			pix->bytesperline;
+		image_in[2].phys = vb2_dma_contig_plane_dma_addr(ctx->in, 0);
+	} else if (pix->field == V4L2_FIELD_INTERLACED_TB) {
+		for (i = 0; i < 3; i++)
+			image_in[i].pix.bytesperline = pix->bytesperline * 2;
+
+		image_in[0].phys = vb2_dma_contig_plane_dma_addr(ctx->in_p, 0) +
+			pix->bytesperline;
+		image_in[1].phys = vb2_dma_contig_plane_dma_addr(ctx->in, 0);
+		image_in[2].phys = vb2_dma_contig_plane_dma_addr(ctx->in, 0) +
+			pix->bytesperline;
+	} else {
+		image_in[0].phys = vb2_dma_contig_plane_dma_addr(ctx->in_p, 0) +
+			pix->bytesperline * pix->height / 2;
+		image_in[1].phys = vb2_dma_contig_plane_dma_addr(ctx->in, 0);
+		image_in[2].phys = vb2_dma_contig_plane_dma_addr(ctx->in, 0) +
+			pix->bytesperline * pix->height / 2;
+	}
+
+	q_data = get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE);
+	pix = &q_data->cur_fmt;
+
+	image_out.pix.width = pix->width;
+	image_out.pix.height = pix->height;
+	image_out.pix.bytesperline = pix->bytesperline;
+	image_out.pix.pixelformat = pix->pixelformat;
+	image_out.rect.left = 0;
+	image_out.rect.top = 0;
+	image_out.rect.width = pix->width;
+	image_out.rect.height = pix->height;
+	image_out.phys = vb2_dma_contig_plane_dma_addr(dst_buf, 0);
+
+	ipu_image_deinterlace_convert(ipu_vdic->ipu, &image_in[0], &image_in[1],
+				      &image_in[2], &image_out, ipu_complete,
+				      ipu_vdic);
+}
+
+/*
+ * video ioctls
+ */
+static int vidioc_querycap(struct file *file, void *priv,
+			   struct v4l2_capability *cap)
+{
+	strncpy(cap->driver, MEM2MEM_NAME, sizeof(cap->driver) - 1);
+	strncpy(cap->card, MEM2MEM_NAME, sizeof(cap->card) - 1);
+	strncpy(cap->bus_info, "platform:" MEM2MEM_NAME,
+		sizeof(cap->bus_info) - 1);
+	/*
+	 * This is only a mem-to-mem video device. The capture and output
+	 * device capability flags are left for backward compatibility and
+	 * are scheduled for removal.
+	 */
+	cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_VIDEO_OUTPUT |
+			   V4L2_CAP_VIDEO_M2M | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+
+	return 0;
+}
+
+static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return ipu_enum_fmt(file, priv, f);
+}
+
+static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return ipu_enum_fmt(file, priv, f);
+}
+
+static int vidioc_g_fmt(struct ipu_vdic_ctx *ctx, struct v4l2_format *f)
+{
+	struct vb2_queue *vq;
+	struct ipu_vdic_q_data *q_data;
+	struct v4l2_pix_format *pix;
+
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
+	if (!vq)
+		return -EINVAL;
+
+	q_data = get_q_data(ctx, f->type);
+	pix = &q_data->cur_fmt;
+
+	return ipu_g_fmt(f, pix);
+}
+
+static int vidioc_g_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(priv, f);
+}
+
+static int vidioc_g_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(priv, f);
+}
+
+static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	return ipu_try_fmt(file, priv, f);
+}
+
+static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	enum v4l2_field field = f->fmt.pix.field;
+	int ret;
+
+	switch (f->fmt.pix.field) {
+	case V4L2_FIELD_SEQ_BT:
+	case V4L2_FIELD_INTERLACED_BT:
+		pr_warn("%s: %s not supported yet\n", __func__,
+			v4l2_field_names[f->fmt.pix.field]);
+		/* fallthrough */
+	case V4L2_FIELD_ANY:
+	case V4L2_FIELD_NONE:
+	case V4L2_FIELD_TOP:
+	case V4L2_FIELD_BOTTOM:
+	case V4L2_FIELD_INTERLACED:
+		field = V4L2_FIELD_INTERLACED_TB;
+		break;
+	case V4L2_FIELD_SEQ_TB:
+	case V4L2_FIELD_INTERLACED_TB:
+		break;
+	}
+
+	ret = ipu_try_fmt(file, priv, f);
+
+	f->fmt.pix.field = field;
+
+	return ret;
+}
+
+static int vidioc_s_fmt(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	struct ipu_vdic_q_data *q_data;
+	struct vb2_queue *vq;
+	struct ipu_vdic_ctx *ctx = fh_to_ctx(priv);
+	enum v4l2_field field = V4L2_FIELD_ANY;
+	int ret;
+
+	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
+	if (!vq)
+		return -EINVAL;
+
+	q_data = get_q_data(ctx, f->type);
+	if (!q_data)
+		return -EINVAL;
+
+	if (vb2_is_busy(vq)) {
+		v4l2_err(&ctx->ipu_vdic->v4l2_dev, "%s queue busy\n", __func__);
+		return -EBUSY;
+	}
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) {
+		vidioc_try_fmt_vid_out(file, priv, f);
+		field = f->fmt.pix.field;
+	}
+
+	ret = ipu_s_fmt(file, priv, f, &q_data->cur_fmt);
+
+	if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) {
+		f->fmt.pix.field = field;
+		q_data->cur_fmt.field = field;
+	}
+
+	return ret;
+}
+
+static int vidioc_enum_framesizes(struct file *file, void *fh,
+				  struct v4l2_frmsizeenum *fsize)
+{
+	return ipu_enum_framesizes(file, fh, fsize);
+}
+
+static const struct v4l2_ioctl_ops ipu_vdic_ioctl_ops = {
+	.vidioc_querycap	= vidioc_querycap,
+
+	.vidioc_enum_fmt_vid_cap = vidioc_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap	= vidioc_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap	= vidioc_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap	= vidioc_s_fmt,
+
+	.vidioc_enum_fmt_vid_out = vidioc_enum_fmt_vid_out,
+	.vidioc_g_fmt_vid_out	= vidioc_g_fmt_vid_out,
+	.vidioc_try_fmt_vid_out	= vidioc_try_fmt_vid_out,
+	.vidioc_s_fmt_vid_out	= vidioc_s_fmt,
+
+	.vidioc_reqbufs		= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf	= v4l2_m2m_ioctl_querybuf,
+
+	.vidioc_qbuf		= v4l2_m2m_ioctl_qbuf,
+	.vidioc_expbuf		= v4l2_m2m_ioctl_expbuf,
+	.vidioc_dqbuf		= v4l2_m2m_ioctl_dqbuf,
+	.vidioc_create_bufs	= v4l2_m2m_ioctl_create_bufs,
+
+	.vidioc_streamon	= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff	= v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_enum_framesizes = vidioc_enum_framesizes,
+};
+
+
+/*
+ * Queue operations
+ */
+
+static int ipu_vdic_queue_setup(struct vb2_queue *vq,
+		const struct v4l2_format *fmt,
+		unsigned int *nbuffers,
+		unsigned int *nplanes, unsigned int sizes[],
+		void *alloc_ctxs[])
+{
+	struct ipu_vdic_ctx *ctx = vb2_get_drv_priv(vq);
+	struct ipu_vdic_q_data *q_data;
+	unsigned int size, count = *nbuffers;
+	struct v4l2_pix_format *pix;
+
+	q_data = get_q_data(ctx, vq->type);
+	pix = &q_data->cur_fmt;
+
+	size = pix->sizeimage;
+
+	while (size * count > MEM2MEM_VID_MEM_LIMIT)
+		(count)--;
+
+	*nplanes = 1;
+	*nbuffers = count;
+	sizes[0] = size;
+
+	ctx->alloc_ctx = vb2_dma_contig_init_ctx(ctx->ipu_vdic->dev);
+	if (IS_ERR(ctx->alloc_ctx))
+		return PTR_ERR(ctx->alloc_ctx);
+
+	alloc_ctxs[0] = ctx->alloc_ctx;
+
+	dev_dbg(ctx->ipu_vdic->dev, "get %d buffer(s) of size %d each.\n",
+		count, size);
+
+	return 0;
+}
+
+static int ipu_vdic_buf_prepare(struct vb2_buffer *vb)
+{
+	struct ipu_vdic_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct ipu_vdic_q_data *q_data;
+	struct v4l2_pix_format *pix;
+
+	dev_dbg(ctx->ipu_vdic->dev, "type: %d\n", vb->vb2_queue->type);
+
+	q_data = get_q_data(ctx, vb->vb2_queue->type);
+	pix = &q_data->cur_fmt;
+
+	if (vb2_plane_size(vb, 0) < pix->sizeimage) {
+		dev_dbg(ctx->ipu_vdic->dev,
+				"%s data will not fit into plane (%lu < %lu)\n",
+				__func__, vb2_plane_size(vb, 0),
+				(long)pix->sizeimage);
+		return -EINVAL;
+	}
+
+	vb2_set_plane_payload(vb, 0, pix->sizeimage);
+
+	return 0;
+}
+
+static void ipu_vdic_buf_queue(struct vb2_buffer *vb)
+{
+	struct ipu_vdic_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vb);
+}
+
+static int ipu_vdic_stop_streaming(struct vb2_queue *q)
+{
+	struct ipu_vdic_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_buffer *buf;
+
+	if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT) {
+		while ((buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx)))
+			v4l2_m2m_buf_done(buf, VB2_BUF_STATE_ERROR);
+		if (ctx->in_p) {
+			v4l2_m2m_buf_done(ctx->in_p, VB2_BUF_STATE_ERROR);
+			ctx->in_p = NULL;
+		}
+	} else {
+		while ((buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx)))
+			v4l2_m2m_buf_done(buf, VB2_BUF_STATE_ERROR);
+	}
+
+	return 0;
+}
+
+static struct vb2_ops ipu_vdic_qops = {
+	.queue_setup	= ipu_vdic_queue_setup,
+	.buf_prepare	= ipu_vdic_buf_prepare,
+	.buf_queue	= ipu_vdic_buf_queue,
+	.wait_prepare	= vb2_ops_wait_prepare,
+	.wait_finish	= vb2_ops_wait_finish,
+	.stop_streaming = ipu_vdic_stop_streaming,
+};
+
+static int queue_init(void *priv, struct vb2_queue *src_vq,
+		      struct vb2_queue *dst_vq)
+{
+	struct ipu_vdic_ctx *ctx = priv;
+	int ret;
+
+	memset(src_vq, 0, sizeof(*src_vq));
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
+	src_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	src_vq->ops = &ipu_vdic_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->ipu_vdic->dev_mutex;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	memset(dst_vq, 0, sizeof(*dst_vq));
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	dst_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct v4l2_m2m_buffer);
+	dst_vq->ops = &ipu_vdic_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->ipu_vdic->dev_mutex;
+
+	return vb2_queue_init(dst_vq);
+}
+
+/*
+ * File operations
+ */
+static int ipu_vdic_open(struct file *file)
+{
+	struct ipu_vdic_dev *ipu_vdic = video_drvdata(file);
+	struct ipu_vdic_ctx *ctx = NULL;
+	const int width = 720;
+	const int height = 576;
+	int i;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx)
+		return -ENOMEM;
+
+	file->private_data = ctx;
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	v4l2_fh_add(&ctx->fh);
+	ctx->ipu_vdic = ipu_vdic;
+
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(ipu_vdic->m2m_dev, ctx,
+					    &queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		int ret = PTR_ERR(ctx->fh.m2m_ctx);
+
+		kfree(ctx);
+		return ret;
+	}
+
+	for (i = 0; i < 2; i++) {
+		ctx->q_data[i].cur_fmt.width = width;
+		ctx->q_data[i].cur_fmt.height = height;
+		ctx->q_data[i].cur_fmt.bytesperline = width;
+		ctx->q_data[i].cur_fmt.pixelformat = V4L2_PIX_FMT_YUV420;
+		ctx->q_data[i].cur_fmt.sizeimage = width * height * 3 / 2;
+		ctx->q_data[i].cur_fmt.colorspace = V4L2_COLORSPACE_REC709;
+	}
+
+	atomic_inc(&ipu_vdic->num_inst);
+
+	dev_dbg(ipu_vdic->dev, "Created instance %p, m2m_ctx: %p\n",
+			ctx, ctx->fh.m2m_ctx);
+
+	return 0;
+}
+
+static int ipu_vdic_release(struct file *file)
+{
+	struct ipu_vdic_dev *ipu_vdic = video_drvdata(file);
+	struct ipu_vdic_ctx *ctx = fh_to_ctx(file->private_data);
+
+	dev_dbg(ipu_vdic->dev, "Releasing instance %p\n", ctx);
+
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	kfree(ctx);
+
+	atomic_dec(&ipu_vdic->num_inst);
+
+	return 0;
+}
+
+static const struct v4l2_file_operations ipu_vdic_fops = {
+	.owner		= THIS_MODULE,
+	.open		= ipu_vdic_open,
+	.release	= ipu_vdic_release,
+	.poll		= v4l2_m2m_fop_poll,
+	.unlocked_ioctl	= video_ioctl2,
+	.mmap		= v4l2_m2m_fop_mmap,
+};
+
+static struct video_device ipu_vdic_videodev = {
+	.name		= MEM2MEM_NAME,
+	.fops		= &ipu_vdic_fops,
+	.ioctl_ops	= &ipu_vdic_ioctl_ops,
+	.minor		= -1,
+	.release	= video_device_release,
+	.vfl_dir	= VFL_DIR_M2M,
+};
+
+static struct v4l2_m2m_ops m2m_ops = {
+	.device_run	= device_run,
+	.job_abort	= job_abort,
+};
+
+static u64 vout_dmamask = ~(u32)0;
+
+static int ipu_vdic_probe(struct platform_device *pdev)
+{
+	struct ipu_vdic_dev *ipu_vdic;
+	struct video_device *vfd;
+	struct ipu_soc *ipu = dev_get_drvdata(pdev->dev.parent);
+	int ret;
+
+	pdev->dev.dma_mask = &vout_dmamask;
+	pdev->dev.coherent_dma_mask = 0xffffffff;
+
+	ipu_vdic = devm_kzalloc(&pdev->dev, sizeof(*ipu_vdic), GFP_KERNEL);
+	if (!ipu_vdic)
+		return -ENOMEM;
+
+	ipu_vdic->ipu = ipu;
+	ipu_vdic->dev = &pdev->dev;
+
+	spin_lock_init(&ipu_vdic->irqlock);
+	mutex_init(&ipu_vdic->dev_mutex);
+
+	ret = v4l2_device_register(&pdev->dev, &ipu_vdic->v4l2_dev);
+	if (ret)
+		return ret;
+
+	atomic_set(&ipu_vdic->num_inst, 0);
+
+	vfd = video_device_alloc();
+	if (!vfd) {
+		dev_err(ipu_vdic->dev, "Failed to allocate video device\n");
+		ret = -ENOMEM;
+		goto unreg_dev;
+	}
+
+	*vfd = ipu_vdic_videodev;
+	vfd->lock = &ipu_vdic->dev_mutex;
+	vfd->v4l2_dev = &ipu_vdic->v4l2_dev;
+
+	ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
+	if (ret) {
+		dev_err(ipu_vdic->dev, "Failed to register video device\n");
+		goto rel_vdev;
+	}
+
+	video_set_drvdata(vfd, ipu_vdic);
+	snprintf(vfd->name, sizeof(vfd->name), "%s", ipu_vdic_videodev.name);
+	ipu_vdic->vfd = vfd;
+	dev_dbg(ipu_vdic->dev, "Device registered as /dev/video%d\n", vfd->num);
+
+	platform_set_drvdata(pdev, ipu_vdic);
+
+	ipu_vdic->m2m_dev = v4l2_m2m_init(&m2m_ops);
+	if (IS_ERR(ipu_vdic->m2m_dev)) {
+		dev_err(ipu_vdic->dev, "Failed to init mem2mem device\n");
+		ret = PTR_ERR(ipu_vdic->m2m_dev);
+		goto err_m2m;
+	}
+
+	return 0;
+
+	v4l2_m2m_release(ipu_vdic->m2m_dev);
+err_m2m:
+	video_unregister_device(ipu_vdic->vfd);
+rel_vdev:
+	video_device_release(vfd);
+unreg_dev:
+	v4l2_device_unregister(&ipu_vdic->v4l2_dev);
+
+	return ret;
+}
+
+static int ipu_vdic_remove(struct platform_device *pdev)
+{
+	struct ipu_vdic_dev *ipu_vdic =
+		(struct ipu_vdic_dev *)platform_get_drvdata(pdev);
+
+	v4l2_m2m_release(ipu_vdic->m2m_dev);
+	video_unregister_device(ipu_vdic->vfd);
+	v4l2_device_unregister(&ipu_vdic->v4l2_dev);
+	kfree(ipu_vdic);
+
+	return 0;
+}
+
+static struct platform_driver ipu_vdic_pdrv = {
+	.probe		= ipu_vdic_probe,
+	.remove		= ipu_vdic_remove,
+	.driver		= {
+		.name	= "imx-ipuv3-vdic",
+		.owner	= THIS_MODULE,
+	},
+};
+
+static void __exit ipu_vdic_exit(void)
+{
+	platform_driver_unregister(&ipu_vdic_pdrv);
+}
+
+static int __init ipu_vdic_init(void)
+{
+	return  platform_driver_register(&ipu_vdic_pdrv);
+}
+
+module_init(ipu_vdic_init);
+module_exit(ipu_vdic_exit);
+
+MODULE_DESCRIPTION("Virtual device for mem2mem framework testing");
+MODULE_AUTHOR("Sascha Hauer <s.hauer@pengutronix.de>");
+MODULE_LICENSE("GPL");
-- 
2.0.0.rc2

[RFC PATCH 11/26] [media] v4l2: subdev: Add v4l2_device_register_subdev_node function

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

We currently only have a function that registers all subdev
device nodes for a v4l2 device at once. This assumes that
there is a point when all subdevices are known and that all
subdevices are needed to make a functional device. With the
advent of asynchronous subdevices this may no longer be the
case, so add a function which registers a single subdevice for
a given v4l2 device only and let v4l2_device_register_subdev_nodes
use this function.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 drivers/media/v4l2-core/v4l2-device.c | 63 ++++++++++++++++++++---------------
 include/media/v4l2-device.h           |  5 +++
 2 files changed, 42 insertions(+), 26 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-device.c b/drivers/media/v4l2-core/v4l2-device.c
index 02d1b63..4211163 100644
--- a/drivers/media/v4l2-core/v4l2-device.c
+++ b/drivers/media/v4l2-core/v4l2-device.c
@@ -205,9 +205,43 @@ static void v4l2_device_release_subdev_node(struct video_device *vdev)
 	kfree(vdev);
 }
 
-int v4l2_device_register_subdev_nodes(struct v4l2_device *v4l2_dev)
+int v4l2_device_register_subdev_node(struct v4l2_device *v4l2_dev,
+		struct v4l2_subdev *sd)
 {
 	struct video_device *vdev;
+	int err;
+
+	if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
+		return 0;
+
+	vdev = kzalloc(sizeof(*vdev), GFP_KERNEL);
+	if (!vdev)
+		return -ENOMEM;
+
+	video_set_drvdata(vdev, sd);
+	strlcpy(vdev->name, sd->name, sizeof(vdev->name));
+	vdev->v4l2_dev = v4l2_dev;
+	vdev->fops = &v4l2_subdev_fops;
+	vdev->release = v4l2_device_release_subdev_node;
+	vdev->ctrl_handler = sd->ctrl_handler;
+	err = __video_register_device(vdev, VFL_TYPE_SUBDEV, -1, 1,
+				      sd->owner);
+	if (err < 0) {
+		kfree(vdev);
+		return err;
+	}
+#if defined(CONFIG_MEDIA_CONTROLLER)
+	sd->entity.info.v4l.major = VIDEO_MAJOR;
+	sd->entity.info.v4l.minor = vdev->minor;
+#endif
+	sd->devnode = vdev;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(v4l2_device_register_subdev_node);
+
+int v4l2_device_register_subdev_nodes(struct v4l2_device *v4l2_dev)
+{
 	struct v4l2_subdev *sd;
 	int err;
 
@@ -215,32 +249,9 @@ int v4l2_device_register_subdev_nodes(struct v4l2_device *v4l2_dev)
 	 * V4L2_SUBDEV_FL_HAS_DEVNODE flag.
 	 */
 	list_for_each_entry(sd, &v4l2_dev->subdevs, list) {
-		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE))
-			continue;
-
-		vdev = kzalloc(sizeof(*vdev), GFP_KERNEL);
-		if (!vdev) {
-			err = -ENOMEM;
-			goto clean_up;
-		}
-
-		video_set_drvdata(vdev, sd);
-		strlcpy(vdev->name, sd->name, sizeof(vdev->name));
-		vdev->v4l2_dev = v4l2_dev;
-		vdev->fops = &v4l2_subdev_fops;
-		vdev->release = v4l2_device_release_subdev_node;
-		vdev->ctrl_handler = sd->ctrl_handler;
-		err = __video_register_device(vdev, VFL_TYPE_SUBDEV, -1, 1,
-					      sd->owner);
-		if (err < 0) {
-			kfree(vdev);
+		err = v4l2_device_register_subdev_node(v4l2_dev, sd);
+		if (err)
 			goto clean_up;
-		}
-#if defined(CONFIG_MEDIA_CONTROLLER)
-		sd->entity.info.v4l.major = VIDEO_MAJOR;
-		sd->entity.info.v4l.minor = vdev->minor;
-#endif
-		sd->devnode = vdev;
 	}
 	return 0;
 
diff --git a/include/media/v4l2-device.h b/include/media/v4l2-device.h
index c9b1593..512cc4b 100644
--- a/include/media/v4l2-device.h
+++ b/include/media/v4l2-device.h
@@ -120,6 +120,11 @@ void v4l2_device_unregister_subdev(struct v4l2_subdev *sd);
 int __must_check
 v4l2_device_register_subdev_nodes(struct v4l2_device *v4l2_dev);
 
+/* Register a single device node for a subdev of a v4l2 device. */
+int __must_check
+v4l2_device_register_subdev_node(struct v4l2_device *v4l2_dev,
+				struct v4l2_subdev *sd);
+
 /* Iterate over all subdevs. */
 #define v4l2_device_for_each_subdev(sd, v4l2_dev)			\
 	list_for_each_entry(sd, &(v4l2_dev)->subdevs, list)
-- 
2.0.0.rc2

[RFC PATCH 12/26] [media] v4l2: Fix V4L2_CID_PIXEL_RATE

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 55c6832..6a6ccc5 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1048,7 +1048,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_PIXEL_RATE:
 		*type = V4L2_CTRL_TYPE_INTEGER64;
 		*flags |= V4L2_CTRL_FLAG_READ_ONLY;
-		*min = *max = *step = *def = 0;
+		*min = *max = *step = 0;
 		break;
 	default:
 		*type = V4L2_CTRL_TYPE_INTEGER;
@@ -1710,7 +1710,11 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 	else if (type == V4L2_CTRL_TYPE_INTEGER_MENU)
 		ctrl->qmenu_int = qmenu_int;
 	ctrl->priv = priv;
-	ctrl->cur.val = ctrl->val = ctrl->default_value = def;
+
+	if (type == V4L2_CTRL_TYPE_INTEGER64)
+		ctrl->val64 = ctrl->cur.val64 = def;
+	else
+		ctrl->cur.val = ctrl->val = ctrl->default_value = def;
 
 	if (ctrl->type == V4L2_CTRL_TYPE_STRING) {
 		ctrl->cur.string = (char *)&ctrl[1] + sz_extra - (max + 1);
-- 
2.0.0.rc2

[RFC PATCH 13/26] [media] v4l2 async: remove from notifier list

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

In v4l2_async_notifier_register remove the notifier from the notifier
list in case of an error.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 drivers/media/v4l2-core/v4l2-async.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
index 85a6a34..1e1e58c 100644
--- a/drivers/media/v4l2-core/v4l2-async.c
+++ b/drivers/media/v4l2-core/v4l2-async.c
@@ -173,6 +173,7 @@ int v4l2_async_notifier_register(struct v4l2_device *v4l2_dev,
 
 		ret = v4l2_async_test_notify(notifier, sd, asd);
 		if (ret < 0) {
+			list_del(&notifier->list);
 			mutex_unlock(&list_lock);
 			return ret;
 		}
-- 
2.0.0.rc2

[RFC PATCH 14/26] [media] Add i.MX SoC wide media device driver

[Philipp Zabel]

This driver registers a single, SoC wide media device, which all entities
in the media graph can be registered with via OF graph bindings.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/Kconfig         |   2 +
 drivers/media/platform/imx/Kconfig     |  13 +++
 drivers/media/platform/imx/Makefile    |   1 +
 drivers/media/platform/imx/imx-media.c | 174 +++++++++++++++++++++++++++++++++
 include/media/imx.h                    |  25 +++++
 5 files changed, 215 insertions(+)
 create mode 100644 drivers/media/platform/imx/imx-media.c
 create mode 100644 include/media/imx.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 8e9c26c..3fe7e28 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -35,6 +35,8 @@ source "drivers/media/platform/omap/Kconfig"
 
 source "drivers/media/platform/blackfin/Kconfig"
 
+source "drivers/media/platform/imx/Kconfig"
+
 config VIDEO_SH_VOU
 	tristate "SuperH VOU video output driver"
 	depends on MEDIA_CAMERA_SUPPORT
diff --git a/drivers/media/platform/imx/Kconfig b/drivers/media/platform/imx/Kconfig
index 506326a..d2f1693 100644
--- a/drivers/media/platform/imx/Kconfig
+++ b/drivers/media/platform/imx/Kconfig
@@ -1,3 +1,16 @@
+config MEDIA_IMX
+	tristate "Multimedia Support for Freescale i.MX"
+	depends on MEDIA_CONTROLLER
+	---help---
+	  This driver provides a SoC wide media controller device that all
+	  multimedia components in i.MX5 and i.MX6 SoCs can register with.
+
+config MEDIA_IMX_IPU
+	tristate "i.MX Image Processing Unit (v3) core driver"
+	---help---
+	  This driver provides core support for the i.MX IPUv3 contained in
+	  i.MX5 and i.MX6 SoCs.
+
 config VIDEO_IMX_IPU_COMMON
 	tristate
 
diff --git a/drivers/media/platform/imx/Makefile b/drivers/media/platform/imx/Makefile
index 5c9da82..60f451a 100644
--- a/drivers/media/platform/imx/Makefile
+++ b/drivers/media/platform/imx/Makefile
@@ -1,3 +1,4 @@
+obj-$(CONFIG_MEDIA_IMX)			+= imx-media.o
 obj-$(CONFIG_VIDEO_IMX_IPU_COMMON)	+= imx-ipu.o
 obj-$(CONFIG_VIDEO_IMX_IPU_SCALER)	+= imx-ipu-scaler.o
 obj-$(CONFIG_VIDEO_IMX_IPU_VDIC)	+= imx-ipu-vdic.o
diff --git a/drivers/media/platform/imx/imx-media.c b/drivers/media/platform/imx/imx-media.c
new file mode 100644
index 0000000..ae63c49
--- /dev/null
+++ b/drivers/media/platform/imx/imx-media.c
@@ -0,0 +1,174 @@
+#include <linux/module.h>
+#include <linux/export.h>
+#include <linux/types.h>
+#include <linux/init.h>
+#include <linux/err.h>
+#include <linux/spinlock.h>
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+#include <media/media-device.h>
+#include <media/v4l2-device.h>
+#include <video/imx-ipu-v3.h>
+
+struct ipu_media_controller {
+	struct v4l2_device v4l2_dev;
+	struct media_device mdev;
+};
+
+static struct ipu_media_controller *ipu_media;
+
+struct media_device *ipu_find_media_device(void)
+{
+	return &ipu_media->mdev;
+}
+EXPORT_SYMBOL_GPL(ipu_find_media_device);
+
+struct ipu_media_link {
+	struct v4l2_async_notifier	asn;
+	struct v4l2_async_subdev	asd;
+	struct v4l2_async_subdev	*asdp;
+	struct v4l2_subdev		*sd;
+	int padno;
+	struct device_node		*endpoint;
+	u32				media_link_flags;
+};
+
+static int ipu_media_bound(struct v4l2_async_notifier *notifier,
+			   struct v4l2_subdev *sd,
+			   struct v4l2_async_subdev *asd)
+{
+	struct ipu_media_controller *im = ipu_media;
+	struct ipu_media_link *link = container_of(notifier,
+						   struct ipu_media_link, asn);
+	struct device_node *np, *rp;
+	uint32_t portno = 0;
+	int ret;
+
+	if ((sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE)) {
+		ret = v4l2_device_register_subdev_node(&im->v4l2_dev, sd);
+		if (ret)
+			return ret;
+	}
+
+	np = link->endpoint;
+	rp = of_graph_get_remote_port(np);
+	of_property_read_u32(rp, "reg", &portno);
+
+	ret = media_entity_create_link(&sd->entity, portno, &link->sd->entity,
+			link->padno, link->media_link_flags);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static void ipu_media_unbind(struct v4l2_async_notifier *notifier,
+			     struct v4l2_subdev *sd,
+			     struct v4l2_async_subdev *asd)
+{
+	if ((sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE)) {
+		video_unregister_device(sd->devnode);
+		kfree(sd->devnode);
+	}
+}
+
+struct ipu_media_link *ipu_media_entity_create_link(struct v4l2_subdev *sd,
+		int padno, struct device_node *endpoint,
+		u32 media_link_flags)
+{
+	struct ipu_media_controller *im = ipu_media;
+	struct ipu_media_link *link;
+	int ret;
+	struct device_node *rpp;
+
+	rpp = of_graph_get_remote_port_parent(endpoint);
+	if (!rpp)
+		return ERR_PTR(-EINVAL);
+
+	pr_info("%s: link on %s pad %d endpoint: %s remotenodeparent: %s\n",
+		__func__, sd->name, padno, endpoint->full_name, rpp->full_name);
+	if (!im)
+		return ERR_PTR(-ENODEV);
+
+	link = kzalloc(sizeof(*link), GFP_KERNEL);
+	if (!link)
+		return ERR_PTR(-ENOMEM);
+
+	link->sd = sd;
+	link->padno = padno;
+	link->endpoint = endpoint;
+	link->media_link_flags = media_link_flags;
+
+	link->asd.match_type = V4L2_ASYNC_MATCH_OF;
+	link->asd.match.of.node = rpp;
+
+	link->asdp = &link->asd;
+
+	link->asn.bound = ipu_media_bound;
+	link->asn.unbind = ipu_media_unbind;
+	link->asn.subdevs = &link->asdp;
+	link->asn.num_subdevs = 1;
+	link->asn.v4l2_dev = &im->v4l2_dev;
+
+	ret = v4l2_async_notifier_register(&im->v4l2_dev, &link->asn);
+	if (ret) {
+		kfree(link);
+		return ERR_PTR(ret);
+	}
+
+	return link;
+}
+EXPORT_SYMBOL_GPL(ipu_media_entity_create_link);
+
+void ipu_media_entity_remove_link(struct ipu_media_link *link)
+{
+	v4l2_async_notifier_unregister(&link->asn);
+
+	kfree(link);
+}
+EXPORT_SYMBOL_GPL(ipu_media_entity_remove_link);
+
+struct v4l2_device *ipu_media_get_v4l2_dev(void)
+{
+	if (!ipu_media)
+		return NULL;
+
+	return &ipu_media->v4l2_dev;
+}
+EXPORT_SYMBOL_GPL(ipu_media_get_v4l2_dev);
+
+int ipu_media_device_register(struct device *dev)
+{
+	struct media_device *mdev;
+	int ret;
+
+	if (ipu_media)
+		return 0;
+
+	ipu_media = devm_kzalloc(dev, sizeof(*ipu_media), GFP_KERNEL);
+	if (!ipu_media)
+		return -ENOMEM;
+
+	mdev = &ipu_media->mdev;
+
+	mdev->dev = dev;
+
+	strlcpy(mdev->model, "i.MX IPUv3", sizeof(mdev->model));
+
+	ret = media_device_register(mdev);
+	if (ret)
+		return ret;
+
+	ipu_media->v4l2_dev.mdev = mdev;
+
+	ret = v4l2_device_register(mdev->dev, &ipu_media->v4l2_dev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ipu_media_device_register);
diff --git a/include/media/imx.h b/include/media/imx.h
new file mode 100644
index 0000000..42be5f4
--- /dev/null
+++ b/include/media/imx.h
@@ -0,0 +1,25 @@
+struct v4l2_subdev;
+struct device_node;
+struct ipu_media_link;
+struct v4l2_device;
+struct media_device;
+struct device;
+
+struct ipu_media_link *ipu_media_entity_create_link(struct v4l2_subdev *sd,
+		int padno, struct device_node *remote_node,
+		u32 media_link_flags);
+
+void ipu_media_entity_remove_link(struct ipu_media_link *link);
+
+struct v4l2_device *ipu_media_get_v4l2_dev(void);
+
+struct media_device *ipu_find_media_device(void);
+
+#ifdef CONFIG_MEDIA_IMX
+int ipu_media_device_register(struct device *dev);
+#else
+static inline int ipu_media_device_register(struct device *dev)
+{
+	return 0;
+}
+#endif
-- 
2.0.0.rc2

[RFC PATCH 15/26] [media] imx-ipu: Add i.MX IPUv3 capture driver

[Philipp Zabel]

This driver uses the IDMA controller's double buffering feature to do the
processing of finished frames in the new frame acknowledge (NFACK) interrupt
handler while the next frame is already being captured. This avoids a race
condition between the end of frame interrupt and NFACK for very short blanking
intervals. On the other hand, the driver needs at least two buffers. It will
never hand out any frames if there is only one.
It supports interlaced input and allows to translate between sequential and
interlaced field formats using the IDMAC scan order and interlace offset
parameters.
If the source entity connected to the CSI subdevice sink pad is a multiplexer,
the mbus_format can change dynamically when the media graph is reconfigured.
Use the current source mbus_format and only then fall back to the static
format read from the device tree.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Marc Kleine-Budde <mkl@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/imx/Kconfig         |    9 +
 drivers/media/platform/imx/Makefile        |    1 +
 drivers/media/platform/imx/imx-ipuv3-csi.c | 1527 ++++++++++++++++++++++++++++
 3 files changed, 1537 insertions(+)
 create mode 100644 drivers/media/platform/imx/imx-ipuv3-csi.c

diff --git a/drivers/media/platform/imx/Kconfig b/drivers/media/platform/imx/Kconfig
index d2f1693..8431441 100644
--- a/drivers/media/platform/imx/Kconfig
+++ b/drivers/media/platform/imx/Kconfig
@@ -14,6 +14,15 @@ config MEDIA_IMX_IPU
 config VIDEO_IMX_IPU_COMMON
 	tristate
 
+config VIDEO_IMX_IPU_CSI
+	tristate "i.MX5/6 CMOS Sensor Interface driver"
+	depends on VIDEO_DEV && IMX_IPUV3_CORE && MEDIA_CONTROLLER
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEO_IMX_IPU_COMMON
+	select VIDEO_IMX_IPUV3
+	---help---
+	  This is a v4l2 video capture driver for the IPUv3 on i.MX5/6.
+
 config VIDEO_IMX_IPU_SCALER
 	tristate "i.MX5/6 IPUv3 based image scaler driver"
 	depends on VIDEO_DEV && IMX_IPUV3_CORE
diff --git a/drivers/media/platform/imx/Makefile b/drivers/media/platform/imx/Makefile
index 60f451a..49d8fab 100644
--- a/drivers/media/platform/imx/Makefile
+++ b/drivers/media/platform/imx/Makefile
@@ -1,4 +1,5 @@
 obj-$(CONFIG_MEDIA_IMX)			+= imx-media.o
 obj-$(CONFIG_VIDEO_IMX_IPU_COMMON)	+= imx-ipu.o
+obj-$(CONFIG_VIDEO_IMX_IPU_CSI)		+= imx-ipuv3-csi.o
 obj-$(CONFIG_VIDEO_IMX_IPU_SCALER)	+= imx-ipu-scaler.o
 obj-$(CONFIG_VIDEO_IMX_IPU_VDIC)	+= imx-ipu-vdic.o
diff --git a/drivers/media/platform/imx/imx-ipuv3-csi.c b/drivers/media/platform/imx/imx-ipuv3-csi.c
new file mode 100644
index 0000000..d9326a8
--- /dev/null
+++ b/drivers/media/platform/imx/imx-ipuv3-csi.c
@@ -0,0 +1,1527 @@
+/*
+ * V4L2 Driver for i.MXL/i.MXL camera (CSI) host
+ *
+ * Copyright (C) 2008, Paulius Zaleckas <paulius.zaleckas@teltonika.lt>
+ * Copyright (C) 2009, Darius Augulis <augulis.darius@gmail.com>
+ *
+ * Based on PXA SoC camera driver
+ * Copyright (C) 2006, Sascha Hauer, Pengutronix
+ * Copyright (C) 2008, Guennadi Liakhovetski <kernel@pengutronix.de>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/platform_device.h>
+#include <linux/dma-mapping.h>
+#include <linux/moduleparam.h>
+#include <linux/interrupt.h>
+#include <linux/videodev2.h>
+#include <linux/version.h>
+#include <linux/device.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/delay.h>
+#include <linux/errno.h>
+#include <linux/mutex.h>
+#include <linux/sched.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <linux/time.h>
+#include <linux/clk.h>
+#include <linux/fs.h>
+#include <linux/io.h>
+#include <linux/mm.h>
+
+#include <video/imx-ipu-v3.h>
+#include "imx-ipu.h"
+
+#include <media/imx.h>
+#include <linux/of_graph.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-of.h>
+
+#define DRIVER_NAME "imx-ipuv3-camera"
+
+/* CMOS Sensor Interface Registers */
+#define CSI_SENS_CONF		0x0000
+#define CSI_SENS_FRM_SIZE	0x0004
+#define CSI_ACT_FRM_SIZE	0x0008
+#define CSI_OUT_FRM_CTRL	0x000c
+#define CSI_TST_CTRL		0x0010
+#define CSI_CCIR_CODE_1		0x0014
+#define CSI_CCIR_CODE_2		0x0018
+#define CSI_CCIR_CODE_3		0x001c
+#define CSI_MIPI_DI		0x0020
+#define CSI_SKIP		0x0024
+#define CSI_CPD_CTRL		0x0028
+#define CSI_CPD_RC(n)		(0x002c + 4 * (n))
+#define CSI_CPD_RS(n)		(0x004c + 4 * (n))
+#define CSI_CPD_GRC(n)		(0x005c + 4 * (n))
+#define CSI_CPD_GRS(n)		(0x007c + 4 * (n))
+#define CSI_CPD_GBC(n)		(0x008c + 4 * (n))
+#define CSI_CPD_GBS(n)		(0x00ac + 4 * (n))
+#define CSI_CPD_BC(n)		(0x00bc + 4 * (n))
+#define CSI_CPD_BS(n)		(0x00dc + 4 * (n))
+#define CSI_CPD_OFFSET1		0x00ec
+#define CSI_CPD_OFFSET2		0x00f0
+
+#define CSI_SENS_CONF_VSYNC_POL			(1 << 0)
+#define CSI_SENS_CONF_HSYNC_POL			(1 << 1)
+#define CSI_SENS_CONF_DATA_POL			(1 << 2)
+#define CSI_SENS_CONF_PIX_CLK_POL		(1 << 3)
+#define CSI_SENS_PRTCL_GATED			(0 << 4)
+#define CSI_SENS_PRTCL_NON_GATED		(1 << 4)
+#define CSI_SENS_PRTCL_BT656_PROGRESSIVE	(2 << 4)
+#define CSI_SENS_PRTCL_BT656_INTERLACED		(3 << 4)
+#define CSI_SENS_PRTCL_BT1120_DDR_PROGRESSIVE	(4 << 4)
+#define CSI_SENS_PRTCL_BT1120_SDR_PROGRESSIVE	(5 << 4)
+#define CSI_SENS_PRTCL_BT1120_DDR_INTERLACED	(6 << 4)
+#define CSI_SENS_PRTCL_BT1120_SDR_INTERLACED	(7 << 4)
+#define CSI_SENS_CONF_PACK_TIGHT		(1 << 7)
+#define CSI_SENS_CONF_DATA_FMT_RGB_YUV444	(0 << 8)
+#define CSI_SENS_CONF_DATA_FMT_YUV422_YUYV	(1 << 8)
+#define CSI_SENS_CONF_DATA_FMT_YUV422_UYVY	(2 << 8)
+#define CSI_SENS_CONF_DATA_FMT_GENERIC		(3 << 8)
+#define CSI_SENS_CONF_DATA_FMT_BAYER		(3 << 8)
+#define CSI_SENS_CONF_DATA_FMT_RGB565		(4 << 8)
+#define CSI_SENS_CONF_DATA_FMT_RGB555		(5 << 8)
+#define CSI_SENS_CONF_DATA_FMT_RGB444		(6 << 8)
+#define CSI_SENS_CONF_DATA_FMT_JPEG		(7 << 8)
+#define CSI_SENS_CONF_DATA_WIDTH_8		(1 << 11)
+#define CSI_SENS_CONF_DATA_WIDTH_10		(3 << 11)
+#define CSI_SENS_CONF_DATA_WIDTH_12		(5 << 11)
+#define CSI_SENS_CONF_DATA_WIDTH_16		(9 << 11)
+#define CSI_SENS_CONF_EXT_VSYNC			(1 << 15)
+#define CSI_SENS_CONF_DATA_DEST_ISP		(1 << 24)
+#define CSI_SENS_CONF_DATA_DEST_IC		(1 << 25)
+#define CSI_SENS_CONF_DATA_DEST_IDMAC		(1 << 26)
+#define CSI_SENS_CONF_JPEG8_EN			(1 << 27)
+#define CSI_SENS_CONF_JPEG_EN			(1 << 28)
+#define CSI_SENS_CONF_FORCE_EOF			(1 << 29)
+#define CSI_SENS_CONF_DATA_EN_POL		(1 << 31)
+
+#define CSI_CCIRx_END_FLD_BLNK_1ST(x)		(((x) & 0x7) << 0)
+#define CSI_CCIRx_START_FLD_BLNK_1ST(x)		(((x) & 0x7) << 3)
+#define CSI_CCIRx_END_FLD_BLNK_2ND(x)		(((x) & 0x7) << 6)
+#define CSI_CCIRx_START_FLD_BLNK_2ND(x)		(((x) & 0x7) << 9)
+#define CSI_CCIRx_END_FLD_ACTV(x)		(((x) & 0x7) << 16)
+#define CSI_CCIRx_START_FLD_ACTV(x)		(((x) & 0x7) << 19)
+#define CSI_CCIRx_ERR_DET_EN			(1 << 24)
+
+#define CSI_SENS_FRM_WIDTH(w)			((((w) - 1) & 0x1fff) << 0)
+#define CSI_SENS_FRM_HEIGHT(h)			((((h) - 1) & 0x1fff) << 16)
+
+#define CSI_ACT_FRM_WIDTH(w)			((((w) - 1) & 0x1fff) << 0)
+#define CSI_ACT_FRM_HEIGHT(h)			((((h) - 1) & 0x1fff) << 16)
+
+#define CSI_OUT_FRM_CTRL_HORI_DOWNSIZE_EN	(1 << 31)
+#define CSI_OUT_FRM_CTRL_VERT_DOWNSIZE_EN	(1 << 30)
+#define CSI_OUT_FRM_CTRL_HSC(left)		(((left) & 0x1fff) << 16)
+#define CSI_OUT_FRM_CTRL_VSC(top)		(((top) & 0xfff) << 0)
+
+#define CSI_TST_CTRL_TEST_GEN_MODE_EN		(1 << 24)
+#define CSI_TST_CTRL_R(x)			(((x) & 0xff) << 0)
+#define CSI_TST_CTRL_G(x)			(((x) & 0xff) << 8)
+#define CSI_TST_CTRL_B(x)			(((x) & 0xff) << 16)
+
+#define CSI_COLOR_FIRST_ROW_MASK	0x00000002L
+#define CSI_COLOR_FIRST_COMP_MASK	0x00000001L
+
+struct ipucsi_format {
+	const char *name;
+	u32 mbus_code;
+	u32 fourcc;
+	u32 sens_conf;
+	u32 bytes_per_pixel; /* memory */
+	int bytes_per_sample; /* mbus */
+	unsigned rgb:1;
+	unsigned yuv:1;
+	unsigned raw:1;
+};
+
+int v4l2_media_subdev_s_stream(struct media_entity *entity, int enable);
+
+static struct ipucsi_format ipucsi_formats[] = {
+	{
+		.name = "Monochrome 8 bit",
+		.fourcc = V4L2_PIX_FMT_GREY,
+		.mbus_code = V4L2_MBUS_FMT_Y8_1X8,
+		.sens_conf = CSI_SENS_CONF_DATA_FMT_GENERIC | CSI_SENS_CONF_DATA_WIDTH_8,
+		.bytes_per_pixel = 1,
+		.bytes_per_sample = 1,
+		.raw = 1,
+	}, {
+		.name = "Monochrome 10 bit",
+		.fourcc = V4L2_PIX_FMT_Y10,
+		.mbus_code = V4L2_MBUS_FMT_Y10_1X10,
+		.sens_conf = CSI_SENS_CONF_DATA_FMT_GENERIC | CSI_SENS_CONF_DATA_WIDTH_10,
+		.bytes_per_pixel = 2,
+		.bytes_per_sample = 2,
+		.raw = 1,
+	}, {
+		.name = "Monochrome 12 bit",
+		.fourcc = V4L2_PIX_FMT_Y16,
+		.mbus_code = V4L2_MBUS_FMT_Y12_1X12,
+		.sens_conf = CSI_SENS_CONF_DATA_FMT_GENERIC | CSI_SENS_CONF_DATA_WIDTH_16,
+		.bytes_per_pixel = 2,
+		.bytes_per_sample = 2,
+		.raw = 1,
+	}, {
+		.name = "UYUV 2x8 bit",
+		.fourcc = V4L2_PIX_FMT_UYVY,
+		.mbus_code = V4L2_MBUS_FMT_UYVY8_2X8,
+		.sens_conf = CSI_SENS_CONF_DATA_FMT_YUV422_UYVY | CSI_SENS_CONF_DATA_WIDTH_8,
+		.bytes_per_pixel = 2,
+		.bytes_per_sample = 1,
+		.yuv = 1,
+	}, {
+		.name = "YUYV 2x8 bit",
+		.fourcc = V4L2_PIX_FMT_YUYV,
+		.mbus_code = V4L2_MBUS_FMT_YUYV8_2X8,
+		.sens_conf = CSI_SENS_CONF_DATA_FMT_YUV422_YUYV | CSI_SENS_CONF_DATA_WIDTH_8,
+		.bytes_per_pixel = 2,
+		.bytes_per_sample = 1,
+		.yuv = 1,
+	}, {
+		.name = "UYUV 1x16 bit",
+		.fourcc = V4L2_PIX_FMT_UYVY,
+		.mbus_code = V4L2_MBUS_FMT_UYVY8_1X16,
+		.sens_conf = CSI_SENS_CONF_DATA_FMT_GENERIC | CSI_SENS_CONF_DATA_WIDTH_16,
+		.bytes_per_pixel = 2,
+		.bytes_per_sample = 2,
+		.raw = 1,
+	}, {
+		.name = "YUYV 1x16 bit",
+		.fourcc = V4L2_PIX_FMT_YUYV,
+		.mbus_code = V4L2_MBUS_FMT_YUYV8_1X16,
+		.sens_conf = CSI_SENS_CONF_DATA_FMT_GENERIC | CSI_SENS_CONF_DATA_WIDTH_16,
+		.bytes_per_pixel = 2,
+		.bytes_per_sample = 2,
+		.raw = 1,
+	},
+};
+
+static struct ipucsi_format ipucsi_format_testpattern = {
+	.name = "RGB888 32bit",
+	.fourcc = V4L2_PIX_FMT_RGB32,
+	.mbus_code = V4L2_MBUS_FMT_FIXED,
+	.sens_conf = CSI_SENS_CONF_DATA_FMT_GENERIC | CSI_SENS_CONF_DATA_WIDTH_12,
+	.bytes_per_pixel = 4,
+	.bytes_per_sample = 4,
+	.rgb = 1,
+};
+
+/* buffer for one video frame */
+struct ipucsi_buffer {
+	struct vb2_buffer		vb;
+	struct list_head		queue;
+};
+
+struct ipucsi {
+	struct device			*dev;
+	struct v4l2_device		*v4l2_dev;
+	struct video_device		vdev;
+	struct media_pad		pad;
+	/* The currently active buffer, set by NFACK and cleared by EOF interrupt */
+	struct ipucsi_buffer		*active;
+	struct list_head		capture;
+	int				ilo;
+
+	struct vb2_queue		vb2_vidq;
+
+	int				id; /* CSI<id> - 0 or 1 */
+	void __iomem			*base;
+
+	spinlock_t			lock; /* locks CSI register access */
+	struct mutex			mutex;
+	struct vb2_alloc_ctx		*alloc_ctx;
+	enum v4l2_field			field;
+	int				sequence;
+	struct ipuv3_channel		*ipuch;
+	struct ipu_soc			*ipu;
+	struct v4l2_of_endpoint		endpoint;
+
+	struct v4l2_format		format;
+	struct ipucsi_format		ipucsifmt;
+	struct v4l2_ctrl_handler	ctrls;
+	struct v4l2_ctrl		*ctrl_test_pattern;
+	struct media_pad		media_pad;
+	struct media_pipeline		pipe;
+
+	struct v4l2_subdev		subdev;
+	struct media_pad		subdev_pad[2];
+	struct v4l2_mbus_framefmt	format_mbus[2];
+	struct ipu_media_link		*link;
+	struct v4l2_fh			fh;
+};
+
+static struct ipucsi_buffer *to_ipucsi_vb(struct vb2_buffer *vb)
+{
+	return container_of(vb, struct ipucsi_buffer, vb);
+}
+
+static u32 ipu_csi_read(struct ipucsi *csi, unsigned offset)
+{
+	return readl(csi->base + offset);
+}
+
+static void ipu_csi_write(struct ipucsi *csi, u32 value, unsigned offset)
+{
+	writel(value, csi->base + offset);
+}
+
+static u32 csi_test_ctrl_patterns[] = {
+	CSI_TST_CTRL_R(0xff) | CSI_TST_CTRL_G(0x0) | CSI_TST_CTRL_B(0x0),
+	CSI_TST_CTRL_R(0x0) | CSI_TST_CTRL_G(0xff) | CSI_TST_CTRL_B(0x0),
+	CSI_TST_CTRL_R(0x0) | CSI_TST_CTRL_G(0x0) | CSI_TST_CTRL_B(0xff),
+};
+
+static int ipu_csi_get_mbus_config(struct ipucsi *ipucsi,
+				   struct v4l2_mbus_config *config)
+{
+	struct v4l2_subdev *sd;
+	struct media_pad *pad;
+	int ret;
+
+	/*
+	 * Retrieve media bus configuration from the entity connected directly
+	 * to the CSI subdev sink pad.
+	 */
+	pad = media_entity_remote_pad(&ipucsi->subdev_pad[0]);
+	sd = media_entity_to_v4l2_subdev(pad->entity);
+	ret = v4l2_subdev_call(sd, video, g_mbus_config, config);
+	if (ret == -ENOIOCTLCMD) {
+		/* Fall back to static mbus configuration from device tree */
+		config->type = ipucsi->endpoint.bus_type;
+		config->flags = ipucsi->endpoint.bus.parallel.flags;
+		ret = 0;
+	}
+
+	return ret;
+}
+
+static int ipu_csi_init_interface(struct ipucsi *ipucsi,
+			   uint16_t width, uint16_t height)
+{
+	struct device *dev = ipucsi->dev;
+	u32 sens_conf, mbus_flags;
+	int interlaced = 0;
+	u32 ccir1, ccir2, ccir3, test_ctrl = 0;
+	struct v4l2_mbus_config mbus_config;
+	int ret;
+
+	sens_conf = ipucsi->ipucsifmt.sens_conf;
+	sens_conf |= CSI_SENS_CONF_DATA_DEST_IDMAC;
+
+	ret = ipu_csi_get_mbus_config(ipucsi, &mbus_config);
+	if (ret)
+		return ret;
+
+	switch (mbus_config.type) {
+	case V4L2_MBUS_PARALLEL:
+		mbus_flags = mbus_config.flags;
+
+		if (mbus_flags & V4L2_MBUS_PCLK_SAMPLE_FALLING)
+			sens_conf |= CSI_SENS_CONF_PIX_CLK_POL;
+		if (mbus_flags & V4L2_MBUS_HSYNC_ACTIVE_LOW)
+			sens_conf |= CSI_SENS_CONF_HSYNC_POL;
+		if (mbus_flags & V4L2_MBUS_VSYNC_ACTIVE_LOW)
+			sens_conf |= CSI_SENS_CONF_VSYNC_POL;
+		if (mbus_flags & V4L2_MBUS_DATA_ACTIVE_LOW)
+			sens_conf |= CSI_SENS_CONF_DATA_POL;
+		break;
+	case V4L2_MBUS_BT656:
+		switch (ipucsi->format_mbus[0].field) {
+		case V4L2_FIELD_SEQ_TB:
+		case V4L2_FIELD_SEQ_BT:
+			interlaced = true;
+			break;
+		default:
+			interlaced = false;
+			break;
+		}
+		if (interlaced) {
+			sens_conf |= CSI_SENS_PRTCL_BT656_INTERLACED;
+			if (width == 720 && height == 576) {
+				/* PAL case */
+				ccir1 = CSI_CCIRx_ERR_DET_EN |
+					CSI_CCIRx_END_FLD_BLNK_1ST(6) |
+					CSI_CCIRx_START_FLD_BLNK_1ST(2) |
+					CSI_CCIRx_END_FLD_BLNK_2ND(6) |
+					CSI_CCIRx_START_FLD_BLNK_2ND(2) |
+					CSI_CCIRx_END_FLD_ACTV(4) |
+					CSI_CCIRx_START_FLD_ACTV(0);
+
+				ccir2 = CSI_CCIRx_END_FLD_BLNK_1ST(7) |
+					CSI_CCIRx_START_FLD_BLNK_1ST(3) |
+					CSI_CCIRx_END_FLD_BLNK_2ND(7) |
+					CSI_CCIRx_START_FLD_BLNK_2ND(3) |
+					CSI_CCIRx_END_FLD_ACTV(5) |
+					CSI_CCIRx_START_FLD_ACTV(1);
+			} else if (width == 720 && height == 480) {
+				/* NTSC case */
+				ccir1 = CSI_CCIRx_ERR_DET_EN |
+					CSI_CCIRx_END_FLD_BLNK_1ST(7) |
+					CSI_CCIRx_START_FLD_BLNK_1ST(3) |
+					CSI_CCIRx_END_FLD_BLNK_2ND(7) |
+					CSI_CCIRx_START_FLD_BLNK_2ND(3) |
+					CSI_CCIRx_END_FLD_ACTV(5) |
+					CSI_CCIRx_START_FLD_ACTV(1);
+
+				ccir2 = CSI_CCIRx_END_FLD_BLNK_1ST(6) |
+					CSI_CCIRx_START_FLD_BLNK_1ST(2) |
+					CSI_CCIRx_END_FLD_BLNK_2ND(6) |
+					CSI_CCIRx_START_FLD_BLNK_2ND(2) |
+					CSI_CCIRx_END_FLD_ACTV(4) |
+					CSI_CCIRx_START_FLD_ACTV(0);
+
+				/* 0b111 - 0bHVF */
+			} else {
+				dev_err(dev,
+					"Unsupported CCIR656 interlaced video mode\n");
+				return -EINVAL;
+			}
+		} else {
+			sens_conf |= CSI_SENS_PRTCL_BT656_PROGRESSIVE;
+			ccir1 = CSI_CCIRx_START_FLD_BLNK_1ST(6) | CSI_CCIRx_END_FLD_ACTV(4);
+			ccir2 = 0;
+		}
+
+		ccir3 = 0xff0000;
+
+		ipu_csi_write(ipucsi, ccir1, CSI_CCIR_CODE_1);
+		ipu_csi_write(ipucsi, ccir2, CSI_CCIR_CODE_2);
+		ipu_csi_write(ipucsi, ccir3, CSI_CCIR_CODE_3);
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	if (ipucsi->ctrl_test_pattern->val) {
+		BUG_ON(ipucsi->ctrl_test_pattern->val > ARRAY_SIZE(csi_test_ctrl_patterns));
+		test_ctrl = csi_test_ctrl_patterns[ipucsi->ctrl_test_pattern->val - 1];
+		test_ctrl |= CSI_TST_CTRL_TEST_GEN_MODE_EN;
+		sens_conf = CSI_SENS_CONF_DATA_DEST_IDMAC |
+			CSI_SENS_CONF_EXT_VSYNC |
+			CSI_SENS_CONF_DATA_WIDTH_8 |
+			CSI_SENS_PRTCL_NON_GATED |
+			CSI_SENS_CONF_PIX_CLK_POL;
+	}
+
+	ipu_csi_write(ipucsi, test_ctrl, CSI_TST_CTRL);
+
+	ipu_csi_write(ipucsi, sens_conf, CSI_SENS_CONF);
+	ipu_csi_write(ipucsi, CSI_SENS_FRM_WIDTH(width) |
+			CSI_SENS_FRM_HEIGHT(height), CSI_SENS_FRM_SIZE);
+
+	dev_dbg(dev, "CSI_SENS_CONF = 0x%08X\n",
+		ipu_csi_read(ipucsi, CSI_SENS_CONF));
+	dev_dbg(dev, "CSI_SENS_FRM_SIZE = 0x%08X\n",
+		ipu_csi_read(ipucsi, CSI_SENS_FRM_SIZE));
+
+	return 0;
+}
+
+static inline void ipucsi_set_inactive_buffer(struct ipucsi *ipucsi,
+					      struct vb2_buffer *vb)
+{
+	int bufptr = !ipu_idmac_get_current_buffer(ipucsi->ipuch);
+	dma_addr_t eba = vb2_dma_contig_plane_dma_addr(vb, 0);
+
+	if (ipucsi->ilo < 0)
+		eba -= ipucsi->ilo;
+
+	ipu_cpmem_set_buffer(ipu_get_cpmem(ipucsi->ipuch), bufptr, eba);
+
+	ipu_idmac_select_buffer(ipucsi->ipuch, bufptr);
+}
+
+static irqreturn_t ipucsi_new_frame_handler(int irq, void *context)
+{
+	struct ipucsi *ipucsi = context;
+	struct ipucsi_buffer *buf;
+	struct vb2_buffer *vb;
+	unsigned long flags;
+
+	/* The IDMAC just started to write pixel data into the current buffer */
+
+	spin_lock_irqsave(&ipucsi->lock, flags);
+
+	/*
+	 * If there is a previously active frame, mark it as done to hand it off
+	 * to userspace. Or, if there are no further frames queued, hold on to it.
+	 */
+	if (ipucsi->active) {
+		vb = &ipucsi->active->vb;
+		buf = to_ipucsi_vb(vb);
+
+		if (vb2_is_streaming(vb->vb2_queue) && list_is_singular(&ipucsi->capture)) {
+			pr_debug("%s: reusing 0x%08x\n", __func__,
+				vb2_dma_contig_plane_dma_addr(vb, 0));
+			/* DEBUG: check if buf == EBA(active) */
+		} else {
+			/* Otherwise, mark buffer as finished */
+			list_del_init(&buf->queue);
+
+			vb2_buffer_done(vb, VB2_BUF_STATE_DONE);
+		}
+	}
+
+	if (list_empty(&ipucsi->capture))
+		goto out;
+
+	ipucsi->active = list_first_entry(&ipucsi->capture,
+					   struct ipucsi_buffer, queue);
+	vb = &ipucsi->active->vb;
+	do_gettimeofday(&vb->v4l2_buf.timestamp);
+	vb->v4l2_buf.field = ipucsi->format.fmt.pix.field;
+	vb->v4l2_buf.sequence = ipucsi->sequence++;
+
+	/*
+	 * Point the inactive buffer address to the next queued buffer,
+	 * if available. Otherwise, prepare to reuse the currently active
+	 * buffer, unless ipucsi_videobuf_queue gets called in time.
+	 */
+	if (!list_is_singular(&ipucsi->capture)) {
+		buf = list_entry(ipucsi->capture.next->next,
+				 struct ipucsi_buffer, queue);
+		vb = &buf->vb;
+	}
+	ipucsi_set_inactive_buffer(ipucsi, vb);
+out:
+	spin_unlock_irqrestore(&ipucsi->lock, flags);
+
+	return IRQ_HANDLED;
+}
+
+static struct ipucsi_format *ipucsi_current_format(struct ipucsi *ipucsi)
+{
+	if (ipucsi->ctrl_test_pattern->val)
+		return &ipucsi_format_testpattern;
+	else
+		return &ipucsi->ipucsifmt;
+}
+
+/*
+ *  Videobuf operations
+ */
+static int ipucsi_videobuf_setup(struct vb2_queue *vq, const struct v4l2_format *fmt,
+		unsigned int *count, unsigned int *num_planes,
+		unsigned int sizes[], void *alloc_ctxs[])
+{
+	struct ipucsi *ipucsi = vq->drv_priv;
+	int bytes_per_line;
+	struct ipucsi_format *ipucsifmt = ipucsi_current_format(ipucsi);
+
+	if (!fmt)
+		fmt = &ipucsi->format;
+
+	bytes_per_line = fmt->fmt.pix.width * ipucsifmt->bytes_per_pixel;
+
+	dev_dbg(ipucsi->dev, "bytes: %d x: %d y: %d",
+			bytes_per_line, fmt->fmt.pix.width, fmt->fmt.pix.height);
+
+	*num_planes = 1;
+
+	ipucsi->sequence = 0;
+	sizes[0] = bytes_per_line * fmt->fmt.pix.height;
+	alloc_ctxs[0] = ipucsi->alloc_ctx;
+
+	if (!*count)
+		*count = 32;
+
+	return 0;
+}
+
+static int ipucsi_videobuf_prepare(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct ipucsi *ipucsi = vq->drv_priv;
+	size_t new_size;
+	struct ipucsi_buffer *buf;
+	struct v4l2_pix_format *pix = &ipucsi->format.fmt.pix;
+	struct ipucsi_format *ipucsifmt = ipucsi_current_format(ipucsi);
+
+	buf = to_ipucsi_vb(vb);
+
+	new_size = pix->width * pix->height * ipucsifmt->bytes_per_pixel;
+
+	if (vb2_plane_size(vb, 0) < new_size)
+		return -ENOBUFS;
+
+	vb2_set_plane_payload(vb, 0, new_size);
+
+	return 0;
+}
+
+static void ipucsi_videobuf_queue(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct ipucsi *ipucsi = vq->drv_priv;
+	struct ipucsi_buffer *buf = to_ipucsi_vb(vb);
+	unsigned long flags;
+
+	spin_lock_irqsave(&ipucsi->lock, flags);
+
+	/*
+	 * If there is no next buffer queued, point the inactive buffer
+	 * address to the incoming buffer
+	 */
+	if (vb2_is_streaming(vb->vb2_queue) && list_is_singular(&ipucsi->capture))
+		ipucsi_set_inactive_buffer(ipucsi, vb);
+
+	list_add_tail(&buf->queue, &ipucsi->capture);
+
+	spin_unlock_irqrestore(&ipucsi->lock, flags);
+}
+
+static void ipucsi_videobuf_release(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct ipucsi *ipucsi = vq->drv_priv;
+	struct ipucsi_buffer *buf = to_ipucsi_vb(vb);
+	unsigned long flags;
+
+	spin_lock_irqsave(&ipucsi->lock, flags);
+
+	if (ipucsi->active == buf)
+		ipucsi->active = NULL;
+
+	if (!list_empty(&buf->queue))
+		list_del_init(&buf->queue);
+
+	spin_unlock_irqrestore(&ipucsi->lock, flags);
+}
+
+static int ipucsi_videobuf_init(struct vb2_buffer *vb)
+{
+	struct ipucsi_buffer *buf = to_ipucsi_vb(vb);
+
+	/* This is for locking debugging only */
+	INIT_LIST_HEAD(&buf->queue);
+
+	return 0;
+}
+
+#define pixfmtstr(x) (x) & 0xff, ((x) >> 8) & 0xff, ((x) >> 16) & 0xff, \
+	((x) >> 24) & 0xff
+
+static int ipucsi_videobuf_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+	struct ipucsi *ipucsi = vq->drv_priv;
+	struct ipucsi_format *ipucsifmt = ipucsi_current_format(ipucsi);
+	int xres = ipucsi->format.fmt.pix.width;
+	int yres = ipucsi->format.fmt.pix.height;
+	struct ipu_ch_param *cpmem = ipu_get_cpmem(ipucsi->ipuch);
+	struct device *dev = ipucsi->dev;
+	int capture_channel, burstsize;
+	struct vb2_buffer *vb;
+	struct ipucsi_buffer *buf;
+	int nfack_irq;
+	int ret;
+
+	memset(cpmem, 0, sizeof(*cpmem));
+
+	nfack_irq = ipu_idmac_channel_irq(ipucsi->ipu, ipucsi->ipuch,
+			IPU_IRQ_NFACK);
+	ret = request_threaded_irq(nfack_irq, NULL, ipucsi_new_frame_handler, IRQF_ONESHOT,
+			"ipucsi-nfack", ipucsi);
+	if (ret) {
+		dev_err(dev, "Failed to request NFACK interrupt: %d\n", nfack_irq);
+		return ret;
+	}
+
+	dev_dbg(dev, "width: %d height: %d, %c%c%c%c\n",
+			xres, yres, pixfmtstr(ipucsi->format.fmt.pix.pixelformat));
+
+	ipu_cpmem_set_resolution(cpmem, xres, yres);
+
+	if (ipucsifmt->raw) {
+		/*
+		 * raw formats. We can only pass them through to memory
+		 */
+		ipu_cpmem_set_stride(cpmem, xres * ipucsifmt->bytes_per_pixel);
+		ipu_cpmem_set_format_passthrough(cpmem, ipucsifmt->bytes_per_sample * 8);
+	} else {
+		/*
+		 * formats we understand, we can write it in any format not requiring
+		 * colorspace conversion.
+		 */
+		u32 fourcc = ipucsi->format.fmt.pix.pixelformat;
+		switch (fourcc) {
+		case V4L2_PIX_FMT_RGB32:
+			ipu_cpmem_set_stride(cpmem, xres * 4);
+			ipu_cpmem_set_fmt(cpmem, fourcc);
+			break;
+		case V4L2_PIX_FMT_UYVY:
+		case V4L2_PIX_FMT_YUYV:
+			ipu_cpmem_set_stride(cpmem, xres * 2);
+			ipu_cpmem_set_yuv_interleaved(cpmem, fourcc);
+			break;
+		case V4L2_PIX_FMT_YUV420:
+			ipu_cpmem_set_stride(cpmem, xres);
+			ipu_cpmem_set_yuv_planar(cpmem, V4L2_PIX_FMT_YUV420, xres, yres);
+			break;
+		default:
+			ret = -EINVAL;
+			goto free_irq;
+		}
+	}
+
+	if (ipucsi->ilo) {
+		ipu_ch_cpmem_set_interlaced_scan(ipucsi->ipuch);
+		if (ipucsi->ilo < 0) {
+			ipu_ch_param_write_field(cpmem, IPU_FIELD_ILO,
+						 0x100000 - (ipucsi->ilo/8));
+		}
+	}
+
+	capture_channel = ipucsi->id; /* CSI0: channel 0, CSI1: channel 1 */
+
+	/*
+	 * Some random value. The reference manual tells us that the burstsize
+	 * is a function of the IDMACs PFS, BPP and NPB settings. Unfortunately
+	 * it doesn't tell us which function this is.
+	 */
+	burstsize = 8;
+
+	ipu_smfc_set_burstsize(ipucsi->ipu, capture_channel, burstsize - 1);
+	ipu_smfc_map_channel(ipucsi->ipu, capture_channel, ipucsi->id, 0);
+
+	ipu_cpmem_set_high_priority(ipucsi->ipuch);
+
+	ipu_csi_write(ipucsi, CSI_ACT_FRM_HEIGHT(yres) | CSI_ACT_FRM_WIDTH(xres),
+			CSI_ACT_FRM_SIZE);
+	ipu_csi_write(ipucsi, CSI_OUT_FRM_CTRL_HSC(0) | CSI_OUT_FRM_CTRL_VSC(0),
+			CSI_OUT_FRM_CTRL);
+
+	ret = media_entity_pipeline_start(&ipucsi->subdev.entity, &ipucsi->pipe);
+	if (ret)
+		goto free_irq;
+
+	ret = ipu_csi_init_interface(ipucsi, xres, yres);
+	if (ret)
+		goto free_irq;
+
+	ipu_idmac_set_double_buffer(ipucsi->ipuch, 1);
+
+	if (list_empty(&ipucsi->capture)) {
+		dev_err(dev, "No capture buffers\n");
+		ret = -ENOMEM;
+		goto free_irq;
+	}
+
+	ipucsi->active = NULL;
+
+	/* Point the inactive buffer address to the first buffer */
+	buf = list_first_entry(&ipucsi->capture, struct ipucsi_buffer, queue);
+	vb = &buf->vb;
+	ipucsi_set_inactive_buffer(ipucsi, vb);
+
+	ipu_idmac_enable_channel(ipucsi->ipuch);
+	ipu_smfc_enable(ipucsi->ipu);
+	ipu_csi_enable(ipucsi->ipu, ipucsi->id);
+
+	ret = v4l2_media_subdev_s_stream(&ipucsi->subdev.entity, 1);
+	if (ret)
+		goto free_irq;
+
+	return 0;
+
+free_irq:
+	free_irq(nfack_irq, ipucsi);
+	return ret;
+}
+
+static int ipucsi_videobuf_stop_streaming(struct vb2_queue *vq)
+{
+	struct ipucsi *ipucsi = vq->drv_priv;
+	unsigned long flags;
+	int nfack_irq = ipu_idmac_channel_irq(ipucsi->ipu, ipucsi->ipuch,
+				IPU_IRQ_NFACK);
+
+	free_irq(nfack_irq, ipucsi);
+	ipu_csi_disable(ipucsi->ipu, ipucsi->id);
+	ipu_idmac_disable_channel(ipucsi->ipuch);
+	ipu_smfc_disable(ipucsi->ipu);
+
+	spin_lock_irqsave(&ipucsi->lock, flags);
+	while (!list_empty(&ipucsi->capture)) {
+		struct ipucsi_buffer *buf = list_entry(ipucsi->capture.next,
+						 struct ipucsi_buffer, queue);
+		list_del_init(ipucsi->capture.next);
+		vb2_buffer_done(&buf->vb, VB2_BUF_STATE_ERROR);
+	}
+	spin_unlock_irqrestore(&ipucsi->lock, flags);
+
+	media_entity_pipeline_stop(&ipucsi->subdev.entity);
+
+	v4l2_media_subdev_s_stream(&ipucsi->subdev.entity, 0);
+
+	return 0;
+}
+
+static void ipucsi_lock(struct vb2_queue *vq)
+{
+	struct ipucsi *ipucsi = vq->drv_priv;
+	mutex_lock(&ipucsi->mutex);
+}
+
+static void ipucsi_unlock(struct vb2_queue *vq)
+{
+	struct ipucsi *ipucsi = vq->drv_priv;
+	mutex_unlock(&ipucsi->mutex);
+}
+
+static struct vb2_ops ipucsi_videobuf_ops = {
+	.queue_setup		= ipucsi_videobuf_setup,
+	.buf_prepare		= ipucsi_videobuf_prepare,
+	.buf_queue		= ipucsi_videobuf_queue,
+	.buf_cleanup		= ipucsi_videobuf_release,
+	.buf_init		= ipucsi_videobuf_init,
+	.start_streaming	= ipucsi_videobuf_start_streaming,
+	.stop_streaming		= ipucsi_videobuf_stop_streaming,
+	.wait_prepare		= ipucsi_unlock,
+	.wait_finish		= ipucsi_lock,
+};
+
+static int ipucsi_querycap(struct file *file, void *priv,
+					struct v4l2_capability *cap)
+{
+	strlcpy(cap->driver, "imx-ipuv3-csi", sizeof(cap->driver));
+	/* cap->name is set by the friendly caller:-> */
+	strlcpy(cap->card, "imx-ipuv3-camera", sizeof(cap->card));
+	strlcpy(cap->bus_info, "platform:imx-ipuv3-csi", sizeof(cap->bus_info));
+	cap->version = 0;
+	cap->capabilities = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+
+	return 0;
+}
+
+static int ipucsi_try_fmt(struct file *file, void *fh,
+		struct v4l2_format *f)
+{
+	struct ipucsi *ipucsi = video_drvdata(file);
+	struct ipucsi_format *ipucsifmt = ipucsi_current_format(ipucsi);
+	enum v4l2_field in = ipucsi->format_mbus[1].field;
+	enum v4l2_field out = f->fmt.pix.field;
+	struct ipu_fmt *fmt = NULL;
+	int bytes_per_pixel;
+
+	if (ipucsifmt->rgb)
+		fmt = ipu_find_fmt_rgb(f->fmt.pix.pixelformat);
+	if (ipucsifmt->yuv)
+		fmt = ipu_find_fmt_yuv(f->fmt.pix.pixelformat);
+
+	if (ipucsifmt->raw) {
+		f->fmt.pix.pixelformat = ipucsifmt->fourcc;
+		bytes_per_pixel = ipucsifmt->bytes_per_pixel;
+	} else {
+		if (!fmt)
+			return -EINVAL;
+		bytes_per_pixel = fmt->bytes_per_pixel;
+	}
+
+	v4l_bound_align_image(&f->fmt.pix.width, 128,
+			      ipucsi->format_mbus[1].width, 3,
+			      &f->fmt.pix.height, 128,
+			      ipucsi->format_mbus[1].height, 1, 0);
+
+	f->fmt.pix.bytesperline = f->fmt.pix.width * bytes_per_pixel;
+	f->fmt.pix.sizeimage = f->fmt.pix.bytesperline * f->fmt.pix.height;
+
+	if ((in == V4L2_FIELD_SEQ_TB && out == V4L2_FIELD_INTERLACED_TB) ||
+	    (in == V4L2_FIELD_INTERLACED_TB && out == V4L2_FIELD_SEQ_TB) ||
+	    (in == V4L2_FIELD_SEQ_BT && out == V4L2_FIELD_INTERLACED_BT) ||
+	    (in == V4L2_FIELD_INTERLACED_BT && out == V4L2_FIELD_SEQ_BT)) {
+		/*
+		 * IDMAC scan order can be used for translation between
+		 * interlaced and sequential field formats.
+		 */
+	} else if (out == V4L2_FIELD_NONE || out == V4L2_FIELD_INTERLACED) {
+		/*
+		 * If userspace requests progressive or interlaced frames,
+		 * interlace sequential fields as closest approximation.
+		 */
+		if (in == V4L2_FIELD_SEQ_TB)
+			out = V4L2_FIELD_INTERLACED_TB;
+		else if (in == V4L2_FIELD_SEQ_BT)
+			out = V4L2_FIELD_INTERLACED_BT;
+		else
+			out = in;
+	} else {
+		/* Translation impossible or userspace doesn't care */
+		out = in;
+	}
+	f->fmt.pix.field = out;
+
+	return 0;
+}
+
+static int ipucsi_s_fmt(struct file *file, void *fh,
+		struct v4l2_format *f)
+{
+	struct ipucsi *ipucsi = video_drvdata(file);
+	enum v4l2_field in, out;
+	int ret;
+
+	ret = ipucsi_try_fmt(file, fh, f);
+	if (ret)
+		return ret;
+
+	ipucsi->format = *f;
+
+	/*
+	 * Set IDMAC scan order interlace offset (ILO) for translation between
+	 * interlaced and sequential field formats.
+	 */
+	in = ipucsi->format_mbus[1].field;
+	out = f->fmt.pix.field;
+	if ((in == V4L2_FIELD_SEQ_TB && out == V4L2_FIELD_INTERLACED_TB) ||
+	    (in == V4L2_FIELD_INTERLACED_TB && out == V4L2_FIELD_SEQ_TB))
+		ipucsi->ilo = f->fmt.pix.bytesperline;
+	else if ((in == V4L2_FIELD_SEQ_BT && out == V4L2_FIELD_INTERLACED_BT) ||
+		 (in == V4L2_FIELD_INTERLACED_BT && out == V4L2_FIELD_SEQ_BT))
+		ipucsi->ilo = -f->fmt.pix.bytesperline;
+	else
+		ipucsi->ilo = 0;
+
+	return 0;
+}
+
+static int ipucsi_g_fmt(struct file *file, void *fh,
+		struct v4l2_format *f)
+{
+	struct ipucsi *ipucsi = video_drvdata(file);
+
+	if (f->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		return -EINVAL;
+
+	*f = ipucsi->format;
+
+	return 0;
+}
+
+static int ipucsi_enum_fmt(struct file *file, void *priv,
+				    struct v4l2_fmtdesc *f)
+{
+	struct ipucsi *ipucsi = video_drvdata(file);
+
+	if (ipucsi->ctrl_test_pattern->val) {
+		if (f->index)
+			return -EINVAL;
+		strlcpy(f->description, ipucsi_format_testpattern.name,
+				sizeof(f->description));
+		f->pixelformat = ipucsi_format_testpattern.fourcc;
+
+		return 0;
+	}
+
+	if (ipucsi->ipucsifmt.rgb)
+		return ipu_enum_fmt_rgb(file, priv, f);
+	if (ipucsi->ipucsifmt.yuv)
+		return ipu_enum_fmt_yuv(file, priv, f);
+
+	if (f->index)
+		return -EINVAL;
+
+	if (!ipucsi->ipucsifmt.name)
+		return -EINVAL;
+
+	strlcpy(f->description, ipucsi->ipucsifmt.name, sizeof(f->description));
+	f->pixelformat = ipucsi->ipucsifmt.fourcc;
+
+	return 0;
+}
+
+static struct v4l2_mbus_framefmt *
+__ipucsi_get_pad_format(struct ipucsi *ipucsi, struct v4l2_subdev_fh *fh,
+			unsigned int pad, u32 which)
+{
+	switch (which) {
+	case V4L2_SUBDEV_FORMAT_TRY:
+		return v4l2_subdev_get_try_format(fh, pad);
+	case V4L2_SUBDEV_FORMAT_ACTIVE:
+		return &ipucsi->format_mbus[pad];
+	default:
+		return NULL;
+	}
+}
+
+static int ipucsi_subdev_get_format(struct v4l2_subdev *subdev,
+		struct v4l2_subdev_fh *fh,
+		struct v4l2_subdev_format *sdformat)
+{
+	struct ipucsi *ipucsi = container_of(subdev, struct ipucsi, subdev);
+
+	sdformat->format = *__ipucsi_get_pad_format(ipucsi, fh, sdformat->pad,
+						sdformat->which);
+	return 0;
+}
+
+static struct ipucsi_format *ipucsi_find_subdev_format(const u32 *fourcc,
+		const u32 *mbus_code)
+{
+	struct ipucsi_format *ipucsifmt;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(ipucsi_formats); i++) {
+		ipucsifmt = &ipucsi_formats[i];
+		if (fourcc && *fourcc == ipucsifmt->fourcc)
+			return ipucsifmt;
+		if (mbus_code && *mbus_code == ipucsifmt->mbus_code)
+			return ipucsifmt;
+	}
+
+	return NULL;
+}
+
+static int ipucsi_subdev_set_format(struct v4l2_subdev *subdev,
+		struct v4l2_subdev_fh *fh,
+		struct v4l2_subdev_format *sdformat)
+{
+	struct ipucsi *ipucsi = container_of(subdev, struct ipucsi, subdev);
+	struct v4l2_mbus_framefmt *mbusformat;
+	struct ipucsi_format *ipucsiformat;
+	unsigned int width, height;
+
+	ipucsiformat = ipucsi_find_subdev_format(NULL, &sdformat->format.code);
+	if (!ipucsiformat)
+		return -EINVAL;
+
+	width = clamp_t(unsigned int, sdformat->format.width, 16, 8192);
+	height = clamp_t(unsigned int, sdformat->format.height, 16, 4096);
+
+	mbusformat = __ipucsi_get_pad_format(ipucsi, fh, sdformat->pad,
+					    sdformat->which);
+	mbusformat->width = width;
+	mbusformat->height = height;
+	mbusformat->code = ipucsiformat->mbus_code;
+	mbusformat->field = sdformat->format.field;
+
+	if (mbusformat->field == V4L2_FIELD_SEQ_TB &&
+	    mbusformat->width == 720 && mbusformat->height == 480 &&
+	    ipucsi->endpoint.bus_type == V4L2_MBUS_BT656) {
+		/* We capture NTSC bottom field first */
+		mbusformat->field = V4L2_FIELD_SEQ_BT;
+	} else if (mbusformat->field == V4L2_FIELD_ANY) {
+		mbusformat->field = ipucsi->format_mbus[!sdformat->pad].field;
+	}
+
+	sdformat->format = *mbusformat;
+
+	ipucsi->ipucsifmt = *ipucsiformat;
+
+	return 0;
+}
+
+static struct v4l2_subdev_pad_ops ipucsi_subdev_pad_ops = {
+	.get_fmt = ipucsi_subdev_get_format,
+	.set_fmt = ipucsi_subdev_set_format,
+};
+
+static const struct v4l2_subdev_ops ipucsi_subdev_ops = {
+	.pad    = &ipucsi_subdev_pad_ops,
+};
+
+struct media_entity_operations ipucsi_entity_ops = {
+	.link_validate = v4l2_subdev_link_validate,
+};
+
+int v4l2_media_subdev_s_stream(struct media_entity *entity, int enable)
+{
+	struct media_entity_graph graph;
+	struct media_entity *first;
+	struct v4l2_subdev *sd;
+	int ret = 0;
+
+	first = entity;
+
+	media_entity_graph_walk_start(&graph, entity);
+	if (!enable) {
+		first = NULL;
+		goto disable;
+	}
+
+	while (!ret && (entity = media_entity_graph_walk_next(&graph))) {
+		if (media_entity_type(entity) == MEDIA_ENT_T_V4L2_SUBDEV) {
+			sd = media_entity_to_v4l2_subdev(entity);
+			ret = v4l2_subdev_call(sd, video, s_stream, 1);
+			if (ret == -ENOIOCTLCMD)
+				ret = 0;
+		}
+	}
+
+	if (!ret)
+		return 0;
+
+	media_entity_graph_walk_start(&graph, first);
+
+disable:
+	while ((entity = media_entity_graph_walk_next(&graph)) && first != entity) {
+		if (media_entity_type(entity) == MEDIA_ENT_T_V4L2_SUBDEV) {
+			sd = media_entity_to_v4l2_subdev(entity);
+			v4l2_subdev_call(sd, video, s_stream, 0);
+		}
+	}
+
+	return ret;
+}
+
+int v4l2_media_subdev_s_power(struct media_entity *entity, int enable)
+{
+	struct media_entity_graph graph;
+	struct media_entity *first;
+	struct v4l2_subdev *sd;
+	int ret = 0;
+
+	first = entity;
+
+	media_entity_graph_walk_start(&graph, entity);
+	if (!enable) {
+		first = NULL;
+		goto disable;
+	}
+
+	while (!ret && (entity = media_entity_graph_walk_next(&graph))) {
+		if (media_entity_type(entity) == MEDIA_ENT_T_V4L2_SUBDEV) {
+			sd = media_entity_to_v4l2_subdev(entity);
+			ret = v4l2_subdev_call(sd, core, s_power, 1);
+			if (ret == -ENOIOCTLCMD)
+				ret = 0;
+		}
+	}
+
+	if (!ret)
+		return 0;
+
+	media_entity_graph_walk_start(&graph, first);
+
+disable:
+	while ((entity = media_entity_graph_walk_next(&graph)) && first != entity) {
+		if (media_entity_type(entity) == MEDIA_ENT_T_V4L2_SUBDEV) {
+			sd = media_entity_to_v4l2_subdev(entity);
+			v4l2_subdev_call(sd, core, s_power, 0);
+		}
+	}
+
+	return ret;
+}
+
+static int ipucsi_open(struct file *file)
+{
+	struct ipucsi *ipucsi = video_drvdata(file);
+	int ret;
+
+	mutex_lock(&ipucsi->mutex);
+	ret = v4l2_fh_open(file);
+	if (ret)
+		goto out;
+
+	if (v4l2_fh_is_singular_file(file))
+		ret = v4l2_media_subdev_s_power(&ipucsi->subdev.entity, 1);
+
+out:
+	mutex_unlock(&ipucsi->mutex);
+	return ret;
+}
+
+static int ipucsi_release(struct file *file)
+{
+	struct ipucsi *ipucsi = video_drvdata(file);
+
+	mutex_lock(&ipucsi->mutex);
+	if (v4l2_fh_is_singular_file(file)) {
+		v4l2_media_subdev_s_power(&ipucsi->subdev.entity, 0);
+
+		vb2_fop_release(file);
+	} else {
+		v4l2_fh_release(file);
+	}
+
+	mutex_unlock(&ipucsi->mutex);
+	return 0;
+}
+
+static u64 camera_mask = DMA_BIT_MASK(32);
+
+static const struct v4l2_file_operations ipucsi_capture_fops = {
+	.owner		= THIS_MODULE,
+	.open		= ipucsi_open,
+	.release	= ipucsi_release,
+	.unlocked_ioctl	= video_ioctl2,
+	.mmap		= vb2_fop_mmap,
+	.poll		= vb2_fop_poll,
+};
+
+static int ipucsi_enum_framesizes(struct file *file, void *fh,
+				  struct v4l2_frmsizeenum *fsize)
+{
+	struct ipucsi *ipucsi = video_drvdata(file);
+	struct ipucsi_format *ipucsifmt = ipucsi_current_format(ipucsi);
+	struct v4l2_mbus_config mbus_config;
+	struct ipu_fmt *fmt = NULL;
+	int ret;
+
+	ret = ipu_csi_get_mbus_config(ipucsi, &mbus_config);
+	if (ret)
+		return ret;
+
+	if (((fsize->index != 0) &&
+	     (mbus_config.type != V4L2_MBUS_BT656)) ||
+	    (fsize->index > 1))
+		return -EINVAL;
+
+	if (ipucsifmt->rgb)
+		fmt = ipu_find_fmt_rgb(fsize->pixel_format);
+	if (ipucsifmt->yuv)
+		fmt = ipu_find_fmt_yuv(fsize->pixel_format);
+	if (!fmt && !ipucsifmt->raw)
+		return -EINVAL;
+
+	if (mbus_config.type == V4L2_MBUS_BT656) {
+		fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+		fsize->discrete.width = 720;
+		fsize->discrete.height = fsize->index ? 576 : 480;
+	} else {
+		fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+		fsize->stepwise.min_width = 1;
+		fsize->stepwise.min_height = 1;
+		fsize->stepwise.max_width = ipucsi->format_mbus[1].width;
+		fsize->stepwise.max_height = ipucsi->format_mbus[1].height;
+		fsize->stepwise.step_width = 1;
+		fsize->stepwise.step_height = 1;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops ipucsi_capture_ioctl_ops = {
+	.vidioc_querycap		= ipucsi_querycap,
+
+	.vidioc_enum_fmt_vid_cap	= ipucsi_enum_fmt,
+	.vidioc_try_fmt_vid_cap		= ipucsi_try_fmt,
+	.vidioc_s_fmt_vid_cap		= ipucsi_s_fmt,
+	.vidioc_g_fmt_vid_cap		= ipucsi_g_fmt,
+
+	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
+	.vidioc_querybuf		= vb2_ioctl_querybuf,
+
+	.vidioc_qbuf			= vb2_ioctl_qbuf,
+	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
+	.vidioc_expbuf			= vb2_ioctl_expbuf,
+
+	.vidioc_streamon		= vb2_ioctl_streamon,
+	.vidioc_streamoff		= vb2_ioctl_streamoff,
+
+	.vidioc_enum_framesizes		= ipucsi_enum_framesizes,
+};
+
+static int ipucsi_subdev_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	switch (ctrl->id) {
+	case V4L2_CID_TEST_PATTERN:
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops ipucsi_subdev_ctrl_ops = {
+	.s_ctrl = ipucsi_subdev_s_ctrl,
+};
+
+static const char * const ipucsi_test_pattern_menu[] = {
+	"Disabled",
+	"Checkerboard-red",
+	"Checkerboard-green",
+	"Checkerboard-blue",
+};
+
+static int ipucsi_create_controls(struct ipucsi *ipucsi)
+{
+	struct v4l2_ctrl_handler *handler = &ipucsi->ctrls;
+
+	v4l2_ctrl_handler_init(handler, 1);
+
+	ipucsi->ctrl_test_pattern = v4l2_ctrl_new_std_menu_items(handler,
+			&ipucsi_subdev_ctrl_ops, V4L2_CID_TEST_PATTERN,
+			ARRAY_SIZE(ipucsi_test_pattern_menu) - 1, 0, 0,
+			ipucsi_test_pattern_menu);
+
+	return ipucsi->ctrl_test_pattern ? 0 : -ENOMEM;
+}
+
+static int ipucsi_subdev_init(struct ipucsi *ipucsi, struct device_node *node)
+{
+	struct device_node *endpoint;
+	int ret;
+
+	v4l2_subdev_init(&ipucsi->subdev, &ipucsi_subdev_ops);
+
+	ipucsi->subdev.ctrl_handler = &ipucsi->ctrls;
+
+	snprintf(ipucsi->subdev.name, sizeof(ipucsi->subdev.name), "%s",
+			node->full_name);
+
+	endpoint = of_get_next_child(node, NULL);
+	if (endpoint)
+		v4l2_of_parse_endpoint(endpoint, &ipucsi->endpoint);
+
+	ipucsi->subdev.entity.ops = &ipucsi_entity_ops;
+
+	ipucsi->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+	ipucsi->subdev_pad[0].flags = MEDIA_PAD_FL_SINK;
+	ipucsi->subdev_pad[1].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_init(&ipucsi->subdev.entity, 2, ipucsi->subdev_pad, 0);
+	if (ret < 0)
+		return ret;
+
+	ret = v4l2_device_register_subdev(ipucsi->v4l2_dev, &ipucsi->subdev);
+	if (ret < 0)
+		return ret;
+
+	ret = v4l2_device_register_subdev_node(ipucsi->v4l2_dev,
+					       &ipucsi->subdev);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
+static int ipucsi_video_device_init(struct platform_device *pdev,
+		struct ipucsi *ipucsi)
+{
+	struct video_device *vdev = &ipucsi->vdev;
+	int ret;
+
+	snprintf(vdev->name, sizeof(vdev->name), DRIVER_NAME ".%d", pdev->id);
+	vdev->release	= video_device_release_empty;
+	vdev->fops	= &ipucsi_capture_fops;
+	vdev->ioctl_ops	= &ipucsi_capture_ioctl_ops;
+	vdev->v4l2_dev	= ipucsi->v4l2_dev;
+	vdev->minor	= -1;
+	vdev->release	= video_device_release_empty;
+	vdev->lock	= &ipucsi->mutex;
+	vdev->queue	= &ipucsi->vb2_vidq;
+
+	video_set_drvdata(vdev, ipucsi);
+
+	ipucsi->media_pad.flags = MEDIA_PAD_FL_SINK;
+	ret = media_entity_init(&vdev->entity, 1, &ipucsi->media_pad, 0);
+	if (ret < 0)
+		video_device_release(vdev);
+
+	return ret;
+}
+
+static int ipucsi_vb2_init(struct ipucsi *ipucsi)
+{
+	struct vb2_queue *q;
+	int ret;
+
+	ipucsi->alloc_ctx = vb2_dma_contig_init_ctx(ipucsi->dev);
+	if (IS_ERR(ipucsi->alloc_ctx))
+		return PTR_ERR(ipucsi->alloc_ctx);
+
+	q = &ipucsi->vb2_vidq;
+	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	q->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+	q->drv_priv = ipucsi;
+	q->ops = &ipucsi_videobuf_ops;
+	q->mem_ops = &vb2_dma_contig_memops;
+	q->buf_struct_size = sizeof(struct ipucsi_buffer);
+	q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+
+	ret = vb2_queue_init(q);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int ipucsi_async_init(struct ipucsi *ipucsi, struct device_node *node)
+{
+	struct device_node *rp;
+
+	rp = of_get_next_child(node, NULL);
+	if (!rp)
+		return 0;
+
+	ipucsi->link = ipu_media_entity_create_link(&ipucsi->subdev, 0, rp,
+			MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
+
+	if (IS_ERR(ipucsi->link)) {
+		ipucsi->link = NULL;
+		return PTR_ERR(ipucsi->link);
+	}
+	return 0;
+}
+
+static struct device_node *ipucsi_get_port(struct device_node *node, int id)
+{
+	struct device_node *port;
+	int reg;
+
+	for_each_child_of_node(node, port) {
+		if (!of_property_read_u32(port, "reg", &reg) && reg == id)
+			return of_node_get(port);
+	}
+
+	return NULL;
+}
+
+static int ipucsi_probe(struct platform_device *pdev)
+{
+	struct ipu_client_platformdata *pdata = pdev->dev.platform_data;
+	struct ipu_soc *ipu = dev_get_drvdata(pdev->dev.parent);
+	struct ipucsi *ipucsi;
+	struct resource *res;
+	int ret;
+	struct device_node *node;
+
+	pdev->dev.dma_mask = &camera_mask,
+	pdev->dev.coherent_dma_mask = DMA_BIT_MASK(32),
+
+	res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+	if (!res)
+		return -ENODEV;
+
+	if (!pdata)
+		return -EINVAL;
+
+	ipucsi = devm_kzalloc(&pdev->dev, sizeof(*ipucsi), GFP_KERNEL);
+	if (!ipucsi)
+		return -ENOMEM;
+
+	INIT_LIST_HEAD(&ipucsi->capture);
+	spin_lock_init(&ipucsi->lock);
+	mutex_init(&ipucsi->mutex);
+
+	ipucsi->base = devm_ioremap(&pdev->dev, res->start, resource_size(res));
+	if (!ipucsi->base) {
+		dev_err(&pdev->dev, "Couldn't map %pR\n", res);
+		return -ENOMEM;
+	}
+
+	/* pdev->id = 0: IPU1 CSI0, 1: IPU1 CSI1, 2: IPU2 CSI0, 3: IPU2 CSI1 */
+	ipucsi->id = pdata->csi; /* CSI0 or CSI1 */
+	ipucsi->ipu = ipu;
+	ipucsi->dev = &pdev->dev;
+	ipucsi->v4l2_dev = ipu_media_get_v4l2_dev();
+	if (!ipucsi->v4l2_dev)
+		return -EPROBE_DEFER;
+
+	node = ipucsi_get_port(pdev->dev.parent->of_node, pdata->csi);
+	if (!node) {
+		dev_err(&pdev->dev, "cannot find node port@%d\n", pdata->csi);
+		return -ENODEV;
+	}
+
+	ipucsi->ipuch = ipu_idmac_get(ipu, pdata->dma[0]);
+	if (!ipucsi->ipuch)
+		return -EBUSY;
+
+	ret = ipucsi_video_device_init(pdev, ipucsi);
+	if (ret)
+		goto failed;
+
+	ret = ipucsi_create_controls(ipucsi);
+	if (ret)
+		goto failed;
+
+	ret = ipucsi_vb2_init(ipucsi);
+	if (ret)
+		goto failed;
+
+	ret = ipucsi_subdev_init(ipucsi, node);
+	if (ret)
+		goto failed;
+
+	ret = ipucsi_async_init(ipucsi, node);
+	if (ret)
+		goto failed;
+
+	of_node_put(node);
+
+	platform_set_drvdata(pdev, ipucsi);
+
+	ret = video_register_device(&ipucsi->vdev, VFL_TYPE_GRABBER, -1);
+	if (ret)
+		goto failed;
+
+	ret = media_entity_create_link(&ipucsi->subdev.entity, 1,
+			&ipucsi->vdev.entity, 0,
+			MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
+	if (ret < 0) {
+		video_unregister_device(&ipucsi->vdev);
+		goto failed;
+	}
+
+	dev_info(&pdev->dev, "loaded\n");
+
+	return 0;
+
+failed:
+	v4l2_ctrl_handler_free(&ipucsi->ctrls);
+	if (ipucsi->link)
+		ipu_media_entity_remove_link(ipucsi->link);
+	if (ipucsi->vdev.entity.links)
+		media_entity_cleanup(&ipucsi->vdev.entity);
+	if (ipucsi->alloc_ctx)
+		vb2_dma_contig_cleanup_ctx(ipucsi->alloc_ctx);
+	if (ipucsi->ipuch)
+		ipu_idmac_put(ipucsi->ipuch);
+
+	return ret;
+}
+
+static int ipucsi_remove(struct platform_device *pdev)
+{
+	struct ipucsi *ipucsi = platform_get_drvdata(pdev);
+
+	video_unregister_device(&ipucsi->vdev);
+	ipu_media_entity_remove_link(ipucsi->link);
+	media_entity_cleanup(&ipucsi->vdev.entity);
+	vb2_dma_contig_cleanup_ctx(ipucsi->alloc_ctx);
+	ipu_idmac_put(ipucsi->ipuch);
+	v4l2_ctrl_handler_free(&ipucsi->ctrls);
+
+	return 0;
+}
+
+static struct platform_driver ipucsi_driver = {
+	.driver = {
+		.name = DRIVER_NAME,
+	},
+	.probe = ipucsi_probe,
+	.remove = ipucsi_remove,
+};
+
+static int __init ipucsi_init(void)
+{
+	return platform_driver_register(&ipucsi_driver);
+}
+
+static void __exit ipucsi_exit(void)
+{
+	return platform_driver_unregister(&ipucsi_driver);
+}
+
+subsys_initcall(ipucsi_init);
+module_exit(ipucsi_exit);
+
+MODULE_DESCRIPTION("i.MX51/53 IPUv3 Camera interface driver");
+MODULE_AUTHOR("Sascha Hauer <s.hauer@pengutronix.de>");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:" DRIVER_NAME);
-- 
2.0.0.rc2

[RFC PATCH 16/26] [media] ipuv3-csi: Skip 3 lines for NTSC BT.656

[Philipp Zabel]

TVP5150 creates 480 visible lines, but synchronisation
signals for 486 visible lines.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/imx/imx-ipuv3-csi.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/imx/imx-ipuv3-csi.c b/drivers/media/platform/imx/imx-ipuv3-csi.c
index d9326a8..dfa2daa 100644
--- a/drivers/media/platform/imx/imx-ipuv3-csi.c
+++ b/drivers/media/platform/imx/imx-ipuv3-csi.c
@@ -698,8 +698,14 @@ static int ipucsi_videobuf_start_streaming(struct vb2_queue *vq, unsigned int co
 
 	ipu_csi_write(ipucsi, CSI_ACT_FRM_HEIGHT(yres) | CSI_ACT_FRM_WIDTH(xres),
 			CSI_ACT_FRM_SIZE);
-	ipu_csi_write(ipucsi, CSI_OUT_FRM_CTRL_HSC(0) | CSI_OUT_FRM_CTRL_VSC(0),
-			CSI_OUT_FRM_CTRL);
+	/* FIXME */
+	if (xres == 720 && yres == 480) {
+		ipu_csi_write(ipucsi, CSI_OUT_FRM_CTRL_HSC(0) |
+				CSI_OUT_FRM_CTRL_VSC(3), CSI_OUT_FRM_CTRL);
+	} else {
+		ipu_csi_write(ipucsi, CSI_OUT_FRM_CTRL_HSC(0) |
+				CSI_OUT_FRM_CTRL_VSC(0), CSI_OUT_FRM_CTRL);
+	}
 
 	ret = media_entity_pipeline_start(&ipucsi->subdev.entity, &ipucsi->pipe);
 	if (ret)
-- 
2.0.0.rc2

[RFC PATCH 17/26] [media] ipuv3-csi: Pass ipucsi to v4l2_media_subdev_s_power

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

Makes it easier to access ipucsi from v4l2_media_subdev_s_power which
is needed in subsequent patches.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 drivers/media/platform/imx/imx-ipuv3-csi.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/imx/imx-ipuv3-csi.c b/drivers/media/platform/imx/imx-ipuv3-csi.c
index dfa2daa..e75d7f5 100644
--- a/drivers/media/platform/imx/imx-ipuv3-csi.c
+++ b/drivers/media/platform/imx/imx-ipuv3-csi.c
@@ -1080,8 +1080,9 @@ disable:
 	return ret;
 }
 
-int v4l2_media_subdev_s_power(struct media_entity *entity, int enable)
+int v4l2_media_subdev_s_power(struct ipucsi *ipucsi, int enable)
 {
+	struct media_entity *entity = &ipucsi->subdev.entity;
 	struct media_entity_graph graph;
 	struct media_entity *first;
 	struct v4l2_subdev *sd;
@@ -1131,7 +1132,7 @@ static int ipucsi_open(struct file *file)
 		goto out;
 
 	if (v4l2_fh_is_singular_file(file))
-		ret = v4l2_media_subdev_s_power(&ipucsi->subdev.entity, 1);
+		ret = v4l2_media_subdev_s_power(ipucsi, 1);
 
 out:
 	mutex_unlock(&ipucsi->mutex);
@@ -1144,7 +1145,7 @@ static int ipucsi_release(struct file *file)
 
 	mutex_lock(&ipucsi->mutex);
 	if (v4l2_fh_is_singular_file(file)) {
-		v4l2_media_subdev_s_power(&ipucsi->subdev.entity, 0);
+		v4l2_media_subdev_s_power(ipucsi, 0);
 
 		vb2_fop_release(file);
 	} else {
-- 
2.0.0.rc2

[RFC PATCH 18/26] [media] ipuv3-csi: make subdev controls available on video device

[Philipp Zabel]

From: Sascha Hauer <s.hauer@pengutronix.de>

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 drivers/media/platform/imx/imx-ipuv3-csi.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/drivers/media/platform/imx/imx-ipuv3-csi.c b/drivers/media/platform/imx/imx-ipuv3-csi.c
index e75d7f5..0dd40a4 100644
--- a/drivers/media/platform/imx/imx-ipuv3-csi.c
+++ b/drivers/media/platform/imx/imx-ipuv3-csi.c
@@ -250,6 +250,7 @@ struct ipucsi {
 	struct v4l2_format		format;
 	struct ipucsi_format		ipucsifmt;
 	struct v4l2_ctrl_handler	ctrls;
+	struct v4l2_ctrl_handler	ctrls_vdev;
 	struct v4l2_ctrl		*ctrl_test_pattern;
 	struct media_pad		media_pad;
 	struct media_pipeline		pipe;
@@ -1096,12 +1097,19 @@ int v4l2_media_subdev_s_power(struct ipucsi *ipucsi, int enable)
 		goto disable;
 	}
 
+	v4l2_ctrl_handler_init(&ipucsi->ctrls_vdev, 1);
+
 	while (!ret && (entity = media_entity_graph_walk_next(&graph))) {
 		if (media_entity_type(entity) == MEDIA_ENT_T_V4L2_SUBDEV) {
 			sd = media_entity_to_v4l2_subdev(entity);
 			ret = v4l2_subdev_call(sd, core, s_power, 1);
 			if (ret == -ENOIOCTLCMD)
 				ret = 0;
+
+			ret = v4l2_ctrl_add_handler(&ipucsi->ctrls_vdev,
+						    sd->ctrl_handler, NULL);
+			if (ret)
+				return ret;
 		}
 	}
 
@@ -1147,6 +1155,8 @@ static int ipucsi_release(struct file *file)
 	if (v4l2_fh_is_singular_file(file)) {
 		v4l2_media_subdev_s_power(ipucsi, 0);
 
+		v4l2_ctrl_handler_free(&ipucsi->ctrls_vdev);
+
 		vb2_fop_release(file);
 	} else {
 		v4l2_fh_release(file);
@@ -1320,6 +1330,7 @@ static int ipucsi_video_device_init(struct platform_device *pdev,
 	vdev->minor	= -1;
 	vdev->release	= video_device_release_empty;
 	vdev->lock	= &ipucsi->mutex;
+	vdev->ctrl_handler = &ipucsi->ctrls_vdev;
 	vdev->queue	= &ipucsi->vb2_vidq;
 
 	video_set_drvdata(vdev, ipucsi);
-- 
2.0.0.rc2

[RFC PATCH 19/26] [media] imx-ipuv3-csi: Add support for temporarily stopping the stream on sync loss

[Philipp Zabel]

This patch allows to receive sync lock notifications from
the source subdevice and temporarily pauses the stream until
the source can lock onto its signal again.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/imx/imx-ipuv3-csi.c | 160 +++++++++++++++++++++++++++++
 1 file changed, 160 insertions(+)

diff --git a/drivers/media/platform/imx/imx-ipuv3-csi.c b/drivers/media/platform/imx/imx-ipuv3-csi.c
index 0dd40a4..ab22cad 100644
--- a/drivers/media/platform/imx/imx-ipuv3-csi.c
+++ b/drivers/media/platform/imx/imx-ipuv3-csi.c
@@ -17,6 +17,7 @@
 #include <linux/moduleparam.h>
 #include <linux/interrupt.h>
 #include <linux/videodev2.h>
+#include <linux/workqueue.h>
 #include <linux/version.h>
 #include <linux/device.h>
 #include <linux/kernel.h>
@@ -260,6 +261,7 @@ struct ipucsi {
 	struct v4l2_mbus_framefmt	format_mbus[2];
 	struct ipu_media_link		*link;
 	struct v4l2_fh			fh;
+	bool				paused;
 };
 
 static struct ipucsi_buffer *to_ipucsi_vb(struct vb2_buffer *vb)
@@ -443,6 +445,159 @@ static inline void ipucsi_set_inactive_buffer(struct ipucsi *ipucsi,
 	ipu_idmac_select_buffer(ipucsi->ipuch, bufptr);
 }
 
+int ipucsi_resume_stream(struct ipucsi *ipucsi)
+{
+	struct ipucsi_buffer *buf;
+	struct vb2_buffer *vb;
+	unsigned long flags;
+	dma_addr_t eba;
+
+	if (!ipucsi->paused)
+		return 0;
+
+	spin_lock_irqsave(&ipucsi->lock, flags);
+
+	if (list_empty(&ipucsi->capture)) {
+		spin_unlock_irqrestore(&ipucsi->lock, flags);
+		return -EAGAIN;
+	}
+
+	buf = list_first_entry(&ipucsi->capture, struct ipucsi_buffer, queue);
+	vb = &buf->vb;
+
+	ipu_idmac_set_double_buffer(ipucsi->ipuch, 1);
+
+	eba = vb2_dma_contig_plane_dma_addr(vb, 0);
+	if (ipucsi->ilo < 0)
+		eba -= ipucsi->ilo;
+
+	ipu_cpmem_set_buffer(ipu_get_cpmem(ipucsi->ipuch), 0, eba);
+
+	ipu_idmac_select_buffer(ipucsi->ipuch, 0);
+
+	/*
+	 * Point the inactive buffer address to the next queued buffer,
+	 * if available. Otherwise, prepare to reuse the currently active
+	 * buffer, unless ipucsi_videobuf_queue gets called in time.
+	 */
+	if (!list_is_singular(&ipucsi->capture)) {
+		buf = list_entry(ipucsi->capture.next->next,
+				 struct ipucsi_buffer, queue);
+		vb = &buf->vb;
+	}
+
+	eba = vb2_dma_contig_plane_dma_addr(vb, 0);
+	if (ipucsi->ilo < 0)
+		eba -= ipucsi->ilo;
+
+	ipu_cpmem_set_buffer(ipu_get_cpmem(ipucsi->ipuch), 1, eba);
+
+	ipu_idmac_select_buffer(ipucsi->ipuch, 1);
+
+	spin_unlock_irqrestore(&ipucsi->lock, flags);
+
+	ipu_smfc_enable(ipucsi->ipu);
+	ipu_idmac_enable_channel(ipucsi->ipuch);
+	ipu_csi_enable(ipucsi->ipu, ipucsi->id);
+
+	ipucsi->active = buf;
+	ipucsi->paused = false;
+
+	return 0;
+}
+
+static void ipucsi_clear_buffer(struct vb2_buffer *vb, u32 fourcc)
+{
+	u32 black, *p, *end;
+
+	switch (fourcc) {
+	case V4L2_PIX_FMT_UYVY:
+		black = 0x00800080;
+		break;
+	case V4L2_PIX_FMT_YUYV:
+		black = 0x80008000;
+		break;
+	default:
+		black = 0x00000000;
+		break;
+	}
+
+	p = vb2_plane_vaddr(vb, 0);
+	end = p + vb2_plane_size(vb, 0) / 4;
+
+	while (p < end)
+		*p++ = black;
+}
+
+int ipucsi_pause_stream(struct ipucsi *ipucsi)
+{
+	unsigned long flags;
+
+	if (ipucsi->paused)
+		return 0;
+
+	ipu_csi_disable(ipucsi->ipu, ipucsi->id);
+	ipu_idmac_disable_channel(ipucsi->ipuch);
+	ipu_smfc_disable(ipucsi->ipu);
+
+	ipucsi->paused = true;
+
+	/*
+	 * If there is a previously active frame, clear it to black and mark
+	 * it as done to hand it off to userspace, unless the list is singular
+	 */
+
+	spin_lock_irqsave(&ipucsi->lock, flags);
+
+	if (ipucsi->active && !list_is_singular(&ipucsi->capture)) {
+		struct ipucsi_buffer *buf = ipucsi->active;
+
+		ipucsi->active = NULL;
+		list_del_init(&buf->queue);
+
+		spin_unlock_irqrestore(&ipucsi->lock, flags);
+
+		ipucsi_clear_buffer(&buf->vb,
+				    ipucsi->format.fmt.pix.pixelformat);
+
+		vb2_buffer_done(&buf->vb, VB2_BUF_STATE_DONE);
+
+		spin_lock_irqsave(&ipucsi->lock, flags);
+	}
+
+	spin_unlock_irqrestore(&ipucsi->lock, flags);
+
+	return 0;
+}
+
+static void ipucsi_v4l2_dev_notify(struct v4l2_subdev *sd,
+				   unsigned int notification, void *arg)
+{
+	if (sd == NULL)
+		return;
+
+	if (notification == V4L2_SUBDEV_SYNC_LOCK_NOTIFY) {
+		struct media_entity_graph graph;
+		struct media_entity *entity;
+		struct ipucsi *ipucsi;
+		bool lock = *(bool *)arg;
+
+		/* Find the CSI (first subdevice entity of the graph) */
+		media_entity_graph_walk_start(&graph, &sd->entity);
+		while ((entity = media_entity_graph_walk_next(&graph)) &&
+		       media_entity_type(entity) != MEDIA_ENT_T_V4L2_SUBDEV);
+		if (!entity)
+			return;
+		sd = media_entity_to_v4l2_subdev(entity);
+		ipucsi = container_of(sd, struct ipucsi, subdev);
+
+		if (lock)
+			ipucsi_resume_stream(ipucsi);
+		else
+			ipucsi_pause_stream(ipucsi);
+	}
+}
+
 static irqreturn_t ipucsi_new_frame_handler(int irq, void *context)
 {
 	struct ipucsi *ipucsi = context;
@@ -735,6 +890,8 @@ static int ipucsi_videobuf_start_streaming(struct vb2_queue *vq, unsigned int co
 	ipu_smfc_enable(ipucsi->ipu);
 	ipu_csi_enable(ipucsi->ipu, ipucsi->id);
 
+	ipucsi->paused = false;
+
 	ret = v4l2_media_subdev_s_stream(&ipucsi->subdev.entity, 1);
 	if (ret)
 		goto free_irq;
@@ -758,6 +915,8 @@ static int ipucsi_videobuf_stop_streaming(struct vb2_queue *vq)
 	ipu_idmac_disable_channel(ipucsi->ipuch);
 	ipu_smfc_disable(ipucsi->ipu);
 
+	ipucsi->paused = false;
+
 	spin_lock_irqsave(&ipucsi->lock, flags);
 	while (!list_empty(&ipucsi->capture)) {
 		struct ipucsi_buffer *buf = list_entry(ipucsi->capture.next,
@@ -1437,6 +1596,7 @@ static int ipucsi_probe(struct platform_device *pdev)
 	ipucsi->ipu = ipu;
 	ipucsi->dev = &pdev->dev;
 	ipucsi->v4l2_dev = ipu_media_get_v4l2_dev();
+	ipucsi->v4l2_dev->notify = ipucsi_v4l2_dev_notify;
 	if (!ipucsi->v4l2_dev)
 		return -EPROBE_DEFER;
 
-- 
2.0.0.rc2

[RFC PATCH 20/26] [media] imx-ipuv3-csi: Export sync lock event to userspace

[Philipp Zabel]

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/imx/imx-ipuv3-csi.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/media/platform/imx/imx-ipuv3-csi.c b/drivers/media/platform/imx/imx-ipuv3-csi.c
index ab22cad..86fadd0 100644
--- a/drivers/media/platform/imx/imx-ipuv3-csi.c
+++ b/drivers/media/platform/imx/imx-ipuv3-csi.c
@@ -43,12 +43,19 @@
 #include <media/v4l2-common.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
 #include <media/v4l2-ioctl.h>
 #include <media/v4l2-dev.h>
 #include <media/v4l2-of.h>
 
 #define DRIVER_NAME "imx-ipuv3-camera"
 
+#define V4L2_EVENT_SYNC_LOCK	(V4L2_EVENT_PRIVATE_START | 0x200)
+
+struct v4l2_event_sync_lock {
+	__u8 lock;
+} __attribute__ ((packed));
+
 /* CMOS Sensor Interface Registers */
 #define CSI_SENS_CONF		0x0000
 #define CSI_SENS_FRM_SIZE	0x0004
@@ -579,6 +586,7 @@ static void ipucsi_v4l2_dev_notify(struct v4l2_subdev *sd,
 	if (notification == V4L2_SUBDEV_SYNC_LOCK_NOTIFY) {
 		struct media_entity_graph graph;
 		struct media_entity *entity;
+		struct v4l2_event event;
 		struct ipucsi *ipucsi;
 		bool lock = *(bool *)arg;
 
@@ -595,6 +603,11 @@ static void ipucsi_v4l2_dev_notify(struct v4l2_subdev *sd,
 			ipucsi_resume_stream(ipucsi);
 		else
 			ipucsi_pause_stream(ipucsi);
+
+		memset(&event, 0, sizeof(event));
+		event.type = V4L2_EVENT_SYNC_LOCK;
+		((struct v4l2_event_sync_lock *)event.u.data)->lock = lock;
+		v4l2_event_queue(&ipucsi->vdev, &event);
 	}
 }
 
@@ -1378,6 +1391,14 @@ static int ipucsi_enum_framesizes(struct file *file, void *fh,
 	return 0;
 }
 
+static int ipucsi_subscribe_event(struct v4l2_fh *fh,
+				  const struct v4l2_event_subscription *sub)
+{
+	if (sub->type == V4L2_EVENT_SYNC_LOCK)
+		return v4l2_event_subscribe(fh, sub, 0, NULL);
+	return -EINVAL;
+}
+
 static const struct v4l2_ioctl_ops ipucsi_capture_ioctl_ops = {
 	.vidioc_querycap		= ipucsi_querycap,
 
@@ -1397,6 +1418,9 @@ static const struct v4l2_ioctl_ops ipucsi_capture_ioctl_ops = {
 	.vidioc_streamoff		= vb2_ioctl_streamoff,
 
 	.vidioc_enum_framesizes		= ipucsi_enum_framesizes,
+
+	.vidioc_subscribe_event		= ipucsi_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
 };
 
 static int ipucsi_subdev_s_ctrl(struct v4l2_ctrl *ctrl)
-- 
2.0.0.rc2

[RFC PATCH 21/26] [media] v4l2-subdev.h: Add lock status notification

[Philipp Zabel]

This notification type can be used by ADC converters with their
own interrupt handler to notify the bridge or capture interface
driver about the signal lock status.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 include/media/v4l2-subdev.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 28f4d8c..0af5e08 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -40,6 +40,9 @@
 #define V4L2_SUBDEV_IR_TX_NOTIFY		_IOW('v', 1, u32)
 #define V4L2_SUBDEV_IR_TX_FIFO_SERVICE_REQ	0x00000001
 
+#define V4L2_SUBDEV_SYNC_LOCK_NOTIFY		_IOW('v', 2, u32)
+#define V4L2_SUBDEV_SYNC_LOCK			0x00000001
+
 struct v4l2_device;
 struct v4l2_ctrl_handler;
 struct v4l2_event_subscription;
-- 
2.0.0.rc2

[RFC PATCH 22/26] [media] v4l2-subdev: Export v4l2_subdev_fops

[Philipp Zabel]

This is needed by the imx-ipuv3-csi driver when compiled as a module.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index aea84ac..c4dc495 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -406,6 +406,7 @@ const struct v4l2_file_operations v4l2_subdev_fops = {
 	.release = subdev_close,
 	.poll = subdev_poll,
 };
+EXPORT_SYMBOL_GPL(v4l2_subdev_fops);
 
 #ifdef CONFIG_MEDIA_CONTROLLER
 int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
-- 
2.0.0.rc2

[RFC PATCH 23/26] mfd: syscon: add child device support

[Philipp Zabel]

For devices which have a complete register for themselves, it is possible to
place them next to the syscon device with overlapping reg ranges. The same is
not possible for devices which only occupy bitfields in registers shared with
other users.
For devices that are completely controlled by bitfields in the syscon address
range, such as multiplexers or voltage regulators, allow to put child devices
into the syscon device node.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 Documentation/devicetree/bindings/mfd/syscon.txt | 11 +++++++++++
 drivers/mfd/syscon.c                             |  3 +++
 2 files changed, 14 insertions(+)

diff --git a/Documentation/devicetree/bindings/mfd/syscon.txt b/Documentation/devicetree/bindings/mfd/syscon.txt
index fe8150b..a7e11d5 100644
--- a/Documentation/devicetree/bindings/mfd/syscon.txt
+++ b/Documentation/devicetree/bindings/mfd/syscon.txt
@@ -9,10 +9,21 @@ using a specific compatible value), interrogate the node (or associated
 OS driver) to determine the location of the registers, and access the
 registers directly.
 
+Optionally, devices that are only controlled through single syscon
+registers or bitfields can also be added as child nodes to the syscon
+device node. These devices can implicitly assume their parent node
+as syscon provider without referencing it explicitly via phandle.
+In this case, the syscon node should have #address-cells = <1> and
+#size-cells = <0> and no ranges property.
+
 Required properties:
 - compatible: Should contain "syscon".
 - reg: the register region can be accessed from syscon
 
+Optional properties:
+- #address-cells: Should be 1.
+- #size-cells: Should be 0.
+
 Examples:
 gpr: iomuxc-gpr@020e0000 {
 	compatible = "fsl,imx6q-iomuxc-gpr", "syscon";
diff --git a/drivers/mfd/syscon.c b/drivers/mfd/syscon.c
index dbea55d..4b5d237 100644
--- a/drivers/mfd/syscon.c
+++ b/drivers/mfd/syscon.c
@@ -147,6 +147,9 @@ static int syscon_probe(struct platform_device *pdev)
 
 	dev_dbg(dev, "regmap %pR registered\n", res);
 
+	if (!of_device_is_compatible(pdev->dev.of_node, "simple-bus"))
+		of_platform_populate(pdev->dev.of_node, NULL, NULL, &pdev->dev);
+
 	return 0;
 }
 
-- 
2.0.0.rc2

[RFC PATCH 24/26] [media] imx: Add video switch

[Philipp Zabel]

This driver can handle SoC internal and extern video bus multiplexers,
controlled either by register bit fields or by GPIO.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 drivers/media/platform/imx/Kconfig            |   8 +
 drivers/media/platform/imx/Makefile           |   1 +
 drivers/media/platform/imx/imx-video-switch.c | 347 ++++++++++++++++++++++++++
 3 files changed, 356 insertions(+)
 create mode 100644 drivers/media/platform/imx/imx-video-switch.c

diff --git a/drivers/media/platform/imx/Kconfig b/drivers/media/platform/imx/Kconfig
index 8431441..ffced1c 100644
--- a/drivers/media/platform/imx/Kconfig
+++ b/drivers/media/platform/imx/Kconfig
@@ -11,6 +11,14 @@ config MEDIA_IMX_IPU
 	  This driver provides core support for the i.MX IPUv3 contained in
 	  i.MX5 and i.MX6 SoCs.
 
+config MEDIA_IMX_VIDEO_SWITCH
+	tristate "i.MX Video Bus Multiplexer"
+	depends on VIDEO_V4L2_SUBDEV_API
+	help
+	  This driver provides support for SoC internal video bus multiplexers
+	  controlled by register bitfields as well as external multiplexers
+	  controller by a GPIO.
+
 config VIDEO_IMX_IPU_COMMON
 	tristate
 
diff --git a/drivers/media/platform/imx/Makefile b/drivers/media/platform/imx/Makefile
index 49d8fab..d972674 100644
--- a/drivers/media/platform/imx/Makefile
+++ b/drivers/media/platform/imx/Makefile
@@ -1,4 +1,5 @@
 obj-$(CONFIG_MEDIA_IMX)			+= imx-media.o
+obj-$(CONFIG_MEDIA_IMX_VIDEO_SWITCH)	+= imx-video-switch.o
 obj-$(CONFIG_VIDEO_IMX_IPU_COMMON)	+= imx-ipu.o
 obj-$(CONFIG_VIDEO_IMX_IPU_CSI)		+= imx-ipuv3-csi.o
 obj-$(CONFIG_VIDEO_IMX_IPU_SCALER)	+= imx-ipu-scaler.o
diff --git a/drivers/media/platform/imx/imx-video-switch.c b/drivers/media/platform/imx/imx-video-switch.c
new file mode 100644
index 0000000..7a5f64f
--- /dev/null
+++ b/drivers/media/platform/imx/imx-video-switch.c
@@ -0,0 +1,347 @@
+/*
+ * devicetree probed mediacontrol multiplexer.
+ *
+ * Copyright (C) 2013 Sascha Hauer, Pengutronix
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/err.h>
+#include <linux/gpio.h>
+#include <linux/mfd/syscon.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_gpio.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/of_graph.h>
+#include <media/v4l2-subdev.h>
+#include <media/v4l2-of.h>
+
+#include <media/imx.h> /* for ipu_media_entity_create_link */
+
+enum {
+	PAD_INPUT0,
+	PAD_INPUT1,
+	PAD_OUTPUT,
+	PAD_NUM,
+};
+
+struct vidsw {
+	struct device *dev;
+	struct v4l2_subdev subdev;
+	struct media_pad pads[PAD_NUM];
+	struct v4l2_mbus_framefmt format_mbus[PAD_NUM];
+	struct v4l2_of_endpoint endpoint[PAD_NUM - 1];
+	struct regmap_field *field;
+	unsigned int gpio;
+	int streaming;
+	int active;
+};
+
+#define to_vidsw(sd) container_of(sd, struct vidsw, subdev)
+
+static int vidsw_link_setup(struct media_entity *entity,
+		const struct media_pad *local,
+		const struct media_pad *remote, u32 flags)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct vidsw *vidsw = to_vidsw(sd);
+
+	dev_dbg(vidsw->dev, "link setup %s -> %s", remote->entity->name,
+		local->entity->name);
+
+	if (vidsw->streaming)
+		return -EBUSY;
+
+	if (!(flags & MEDIA_LNK_FL_ENABLED)) {
+		if (local->index == vidsw->active) {
+			dev_dbg(vidsw->dev, "going inactive\n");
+			vidsw->active = -1;
+		}
+		return 0;
+	}
+
+	if (vidsw->active >= 0) {
+		if (vidsw->active == local->index)
+			return 0;
+		else
+			return -EBUSY;
+	}
+
+	vidsw->active = local->index;
+
+	dev_dbg(vidsw->dev, "setting %d active\n", vidsw->active);
+
+	if (vidsw->field)
+		regmap_field_write(vidsw->field, vidsw->active);
+	else if (gpio_is_valid(vidsw->gpio))
+		gpio_set_value(vidsw->gpio, vidsw->active);
+
+	return 0;
+}
+
+static struct media_entity_operations vidsw_ops = {
+	.link_setup = vidsw_link_setup,
+};
+
+static int vidsw_async_init(struct vidsw *vidsw, struct device_node *node)
+{
+	struct v4l2_of_endpoint endpoint;
+	struct device_node *rp, *port;
+	u32 portno;
+	int numports;
+	int ret;
+
+	numports = PAD_NUM;
+
+	vidsw->pads[PAD_INPUT0].flags = MEDIA_PAD_FL_SINK;
+	vidsw->pads[PAD_INPUT1].flags = MEDIA_PAD_FL_SINK;
+	vidsw->pads[PAD_OUTPUT].flags = MEDIA_PAD_FL_SOURCE;
+
+	ret = media_entity_init(&vidsw->subdev.entity, numports, vidsw->pads, 0);
+	if (ret < 0)
+		return ret;
+
+	vidsw->subdev.entity.ops = &vidsw_ops;
+
+	rp = NULL;
+
+	while (1) {
+		rp = of_graph_get_next_endpoint(node, rp);
+		if (!rp)
+			break;
+
+		port = of_get_parent(rp);
+		if (!port)
+			return -EINVAL;
+
+		portno = 0;
+		of_property_read_u32(port, "reg", &portno);
+
+		v4l2_of_parse_endpoint(rp, &endpoint);
+		of_node_put(rp);
+
+		if (portno > 1)
+			break;
+
+		vidsw->endpoint[portno] = endpoint;
+
+		/* FIXME */
+		ipu_media_entity_create_link(&vidsw->subdev, portno, rp, 0);
+	}
+
+	return 0;
+}
+
+static int vidsw_registered(struct v4l2_subdev *sd)
+{
+	return 0;
+}
+
+int vidsw_g_mbus_config(struct v4l2_subdev *sd, struct v4l2_mbus_config *cfg)
+{
+	struct vidsw *vidsw = container_of(sd, struct vidsw, subdev);
+
+	dev_dbg(vidsw->dev, "reporting configration %d\n", vidsw->active);
+
+	/* Mirror the input side on the output side */
+	cfg->type = vidsw->endpoint[vidsw->active].bus_type;
+	if (cfg->type == V4L2_MBUS_PARALLEL || cfg->type == V4L2_MBUS_BT656)
+		cfg->flags = vidsw->endpoint[vidsw->active].bus.parallel.flags;
+
+	return 0;
+}
+
+static const struct v4l2_subdev_video_ops vidsw_subdev_video_ops = {
+	.g_mbus_config = vidsw_g_mbus_config,
+};
+
+static struct v4l2_mbus_framefmt *
+__vidsw_get_pad_format(struct vidsw *vidsw, struct v4l2_subdev_fh *fh,
+		       unsigned int pad, u32 which)
+{
+	switch (which) {
+	case V4L2_SUBDEV_FORMAT_TRY:
+		return v4l2_subdev_get_try_format(fh, pad);
+	case V4L2_SUBDEV_FORMAT_ACTIVE:
+		return &vidsw->format_mbus[pad];
+	default:
+		return NULL;
+	}
+}
+
+static int vidsw_get_format(struct v4l2_subdev *sd,
+			    struct v4l2_subdev_fh *fh,
+			    struct v4l2_subdev_format *sdformat)
+{
+	struct vidsw *vidsw = container_of(sd, struct vidsw, subdev);
+
+	sdformat->format = *__vidsw_get_pad_format(vidsw, fh, sdformat->pad,
+						   sdformat->which);
+	return 0;
+}
+
+static int vidsw_set_format(struct v4l2_subdev *sd,
+			    struct v4l2_subdev_fh *fh,
+			    struct v4l2_subdev_format *sdformat)
+{
+	struct vidsw *vidsw = container_of(sd, struct vidsw, subdev);
+	struct v4l2_mbus_framefmt *mbusformat;
+
+	if (sdformat->pad >= PAD_NUM)
+		return -EINVAL;
+
+	mbusformat = __vidsw_get_pad_format(vidsw, fh, sdformat->pad,
+					    sdformat->which);
+	if (!mbusformat)
+		return -EINVAL;
+
+	/* Output pad mirrors active input pad, no limitations on input pads */
+	if (sdformat->pad == PAD_OUTPUT && vidsw->active >= 0)
+		*mbusformat = vidsw->format_mbus[vidsw->active];
+	else
+		*mbusformat = sdformat->format;
+
+	sdformat->format = *mbusformat;
+
+	return 0;
+}
+
+static struct v4l2_subdev_pad_ops vidsw_pad_ops = {
+	.get_fmt = vidsw_get_format,
+	.set_fmt = vidsw_set_format,
+};
+
+static struct v4l2_subdev_ops vidsw_subdev_ops = {
+	.pad = &vidsw_pad_ops,
+	.video = &vidsw_subdev_video_ops,
+};
+
+static struct v4l2_subdev_internal_ops vidsw_internal_ops = {
+	.registered = vidsw_registered,
+};
+
+static int of_get_reg_field(struct device_node *node, struct reg_field *field)
+{
+	u32 bit_mask;
+	int ret;
+
+	ret = of_property_read_u32(node, "reg", &field->reg);
+	if (ret < 0)
+		return ret;
+
+	ret = of_property_read_u32(node, "bit-mask", &bit_mask);
+	if (ret < 0)
+		return ret;
+
+	ret = of_property_read_u32(node, "bit-shift", &field->lsb);
+	if (ret < 0)
+		return ret;
+
+	field->msb = field->lsb + ffs(bit_mask) - 1;
+
+	return 0;
+}
+
+static int vidsw_probe(struct platform_device *pdev)
+{
+	struct device_node *np = pdev->dev.of_node;
+	struct reg_field field;
+	struct vidsw *vidsw;
+	struct regmap *map;
+	int num_pads;
+	int ret;
+
+	vidsw = devm_kzalloc(&pdev->dev, sizeof(*vidsw), GFP_KERNEL);
+	if (!vidsw)
+		return -ENOMEM;
+
+	platform_set_drvdata(pdev, vidsw);
+
+	v4l2_subdev_init(&vidsw->subdev, &vidsw_subdev_ops);
+	vidsw->subdev.internal_ops = &vidsw_internal_ops;
+	snprintf(vidsw->subdev.name, sizeof(vidsw->subdev.name), "%s",
+			np->name);
+	vidsw->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	vidsw->subdev.dev = &pdev->dev;
+	vidsw->dev = &pdev->dev;
+	vidsw->active = -1;
+
+	num_pads = of_get_child_count(np);
+	if (num_pads < 2) {
+		dev_err(&pdev->dev, "Not enough ports %d\n", num_pads);
+		return -EINVAL;
+	}
+
+	ret = of_get_reg_field(np, &field);
+	if (ret == 0) {
+		map = syscon_node_to_regmap(np->parent);
+		if (!map) {
+			dev_err(&pdev->dev, "Failed to get syscon register map\n");
+			return PTR_ERR(map);
+		}
+
+		vidsw->field = devm_regmap_field_alloc(&pdev->dev, map, field);
+		if (IS_ERR(vidsw->field)) {
+			dev_err(&pdev->dev, "Failed to allocate regmap field\n");
+			return PTR_ERR(vidsw->field);
+		}
+	} else {
+		vidsw->gpio = of_get_named_gpio_flags(np, "gpios", 0,
+						    NULL);
+		ret = gpio_request_one(vidsw->gpio,
+				       GPIOF_OUT_INIT_LOW, np->name);
+		if (ret < 0) {
+			dev_warn(&pdev->dev,
+				 "could not request control gpio %d: %d\n",
+				 vidsw->gpio, ret);
+			vidsw->gpio = -1;
+		}
+	}
+
+	ret = vidsw_async_init(vidsw, np);
+	if (ret)
+		return ret;
+
+	ret = v4l2_async_register_subdev(&vidsw->subdev);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int vidsw_remove(struct platform_device *pdev)
+{
+	/* FIXME */
+
+	return -EBUSY;
+}
+
+static const struct of_device_id vidsw_dt_ids[] = {
+	{ .compatible = "video-multiplexer", },
+	{ /* sentinel */ }
+};
+
+static struct platform_driver vidsw_driver = {
+	.probe		= vidsw_probe,
+	.remove		= vidsw_remove,
+	.driver		= {
+		.of_match_table = vidsw_dt_ids,
+		.name	= "video-multiplexer",
+		.owner	= THIS_MODULE,
+	},
+};
+
+module_platform_driver(vidsw_driver);
+
+MODULE_DESCRIPTION("i.MX video stream multiplexer");
+MODULE_AUTHOR("Sascha Hauer, Pengutronix");
+MODULE_LICENSE("GPL");
-- 
2.0.0.rc2

[RFC PATCH 25/26] ARM: dts: Add IPU aliases on i.MX6

[Philipp Zabel]

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 arch/arm/boot/dts/imx6q.dtsi   | 1 +
 arch/arm/boot/dts/imx6qdl.dtsi | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index fb6185f..919a18a 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -14,6 +14,7 @@
 
 / {
 	aliases {
+		ipu1 = &ipu2;
 		spi4 = &ecspi5;
 	};
 
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index 84ddc94..27303d6 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -28,6 +28,7 @@
 		i2c0 = &i2c1;
 		i2c1 = &i2c2;
 		i2c2 = &i2c3;
+		ipu0 = &ipu1;
 		mmc0 = &usdhc1;
 		mmc1 = &usdhc2;
 		mmc2 = &usdhc3;
-- 
2.0.0.rc2

[RFC PATCH 26/26] ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their connections

[Philipp Zabel]

This patch adds the device tree graph connecting the input multiplexers
to the IPU CSIs and the MIPI-CSI2 gasket on i.MX6. The MIPI_IPU multiplexers
are added as children of the iomuxc-gpr syscon device node.
On i.MX6Q/D two two-input multiplexers in front of IPU1 CSI0 and IPU2 CSI1
allow to select between CSI0/1 parallel input pads and the MIPI CSI-2 virtual
channels 0/3.
On i.MX6DL/S two five-input multiplexers in front of IPU1 CSI0 and IPU1 CSI1
allow to select between CSI0/1 parallel input pads and any of the four MIPI
CSI-2 virtual channels.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 arch/arm/boot/dts/imx6dl.dtsi  | 182 +++++++++++++++++++++++++++++++++++++++++
 arch/arm/boot/dts/imx6q.dtsi   | 118 ++++++++++++++++++++++++++
 arch/arm/boot/dts/imx6qdl.dtsi |   8 ++
 3 files changed, 308 insertions(+)

diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 5c5f574..619c7ce 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -110,3 +110,185 @@
 		      "di0_sel", "di1_sel",
 		      "di0", "di1";
 };
+
+&gpr {
+	ipu_csi0_mux {
+		compatible = "video-multiplexer";
+		reg = <0x34>;
+		bit-mask = <0x7>;
+		bit-shift = <0>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		port@0 {
+			reg = <0>;
+
+			ipu_csi0_mux_from_mipi_csi0: endpoint {
+				remote-endpoint = <&mipi_csi0_to_ipu_csi0_mux>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+
+			ipu_csi0_mux_from_mipi_csi1: endpoint {
+				remote-endpoint = <&mipi_csi1_to_ipu_csi0_mux>;
+			};
+		};
+
+		port@2 {
+			reg = <2>;
+
+			ipu_csi0_mux_from_mipi_csi2: endpoint {
+				remote-endpoint = <&mipi_csi2_to_ipu_csi0_mux>;
+			};
+		};
+
+		port@3 {
+			reg = <3>;
+
+			ipu_csi0_mux_from_mipi_csi3: endpoint {
+				remote-endpoint = <&mipi_csi3_to_ipu_csi0_mux>;
+			};
+		};
+
+		csi0: port@4 {
+			reg = <4>;
+		};
+
+		port@5 {
+			reg = <5>;
+
+			ipu_csi0_mux_to_ipu1_csi0: endpoint {
+				remote-endpoint = <&ipu1_csi0_from_ipu_csi0_mux>;
+			};
+		};
+	};
+
+	ipu_csi1_mux {
+		compatible = "video-multiplexer";
+		reg = <0x34>;
+		bit-mask = <0x7>;
+		bit-shift = <0>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		port@0 {
+			reg = <0>;
+
+			ipu_csi1_mux_from_mipi_csi0: endpoint {
+				remote-endpoint = <&mipi_csi0_to_ipu_csi1_mux>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+
+			ipu_csi1_mux_from_mipi_csi1: endpoint {
+				remote-endpoint = <&mipi_csi1_to_ipu_csi1_mux>;
+			};
+		};
+
+		port@2 {
+			reg = <2>;
+
+			ipu_csi1_mux_from_mipi_csi2: endpoint {
+				remote-endpoint = <&mipi_csi2_to_ipu_csi1_mux>;
+			};
+		};
+
+		port@3 {
+			reg = <3>;
+
+			ipu_csi1_mux_from_mipi_csi3: endpoint {
+				remote-endpoint = <&mipi_csi3_to_ipu_csi1_mux>;
+			};
+		};
+
+		csi1: port@4 {
+			reg = <4>;
+		};
+
+		port@5 {
+			reg = <5>;
+
+			ipu_csi1_mux_to_ipu1_csi1: endpoint {
+				remote-endpoint = <&ipu1_csi1_from_ipu_csi1_mux>;
+			};
+		};
+	};
+};
+
+&ipu1_csi0 {
+	ipu1_csi0_from_ipu_csi0_mux: endpoint {
+		remote-endpoint = <&ipu_csi0_mux_to_ipu1_csi0>;
+	};
+};
+
+&ipu1_csi1 {
+	ipu1_csi1_from_ipu_csi1_mux: endpoint {
+		remote-endpoint = <&ipu_csi1_mux_to_ipu1_csi1>;
+	};
+};
+
+&mipi_csi {
+	port@0 {
+		reg = <0>;
+	};
+
+	port@1 {
+		reg = <1>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_csi0_to_ipu_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu_csi0_mux_from_mipi_csi0>;
+		};
+
+		mipi_csi0_to_ipu_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu_csi1_mux_from_mipi_csi0>;
+		};
+	};
+
+	port@2 {
+		reg = <2>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_csi1_to_ipu_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu_csi0_mux_from_mipi_csi1>;
+		};
+
+		mipi_csi1_to_ipu_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu_csi1_mux_from_mipi_csi1>;
+		};
+	};
+
+	port@3 {
+		reg = <3>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_csi2_to_ipu_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu_csi0_mux_from_mipi_csi2>;
+		};
+
+		mipi_csi2_to_ipu_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu_csi1_mux_from_mipi_csi2>;
+		};
+	};
+
+	port@4 {
+		reg = <4>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_csi3_to_ipu_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu_csi0_mux_from_mipi_csi3>;
+		};
+
+		mipi_csi3_to_ipu_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu_csi1_mux_from_mipi_csi3>;
+		};
+	};
+};
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index 919a18a..bac825e 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -159,10 +159,18 @@
 
 			ipu2_csi0: port@0 {
 				reg = <0>;
+
+				ipu2_csi0_from_csi2ipu: endpoint {
+					remote-endpoint = <&csi2ipu_to_ipu2_csi0>;
+				};
 			};
 
 			ipu2_csi1: port@1 {
 				reg = <1>;
+
+				ipu2_csi1_from_mipi_ipu2_mux: endpoint {
+					remote-endpoint = <&mipi_ipu2_mux_to_ipu2_csi1>;
+				};
 			};
 
 			ipu2_di0: port@2 {
@@ -238,6 +246,78 @@
 	};
 };
 
+&gpr {
+	mipi_ipu1_mux {
+		compatible = "video-multiplexer";
+		reg = <0x04>;
+		bit-mask = <1>;
+		bit-shift = <19>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		port@0 {
+			reg = <0>;
+
+			mipi_ipu1_mux_from_mipi_csi0: endpoint {
+				remote-endpoint = <&mipi_csi0_to_mipi_ipu1_mux>;
+			};
+		};
+
+		csi0: port@1 {
+			reg = <1>;
+		};
+
+		port@2 {
+			reg = <2>;
+
+			mipi_ipu1_mux_to_ipu1_csi0: endpoint {
+				remote-endpoint = <&ipu1_csi0_from_mipi_ipu1_mux>;
+			};
+		};
+	};
+
+	mipi_ipu2_mux {
+		compatible = "video-multiplexer";
+		reg = <0x04>;
+		bit-mask = <1>;
+		bit-shift = <20>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		port@0 {
+			reg = <0>;
+
+			mipi_ipu2_mux_from_mipi_csi3: endpoint {
+				remote-endpoint = <&mipi_csi3_to_mipi_ipu2_mux>;
+			};
+		};
+
+		csi1: port@1 {
+			reg = <1>;
+		};
+
+		port@2 {
+			reg = <2>;
+
+			mipi_ipu2_mux_to_ipu2_csi1: endpoint {
+				remote-endpoint = <&ipu2_csi1_from_mipi_ipu2_mux>;
+			};
+		};
+	};
+};
+
+&ipu1_csi1 {
+	ipu1_csi1_from_mipi_csi1: endpoint {
+		remote-endpoint = <&mipi_csi1_to_ipu1_csi1>;
+	};
+};
+
+&ipu2_csi0 {
+	ipu2_csi0_from_mipi_csi2: endpoint {
+		remote-endpoint = <&mipi_csi2_to_ipu2_csi0>;
+	};
+};
+
 &ldb {
 	clocks = <&clks 33>, <&clks 34>,
 		 <&clks 39>, <&clks 40>, <&clks 41>, <&clks 42>,
@@ -283,6 +363,44 @@
 	};
 };
 
+&mipi_csi {
+	port@0 {
+		reg = <0>;
+	};
+
+	port@1 {
+		reg = <1>;
+
+		mipi_csi0_to_mipi_ipu1_mux: endpoint {
+			remote-endpoint = <&mipi_ipu1_mux_from_mipi_csi0>;
+		};
+	};
+
+	port@2 {
+		reg = <2>;
+
+		mipi_csi1_to_ipu1_csi1: endpoint {
+			remote-endpoint = <&ipu1_csi1_from_mipi_csi1>;
+		};
+	};
+
+	port@3 {
+		reg = <3>;
+
+		mipi_csi2_to_ipu2_csi0: endpoint {
+			remote-endpoint = <&ipu2_csi0_from_mipi_csi2>;
+		};
+	};
+
+	port@4 {
+		reg = <4>;
+
+		mipi_csi3_to_mipi_ipu2_mux: endpoint {
+			remote-endpoint = <&mipi_ipu2_mux_from_mipi_csi3>;
+		};
+	};
+};
+
 &mipi_dsi {
 	port@2 {
 		reg = <2>;
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index 27303d6..e2f8924 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -659,6 +659,8 @@
 			gpr: iomuxc-gpr@020e0000 {
 				compatible = "fsl,imx6q-iomuxc-gpr", "syscon";
 				reg = <0x020e0000 0x38>;
+				#address-cells = <1>;
+				#size-cells = <0>;
 			};
 
 			iomuxc: iomuxc@020e0000 {
@@ -961,6 +963,8 @@
 
 			mipi_csi: mipi@021dc000 {
 				reg = <0x021dc000 0x4000>;
+				#address-cells = <1>;
+				#size-cells = <0>;
 			};
 
 			mipi_dsi: mipi@021e0000 {
@@ -1049,6 +1053,10 @@
 
 			ipu1_csi0: port@0 {
 				reg = <0>;
+
+				ipu1_csi0_from_mipi_ipu1_mux: endpoint {
+					remote-endpoint = <&mipi_ipu1_mux_to_ipu1_csi0>;
+				};
 			};
 
 			ipu1_csi1: port@1 {
-- 
2.0.0.rc2

Re: [RFC PATCH 14/26] [media] Add i.MX SoC wide media device driver

[Dave Müller]

Hello

Philipp Zabel <p.zabel@pengutronix.de> writes:

> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index 8e9c26c..3fe7e28 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -35,6 +35,8 @@  source "drivers/media/platform/omap/Kconfig"
> 
>  source "drivers/media/platform/blackfin/Kconfig"
> 
> +source "drivers/media/platform/imx/Kconfig"
> +

This was added already in patch #8 of this serie.

Re: [RFC PATCH 14/26] [media] Add i.MX SoC wide media device driver

[Dave Müller]

Hello again

Philipp Zabel <p.zabel@pengutronix.de> writes:

> +struct media_device *ipu_find_media_device(void)
> +{
> +	return &ipu_media->mdev;
> +}
> +EXPORT_SYMBOL_GPL(ipu_find_media_device);

Where is ipu_find_media_device() being called?

> +int ipu_media_device_register(struct device *dev)
> +{

[snip]

> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(ipu_media_device_register);

Where is ipu_media_device_register() being called?

Re: [RFC PATCH 00/26] i.MX5/6 IPUv3 CSI/IC

[Zahari Doychev]

Hi Philipp,

can you tell which kernel tree I have to use for this patch set?

Thanks and Regards,
Zahari

On Thu, Jun 12, 2014 at 07:06:14PM +0200, Philipp Zabel wrote:
> Hi,
> 
> attached is a series of our work in progress i.MX6 capture drivers.
> I'm posting this now in reaction to Steve's i.MX6 Video capture series,
> as a reference for further discussion.
> Of the Image Converter (IC) we only use the postprocessor task, with
> tiling for larger frames, to implement v4l2 mem2mem scaler/colorspace
> converter and deinterlacer devices.
> The capture code capture code already uses the media controller framework
> and creates a subdevice representing the CSI, but the path to memory is
> fixed to IDMAC via SMFC, which is the only possible path for grayscale
> and  and anything with multiple output ports connected
> to the CSIs (such as the CSI2IPU gasket on i.MX6) doesn't work yet. Also,
> I think the CSI subdevice driver should be completely separate from the
> capture driver.
> 
> regards
> Philipp
> 
> Philipp Zabel (16):
>   gpu: ipu-v3: Add function to setup CP channel as interlaced
>   gpu: ipu-v3: Add ipu_cpmem_get_buffer function
>   gpu: ipu-v3: Add support for partial interleaved YCbCr 4:2:0 (NV12)
>     format
>   gpu: ipu-v3: Add support for planar YUV 4:2:2 (YUV422P) format
>   imx-drm: currently only IPUv3 is supported, make it mandatory
>   [media] Add i.MX SoC wide media device driver
>   [media] imx-ipu: Add i.MX IPUv3 capture driver
>   [media] ipuv3-csi: Skip 3 lines for NTSC BT.656
>   [media] imx-ipuv3-csi: Add support for temporarily stopping the stream
>     on sync loss
>   [media] imx-ipuv3-csi: Export sync lock event to userspace
>   [media] v4l2-subdev.h: Add lock status notification
>   [media] v4l2-subdev: Export v4l2_subdev_fops
>   mfd: syscon: add child device support
>   [media] imx: Add video switch
>   ARM: dts: Add IPU aliases on i.MX6
>   ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
>     connections
> 
> Sascha Hauer (10):
>   gpu: ipu-v3: Add IC support
>   gpu: ipu-v3: Register IC with IPUv3
>   [media] imx-ipu: add ipu media common code
>   [media] imx-ipu: Add i.MX IPUv3 scaler driver
>   [media] imx-ipu: Add i.MX IPUv3 deinterlacer driver
>   [media] v4l2: subdev: Add v4l2_device_register_subdev_node function
>   [media] v4l2: Fix V4L2_CID_PIXEL_RATE
>   [media] v4l2 async: remove from notifier list
>   [media] ipuv3-csi: Pass ipucsi to v4l2_media_subdev_s_power
>   [media] ipuv3-csi: make subdev controls available on video device
> 
>  Documentation/devicetree/bindings/mfd/syscon.txt |   11 +
>  arch/arm/boot/dts/imx6dl.dtsi                    |  182 +++
>  arch/arm/boot/dts/imx6q.dtsi                     |  119 ++
>  arch/arm/boot/dts/imx6qdl.dtsi                   |    9 +
>  drivers/gpu/ipu-v3/Makefile                      |    2 +-
>  drivers/gpu/ipu-v3/ipu-common.c                  |  119 ++
>  drivers/gpu/ipu-v3/ipu-ic.c                      | 1227 +++++++++++++++
>  drivers/gpu/ipu-v3/ipu-prv.h                     |    6 +
>  drivers/media/platform/Kconfig                   |    4 +
>  drivers/media/platform/Makefile                  |    1 +
>  drivers/media/platform/imx/Kconfig               |   50 +
>  drivers/media/platform/imx/Makefile              |    6 +
>  drivers/media/platform/imx/imx-ipu-scaler.c      |  825 +++++++++++
>  drivers/media/platform/imx/imx-ipu-vdic.c        |  716 +++++++++
>  drivers/media/platform/imx/imx-ipu.c             |  313 ++++
>  drivers/media/platform/imx/imx-ipu.h             |   36 +
>  drivers/media/platform/imx/imx-ipuv3-csi.c       | 1729 ++++++++++++++++++++++
>  drivers/media/platform/imx/imx-media.c           |  174 +++
>  drivers/media/platform/imx/imx-video-switch.c    |  347 +++++
>  drivers/media/v4l2-core/v4l2-async.c             |    1 +
>  drivers/media/v4l2-core/v4l2-ctrls.c             |    8 +-
>  drivers/media/v4l2-core/v4l2-device.c            |   63 +-
>  drivers/media/v4l2-core/v4l2-subdev.c            |    1 +
>  drivers/mfd/syscon.c                             |    3 +
>  drivers/staging/imx-drm/Kconfig                  |    7 +-
>  include/media/imx.h                              |   25 +
>  include/media/v4l2-device.h                      |    5 +
>  include/media/v4l2-subdev.h                      |    3 +
>  include/video/imx-ipu-v3.h                       |   16 +
>  29 files changed, 5976 insertions(+), 32 deletions(-)
>  create mode 100644 drivers/gpu/ipu-v3/ipu-ic.c
>  create mode 100644 drivers/media/platform/imx/Kconfig
>  create mode 100644 drivers/media/platform/imx/Makefile
>  create mode 100644 drivers/media/platform/imx/imx-ipu-scaler.c
>  create mode 100644 drivers/media/platform/imx/imx-ipu-vdic.c
>  create mode 100644 drivers/media/platform/imx/imx-ipu.c
>  create mode 100644 drivers/media/platform/imx/imx-ipu.h
>  create mode 100644 drivers/media/platform/imx/imx-ipuv3-csi.c
>  create mode 100644 drivers/media/platform/imx/imx-media.c
>  create mode 100644 drivers/media/platform/imx/imx-video-switch.c
>  create mode 100644 include/media/imx.h
> 
> -- 
> 2.0.0.rc2
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 00/26] i.MX5/6 IPUv3 CSI/IC

[Fabio Estevam]

Hi Philipp,


On Thu, Jun 12, 2014 at 2:06 PM, Philipp Zabel <p.zabel@pengutronix.de> wrote:
> Hi,
>
> attached is a series of our work in progress i.MX6 capture drivers.
> I'm posting this now in reaction to Steve's i.MX6 Video capture series,
> as a reference for further discussion.
> Of the Image Converter (IC) we only use the postprocessor task, with
> tiling for larger frames, to implement v4l2 mem2mem scaler/colorspace
> converter and deinterlacer devices.
> The capture code capture code already uses the media controller framework
> and creates a subdevice representing the CSI, but the path to memory is
> fixed to IDMAC via SMFC, which is the only possible path for grayscale
> and  and anything with multiple output ports connected
> to the CSIs (such as the CSI2IPU gasket on i.MX6) doesn't work yet. Also,
> I think the CSI subdevice driver should be completely separate from the
> capture driver.
>
> regards
> Philipp
>
> Philipp Zabel (16):
>   gpu: ipu-v3: Add function to setup CP channel as interlaced
>   gpu: ipu-v3: Add ipu_cpmem_get_buffer function
>   gpu: ipu-v3: Add support for partial interleaved YCbCr 4:2:0 (NV12)
>     format
>   gpu: ipu-v3: Add support for planar YUV 4:2:2 (YUV422P) format
>   imx-drm: currently only IPUv3 is supported, make it mandatory
>   [media] Add i.MX SoC wide media device driver
>   [media] imx-ipu: Add i.MX IPUv3 capture driver
>   [media] ipuv3-csi: Skip 3 lines for NTSC BT.656
>   [media] imx-ipuv3-csi: Add support for temporarily stopping the stream
>     on sync loss
>   [media] imx-ipuv3-csi: Export sync lock event to userspace
>   [media] v4l2-subdev.h: Add lock status notification
>   [media] v4l2-subdev: Export v4l2_subdev_fops
>   mfd: syscon: add child device support
>   [media] imx: Add video switch
>   ARM: dts: Add IPU aliases on i.MX6
>   ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
>     connections
>
> Sascha Hauer (10):
>   gpu: ipu-v3: Add IC support
>   gpu: ipu-v3: Register IC with IPUv3
>   [media] imx-ipu: add ipu media common code
>   [media] imx-ipu: Add i.MX IPUv3 scaler driver
>   [media] imx-ipu: Add i.MX IPUv3 deinterlacer driver
>   [media] v4l2: subdev: Add v4l2_device_register_subdev_node function
>   [media] v4l2: Fix V4L2_CID_PIXEL_RATE
>   [media] v4l2 async: remove from notifier list
>   [media] ipuv3-csi: Pass ipucsi to v4l2_media_subdev_s_power
>   [media] ipuv3-csi: make subdev controls available on video device

Do you have plans to resubmit this series?

Regards,

Fabio Estevam

Re: [RFC PATCH 00/26] i.MX5/6 IPUv3 CSI/IC

[Fabio Estevam]

Hi Philipp,

On Tue, Oct 27, 2015 at 11:10 AM, Fabio Estevam <festevam@gmail.com> wrote:
> Hi Philipp,
>
>
> On Thu, Jun 12, 2014 at 2:06 PM, Philipp Zabel <p.zabel@pengutronix.de> wrote:
>> Hi,
>>
>> attached is a series of our work in progress i.MX6 capture drivers.
>> I'm posting this now in reaction to Steve's i.MX6 Video capture series,
>> as a reference for further discussion.
>> Of the Image Converter (IC) we only use the postprocessor task, with
>> tiling for larger frames, to implement v4l2 mem2mem scaler/colorspace
>> converter and deinterlacer devices.
>> The capture code capture code already uses the media controller framework
>> and creates a subdevice representing the CSI, but the path to memory is
>> fixed to IDMAC via SMFC, which is the only possible path for grayscale
>> and  and anything with multiple output ports connected
>> to the CSIs (such as the CSI2IPU gasket on i.MX6) doesn't work yet. Also,
>> I think the CSI subdevice driver should be completely separate from the
>> capture driver.
>>
>> regards
>> Philipp
>>
>> Philipp Zabel (16):
>>   gpu: ipu-v3: Add function to setup CP channel as interlaced
>>   gpu: ipu-v3: Add ipu_cpmem_get_buffer function
>>   gpu: ipu-v3: Add support for partial interleaved YCbCr 4:2:0 (NV12)
>>     format
>>   gpu: ipu-v3: Add support for planar YUV 4:2:2 (YUV422P) format
>>   imx-drm: currently only IPUv3 is supported, make it mandatory
>>   [media] Add i.MX SoC wide media device driver
>>   [media] imx-ipu: Add i.MX IPUv3 capture driver
>>   [media] ipuv3-csi: Skip 3 lines for NTSC BT.656
>>   [media] imx-ipuv3-csi: Add support for temporarily stopping the stream
>>     on sync loss
>>   [media] imx-ipuv3-csi: Export sync lock event to userspace
>>   [media] v4l2-subdev.h: Add lock status notification
>>   [media] v4l2-subdev: Export v4l2_subdev_fops
>>   mfd: syscon: add child device support
>>   [media] imx: Add video switch
>>   ARM: dts: Add IPU aliases on i.MX6
>>   ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
>>     connections
>>
>> Sascha Hauer (10):
>>   gpu: ipu-v3: Add IC support
>>   gpu: ipu-v3: Register IC with IPUv3
>>   [media] imx-ipu: add ipu media common code
>>   [media] imx-ipu: Add i.MX IPUv3 scaler driver
>>   [media] imx-ipu: Add i.MX IPUv3 deinterlacer driver
>>   [media] v4l2: subdev: Add v4l2_device_register_subdev_node function
>>   [media] v4l2: Fix V4L2_CID_PIXEL_RATE
>>   [media] v4l2 async: remove from notifier list
>>   [media] ipuv3-csi: Pass ipucsi to v4l2_media_subdev_s_power
>>   [media] ipuv3-csi: make subdev controls available on video device
>
> Do you have plans to resubmit this series?

Any news about this series?

Thanks

Re: [RFC PATCH 00/26] i.MX5/6 IPUv3 CSI/IC

[Tim Harvey]

On Tue, Dec 1, 2015 at 4:08 AM, Fabio Estevam <festevam@gmail.com> wrote:
> Hi Philipp,
>
> On Tue, Oct 27, 2015 at 11:10 AM, Fabio Estevam <festevam@gmail.com> wrote:
>> Hi Philipp,
>>
>>
>> On Thu, Jun 12, 2014 at 2:06 PM, Philipp Zabel <p.zabel@pengutronix.de> wrote:
>>> Hi,
>>>
>>> attached is a series of our work in progress i.MX6 capture drivers.
>>> I'm posting this now in reaction to Steve's i.MX6 Video capture series,
>>> as a reference for further discussion.
>>> Of the Image Converter (IC) we only use the postprocessor task, with
>>> tiling for larger frames, to implement v4l2 mem2mem scaler/colorspace
>>> converter and deinterlacer devices.
>>> The capture code capture code already uses the media controller framework
>>> and creates a subdevice representing the CSI, but the path to memory is
>>> fixed to IDMAC via SMFC, which is the only possible path for grayscale
>>> and  and anything with multiple output ports connected
>>> to the CSIs (such as the CSI2IPU gasket on i.MX6) doesn't work yet. Also,
>>> I think the CSI subdevice driver should be completely separate from the
>>> capture driver.
>>>
>>> regards
>>> Philipp
>>>
>>> Philipp Zabel (16):
>>>   gpu: ipu-v3: Add function to setup CP channel as interlaced
>>>   gpu: ipu-v3: Add ipu_cpmem_get_buffer function
>>>   gpu: ipu-v3: Add support for partial interleaved YCbCr 4:2:0 (NV12)
>>>     format
>>>   gpu: ipu-v3: Add support for planar YUV 4:2:2 (YUV422P) format
>>>   imx-drm: currently only IPUv3 is supported, make it mandatory
>>>   [media] Add i.MX SoC wide media device driver
>>>   [media] imx-ipu: Add i.MX IPUv3 capture driver
>>>   [media] ipuv3-csi: Skip 3 lines for NTSC BT.656
>>>   [media] imx-ipuv3-csi: Add support for temporarily stopping the stream
>>>     on sync loss
>>>   [media] imx-ipuv3-csi: Export sync lock event to userspace
>>>   [media] v4l2-subdev.h: Add lock status notification
>>>   [media] v4l2-subdev: Export v4l2_subdev_fops
>>>   mfd: syscon: add child device support
>>>   [media] imx: Add video switch
>>>   ARM: dts: Add IPU aliases on i.MX6
>>>   ARM: dts: imx6qdl: Add mipi_ipu1/2 multiplexers, mipi_csi, and their
>>>     connections
>>>
>>> Sascha Hauer (10):
>>>   gpu: ipu-v3: Add IC support
>>>   gpu: ipu-v3: Register IC with IPUv3
>>>   [media] imx-ipu: add ipu media common code
>>>   [media] imx-ipu: Add i.MX IPUv3 scaler driver
>>>   [media] imx-ipu: Add i.MX IPUv3 deinterlacer driver
>>>   [media] v4l2: subdev: Add v4l2_device_register_subdev_node function
>>>   [media] v4l2: Fix V4L2_CID_PIXEL_RATE
>>>   [media] v4l2 async: remove from notifier list
>>>   [media] ipuv3-csi: Pass ipucsi to v4l2_media_subdev_s_power
>>>   [media] ipuv3-csi: make subdev controls available on video device
>>
>> Do you have plans to resubmit this series?
>
> Any news about this series?
>
> Thanks

Philipp / Steve,

I'm also curious as to what your plans are with this work. I have IMX6
boards in mainline (Gateworks Ventana product family) that have both
composite input (adv7180) and HDMI input (tda1997x) via the IMX6 CSI
and am wondering what is left to achieve mainline video capture on the
IMX6. Video capture and crypto are the last things I know of that can
eliminate the common need of a downstream vendor kernel with 100's of
Freescale patches.

Can you point me to the most recent git tree that you were working
with and tell me what devices you were using to capture with?

Regards,

Tim

Tim Harvey - Principal Software Engineer
Gateworks Corporation - http://www.gateworks.com/
3026 S. Higuera St. San Luis Obispo CA 93401
805-781-2000