[RFC 00/12] Adding media device driver for Exynos imaging subsystem

[RFC 00/12] Adding media device driver for Exynos imaging subsystem

[Shaik Ameer Basha]

The following patchset features:

1] Creating a common pipeline framework which can be used by all
Exynos series SoCs for developing media device drivers.
2] Modified the existing fimc-mdevice for exynos4 to use the common
pipeline framework.
3] Adding of media device driver for Exynos5 Imaging subsystem.
4] Upgrading mipi-csis and fimc-lite drivers for Exynos5 SoCs.
5] Adding DT support to m5mols driver and tested with Exynos5 media
device driver.

Current changes are not tested on exynos4 series SoCs. Current media
device driver only support one pipeline (pipeline0) which consists of 
	Sensor --> MIPI-CSIS --> FIMC-LITE
	Sensor --> FIMC-LITE
G-Scaler support to pipeline0 will be added later.

Once the fimc-is device driver is posted, one more pipeline (pipeline1)
will be added for exynos5 media device driver for fimc-is sub-devices.

This patchset is rebased on:
git://linuxtv.org/media_tree.git:staging/for_v3.9

This patchset depends on:
    from Thomas Abraham:
    [1] pinctrl: exynos: add support for Samsung's Exynos5250
    [2] ARM: dts: add pinctrl nodes for Exynos5250 SoC

from Sylwester Nawrocki:
    [1] Device tree support for Exynos SoC camera subsystem

Shaik Ameer Basha (12):
  media: s5p-fimc: modify existing mdev to use common pipeline
  fimc-lite: Adding Exynos5 compatibility to fimc-lite driver
  media: fimc-lite: Adding support for Exynos5
  s5p-csis: Adding Exynos5250 compatibility
  ARM: EXYNOS: Add devicetree node for mipi-csis driver for exynos5
  ARM: EXYNOS: Add devicetree node for FIMC-LITE driver for exynos5
  media: exynos5-is: Adding media device driver for exynos5
  ARM: dts: add camera specific pinctrl nodes for Exynos5250 SoC
  ARM: dts: Adding pinctrl support to Exynos5250 i2c nodes
  ARM: dts: Adding media device nodes to Exynos5 SoCs
  media: m5mols: Adding dt support to m5mols driver
  ARM: dts: Add camera node to exynos5250-smdk5250.dts

 arch/arm/boot/dts/exynos5250-pinctrl.dtsi        |   41 +
 arch/arm/boot/dts/exynos5250-smdk5250.dts        |   65 +-
 arch/arm/boot/dts/exynos5250.dtsi                |   54 +
 arch/arm/mach-exynos/clock-exynos5.c             |   20 +-
 arch/arm/mach-exynos/include/mach/map.h          |    7 +
 arch/arm/mach-exynos/mach-exynos5-dt.c           |   10 +
 drivers/media/i2c/m5mols/m5mols_core.c           |   54 +-
 drivers/media/platform/Kconfig                   |    1 +
 drivers/media/platform/Makefile                  |    1 +
 drivers/media/platform/exynos5-is/Kconfig        |    7 +
 drivers/media/platform/exynos5-is/Makefile       |    4 +
 drivers/media/platform/exynos5-is/exynos5-mdev.c | 1309 ++++++++++++++++++++++
 drivers/media/platform/exynos5-is/exynos5-mdev.h |  107 ++
 drivers/media/platform/s5p-fimc/fimc-capture.c   |   96 +-
 drivers/media/platform/s5p-fimc/fimc-core.h      |    4 +-
 drivers/media/platform/s5p-fimc/fimc-lite-reg.c  |   16 +-
 drivers/media/platform/s5p-fimc/fimc-lite-reg.h  |   41 +-
 drivers/media/platform/s5p-fimc/fimc-lite.c      |  292 ++++-
 drivers/media/platform/s5p-fimc/fimc-lite.h      |   12 +-
 drivers/media/platform/s5p-fimc/fimc-mdevice.c   |  186 ++-
 drivers/media/platform/s5p-fimc/fimc-mdevice.h   |   41 +-
 drivers/media/platform/s5p-fimc/mipi-csis.c      |    1 +
 include/media/s5p_fimc.h                         |   66 +-
 23 files changed, 2261 insertions(+), 174 deletions(-)
 create mode 100644 drivers/media/platform/exynos5-is/Kconfig
 create mode 100644 drivers/media/platform/exynos5-is/Makefile
 create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.c
 create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.h

-- 
1.7.9.5

[RFC 01/12] media: s5p-fimc: modify existing mdev to use common pipeline

[Shaik Ameer Basha]

This patch modifies the current fimc_pipeline to exynos_pipeline,
which can be used across multiple media device drivers.

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 drivers/media/platform/s5p-fimc/fimc-capture.c |   96 +++++++-----
 drivers/media/platform/s5p-fimc/fimc-core.h    |    4 +-
 drivers/media/platform/s5p-fimc/fimc-lite.c    |   73 ++++------
 drivers/media/platform/s5p-fimc/fimc-lite.h    |    4 +-
 drivers/media/platform/s5p-fimc/fimc-mdevice.c |  186 ++++++++++++++++++++++--
 drivers/media/platform/s5p-fimc/fimc-mdevice.h |   41 +++---
 include/media/s5p_fimc.h                       |   66 ++++++---
 7 files changed, 326 insertions(+), 144 deletions(-)

diff --git a/drivers/media/platform/s5p-fimc/fimc-capture.c b/drivers/media/platform/s5p-fimc/fimc-capture.c
index 4cbaf46..106466e 100644
--- a/drivers/media/platform/s5p-fimc/fimc-capture.c
+++ b/drivers/media/platform/s5p-fimc/fimc-capture.c
@@ -27,24 +27,26 @@
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-dma-contig.h>
 
-#include "fimc-mdevice.h"
 #include "fimc-core.h"
 #include "fimc-reg.h"
 
 static int fimc_capture_hw_init(struct fimc_dev *fimc)
 {
 	struct fimc_ctx *ctx = fimc->vid_cap.ctx;
-	struct fimc_pipeline *p = &fimc->pipeline;
+	struct exynos_pipeline *p = &fimc->pipeline;
 	struct fimc_sensor_info *sensor;
 	unsigned long flags;
+	struct v4l2_subdev *sd;
 	int ret = 0;
 
-	if (p->subdevs[IDX_SENSOR] == NULL || ctx == NULL)
+	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+					get_subdev_sensor, p);
+	if (sd == NULL || ctx == NULL)
 		return -ENXIO;
 	if (ctx->s_frame.fmt == NULL)
 		return -EINVAL;
 
-	sensor = v4l2_get_subdev_hostdata(p->subdevs[IDX_SENSOR]);
+	sensor = v4l2_get_subdev_hostdata(sd);
 
 	spin_lock_irqsave(&fimc->slock, flags);
 	fimc_prepare_dma_offset(ctx, &ctx->d_frame);
@@ -118,7 +120,7 @@ static int fimc_capture_state_cleanup(struct fimc_dev *fimc, bool suspend)
 	spin_unlock_irqrestore(&fimc->slock, flags);
 
 	if (streaming)
-		return fimc_pipeline_call(fimc, set_stream,
+		return exynos_pipeline_call(fimc, set_stream,
 					  &fimc->pipeline, 0);
 	else
 		return 0;
@@ -177,13 +179,16 @@ static int fimc_capture_config_update(struct fimc_ctx *ctx)
 
 void fimc_capture_irq_handler(struct fimc_dev *fimc, int deq_buf)
 {
-	struct v4l2_subdev *csis = fimc->pipeline.subdevs[IDX_CSIS];
+	struct v4l2_subdev *csis;
 	struct fimc_vid_cap *cap = &fimc->vid_cap;
 	struct fimc_frame *f = &cap->ctx->d_frame;
 	struct fimc_vid_buffer *v_buf;
 	struct timeval *tv;
 	struct timespec ts;
 
+	csis = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+					get_subdev_csis, &fimc->pipeline);
+
 	if (test_and_clear_bit(ST_CAPT_SHUT, &fimc->state)) {
 		wake_up(&fimc->irq_queue);
 		goto done;
@@ -286,7 +291,7 @@ static int start_streaming(struct vb2_queue *q, unsigned int count)
 		fimc_activate_capture(ctx);
 
 		if (!test_and_set_bit(ST_CAPT_ISP_STREAM, &fimc->state))
-			fimc_pipeline_call(fimc, set_stream,
+			exynos_pipeline_call(fimc, set_stream,
 					   &fimc->pipeline, 1);
 	}
 
@@ -311,7 +316,7 @@ int fimc_capture_suspend(struct fimc_dev *fimc)
 	int ret = fimc_stop_capture(fimc, suspend);
 	if (ret)
 		return ret;
-	return fimc_pipeline_call(fimc, close, &fimc->pipeline);
+	return exynos_pipeline_call(fimc, close, &fimc->pipeline);
 }
 
 static void buffer_queue(struct vb2_buffer *vb);
@@ -327,7 +332,7 @@ int fimc_capture_resume(struct fimc_dev *fimc)
 
 	INIT_LIST_HEAD(&fimc->vid_cap.active_buf_q);
 	vid_cap->buf_index = 0;
-	fimc_pipeline_call(fimc, open, &fimc->pipeline,
+	exynos_pipeline_call(fimc, open, &fimc->pipeline,
 			   &vid_cap->vfd.entity, false);
 	fimc_capture_hw_init(fimc);
 
@@ -447,7 +452,7 @@ static void buffer_queue(struct vb2_buffer *vb)
 		spin_unlock_irqrestore(&fimc->slock, flags);
 
 		if (!test_and_set_bit(ST_CAPT_ISP_STREAM, &fimc->state))
-			fimc_pipeline_call(fimc, set_stream,
+			exynos_pipeline_call(fimc, set_stream,
 					   &fimc->pipeline, 1);
 		return;
 	}
@@ -486,9 +491,12 @@ static struct vb2_ops fimc_capture_qops = {
 int fimc_capture_ctrls_create(struct fimc_dev *fimc)
 {
 	struct fimc_vid_cap *vid_cap = &fimc->vid_cap;
-	struct v4l2_subdev *sensor = fimc->pipeline.subdevs[IDX_SENSOR];
+	struct v4l2_subdev *sensor;
 	int ret;
 
+	sensor = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+					get_subdev_sensor, &fimc->pipeline);
+
 	if (WARN_ON(vid_cap->ctx == NULL))
 		return -ENXIO;
 	if (vid_cap->ctx->ctrls.ready)
@@ -513,7 +521,7 @@ static int fimc_capture_open(struct file *file)
 
 	dbg("pid: %d, state: 0x%lx", task_pid_nr(current), fimc->state);
 
-	fimc_md_graph_lock(fimc);
+	exynos_pipeline_graph_lock(fimc->pipeline_ops, &fimc->pipeline);
 	mutex_lock(&fimc->lock);
 
 	if (fimc_m2m_active(fimc))
@@ -531,7 +539,7 @@ static int fimc_capture_open(struct file *file)
 	}
 
 	if (++fimc->vid_cap.refcnt == 1) {
-		ret = fimc_pipeline_call(fimc, open, &fimc->pipeline,
+		ret = exynos_pipeline_call(fimc, open, &fimc->pipeline,
 					 &fimc->vid_cap.vfd.entity, true);
 
 		if (!ret && !fimc->vid_cap.user_subdev_api)
@@ -549,7 +557,7 @@ static int fimc_capture_open(struct file *file)
 	}
 unlock:
 	mutex_unlock(&fimc->lock);
-	fimc_md_graph_unlock(fimc);
+	exynos_pipeline_graph_unlock(fimc->pipeline_ops, &fimc->pipeline);
 	return ret;
 }
 
@@ -565,7 +573,7 @@ static int fimc_capture_close(struct file *file)
 	if (--fimc->vid_cap.refcnt == 0) {
 		clear_bit(ST_CAPT_BUSY, &fimc->state);
 		fimc_stop_capture(fimc, false);
-		fimc_pipeline_call(fimc, close, &fimc->pipeline);
+		exynos_pipeline_call(fimc, close, &fimc->pipeline);
 		clear_bit(ST_CAPT_SUSPENDED, &fimc->state);
 	}
 
@@ -826,7 +834,7 @@ static int fimc_pipeline_try_format(struct fimc_ctx *ctx,
 				    bool set)
 {
 	struct fimc_dev *fimc = ctx->fimc_dev;
-	struct v4l2_subdev *sd = fimc->pipeline.subdevs[IDX_SENSOR];
+	struct v4l2_subdev *sd;
 	struct v4l2_subdev_format sfmt;
 	struct v4l2_mbus_framefmt *mf = &sfmt.format;
 	struct media_entity *me;
@@ -835,6 +843,9 @@ static int fimc_pipeline_try_format(struct fimc_ctx *ctx,
 	int ret, i = 1;
 	u32 fcc;
 
+	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+					get_subdev_sensor, &fimc->pipeline);
+
 	if (WARN_ON(!sd || !tfmt))
 		return -EINVAL;
 
@@ -968,7 +979,7 @@ static int fimc_cap_try_fmt_mplane(struct file *file, void *fh,
 	struct fimc_fmt *ffmt = NULL;
 	int ret = 0;
 
-	fimc_md_graph_lock(fimc);
+	exynos_pipeline_graph_lock(fimc->pipeline_ops, &fimc->pipeline);
 	mutex_lock(&fimc->lock);
 
 	if (fimc_jpeg_fourcc(pix->pixelformat)) {
@@ -1000,11 +1011,13 @@ static int fimc_cap_try_fmt_mplane(struct file *file, void *fh,
 	fimc_adjust_mplane_format(ffmt, pix->width, pix->height, pix);
 
 	if (ffmt->flags & FMT_FLAGS_COMPRESSED)
-		fimc_get_sensor_frame_desc(fimc->pipeline.subdevs[IDX_SENSOR],
-					pix->plane_fmt, ffmt->memplanes, true);
+		fimc_get_sensor_frame_desc(
+			exynos_pipeline_get_subdev(fimc->pipeline_ops,
+				get_subdev_sensor, &fimc->pipeline),
+			pix->plane_fmt, ffmt->memplanes, true);
 unlock:
 	mutex_unlock(&fimc->lock);
-	fimc_md_graph_unlock(fimc);
+	exynos_pipeline_graph_unlock(fimc->pipeline_ops, &fimc->pipeline);
 
 	return ret;
 }
@@ -1070,9 +1083,10 @@ static int __fimc_capture_set_format(struct fimc_dev *fimc,
 	fimc_adjust_mplane_format(ff->fmt, pix->width, pix->height, pix);
 
 	if (ff->fmt->flags & FMT_FLAGS_COMPRESSED) {
-		ret = fimc_get_sensor_frame_desc(fimc->pipeline.subdevs[IDX_SENSOR],
-					pix->plane_fmt, ff->fmt->memplanes,
-					true);
+		ret = fimc_get_sensor_frame_desc(
+			exynos_pipeline_get_subdev(fimc->pipeline_ops,
+				get_subdev_sensor, &fimc->pipeline),
+			pix->plane_fmt, ff->fmt->memplanes, true);
 		if (ret < 0)
 			return ret;
 	}
@@ -1105,7 +1119,7 @@ static int fimc_cap_s_fmt_mplane(struct file *file, void *priv,
 	struct fimc_dev *fimc = video_drvdata(file);
 	int ret;
 
-	fimc_md_graph_lock(fimc);
+	exynos_pipeline_graph_lock(fimc->pipeline_ops, &fimc->pipeline);
 	mutex_lock(&fimc->lock);
 	/*
 	 * The graph is walked within __fimc_capture_set_format() to set
@@ -1118,7 +1132,7 @@ static int fimc_cap_s_fmt_mplane(struct file *file, void *priv,
 	ret = __fimc_capture_set_format(fimc, f);
 
 	mutex_unlock(&fimc->lock);
-	fimc_md_graph_unlock(fimc);
+	exynos_pipeline_graph_unlock(fimc->pipeline_ops, &fimc->pipeline);
 	return ret;
 }
 
@@ -1126,7 +1140,10 @@ static int fimc_cap_enum_input(struct file *file, void *priv,
 			       struct v4l2_input *i)
 {
 	struct fimc_dev *fimc = video_drvdata(file);
-	struct v4l2_subdev *sd = fimc->pipeline.subdevs[IDX_SENSOR];
+	struct v4l2_subdev *sd;
+
+	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops, get_subdev_sensor,
+							&fimc->pipeline);
 
 	if (i->index != 0)
 		return -EINVAL;
@@ -1205,8 +1222,9 @@ static int fimc_pipeline_validate(struct fimc_dev *fimc)
 		    src_fmt.format.code != sink_fmt.format.code)
 			return -EPIPE;
 
-		if (sd == fimc->pipeline.subdevs[IDX_SENSOR] &&
-		    fimc_user_defined_mbus_fmt(src_fmt.format.code)) {
+		if (sd == exynos_pipeline_get_subdev(fimc->pipeline_ops,
+					get_subdev_sensor, &fimc->pipeline) &&
+			  fimc_user_defined_mbus_fmt(src_fmt.format.code)) {
 			struct v4l2_plane_pix_format plane_fmt[FIMC_MAX_PLANES];
 			struct fimc_frame *frame = &vid_cap->ctx->d_frame;
 			unsigned int i;
@@ -1229,14 +1247,17 @@ static int fimc_cap_streamon(struct file *file, void *priv,
 			     enum v4l2_buf_type type)
 {
 	struct fimc_dev *fimc = video_drvdata(file);
-	struct fimc_pipeline *p = &fimc->pipeline;
-	struct v4l2_subdev *sd = p->subdevs[IDX_SENSOR];
+	struct exynos_pipeline *p = &fimc->pipeline;
+	struct v4l2_subdev *sd;
 	int ret;
 
+	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+						get_subdev_sensor, p);
+
 	if (fimc_capture_active(fimc))
 		return -EBUSY;
 
-	ret = media_entity_pipeline_start(&sd->entity, p->m_pipeline);
+	ret = media_entity_pipeline_start(&sd->entity, &p->m_pipeline);
 	if (ret < 0)
 		return ret;
 
@@ -1254,9 +1275,12 @@ static int fimc_cap_streamoff(struct file *file, void *priv,
 			    enum v4l2_buf_type type)
 {
 	struct fimc_dev *fimc = video_drvdata(file);
-	struct v4l2_subdev *sd = fimc->pipeline.subdevs[IDX_SENSOR];
+	struct v4l2_subdev *sd;
 	int ret;
 
+	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops, get_subdev_sensor,
+							&fimc->pipeline);
+
 	ret = vb2_streamoff(&fimc->vid_cap.vbq, type);
 	if (ret == 0)
 		media_entity_pipeline_stop(&sd->entity);
@@ -1489,18 +1513,13 @@ void fimc_sensor_notify(struct v4l2_subdev *sd, unsigned int notification,
 {
 	struct fimc_sensor_info	*sensor;
 	struct fimc_vid_buffer *buf;
-	struct fimc_md *fmd;
 	struct fimc_dev *fimc;
-	unsigned long flags;
 
 	if (sd == NULL)
 		return;
 
 	sensor = v4l2_get_subdev_hostdata(sd);
-	fmd = entity_to_fimc_mdev(&sd->entity);
-
-	spin_lock_irqsave(&fmd->slock, flags);
-	fimc = sensor ? sensor->host : NULL;
+	fimc = sensor ? (struct fimc_dev *)sensor->host : NULL;
 
 	if (fimc && arg && notification == S5P_FIMC_TX_END_NOTIFY &&
 	    test_bit(ST_CAPT_PEND, &fimc->state)) {
@@ -1515,7 +1534,6 @@ void fimc_sensor_notify(struct v4l2_subdev *sd, unsigned int notification,
 		fimc_deactivate_capture(fimc);
 		spin_unlock_irqrestore(&fimc->slock, irq_flags);
 	}
-	spin_unlock_irqrestore(&fmd->slock, flags);
 }
 
 static int fimc_subdev_enum_mbus_code(struct v4l2_subdev *sd,
diff --git a/drivers/media/platform/s5p-fimc/fimc-core.h b/drivers/media/platform/s5p-fimc/fimc-core.h
index 412d507..57bf708 100644
--- a/drivers/media/platform/s5p-fimc/fimc-core.h
+++ b/drivers/media/platform/s5p-fimc/fimc-core.h
@@ -447,8 +447,8 @@ struct fimc_dev {
 	struct fimc_vid_cap		vid_cap;
 	unsigned long			state;
 	struct vb2_alloc_ctx		*alloc_ctx;
-	struct fimc_pipeline		pipeline;
-	const struct fimc_pipeline_ops	*pipeline_ops;
+	struct exynos_pipeline		pipeline;
+	const struct exynos_pipeline_ops	*pipeline_ops;
 };
 
 /**
diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c b/drivers/media/platform/s5p-fimc/fimc-lite.c
index 3266c3f..122cf95 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite.c
+++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
@@ -11,6 +11,7 @@
 #define pr_fmt(fmt) "%s:%d " fmt, __func__, __LINE__
 
 #include <linux/bug.h>
+#include <linux/clk.h>
 #include <linux/device.h>
 #include <linux/errno.h>
 #include <linux/interrupt.h>
@@ -31,8 +32,6 @@
 #include <media/videobuf2-dma-contig.h>
 #include <media/s5p_fimc.h>
 
-#include "fimc-mdevice.h"
-#include "fimc-core.h"
 #include "fimc-lite.h"
 #include "fimc-lite-reg.h"
 
@@ -123,12 +122,13 @@ static const struct fimc_fmt *fimc_lite_find_format(const u32 *pixelformat,
 
 static int fimc_lite_hw_init(struct fimc_lite *fimc, bool isp_output)
 {
-	struct fimc_pipeline *pipeline = &fimc->pipeline;
 	struct v4l2_subdev *sensor;
-	struct fimc_sensor_info *si;
+	struct fimc_source_info *src_info;
 	unsigned long flags;
 
-	sensor = isp_output ? fimc->sensor : pipeline->subdevs[IDX_SENSOR];
+	sensor = isp_output ? fimc->sensor :
+				exynos_pipeline_get_subdev(fimc->pipeline_ops,
+					get_subdev_sensor, &fimc->pipeline);
 
 	if (sensor == NULL)
 		return -ENXIO;
@@ -137,10 +137,10 @@ static int fimc_lite_hw_init(struct fimc_lite *fimc, bool isp_output)
 		return -EINVAL;
 
 	/* Get sensor configuration data from the sensor subdev */
-	si = v4l2_get_subdev_hostdata(sensor);
+	src_info = v4l2_get_subdev_hostdata(sensor);
 	spin_lock_irqsave(&fimc->slock, flags);
 
-	flite_hw_set_camera_bus(fimc, &si->pdata);
+	flite_hw_set_camera_bus(fimc, src_info);
 	flite_hw_set_source_format(fimc, &fimc->inp_frame);
 	flite_hw_set_window_offset(fimc, &fimc->inp_frame);
 	flite_hw_set_output_dma(fimc, &fimc->out_frame, !isp_output);
@@ -200,7 +200,7 @@ static int fimc_lite_reinit(struct fimc_lite *fimc, bool suspend)
 	if (!streaming)
 		return 0;
 
-	return fimc_pipeline_call(fimc, set_stream, &fimc->pipeline, 0);
+	return exynos_pipeline_call(fimc, set_stream, &fimc->pipeline, 0);
 }
 
 static int fimc_lite_stop_capture(struct fimc_lite *fimc, bool suspend)
@@ -314,7 +314,7 @@ static int start_streaming(struct vb2_queue *q, unsigned int count)
 		flite_hw_capture_start(fimc);
 
 		if (!test_and_set_bit(ST_SENSOR_STREAM, &fimc->state))
-			fimc_pipeline_call(fimc, set_stream,
+			exynos_pipeline_call(fimc, set_stream,
 					   &fimc->pipeline, 1);
 	}
 	if (debug > 0)
@@ -419,7 +419,7 @@ static void buffer_queue(struct vb2_buffer *vb)
 		spin_unlock_irqrestore(&fimc->slock, flags);
 
 		if (!test_and_set_bit(ST_SENSOR_STREAM, &fimc->state))
-			fimc_pipeline_call(fimc, set_stream,
+			exynos_pipeline_call(fimc, set_stream,
 					   &fimc->pipeline, 1);
 		return;
 	}
@@ -482,7 +482,7 @@ static int fimc_lite_open(struct file *file)
 
 	if (++fimc->ref_count == 1 &&
 	    atomic_read(&fimc->out_path) == FIMC_IO_DMA) {
-		ret = fimc_pipeline_call(fimc, open, &fimc->pipeline,
+		ret = exynos_pipeline_call(fimc, open, &fimc->pipeline,
 					 &fimc->vfd.entity, true);
 		if (ret < 0) {
 			pm_runtime_put_sync(&fimc->pdev->dev);
@@ -510,7 +510,7 @@ static int fimc_lite_close(struct file *file)
 	    atomic_read(&fimc->out_path) == FIMC_IO_DMA) {
 		clear_bit(ST_FLITE_IN_USE, &fimc->state);
 		fimc_lite_stop_capture(fimc, false);
-		fimc_pipeline_call(fimc, close, &fimc->pipeline);
+		exynos_pipeline_call(fimc, close, &fimc->pipeline);
 		clear_bit(ST_FLITE_SUSPENDED, &fimc->state);
 	}
 
@@ -801,14 +801,17 @@ static int fimc_lite_streamon(struct file *file, void *priv,
 			      enum v4l2_buf_type type)
 {
 	struct fimc_lite *fimc = video_drvdata(file);
-	struct v4l2_subdev *sensor = fimc->pipeline.subdevs[IDX_SENSOR];
-	struct fimc_pipeline *p = &fimc->pipeline;
+	struct v4l2_subdev *sensor;
+	struct exynos_pipeline *p = &fimc->pipeline;
 	int ret;
 
+	sensor = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+			get_subdev_sensor, &fimc->pipeline);
+
 	if (fimc_lite_active(fimc))
 		return -EBUSY;
 
-	ret = media_entity_pipeline_start(&sensor->entity, p->m_pipeline);
+	ret = media_entity_pipeline_start(&sensor->entity, &p->m_pipeline);
 	if (ret < 0)
 		return ret;
 
@@ -825,9 +828,12 @@ static int fimc_lite_streamoff(struct file *file, void *priv,
 			       enum v4l2_buf_type type)
 {
 	struct fimc_lite *fimc = video_drvdata(file);
-	struct v4l2_subdev *sd = fimc->pipeline.subdevs[IDX_SENSOR];
+	struct v4l2_subdev *sd;
 	int ret;
 
+	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+			get_subdev_sensor, &fimc->pipeline);
+
 	ret = vb2_streamoff(&fimc->vb_queue, type);
 	if (ret == 0)
 		media_entity_pipeline_stop(&sd->entity);
@@ -976,29 +982,6 @@ static const struct v4l2_ioctl_ops fimc_lite_ioctl_ops = {
 	.vidioc_streamoff		= fimc_lite_streamoff,
 };
 
-/* Called with the media graph mutex held */
-static struct v4l2_subdev *__find_remote_sensor(struct media_entity *me)
-{
-	struct media_pad *pad = &me->pads[0];
-	struct v4l2_subdev *sd;
-
-	while (pad->flags & MEDIA_PAD_FL_SINK) {
-		/* source pad */
-		pad = media_entity_remote_source(pad);
-		if (pad == NULL ||
-		    media_entity_type(pad->entity) != MEDIA_ENT_T_V4L2_SUBDEV)
-			break;
-
-		sd = media_entity_to_v4l2_subdev(pad->entity);
-
-		if (sd->grp_id == GRP_ID_FIMC_IS_SENSOR)
-			return sd;
-		/* sink pad */
-		pad = &sd->entity.pads[0];
-	}
-	return NULL;
-}
-
 /* Capture subdev media entity operations */
 static int fimc_lite_link_setup(struct media_entity *entity,
 				const struct media_pad *local,
@@ -1241,6 +1224,8 @@ static int fimc_lite_subdev_set_selection(struct v4l2_subdev *sd,
 static int fimc_lite_subdev_s_stream(struct v4l2_subdev *sd, int on)
 {
 	struct fimc_lite *fimc = v4l2_get_subdevdata(sd);
+	struct exynos_pipeline *ep;
+	struct exynos_pipeline_ops *ep_ops;
 	unsigned long flags;
 	int ret;
 
@@ -1251,7 +1236,10 @@ static int fimc_lite_subdev_s_stream(struct v4l2_subdev *sd, int on)
 	 * The pipeline links are protected through entity.stream_count
 	 * so there is no need to take the media graph mutex here.
 	 */
-	fimc->sensor = __find_remote_sensor(&sd->entity);
+	ep_ops = v4l2_get_subdev_hostdata(sd);
+	ep = media_pipe_to_exynos_pipeline(sd->entity.pipe);
+	fimc->sensor = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+							get_subdev_sensor, ep);
 
 	if (atomic_read(&fimc->out_path) != FIMC_IO_ISP)
 		return -ENOIOCTLCMD;
@@ -1338,6 +1326,7 @@ static int fimc_lite_subdev_registered(struct v4l2_subdev *sd)
 
 	video_set_drvdata(vfd, fimc);
 	fimc->pipeline_ops = v4l2_get_subdev_hostdata(sd);
+	fimc->pipeline_ops->init(&fimc->pipeline);
 
 	ret = video_register_device(vfd, VFL_TYPE_GRABBER, -1);
 	if (ret < 0) {
@@ -1606,7 +1595,7 @@ static int fimc_lite_resume(struct device *dev)
 		return 0;
 
 	INIT_LIST_HEAD(&fimc->active_buf_q);
-	fimc_pipeline_call(fimc, open, &fimc->pipeline,
+	exynos_pipeline_call(fimc, open, &fimc->pipeline,
 			   &fimc->vfd.entity, false);
 	fimc_lite_hw_init(fimc, atomic_read(&fimc->out_path) == FIMC_IO_ISP);
 	clear_bit(ST_FLITE_SUSPENDED, &fimc->state);
@@ -1633,7 +1622,7 @@ static int fimc_lite_suspend(struct device *dev)
 	if (ret < 0 || !fimc_lite_active(fimc))
 		return ret;
 
-	return fimc_pipeline_call(fimc, close, &fimc->pipeline);
+	return exynos_pipeline_call(fimc, close, &fimc->pipeline);
 }
 #endif /* CONFIG_PM_SLEEP */
 
diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.h b/drivers/media/platform/s5p-fimc/fimc-lite.h
index 7085761..66d6eeb 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite.h
+++ b/drivers/media/platform/s5p-fimc/fimc-lite.h
@@ -145,8 +145,8 @@ struct fimc_lite {
 	struct v4l2_ctrl_handler ctrl_handler;
 	struct v4l2_ctrl	*test_pattern;
 	u32			index;
-	struct fimc_pipeline	pipeline;
-	const struct fimc_pipeline_ops *pipeline_ops;
+	struct exynos_pipeline	pipeline;
+	const struct exynos_pipeline_ops *pipeline_ops;
 
 	struct mutex		lock;
 	spinlock_t		slock;
diff --git a/drivers/media/platform/s5p-fimc/fimc-mdevice.c b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
index fc93fad..938cc56 100644
--- a/drivers/media/platform/s5p-fimc/fimc-mdevice.c
+++ b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
@@ -36,6 +36,8 @@
 #include "fimc-mdevice.h"
 #include "mipi-csis.h"
 
+static struct fimc_md *g_fimc_mdev;
+
 static int __fimc_md_set_camclk(struct fimc_md *fmd,
 				struct fimc_sensor_info *s_info,
 				bool on);
@@ -143,6 +145,73 @@ static int fimc_pipeline_s_power(struct fimc_pipeline *p, bool state)
 }
 
 /**
+ * __fimc_pipeline_init
+ *      allocate the fimc_pipeline structure and do the basic initialization
+ */
+static int __fimc_pipeline_init(struct exynos_pipeline *ep)
+{
+	struct fimc_pipeline *p;
+
+	p = kzalloc(sizeof(*p), GFP_KERNEL);
+	if (!p)
+		return -ENOMEM;
+
+	p->is_init = true;
+	p->fmd = g_fimc_mdev;
+	ep->priv = (void *)p;
+	return 0;
+}
+
+/**
+ * __fimc_pipeline_deinit
+ *      free the allocated resources for fimc_pipeline
+ */
+static int __fimc_pipeline_deinit(struct exynos_pipeline *ep)
+{
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
+
+	if (!p || !p->is_init)
+		return -EINVAL;
+
+	p->is_init = false;
+	kfree(p);
+
+	return 0;
+}
+
+/**
+ * __fimc_pipeline_get_subdev_sensor
+ *      if valid pipeline, returns the sensor subdev pointer
+ *      else returns NULL
+ */
+static struct v4l2_subdev *__fimc_pipeline_get_subdev_sensor(
+					struct exynos_pipeline *ep)
+{
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
+
+	if (!p || !p->is_init)
+		return NULL;
+
+	return p->subdevs[IDX_SENSOR];
+}
+
+/**
+ * __fimc_pipeline_get_subdev_csis
+ *      if valid pipeline, returns the csis subdev pointer
+ *      else returns NULL
+ */
+static struct v4l2_subdev *__fimc_pipeline_get_subdev_csis(
+					struct exynos_pipeline *ep)
+{
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
+
+	if (!p || !p->is_init)
+		return NULL;
+
+	return p->subdevs[IDX_CSIS];
+}
+
+/**
  * __fimc_pipeline_open - update the pipeline information, enable power
  *                        of all pipeline subdevs and the sensor clock
  * @me: media entity to start graph walk with
@@ -150,11 +219,15 @@ static int fimc_pipeline_s_power(struct fimc_pipeline *p, bool state)
  *
  * Called with the graph mutex held.
  */
-static int __fimc_pipeline_open(struct fimc_pipeline *p,
+static int __fimc_pipeline_open(struct exynos_pipeline *ep,
 				struct media_entity *me, bool prep)
 {
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
 	int ret;
 
+	if (!p || !p->is_init)
+		return -EINVAL;
+
 	if (prep)
 		fimc_pipeline_prepare(p, me);
 
@@ -174,17 +247,20 @@ static int __fimc_pipeline_open(struct fimc_pipeline *p,
  *
  * Disable power of all subdevs and turn the external sensor clock off.
  */
-static int __fimc_pipeline_close(struct fimc_pipeline *p)
+static int __fimc_pipeline_close(struct exynos_pipeline *ep)
 {
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
 	int ret = 0;
 
-	if (!p || !p->subdevs[IDX_SENSOR])
+	if (!p || !p->is_init)
 		return -EINVAL;
 
-	if (p->subdevs[IDX_SENSOR]) {
-		ret = fimc_pipeline_s_power(p, 0);
-		fimc_md_set_camclk(p->subdevs[IDX_SENSOR], false);
-	}
+	if (!p->subdevs[IDX_SENSOR])
+		return -EINVAL;
+
+	ret = fimc_pipeline_s_power(p, 0);
+	fimc_md_set_camclk(p->subdevs[IDX_SENSOR], false);
+
 	return ret == -ENXIO ? 0 : ret;
 }
 
@@ -193,10 +269,14 @@ static int __fimc_pipeline_close(struct fimc_pipeline *p)
  * @pipeline: video pipeline structure
  * @on: passed as the s_stream call argument
  */
-static int __fimc_pipeline_s_stream(struct fimc_pipeline *p, bool on)
+static int __fimc_pipeline_s_stream(struct exynos_pipeline *ep, bool on)
 {
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
 	int i, ret;
 
+	if (!p || !p->is_init)
+		return -EINVAL;
+
 	if (p->subdevs[IDX_SENSOR] == NULL)
 		return -ENODEV;
 
@@ -213,11 +293,47 @@ static int __fimc_pipeline_s_stream(struct fimc_pipeline *p, bool on)
 
 }
 
+static void __fimc_pipeline_graph_lock(struct exynos_pipeline *ep)
+{
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
+	struct fimc_md *fmd = p->fmd;
+
+	mutex_lock(&fmd->media_dev.graph_mutex);
+}
+
+static void __fimc_pipeline_graph_unlock(struct exynos_pipeline *ep)
+{
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
+	struct fimc_md *fmd = p->fmd;
+
+	mutex_unlock(&fmd->media_dev.graph_mutex);
+}
+
+static void __fimc_pipeline_register_notify_callback(
+		struct exynos_pipeline *ep,
+		void (*notify_cb)(struct v4l2_subdev *sd,
+				unsigned int notification, void *arg))
+{
+	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
+
+	if (!notify_cb)
+		return;
+
+	p->sensor_notify = notify_cb;
+}
+
 /* Media pipeline operations for the FIMC/FIMC-LITE video device driver */
-static const struct fimc_pipeline_ops fimc_pipeline_ops = {
-	.open		= __fimc_pipeline_open,
-	.close		= __fimc_pipeline_close,
-	.set_stream	= __fimc_pipeline_s_stream,
+static const struct exynos_pipeline_ops fimc_pipeline_ops = {
+	.init			= __fimc_pipeline_init,
+	.deinit			= __fimc_pipeline_deinit,
+	.open			= __fimc_pipeline_open,
+	.close			= __fimc_pipeline_close,
+	.set_stream		= __fimc_pipeline_s_stream,
+	.get_subdev_sensor	= __fimc_pipeline_get_subdev_sensor,
+	.get_subdev_csis	= __fimc_pipeline_get_subdev_csis,
+	.graph_lock		= __fimc_pipeline_graph_lock,
+	.graph_unlock		= __fimc_pipeline_graph_unlock,
+	.register_notify_cb	= __fimc_pipeline_register_notify_callback,
 };
 
 /*
@@ -769,7 +885,7 @@ static int __fimc_md_create_fimc_sink_links(struct fimc_md *fmd,
 		if (!WARN_ON(s_info == NULL)) {
 			unsigned long irq_flags;
 			spin_lock_irqsave(&fmd->slock, irq_flags);
-			s_info->host = fmd->fimc[i];
+			s_info->host = (void *)fmd->fimc[i];
 			spin_unlock_irqrestore(&fmd->slock, irq_flags);
 		}
 	}
@@ -1051,7 +1167,8 @@ static int fimc_md_link_notify(struct media_pad *source,
 {
 	struct fimc_lite *fimc_lite = NULL;
 	struct fimc_dev *fimc = NULL;
-	struct fimc_pipeline *pipeline;
+	struct exynos_pipeline *pipeline;
+	struct fimc_pipeline *p;
 	struct v4l2_subdev *sd;
 	struct mutex *lock;
 	int ret = 0;
@@ -1081,12 +1198,16 @@ static int fimc_md_link_notify(struct media_pad *source,
 		return 0;
 	}
 
+	p = (struct fimc_pipeline *)pipeline->priv;
+	if (!p || !p->is_init)
+		return -EINVAL;
+
 	if (!(flags & MEDIA_LNK_FL_ENABLED)) {
 		int i;
 		mutex_lock(lock);
 		ret = __fimc_pipeline_close(pipeline);
 		for (i = 0; i < IDX_MAX; i++)
-			pipeline->subdevs[i] = NULL;
+			p->subdevs[i] = NULL;
 		if (fimc)
 			fimc_ctrls_delete(fimc->vid_cap.ctx);
 		mutex_unlock(lock);
@@ -1154,6 +1275,37 @@ static ssize_t fimc_md_sysfs_store(struct device *dev,
 static DEVICE_ATTR(subdev_conf_mode, S_IWUSR | S_IRUGO,
 		   fimc_md_sysfs_show, fimc_md_sysfs_store);
 
+/**
+ * fimc_md_sensor_notify - v4l2_device notification from a sensor subdev
+ * @sd: pointer to a subdev generating the notification
+ * @notification: the notification type, must be S5P_FIMC_TX_END_NOTIFY
+ * @arg: pointer to an u32 type integer that stores the frame payload value
+ *
+ * Passes the sensor notification to the capture device
+ */
+void fimc_md_sensor_notify(struct v4l2_subdev *sd, unsigned int notification,
+			void *arg)
+{
+	struct fimc_md *fmd;
+	struct exynos_pipeline *ep;
+	struct fimc_pipeline *p;
+	unsigned long flags;
+
+	if (sd == NULL)
+		return;
+
+	ep = media_pipe_to_exynos_pipeline(sd->entity.pipe);
+	p = (struct fimc_pipeline *)ep->priv;
+
+	spin_lock_irqsave(&fmd->slock, flags);
+
+	if (p->sensor_notify)
+		p->sensor_notify(sd, notification, arg);
+
+	spin_unlock_irqrestore(&fmd->slock, flags);
+}
+
+
 static int fimc_md_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -1175,7 +1327,7 @@ static int fimc_md_probe(struct platform_device *pdev)
 
 	v4l2_dev = &fmd->v4l2_dev;
 	v4l2_dev->mdev = &fmd->media_dev;
-	v4l2_dev->notify = fimc_sensor_notify;
+	v4l2_dev->notify = fimc_md_sensor_notify;
 	strlcpy(v4l2_dev->name, "s5p-fimc-md", sizeof(v4l2_dev->name));
 
 
@@ -1194,6 +1346,7 @@ static int fimc_md_probe(struct platform_device *pdev)
 		goto err_clk;
 
 	fmd->user_subdev_api = (dev->of_node != NULL);
+	g_fimc_mdev = fmd;
 
 	/* Protect the media graph while we're registering entities */
 	mutex_lock(&fmd->media_dev.graph_mutex);
@@ -1252,6 +1405,7 @@ static int fimc_md_remove(struct platform_device *pdev)
 
 	if (!fmd)
 		return 0;
+	g_fimc_mdev = NULL;
 	device_remove_file(&pdev->dev, &dev_attr_subdev_conf_mode);
 	fimc_md_unregister_entities(fmd);
 	media_device_unregister(&fmd->media_dev);
diff --git a/drivers/media/platform/s5p-fimc/fimc-mdevice.h b/drivers/media/platform/s5p-fimc/fimc-mdevice.h
index f3e0251..1ea7acf 100644
--- a/drivers/media/platform/s5p-fimc/fimc-mdevice.h
+++ b/drivers/media/platform/s5p-fimc/fimc-mdevice.h
@@ -37,6 +37,15 @@
 #define FIMC_MAX_SENSORS	8
 #define FIMC_MAX_CAMCLKS	2
 
+enum fimc_subdev_index {
+	IDX_SENSOR,
+	IDX_CSIS,
+	IDX_FLITE,
+	IDX_IS_ISP,
+	IDX_FIMC,
+	IDX_MAX,
+};
+
 struct fimc_csis_info {
 	struct v4l2_subdev *sd;
 	int id;
@@ -49,20 +58,6 @@ struct fimc_camclk_info {
 };
 
 /**
- * struct fimc_sensor_info - image data source subdev information
- * @pdata: sensor's atrributes passed as media device's platform data
- * @subdev: image sensor v4l2 subdev
- * @host: fimc device the sensor is currently linked to
- *
- * This data structure applies to image sensor and the writeback subdevs.
- */
-struct fimc_sensor_info {
-	struct fimc_source_info pdata;
-	struct v4l2_subdev *subdev;
-	struct fimc_dev *host;
-};
-
-/**
  * struct fimc_md - fimc media device information
  * @csis: MIPI CSIS subdevs data
  * @sensor: array of registered sensor subdevs
@@ -89,6 +84,14 @@ struct fimc_md {
 	spinlock_t slock;
 };
 
+struct fimc_pipeline {
+	int is_init;
+	struct fimc_md *fmd;
+	struct v4l2_subdev *subdevs[IDX_MAX];
+	void (*sensor_notify)(struct v4l2_subdev *sd,
+			unsigned int notification, void *arg);
+};
+
 #define is_subdev_pad(pad) (pad == NULL || \
 	media_entity_type(pad->entity) == MEDIA_ENT_T_V4L2_SUBDEV)
 
@@ -103,16 +106,6 @@ static inline struct fimc_md *entity_to_fimc_mdev(struct media_entity *me)
 		container_of(me->parent, struct fimc_md, media_dev);
 }
 
-static inline void fimc_md_graph_lock(struct fimc_dev *fimc)
-{
-	mutex_lock(&fimc->vid_cap.vfd.entity.parent->graph_mutex);
-}
-
-static inline void fimc_md_graph_unlock(struct fimc_dev *fimc)
-{
-	mutex_unlock(&fimc->vid_cap.vfd.entity.parent->graph_mutex);
-}
-
 int fimc_md_set_camclk(struct v4l2_subdev *sd, bool on);
 
 #endif
diff --git a/include/media/s5p_fimc.h b/include/media/s5p_fimc.h
index e2434bb..007e998 100644
--- a/include/media/s5p_fimc.h
+++ b/include/media/s5p_fimc.h
@@ -13,6 +13,7 @@
 #define S5P_FIMC_H_
 
 #include <media/media-entity.h>
+#include <media/v4l2-subdev.h>
 
 /*
  * Enumeration of data inputs to the camera subsystem.
@@ -75,6 +76,20 @@ struct fimc_source_info {
 };
 
 /**
+ * struct fimc_sensor_info - image data source subdev information
+ * @pdata: sensor's atrributes passed as media device's platform data
+ * @subdev: image sensor v4l2 subdev
+ * @host: capture device the sensor is currently linked to
+ *
+ * This data structure applies to image sensor and the writeback subdevs.
+ */
+struct fimc_sensor_info {
+	struct fimc_source_info pdata;
+	struct v4l2_subdev *subdev;
+	void *host;
+};
+
+/**
  * struct s5p_platform_fimc - camera host interface platform data
  *
  * @source_info: properties of an image source for the host interface setup
@@ -93,21 +108,10 @@ struct s5p_platform_fimc {
  */
 #define S5P_FIMC_TX_END_NOTIFY _IO('e', 0)
 
-enum fimc_subdev_index {
-	IDX_SENSOR,
-	IDX_CSIS,
-	IDX_FLITE,
-	IDX_IS_ISP,
-	IDX_FIMC,
-	IDX_MAX,
-};
-
-struct media_pipeline;
-struct v4l2_subdev;
 
-struct fimc_pipeline {
-	struct v4l2_subdev *subdevs[IDX_MAX];
-	struct media_pipeline *m_pipeline;
+struct exynos_pipeline {
+	struct media_pipeline m_pipeline;
+	void *priv;
 };
 
 /*
@@ -115,15 +119,39 @@ struct fimc_pipeline {
  * video node when it is the last entity of the pipeline. Implemented
  * by corresponding media device driver.
  */
-struct fimc_pipeline_ops {
-	int (*open)(struct fimc_pipeline *p, struct media_entity *me,
+struct exynos_pipeline_ops {
+	int (*init) (struct exynos_pipeline *p);
+	int (*deinit) (struct exynos_pipeline *p);
+	int (*open)(struct exynos_pipeline *p, struct media_entity *me,
 			  bool resume);
-	int (*close)(struct fimc_pipeline *p);
-	int (*set_stream)(struct fimc_pipeline *p, bool state);
+	int (*close)(struct exynos_pipeline *p);
+	int (*set_stream)(struct exynos_pipeline *p, bool state);
+	void (*graph_lock)(struct exynos_pipeline *p);
+	void (*graph_unlock)(struct exynos_pipeline *p);
+	struct v4l2_subdev *(*get_subdev_sensor)(struct exynos_pipeline *p);
+	struct v4l2_subdev *(*get_subdev_csis)(struct exynos_pipeline *p);
+	void (*register_notify_cb)(struct exynos_pipeline *p,
+		void (*cb)(struct v4l2_subdev *sd,
+				unsigned int notification, void *arg));
+
 };
 
-#define fimc_pipeline_call(f, op, p, args...)				\
+#define exynos_pipeline_call(f, op, p, args...)				\
 	(!(f) ? -ENODEV : (((f)->pipeline_ops && (f)->pipeline_ops->op) ? \
 			    (f)->pipeline_ops->op((p), ##args) : -ENOIOCTLCMD))
 
+#define exynos_pipeline_get_subdev(ops, op, p)				\
+	((ops && ops->op) ? ops->op(p) : NULL)
+
+#define exynos_pipeline_graph_lock(ops, p)				\
+	((ops && ops->graph_lock) ?					\
+		   ops->graph_lock(p) : -ENOIOCTLCMD)
+
+#define exynos_pipeline_graph_unlock(ops, p)				\
+	((ops && ops->graph_unlock) ?					\
+		   ops->graph_unlock(p) : -ENOIOCTLCMD)
+
+#define media_pipe_to_exynos_pipeline(mp) \
+	container_of(mp, struct exynos_pipeline, m_pipeline)
+
 #endif /* S5P_FIMC_H_ */
-- 
1.7.9.5

[RFC 02/12] fimc-lite: Adding Exynos5 compatibility to fimc-lite driver

[Shaik Ameer Basha]

This patch adds the Exynos5 soc compatibility to the fimc-lite driver.
It also adds a version checking to deal with the changes between
different fimc-lite hardware versions.

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 drivers/media/platform/s5p-fimc/fimc-lite.c |   23 +++++++++++++++++++++++
 drivers/media/platform/s5p-fimc/fimc-lite.h |    7 ++++++-
 2 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c b/drivers/media/platform/s5p-fimc/fimc-lite.c
index 122cf95..eb64f87 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite.c
+++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
@@ -1653,6 +1653,16 @@ static struct flite_variant fimc_lite0_variant_exynos4 = {
 	.out_width_align	= 8,
 	.win_hor_offs_align	= 2,
 	.out_hor_offs_align	= 8,
+	.version		= FLITE_VER_EXYNOS4,
+};
+
+static struct flite_variant fimc_lite0_variant_exynos5 = {
+	.max_width		= 8192,
+	.max_height		= 8192,
+	.out_width_align	= 8,
+	.win_hor_offs_align	= 2,
+	.out_hor_offs_align	= 8,
+	.version		= FLITE_VER_EXYNOS5,
 };
 
 /* EXYNOS4212, EXYNOS4412 */
@@ -1663,6 +1673,15 @@ static struct flite_drvdata fimc_lite_drvdata_exynos4 = {
 	},
 };
 
+/* EXYNOS5250 */
+static struct flite_drvdata fimc_lite_drvdata_exynos5 = {
+	.variant = {
+		[0] = &fimc_lite0_variant_exynos5,
+		[1] = &fimc_lite0_variant_exynos5,
+		[2] = &fimc_lite0_variant_exynos5,
+	},
+};
+
 static struct platform_device_id fimc_lite_driver_ids[] = {
 	{
 		.name		= "exynos-fimc-lite",
@@ -1677,6 +1696,10 @@ static const struct of_device_id flite_of_match[] = {
 		.compatible = "samsung,exynos4212-fimc-lite",
 		.data = &fimc_lite_drvdata_exynos4,
 	},
+	{
+		.compatible = "samsung,exynos5250-fimc-lite",
+		.data = &fimc_lite_drvdata_exynos5,
+	},
 	{ /* sentinel */ },
 };
 MODULE_DEVICE_TABLE(of, flite_of_match);
diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.h b/drivers/media/platform/s5p-fimc/fimc-lite.h
index 66d6eeb..ef43fe0 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite.h
+++ b/drivers/media/platform/s5p-fimc/fimc-lite.h
@@ -28,7 +28,7 @@
 
 #define FIMC_LITE_DRV_NAME	"exynos-fimc-lite"
 #define FLITE_CLK_NAME		"flite"
-#define FIMC_LITE_MAX_DEVS	2
+#define FIMC_LITE_MAX_DEVS	3
 #define FLITE_REQ_BUFS_MIN	2
 
 /* Bit index definitions for struct fimc_lite::state */
@@ -49,12 +49,17 @@ enum {
 #define FLITE_SD_PAD_SOURCE_ISP	2
 #define FLITE_SD_PADS_NUM	3
 
+#define FLITE_VER_EXYNOS4	0
+#define FLITE_VER_EXYNOS5	1
+
+
 struct flite_variant {
 	unsigned short max_width;
 	unsigned short max_height;
 	unsigned short out_width_align;
 	unsigned short win_hor_offs_align;
 	unsigned short out_hor_offs_align;
+	unsigned short version;
 };
 
 struct flite_drvdata {
-- 
1.7.9.5

[RFC 03/12] media: fimc-lite: Adding support for Exynos5

[Shaik Ameer Basha]

This patch adds the following functionalities to existing driver

1] FIMC-LITE supports multiple DMA shadow registers from Exynos5 onwards.
This patch adds the functionality of using shadow registers by
checking the current FIMC-LITE hardware version.
2] Fixes Buffer corruption on DMA output from fimc-lite
3] Modified the driver to be used as pipeline endpoint

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
Signed-off-by: Arun Kumar K <arun.kk@samsung.com>
---
 drivers/media/platform/s5p-fimc/fimc-lite-reg.c |   16 +-
 drivers/media/platform/s5p-fimc/fimc-lite-reg.h |   41 ++++-
 drivers/media/platform/s5p-fimc/fimc-lite.c     |  196 +++++++++++++++++++++--
 drivers/media/platform/s5p-fimc/fimc-lite.h     |    3 +-
 4 files changed, 236 insertions(+), 20 deletions(-)

diff --git a/drivers/media/platform/s5p-fimc/fimc-lite-reg.c b/drivers/media/platform/s5p-fimc/fimc-lite-reg.c
index 3c7dd65..3d63526 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite-reg.c
+++ b/drivers/media/platform/s5p-fimc/fimc-lite-reg.c
@@ -68,7 +68,8 @@ void flite_hw_set_interrupt_mask(struct fimc_lite *dev)
 	if (atomic_read(&dev->out_path) == FIMC_IO_DMA) {
 		intsrc = FLITE_REG_CIGCTRL_IRQ_OVFEN |
 			 FLITE_REG_CIGCTRL_IRQ_LASTEN |
-			 FLITE_REG_CIGCTRL_IRQ_STARTEN;
+			 FLITE_REG_CIGCTRL_IRQ_STARTEN |
+			 FLITE_REG_CIGCTRL_IRQ_ENDEN;
 	} else {
 		/* An output to the FIMC-IS */
 		intsrc = FLITE_REG_CIGCTRL_IRQ_OVFEN |
@@ -215,6 +216,18 @@ void flite_hw_set_camera_bus(struct fimc_lite *dev,
 	flite_hw_set_camera_port(dev, si->mux_id);
 }
 
+static void flite_hw_set_pack12(struct fimc_lite *dev, int on)
+{
+	u32 cfg = readl(dev->regs + FLITE_REG_CIODMAFMT);
+
+	cfg &= ~FLITE_REG_CIODMAFMT_PACK12;
+
+	if (on)
+		cfg |= FLITE_REG_CIODMAFMT_PACK12;
+
+	writel(cfg, dev->regs + FLITE_REG_CIODMAFMT);
+}
+
 static void flite_hw_set_out_order(struct fimc_lite *dev, struct flite_frame *f)
 {
 	static const u32 pixcode[4][2] = {
@@ -267,6 +280,7 @@ void flite_hw_set_output_dma(struct fimc_lite *dev, struct flite_frame *f,
 
 	flite_hw_set_out_order(dev, f);
 	flite_hw_set_dma_window(dev, f);
+	flite_hw_set_pack12(dev, 0);
 }
 
 void flite_hw_dump_regs(struct fimc_lite *dev, const char *label)
diff --git a/drivers/media/platform/s5p-fimc/fimc-lite-reg.h b/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
index 0e34584..716df6c 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
+++ b/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
@@ -120,6 +120,10 @@
 /* b0: 1 - camera B, 0 - camera A */
 #define FLITE_REG_CIGENERAL_CAM_B		(1 << 0)
 
+
+#define FLITE_REG_CIFCNTSEQ			0x100
+#define FLITE_REG_CIOSAN(x)			(0x200 + (4 * (x)))
+
 /* ----------------------------------------------------------------------------
  * Function declarations
  */
@@ -143,8 +147,41 @@ void flite_hw_set_dma_window(struct fimc_lite *dev, struct flite_frame *f);
 void flite_hw_set_test_pattern(struct fimc_lite *dev, bool on);
 void flite_hw_dump_regs(struct fimc_lite *dev, const char *label);
 
-static inline void flite_hw_set_output_addr(struct fimc_lite *dev, u32 paddr)
+static inline void flite_hw_set_output_addr(struct fimc_lite *dev,
+	u32 paddr, u32 index)
+{
+	u32 config;
+
+	/* FLITE in EXYNOS4 has only one DMA register */
+	if (dev->variant->version == FLITE_VER_EXYNOS4)
+		index = 0;
+
+	config = readl(dev->regs + FLITE_REG_CIFCNTSEQ);
+	config |= 1 << index;
+	writel(config, dev->regs + FLITE_REG_CIFCNTSEQ);
+
+	if (index == 0)
+		writel(paddr, dev->regs + FLITE_REG_CIOSA);
+	else
+		writel(paddr, dev->regs + FLITE_REG_CIOSAN(index-1));
+}
+
+static inline void flite_hw_clear_output_addr(struct fimc_lite *dev, u32 index)
 {
-	writel(paddr, dev->regs + FLITE_REG_CIOSA);
+	u32 config;
+
+	/* FLITE in EXYNOS4 has only one DMA register */
+	if (dev->variant->version == FLITE_VER_EXYNOS4)
+		index = 0;
+
+	config = readl(dev->regs + FLITE_REG_CIFCNTSEQ);
+	config &= ~(1 << index);
+	writel(config, dev->regs + FLITE_REG_CIFCNTSEQ);
 }
+
+static inline void flite_hw_clear_output_index(struct fimc_lite *dev)
+{
+	writel(0, dev->regs + FLITE_REG_CIFCNTSEQ);
+}
+
 #endif /* FIMC_LITE_REG_H */
diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c b/drivers/media/platform/s5p-fimc/fimc-lite.c
index eb64f87..1edc5ce 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite.c
+++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
@@ -136,6 +136,8 @@ static int fimc_lite_hw_init(struct fimc_lite *fimc, bool isp_output)
 	if (fimc->fmt == NULL)
 		return -EINVAL;
 
+	flite_hw_clear_output_index(fimc);
+
 	/* Get sensor configuration data from the sensor subdev */
 	src_info = v4l2_get_subdev_hostdata(sensor);
 	spin_lock_irqsave(&fimc->slock, flags);
@@ -266,19 +268,24 @@ static irqreturn_t flite_irq_handler(int irq, void *priv)
 
 	if ((intsrc & FLITE_REG_CISTATUS_IRQ_SRC_FRMSTART) &&
 	    test_bit(ST_FLITE_RUN, &fimc->state) &&
-	    !list_empty(&fimc->active_buf_q) &&
 	    !list_empty(&fimc->pending_buf_q)) {
+		vbuf = fimc_lite_pending_queue_pop(fimc);
+		flite_hw_set_output_addr(fimc, vbuf->paddr,
+					vbuf->vb.v4l2_buf.index);
+		fimc_lite_active_queue_add(fimc, vbuf);
+	}
+
+	if ((intsrc & FLITE_REG_CISTATUS_IRQ_SRC_FRMEND) &&
+	    test_bit(ST_FLITE_RUN, &fimc->state) &&
+	    !list_empty(&fimc->active_buf_q)) {
 		vbuf = fimc_lite_active_queue_pop(fimc);
 		ktime_get_ts(&ts);
 		tv = &vbuf->vb.v4l2_buf.timestamp;
 		tv->tv_sec = ts.tv_sec;
 		tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
 		vbuf->vb.v4l2_buf.sequence = fimc->frame_count++;
+		flite_hw_clear_output_addr(fimc, vbuf->vb.v4l2_buf.index);
 		vb2_buffer_done(&vbuf->vb, VB2_BUF_STATE_DONE);
-
-		vbuf = fimc_lite_pending_queue_pop(fimc);
-		flite_hw_set_output_addr(fimc, vbuf->paddr);
-		fimc_lite_active_queue_add(fimc, vbuf);
 	}
 
 	if (test_bit(ST_FLITE_CONFIG, &fimc->state))
@@ -406,7 +413,8 @@ static void buffer_queue(struct vb2_buffer *vb)
 	if (!test_bit(ST_FLITE_SUSPENDED, &fimc->state) &&
 	    !test_bit(ST_FLITE_STREAM, &fimc->state) &&
 	    list_empty(&fimc->active_buf_q)) {
-		flite_hw_set_output_addr(fimc, buf->paddr);
+		flite_hw_set_output_addr(fimc, buf->paddr,
+					buf->vb.v4l2_buf.index);
 		fimc_lite_active_queue_add(fimc, buf);
 	} else {
 		fimc_lite_pending_queue_add(fimc, buf);
@@ -646,7 +654,7 @@ static int fimc_vidioc_querycap_capture(struct file *file, void *priv,
 	strlcpy(cap->driver, FIMC_LITE_DRV_NAME, sizeof(cap->driver));
 	cap->bus_info[0] = 0;
 	cap->card[0] = 0;
-	cap->capabilities = V4L2_CAP_STREAMING;
+	cap->capabilities = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE_MPLANE;
 	return 0;
 }
 
@@ -725,13 +733,125 @@ static int fimc_lite_try_fmt_mplane(struct file *file, void *fh,
 	return fimc_lite_try_fmt(fimc, &f->fmt.pix_mp, NULL);
 }
 
-static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
-				  struct v4l2_format *f)
+static struct media_entity *fimc_pipeline_get_head(struct media_entity *me)
 {
-	struct v4l2_pix_format_mplane *pixm = &f->fmt.pix_mp;
-	struct fimc_lite *fimc = video_drvdata(file);
+	struct media_pad *pad = &me->pads[0];
+
+	while (!(pad->flags & MEDIA_PAD_FL_SOURCE)) {
+		pad = media_entity_remote_source(pad);
+		if (!pad)
+			break;
+		me = pad->entity;
+		pad = &me->pads[0];
+	}
+
+	return me;
+}
+
+/**
+ * fimc_pipeline_try_format - negotiate and/or set formats at pipeline
+ *                            elements
+ * @ctx: FIMC capture context
+ * @tfmt: media bus format to try/set on subdevs
+ * @fmt_id: fimc pixel format id corresponding to returned @tfmt (output)
+ * @set: true to set format on subdevs, false to try only
+ */
+static int fimc_pipeline_try_format(struct fimc_lite *fimc,
+				    struct v4l2_mbus_framefmt *tfmt,
+				    struct fimc_fmt **fmt_id,
+				    bool set)
+{
+	struct v4l2_subdev *sd;
+	struct v4l2_subdev_format sfmt;
+	struct v4l2_mbus_framefmt *mf = &sfmt.format;
+	struct media_entity *me;
+	struct fimc_fmt *ffmt;
+	struct media_pad *pad;
+	int ret, i = 1;
+	u32 fcc;
+
+	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
+					get_subdev_sensor, &fimc->pipeline);
+
+	if (WARN_ON(!sd || !tfmt))
+		return -EINVAL;
+
+	memset(&sfmt, 0, sizeof(sfmt));
+	sfmt.format = *tfmt;
+	sfmt.which = set ? V4L2_SUBDEV_FORMAT_ACTIVE : V4L2_SUBDEV_FORMAT_TRY;
+
+	me = fimc_pipeline_get_head(&sd->entity);
+
+	while (1) {
+		ffmt = fimc_lite_find_format(NULL,
+				mf->code != 0 ? &mf->code : NULL, i++);
+		if (ffmt == NULL) {
+			/*
+			 * Notify user-space if common pixel code for
+			 * host and sensor does not exist.
+			 */
+			return -EINVAL;
+		}
+		mf->code = tfmt->code = ffmt->mbus_code;
+
+		/* set format on all pipeline subdevs */
+		while (me != &fimc->subdev.entity) {
+			sd = media_entity_to_v4l2_subdev(me);
+
+			sfmt.pad = 0;
+			ret = v4l2_subdev_call(sd, pad, set_fmt, NULL, &sfmt);
+			if (ret)
+				return ret;
+
+			if (me->pads[0].flags & MEDIA_PAD_FL_SINK) {
+				sfmt.pad = me->num_pads - 1;
+				sfmt.format.code = tfmt->code;
+				ret = v4l2_subdev_call(sd, pad, set_fmt, NULL,
+									&sfmt);
+				if (ret)
+					return ret;
+			}
+
+			pad = media_entity_remote_source(&me->pads[sfmt.pad]);
+			if (!pad)
+				return -EINVAL;
+			me = pad->entity;
+		}
+
+		if (mf->code != tfmt->code)
+			continue;
+
+		fcc = ffmt->fourcc;
+		tfmt->width  = mf->width;
+		tfmt->height = mf->height;
+		ffmt = fimc_lite_try_format(fimc, &tfmt->width, &tfmt->height,
+					NULL, &fcc, FIMC_SD_PAD_SINK);
+		ffmt = fimc_lite_try_format(fimc, &tfmt->width, &tfmt->height,
+					NULL, &fcc, FIMC_SD_PAD_SOURCE);
+		if (ffmt && ffmt->mbus_code)
+			mf->code = ffmt->mbus_code;
+		if (mf->width != tfmt->width || mf->height != tfmt->height)
+			continue;
+		tfmt->code = mf->code;
+		break;
+	}
+
+	if (fmt_id && ffmt)
+		*fmt_id = ffmt;
+	*tfmt = *mf;
+
+	return 0;
+}
+
+
+static int __fimc_lite_set_format(struct fimc_lite *fimc,
+				     struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp;
 	struct flite_frame *frame = &fimc->out_frame;
 	const struct fimc_fmt *fmt = NULL;
+	struct v4l2_mbus_framefmt mf;
+	struct fimc_fmt *s_fmt = NULL;
 	int ret;
 
 	if (vb2_is_busy(&fimc->vb_queue))
@@ -741,15 +861,59 @@ static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
 	if (ret < 0)
 		return ret;
 
+	/* Reset cropping and set format at the camera interface input */
+	if (!fimc->user_subdev_api) {
+		fimc->inp_frame.f_width = pix_mp->width;
+		fimc->inp_frame.f_height = pix_mp->height;
+		fimc->inp_frame.rect.top = 0;
+		fimc->inp_frame.rect.left = 0;
+		fimc->inp_frame.rect.width = pix_mp->width;
+		fimc->inp_frame.rect.height = pix_mp->height;
+	}
+
+	/* Try to match format at the host and the sensor */
+	if (!fimc->user_subdev_api) {
+		mf.code   = fmt->mbus_code;
+		mf.width  = pix_mp->width;
+		mf.height = pix_mp->height;
+		ret = fimc_pipeline_try_format(fimc, &mf, &s_fmt, true);
+		if (ret)
+			return ret;
+
+		pix_mp->width  = mf.width;
+		pix_mp->height = mf.height;
+	}
+
 	fimc->fmt = fmt;
-	fimc->payload[0] = max((pixm->width * pixm->height * fmt->depth[0]) / 8,
-			       pixm->plane_fmt[0].sizeimage);
-	frame->f_width = pixm->width;
-	frame->f_height = pixm->height;
+	fimc->payload[0] = max((pix_mp->width * pix_mp->height *
+			fmt->depth[0]) / 8, pix_mp->plane_fmt[0].sizeimage);
+	frame->f_width = pix_mp->width;
+	frame->f_height = pix_mp->height;
 
 	return 0;
 }
 
+static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct fimc_lite *fimc = video_drvdata(file);
+	int ret;
+
+	exynos_pipeline_graph_lock(fimc->pipeline_ops, &fimc->pipeline);
+	/*
+	 * The graph is walked within __fimc_lite_set_format() to set
+	 * the format at subdevs thus the graph mutex needs to be held at
+	 * this point and acquired before the video mutex, to avoid  AB-BA
+	 * deadlock when fimc_md_link_notify() is called by other thread.
+	 * Ideally the graph walking and setting format at the whole pipeline
+	 * should be removed from this driver and handled in userspace only.
+	 */
+	ret = __fimc_lite_set_format(fimc, f);
+
+	exynos_pipeline_graph_unlock(fimc->pipeline_ops, &fimc->pipeline);
+	return ret;
+}
+
 static int fimc_pipeline_validate(struct fimc_lite *fimc)
 {
 	struct v4l2_subdev *sd = &fimc->subdev;
@@ -1247,7 +1411,7 @@ static int fimc_lite_subdev_s_stream(struct v4l2_subdev *sd, int on)
 	mutex_lock(&fimc->lock);
 	if (on) {
 		flite_hw_reset(fimc);
-		ret = fimc_lite_hw_init(fimc, true);
+		ret = fimc_lite_hw_init(fimc, false);
 		if (!ret) {
 			spin_lock_irqsave(&fimc->slock, flags);
 			flite_hw_capture_start(fimc);
diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.h b/drivers/media/platform/s5p-fimc/fimc-lite.h
index ef43fe0..7ea57c8 100644
--- a/drivers/media/platform/s5p-fimc/fimc-lite.h
+++ b/drivers/media/platform/s5p-fimc/fimc-lite.h
@@ -52,7 +52,6 @@ enum {
 #define FLITE_VER_EXYNOS4	0
 #define FLITE_VER_EXYNOS5	1
 
-
 struct flite_variant {
 	unsigned short max_width;
 	unsigned short max_height;
@@ -175,6 +174,8 @@ struct fimc_lite {
 	unsigned int		reqbufs_count;
 	int			ref_count;
 
+	bool			user_subdev_api;
+
 	struct fimc_lite_events	events;
 };
 
-- 
1.7.9.5

[RFC 04/12] s5p-csis: Adding Exynos5250 compatibility

[Shaik Ameer Basha]

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 drivers/media/platform/s5p-fimc/mipi-csis.c |    1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/platform/s5p-fimc/mipi-csis.c b/drivers/media/platform/s5p-fimc/mipi-csis.c
index df4411c..debda7c 100644
--- a/drivers/media/platform/s5p-fimc/mipi-csis.c
+++ b/drivers/media/platform/s5p-fimc/mipi-csis.c
@@ -1002,6 +1002,7 @@ static const struct dev_pm_ops s5pcsis_pm_ops = {
 static const struct of_device_id s5pcsis_of_match[] __devinitconst = {
 	{ .compatible = "samsung,exynos3110-csis" },
 	{ .compatible = "samsung,exynos4210-csis" },
+	{ .compatible = "samsung,exynos5250-csis" },
 	{ /* sentinel */ },
 };
 MODULE_DEVICE_TABLE(of, s5pcsis_of_match);
-- 
1.7.9.5

[RFC 05/12] ARM: EXYNOS: Add devicetree node for mipi-csis driver for exynos5

[Shaik Ameer Basha]

This patch adds necessary source definations needed for mipi-csis
driver and adds devicetree node for exynos5250.

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 arch/arm/boot/dts/exynos5250.dtsi       |   18 ++++++++++++++++++
 arch/arm/mach-exynos/clock-exynos5.c    |   16 ++++++++++++++--
 arch/arm/mach-exynos/include/mach/map.h |    3 +++
 arch/arm/mach-exynos/mach-exynos5-dt.c  |    4 ++++
 4 files changed, 39 insertions(+), 2 deletions(-)

diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
index 3a2cd9a..4fff98b 100644
--- a/arch/arm/boot/dts/exynos5250.dtsi
+++ b/arch/arm/boot/dts/exynos5250.dtsi
@@ -47,6 +47,8 @@
 		i2c6 = &i2c_6;
 		i2c7 = &i2c_7;
 		i2c8 = &i2c_8;
+		csis0 = &csis_0;
+		csis1 = &csis_1;
 	};
 
 	gic:interrupt-controller@10481000 {
@@ -357,4 +359,20 @@
 		reg = <0x14450000 0x10000>;
 		interrupts = <0 94 0>;
 	};
+
+	csis_0: csis@13C20000 {
+		compatible = "samsung,exynos5250-csis";
+		reg = <0x13C20000 0x4000>;
+		interrupts = <0 79 0>;
+		bus-width = <4>;
+		status = "disabled";
+	};
+
+	csis_1: csis@13C30000 {
+		compatible = "samsung,exynos5250-csis";
+		reg = <0x13C30000 0x4000>;
+		interrupts = <0 80 0>;
+		bus-width = <4>;
+		status = "disabled";
+	};
 };
diff --git a/arch/arm/mach-exynos/clock-exynos5.c b/arch/arm/mach-exynos/clock-exynos5.c
index e9d7b80..34a22ff 100644
--- a/arch/arm/mach-exynos/clock-exynos5.c
+++ b/arch/arm/mach-exynos/clock-exynos5.c
@@ -859,6 +859,16 @@ static struct clk exynos5_init_clocks_off[] = {
 		.enable		= exynos5_clk_ip_gscl_ctrl,
 		.ctrlbit	= (1 << 3),
 	}, {
+		.name		= "csis",
+		.devname	= "s5p-mipi-csis.0",
+		.enable		= exynos5_clk_ip_gscl_ctrl,
+		.ctrlbit	= (1 << 5),
+	}, {
+		.name		= "csis",
+		.devname	= "s5p-mipi-csis.1",
+		.enable		= exynos5_clk_ip_gscl_ctrl,
+		.ctrlbit	= (1 << 6),
+	}, {
 		.name		= SYSMMU_CLOCK_NAME,
 		.devname	= SYSMMU_CLOCK_DEVNAME(mfc_l, 0),
 		.enable		= &exynos5_clk_ip_mfc_ctrl,
@@ -1263,9 +1273,10 @@ static struct clksrc_clk exynos5_clksrcs[] = {
 		.reg_div = { .reg = EXYNOS5_CLKDIV_FSYS0, .shift = 20, .size = 4 },
 	}, {
 		.clk	= {
-			.name		= "sclk_gscl_wrap",
+			.name		= "sclk_csis",
 			.devname	= "s5p-mipi-csis.0",
 			.enable		= exynos5_clksrc_mask_gscl_ctrl,
+			.parent		= &exynos5_clk_mout_mpll_user.clk,
 			.ctrlbit	= (1 << 24),
 		},
 		.sources = &exynos5_clkset_group,
@@ -1273,9 +1284,10 @@ static struct clksrc_clk exynos5_clksrcs[] = {
 		.reg_div = { .reg = EXYNOS5_CLKDIV_GSCL, .shift = 24, .size = 4 },
 	}, {
 		.clk	= {
-			.name		= "sclk_gscl_wrap",
+			.name		= "sclk_csis",
 			.devname	= "s5p-mipi-csis.1",
 			.enable		= exynos5_clksrc_mask_gscl_ctrl,
+			.parent		= &exynos5_clk_mout_mpll_user.clk,
 			.ctrlbit	= (1 << 28),
 		},
 		.sources = &exynos5_clkset_group,
diff --git a/arch/arm/mach-exynos/include/mach/map.h b/arch/arm/mach-exynos/include/mach/map.h
index 1df6abb..c834321 100644
--- a/arch/arm/mach-exynos/include/mach/map.h
+++ b/arch/arm/mach-exynos/include/mach/map.h
@@ -177,6 +177,9 @@
 #define EXYNOS4_PA_MIPI_CSIS0		0x11880000
 #define EXYNOS4_PA_MIPI_CSIS1		0x11890000
 
+#define EXYNOS5_PA_MIPI_CSIS0		0x13C20000
+#define EXYNOS5_PA_MIPI_CSIS1		0x13C30000
+
 #define EXYNOS4_PA_FIMD0		0x11C00000
 
 #define EXYNOS4_PA_HSMMC(x)		(0x12510000 + ((x) * 0x10000))
diff --git a/arch/arm/mach-exynos/mach-exynos5-dt.c b/arch/arm/mach-exynos/mach-exynos5-dt.c
index e99d3d8..c420349 100644
--- a/arch/arm/mach-exynos/mach-exynos5-dt.c
+++ b/arch/arm/mach-exynos/mach-exynos5-dt.c
@@ -104,6 +104,10 @@ static const struct of_dev_auxdata exynos5250_auxdata_lookup[] __initconst = {
 	OF_DEV_AUXDATA("samsung,mfc-v6", 0x11000000, "s5p-mfc-v6", NULL),
 	OF_DEV_AUXDATA("samsung,exynos5250-tmu", 0x10060000,
 				"exynos-tmu", NULL),
+	OF_DEV_AUXDATA("samsung,exynos5250-csis", EXYNOS5_PA_MIPI_CSIS0,
+				"s5p-mipi-csis.0", NULL),
+	OF_DEV_AUXDATA("samsung,exynos5250-csis", EXYNOS5_PA_MIPI_CSIS1,
+				"s5p-mipi-csis.1", NULL),
 	{},
 };
 
-- 
1.7.9.5

[RFC 06/12] ARM: EXYNOS: Add devicetree node for FIMC-LITE driver for exynos5

[Shaik Ameer Basha]

This patch adds necessary source definitions needed for FIMC-LITE
driver and adds devicetree node for exynos5250.

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 arch/arm/boot/dts/exynos5250.dtsi       |   21 +++++++++++++++++++++
 arch/arm/mach-exynos/clock-exynos5.c    |    4 ++++
 arch/arm/mach-exynos/include/mach/map.h |    4 ++++
 arch/arm/mach-exynos/mach-exynos5-dt.c  |    6 ++++++
 4 files changed, 35 insertions(+)

diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
index 4fff98b..4754865 100644
--- a/arch/arm/boot/dts/exynos5250.dtsi
+++ b/arch/arm/boot/dts/exynos5250.dtsi
@@ -49,6 +49,9 @@
 		i2c8 = &i2c_8;
 		csis0 = &csis_0;
 		csis1 = &csis_1;
+		fimc-lite0 = &fimc_lite_0;
+		fimc-lite1 = &fimc_lite_1;
+		fimc-lite2 = &fimc_lite_2;
 	};
 
 	gic:interrupt-controller@10481000 {
@@ -375,4 +378,22 @@
 		bus-width = <4>;
 		status = "disabled";
 	};
+
+	fimc_lite_0: fimc-lite@13C00000 {
+		compatible = "samsung,exynos5250-fimc-lite";
+		reg = <0x13C00000 0x1000>;
+		interrupts = <0 125 0>;
+	};
+
+	fimc_lite_1: fimc-lite@13C10000 {
+		compatible = "samsung,exynos5250-fimc-lite";
+		reg = <0x13C10000 0x1000>;
+		interrupts = <0 126 0>;
+	};
+
+	fimc_lite_2: fimc-lite@13C90000 {
+		compatible = "samsung,exynos5250-fimc-lite";
+		reg = <0x13C90000 0x1000>;
+		interrupts = <0 110 0>;
+	};
 };
diff --git a/arch/arm/mach-exynos/clock-exynos5.c b/arch/arm/mach-exynos/clock-exynos5.c
index 34a22ff..4536515 100644
--- a/arch/arm/mach-exynos/clock-exynos5.c
+++ b/arch/arm/mach-exynos/clock-exynos5.c
@@ -859,6 +859,10 @@ static struct clk exynos5_init_clocks_off[] = {
 		.enable		= exynos5_clk_ip_gscl_ctrl,
 		.ctrlbit	= (1 << 3),
 	}, {
+		.name		= "flite",
+		.enable		= exynos5_clk_ip_gscl_ctrl,
+		.ctrlbit	= (1 << 4),
+	}, {
 		.name		= "csis",
 		.devname	= "s5p-mipi-csis.0",
 		.enable		= exynos5_clk_ip_gscl_ctrl,
diff --git a/arch/arm/mach-exynos/include/mach/map.h b/arch/arm/mach-exynos/include/mach/map.h
index c834321..5bfc744 100644
--- a/arch/arm/mach-exynos/include/mach/map.h
+++ b/arch/arm/mach-exynos/include/mach/map.h
@@ -125,6 +125,10 @@
 #define EXYNOS4_PA_SYSMMU_MFC_L		0x13620000
 #define EXYNOS4_PA_SYSMMU_MFC_R		0x13630000
 
+#define EXYNOS5_PA_FIMC_LITE0           0x13C00000
+#define EXYNOS5_PA_FIMC_LITE1           0x13C10000
+#define EXYNOS5_PA_FIMC_LITE2           0x13C90000
+
 #define EXYNOS5_PA_GSC0			0x13E00000
 #define EXYNOS5_PA_GSC1			0x13E10000
 #define EXYNOS5_PA_GSC2			0x13E20000
diff --git a/arch/arm/mach-exynos/mach-exynos5-dt.c b/arch/arm/mach-exynos/mach-exynos5-dt.c
index c420349..f6c3223 100644
--- a/arch/arm/mach-exynos/mach-exynos5-dt.c
+++ b/arch/arm/mach-exynos/mach-exynos5-dt.c
@@ -108,6 +108,12 @@ static const struct of_dev_auxdata exynos5250_auxdata_lookup[] __initconst = {
 				"s5p-mipi-csis.0", NULL),
 	OF_DEV_AUXDATA("samsung,exynos5250-csis", EXYNOS5_PA_MIPI_CSIS1,
 				"s5p-mipi-csis.1", NULL),
+	OF_DEV_AUXDATA("samsung,exynos5250-fimc-lite", EXYNOS5_PA_FIMC_LITE0,
+				"exynos5-fimc-lite.0", NULL),
+	OF_DEV_AUXDATA("samsung,exynos5250-fimc-lite", EXYNOS5_PA_FIMC_LITE1,
+				"exynos5-fimc-lite.1", NULL),
+	OF_DEV_AUXDATA("samsung,exynos5250-fimc-lite", EXYNOS5_PA_FIMC_LITE2,
+				"exynos5-fimc-lite.2", NULL),
 	{},
 };
 
-- 
1.7.9.5

[RFC 07/12] media: exynos5-is: Adding media device driver for exynos5

[Shaik Ameer Basha]

This patch adds support for media device for EXYNOS5 SoCs.
The current media device supports the following ips to connect
through the media controller framework.

* MIPI-CSIS
  Support interconnection(subdev interface) between devices

* FIMC-LITE
  Support capture interface from device(Sensor, MIPI-CSIS) to memory
  Support interconnection(subdev interface) between devices

G-Scaler will be added later to the current media device.

* Gscaler: general scaler
  Support memory to memory interface
  Support output interface from memory to display device(LCD, TV)
  Support capture interface from device(FIMC-LITE, FIMD) to memory

--> media 0
  Camera Capture path consists of MIPI-CSIS, FIMC-LITE and G-Scaler
  +--------+     +-----------+     +-----------------+
  | Sensor | --> | FIMC-LITE | --> | G-Scaler-capture |
  +--------+     +-----------+     +-----------------+

  +--------+     +-----------+     +-----------+     +-----------------+
  | Sensor | --> | MIPI-CSIS | --> | FIMC-LITE | --> | G-Scaler-capture |
  +--------+     +-----------+     +-----------+     +-----------------+

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 drivers/media/platform/Kconfig                   |    1 +
 drivers/media/platform/Makefile                  |    1 +
 drivers/media/platform/exynos5-is/Kconfig        |    7 +
 drivers/media/platform/exynos5-is/Makefile       |    4 +
 drivers/media/platform/exynos5-is/exynos5-mdev.c | 1309 ++++++++++++++++++++++
 drivers/media/platform/exynos5-is/exynos5-mdev.h |  107 ++
 6 files changed, 1429 insertions(+)
 create mode 100644 drivers/media/platform/exynos5-is/Kconfig
 create mode 100644 drivers/media/platform/exynos5-is/Makefile
 create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.c
 create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 2433e2b..f74bd92 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -123,6 +123,7 @@ config VIDEO_S3C_CAMIF
 
 source "drivers/media/platform/soc_camera/Kconfig"
 source "drivers/media/platform/s5p-fimc/Kconfig"
+source "drivers/media/platform/exynos5-is/Kconfig"
 source "drivers/media/platform/s5p-tv/Kconfig"
 
 endif # V4L_PLATFORM_DRIVERS
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 42089ba..43da7ab 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -37,6 +37,7 @@ obj-$(CONFIG_VIDEO_SAMSUNG_S5P_TV)	+= s5p-tv/
 
 obj-$(CONFIG_VIDEO_SAMSUNG_S5P_G2D)	+= s5p-g2d/
 obj-$(CONFIG_VIDEO_SAMSUNG_EXYNOS_GSC)	+= exynos-gsc/
+obj-$(CONFIG_VIDEO_SAMSUNG_EXYNOS5_MDEV)	+= exynos5-is/
 
 obj-$(CONFIG_BLACKFIN)                  += blackfin/
 
diff --git a/drivers/media/platform/exynos5-is/Kconfig b/drivers/media/platform/exynos5-is/Kconfig
new file mode 100644
index 0000000..7aacf3b
--- /dev/null
+++ b/drivers/media/platform/exynos5-is/Kconfig
@@ -0,0 +1,7 @@
+config VIDEO_SAMSUNG_EXYNOS5_MDEV
+	bool "Samsung Exynos5 Media Device driver"
+	depends on VIDEO_DEV && VIDEO_V4L2 && ARCH_EXYNOS5
+	help
+	  This is a v4l2 based media controller driver for
+	  Exynos5 SoC.
+
diff --git a/drivers/media/platform/exynos5-is/Makefile b/drivers/media/platform/exynos5-is/Makefile
new file mode 100644
index 0000000..472d8e1
--- /dev/null
+++ b/drivers/media/platform/exynos5-is/Makefile
@@ -0,0 +1,4 @@
+ccflags-y += -Idrivers/media/platform/s5p-fimc
+exynos-mdevice-objs := exynos5-mdev.o
+
+obj-$(CONFIG_VIDEO_SAMSUNG_EXYNOS5_MDEV) += exynos-mdevice.o
diff --git a/drivers/media/platform/exynos5-is/exynos5-mdev.c b/drivers/media/platform/exynos5-is/exynos5-mdev.c
new file mode 100644
index 0000000..1158696
--- /dev/null
+++ b/drivers/media/platform/exynos5-is/exynos5-mdev.c
@@ -0,0 +1,1309 @@
+/*
+ * S5P/EXYNOS4 SoC series camera host interface media device driver
+ *
+ * Copyright (C) 2011 - 2012 Samsung Electronics Co., Ltd.
+ * Sylwester Nawrocki <s.nawrocki@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation, either version 2 of the License,
+ * or (at your option) any later version.
+ */
+
+#include <linux/bug.h>
+#include <linux/device.h>
+#include <linux/errno.h>
+#include <linux/i2c.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/of_device.h>
+#include <linux/of_i2c.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/types.h>
+#include <linux/slab.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-of.h>
+#include <media/media-device.h>
+
+#include "exynos5-mdev.h"
+
+#define dbg(fmt, args...) \
+	pr_debug("%s:%d: " fmt "\n", __func__, __LINE__, ##args)
+
+static struct fimc_md *g_exynos_mdev;
+
+static int fimc_md_set_camclk(struct v4l2_subdev *sd, bool on);
+static int __fimc_md_set_camclk(struct fimc_md *fmd,
+				struct fimc_sensor_info *s_info,
+				bool on);
+/**
+ * fimc_pipeline_prepare - update pipeline information with subdevice pointers
+ * @fimc: fimc device terminating the pipeline
+ *
+ * Caller holds the graph mutex.
+ */
+static void fimc_pipeline_prepare(struct exynos5_pipeline0 *p,
+				  struct media_entity *me)
+{
+	struct media_pad *pad = &me->pads[0];
+	struct v4l2_subdev *sd;
+	int i;
+
+	for (i = 0; i < IDX_MAX; i++)
+		p->subdevs[i] = NULL;
+
+	while (1) {
+
+		if (!(pad->flags & MEDIA_PAD_FL_SINK))
+			break;
+
+		/* source pad */
+		pad = media_entity_remote_source(pad);
+
+		if (pad != NULL)
+			pr_err("entity type: %d, entity name: %s\n",
+			    media_entity_type(pad->entity), pad->entity->name);
+
+		if (pad == NULL ||
+		    media_entity_type(pad->entity) != MEDIA_ENT_T_V4L2_SUBDEV)
+			break;
+
+		sd = media_entity_to_v4l2_subdev(pad->entity);
+
+		switch (sd->grp_id) {
+		case GRP_ID_FIMC_IS_SENSOR:
+		case GRP_ID_SENSOR:
+			p->subdevs[IDX_SENSOR] = sd;
+			break;
+		case GRP_ID_CSIS:
+			p->subdevs[IDX_CSIS] = sd;
+			break;
+		case GRP_ID_FLITE:
+			p->subdevs[IDX_FLITE] = sd;
+			break;
+		default:
+			pr_warn("%s: Unknown subdev grp_id: %#x\n",
+				__func__, sd->grp_id);
+		}
+
+		/* sink pad */
+		pad = &sd->entity.pads[0];
+	}
+}
+
+/**
+ * __subdev_set_power - change power state of a single subdev
+ * @sd: subdevice to change power state for
+ * @on: 1 to enable power or 0 to disable
+ *
+ * Return result of s_power subdev operation or -ENXIO if sd argument
+ * is NULL. Return 0 if the subdevice does not implement s_power.
+ */
+static int __subdev_set_power(struct v4l2_subdev *sd, int on)
+{
+	int *use_count;
+	int ret;
+
+	if (sd == NULL)
+		return -ENXIO;
+
+	use_count = &sd->entity.use_count;
+	if (on && (*use_count)++ > 0)
+		return 0;
+	else if (!on && (*use_count == 0 || --(*use_count) > 0))
+		return 0;
+
+	ret = v4l2_subdev_call(sd, core, s_power, on);
+
+	return ret != -ENOIOCTLCMD ? ret : 0;
+}
+
+/**
+ * fimc_pipeline_s_power - change power state of all pipeline subdevs
+ * @fimc: fimc device terminating the pipeline
+ * @state: true to power on, false to power off
+ *
+ * Needs to be called with the graph mutex held.
+ */
+static int fimc_pipeline_s_power(struct exynos5_pipeline0 *p, bool state)
+{
+	unsigned int i;
+	int ret;
+
+	if (p->subdevs[IDX_SENSOR] == NULL)
+		return -ENXIO;
+
+	for (i = 0; i < IDX_MAX; i++) {
+		unsigned int idx = state ? (IDX_MAX - 1) - i : i;
+
+		ret = __subdev_set_power(p->subdevs[idx], state);
+		if (ret < 0 && ret != -ENXIO)
+			return ret;
+	}
+
+	return 0;
+}
+
+/**
+ * __fimc_pipeline_init
+ *    allocate the fimc_pipeline structure and do the basic initialization
+ */
+static int __fimc_pipeline_init(struct exynos_pipeline *ep)
+{
+	struct exynos5_pipeline0 *p;
+
+	p = kzalloc(sizeof(*p), GFP_KERNEL);
+	if (!p)
+		return -ENOMEM;
+
+	p->is_init = true;
+	p->fmd = g_exynos_mdev;
+	ep->priv = (void *)p;
+	return 0;
+}
+
+/**
+ * __fimc_pipeline_deinit
+ *    free the allocated resources for fimc_pipeline
+ */
+static int __fimc_pipeline_deinit(struct exynos_pipeline *ep)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+
+	if (!p || !p->is_init)
+		return -EINVAL;
+
+	p->is_init = false;
+	kfree(p);
+
+	return 0;
+}
+
+/**
+ * __fimc_pipeline_get_subdev_sensor
+ *     if valid pipeline, returns the sensor subdev pointer
+ *     else returns NULL
+ */
+static struct v4l2_subdev *__fimc_pipeline_get_subdev_sensor(
+					struct exynos_pipeline *ep)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+
+	if (!p || !p->is_init)
+		return NULL;
+
+	return p->subdevs[IDX_SENSOR];
+}
+
+/**
+ * __fimc_pipeline_get_subdev_csis
+ *      if valid pipeline, returns the csis subdev pointer
+ *      else returns NULL
+ */
+static struct v4l2_subdev *__fimc_pipeline_get_subdev_csis(
+					struct exynos_pipeline *ep)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+
+	if (!p || !p->is_init)
+		return NULL;
+
+	return p->subdevs[IDX_CSIS];
+}
+/**
+ * __fimc_pipeline_open - update the pipeline information, enable power
+ *                        of all pipeline subdevs and the sensor clock
+ * @me: media entity to start graph walk with
+ * @prep: true to acquire sensor (and csis) subdevs
+ *
+ * Called with the graph mutex held.
+ */
+static int __fimc_pipeline_open(struct exynos_pipeline *ep,
+				struct media_entity *me, bool prep)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+	int ret;
+
+	if (prep)
+		fimc_pipeline_prepare(p, me);
+
+	if (p->subdevs[IDX_SENSOR] == NULL)
+		return -EINVAL;
+
+	ret = fimc_md_set_camclk(p->subdevs[IDX_SENSOR], true);
+	if (ret)
+		return ret;
+
+	return fimc_pipeline_s_power(p, 1);
+}
+
+/**
+ * __fimc_pipeline_close - disable the sensor clock and pipeline power
+ * @fimc: fimc device terminating the pipeline
+ *
+ * Disable power of all subdevs and turn the external sensor clock off.
+ */
+static int __fimc_pipeline_close(struct exynos_pipeline *ep)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+	int ret = 0;
+
+	if (!p || !p->subdevs[IDX_SENSOR])
+		return -EINVAL;
+
+	if (p->subdevs[IDX_SENSOR]) {
+		ret = fimc_pipeline_s_power(p, 0);
+		fimc_md_set_camclk(p->subdevs[IDX_SENSOR], false);
+	}
+	return ret == -ENXIO ? 0 : ret;
+}
+
+/**
+ * __fimc_pipeline_s_stream - invoke s_stream on pipeline subdevs
+ * @pipeline: video pipeline structure
+ * @on: passed as the s_stream call argument
+ */
+static int __fimc_pipeline_s_stream(struct exynos_pipeline *ep, bool on)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+	int i, ret;
+
+	if (p->subdevs[IDX_SENSOR] == NULL)
+		return -ENODEV;
+
+	for (i = 0; i < IDX_MAX; i++) {
+		unsigned int idx = on ? (IDX_MAX - 1) - i : i;
+
+		ret = v4l2_subdev_call(p->subdevs[idx], video, s_stream, on);
+
+		if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
+			return ret;
+	}
+
+	return 0;
+
+}
+
+static void __fimc_pipeline_graph_lock(struct exynos_pipeline *ep)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+	struct fimc_md *fmd = p->fmd;
+
+	mutex_lock(&fmd->media_dev.graph_mutex);
+}
+
+static void __fimc_pipeline_graph_unlock(struct exynos_pipeline *ep)
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+	struct fimc_md *fmd = p->fmd;
+
+	mutex_unlock(&fmd->media_dev.graph_mutex);
+}
+
+static void __fimc_pipeline_register_notify_callback(
+		struct exynos_pipeline *ep,
+		void (*notify_cb)(struct v4l2_subdev *sd,
+				unsigned int notification, void *arg))
+{
+	struct exynos5_pipeline0 *p = (struct exynos5_pipeline0 *)ep->priv;
+
+	if (!notify_cb)
+		return;
+
+	p->sensor_notify = notify_cb;
+}
+
+/* Media pipeline operations for the FIMC/FIMC-LITE video device driver */
+static const struct exynos_pipeline_ops exynos5_pipeline0_ops = {
+	.init			= __fimc_pipeline_init,
+	.deinit			= __fimc_pipeline_deinit,
+	.open			= __fimc_pipeline_open,
+	.close			= __fimc_pipeline_close,
+	.set_stream		= __fimc_pipeline_s_stream,
+	.get_subdev_sensor	= __fimc_pipeline_get_subdev_sensor,
+	.get_subdev_csis	= __fimc_pipeline_get_subdev_csis,
+	.graph_lock		= __fimc_pipeline_graph_lock,
+	.graph_unlock		= __fimc_pipeline_graph_unlock,
+	.register_notify_cb	= __fimc_pipeline_register_notify_callback,
+};
+
+/*
+ * Sensor subdevice helper functions
+ */
+static struct v4l2_subdev *fimc_md_register_sensor(struct fimc_md *fmd,
+				   struct fimc_sensor_info *s_info)
+{
+	struct i2c_adapter *adapter;
+	struct v4l2_subdev *sd = NULL;
+
+	if (!s_info || !fmd)
+		return NULL;
+
+	adapter = i2c_get_adapter(s_info->pdata.i2c_bus_num);
+	if (!adapter) {
+		v4l2_warn(&fmd->v4l2_dev,
+			  "Failed to get I2C adapter %d, deferring probe\n",
+			  s_info->pdata.i2c_bus_num);
+		return ERR_PTR(-EPROBE_DEFER);
+	}
+	sd = v4l2_i2c_new_subdev_board(&fmd->v4l2_dev, adapter,
+				       s_info->pdata.board_info, NULL);
+	if (IS_ERR_OR_NULL(sd)) {
+		i2c_put_adapter(adapter);
+		v4l2_warn(&fmd->v4l2_dev,
+			  "Failed to acquire subdev %s, deferring probe\n",
+			  s_info->pdata.board_info->type);
+		return ERR_PTR(-EPROBE_DEFER);
+	}
+	v4l2_set_subdev_hostdata(sd, s_info);
+	sd->grp_id = GRP_ID_SENSOR;
+
+	v4l2_info(&fmd->v4l2_dev, "Registered sensor subdevice %s\n",
+		  sd->name);
+	return sd;
+}
+
+static void fimc_md_unregister_sensor(struct v4l2_subdev *sd)
+{
+	struct i2c_client *client = v4l2_get_subdevdata(sd);
+	struct i2c_adapter *adapter;
+
+	if (!client)
+		return;
+	v4l2_device_unregister_subdev(sd);
+
+	if (!client->dev.of_node) {
+		adapter = client->adapter;
+		i2c_unregister_device(client);
+		if (adapter)
+			i2c_put_adapter(adapter);
+	}
+}
+
+#ifdef CONFIG_OF
+/* Register I2C client subdev associated with @node. */
+static int fimc_md_of_add_sensor(struct fimc_md *fmd,
+				 struct device_node *node, int index)
+{
+	struct fimc_sensor_info *si;
+	struct i2c_client *client;
+	struct v4l2_subdev *sd;
+	int ret;
+
+	if (WARN_ON(index >= ARRAY_SIZE(fmd->sensor)))
+		return -EINVAL;
+
+	si = &fmd->sensor[index];
+
+	client = of_find_i2c_device_by_node(node);
+	if (!client)
+		return -EPROBE_DEFER;
+
+	device_lock(&client->dev);
+
+	if (!client->driver ||
+	    !try_module_get(client->driver->driver.owner)) {
+		ret = -EPROBE_DEFER;
+		goto dev_put;
+	}
+
+	/* Enable sensor's master clock */
+	ret = __fimc_md_set_camclk(fmd, si, true);
+	if (ret < 0)
+		goto mod_put;
+
+	sd = i2c_get_clientdata(client);
+
+	ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
+	__fimc_md_set_camclk(fmd, si, false);
+	if (ret < 0)
+		goto mod_put;
+
+	v4l2_set_subdev_hostdata(sd, si);
+	sd->grp_id = GRP_ID_SENSOR;
+	si->subdev = sd;
+	v4l2_info(&fmd->v4l2_dev, "Registered sensor subdevice: %s (%d)\n",
+		  sd->name, fmd->num_sensors);
+	fmd->num_sensors++;
+
+mod_put:
+	module_put(client->driver->driver.owner);
+dev_put:
+	device_unlock(&client->dev);
+	put_device(&client->dev);
+	return ret;
+}
+
+/* Parse port node and register as a sub-device any sensor specified there. */
+static int fimc_md_parse_port_node(struct fimc_md *fmd,
+				   struct device_node *port,
+				   unsigned int index)
+{
+	struct device_node *rem, *endpoint;
+	struct fimc_source_info *pd;
+	struct v4l2_of_endpoint bus_cfg;
+	u32 tmp, reg = 0;
+	int ret;
+
+	if (WARN_ON(of_property_read_u32(port, "reg", &reg) ||
+	    reg >= FIMC_MAX_SENSORS))
+		return -EINVAL;
+
+	pd = &fmd->sensor[index].pdata;
+	pd->mux_id = (reg - 1) & 0x1;
+
+	/* Assume here a port node can have only one endpoint node. */
+	endpoint = of_get_next_child(port, NULL);
+	if (!endpoint)
+		return 0;
+
+	rem = v4l2_of_get_remote_port_parent(endpoint);
+	of_node_put(endpoint);
+	if (rem == NULL) {
+		v4l2_info(&fmd->v4l2_dev, "Remote device at %s not found\n",
+			  endpoint->full_name);
+		return 0;
+	}
+
+	if (!of_property_read_u32(rem, "samsung,camclk-out", &tmp))
+		pd->clk_id = tmp;
+
+	if (!of_property_read_u32(rem, "clock-frequency", &tmp))
+		pd->clk_frequency = tmp;
+
+	if (pd->clk_frequency == 0) {
+		v4l2_err(&fmd->v4l2_dev, "Wrong clock frequency at node %s\n",
+			 rem->full_name);
+		of_node_put(rem);
+		return -EINVAL;
+	}
+
+	if (fimc_input_is_parallel(reg)) {
+		v4l2_of_parse_parallel_bus(endpoint, &bus_cfg);
+		if (bus_cfg.mbus.type == V4L2_MBUS_PARALLEL)
+			pd->sensor_bus_type = FIMC_BUS_TYPE_ITU_601;
+		else
+			pd->sensor_bus_type = FIMC_BUS_TYPE_ITU_656;
+		pd->flags = bus_cfg.mbus.flags;
+	} else if (fimc_input_is_mipi_csi(reg)) {
+		/*
+		 * MIPI CSI-2: only input mux selection
+		 * and sensor's clock frequency is needed.
+		 */
+		pd->sensor_bus_type = FIMC_BUS_TYPE_MIPI_CSI2;
+	} else {
+		v4l2_err(&fmd->v4l2_dev, "Wrong port id (%u) at node %s\n",
+			 reg, rem->full_name);
+	}
+
+	ret = fimc_md_of_add_sensor(fmd, rem, index);
+	of_node_put(rem);
+
+	return ret;
+}
+
+/* Register all SoC external sub-devices */
+static int fimc_md_of_sensors_register(struct fimc_md *fmd,
+				       struct device_node *np)
+{
+	struct device_node *parent = fmd->pdev->dev.of_node;
+	struct device_node *node, *ports;
+	int index = 0;
+	int ret;
+
+	/* Attach sensors linked to MIPI CSI-2 receivers */
+	for_each_available_child_of_node(parent, node) {
+		struct device_node *port;
+
+		if (of_node_cmp(node->name, "csis"))
+			continue;
+
+		/* The csis node can have only port subnode. */
+		port = of_get_next_child(node, NULL);
+		if (!port)
+			continue;
+
+		ret = fimc_md_parse_port_node(fmd, port, index);
+		if (ret < 0)
+			return ret;
+		index++;
+	}
+
+	/* Attach sensors listed in the parallel-ports node */
+	ports = of_get_child_by_name(parent, "parallel-ports");
+	if (!ports)
+		return 0;
+
+	for_each_child_of_node(ports, node) {
+		ret = fimc_md_parse_port_node(fmd, node, index);
+		if (ret < 0)
+			break;
+		index++;
+	}
+
+	return 0;
+}
+#else
+#define fimc_md_of_sensors_register(fmd, np) (-ENOSYS)
+#endif
+
+static int fimc_md_register_sensor_entities(struct fimc_md *fmd)
+{
+	struct s5p_platform_fimc *pdata = fmd->pdev->dev.platform_data;
+	struct device_node *of_node = fmd->pdev->dev.of_node;
+	int num_clients = 0;
+	int ret, i;
+
+	if (of_node) {
+		fmd->num_sensors = 0;
+		ret = fimc_md_of_sensors_register(fmd, of_node);
+	} else if (pdata) {
+		WARN_ON(pdata->num_clients > ARRAY_SIZE(fmd->sensor));
+		num_clients = min_t(u32, pdata->num_clients,
+				    ARRAY_SIZE(fmd->sensor));
+		fmd->num_sensors = num_clients;
+
+		fmd->num_sensors = num_clients;
+		for (i = 0; i < num_clients; i++) {
+			struct v4l2_subdev *sd;
+
+			fmd->sensor[i].pdata = pdata->source_info[i];
+			ret = __fimc_md_set_camclk(fmd, &fmd->sensor[i], true);
+			if (ret)
+				break;
+			sd = fimc_md_register_sensor(fmd, &fmd->sensor[i]);
+			ret = __fimc_md_set_camclk(fmd, &fmd->sensor[i], false);
+
+			if (IS_ERR(sd)) {
+				fmd->sensor[i].subdev = NULL;
+				ret = PTR_ERR(sd);
+				break;
+			}
+			fmd->sensor[i].subdev = sd;
+			if (ret)
+				break;
+		}
+	}
+
+	return ret;
+}
+
+/*
+ * MIPI-CSIS, FIMC and FIMC-LITE platform devices registration.
+ */
+
+static int register_fimc_lite_entity(struct fimc_md *fmd,
+				     struct fimc_lite *fimc_lite)
+{
+	struct v4l2_subdev *sd;
+	int ret;
+
+	if (WARN_ON(fimc_lite->index >= FIMC_LITE_MAX_DEVS ||
+		    fmd->fimc_lite[fimc_lite->index]))
+		return -EBUSY;
+
+	sd = &fimc_lite->subdev;
+	sd->grp_id = GRP_ID_FLITE;
+	v4l2_set_subdev_hostdata(sd, (void *)&exynos5_pipeline0_ops);
+
+	ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
+	if (!ret)
+		fmd->fimc_lite[fimc_lite->index] = fimc_lite;
+	else
+		v4l2_err(&fmd->v4l2_dev, "Failed to register FIMC.LITE%d\n",
+			 fimc_lite->index);
+	return ret;
+}
+
+static int register_csis_entity(struct fimc_md *fmd,
+				struct platform_device *pdev,
+				struct v4l2_subdev *sd)
+{
+	struct device_node *node = pdev->dev.of_node;
+	int id, ret;
+
+	id = node ? of_alias_get_id(node, "csis") : max(0, pdev->id);
+
+	if (WARN_ON(id >= CSIS_MAX_ENTITIES || fmd->csis[id].sd))
+		return -EBUSY;
+
+	if (WARN_ON(id >= CSIS_MAX_ENTITIES))
+		return 0;
+
+	sd->grp_id = GRP_ID_CSIS;
+	ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
+	if (!ret)
+		fmd->csis[id].sd = sd;
+	else
+		v4l2_err(&fmd->v4l2_dev,
+			 "Failed to register MIPI-CSIS.%d (%d)\n", id, ret);
+
+	return ret;
+}
+
+static int fimc_md_register_platform_entity(struct fimc_md *fmd,
+					    struct platform_device *pdev,
+					    int plat_entity)
+{
+	struct device *dev = &pdev->dev;
+	int ret = -EPROBE_DEFER;
+	void *drvdata;
+
+	/* Lock to ensure dev->driver won't change. */
+	device_lock(dev);
+
+	if (!dev->driver || !try_module_get(dev->driver->owner))
+		goto dev_unlock;
+
+	drvdata = dev_get_drvdata(dev);
+	/* Some subdev didn't probe succesfully id drvdata is NULL */
+	if (drvdata) {
+		switch (plat_entity) {
+		case IDX_FLITE:
+			ret = register_fimc_lite_entity(fmd, drvdata);
+			break;
+		case IDX_CSIS:
+			ret = register_csis_entity(fmd, pdev, drvdata);
+			break;
+		default:
+			ret = -ENODEV;
+		}
+	}
+
+	module_put(dev->driver->owner);
+dev_unlock:
+	device_unlock(dev);
+	if (ret == -EPROBE_DEFER)
+		dev_info(&fmd->pdev->dev, "deferring %s device registration\n",
+			dev_name(dev));
+	else if (ret < 0)
+		dev_err(&fmd->pdev->dev, "%s device registration failed (%d)\n",
+			dev_name(dev), ret);
+	return ret;
+}
+
+static int fimc_md_pdev_match(struct device *dev, void *data)
+{
+	struct platform_device *pdev = to_platform_device(dev);
+	int plat_entity = -1;
+	int ret;
+
+	if (!get_device(dev))
+		return -ENODEV;
+
+	if (!strcmp(pdev->name, CSIS_DRIVER_NAME))
+		plat_entity = IDX_CSIS;
+	else if (!strcmp(pdev->name, FIMC_LITE_DRV_NAME))
+		plat_entity = IDX_FLITE;
+
+	if (plat_entity >= 0)
+		ret = fimc_md_register_platform_entity(data, pdev,
+						       plat_entity);
+	put_device(dev);
+	return 0;
+}
+
+/* Register FIMC, FIMC-LITE and CSIS media entities */
+#ifdef CONFIG_OF
+static int fimc_md_register_of_platform_entities(struct fimc_md *fmd,
+						 struct device_node *parent)
+{
+	struct device_node *node;
+	int ret = 0;
+
+	for_each_available_child_of_node(parent, node) {
+		struct platform_device *pdev;
+		int plat_entity = -1;
+
+		pdev = of_find_device_by_node(node);
+		if (!pdev)
+			continue;
+
+		/* If driver of any entity isn't ready try all again later. */
+		if (!strcmp(node->name, CSIS_OF_NODE_NAME))
+			plat_entity = IDX_CSIS;
+		else if (!strcmp(node->name, FIMC_LITE_OF_NODE_NAME))
+			plat_entity = IDX_FLITE;
+
+		if (plat_entity >= 0)
+			ret = fimc_md_register_platform_entity(fmd, pdev,
+							plat_entity);
+		put_device(&pdev->dev);
+		if (ret < 0)
+			break;
+	}
+
+	return ret;
+}
+#else
+#define fimc_md_register_platform_entities(fmd) (-ENOSYS)
+#endif
+
+static void fimc_md_unregister_entities(struct fimc_md *fmd)
+{
+	int i;
+
+	for (i = 0; i < FIMC_LITE_MAX_DEVS; i++) {
+		if (fmd->fimc_lite[i] == NULL)
+			continue;
+		v4l2_device_unregister_subdev(&fmd->fimc_lite[i]->subdev);
+		fmd->fimc_lite[i]->pipeline_ops = NULL;
+		fmd->fimc_lite[i] = NULL;
+	}
+	for (i = 0; i < CSIS_MAX_ENTITIES; i++) {
+		if (fmd->csis[i].sd == NULL)
+			continue;
+		v4l2_device_unregister_subdev(fmd->csis[i].sd);
+		module_put(fmd->csis[i].sd->owner);
+		fmd->csis[i].sd = NULL;
+	}
+	for (i = 0; i < fmd->num_sensors; i++) {
+		if (fmd->sensor[i].subdev == NULL)
+			continue;
+		fimc_md_unregister_sensor(fmd->sensor[i].subdev);
+		fmd->sensor[i].subdev = NULL;
+	}
+	v4l2_info(&fmd->v4l2_dev, "Unregistered all entities\n");
+}
+
+/**
+ * __fimc_md_create_fimc_links - create links to all FIMC entities
+ * @fmd: fimc media device
+ * @source: the source entity to create links to all fimc entities from
+ * @sensor: sensor subdev linked to FIMC[fimc_id] entity, may be null
+ * @pad: the source entity pad index
+ * @link_mask: bitmask of the fimc devices for which link should be enabled
+ */
+static int __fimc_md_create_fimc_sink_links(struct fimc_md *fmd,
+					    struct media_entity *source,
+					    struct v4l2_subdev *sensor,
+					    int pad, int link_mask)
+{
+	struct media_entity *sink;
+	unsigned int flags = 0;
+	int ret, i;
+
+	for (i = 0; i < FIMC_LITE_MAX_DEVS; i++) {
+		if (!fmd->fimc_lite[i])
+			continue;
+
+		flags = ((1 << i) & link_mask) ? MEDIA_LNK_FL_ENABLED : 0;
+		sink = &fmd->fimc_lite[i]->subdev.entity;
+		ret = media_entity_create_link(source, pad, sink,
+					       FLITE_SD_PAD_SINK, flags);
+		if (ret)
+			return ret;
+
+		/* Notify FIMC-LITE subdev entity */
+		ret = media_entity_call(sink, link_setup, &sink->pads[0],
+					&source->pads[pad], flags);
+		if (ret)
+			break;
+
+		v4l2_info(&fmd->v4l2_dev, "created link [%s] %c> [%s]\n",
+			  source->name, flags ? '=' : '-', sink->name);
+	}
+	return 0;
+}
+
+/**
+ * fimc_md_create_links - create default links between registered entities
+ *
+ * Parallel interface sensor entities are connected directly to FIMC capture
+ * entities. The sensors using MIPI CSIS bus are connected through immutable
+ * link with CSI receiver entity specified by mux_id. Any registered CSIS
+ * entity has a link to each registered FIMC capture entity. Enabled links
+ * are created by default between each subsequent registered sensor and
+ * subsequent FIMC capture entity. The number of default active links is
+ * determined by the number of available sensors or FIMC entities,
+ * whichever is less.
+ */
+static int fimc_md_create_links(struct fimc_md *fmd)
+{
+	struct v4l2_subdev *csi_sensors[CSIS_MAX_ENTITIES] = { NULL };
+	struct v4l2_subdev *sensor, *csis;
+	struct fimc_source_info *pdata;
+	struct fimc_sensor_info *s_info;
+	struct media_entity *source, *sink;
+	int i, pad, fimc_id = 0, ret = 0;
+	u32 flags, link_mask = 0;
+
+	for (i = 0; i < fmd->num_sensors; i++) {
+		if (fmd->sensor[i].subdev == NULL)
+			continue;
+
+		sensor = fmd->sensor[i].subdev;
+		s_info = v4l2_get_subdev_hostdata(sensor);
+		if (!s_info)
+			continue;
+
+		source = NULL;
+		pdata = &s_info->pdata;
+
+		switch (pdata->sensor_bus_type) {
+		case FIMC_BUS_TYPE_MIPI_CSI2:
+			if (WARN(pdata->mux_id >= CSIS_MAX_ENTITIES,
+				"Wrong CSI channel id: %d\n", pdata->mux_id))
+				return -EINVAL;
+
+			csis = fmd->csis[pdata->mux_id].sd;
+			if (WARN(csis == NULL,
+				 "MIPI-CSI interface specified "
+				 "but s5p-csis module is not loaded!\n"))
+				return -EINVAL;
+
+			pad = sensor->entity.num_pads - 1;
+			ret = media_entity_create_link(&sensor->entity, pad,
+					      &csis->entity, CSIS_PAD_SINK,
+					      MEDIA_LNK_FL_IMMUTABLE |
+					      MEDIA_LNK_FL_ENABLED);
+			if (ret)
+				return ret;
+
+			v4l2_info(&fmd->v4l2_dev, "created link [%s] => [%s]\n",
+				  sensor->entity.name, csis->entity.name);
+
+			source = NULL;
+			csi_sensors[pdata->mux_id] = sensor;
+			break;
+
+		case FIMC_BUS_TYPE_ITU_601...FIMC_BUS_TYPE_ITU_656:
+			source = &sensor->entity;
+			pad = 0;
+			break;
+
+		default:
+			v4l2_err(&fmd->v4l2_dev, "Wrong bus_type: %x\n",
+				 pdata->sensor_bus_type);
+			return -EINVAL;
+		}
+		if (source == NULL)
+			continue;
+
+		link_mask = 1 << fimc_id++;
+		ret = __fimc_md_create_fimc_sink_links(fmd, source, sensor,
+						       pad, link_mask);
+	}
+
+	for (i = 0; i < CSIS_MAX_ENTITIES; i++) {
+		if (fmd->csis[i].sd == NULL)
+			continue;
+		source = &fmd->csis[i].sd->entity;
+		pad = CSIS_PAD_SOURCE;
+		sensor = csi_sensors[i];
+
+		link_mask = 1 << fimc_id++;
+		ret = __fimc_md_create_fimc_sink_links(fmd, source, sensor,
+						       pad, link_mask);
+	}
+
+	/* Create immutable links b/w each FIMC-LITE's subdev and video node */
+	flags = MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED;
+	for (i = 0; i < FIMC_LITE_MAX_DEVS; i++) {
+		if (!fmd->fimc_lite[i])
+			continue;
+		source = &fmd->fimc_lite[i]->subdev.entity;
+		sink = &fmd->fimc_lite[i]->vfd.entity;
+		ret = media_entity_create_link(source, FLITE_SD_PAD_SOURCE_DMA,
+					      sink, 0, flags);
+		if (ret)
+			break;
+	}
+
+	return 0;
+}
+
+/*
+ * The peripheral sensor clock management.
+ */
+static void fimc_md_put_clocks(struct fimc_md *fmd)
+{
+	int i = FIMC_MAX_CAMCLKS;
+
+	while (--i >= 0) {
+		if (IS_ERR(fmd->camclk[i].clock))
+			continue;
+		clk_unprepare(fmd->camclk[i].clock);
+		clk_put(fmd->camclk[i].clock);
+		fmd->camclk[i].clock = ERR_PTR(-EINVAL);
+	}
+}
+
+static int fimc_md_get_clocks(struct fimc_md *fmd)
+{
+	struct device *dev = NULL;
+	char clk_name[32];
+	struct clk *clock;
+	int ret, i;
+
+	for (i = 0; i < FIMC_MAX_CAMCLKS; i++)
+		fmd->camclk[i].clock = ERR_PTR(-EINVAL);
+
+	if (fmd->pdev->dev.of_node)
+		dev = &fmd->pdev->dev;
+
+	for (i = 0; i < FIMC_MAX_CAMCLKS; i++) {
+		snprintf(clk_name, sizeof(clk_name), "sclk_cam%u", i);
+		clock = clk_get(dev, clk_name);
+
+		if (IS_ERR(clock)) {
+			dev_err(&fmd->pdev->dev, "Failed to get clock: %s\n",
+								clk_name);
+			ret = PTR_ERR(clock);
+			break;
+		}
+		ret = clk_prepare(clock);
+		if (ret < 0) {
+			clk_put(clock);
+			fmd->camclk[i].clock = ERR_PTR(-EINVAL);
+			break;
+		}
+		fmd->camclk[i].clock = clock;
+	}
+	if (ret)
+		fimc_md_put_clocks(fmd);
+
+	return ret;
+}
+
+static int __fimc_md_set_camclk(struct fimc_md *fmd,
+				struct fimc_sensor_info *s_info,
+				bool on)
+{
+	struct fimc_source_info *pdata = &s_info->pdata;
+	struct fimc_camclk_info *camclk;
+	int ret = 0;
+
+	if (WARN_ON(pdata->clk_id >= FIMC_MAX_CAMCLKS) || fmd == NULL)
+		return -EINVAL;
+
+	camclk = &fmd->camclk[pdata->clk_id];
+
+	dbg("camclk %d, f: %lu, use_count: %d, on: %d",
+	    pdata->clk_id, pdata->clk_frequency, camclk->use_count, on);
+
+	if (on) {
+		if (camclk->use_count > 0 &&
+		    camclk->frequency != pdata->clk_frequency)
+			return -EINVAL;
+
+		if (camclk->use_count++ == 0) {
+			clk_set_rate(camclk->clock, pdata->clk_frequency);
+			camclk->frequency = pdata->clk_frequency;
+			ret = clk_enable(camclk->clock);
+			dbg("Enabled camclk %d: f: %lu", pdata->clk_id,
+			    clk_get_rate(camclk->clock));
+		}
+		return ret;
+	}
+
+	if (WARN_ON(camclk->use_count == 0))
+		return 0;
+
+	if (--camclk->use_count == 0) {
+		clk_disable(camclk->clock);
+		dbg("Disabled camclk %d", pdata->clk_id);
+	}
+
+	return ret;
+}
+
+/**
+ * fimc_md_set_camclk - peripheral sensor clock setup
+ * @sd: sensor subdev to configure sclk_cam clock for
+ * @on: 1 to enable or 0 to disable the clock
+ *
+ * There are 2 separate clock outputs available in the SoC for external
+ * image processors. These clocks are shared between all registered FIMC
+ * devices to which sensors can be attached, either directly or through
+ * the MIPI CSI receiver. The clock is allowed here to be used by
+ * multiple sensors concurrently if they use same frequency.
+ * This function should only be called when the graph mutex is held.
+ */
+static int fimc_md_set_camclk(struct v4l2_subdev *sd, bool on)
+{
+	struct fimc_sensor_info *s_info = v4l2_get_subdev_hostdata(sd);
+	struct fimc_md *fmd = entity_to_fimc_mdev(&sd->entity);
+
+	return __fimc_md_set_camclk(fmd, s_info, on);
+}
+
+static int fimc_md_link_notify(struct media_pad *source,
+			       struct media_pad *sink, u32 flags)
+{
+	struct fimc_lite *fimc_lite = NULL;
+	struct exynos_pipeline *pipeline;
+	struct exynos5_pipeline0 *p;
+	struct v4l2_subdev *sd;
+	struct mutex *lock;
+	int ret = 0;
+	int ref_count;
+
+	if (media_entity_type(sink->entity) != MEDIA_ENT_T_V4L2_SUBDEV)
+		return 0;
+
+	sd = media_entity_to_v4l2_subdev(sink->entity);
+
+	switch (sd->grp_id) {
+	case GRP_ID_FLITE:
+		fimc_lite = v4l2_get_subdevdata(sd);
+		if (WARN_ON(fimc_lite == NULL))
+			return 0;
+		pipeline = &fimc_lite->pipeline;
+		lock = &fimc_lite->lock;
+		break;
+	default:
+		return 0;
+	}
+
+	p = (struct exynos5_pipeline0 *)pipeline->priv;
+	if (!p || !p->is_init)
+		return -EINVAL;
+
+	if (!(flags & MEDIA_LNK_FL_ENABLED)) {
+		int i;
+		mutex_lock(lock);
+		ret = __fimc_pipeline_close(pipeline);
+		for (i = 0; i < IDX_MAX; i++)
+			p->subdevs[i] = NULL;
+		mutex_unlock(lock);
+		return ret;
+	}
+	/*
+	 * Link activation. Enable power of pipeline elements only if the
+	 * pipeline is already in use, i.e. its video node is opened.
+	 * Recreate the controls destroyed during the link deactivation.
+	 */
+	mutex_lock(lock);
+
+	ref_count = fimc_lite->ref_count;
+	if (ref_count > 0)
+		ret = __fimc_pipeline_open(pipeline, source->entity, true);
+
+	mutex_unlock(lock);
+	return ret ? -EPIPE : ret;
+}
+
+static ssize_t fimc_md_sysfs_show(struct device *dev,
+				  struct device_attribute *attr, char *buf)
+{
+	struct platform_device *pdev = to_platform_device(dev);
+	struct fimc_md *fmd = platform_get_drvdata(pdev);
+
+	if (fmd->user_subdev_api)
+		return strlcpy(buf, "Sub-device API (sub-dev)\n", PAGE_SIZE);
+
+	return strlcpy(buf, "V4L2 video node only API (vid-dev)\n", PAGE_SIZE);
+}
+
+static ssize_t fimc_md_sysfs_store(struct device *dev,
+				   struct device_attribute *attr,
+				   const char *buf, size_t count)
+{
+	struct platform_device *pdev = to_platform_device(dev);
+	struct fimc_md *fmd = platform_get_drvdata(pdev);
+	bool subdev_api;
+	int i;
+
+	if (!strcmp(buf, "vid-dev\n"))
+		subdev_api = false;
+	else if (!strcmp(buf, "sub-dev\n"))
+		subdev_api = true;
+	else
+		return count;
+
+	fmd->user_subdev_api = subdev_api;
+	for (i = 0; i < FIMC_LITE_MAX_DEVS; i++)
+		if (fmd->fimc_lite[i])
+			fmd->fimc_lite[i]->user_subdev_api = subdev_api;
+
+	return count;
+}
+/*
+ * This device attribute is to select video pipeline configuration method.
+ * There are following valid values:
+ *  vid-dev - for V4L2 video node API only, subdevice will be configured
+ *  by the host driver.
+ *  sub-dev - for media controller API, subdevs must be configured in user
+ *  space before starting streaming.
+ */
+static DEVICE_ATTR(subdev_conf_mode, S_IWUSR | S_IRUGO,
+		   fimc_md_sysfs_show, fimc_md_sysfs_store);
+
+/**
+ * fimc_md_sensor_notify - v4l2_device notification from a sensor subdev
+ * @sd: pointer to a subdev generating the notification
+ * @notification: the notification type, must be S5P_FIMC_TX_END_NOTIFY
+ * @arg: pointer to an u32 type integer that stores the frame payload value
+ *
+ * Passes the sensor notification to the capture device
+ */
+static void fimc_md_sensor_notify(struct v4l2_subdev *sd,
+				unsigned int notification, void *arg)
+{
+	struct fimc_md *fmd;
+	struct exynos_pipeline *ep;
+	struct exynos5_pipeline0 *p;
+	unsigned long flags;
+
+	if (sd == NULL)
+		return;
+
+	ep = media_pipe_to_exynos_pipeline(sd->entity.pipe);
+	p = (struct exynos5_pipeline0 *)ep->priv;
+
+	spin_lock_irqsave(&fmd->slock, flags);
+
+	if (p->sensor_notify)
+		p->sensor_notify(sd, notification, arg);
+
+	spin_unlock_irqrestore(&fmd->slock, flags);
+}
+
+static int fimc_md_probe(struct platform_device *pdev)
+{
+	struct device *dev = &pdev->dev;
+	struct v4l2_device *v4l2_dev;
+	struct fimc_md *fmd;
+	int ret;
+
+	fmd = devm_kzalloc(dev, sizeof(*fmd), GFP_KERNEL);
+	if (!fmd)
+		return -ENOMEM;
+
+	spin_lock_init(&fmd->slock);
+	fmd->pdev = pdev;
+
+	strlcpy(fmd->media_dev.model, "SAMSUNG S5P FIMC",
+		sizeof(fmd->media_dev.model));
+	fmd->media_dev.link_notify = fimc_md_link_notify;
+	fmd->media_dev.dev = dev;
+
+	v4l2_dev = &fmd->v4l2_dev;
+	v4l2_dev->mdev = &fmd->media_dev;
+	v4l2_dev->notify = fimc_md_sensor_notify;
+	strlcpy(v4l2_dev->name, "s5p-fimc-md", sizeof(v4l2_dev->name));
+
+	ret = v4l2_device_register(dev, &fmd->v4l2_dev);
+	if (ret < 0) {
+		v4l2_err(v4l2_dev, "Failed to register v4l2_device: %d\n", ret);
+
+		return ret;
+
+	}
+	ret = media_device_register(&fmd->media_dev);
+	if (ret < 0) {
+		v4l2_err(v4l2_dev, "Failed to register media dev: %d\n", ret);
+		goto err_md;
+	}
+	ret = fimc_md_get_clocks(fmd);
+	if (ret)
+		goto err_clk;
+
+	fmd->user_subdev_api = (dev->of_node != NULL);
+	g_exynos_mdev = fmd;
+
+	/* Protect the media graph while we're registering entities */
+	mutex_lock(&fmd->media_dev.graph_mutex);
+
+	if (fmd->pdev->dev.of_node)
+		ret = fimc_md_register_of_platform_entities(fmd, dev->of_node);
+	else
+		ret = bus_for_each_dev(&platform_bus_type, NULL, fmd,
+						fimc_md_pdev_match);
+
+	if (ret)
+		goto err_unlock;
+
+	if (dev->platform_data || dev->of_node) {
+		ret = fimc_md_register_sensor_entities(fmd);
+		if (ret)
+			goto err_unlock;
+	}
+
+	ret = fimc_md_create_links(fmd);
+	if (ret)
+		goto err_unlock;
+
+	ret = v4l2_device_register_subdev_nodes(&fmd->v4l2_dev);
+	if (ret)
+		goto err_unlock;
+
+	ret = device_create_file(&pdev->dev, &dev_attr_subdev_conf_mode);
+	if (ret)
+		goto err_unlock;
+
+	platform_set_drvdata(pdev, fmd);
+	mutex_unlock(&fmd->media_dev.graph_mutex);
+	return 0;
+
+err_unlock:
+	mutex_unlock(&fmd->media_dev.graph_mutex);
+err_clk:
+	media_device_unregister(&fmd->media_dev);
+	fimc_md_put_clocks(fmd);
+	fimc_md_unregister_entities(fmd);
+err_md:
+	v4l2_device_unregister(&fmd->v4l2_dev);
+	return ret;
+}
+
+static int fimc_md_remove(struct platform_device *pdev)
+{
+	struct fimc_md *fmd = platform_get_drvdata(pdev);
+
+	if (!fmd)
+		return 0;
+	g_exynos_mdev = NULL;
+	device_remove_file(&pdev->dev, &dev_attr_subdev_conf_mode);
+	fimc_md_unregister_entities(fmd);
+	media_device_unregister(&fmd->media_dev);
+	fimc_md_put_clocks(fmd);
+	return 0;
+}
+
+static struct platform_device_id fimc_driver_ids[] __always_unused = {
+	{ .name = "exynos-fimc-md" },
+	{ },
+};
+MODULE_DEVICE_TABLE(platform, fimc_driver_ids);
+
+static const struct of_device_id fimc_md_of_match[] __initconst = {
+	{ .compatible = "samsung,exynos5-fimc" },
+	{ },
+};
+MODULE_DEVICE_TABLE(of, fimc_md_of_match);
+
+static struct platform_driver fimc_md_driver = {
+	.probe		= fimc_md_probe,
+	.remove		= fimc_md_remove,
+	.driver = {
+		.of_match_table = fimc_md_of_match,
+		.name		= "exynos-fimc-md",
+		.owner		= THIS_MODULE,
+	}
+};
+
+static int __init fimc_md_init(void)
+{
+	request_module("s5p-csis");
+	return platform_driver_register(&fimc_md_driver);
+}
+
+static void __exit fimc_md_exit(void)
+{
+	platform_driver_unregister(&fimc_md_driver);
+}
+
+module_init(fimc_md_init);
+module_exit(fimc_md_exit);
+
+MODULE_AUTHOR("Sylwester Nawrocki <s.nawrocki@samsung.com>");
+MODULE_AUTHOR("Shaik Ameer Basha <shaik.ameer@samsung.com>");
+MODULE_DESCRIPTION("EXYNOS5 FIMC camera host interface/video postprocessor driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/exynos5-is/exynos5-mdev.h b/drivers/media/platform/exynos5-is/exynos5-mdev.h
new file mode 100644
index 0000000..de0ad11
--- /dev/null
+++ b/drivers/media/platform/exynos5-is/exynos5-mdev.h
@@ -0,0 +1,107 @@
+/*
+ * Copyright (C) 2011 - 2012 Samsung Electronics Co., Ltd.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef EXYNOS5_MDEVICE_H_
+#define EXYNOS5_MDEVICE_H_
+
+#include <linux/clk.h>
+#include <linux/platform_device.h>
+#include <linux/mutex.h>
+#include <media/media-device.h>
+#include <media/media-entity.h>
+#include <media/s5p_fimc.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-subdev.h>
+
+#include "fimc-lite.h"
+#include "mipi-csis.h"
+
+#define FIMC_OF_NODE_NAME	"fimc"
+#define FIMC_LITE_OF_NODE_NAME	"fimc-lite"
+#define CSIS_OF_NODE_NAME	"csis"
+#define FIMC_IS_OF_NODE_NAME	"fimc-is"
+
+/* Group IDs of sensor, MIPI-CSIS, FIMC-LITE and the writeback subdevs. */
+#define GRP_ID_SENSOR		(1 << 8)
+#define GRP_ID_FIMC_IS_SENSOR	(1 << 9)
+#define GRP_ID_WRITEBACK	(1 << 10)
+#define GRP_ID_CSIS		(1 << 11)
+#define GRP_ID_FLITE		(1 << 13)
+#define GRP_ID_FIMC_IS		(1 << 14)
+
+#define FIMC_MAX_SENSORS	8
+#define FIMC_MAX_CAMCLKS	2
+
+enum fimc_subdev_index {
+	IDX_SENSOR,
+	IDX_CSIS,
+	IDX_FLITE,
+	IDX_FIMC,
+	IDX_MAX,
+};
+
+struct fimc_csis_info {
+	struct v4l2_subdev *sd;
+	int id;
+};
+
+struct fimc_camclk_info {
+	struct clk *clock;
+	int use_count;
+	unsigned long frequency;
+};
+
+/**
+ * struct fimc_md - fimc media device information
+ * @csis: MIPI CSIS subdevs data
+ * @sensor: array of registered sensor subdevs
+ * @num_sensors: actual number of registered sensors
+ * @camclk: external sensor clock information
+ * @fimc: array of registered fimc devices
+ * @media_dev: top level media device
+ * @v4l2_dev: top level v4l2_device holding up the subdevs
+ * @pdev: platform device this media device is hooked up into
+ * @user_subdev_api: true if subdevs are not configured by the host driver
+ * @slock: spinlock protecting @sensor array
+ */
+struct fimc_md {
+	struct fimc_csis_info csis[CSIS_MAX_ENTITIES];
+	struct fimc_sensor_info sensor[FIMC_MAX_SENSORS];
+	int num_sensors;
+	struct fimc_camclk_info camclk[FIMC_MAX_CAMCLKS];
+	struct fimc_lite *fimc_lite[FIMC_LITE_MAX_DEVS];
+	struct media_device media_dev;
+	struct v4l2_device v4l2_dev;
+	struct platform_device *pdev;
+	bool user_subdev_api;
+	spinlock_t slock;
+};
+
+struct exynos5_pipeline0 {
+	int is_init;
+	struct fimc_md *fmd;
+	struct v4l2_subdev *subdevs[IDX_MAX];
+	void (*sensor_notify)(struct v4l2_subdev *sd,
+			unsigned int notification, void *arg);
+};
+
+#define is_subdev_pad(pad) (pad == NULL || \
+	media_entity_type(pad->entity) == MEDIA_ENT_T_V4L2_SUBDEV)
+
+#define me_subtype(me) \
+	((me->type) & (MEDIA_ENT_TYPE_MASK | MEDIA_ENT_SUBTYPE_MASK))
+
+#define subdev_has_devnode(__sd) (__sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE)
+
+static inline struct fimc_md *entity_to_fimc_mdev(struct media_entity *me)
+{
+	return me->parent == NULL ? NULL :
+		container_of(me->parent, struct fimc_md, media_dev);
+}
+
+#endif /* EXYNOS5_MDEVICE_H_ */
-- 
1.7.9.5

[RFC 08/12] ARM: dts: add camera specific pinctrl nodes for Exynos5250 SoC

[Shaik Ameer Basha]

Add device nodes for pinctrl group-1 for Exynos5250 SoC.
This only adds cam1 specific pinctrl nodes to the file.

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 arch/arm/boot/dts/exynos5250-pinctrl.dtsi |   41 +++++++++++++++++++++++++++++
 arch/arm/boot/dts/exynos5250.dtsi         |    7 +++++
 2 files changed, 48 insertions(+)

diff --git a/arch/arm/boot/dts/exynos5250-pinctrl.dtsi b/arch/arm/boot/dts/exynos5250-pinctrl.dtsi
index 24180fc..3caaa21 100644
--- a/arch/arm/boot/dts/exynos5250-pinctrl.dtsi
+++ b/arch/arm/boot/dts/exynos5250-pinctrl.dtsi
@@ -555,6 +555,47 @@
 		};
 	};
 
+	pinctrl@13400000 {
+		gph0: gph0 {
+			gpio-controller;
+			#gpio-cells = <2>;
+
+			interrupt-controller;
+			#interrupt-cells = <2>;
+		};
+
+		gph1: gph1 {
+			gpio-controller;
+			#gpio-cells = <2>;
+
+			interrupt-controller;
+			#interrupt-cells = <2>;
+		};
+
+		cam_port_a_io: cam-port-a-io {
+			samsung,pins = "gph0-0", "gph0-1", "gph0-2", "gph0-3",
+				"gph1-0", "gph1-1", "gph1-2", "gph1-3",
+				"gph1-4", "gph1-5", "gph1-6", "gph1-7";
+			samsung,pin-function = <2>;
+			samsung,pin-pud = <0>;
+			samsung,pin-drv = <3>;
+		};
+
+		cam_port_a_clk_active: cam-port-a-clk-active {
+			samsung,pins = "gph0-3";
+			samsung,pin-function = <2>;
+			samsung,pin-pud = <0>;
+			samsung,pin-drv = <3>;
+		};
+
+		cam_port_a_clk_idle: cam-port-a-clk-idle {
+			samsung,pins = "gph0-3";
+			samsung,pin-function = <0>;
+			samsung,pin-pud = <0>;
+			samsung,pin-drv = <0>;
+		};
+	};
+
 	pinctrl_3: pinctrl@03680000 {
 		gpz: gpz {
 			gpio-controller;
diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
index 4754865..e09cda0 100644
--- a/arch/arm/boot/dts/exynos5250.dtsi
+++ b/arch/arm/boot/dts/exynos5250.dtsi
@@ -37,6 +37,7 @@
 		mshc2 = &dwmmc_2;
 		mshc3 = &dwmmc_3;
 		pinctrl0 = &pinctrl_0;
+		pinctrl1 = &pinctrl_1;
 		pinctrl3 = &pinctrl_3;
 		i2c0 = &i2c_0;
 		i2c1 = &i2c_1;
@@ -95,6 +96,12 @@
 		};
 	};
 
+	pinctrl_1: pinctrl@13400000 {
+		compatible = "samsung,pinctrl-exynos5250";
+		reg = <0x13400000 0x1000>;
+		interrupts = <0 47 0>;
+	};
+
 	pinctrl_3: pinctrl@03680000 {
 		compatible = "samsung,pinctrl-exynos5250";
 		reg = <0x0368000 0x1000>;
-- 
1.7.9.5

[RFC 09/12] ARM: dts: Adding pinctrl support to Exynos5250 i2c nodes

[Shaik Ameer Basha]

This patch adds the default pinctrl functionality to the
i2c device nodes on exynos5250-smdk5250.dts file

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 arch/arm/boot/dts/exynos5250-smdk5250.dts |   22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/arch/arm/boot/dts/exynos5250-smdk5250.dts b/arch/arm/boot/dts/exynos5250-smdk5250.dts
index 942d576..4b10744 100644
--- a/arch/arm/boot/dts/exynos5250-smdk5250.dts
+++ b/arch/arm/boot/dts/exynos5250-smdk5250.dts
@@ -30,8 +30,8 @@
 	i2c@12C60000 {
 		samsung,i2c-sda-delay = <100>;
 		samsung,i2c-max-bus-freq = <20000>;
-		gpios = <&gpb3 0 2 3 0>,
-			<&gpb3 1 2 3 0>;
+		pinctrl-0 = <&i2c0_bus>;
+		pinctrl-names = "default";
 
 		eeprom@50 {
 			compatible = "samsung,s524ad0xd1";
@@ -42,8 +42,8 @@
 	i2c@12C70000 {
 		samsung,i2c-sda-delay = <100>;
 		samsung,i2c-max-bus-freq = <20000>;
-		gpios = <&gpb3 2 2 3 0>,
-			<&gpb3 3 2 3 0>;
+		pinctrl-0 = <&i2c1_bus>;
+		pinctrl-names = "default";
 
 		eeprom@51 {
 			compatible = "samsung,s524ad0xd1";
@@ -69,8 +69,8 @@
 	i2c@12C80000 {
 		samsung,i2c-sda-delay = <100>;
 		samsung,i2c-max-bus-freq = <66000>;
-		gpios = <&gpa0 6 3 3 0>,
-			<&gpa0 7 3 3 0>;
+		pinctrl-0 = <&i2c2_bus>;
+		pinctrl-names = "default";
 
 		hdmiddc@50 {
 			compatible = "samsung,exynos5-hdmiddc";
@@ -80,22 +80,32 @@
 
 	i2c@12C90000 {
 		status = "disabled";
+		pinctrl-0 = <&i2c3_bus>;
+		pinctrl-names = "default";
 	};
 
 	i2c@12CA0000 {
 		status = "disabled";
+		pinctrl-0 = <&i2c4_bus>;
+		pinctrl-names = "default";
 	};
 
 	i2c@12CB0000 {
 		status = "disabled";
+		pinctrl-0 = <&i2c5_bus>;
+		pinctrl-names = "default";
 	};
 
 	i2c@12CC0000 {
 		status = "disabled";
+		pinctrl-0 = <&i2c6_bus>;
+		pinctrl-names = "default";
 	};
 
 	i2c@12CD0000 {
 		status = "disabled";
+		pinctrl-0 = <&i2c7_bus>;
+		pinctrl-names = "default";
 	};
 
 	i2c@12CE0000 {
-- 
1.7.9.5

[RFC 10/12] ARM: dts: Adding media device nodes to Exynos5 SoCs

[Shaik Ameer Basha]

This patch adds the media device driver specific dt bindings
to the Exynos5 specific SoCs.

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 arch/arm/boot/dts/exynos5250.dtsi |   64 +++++++++++++++++++++----------------
 1 file changed, 36 insertions(+), 28 deletions(-)

diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
index e09cda0..564c05f 100644
--- a/arch/arm/boot/dts/exynos5250.dtsi
+++ b/arch/arm/boot/dts/exynos5250.dtsi
@@ -370,37 +370,45 @@
 		interrupts = <0 94 0>;
 	};
 
-	csis_0: csis@13C20000 {
-		compatible = "samsung,exynos5250-csis";
-		reg = <0x13C20000 0x4000>;
-		interrupts = <0 79 0>;
-		bus-width = <4>;
-		status = "disabled";
-	};
+	camera {
+		compatible = "samsung,exynos5-fimc", "simple-bus";
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
 
-	csis_1: csis@13C30000 {
-		compatible = "samsung,exynos5250-csis";
-		reg = <0x13C30000 0x4000>;
-		interrupts = <0 80 0>;
-		bus-width = <4>;
-		status = "disabled";
-	};
+		fimc_lite_0: fimc-lite@13C00000 {
+			compatible = "samsung,exynos5250-fimc-lite";
+			reg = <0x13C00000 0x1000>;
+			interrupts = <0 125 0>;
+		};
 
-	fimc_lite_0: fimc-lite@13C00000 {
-		compatible = "samsung,exynos5250-fimc-lite";
-		reg = <0x13C00000 0x1000>;
-		interrupts = <0 125 0>;
-	};
+		fimc_lite_1: fimc-lite@13C10000 {
+			compatible = "samsung,exynos5250-fimc-lite";
+			reg = <0x13C10000 0x1000>;
+			interrupts = <0 126 0>;
+		};
 
-	fimc_lite_1: fimc-lite@13C10000 {
-		compatible = "samsung,exynos5250-fimc-lite";
-		reg = <0x13C10000 0x1000>;
-		interrupts = <0 126 0>;
-	};
+		fimc_lite_2: fimc-lite@13C90000 {
+			compatible = "samsung,exynos5250-fimc-lite";
+			reg = <0x13C90000 0x1000>;
+			interrupts = <0 110 0>;
+		};
 
-	fimc_lite_2: fimc-lite@13C90000 {
-		compatible = "samsung,exynos5250-fimc-lite";
-		reg = <0x13C90000 0x1000>;
-		interrupts = <0 110 0>;
+		csis_0: csis@13C20000 {
+			compatible = "samsung,exynos5250-csis";
+			reg = <0x13C20000 0x4000>;
+			interrupts = <0 79 0>;
+			bus-width = <4>;
+			clock-names = "csis", "sclk_csis", "mux", "parent";
+			status = "disabled";
+		};
+
+		csis_1: csis@13C30000 {
+			compatible = "samsung,exynos5250-csis";
+			reg = <0x13C30000 0x4000>;
+			interrupts = <0 80 0>;
+			bus-width = <2>;
+			status = "disabled";
+		};
 	};
 };
-- 
1.7.9.5

[RFC 11/12] media: m5mols: Adding dt support to m5mols driver

[Shaik Ameer Basha]

This patch adds the dt support to m5mols driver.

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 drivers/media/i2c/m5mols/m5mols_core.c |   54 +++++++++++++++++++++++++++++++-
 1 file changed, 53 insertions(+), 1 deletion(-)

diff --git a/drivers/media/i2c/m5mols/m5mols_core.c b/drivers/media/i2c/m5mols/m5mols_core.c
index d4e7567..21c66ef 100644
--- a/drivers/media/i2c/m5mols/m5mols_core.c
+++ b/drivers/media/i2c/m5mols/m5mols_core.c
@@ -19,6 +19,8 @@
 #include <linux/interrupt.h>
 #include <linux/delay.h>
 #include <linux/gpio.h>
+#include <linux/of_gpio.h>
+#include <linux/pinctrl/consumer.h>
 #include <linux/regulator/consumer.h>
 #include <linux/videodev2.h>
 #include <linux/module.h>
@@ -926,13 +928,38 @@ static irqreturn_t m5mols_irq_handler(int irq, void *data)
 	return IRQ_HANDLED;
 }
 
+static const struct of_device_id m5mols_match[];
+
 static int m5mols_probe(struct i2c_client *client,
 			const struct i2c_device_id *id)
 {
-	const struct m5mols_platform_data *pdata = client->dev.platform_data;
+	struct m5mols_platform_data *pdata;
 	struct m5mols_info *info;
+	const struct of_device_id *of_id;
 	struct v4l2_subdev *sd;
 	int ret;
+	struct pinctrl *pctrl;
+	int eint_gpio = 0;
+
+	if (client->dev.of_node) {
+		of_id = of_match_node(m5mols_match, client->dev.of_node);
+		if (of_id)
+			pdata = (struct m5mols_platform_data *)of_id->data;
+		client->dev.platform_data = pdata;
+	} else {
+		pdata = client->dev.platform_data;
+	}
+
+	if (!pdata)
+		return -EINVAL;
+
+	pctrl = devm_pinctrl_get_select_default(&client->dev);
+	if (client->dev.of_node) {
+		eint_gpio = of_get_named_gpio(client->dev.of_node, "gpios", 0);
+		client->irq = gpio_to_irq(eint_gpio);
+		pdata->gpio_reset = of_get_named_gpio(client->dev.of_node,
+								"gpios", 1);
+	}
 
 	if (pdata == NULL) {
 		dev_err(&client->dev, "No platform data\n");
@@ -1040,9 +1067,34 @@ static const struct i2c_device_id m5mols_id[] = {
 };
 MODULE_DEVICE_TABLE(i2c, m5mols_id);
 
+static int m5mols_set_power(struct device *dev, int on)
+{
+	struct m5mols_platform_data *pdata =
+			(struct m5mols_platform_data *)dev->platform_data;
+	gpio_set_value(pdata->gpio_reset, !on);
+	gpio_set_value(pdata->gpio_reset, !!on);
+	return 0;
+}
+
+static struct m5mols_platform_data m5mols_drvdata = {
+	.gpio_reset	= 0,
+	.reset_polarity	= 0,
+	.set_power	= m5mols_set_power,
+};
+
+static const struct of_device_id m5mols_match[] = {
+	{
+		.compatible = "fujitsu,m-5mols",
+		.data = &m5mols_drvdata,
+	},
+	{},
+};
+MODULE_DEVICE_TABLE(of, m5mols_match);
+
 static struct i2c_driver m5mols_i2c_driver = {
 	.driver = {
 		.name	= MODULE_NAME,
+		.of_match_table = m5mols_match,
 	},
 	.probe		= m5mols_probe,
 	.remove		= m5mols_remove,
-- 
1.7.9.5

[RFC 12/12] ARM: dts: Add camera node to exynos5250-smdk5250.dts

[Shaik Ameer Basha]

Signed-off-by: Shaik Ameer Basha <shaik.ameer@samsung.com>
---
 arch/arm/boot/dts/exynos5250-smdk5250.dts |   43 ++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/exynos5250-smdk5250.dts b/arch/arm/boot/dts/exynos5250-smdk5250.dts
index 4b10744..7fbc236 100644
--- a/arch/arm/boot/dts/exynos5250-smdk5250.dts
+++ b/arch/arm/boot/dts/exynos5250-smdk5250.dts
@@ -85,9 +85,26 @@
 	};
 
 	i2c@12CA0000 {
-		status = "disabled";
+		samsung,i2c-sda-delay = <100>;
+		samsung,i2c-max-bus-freq = <100000>;
 		pinctrl-0 = <&i2c4_bus>;
 		pinctrl-names = "default";
+
+		m5mols@1f {
+			compatible = "fujitsu,m-5mols";
+			reg = <0x1F>;
+			gpios = <&gpx3 3 0xf>, <&gpx1 2 1>;
+			clock-frequency = <24000000>;
+			pinctrl-0 = <&cam_port_a_clk_active>;
+			pinctrl-names = "default";
+
+			port {
+				m5mols_ep: endpoint {
+					remote-endpoint = <&csis0_ep>;
+				};
+			};
+
+		};
 	};
 
 	i2c@12CB0000 {
@@ -214,4 +231,28 @@
 		samsung,mfc-r = <0x43000000 0x800000>;
 		samsung,mfc-l = <0x51000000 0x800000>;
 	};
+
+	camera {
+		compatible = "samsung,exynos5-fimc", "simple-bus";
+		status = "okay";
+
+		csis_0: csis@13C20000 {
+			status = "okay";
+			clock-frequency = <166000000>;
+			#address-cells = <1>;
+			#size-cells = <0>;
+
+			/* Camera C (3) MIPI CSI-2 (CSIS0) */
+			port@3 {
+				reg = <3>;
+				csis0_ep: endpoint {
+					remote-endpoint = <&m5mols_ep>;
+					data-lanes = <1 2 3 4>;
+					samsung,csis-hs-settle = <12>;
+					samsung,csis-wclk;
+				};
+			};
+		};
+
+	};
 };
-- 
1.7.9.5

Re: [RFC 00/12] Adding media device driver for Exynos imaging subsystem

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> The following patchset features:
>
> 1] Creating a common pipeline framework which can be used by all
> Exynos series SoCs for developing media device drivers.
> 2] Modified the existing fimc-mdevice for exynos4 to use the common
> pipeline framework.
> 3] Adding of media device driver for Exynos5 Imaging subsystem.
> 4] Upgrading mipi-csis and fimc-lite drivers for Exynos5 SoCs.
> 5] Adding DT support to m5mols driver and tested with Exynos5 media
> device driver.
>
> Current changes are not tested on exynos4 series SoCs. Current media
> device driver only support one pipeline (pipeline0) which consists of
> 	Sensor -->  MIPI-CSIS -->  FIMC-LITE
> 	Sensor -->  FIMC-LITE

Thanks Shaik. My quick review to follow. I'll focus on most significant
issues for now.

And here is my updated patch series adding device tree support to
s5p-fimc driver [1]. I hope it gets merged to v3.10 without significant
changes. Sorry, for now we need to use that private branch.

[1] http://git.linuxtv.org/snawrocki/samsung.git/devicetree-fimc

Regards,
Sylwester

Re: [RFC 02/12] fimc-lite: Adding Exynos5 compatibility to fimc-lite driver

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> This patch adds the Exynos5 soc compatibility to the fimc-lite driver.
> It also adds a version checking to deal with the changes between
> different fimc-lite hardware versions.

Is there really anything different between the Exynos4 and Exynos5
FIMC-LITE IPs except the maximum number of buffer descriptors in
the output DMA queue ?

> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> ---
>   drivers/media/platform/s5p-fimc/fimc-lite.c |   23 +++++++++++++++++++++++
>   drivers/media/platform/s5p-fimc/fimc-lite.h |    7 ++++++-
>   2 files changed, 29 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c b/drivers/media/platform/s5p-fimc/fimc-lite.c
> index 122cf95..eb64f87 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-lite.c
> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
> @@ -1653,6 +1653,16 @@ static struct flite_variant fimc_lite0_variant_exynos4 = {
>   	.out_width_align	= 8,
>   	.win_hor_offs_align	= 2,
>   	.out_hor_offs_align	= 8,
> +	.version		= FLITE_VER_EXYNOS4,
> +};
> +
> +static struct flite_variant fimc_lite0_variant_exynos5 = {
> +	.max_width		= 8192,
> +	.max_height		= 8192,
> +	.out_width_align	= 8,
> +	.win_hor_offs_align	= 2,
> +	.out_hor_offs_align	= 8,

Please see my comment to patch 03/12.

> +	.version		= FLITE_VER_EXYNOS5,
>   };
>
>   /* EXYNOS4212, EXYNOS4412 */
> @@ -1663,6 +1673,15 @@ static struct flite_drvdata fimc_lite_drvdata_exynos4 = {
>   	},
>   };
>
> +/* EXYNOS5250 */
> +static struct flite_drvdata fimc_lite_drvdata_exynos5 = {
> +	.variant = {
> +		[0] =&fimc_lite0_variant_exynos5,
> +		[1] =&fimc_lite0_variant_exynos5,
> +		[2] =&fimc_lite0_variant_exynos5,
> +	},
> +};
> +
>   static struct platform_device_id fimc_lite_driver_ids[] = {
>   	{
>   		.name		= "exynos-fimc-lite",
> @@ -1677,6 +1696,10 @@ static const struct of_device_id flite_of_match[] = {
>   		.compatible = "samsung,exynos4212-fimc-lite",
>   		.data =&fimc_lite_drvdata_exynos4,
>   	},
> +	{
> +		.compatible = "samsung,exynos5250-fimc-lite",
> +		.data =&fimc_lite_drvdata_exynos5,
> +	},
>   	{ /* sentinel */ },
>   };
>   MODULE_DEVICE_TABLE(of, flite_of_match);
> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.h b/drivers/media/platform/s5p-fimc/fimc-lite.h
> index 66d6eeb..ef43fe0 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-lite.h
> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.h
> @@ -28,7 +28,7 @@
>
>   #define FIMC_LITE_DRV_NAME	"exynos-fimc-lite"
>   #define FLITE_CLK_NAME		"flite"
> -#define FIMC_LITE_MAX_DEVS	2
> +#define FIMC_LITE_MAX_DEVS	3
>   #define FLITE_REQ_BUFS_MIN	2
>
>   /* Bit index definitions for struct fimc_lite::state */
> @@ -49,12 +49,17 @@ enum {
>   #define FLITE_SD_PAD_SOURCE_ISP	2
>   #define FLITE_SD_PADS_NUM	3
>
> +#define FLITE_VER_EXYNOS4	0
> +#define FLITE_VER_EXYNOS5	1

I would prefer not using explicit version and rather put each
quirk in the driver data structure, so we can avoid those
multiple if (version == ...) checks all over in the code, should
more revision of this IP come in future SoCs.

Re: [RFC 03/12] media: fimc-lite: Adding support for Exynos5

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> This patch adds the following functionalities to existing driver
>
> 1] FIMC-LITE supports multiple DMA shadow registers from Exynos5 onwards.
> This patch adds the functionality of using shadow registers by
> checking the current FIMC-LITE hardware version.
> 2] Fixes Buffer corruption on DMA output from fimc-lite
> 3] Modified the driver to be used as pipeline endpoint

There seems to be too many things done in this single patch. Can
we have it split for example to the parts adding:
- registers definitions and hardware interface helpers
   for exynos5
- the DMA handling fix

So it is easier to apply it and test incrementally ?

> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> Signed-off-by: Arun Kumar K<arun.kk@samsung.com>
> ---
>   drivers/media/platform/s5p-fimc/fimc-lite-reg.c |   16 +-
>   drivers/media/platform/s5p-fimc/fimc-lite-reg.h |   41 ++++-
>   drivers/media/platform/s5p-fimc/fimc-lite.c     |  196 +++++++++++++++++++++--
>   drivers/media/platform/s5p-fimc/fimc-lite.h     |    3 +-
>   4 files changed, 236 insertions(+), 20 deletions(-)
...
>   void flite_hw_dump_regs(struct fimc_lite *dev, const char *label)
> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite-reg.h b/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
> index 0e34584..716df6c 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
> +++ b/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
> @@ -120,6 +120,10 @@
>   /* b0: 1 - camera B, 0 - camera A */
>   #define FLITE_REG_CIGENERAL_CAM_B		(1<<  0)
>
> +
> +#define FLITE_REG_CIFCNTSEQ			0x100
> +#define FLITE_REG_CIOSAN(x)			(0x200 + (4 * (x)))
> +
>   /* ----------------------------------------------------------------------------
>    * Function declarations
>    */
> @@ -143,8 +147,41 @@ void flite_hw_set_dma_window(struct fimc_lite *dev, struct flite_frame *f);
>   void flite_hw_set_test_pattern(struct fimc_lite *dev, bool on);
>   void flite_hw_dump_regs(struct fimc_lite *dev, const char *label);
>
> -static inline void flite_hw_set_output_addr(struct fimc_lite *dev, u32 paddr)
> +static inline void flite_hw_set_output_addr(struct fimc_lite *dev,
> +	u32 paddr, u32 index)
> +{
> +	u32 config;
> +
> +	/* FLITE in EXYNOS4 has only one DMA register */
> +	if (dev->variant->version == FLITE_VER_EXYNOS4)
> +		index = 0;
> +
> +	config = readl(dev->regs + FLITE_REG_CIFCNTSEQ);
> +	config |= 1<<  index;
> +	writel(config, dev->regs + FLITE_REG_CIFCNTSEQ);
> +
> +	if (index == 0)
> +		writel(paddr, dev->regs + FLITE_REG_CIOSA);
> +	else
> +		writel(paddr, dev->regs + FLITE_REG_CIOSAN(index-1));
> +}
> +
> +static inline void flite_hw_clear_output_addr(struct fimc_lite *dev, u32 index)
>   {
> -	writel(paddr, dev->regs + FLITE_REG_CIOSA);
> +	u32 config;
> +
> +	/* FLITE in EXYNOS4 has only one DMA register */
> +	if (dev->variant->version == FLITE_VER_EXYNOS4)
> +		index = 0;

I'm planning to remove struct flite_variant and put everything what's
needed in the driver data structure. Are there any differences between
FIMC-LITE IP block instances or Exynos5250 ? Or are these all same ?

That said it seems better to me to add a field like out_dma_bufs to the
driver data structure and embed a pointer to this structure in struct
fimc_lite. The driver data matching would be done automatically, based
on the compatible property and those unpleasant checks
if (variant->version == FLITE_VER_EXYNOS4) ...

> +
> +	config = readl(dev->regs + FLITE_REG_CIFCNTSEQ);
> +	config&= ~(1<<  index);
> +	writel(config, dev->regs + FLITE_REG_CIFCNTSEQ);
>   }
> +
> +static inline void flite_hw_clear_output_index(struct fimc_lite *dev)
> +{
> +	writel(0, dev->regs + FLITE_REG_CIFCNTSEQ);
> +}
> +
>   #endif /* FIMC_LITE_REG_H */
> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c b/drivers/media/platform/s5p-fimc/fimc-lite.c
> index eb64f87..1edc5ce 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-lite.c
> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
> @@ -136,6 +136,8 @@ static int fimc_lite_hw_init(struct fimc_lite *fimc, bool isp_output)
>   	if (fimc->fmt == NULL)
>   		return -EINVAL;
>
> +	flite_hw_clear_output_index(fimc);
> +
>   	/* Get sensor configuration data from the sensor subdev */
>   	src_info = v4l2_get_subdev_hostdata(sensor);
>   	spin_lock_irqsave(&fimc->slock, flags);
> @@ -266,19 +268,24 @@ static irqreturn_t flite_irq_handler(int irq, void *priv)
>
>   	if ((intsrc&  FLITE_REG_CISTATUS_IRQ_SRC_FRMSTART)&&
>   	test_bit(ST_FLITE_RUN,&fimc->state)&&
> -	    !list_empty(&fimc->active_buf_q)&&
>   	!list_empty(&fimc->pending_buf_q)) {
> +		vbuf = fimc_lite_pending_queue_pop(fimc);
> +		flite_hw_set_output_addr(fimc, vbuf->paddr,
> +					vbuf->vb.v4l2_buf.index);
> +		fimc_lite_active_queue_add(fimc, vbuf);
> +	}
> +
> +	if ((intsrc&  FLITE_REG_CISTATUS_IRQ_SRC_FRMEND)&&
> +	    test_bit(ST_FLITE_RUN,&fimc->state)&&
> +	    !list_empty(&fimc->active_buf_q)) {
>   		vbuf = fimc_lite_active_queue_pop(fimc);
>   		ktime_get_ts(&ts);
>   		tv =&vbuf->vb.v4l2_buf.timestamp;
>   		tv->tv_sec = ts.tv_sec;
>   		tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
>   		vbuf->vb.v4l2_buf.sequence = fimc->frame_count++;
> +		flite_hw_clear_output_addr(fimc, vbuf->vb.v4l2_buf.index);
>   		vb2_buffer_done(&vbuf->vb, VB2_BUF_STATE_DONE);
> -
> -		vbuf = fimc_lite_pending_queue_pop(fimc);
> -		flite_hw_set_output_addr(fimc, vbuf->paddr);
> -		fimc_lite_active_queue_add(fimc, vbuf);
>   	}
>
>   	if (test_bit(ST_FLITE_CONFIG,&fimc->state))
> @@ -406,7 +413,8 @@ static void buffer_queue(struct vb2_buffer *vb)
>   	if (!test_bit(ST_FLITE_SUSPENDED,&fimc->state)&&
>   	!test_bit(ST_FLITE_STREAM,&fimc->state)&&
>   	list_empty(&fimc->active_buf_q)) {
> -		flite_hw_set_output_addr(fimc, buf->paddr);
> +		flite_hw_set_output_addr(fimc, buf->paddr,
> +					buf->vb.v4l2_buf.index);
>   		fimc_lite_active_queue_add(fimc, buf);
>   	} else {
>   		fimc_lite_pending_queue_add(fimc, buf);
> @@ -646,7 +654,7 @@ static int fimc_vidioc_querycap_capture(struct file *file, void *priv,
>   	strlcpy(cap->driver, FIMC_LITE_DRV_NAME, sizeof(cap->driver));
>   	cap->bus_info[0] = 0;
>   	cap->card[0] = 0;
> -	cap->capabilities = V4L2_CAP_STREAMING;
> +	cap->capabilities = V4L2_CAP_STREAMING | V4L2_CAP_VIDEO_CAPTURE_MPLANE;
>   	return 0;
>   }
>
> @@ -725,13 +733,125 @@ static int fimc_lite_try_fmt_mplane(struct file *file, void *fh,
>   	return fimc_lite_try_fmt(fimc,&f->fmt.pix_mp, NULL);
>   }
>
> -static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
> -				  struct v4l2_format *f)
> +static struct media_entity *fimc_pipeline_get_head(struct media_entity *me)
>   {
> -	struct v4l2_pix_format_mplane *pixm =&f->fmt.pix_mp;
> -	struct fimc_lite *fimc = video_drvdata(file);
> +	struct media_pad *pad =&me->pads[0];
> +
> +	while (!(pad->flags&  MEDIA_PAD_FL_SOURCE)) {
> +		pad = media_entity_remote_source(pad);
> +		if (!pad)
> +			break;
> +		me = pad->entity;
> +		pad =&me->pads[0];
> +	}
> +
> +	return me;
> +}
> +
> +/**
> + * fimc_pipeline_try_format - negotiate and/or set formats at pipeline
> + *                            elements
> + * @ctx: FIMC capture context
> + * @tfmt: media bus format to try/set on subdevs
> + * @fmt_id: fimc pixel format id corresponding to returned @tfmt (output)
> + * @set: true to set format on subdevs, false to try only
> + */
> +static int fimc_pipeline_try_format(struct fimc_lite *fimc,
> +				    struct v4l2_mbus_framefmt *tfmt,
> +				    struct fimc_fmt **fmt_id,
> +				    bool set)
> +{
[...]
> +}
> +
> +
> +static int __fimc_lite_set_format(struct fimc_lite *fimc,
> +				     struct v4l2_format *f)
> +{
> +	struct v4l2_pix_format_mplane *pix_mp =&f->fmt.pix_mp;
>   	struct flite_frame *frame =&fimc->out_frame;
>   	const struct fimc_fmt *fmt = NULL;
> +	struct v4l2_mbus_framefmt mf;
> +	struct fimc_fmt *s_fmt = NULL;
>   	int ret;
>
>   	if (vb2_is_busy(&fimc->vb_queue))
> @@ -741,15 +861,59 @@ static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
>   	if (ret<  0)
>   		return ret;
>
> +	/* Reset cropping and set format at the camera interface input */
> +	if (!fimc->user_subdev_api) {
> +		fimc->inp_frame.f_width = pix_mp->width;
> +		fimc->inp_frame.f_height = pix_mp->height;
> +		fimc->inp_frame.rect.top = 0;
> +		fimc->inp_frame.rect.left = 0;
> +		fimc->inp_frame.rect.width = pix_mp->width;
> +		fimc->inp_frame.rect.height = pix_mp->height;
> +	}
> +
> +	/* Try to match format at the host and the sensor */
> +	if (!fimc->user_subdev_api) {
> +		mf.code   = fmt->mbus_code;
> +		mf.width  = pix_mp->width;
> +		mf.height = pix_mp->height;
> +		ret = fimc_pipeline_try_format(fimc,&mf,&s_fmt, true);
> +		if (ret)
> +			return ret;
> +
> +		pix_mp->width  = mf.width;
> +		pix_mp->height = mf.height;
> +	}
> +
>   	fimc->fmt = fmt;
> -	fimc->payload[0] = max((pixm->width * pixm->height * fmt->depth[0]) / 8,
> -			       pixm->plane_fmt[0].sizeimage);
> -	frame->f_width = pixm->width;
> -	frame->f_height = pixm->height;
> +	fimc->payload[0] = max((pix_mp->width * pix_mp->height *
> +			fmt->depth[0]) / 8, pix_mp->plane_fmt[0].sizeimage);
> +	frame->f_width = pix_mp->width;
> +	frame->f_height = pix_mp->height;
>
>   	return 0;
>   }
>
> +static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
> +				  struct v4l2_format *f)
> +{
> +	struct fimc_lite *fimc = video_drvdata(file);
> +	int ret;
> +
> +	exynos_pipeline_graph_lock(fimc->pipeline_ops,&fimc->pipeline);
> +	/*
> +	 * The graph is walked within __fimc_lite_set_format() to set
> +	 * the format at subdevs thus the graph mutex needs to be held at
> +	 * this point and acquired before the video mutex, to avoid  AB-BA
> +	 * deadlock when fimc_md_link_notify() is called by other thread.
> +	 * Ideally the graph walking and setting format at the whole pipeline
> +	 * should be removed from this driver and handled in userspace only.

You have copied this format setting code from fimc-capture.c, while I
would rather remove it from there as well. But it have to stay for
backward compatibility. Nevertheless on Exynos4x12 by default image
format on whole pipeline will not be set in kernel by the video capture
node driver, there is simply to many entities involved there and for
full flexibility format setting/negotiation at the whole pipeline needs
to be done by user space library. Or you can use media-ctl to configure
the formats at each subdev.


> --- a/drivers/media/platform/s5p-fimc/fimc-lite.h
> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.h
> @@ -52,7 +52,6 @@ enum {
>   #define FLITE_VER_EXYNOS4	0
>   #define FLITE_VER_EXYNOS5	1
>
> -
>   struct flite_variant {
>   	unsigned short max_width;
>   	unsigned short max_height;
> @@ -175,6 +174,8 @@ struct fimc_lite {
>   	unsigned int		reqbufs_count;
>   	int			ref_count;
>
> +	bool			user_subdev_api;

No, please don't copy this. I don't really want to see that sysfs
workaround in other drivers.

--

Regards,
Sylwester

Re: [RFC 04/12] s5p-csis: Adding Exynos5250 compatibility

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:

Please don't leave the change log empty. I'll apply this patch.
I'm just wondering, if there aren't any further changes needed
to make the driver really working on exynos5250 ?

> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> ---
>   drivers/media/platform/s5p-fimc/mipi-csis.c |    1 +
>   1 file changed, 1 insertion(+)
>
> diff --git a/drivers/media/platform/s5p-fimc/mipi-csis.c b/drivers/media/platform/s5p-fimc/mipi-csis.c
> index df4411c..debda7c 100644
> --- a/drivers/media/platform/s5p-fimc/mipi-csis.c
> +++ b/drivers/media/platform/s5p-fimc/mipi-csis.c
> @@ -1002,6 +1002,7 @@ static const struct dev_pm_ops s5pcsis_pm_ops = {
>   static const struct of_device_id s5pcsis_of_match[] __devinitconst = {
>   	{ .compatible = "samsung,exynos3110-csis" },
>   	{ .compatible = "samsung,exynos4210-csis" },
> +	{ .compatible = "samsung,exynos5250-csis" },
>   	{ /* sentinel */ },
>   };
>   MODULE_DEVICE_TABLE(of, s5pcsis_of_match);

Re: [RFC 05/12] ARM: EXYNOS: Add devicetree node for mipi-csis driver for exynos5

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> This patch adds necessary source definations needed for mipi-csis
> driver and adds devicetree node for exynos5250.
>
> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> ---
>   arch/arm/boot/dts/exynos5250.dtsi       |   18 ++++++++++++++++++
>   arch/arm/mach-exynos/clock-exynos5.c    |   16 ++++++++++++++--
>   arch/arm/mach-exynos/include/mach/map.h |    3 +++
>   arch/arm/mach-exynos/mach-exynos5-dt.c  |    4 ++++
>   4 files changed, 39 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/boot/dts/exynos5250.dtsi b/arch/arm/boot/dts/exynos5250.dtsi
> index 3a2cd9a..4fff98b 100644
> --- a/arch/arm/boot/dts/exynos5250.dtsi
> +++ b/arch/arm/boot/dts/exynos5250.dtsi
> @@ -47,6 +47,8 @@
>   		i2c6 =&i2c_6;
>   		i2c7 =&i2c_7;
>   		i2c8 =&i2c_8;
> +		csis0 =&csis_0;
> +		csis1 =&csis_1;
>   	};
>
>   	gic:interrupt-controller@10481000 {
> @@ -357,4 +359,20 @@
>   		reg =<0x14450000 0x10000>;
>   		interrupts =<0 94 0>;
>   	};
> +
> +	csis_0: csis@13C20000 {
> +		compatible = "samsung,exynos5250-csis";
> +		reg =<0x13C20000 0x4000>;
> +		interrupts =<0 79 0>;
> +		bus-width =<4>;
> +		status = "disabled";
> +	};
> +
> +	csis_1: csis@13C30000 {
> +		compatible = "samsung,exynos5250-csis";
> +		reg =<0x13C30000 0x4000>;
> +		interrupts =<0 80 0>;
> +		bus-width =<4>;

Shouldn't this be 2 ? Anyway what's the point of adding this node here
only to move it in a subsequent patch ? I guess you should first add
'camera' node and then have further patches adding relevant device nodes.

> +		status = "disabled";
> +	};
>   };

> diff --git a/arch/arm/mach-exynos/clock-exynos5.c b/arch/arm/mach-exynos/clock-exynos5.c
> index e9d7b80..34a22ff 100644
> --- a/arch/arm/mach-exynos/clock-exynos5.c
> +++ b/arch/arm/mach-exynos/clock-exynos5.c

This file is already removed in Kukjin's for-next tree.
And for dts changes I would start the patch summary line with "ARM: dts:".

> @@ -859,6 +859,16 @@ static struct clk exynos5_init_clocks_off[] = {
>   		.enable		= exynos5_clk_ip_gscl_ctrl,
>   		.ctrlbit	= (1<<  3),
>   	}, {
> +		.name		= "csis",
> +		.devname	= "s5p-mipi-csis.0",
> +		.enable		= exynos5_clk_ip_gscl_ctrl,
> +		.ctrlbit	= (1<<  5),
> +	}, {
> +		.name		= "csis",
> +		.devname	= "s5p-mipi-csis.1",
> +		.enable		= exynos5_clk_ip_gscl_ctrl,
> +		.ctrlbit	= (1<<  6),
> +	}, {

Re: [RFC 05/12] ARM: EXYNOS: Add devicetree node for mipi-csis driver for exynos5

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> --- a/arch/arm/boot/dts/exynos5250.dtsi
> +++ b/arch/arm/boot/dts/exynos5250.dtsi
> @@ -47,6 +47,8 @@
>   		i2c6 =&i2c_6;
>   		i2c7 =&i2c_7;
>   		i2c8 =&i2c_8;
> +		csis0 =&csis_0;
> +		csis1 =&csis_1;

You can drop these aliases if you use my latest patches as indicated
in the comment to patch 00/12.

> diff --git a/arch/arm/mach-exynos/clock-exynos5.c b/arch/arm/mach-exynos/clock-exynos5.c
> index e9d7b80..34a22ff 100644
> --- a/arch/arm/mach-exynos/clock-exynos5.c
> +++ b/arch/arm/mach-exynos/clock-exynos5.c
> @@ -859,6 +859,16 @@ static struct clk exynos5_init_clocks_off[] = {
>   		.enable		= exynos5_clk_ip_gscl_ctrl,
>   		.ctrlbit	= (1<<  3),
>   	}, {
> +		.name		= "csis",
> +		.devname	= "s5p-mipi-csis.0",
> +		.enable		= exynos5_clk_ip_gscl_ctrl,
> +		.ctrlbit	= (1<<  5),
> +	}, {

Instead you should add relevant clock definitions to the Samsung clocks 
driver,
it's already merged in Kukjin's tree.

Re: [RFC 01/12] media: s5p-fimc: modify existing mdev to use common pipeline

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> This patch modifies the current fimc_pipeline to exynos_pipeline,

I think we could leave it as fimc_pipeline, exynos_pipeline seems
too generic to me.

> which can be used across multiple media device drivers.
> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> ---
>   drivers/media/platform/s5p-fimc/fimc-capture.c |   96 +++++++-----
>   drivers/media/platform/s5p-fimc/fimc-core.h    |    4 +-
>   drivers/media/platform/s5p-fimc/fimc-lite.c    |   73 ++++------
>   drivers/media/platform/s5p-fimc/fimc-lite.h    |    4 +-
>   drivers/media/platform/s5p-fimc/fimc-mdevice.c |  186 ++++++++++++++++++++++--
>   drivers/media/platform/s5p-fimc/fimc-mdevice.h |   41 +++---
>   include/media/s5p_fimc.h                       |   66 ++++++---
>   7 files changed, 326 insertions(+), 144 deletions(-)
>
> diff --git a/drivers/media/platform/s5p-fimc/fimc-capture.c b/drivers/media/platform/s5p-fimc/fimc-capture.c
> index 4cbaf46..106466e 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-capture.c
> +++ b/drivers/media/platform/s5p-fimc/fimc-capture.c
> @@ -27,24 +27,26 @@
>   #include<media/videobuf2-core.h>
>   #include<media/videobuf2-dma-contig.h>
>
> -#include "fimc-mdevice.h"
>   #include "fimc-core.h"
>   #include "fimc-reg.h"
>
>   static int fimc_capture_hw_init(struct fimc_dev *fimc)
>   {
>   	struct fimc_ctx *ctx = fimc->vid_cap.ctx;
> -	struct fimc_pipeline *p =&fimc->pipeline;
> +	struct exynos_pipeline *p =&fimc->pipeline;
>   	struct fimc_sensor_info *sensor;
>   	unsigned long flags;
> +	struct v4l2_subdev *sd;
>   	int ret = 0;
>
> -	if (p->subdevs[IDX_SENSOR] == NULL || ctx == NULL)
> +	sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
> +					get_subdev_sensor, p);

Hmm, it feels it is going wrong path this way. I would keep changes
to the s5p-fimc driver as small as possible. And the modules that
are shared across the exynos4 and exynos5 driver should use generic
media graph walking routines where possible.

> +	if (sd == NULL || ctx == NULL)
>   		return -ENXIO;
>   	if (ctx->s_frame.fmt == NULL)
>   		return -EINVAL;
>
> -	sensor = v4l2_get_subdev_hostdata(p->subdevs[IDX_SENSOR]);
> +	sensor = v4l2_get_subdev_hostdata(sd);
>
>   	spin_lock_irqsave(&fimc->slock, flags);
>   	fimc_prepare_dma_offset(ctx,&ctx->d_frame);
...
> @@ -486,9 +491,12 @@ static struct vb2_ops fimc_capture_qops = {
>   int fimc_capture_ctrls_create(struct fimc_dev *fimc)
>   {
>   	struct fimc_vid_cap *vid_cap =&fimc->vid_cap;
> -	struct v4l2_subdev *sensor = fimc->pipeline.subdevs[IDX_SENSOR];
> +	struct v4l2_subdev *sensor;
>   	int ret;
>
> +	sensor = exynos_pipeline_get_subdev(fimc->pipeline_ops,
> +					get_subdev_sensor,&fimc->pipeline);
> +
>   	if (WARN_ON(vid_cap->ctx == NULL))
>   		return -ENXIO;
>   	if (vid_cap->ctx->ctrls.ready)
> @@ -513,7 +521,7 @@ static int fimc_capture_open(struct file *file)
>
>   	dbg("pid: %d, state: 0x%lx", task_pid_nr(current), fimc->state);
>
> -	fimc_md_graph_lock(fimc);
> +	exynos_pipeline_graph_lock(fimc->pipeline_ops,&fimc->pipeline);

Hmm, this look pretty scary to me. But I suspect this change is not
needed at all. The graph lock is _not_ the pipeline lock. It protects
all media entities registered to the media device, and links between
them. Not only entities linked into specific video processing pipeline
at a moment.

>   	mutex_lock(&fimc->lock);
>
>   	if (fimc_m2m_active(fimc))
> @@ -531,7 +539,7 @@ static int fimc_capture_open(struct file *file)
>   	}
>
>   	if (++fimc->vid_cap.refcnt == 1) {
> -		ret = fimc_pipeline_call(fimc, open,&fimc->pipeline,
> +		ret = exynos_pipeline_call(fimc, open,&fimc->pipeline,
>   					&fimc->vid_cap.vfd.entity, true);
>
>   		if (!ret&&  !fimc->vid_cap.user_subdev_api)
> @@ -549,7 +557,7 @@ static int fimc_capture_open(struct file *file)
>   	}
>   unlock:
>   	mutex_unlock(&fimc->lock);
> -	fimc_md_graph_unlock(fimc);
> +	exynos_pipeline_graph_unlock(fimc->pipeline_ops,&fimc->pipeline);
>   	return ret;
>   }
...
>   /**
> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c b/drivers/media/platform/s5p-fimc/fimc-lite.c
> index 3266c3f..122cf95 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-lite.c
> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
...
> -/* Called with the media graph mutex held */
> -static struct v4l2_subdev *__find_remote_sensor(struct media_entity *me)
> -{
> -	struct media_pad *pad =&me->pads[0];
> -	struct v4l2_subdev *sd;
> -
> -	while (pad->flags&  MEDIA_PAD_FL_SINK) {
> -		/* source pad */
> -		pad = media_entity_remote_source(pad);
> -		if (pad == NULL ||
> -		    media_entity_type(pad->entity) != MEDIA_ENT_T_V4L2_SUBDEV)
> -			break;
> -
> -		sd = media_entity_to_v4l2_subdev(pad->entity);
> -
> -		if (sd->grp_id == GRP_ID_FIMC_IS_SENSOR)
> -			return sd;
> -		/* sink pad */
> -		pad =&sd->entity.pads[0];
> -	}
> -	return NULL;
> -}

What's wrong with this function ? Why doesn't it work for you ?
I think we should drop direct usage of fimc->pipeline->subdevs[IDX_SENSOR]
instead. The sensor group IDs should probably be moved to some common
header file.

> diff --git a/drivers/media/platform/s5p-fimc/fimc-mdevice.c b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
> index fc93fad..938cc56 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-mdevice.c
> +++ b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
> @@ -36,6 +36,8 @@
>   #include "fimc-mdevice.h"
>   #include "mipi-csis.h"
>
> +static struct fimc_md *g_fimc_mdev;

Hm, this is pretty ugly. Can you explain why it is needed ?

>   static int __fimc_md_set_camclk(struct fimc_md *fmd,
>   				struct fimc_sensor_info *s_info,
>   				bool on);
> @@ -143,6 +145,73 @@ static int fimc_pipeline_s_power(struct fimc_pipeline *p, bool state)
>   }
>
>   /**
> + * __fimc_pipeline_init
> + *      allocate the fimc_pipeline structure and do the basic initialization

This is not a proper kernel-doc style.

> + */
> +static int __fimc_pipeline_init(struct exynos_pipeline *ep)
> +{
> +	struct fimc_pipeline *p;
> +
> +	p = kzalloc(sizeof(*p), GFP_KERNEL);
> +	if (!p)
> +		return -ENOMEM;

Why this needs to be allocated dynamically ?

> +	p->is_init = true;
> +	p->fmd = g_fimc_mdev;
> +	ep->priv = (void *)p;
> +	return 0;
> +}
> +
> +/**
> + * __fimc_pipeline_deinit
> + *      free the allocated resources for fimc_pipeline
> + */
> +static int __fimc_pipeline_deinit(struct exynos_pipeline *ep)
> +{
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
> +
> +	if (!p || !p->is_init)
> +		return -EINVAL;
> +
> +	p->is_init = false;
> +	kfree(p);
> +
> +	return 0;
> +}
> +
> +/**
> + * __fimc_pipeline_get_subdev_sensor
> + *      if valid pipeline, returns the sensor subdev pointer
> + *      else returns NULL
> + */
> +static struct v4l2_subdev *__fimc_pipeline_get_subdev_sensor(
> +					struct exynos_pipeline *ep)
> +{
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
> +
> +	if (!p || !p->is_init)
> +		return NULL;
> +
> +	return p->subdevs[IDX_SENSOR];
> +}
> +
> +/**
> + * __fimc_pipeline_get_subdev_csis
> + *      if valid pipeline, returns the csis subdev pointer
> + *      else returns NULL
> + */
> +static struct v4l2_subdev *__fimc_pipeline_get_subdev_csis(
> +					struct exynos_pipeline *ep)
> +{
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
> +
> +	if (!p || !p->is_init)
> +		return NULL;
> +
> +	return p->subdevs[IDX_CSIS];
> +}
> +
> +/**
>    * __fimc_pipeline_open - update the pipeline information, enable power
>    *                        of all pipeline subdevs and the sensor clock
>    * @me: media entity to start graph walk with
> @@ -150,11 +219,15 @@ static int fimc_pipeline_s_power(struct fimc_pipeline *p, bool state)
>    *
>    * Called with the graph mutex held.
>    */
> -static int __fimc_pipeline_open(struct fimc_pipeline *p,
> +static int __fimc_pipeline_open(struct exynos_pipeline *ep,
>   				struct media_entity *me, bool prep)
>   {
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>   	int ret;
>
> +	if (!p || !p->is_init)
> +		return -EINVAL;
> +
>   	if (prep)
>   		fimc_pipeline_prepare(p, me);
>
> @@ -174,17 +247,20 @@ static int __fimc_pipeline_open(struct fimc_pipeline *p,
>    *
>    * Disable power of all subdevs and turn the external sensor clock off.
>    */
> -static int __fimc_pipeline_close(struct fimc_pipeline *p)
> +static int __fimc_pipeline_close(struct exynos_pipeline *ep)
>   {
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>   	int ret = 0;
>
> -	if (!p || !p->subdevs[IDX_SENSOR])
> +	if (!p || !p->is_init)
>   		return -EINVAL;
>
> -	if (p->subdevs[IDX_SENSOR]) {
> -		ret = fimc_pipeline_s_power(p, 0);
> -		fimc_md_set_camclk(p->subdevs[IDX_SENSOR], false);
> -	}
> +	if (!p->subdevs[IDX_SENSOR])
> +		return -EINVAL;
> +
> +	ret = fimc_pipeline_s_power(p, 0);
> +	fimc_md_set_camclk(p->subdevs[IDX_SENSOR], false);
> +
>   	return ret == -ENXIO ? 0 : ret;
>   }
>
> @@ -193,10 +269,14 @@ static int __fimc_pipeline_close(struct fimc_pipeline *p)
>    * @pipeline: video pipeline structure
>    * @on: passed as the s_stream call argument
>    */
> -static int __fimc_pipeline_s_stream(struct fimc_pipeline *p, bool on)
> +static int __fimc_pipeline_s_stream(struct exynos_pipeline *ep, bool on)
>   {
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>   	int i, ret;
>
> +	if (!p || !p->is_init)
> +		return -EINVAL;
> +
>   	if (p->subdevs[IDX_SENSOR] == NULL)
>   		return -ENODEV;
>
> @@ -213,11 +293,47 @@ static int __fimc_pipeline_s_stream(struct fimc_pipeline *p, bool on)
>
>   }
>
> +static void __fimc_pipeline_graph_lock(struct exynos_pipeline *ep)
> +{
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
> +	struct fimc_md *fmd = p->fmd;
> +
> +	mutex_lock(&fmd->media_dev.graph_mutex);
> +}
> +
> +static void __fimc_pipeline_graph_unlock(struct exynos_pipeline *ep)
> +{
> +	struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
> +	struct fimc_md *fmd = p->fmd;
> +
> +	mutex_unlock(&fmd->media_dev.graph_mutex);
> +}

This seems really wrong to me. You can reference the media graph mutex
from a subdev or video device through its 'entity' member, e.g.
sd->entity.parent->graph_mutex.

...
>   static int fimc_md_probe(struct platform_device *pdev)
>   {
>   	struct device *dev =&pdev->dev;
> @@ -1175,7 +1327,7 @@ static int fimc_md_probe(struct platform_device *pdev)
>
>   	v4l2_dev =&fmd->v4l2_dev;
>   	v4l2_dev->mdev =&fmd->media_dev;
> -	v4l2_dev->notify = fimc_sensor_notify;
> +	v4l2_dev->notify = fimc_md_sensor_notify;
>   	strlcpy(v4l2_dev->name, "s5p-fimc-md", sizeof(v4l2_dev->name));
>
>
> @@ -1194,6 +1346,7 @@ static int fimc_md_probe(struct platform_device *pdev)
>   		goto err_clk;
>
>   	fmd->user_subdev_api = (dev->of_node != NULL);
> +	g_fimc_mdev = fmd;
>
>   	/* Protect the media graph while we're registering entities */
>   	mutex_lock(&fmd->media_dev.graph_mutex);
> @@ -1252,6 +1405,7 @@ static int fimc_md_remove(struct platform_device *pdev)
>
>   	if (!fmd)
>   		return 0;
> +	g_fimc_mdev = NULL;
>   	device_remove_file(&pdev->dev,&dev_attr_subdev_conf_mode);
>   	fimc_md_unregister_entities(fmd);
>   	media_device_unregister(&fmd->media_dev);
> diff --git a/drivers/media/platform/s5p-fimc/fimc-mdevice.h b/drivers/media/platform/s5p-fimc/fimc-mdevice.h
> index f3e0251..1ea7acf 100644
> --- a/drivers/media/platform/s5p-fimc/fimc-mdevice.h
> +++ b/drivers/media/platform/s5p-fimc/fimc-mdevice.h
> @@ -37,6 +37,15 @@
>   #define FIMC_MAX_SENSORS	8
>   #define FIMC_MAX_CAMCLKS	2
>
> +enum fimc_subdev_index {
> +	IDX_SENSOR,
> +	IDX_CSIS,
> +	IDX_FLITE,
> +	IDX_IS_ISP,
> +	IDX_FIMC,
> +	IDX_MAX,
> +};
> +
>   struct fimc_csis_info {
>   	struct v4l2_subdev *sd;
>   	int id;
> @@ -49,20 +58,6 @@ struct fimc_camclk_info {
>   };
>
>   /**
> - * struct fimc_sensor_info - image data source subdev information
> - * @pdata: sensor's atrributes passed as media device's platform data
> - * @subdev: image sensor v4l2 subdev
> - * @host: fimc device the sensor is currently linked to
> - *
> - * This data structure applies to image sensor and the writeback subdevs.
> - */
> -struct fimc_sensor_info {
> -	struct fimc_source_info pdata;
> -	struct v4l2_subdev *subdev;
> -	struct fimc_dev *host;
> -};
> -
> -/**
>    * struct fimc_md - fimc media device information
>    * @csis: MIPI CSIS subdevs data
>    * @sensor: array of registered sensor subdevs
> @@ -89,6 +84,14 @@ struct fimc_md {
>   	spinlock_t slock;
>   };
>
> +struct fimc_pipeline {
> +	int is_init;
> +	struct fimc_md *fmd;
> +	struct v4l2_subdev *subdevs[IDX_MAX];
> +	void (*sensor_notify)(struct v4l2_subdev *sd,
> +			unsigned int notification, void *arg);
> +};

This is supposed to be the s5p-fimc driver specific only data structure,
right ?

>   #define is_subdev_pad(pad) (pad == NULL || \
>   	media_entity_type(pad->entity) == MEDIA_ENT_T_V4L2_SUBDEV)
>
> @@ -103,16 +106,6 @@ static inline struct fimc_md *entity_to_fimc_mdev(struct media_entity *me)
>   		container_of(me->parent, struct fimc_md, media_dev);
>   }
>
> -static inline void fimc_md_graph_lock(struct fimc_dev *fimc)
> -{
> -	mutex_lock(&fimc->vid_cap.vfd.entity.parent->graph_mutex);
> -}
> -
> -static inline void fimc_md_graph_unlock(struct fimc_dev *fimc)
> -{
> -	mutex_unlock(&fimc->vid_cap.vfd.entity.parent->graph_mutex);
> -}

Why to remove these s5p-fimc driver private functions ?

>   int fimc_md_set_camclk(struct v4l2_subdev *sd, bool on);
>
>   #endif
> diff --git a/include/media/s5p_fimc.h b/include/media/s5p_fimc.h
> index e2434bb..007e998 100644
> --- a/include/media/s5p_fimc.h
> +++ b/include/media/s5p_fimc.h
> @@ -13,6 +13,7 @@
>   #define S5P_FIMC_H_
>
>   #include<media/media-entity.h>
> +#include<media/v4l2-subdev.h>
>
>   /*
>    * Enumeration of data inputs to the camera subsystem.
> @@ -75,6 +76,20 @@ struct fimc_source_info {
>   };
>
>   /**
> + * struct fimc_sensor_info - image data source subdev information
> + * @pdata: sensor's atrributes passed as media device's platform data
> + * @subdev: image sensor v4l2 subdev
> + * @host: capture device the sensor is currently linked to
> + *
> + * This data structure applies to image sensor and the writeback subdevs.
> + */
> +struct fimc_sensor_info {
> +	struct fimc_source_info pdata;
> +	struct v4l2_subdev *subdev;
> +	void *host;
> +};
> +
> +/**
>    * struct s5p_platform_fimc - camera host interface platform data
>    *
>    * @source_info: properties of an image source for the host interface setup
> @@ -93,21 +108,10 @@ struct s5p_platform_fimc {
>    */
>   #define S5P_FIMC_TX_END_NOTIFY _IO('e', 0)
>
> -enum fimc_subdev_index {
> -	IDX_SENSOR,
> -	IDX_CSIS,
> -	IDX_FLITE,
> -	IDX_IS_ISP,
> -	IDX_FIMC,
> -	IDX_MAX,
> -};
> -
> -struct media_pipeline;
> -struct v4l2_subdev;
>
> -struct fimc_pipeline {
> -	struct v4l2_subdev *subdevs[IDX_MAX];
> -	struct media_pipeline *m_pipeline;
> +struct exynos_pipeline {
> +	struct media_pipeline m_pipeline;
> +	void *priv;
>   };
>
>   /*
> @@ -115,15 +119,39 @@ struct fimc_pipeline {
>    * video node when it is the last entity of the pipeline. Implemented
>    * by corresponding media device driver.
>    */
> -struct fimc_pipeline_ops {
> -	int (*open)(struct fimc_pipeline *p, struct media_entity *me,
> +struct exynos_pipeline_ops {
> +	int (*init) (struct exynos_pipeline *p);
> +	int (*deinit) (struct exynos_pipeline *p);

How about naming it 'free' instead ?

> +	int (*open)(struct exynos_pipeline *p, struct media_entity *me,
>   			  bool resume);
> -	int (*close)(struct fimc_pipeline *p);
> -	int (*set_stream)(struct fimc_pipeline *p, bool state);
> +	int (*close)(struct exynos_pipeline *p);
> +	int (*set_stream)(struct exynos_pipeline *p, bool state);
> +	void (*graph_lock)(struct exynos_pipeline *p);
> +	void (*graph_unlock)(struct exynos_pipeline *p);

Why do you think these graph callbacks are needed here ?

> +	struct v4l2_subdev *(*get_subdev_sensor)(struct exynos_pipeline *p);
> +	struct v4l2_subdev *(*get_subdev_csis)(struct exynos_pipeline *p);

No, we should instead use generic media graph walking routines in the
shared drivers (FIMC-LITE, MIPI-CSIS). FIMC driver could be making
certain assumptions about it's related media device driver. These
drivers are contained in a single module anyway.

> +	void (*register_notify_cb)(struct exynos_pipeline *p,
> +		void (*cb)(struct v4l2_subdev *sd,
> +				unsigned int notification, void *arg));

Hmm, I need to think more about this. AFAIR this would be only needed
for S5P/Exynos4 FIMC.

Re: [RFC 07/12] media: exynos5-is: Adding media device driver for exynos5

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> This patch adds support for media device for EXYNOS5 SoCs.
> The current media device supports the following ips to connect
> through the media controller framework.
>
> * MIPI-CSIS
>    Support interconnection(subdev interface) between devices
>
> * FIMC-LITE
>    Support capture interface from device(Sensor, MIPI-CSIS) to memory
>    Support interconnection(subdev interface) between devices
>
> G-Scaler will be added later to the current media device.
>
> * Gscaler: general scaler
>    Support memory to memory interface
>    Support output interface from memory to display device(LCD, TV)
>    Support capture interface from device(FIMC-LITE, FIMD) to memory
>
> -->  media 0
>    Camera Capture path consists of MIPI-CSIS, FIMC-LITE and G-Scaler
>    +--------+     +-----------+     +-----------------+
>    | Sensor | -->  | FIMC-LITE | -->  | G-Scaler-capture |
>    +--------+     +-----------+     +-----------------+
>
>    +--------+     +-----------+     +-----------+     +-----------------+
>    | Sensor | -->  | MIPI-CSIS | -->  | FIMC-LITE | -->  | G-Scaler-capture |
>    +--------+     +-----------+     +-----------+     +-----------------+
>
> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> ---
>   drivers/media/platform/Kconfig                   |    1 +
>   drivers/media/platform/Makefile                  |    1 +
>   drivers/media/platform/exynos5-is/Kconfig        |    7 +
>   drivers/media/platform/exynos5-is/Makefile       |    4 +
>   drivers/media/platform/exynos5-is/exynos5-mdev.c | 1309 ++++++++++++++++++++++
>   drivers/media/platform/exynos5-is/exynos5-mdev.h |  107 ++
>   6 files changed, 1429 insertions(+)
>   create mode 100644 drivers/media/platform/exynos5-is/Kconfig
>   create mode 100644 drivers/media/platform/exynos5-is/Makefile
>   create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.c
>   create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.h
>
...
> diff --git a/drivers/media/platform/exynos5-is/exynos5-mdev.c b/drivers/media/platform/exynos5-is/exynos5-mdev.c
> new file mode 100644
> index 0000000..1158696
> --- /dev/null
> +++ b/drivers/media/platform/exynos5-is/exynos5-mdev.c
> @@ -0,0 +1,1309 @@
> +/*
> + * S5P/EXYNOS4 SoC series camera host interface media device driver

EXYNOS5

> + * Copyright (C) 2011 - 2012 Samsung Electronics Co., Ltd.
> + * Sylwester Nawrocki<s.nawrocki@samsung.com>

This is incorrect too, you should add your authorship and a note it
is based on other code, with a proper copyright notice.

> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published
> + * by the Free Software Foundation, either version 2 of the License,
> + * or (at your option) any later version.
> + */
> +
> +#include<linux/bug.h>
> +#include<linux/device.h>
> +#include<linux/errno.h>
> +#include<linux/i2c.h>
> +#include<linux/kernel.h>
> +#include<linux/list.h>
> +#include<linux/module.h>
> +#include<linux/of.h>
> +#include<linux/of_platform.h>
> +#include<linux/of_device.h>
> +#include<linux/of_i2c.h>
> +#include<linux/pinctrl/consumer.h>
> +#include<linux/platform_device.h>
> +#include<linux/pm_runtime.h>
> +#include<linux/types.h>
> +#include<linux/slab.h>
> +#include<media/v4l2-ctrls.h>
> +#include<media/v4l2-of.h>
> +#include<media/media-device.h>
> +
> +#include "exynos5-mdev.h"
> +
> +#define dbg(fmt, args...) \
> +	pr_debug("%s:%d: " fmt "\n", __func__, __LINE__, ##args)
> +
> +static struct fimc_md *g_exynos_mdev;
> +
> +static int fimc_md_set_camclk(struct v4l2_subdev *sd, bool on);
> +static int __fimc_md_set_camclk(struct fimc_md *fmd,
> +				struct fimc_sensor_info *s_info,
> +				bool on);
> +/**
> + * fimc_pipeline_prepare - update pipeline information with subdevice pointers
> + * @fimc: fimc device terminating the pipeline
> + *
> + * Caller holds the graph mutex.
> + */
> +static void fimc_pipeline_prepare(struct exynos5_pipeline0 *p,
> +				  struct media_entity *me)
> +{
> +	struct media_pad *pad =&me->pads[0];

This will need to be changed to support subdevs with more than 2 pads.
I should post relevant patch this week.

> +	struct v4l2_subdev *sd;
> +	int i;
> +
> +	for (i = 0; i<  IDX_MAX; i++)
> +		p->subdevs[i] = NULL;
> +
> +	while (1) {
> +
> +		if (!(pad->flags&  MEDIA_PAD_FL_SINK))
> +			break;
> +
> +		/* source pad */
> +		pad = media_entity_remote_source(pad);
> +
> +		if (pad != NULL)
> +			pr_err("entity type: %d, entity name: %s\n",
> +			    media_entity_type(pad->entity), pad->entity->name);
> +
> +		if (pad == NULL ||
> +		    media_entity_type(pad->entity) != MEDIA_ENT_T_V4L2_SUBDEV)
> +			break;
> +
> +		sd = media_entity_to_v4l2_subdev(pad->entity);
> +
> +		switch (sd->grp_id) {
> +		case GRP_ID_FIMC_IS_SENSOR:
> +		case GRP_ID_SENSOR:
> +			p->subdevs[IDX_SENSOR] = sd;
> +			break;
> +		case GRP_ID_CSIS:
> +			p->subdevs[IDX_CSIS] = sd;
> +			break;
> +		case GRP_ID_FLITE:
> +			p->subdevs[IDX_FLITE] = sd;
> +			break;
> +		default:
> +			pr_warn("%s: Unknown subdev grp_id: %#x\n",
> +				__func__, sd->grp_id);
> +		}
> +
> +		/* sink pad */
> +		pad =&sd->entity.pads[0];
> +	}
> +}
...
> +/*
> + * Sensor subdevice helper functions
> + */
> +static struct v4l2_subdev *fimc_md_register_sensor(struct fimc_md *fmd,
> +				   struct fimc_sensor_info *s_info)
> +{
> +	struct i2c_adapter *adapter;
> +	struct v4l2_subdev *sd = NULL;
> +
> +	if (!s_info || !fmd)
> +		return NULL;
> +
> +	adapter = i2c_get_adapter(s_info->pdata.i2c_bus_num);
> +	if (!adapter) {
> +		v4l2_warn(&fmd->v4l2_dev,
> +			  "Failed to get I2C adapter %d, deferring probe\n",
> +			  s_info->pdata.i2c_bus_num);
> +		return ERR_PTR(-EPROBE_DEFER);
> +	}
> +	sd = v4l2_i2c_new_subdev_board(&fmd->v4l2_dev, adapter,
> +				       s_info->pdata.board_info, NULL);
> +	if (IS_ERR_OR_NULL(sd)) {
> +		i2c_put_adapter(adapter);
> +		v4l2_warn(&fmd->v4l2_dev,
> +			  "Failed to acquire subdev %s, deferring probe\n",
> +			  s_info->pdata.board_info->type);
> +		return ERR_PTR(-EPROBE_DEFER);
> +	}
> +	v4l2_set_subdev_hostdata(sd, s_info);
> +	sd->grp_id = GRP_ID_SENSOR;
> +
> +	v4l2_info(&fmd->v4l2_dev, "Registered sensor subdevice %s\n",
> +		  sd->name);
> +	return sd;
> +}
> +
> +static void fimc_md_unregister_sensor(struct v4l2_subdev *sd)
> +{
> +	struct i2c_client *client = v4l2_get_subdevdata(sd);
> +	struct i2c_adapter *adapter;
> +
> +	if (!client)
> +		return;
> +	v4l2_device_unregister_subdev(sd);
> +
> +	if (!client->dev.of_node) {
> +		adapter = client->adapter;
> +		i2c_unregister_device(client);
> +		if (adapter)
> +			i2c_put_adapter(adapter);
> +	}
> +}

You don't need the above code which is for !CONFIG_OF only.

> +#ifdef CONFIG_OF

Just add "depends on OF" in Kconfig for the whole driver and
you can drop all #ifdef CONFIG_OF.

> +/* Register I2C client subdev associated with @node. */
> +static int fimc_md_of_add_sensor(struct fimc_md *fmd,
> +				 struct device_node *node, int index)
> +{
> +	struct fimc_sensor_info *si;
> +	struct i2c_client *client;
> +	struct v4l2_subdev *sd;
> +	int ret;
> +
> +	if (WARN_ON(index>= ARRAY_SIZE(fmd->sensor)))
> +		return -EINVAL;
> +
> +	si =&fmd->sensor[index];
> +
> +	client = of_find_i2c_device_by_node(node);
> +	if (!client)
> +		return -EPROBE_DEFER;
> +
> +	device_lock(&client->dev);
> +
> +	if (!client->driver ||
> +	    !try_module_get(client->driver->driver.owner)) {
> +		ret = -EPROBE_DEFER;
> +		goto dev_put;
> +	}
> +
> +	/* Enable sensor's master clock */
> +	ret = __fimc_md_set_camclk(fmd, si, true);
> +	if (ret<  0)
> +		goto mod_put;
> +
> +	sd = i2c_get_clientdata(client);
> +
> +	ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
> +	__fimc_md_set_camclk(fmd, si, false);
> +	if (ret<  0)
> +		goto mod_put;
> +
> +	v4l2_set_subdev_hostdata(sd, si);
> +	sd->grp_id = GRP_ID_SENSOR;
> +	si->subdev = sd;
> +	v4l2_info(&fmd->v4l2_dev, "Registered sensor subdevice: %s (%d)\n",
> +		  sd->name, fmd->num_sensors);
> +	fmd->num_sensors++;
> +
> +mod_put:
> +	module_put(client->driver->driver.owner);
> +dev_put:
> +	device_unlock(&client->dev);
> +	put_device(&client->dev);
> +	return ret;
> +}
...
> +#else
> +#define fimc_md_of_sensors_register(fmd, np) (-ENOSYS)

This would never be used since Exynos5 is a dt-only platform.

> +#endif
> +
> +static int fimc_md_register_sensor_entities(struct fimc_md *fmd)
> +{
> +	struct s5p_platform_fimc *pdata = fmd->pdev->dev.platform_data;
> +	struct device_node *of_node = fmd->pdev->dev.of_node;
> +	int num_clients = 0;
> +	int ret, i;
> +
> +	if (of_node) {
> +		fmd->num_sensors = 0;
> +		ret = fimc_md_of_sensors_register(fmd, of_node);
> +	} else if (pdata) {

The code below is for !CONFIG_OF only, no need to repeat it for Exynos5.

> +		WARN_ON(pdata->num_clients>  ARRAY_SIZE(fmd->sensor));
> +		num_clients = min_t(u32, pdata->num_clients,
> +				    ARRAY_SIZE(fmd->sensor));
> +		fmd->num_sensors = num_clients;
> +
> +		fmd->num_sensors = num_clients;
> +		for (i = 0; i<  num_clients; i++) {
> +			struct v4l2_subdev *sd;
> +
> +			fmd->sensor[i].pdata = pdata->source_info[i];
> +			ret = __fimc_md_set_camclk(fmd,&fmd->sensor[i], true);
> +			if (ret)
> +				break;
> +			sd = fimc_md_register_sensor(fmd,&fmd->sensor[i]);
> +			ret = __fimc_md_set_camclk(fmd,&fmd->sensor[i], false);
> +
> +			if (IS_ERR(sd)) {
> +				fmd->sensor[i].subdev = NULL;
> +				ret = PTR_ERR(sd);
> +				break;
> +			}
> +			fmd->sensor[i].subdev = sd;
> +			if (ret)
> +				break;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +/*
> + * MIPI-CSIS, FIMC and FIMC-LITE platform devices registration.
> + */
> +
> +static int register_fimc_lite_entity(struct fimc_md *fmd,
> +				     struct fimc_lite *fimc_lite)
> +{
> +	struct v4l2_subdev *sd;
> +	int ret;
> +
> +	if (WARN_ON(fimc_lite->index>= FIMC_LITE_MAX_DEVS ||
> +		    fmd->fimc_lite[fimc_lite->index]))
> +		return -EBUSY;
> +
> +	sd =&fimc_lite->subdev;
> +	sd->grp_id = GRP_ID_FLITE;
> +	v4l2_set_subdev_hostdata(sd, (void *)&exynos5_pipeline0_ops);
> +
> +	ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
> +	if (!ret)
> +		fmd->fimc_lite[fimc_lite->index] = fimc_lite;
> +	else
> +		v4l2_err(&fmd->v4l2_dev, "Failed to register FIMC.LITE%d\n",
> +			 fimc_lite->index);
> +	return ret;
> +}
> +
> +static int register_csis_entity(struct fimc_md *fmd,
> +				struct platform_device *pdev,
> +				struct v4l2_subdev *sd)
> +{
> +	struct device_node *node = pdev->dev.of_node;
> +	int id, ret;
> +
> +	id = node ? of_alias_get_id(node, "csis") : max(0, pdev->id);
> +
> +	if (WARN_ON(id>= CSIS_MAX_ENTITIES || fmd->csis[id].sd))
> +		return -EBUSY;
> +
> +	if (WARN_ON(id>= CSIS_MAX_ENTITIES))
> +		return 0;
> +
> +	sd->grp_id = GRP_ID_CSIS;
> +	ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
> +	if (!ret)
> +		fmd->csis[id].sd = sd;
> +	else
> +		v4l2_err(&fmd->v4l2_dev,
> +			 "Failed to register MIPI-CSIS.%d (%d)\n", id, ret);
> +
> +	return ret;
> +}
> +
> +static int fimc_md_register_platform_entity(struct fimc_md *fmd,
> +					    struct platform_device *pdev,
> +					    int plat_entity)
> +{
> +	struct device *dev =&pdev->dev;
> +	int ret = -EPROBE_DEFER;
> +	void *drvdata;
> +
> +	/* Lock to ensure dev->driver won't change. */
> +	device_lock(dev);
> +
> +	if (!dev->driver || !try_module_get(dev->driver->owner))
> +		goto dev_unlock;
> +
> +	drvdata = dev_get_drvdata(dev);
> +	/* Some subdev didn't probe succesfully id drvdata is NULL */
> +	if (drvdata) {
> +		switch (plat_entity) {
> +		case IDX_FLITE:
> +			ret = register_fimc_lite_entity(fmd, drvdata);
> +			break;
> +		case IDX_CSIS:
> +			ret = register_csis_entity(fmd, pdev, drvdata);
> +			break;
> +		default:
> +			ret = -ENODEV;
> +		}
> +	}
> +
> +	module_put(dev->driver->owner);
> +dev_unlock:
> +	device_unlock(dev);
> +	if (ret == -EPROBE_DEFER)
> +		dev_info(&fmd->pdev->dev, "deferring %s device registration\n",
> +			dev_name(dev));
> +	else if (ret<  0)
> +		dev_err(&fmd->pdev->dev, "%s device registration failed (%d)\n",
> +			dev_name(dev), ret);
> +	return ret;
> +}
> +
> +static int fimc_md_pdev_match(struct device *dev, void *data)

This function is also useless in your case.

> +{
> +	struct platform_device *pdev = to_platform_device(dev);
> +	int plat_entity = -1;
> +	int ret;
> +
> +	if (!get_device(dev))
> +		return -ENODEV;
> +
> +	if (!strcmp(pdev->name, CSIS_DRIVER_NAME))
> +		plat_entity = IDX_CSIS;
> +	else if (!strcmp(pdev->name, FIMC_LITE_DRV_NAME))
> +		plat_entity = IDX_FLITE;
> +
> +	if (plat_entity>= 0)
> +		ret = fimc_md_register_platform_entity(data, pdev,
> +						       plat_entity);
> +	put_device(dev);
> +	return 0;
> +}
> +
> +/* Register FIMC, FIMC-LITE and CSIS media entities */
> +#ifdef CONFIG_OF
> +static int fimc_md_register_of_platform_entities(struct fimc_md *fmd,
> +						 struct device_node *parent)
> +{
> +	struct device_node *node;
> +	int ret = 0;
> +
> +	for_each_available_child_of_node(parent, node) {
> +		struct platform_device *pdev;
> +		int plat_entity = -1;
> +
> +		pdev = of_find_device_by_node(node);
> +		if (!pdev)
> +			continue;
> +
> +		/* If driver of any entity isn't ready try all again later. */
> +		if (!strcmp(node->name, CSIS_OF_NODE_NAME))
> +			plat_entity = IDX_CSIS;
> +		else if (!strcmp(node->name, FIMC_LITE_OF_NODE_NAME))
> +			plat_entity = IDX_FLITE;
> +
> +		if (plat_entity>= 0)
> +			ret = fimc_md_register_platform_entity(fmd, pdev,
> +							plat_entity);
> +		put_device(&pdev->dev);
> +		if (ret<  0)
> +			break;
> +	}
> +
> +	return ret;
> +}
> +#else
> +#define fimc_md_register_platform_entities(fmd) (-ENOSYS)
> +#endif
...
> +static int fimc_md_link_notify(struct media_pad *source,
> +			       struct media_pad *sink, u32 flags)
> +{
> +	struct fimc_lite *fimc_lite = NULL;
> +	struct exynos_pipeline *pipeline;
> +	struct exynos5_pipeline0 *p;
> +	struct v4l2_subdev *sd;
> +	struct mutex *lock;
> +	int ret = 0;
> +	int ref_count;
> +
> +	if (media_entity_type(sink->entity) != MEDIA_ENT_T_V4L2_SUBDEV)
> +		return 0;
> +
> +	sd = media_entity_to_v4l2_subdev(sink->entity);
> +
> +	switch (sd->grp_id) {
> +	case GRP_ID_FLITE:
> +		fimc_lite = v4l2_get_subdevdata(sd);
> +		if (WARN_ON(fimc_lite == NULL))
> +			return 0;
> +		pipeline =&fimc_lite->pipeline;
> +		lock =&fimc_lite->lock;
> +		break;
> +	default:
> +		return 0;
> +	}
> +
> +	p = (struct exynos5_pipeline0 *)pipeline->priv;
> +	if (!p || !p->is_init)
> +		return -EINVAL;
> +
> +	if (!(flags&  MEDIA_LNK_FL_ENABLED)) {
> +		int i;
> +		mutex_lock(lock);

ref_count needs to be checked here as well.

> +		ret = __fimc_pipeline_close(pipeline);
> +		for (i = 0; i<  IDX_MAX; i++)
> +			p->subdevs[i] = NULL;
> +		mutex_unlock(lock);
> +		return ret;
> +	}
> +	/*
> +	 * Link activation. Enable power of pipeline elements only if the
> +	 * pipeline is already in use, i.e. its video node is opened.
> +	 * Recreate the controls destroyed during the link deactivation.

I think this line should be deleted.

> +	 */
> +	mutex_lock(lock);
> +
> +	ref_count = fimc_lite->ref_count;
> +	if (ref_count>  0)
> +		ret = __fimc_pipeline_open(pipeline, source->entity, true);
> +
> +	mutex_unlock(lock);
> +	return ret ? -EPIPE : ret;
> +}
> +
> +static ssize_t fimc_md_sysfs_show(struct device *dev,
> +				  struct device_attribute *attr, char *buf)
> +{
> +	struct platform_device *pdev = to_platform_device(dev);
> +	struct fimc_md *fmd = platform_get_drvdata(pdev);
> +
> +	if (fmd->user_subdev_api)
> +		return strlcpy(buf, "Sub-device API (sub-dev)\n", PAGE_SIZE);
> +
> +	return strlcpy(buf, "V4L2 video node only API (vid-dev)\n", PAGE_SIZE);
> +}
> +
> +static ssize_t fimc_md_sysfs_store(struct device *dev,
> +				   struct device_attribute *attr,
> +				   const char *buf, size_t count)
> +{
> +	struct platform_device *pdev = to_platform_device(dev);
> +	struct fimc_md *fmd = platform_get_drvdata(pdev);
> +	bool subdev_api;
> +	int i;
> +
> +	if (!strcmp(buf, "vid-dev\n"))
> +		subdev_api = false;
> +	else if (!strcmp(buf, "sub-dev\n"))
> +		subdev_api = true;
> +	else
> +		return count;
> +
> +	fmd->user_subdev_api = subdev_api;
> +	for (i = 0; i<  FIMC_LITE_MAX_DEVS; i++)
> +		if (fmd->fimc_lite[i])
> +			fmd->fimc_lite[i]->user_subdev_api = subdev_api;
> +
> +	return count;
> +}
> +/*
> + * This device attribute is to select video pipeline configuration method.
> + * There are following valid values:
> + *  vid-dev - for V4L2 video node API only, subdevice will be configured
> + *  by the host driver.
> + *  sub-dev - for media controller API, subdevs must be configured in user
> + *  space before starting streaming.
> + */
> +static DEVICE_ATTR(subdev_conf_mode, S_IWUSR | S_IRUGO,
> +		   fimc_md_sysfs_show, fimc_md_sysfs_store);

Please don't copy that. It is a not standard attribute that you should not
need in this driver anyway.

> +/**
> + * fimc_md_sensor_notify - v4l2_device notification from a sensor subdev
> + * @sd: pointer to a subdev generating the notification
> + * @notification: the notification type, must be S5P_FIMC_TX_END_NOTIFY
> + * @arg: pointer to an u32 type integer that stores the frame payload value
> + *
> + * Passes the sensor notification to the capture device
> + */
> +static void fimc_md_sensor_notify(struct v4l2_subdev *sd,
> +				unsigned int notification, void *arg)
> +{
> +	struct fimc_md *fmd;
> +	struct exynos_pipeline *ep;
> +	struct exynos5_pipeline0 *p;
> +	unsigned long flags;
> +
> +	if (sd == NULL)
> +		return;
> +
> +	ep = media_pipe_to_exynos_pipeline(sd->entity.pipe);
> +	p = (struct exynos5_pipeline0 *)ep->priv;
> +
> +	spin_lock_irqsave(&fmd->slock, flags);
> +
> +	if (p->sensor_notify)
> +		p->sensor_notify(sd, notification, arg);
> +
> +	spin_unlock_irqrestore(&fmd->slock, flags);
> +}
> +
> +static int fimc_md_probe(struct platform_device *pdev)
> +{
> +	struct device *dev =&pdev->dev;
> +	struct v4l2_device *v4l2_dev;
> +	struct fimc_md *fmd;
> +	int ret;
> +
> +	fmd = devm_kzalloc(dev, sizeof(*fmd), GFP_KERNEL);
> +	if (!fmd)
> +		return -ENOMEM;
> +
> +	spin_lock_init(&fmd->slock);
> +	fmd->pdev = pdev;
> +
> +	strlcpy(fmd->media_dev.model, "SAMSUNG S5P FIMC",

Please change this to "SAMSUNG EXYNOS5 IS" or something similar.
This could confuse user space!

> +		sizeof(fmd->media_dev.model));
> +	fmd->media_dev.link_notify = fimc_md_link_notify;
> +	fmd->media_dev.dev = dev;
> +
> +	v4l2_dev =&fmd->v4l2_dev;
> +	v4l2_dev->mdev =&fmd->media_dev;
> +	v4l2_dev->notify = fimc_md_sensor_notify;
> +	strlcpy(v4l2_dev->name, "s5p-fimc-md", sizeof(v4l2_dev->name));

exynos-fimc-md ?

> +
> +	ret = v4l2_device_register(dev,&fmd->v4l2_dev);
> +	if (ret<  0) {
> +		v4l2_err(v4l2_dev, "Failed to register v4l2_device: %d\n", ret);
> +
> +		return ret;
> +
> +	}
> +	ret = media_device_register(&fmd->media_dev);
> +	if (ret<  0) {
> +		v4l2_err(v4l2_dev, "Failed to register media dev: %d\n", ret);
> +		goto err_md;
> +	}
> +	ret = fimc_md_get_clocks(fmd);
> +	if (ret)
> +		goto err_clk;
> +
> +	fmd->user_subdev_api = (dev->of_node != NULL);
> +	g_exynos_mdev = fmd;
> +
> +	/* Protect the media graph while we're registering entities */
> +	mutex_lock(&fmd->media_dev.graph_mutex);
> +
> +	if (fmd->pdev->dev.of_node)
> +		ret = fimc_md_register_of_platform_entities(fmd, dev->of_node);
> +	else
> +		ret = bus_for_each_dev(&platform_bus_type, NULL, fmd,
> +						fimc_md_pdev_match);
> +
> +	if (ret)
> +		goto err_unlock;
> +
> +	if (dev->platform_data || dev->of_node) {

platform_data will never be used, and dev->of_node would be always not NULL.

> +		ret = fimc_md_register_sensor_entities(fmd);
> +		if (ret)
> +			goto err_unlock;
> +	}
> +
> +	ret = fimc_md_create_links(fmd);
> +	if (ret)
> +		goto err_unlock;
> +
> +	ret = v4l2_device_register_subdev_nodes(&fmd->v4l2_dev);
> +	if (ret)
> +		goto err_unlock;
> +
> +	ret = device_create_file(&pdev->dev,&dev_attr_subdev_conf_mode);
> +	if (ret)
> +		goto err_unlock;
> +
> +	platform_set_drvdata(pdev, fmd);
> +	mutex_unlock(&fmd->media_dev.graph_mutex);
> +	return 0;
> +
> +err_unlock:
> +	mutex_unlock(&fmd->media_dev.graph_mutex);
> +err_clk:
> +	media_device_unregister(&fmd->media_dev);
> +	fimc_md_put_clocks(fmd);
> +	fimc_md_unregister_entities(fmd);
> +err_md:
> +	v4l2_device_unregister(&fmd->v4l2_dev);
> +	return ret;
> +}
> +
...
> +static struct platform_device_id fimc_driver_ids[] __always_unused = {
> +	{ .name = "exynos-fimc-md" },
> +	{ },
> +};
> +MODULE_DEVICE_TABLE(platform, fimc_driver_ids);
> +
> +static const struct of_device_id fimc_md_of_match[] __initconst = {

The attribute is wrong. I would remove that "__initconst".

> +	{ .compatible = "samsung,exynos5-fimc" },
> +	{ },
> +};
> +MODULE_DEVICE_TABLE(of, fimc_md_of_match);
> +

--

Regards,
Sylwester

Re: [RFC 01/12] media: s5p-fimc: modify existing mdev to use common pipeline

[Shaik Ameer Basha]

Hi Sylwester,

Thanks for the review.
Actually I know this is the important patch in this series and I
wanted us to have
enough time to discuss on this patch. That's why I posted this patch
series in hurry.

I will remove this patch from the exynos5-mdev series and will send this as a
separate patch from next time.

please find my review comments inline..


On Mon, Mar 11, 2013 at 3:30 AM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> This patch modifies the current fimc_pipeline to exynos_pipeline,
>
>
> I think we could leave it as fimc_pipeline, exynos_pipeline seems
> too generic to me.

no issues, if we are going to strict to this common pipeline implementation
definitely we can retain fimc_pipeline or we can use some other name which
is not too generic.

>
>> which can be used across multiple media device drivers.
>> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
>> ---
>>   drivers/media/platform/s5p-fimc/fimc-capture.c |   96 +++++++-----
>>   drivers/media/platform/s5p-fimc/fimc-core.h    |    4 +-
>>   drivers/media/platform/s5p-fimc/fimc-lite.c    |   73 ++++------
>>   drivers/media/platform/s5p-fimc/fimc-lite.h    |    4 +-
>>   drivers/media/platform/s5p-fimc/fimc-mdevice.c |  186
>> ++++++++++++++++++++++--
>>   drivers/media/platform/s5p-fimc/fimc-mdevice.h |   41 +++---
>>   include/media/s5p_fimc.h                       |   66 ++++++---
>>   7 files changed, 326 insertions(+), 144 deletions(-)
>>
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-capture.c
>> b/drivers/media/platform/s5p-fimc/fimc-capture.c
>> index 4cbaf46..106466e 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-capture.c
>> +++ b/drivers/media/platform/s5p-fimc/fimc-capture.c
>> @@ -27,24 +27,26 @@
>>   #include<media/videobuf2-core.h>
>>   #include<media/videobuf2-dma-contig.h>
>>
>> -#include "fimc-mdevice.h"
>>   #include "fimc-core.h"
>>   #include "fimc-reg.h"
>>
>>   static int fimc_capture_hw_init(struct fimc_dev *fimc)
>>   {
>>         struct fimc_ctx *ctx = fimc->vid_cap.ctx;
>> -       struct fimc_pipeline *p =&fimc->pipeline;
>> +       struct exynos_pipeline *p =&fimc->pipeline;
>>
>>         struct fimc_sensor_info *sensor;
>>         unsigned long flags;
>> +       struct v4l2_subdev *sd;
>>         int ret = 0;
>>
>> -       if (p->subdevs[IDX_SENSOR] == NULL || ctx == NULL)
>> +       sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
>> +                                       get_subdev_sensor, p);
>
>
> Hmm, it feels it is going wrong path this way. I would keep changes
> to the s5p-fimc driver as small as possible. And the modules that
> are shared across the exynos4 and exynos5 driver should use generic
> media graph walking routines where possible.

The only problem here is, the fimc_subdev_index enum is specific to
fimc-mdevice.
and why should we expose one particular media-device driver specific
enums to other drivers.
My Idea was to remove all media device specific structures, macros
from fimc, fimc-lite,
mipi-csis and fimc-is drivers.

>
>
>> +       if (sd == NULL || ctx == NULL)
>>                 return -ENXIO;
>>         if (ctx->s_frame.fmt == NULL)
>>                 return -EINVAL;
>>
>> -       sensor = v4l2_get_subdev_hostdata(p->subdevs[IDX_SENSOR]);
>> +       sensor = v4l2_get_subdev_hostdata(sd);
>>
>>         spin_lock_irqsave(&fimc->slock, flags);
>>         fimc_prepare_dma_offset(ctx,&ctx->d_frame);
>
> ...
>>
>> @@ -486,9 +491,12 @@ static struct vb2_ops fimc_capture_qops = {
>>   int fimc_capture_ctrls_create(struct fimc_dev *fimc)
>>   {
>>         struct fimc_vid_cap *vid_cap =&fimc->vid_cap;
>>
>> -       struct v4l2_subdev *sensor = fimc->pipeline.subdevs[IDX_SENSOR];
>> +       struct v4l2_subdev *sensor;
>>         int ret;
>>
>> +       sensor = exynos_pipeline_get_subdev(fimc->pipeline_ops,
>> +
>> get_subdev_sensor,&fimc->pipeline);
>>
>> +
>>         if (WARN_ON(vid_cap->ctx == NULL))
>>                 return -ENXIO;
>>         if (vid_cap->ctx->ctrls.ready)
>> @@ -513,7 +521,7 @@ static int fimc_capture_open(struct file *file)
>>
>>         dbg("pid: %d, state: 0x%lx", task_pid_nr(current), fimc->state);
>>
>> -       fimc_md_graph_lock(fimc);
>> +       exynos_pipeline_graph_lock(fimc->pipeline_ops,&fimc->pipeline);
>
>
> Hmm, this look pretty scary to me. But I suspect this change is not
> needed at all. The graph lock is _not_ the pipeline lock. It protects
> all media entities registered to the media device, and links between
> them. Not only entities linked into specific video processing pipeline
> at a moment.

Sorry, here the function name doesn't suits its implementation.
Actually  exynos_pipeline_graph_lock() does what exactly
fimc_md_graph_lock() does.
I thought of having one common function across all the drivers to use
graph lock/unlock functionality.

>
>>         mutex_lock(&fimc->lock);
>>
>>         if (fimc_m2m_active(fimc))
>> @@ -531,7 +539,7 @@ static int fimc_capture_open(struct file *file)
>>         }
>>
>>         if (++fimc->vid_cap.refcnt == 1) {
>> -               ret = fimc_pipeline_call(fimc, open,&fimc->pipeline,
>> +               ret = exynos_pipeline_call(fimc, open,&fimc->pipeline,
>>                                         &fimc->vid_cap.vfd.entity, true);
>>
>>                 if (!ret&&  !fimc->vid_cap.user_subdev_api)
>>
>> @@ -549,7 +557,7 @@ static int fimc_capture_open(struct file *file)
>>         }
>>   unlock:
>>         mutex_unlock(&fimc->lock);
>> -       fimc_md_graph_unlock(fimc);
>> +       exynos_pipeline_graph_unlock(fimc->pipeline_ops,&fimc->pipeline);
>>         return ret;
>>   }
>
> ...
>
>>   /**
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c
>> b/drivers/media/platform/s5p-fimc/fimc-lite.c
>> index 3266c3f..122cf95 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-lite.c
>> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
>
> ...
>>
>> -/* Called with the media graph mutex held */
>> -static struct v4l2_subdev *__find_remote_sensor(struct media_entity *me)
>> -{
>> -       struct media_pad *pad =&me->pads[0];
>>
>> -       struct v4l2_subdev *sd;
>> -
>> -       while (pad->flags&  MEDIA_PAD_FL_SINK) {
>>
>> -               /* source pad */
>> -               pad = media_entity_remote_source(pad);
>> -               if (pad == NULL ||
>> -                   media_entity_type(pad->entity) !=
>> MEDIA_ENT_T_V4L2_SUBDEV)
>> -                       break;
>> -
>> -               sd = media_entity_to_v4l2_subdev(pad->entity);
>> -
>> -               if (sd->grp_id == GRP_ID_FIMC_IS_SENSOR)
>> -                       return sd;
>> -               /* sink pad */
>> -               pad =&sd->entity.pads[0];
>>
>> -       }
>> -       return NULL;
>> -}
>
>
> What's wrong with this function ? Why doesn't it work for you ?
> I think we should drop direct usage of fimc->pipeline->subdevs[IDX_SENSOR]
> instead. The sensor group IDs should probably be moved to some common
> header file.
>

In case of exynos4, fimc-lite only used with fimc-is. That means it only
uses the FIMC IS SENSOR. but in case of exynos5, fimc-lite can use both
GRP_ID_SENSOR and GRP_ID_FIMC_IS_SENSOR. Even this group ID's are
specific to individual media device drivers.

Here the idea is driver need not know about which sensor is connected to the
pipeline, it can be IS sensor or normal sensor. It can use the common pipeline
api's to get the sensor sub-dev.

Or as you suggested, we can move all this GRP_ID's to some common location
and change the checking in this function to
        -  if (sd->grp_id == GRP_ID_FIMC_IS_SENSOR)
        + if ((sd->grp_id == GRP_ID_FIMC_IS_SENSOR) || (sd->grp_id ==
GRP_ID_SENSOR))

My only idea was to reuse the fimc pipeline structure (struct
fimc_pipeline) across
different media device drivers. Having a hard-coded max array index
which is specific to
one media device driver will not help it to be reused by multiple guys.

I may be missing some details here. Can you please suggest a better
way to remove this dependency.

>
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>> b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>> index fc93fad..938cc56 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>> +++ b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>> @@ -36,6 +36,8 @@
>>   #include "fimc-mdevice.h"
>>   #include "mipi-csis.h"
>>
>> +static struct fimc_md *g_fimc_mdev;
>
>
> Hm, this is pretty ugly. Can you explain why it is needed ?

I was expecting this.
Actually i want to associate this fimc_mdev with this exynos_pipeline.
But as the pipeline init happens from the driver context (fimc,
fimc-lite), I should
have a way to access struct fimc_md in pipeline init function. That's
why I stored a
global reference of struct fimc_md here.
I think, I need to find a better way to achieve same functionality if needed. :)

>
>
>>   static int __fimc_md_set_camclk(struct fimc_md *fmd,
>>                                 struct fimc_sensor_info *s_info,
>>                                 bool on);
>> @@ -143,6 +145,73 @@ static int fimc_pipeline_s_power(struct fimc_pipeline
>> *p, bool state)
>>   }
>>
>>   /**
>> + * __fimc_pipeline_init
>> + *      allocate the fimc_pipeline structure and do the basic
>> initialization
>
>
> This is not a proper kernel-doc style.

Ok. Will take care.

>
>
>> + */
>> +static int __fimc_pipeline_init(struct exynos_pipeline *ep)
>> +{
>> +       struct fimc_pipeline *p;
>> +
>> +       p = kzalloc(sizeof(*p), GFP_KERNEL);
>> +       if (!p)
>> +               return -ENOMEM;
>
>
> Why this needs to be allocated dynamically ?

OK. I will check this.

>
>
>> +       p->is_init = true;
>> +       p->fmd = g_fimc_mdev;
>> +       ep->priv = (void *)p;
>> +       return 0;
>> +}
>> +
>> +/**
>> + * __fimc_pipeline_deinit
>> + *      free the allocated resources for fimc_pipeline
>> + */
>> +static int __fimc_pipeline_deinit(struct exynos_pipeline *ep)
>> +{
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>> +
>> +       if (!p || !p->is_init)
>> +               return -EINVAL;
>> +
>> +       p->is_init = false;
>> +       kfree(p);
>> +
>> +       return 0;
>> +}
>> +
>> +/**
>> + * __fimc_pipeline_get_subdev_sensor
>> + *      if valid pipeline, returns the sensor subdev pointer
>> + *      else returns NULL
>> + */
>> +static struct v4l2_subdev *__fimc_pipeline_get_subdev_sensor(
>> +                                       struct exynos_pipeline *ep)
>> +{
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>> +
>> +       if (!p || !p->is_init)
>> +               return NULL;
>> +
>> +       return p->subdevs[IDX_SENSOR];
>> +}
>> +
>> +/**
>> + * __fimc_pipeline_get_subdev_csis
>> + *      if valid pipeline, returns the csis subdev pointer
>> + *      else returns NULL
>> + */
>> +static struct v4l2_subdev *__fimc_pipeline_get_subdev_csis(
>> +                                       struct exynos_pipeline *ep)
>> +{
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>> +
>> +       if (!p || !p->is_init)
>> +               return NULL;
>> +
>> +       return p->subdevs[IDX_CSIS];
>> +}
>> +
>> +/**
>>    * __fimc_pipeline_open - update the pipeline information, enable power
>>    *                        of all pipeline subdevs and the sensor clock
>>    * @me: media entity to start graph walk with
>> @@ -150,11 +219,15 @@ static int fimc_pipeline_s_power(struct
>> fimc_pipeline *p, bool state)
>>    *
>>    * Called with the graph mutex held.
>>    */
>> -static int __fimc_pipeline_open(struct fimc_pipeline *p,
>> +static int __fimc_pipeline_open(struct exynos_pipeline *ep,
>>                                 struct media_entity *me, bool prep)
>>   {
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>>         int ret;
>>
>> +       if (!p || !p->is_init)
>> +               return -EINVAL;
>> +
>>         if (prep)
>>                 fimc_pipeline_prepare(p, me);
>>
>> @@ -174,17 +247,20 @@ static int __fimc_pipeline_open(struct fimc_pipeline
>> *p,
>>    *
>>    * Disable power of all subdevs and turn the external sensor clock off.
>>    */
>> -static int __fimc_pipeline_close(struct fimc_pipeline *p)
>> +static int __fimc_pipeline_close(struct exynos_pipeline *ep)
>>   {
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>>         int ret = 0;
>>
>> -       if (!p || !p->subdevs[IDX_SENSOR])
>> +       if (!p || !p->is_init)
>>                 return -EINVAL;
>>
>> -       if (p->subdevs[IDX_SENSOR]) {
>> -               ret = fimc_pipeline_s_power(p, 0);
>> -               fimc_md_set_camclk(p->subdevs[IDX_SENSOR], false);
>> -       }
>> +       if (!p->subdevs[IDX_SENSOR])
>> +               return -EINVAL;
>> +
>> +       ret = fimc_pipeline_s_power(p, 0);
>> +       fimc_md_set_camclk(p->subdevs[IDX_SENSOR], false);
>> +
>>         return ret == -ENXIO ? 0 : ret;
>>   }
>>
>> @@ -193,10 +269,14 @@ static int __fimc_pipeline_close(struct
>> fimc_pipeline *p)
>>    * @pipeline: video pipeline structure
>>    * @on: passed as the s_stream call argument
>>    */
>> -static int __fimc_pipeline_s_stream(struct fimc_pipeline *p, bool on)
>> +static int __fimc_pipeline_s_stream(struct exynos_pipeline *ep, bool on)
>>   {
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>>         int i, ret;
>>
>> +       if (!p || !p->is_init)
>> +               return -EINVAL;
>> +
>>         if (p->subdevs[IDX_SENSOR] == NULL)
>>                 return -ENODEV;
>>
>> @@ -213,11 +293,47 @@ static int __fimc_pipeline_s_stream(struct
>> fimc_pipeline *p, bool on)
>>
>>   }
>>
>> +static void __fimc_pipeline_graph_lock(struct exynos_pipeline *ep)
>> +{
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>> +       struct fimc_md *fmd = p->fmd;
>> +
>> +       mutex_lock(&fmd->media_dev.graph_mutex);
>> +}
>> +
>> +static void __fimc_pipeline_graph_unlock(struct exynos_pipeline *ep)
>> +{
>> +       struct fimc_pipeline *p = (struct fimc_pipeline *)ep->priv;
>> +       struct fimc_md *fmd = p->fmd;
>> +
>> +       mutex_unlock(&fmd->media_dev.graph_mutex);
>> +}
>
>
> This seems really wrong to me. You can reference the media graph mutex
> from a subdev or video device through its 'entity' member, e.g.
> sd->entity.parent->graph_mutex.
>
> ...
>>
>>   static int fimc_md_probe(struct platform_device *pdev)
>>   {
>>         struct device *dev =&pdev->dev;
>>
>> @@ -1175,7 +1327,7 @@ static int fimc_md_probe(struct platform_device
>> *pdev)
>>
>>         v4l2_dev =&fmd->v4l2_dev;
>>         v4l2_dev->mdev =&fmd->media_dev;
>>
>> -       v4l2_dev->notify = fimc_sensor_notify;
>> +       v4l2_dev->notify = fimc_md_sensor_notify;
>>         strlcpy(v4l2_dev->name, "s5p-fimc-md", sizeof(v4l2_dev->name));
>>
>>
>> @@ -1194,6 +1346,7 @@ static int fimc_md_probe(struct platform_device
>> *pdev)
>>                 goto err_clk;
>>
>>         fmd->user_subdev_api = (dev->of_node != NULL);
>> +       g_fimc_mdev = fmd;
>>
>>         /* Protect the media graph while we're registering entities */
>>         mutex_lock(&fmd->media_dev.graph_mutex);
>> @@ -1252,6 +1405,7 @@ static int fimc_md_remove(struct platform_device
>> *pdev)
>>
>>         if (!fmd)
>>                 return 0;
>> +       g_fimc_mdev = NULL;
>>         device_remove_file(&pdev->dev,&dev_attr_subdev_conf_mode);
>>         fimc_md_unregister_entities(fmd);
>>         media_device_unregister(&fmd->media_dev);
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-mdevice.h
>> b/drivers/media/platform/s5p-fimc/fimc-mdevice.h
>> index f3e0251..1ea7acf 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-mdevice.h
>> +++ b/drivers/media/platform/s5p-fimc/fimc-mdevice.h
>> @@ -37,6 +37,15 @@
>>   #define FIMC_MAX_SENSORS      8
>>   #define FIMC_MAX_CAMCLKS      2
>>
>> +enum fimc_subdev_index {
>> +       IDX_SENSOR,
>> +       IDX_CSIS,
>> +       IDX_FLITE,
>> +       IDX_IS_ISP,
>> +       IDX_FIMC,
>> +       IDX_MAX,
>> +};
>> +
>>   struct fimc_csis_info {
>>         struct v4l2_subdev *sd;
>>         int id;
>> @@ -49,20 +58,6 @@ struct fimc_camclk_info {
>>   };
>>
>>   /**
>> - * struct fimc_sensor_info - image data source subdev information
>> - * @pdata: sensor's atrributes passed as media device's platform data
>> - * @subdev: image sensor v4l2 subdev
>> - * @host: fimc device the sensor is currently linked to
>> - *
>> - * This data structure applies to image sensor and the writeback subdevs.
>> - */
>> -struct fimc_sensor_info {
>> -       struct fimc_source_info pdata;
>> -       struct v4l2_subdev *subdev;
>> -       struct fimc_dev *host;
>> -};
>> -
>> -/**
>>    * struct fimc_md - fimc media device information
>>    * @csis: MIPI CSIS subdevs data
>>    * @sensor: array of registered sensor subdevs
>> @@ -89,6 +84,14 @@ struct fimc_md {
>>         spinlock_t slock;
>>   };
>>
>> +struct fimc_pipeline {
>> +       int is_init;
>> +       struct fimc_md *fmd;
>> +       struct v4l2_subdev *subdevs[IDX_MAX];
>> +       void (*sensor_notify)(struct v4l2_subdev *sd,
>> +                       unsigned int notification, void *arg);
>> +};
>
>
> This is supposed to be the s5p-fimc driver specific only data structure,
> right ?

yes.

>
>
>>   #define is_subdev_pad(pad) (pad == NULL || \
>>         media_entity_type(pad->entity) == MEDIA_ENT_T_V4L2_SUBDEV)
>>
>> @@ -103,16 +106,6 @@ static inline struct fimc_md
>> *entity_to_fimc_mdev(struct media_entity *me)
>>                 container_of(me->parent, struct fimc_md, media_dev);
>>   }
>>
>> -static inline void fimc_md_graph_lock(struct fimc_dev *fimc)
>> -{
>> -       mutex_lock(&fimc->vid_cap.vfd.entity.parent->graph_mutex);
>> -}
>> -
>> -static inline void fimc_md_graph_unlock(struct fimc_dev *fimc)
>> -{
>> -       mutex_unlock(&fimc->vid_cap.vfd.entity.parent->graph_mutex);
>> -}
>
>
> Why to remove these s5p-fimc driver private functions ?

I thought of using the common pipeline graph functions instead.
As the implementaiton of __fimc_pipeline_graph_unlock() looks wrong,
we can retain this as driver private functions.

>
>
>>   int fimc_md_set_camclk(struct v4l2_subdev *sd, bool on);
>>
>>   #endif
>> diff --git a/include/media/s5p_fimc.h b/include/media/s5p_fimc.h
>> index e2434bb..007e998 100644
>> --- a/include/media/s5p_fimc.h
>> +++ b/include/media/s5p_fimc.h
>> @@ -13,6 +13,7 @@
>>   #define S5P_FIMC_H_
>>
>>   #include<media/media-entity.h>
>> +#include<media/v4l2-subdev.h>
>>
>>   /*
>>    * Enumeration of data inputs to the camera subsystem.
>> @@ -75,6 +76,20 @@ struct fimc_source_info {
>>   };
>>
>>   /**
>> + * struct fimc_sensor_info - image data source subdev information
>> + * @pdata: sensor's atrributes passed as media device's platform data
>> + * @subdev: image sensor v4l2 subdev
>> + * @host: capture device the sensor is currently linked to
>> + *
>> + * This data structure applies to image sensor and the writeback subdevs.
>> + */
>> +struct fimc_sensor_info {
>> +       struct fimc_source_info pdata;
>> +       struct v4l2_subdev *subdev;
>> +       void *host;
>> +};
>> +
>> +/**
>>    * struct s5p_platform_fimc - camera host interface platform data
>>    *
>>    * @source_info: properties of an image source for the host interface
>> setup
>> @@ -93,21 +108,10 @@ struct s5p_platform_fimc {
>>    */
>>   #define S5P_FIMC_TX_END_NOTIFY _IO('e', 0)
>>
>> -enum fimc_subdev_index {
>> -       IDX_SENSOR,
>> -       IDX_CSIS,
>> -       IDX_FLITE,
>> -       IDX_IS_ISP,
>> -       IDX_FIMC,
>> -       IDX_MAX,
>> -};
>> -
>> -struct media_pipeline;
>> -struct v4l2_subdev;
>>
>> -struct fimc_pipeline {
>> -       struct v4l2_subdev *subdevs[IDX_MAX];
>> -       struct media_pipeline *m_pipeline;
>> +struct exynos_pipeline {
>> +       struct media_pipeline m_pipeline;
>> +       void *priv;
>>   };
>>
>>   /*
>> @@ -115,15 +119,39 @@ struct fimc_pipeline {
>>    * video node when it is the last entity of the pipeline. Implemented
>>    * by corresponding media device driver.
>>    */
>> -struct fimc_pipeline_ops {
>> -       int (*open)(struct fimc_pipeline *p, struct media_entity *me,
>> +struct exynos_pipeline_ops {
>> +       int (*init) (struct exynos_pipeline *p);
>> +       int (*deinit) (struct exynos_pipeline *p);
>
>
> How about naming it 'free' instead ?

OK

>
>
>> +       int (*open)(struct exynos_pipeline *p, struct media_entity *me,
>>                           bool resume);
>> -       int (*close)(struct fimc_pipeline *p);
>> -       int (*set_stream)(struct fimc_pipeline *p, bool state);
>> +       int (*close)(struct exynos_pipeline *p);
>> +       int (*set_stream)(struct exynos_pipeline *p, bool state);
>> +       void (*graph_lock)(struct exynos_pipeline *p);
>> +       void (*graph_unlock)(struct exynos_pipeline *p);
>
>
> Why do you think these graph callbacks are needed here ?

as mentioned above, we can remove this.

>
>
>> +       struct v4l2_subdev *(*get_subdev_sensor)(struct exynos_pipeline
>> *p);
>> +       struct v4l2_subdev *(*get_subdev_csis)(struct exynos_pipeline *p);
>
>
> No, we should instead use generic media graph walking routines in the
> shared drivers (FIMC-LITE, MIPI-CSIS). FIMC driver could be making
> certain assumptions about it's related media device driver. These
> drivers are contained in a single module anyway.

Cant we expose some macros for subdevs in a common header and implement a
get_subdev function in common pipeline to achieve this.

enum in /include/media/s5p_fimc.h file. Something like

EXYNOS_SD_SENSOR,
EXYNOS_SD_MIPI_CSIS,
...
... etc.

pipeline_ops->get_subdev(pipeline, EXYNOS_SD_SENSOR);

actually, the implementation of get_subdev uses the media device specific
structures which stored the subdev pointers internally by using  generic media
graph walking routines itself. so there should not be any issues with using the
common pipeline api's also.

[Or]

as you already suggested, we can have some common group id's exposed
in a common header file, and use the generic media graph walking
routines in each
driver to achieve this functionality.

Regards,
Shaik Ameer Basha

>
>
>> +       void (*register_notify_cb)(struct exynos_pipeline *p,
>> +               void (*cb)(struct v4l2_subdev *sd,
>> +                               unsigned int notification, void *arg));
>
>
> Hmm, I need to think more about this. AFAIR this would be only needed
> for S5P/Exynos4 FIMC.

Re: [RFC 02/12] fimc-lite: Adding Exynos5 compatibility to fimc-lite driver

[Shaik Ameer Basha]

Hi Sylwester,

On Mon, Mar 11, 2013 at 2:06 AM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> This patch adds the Exynos5 soc compatibility to the fimc-lite driver.
>> It also adds a version checking to deal with the changes between
>> different fimc-lite hardware versions.
>
>
> Is there really anything different between the Exynos4 and Exynos5
> FIMC-LITE IPs except the maximum number of buffer descriptors in
> the output DMA queue ?
>
>
>> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
>> ---
>>   drivers/media/platform/s5p-fimc/fimc-lite.c |   23
>> +++++++++++++++++++++++
>>   drivers/media/platform/s5p-fimc/fimc-lite.h |    7 ++++++-
>>   2 files changed, 29 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c
>> b/drivers/media/platform/s5p-fimc/fimc-lite.c
>> index 122cf95..eb64f87 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-lite.c
>> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
>> @@ -1653,6 +1653,16 @@ static struct flite_variant
>> fimc_lite0_variant_exynos4 = {
>>         .out_width_align        = 8,
>>         .win_hor_offs_align     = 2,
>>         .out_hor_offs_align     = 8,
>> +       .version                = FLITE_VER_EXYNOS4,
>> +};
>> +
>> +static struct flite_variant fimc_lite0_variant_exynos5 = {
>> +       .max_width              = 8192,
>> +       .max_height             = 8192,
>> +       .out_width_align        = 8,
>> +       .win_hor_offs_align     = 2,
>> +       .out_hor_offs_align     = 8,
>
>
> Please see my comment to patch 03/12.
>
>
>> +       .version                = FLITE_VER_EXYNOS5,
>>   };
>>
>>   /* EXYNOS4212, EXYNOS4412 */
>> @@ -1663,6 +1673,15 @@ static struct flite_drvdata
>> fimc_lite_drvdata_exynos4 = {
>>         },
>>   };
>>
>> +/* EXYNOS5250 */
>> +static struct flite_drvdata fimc_lite_drvdata_exynos5 = {
>> +       .variant = {
>> +               [0] =&fimc_lite0_variant_exynos5,
>> +               [1] =&fimc_lite0_variant_exynos5,
>> +               [2] =&fimc_lite0_variant_exynos5,
>> +       },
>> +};
>> +
>>   static struct platform_device_id fimc_lite_driver_ids[] = {
>>         {
>>                 .name           = "exynos-fimc-lite",
>> @@ -1677,6 +1696,10 @@ static const struct of_device_id flite_of_match[] =
>> {
>>                 .compatible = "samsung,exynos4212-fimc-lite",
>>                 .data =&fimc_lite_drvdata_exynos4,
>>         },
>> +       {
>> +               .compatible = "samsung,exynos5250-fimc-lite",
>> +               .data =&fimc_lite_drvdata_exynos5,
>> +       },
>>         { /* sentinel */ },
>>   };
>>   MODULE_DEVICE_TABLE(of, flite_of_match);
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.h
>> b/drivers/media/platform/s5p-fimc/fimc-lite.h
>> index 66d6eeb..ef43fe0 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-lite.h
>> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.h
>> @@ -28,7 +28,7 @@
>>
>>   #define FIMC_LITE_DRV_NAME    "exynos-fimc-lite"
>>   #define FLITE_CLK_NAME                "flite"
>> -#define FIMC_LITE_MAX_DEVS     2
>> +#define FIMC_LITE_MAX_DEVS     3
>>   #define FLITE_REQ_BUFS_MIN    2
>>
>>   /* Bit index definitions for struct fimc_lite::state */
>> @@ -49,12 +49,17 @@ enum {
>>   #define FLITE_SD_PAD_SOURCE_ISP       2
>>   #define FLITE_SD_PADS_NUM     3
>>
>> +#define FLITE_VER_EXYNOS4      0
>> +#define FLITE_VER_EXYNOS5      1
>
>
> I would prefer not using explicit version and rather put each
> quirk in the driver data structure, so we can avoid those
> multiple if (version == ...) checks all over in the code, should
> more revision of this IP come in future SoCs.

Ok. no issues. we can remove this version checking and maintain the
differences in driver data structures.

Regards,
Shaik Ameer Basha

Re: [RFC 03/12] media: fimc-lite: Adding support for Exynos5

[Shaik Ameer Basha]

Hi Sylweter,

On Mon, Mar 11, 2013 at 2:09 AM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> This patch adds the following functionalities to existing driver
>>
>> 1] FIMC-LITE supports multiple DMA shadow registers from Exynos5 onwards.
>> This patch adds the functionality of using shadow registers by
>> checking the current FIMC-LITE hardware version.
>> 2] Fixes Buffer corruption on DMA output from fimc-lite
>> 3] Modified the driver to be used as pipeline endpoint
>
>
> There seems to be too many things done in this single patch. Can
> we have it split for example to the parts adding:
> - registers definitions and hardware interface helpers
>   for exynos5
> - the DMA handling fix
>
> So it is easier to apply it and test incrementally ?

Definitely, I will split this patch in to multiple patches

>
>
>> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
>> Signed-off-by: Arun Kumar K<arun.kk@samsung.com>
>> ---
>>   drivers/media/platform/s5p-fimc/fimc-lite-reg.c |   16 +-
>>   drivers/media/platform/s5p-fimc/fimc-lite-reg.h |   41 ++++-
>>   drivers/media/platform/s5p-fimc/fimc-lite.c     |  196
>> +++++++++++++++++++++--
>>   drivers/media/platform/s5p-fimc/fimc-lite.h     |    3 +-
>>   4 files changed, 236 insertions(+), 20 deletions(-)
>
> ...
>
>>   void flite_hw_dump_regs(struct fimc_lite *dev, const char *label)
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
>> b/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
>> index 0e34584..716df6c 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
>> +++ b/drivers/media/platform/s5p-fimc/fimc-lite-reg.h
>> @@ -120,6 +120,10 @@
>>   /* b0: 1 - camera B, 0 - camera A */
>>   #define FLITE_REG_CIGENERAL_CAM_B             (1<<  0)
>>
>> +
>> +#define FLITE_REG_CIFCNTSEQ                    0x100
>> +#define FLITE_REG_CIOSAN(x)                    (0x200 + (4 * (x)))
>> +
>>   /*
>> ----------------------------------------------------------------------------
>>    * Function declarations
>>    */
>> @@ -143,8 +147,41 @@ void flite_hw_set_dma_window(struct fimc_lite *dev,
>> struct flite_frame *f);
>>   void flite_hw_set_test_pattern(struct fimc_lite *dev, bool on);
>>   void flite_hw_dump_regs(struct fimc_lite *dev, const char *label);
>>
>> -static inline void flite_hw_set_output_addr(struct fimc_lite *dev, u32
>> paddr)
>> +static inline void flite_hw_set_output_addr(struct fimc_lite *dev,
>> +       u32 paddr, u32 index)
>> +{
>> +       u32 config;
>> +
>> +       /* FLITE in EXYNOS4 has only one DMA register */
>> +       if (dev->variant->version == FLITE_VER_EXYNOS4)
>> +               index = 0;
>> +
>> +       config = readl(dev->regs + FLITE_REG_CIFCNTSEQ);
>> +       config |= 1<<  index;
>> +       writel(config, dev->regs + FLITE_REG_CIFCNTSEQ);
>> +
>> +       if (index == 0)
>> +               writel(paddr, dev->regs + FLITE_REG_CIOSA);
>> +       else
>> +               writel(paddr, dev->regs + FLITE_REG_CIOSAN(index-1));
>> +}
>> +
>> +static inline void flite_hw_clear_output_addr(struct fimc_lite *dev, u32
>> index)
>>   {
>> -       writel(paddr, dev->regs + FLITE_REG_CIOSA);
>> +       u32 config;
>> +
>> +       /* FLITE in EXYNOS4 has only one DMA register */
>> +       if (dev->variant->version == FLITE_VER_EXYNOS4)
>> +               index = 0;
>
>
> I'm planning to remove struct flite_variant and put everything what's
> needed in the driver data structure. Are there any differences between
> FIMC-LITE IP block instances or Exynos5250 ? Or are these all same ?

Except the out dma buffers change, I don't think any changes are there in
fimc-lite for Exynos5250.

>
> That said it seems better to me to add a field like out_dma_bufs to the
> driver data structure and embed a pointer to this structure in struct
> fimc_lite. The driver data matching would be done automatically, based
> on the compatible property and those unpleasant checks
> if (variant->version == FLITE_VER_EXYNOS4) ...

Ok. This seems a better option. I will follow your suggestions.

>
>> +
>> +       config = readl(dev->regs + FLITE_REG_CIFCNTSEQ);
>> +       config&= ~(1<<  index);
>>
>> +       writel(config, dev->regs + FLITE_REG_CIFCNTSEQ);
>>   }
>> +
>> +static inline void flite_hw_clear_output_index(struct fimc_lite *dev)
>> +{
>> +       writel(0, dev->regs + FLITE_REG_CIFCNTSEQ);
>> +}
>> +
>>   #endif /* FIMC_LITE_REG_H */
>> diff --git a/drivers/media/platform/s5p-fimc/fimc-lite.c
>> b/drivers/media/platform/s5p-fimc/fimc-lite.c
>> index eb64f87..1edc5ce 100644
>> --- a/drivers/media/platform/s5p-fimc/fimc-lite.c
>> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.c
>> @@ -136,6 +136,8 @@ static int fimc_lite_hw_init(struct fimc_lite *fimc,
>> bool isp_output)
>>         if (fimc->fmt == NULL)
>>                 return -EINVAL;
>>
>> +       flite_hw_clear_output_index(fimc);
>> +
>>         /* Get sensor configuration data from the sensor subdev */
>>         src_info = v4l2_get_subdev_hostdata(sensor);
>>         spin_lock_irqsave(&fimc->slock, flags);
>> @@ -266,19 +268,24 @@ static irqreturn_t flite_irq_handler(int irq, void
>> *priv)
>>
>>         if ((intsrc&  FLITE_REG_CISTATUS_IRQ_SRC_FRMSTART)&&
>>         test_bit(ST_FLITE_RUN,&fimc->state)&&
>> -           !list_empty(&fimc->active_buf_q)&&
>>         !list_empty(&fimc->pending_buf_q)) {
>> +               vbuf = fimc_lite_pending_queue_pop(fimc);
>> +               flite_hw_set_output_addr(fimc, vbuf->paddr,
>> +                                       vbuf->vb.v4l2_buf.index);
>> +               fimc_lite_active_queue_add(fimc, vbuf);
>> +       }
>> +
>> +       if ((intsrc&  FLITE_REG_CISTATUS_IRQ_SRC_FRMEND)&&
>> +           test_bit(ST_FLITE_RUN,&fimc->state)&&
>> +           !list_empty(&fimc->active_buf_q)) {
>>                 vbuf = fimc_lite_active_queue_pop(fimc);
>>                 ktime_get_ts(&ts);
>>                 tv =&vbuf->vb.v4l2_buf.timestamp;
>>
>>                 tv->tv_sec = ts.tv_sec;
>>                 tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
>>                 vbuf->vb.v4l2_buf.sequence = fimc->frame_count++;
>> +               flite_hw_clear_output_addr(fimc, vbuf->vb.v4l2_buf.index);
>>                 vb2_buffer_done(&vbuf->vb, VB2_BUF_STATE_DONE);
>> -
>> -               vbuf = fimc_lite_pending_queue_pop(fimc);
>> -               flite_hw_set_output_addr(fimc, vbuf->paddr);
>> -               fimc_lite_active_queue_add(fimc, vbuf);
>>         }
>>
>>         if (test_bit(ST_FLITE_CONFIG,&fimc->state))
>>
>> @@ -406,7 +413,8 @@ static void buffer_queue(struct vb2_buffer *vb)
>>         if (!test_bit(ST_FLITE_SUSPENDED,&fimc->state)&&
>>         !test_bit(ST_FLITE_STREAM,&fimc->state)&&
>>         list_empty(&fimc->active_buf_q)) {
>> -               flite_hw_set_output_addr(fimc, buf->paddr);
>> +               flite_hw_set_output_addr(fimc, buf->paddr,
>> +                                       buf->vb.v4l2_buf.index);
>>                 fimc_lite_active_queue_add(fimc, buf);
>>         } else {
>>                 fimc_lite_pending_queue_add(fimc, buf);
>> @@ -646,7 +654,7 @@ static int fimc_vidioc_querycap_capture(struct file
>> *file, void *priv,
>>         strlcpy(cap->driver, FIMC_LITE_DRV_NAME, sizeof(cap->driver));
>>         cap->bus_info[0] = 0;
>>         cap->card[0] = 0;
>> -       cap->capabilities = V4L2_CAP_STREAMING;
>> +       cap->capabilities = V4L2_CAP_STREAMING |
>> V4L2_CAP_VIDEO_CAPTURE_MPLANE;
>>         return 0;
>>   }
>>
>> @@ -725,13 +733,125 @@ static int fimc_lite_try_fmt_mplane(struct file
>> *file, void *fh,
>>         return fimc_lite_try_fmt(fimc,&f->fmt.pix_mp, NULL);
>>
>>   }
>>
>> -static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
>> -                                 struct v4l2_format *f)
>> +static struct media_entity *fimc_pipeline_get_head(struct media_entity
>> *me)
>>   {
>> -       struct v4l2_pix_format_mplane *pixm =&f->fmt.pix_mp;
>>
>> -       struct fimc_lite *fimc = video_drvdata(file);
>> +       struct media_pad *pad =&me->pads[0];
>> +
>> +       while (!(pad->flags&  MEDIA_PAD_FL_SOURCE)) {
>>
>> +               pad = media_entity_remote_source(pad);
>> +               if (!pad)
>> +                       break;
>> +               me = pad->entity;
>> +               pad =&me->pads[0];
>>
>> +       }
>> +
>> +       return me;
>> +}
>> +
>> +/**
>> + * fimc_pipeline_try_format - negotiate and/or set formats at pipeline
>> + *                            elements
>> + * @ctx: FIMC capture context
>> + * @tfmt: media bus format to try/set on subdevs
>> + * @fmt_id: fimc pixel format id corresponding to returned @tfmt (output)
>> + * @set: true to set format on subdevs, false to try only
>> + */
>> +static int fimc_pipeline_try_format(struct fimc_lite *fimc,
>> +                                   struct v4l2_mbus_framefmt *tfmt,
>> +                                   struct fimc_fmt **fmt_id,
>> +                                   bool set)
>> +{
>
> [...]
>>
>> +}
>> +
>> +
>> +static int __fimc_lite_set_format(struct fimc_lite *fimc,
>> +                                    struct v4l2_format *f)
>> +{
>> +       struct v4l2_pix_format_mplane *pix_mp =&f->fmt.pix_mp;
>>         struct flite_frame *frame =&fimc->out_frame;
>>
>>         const struct fimc_fmt *fmt = NULL;
>> +       struct v4l2_mbus_framefmt mf;
>> +       struct fimc_fmt *s_fmt = NULL;
>>         int ret;
>>
>>         if (vb2_is_busy(&fimc->vb_queue))
>> @@ -741,15 +861,59 @@ static int fimc_lite_s_fmt_mplane(struct file *file,
>> void *priv,
>>         if (ret<  0)
>>                 return ret;
>>
>> +       /* Reset cropping and set format at the camera interface input */
>> +       if (!fimc->user_subdev_api) {
>> +               fimc->inp_frame.f_width = pix_mp->width;
>> +               fimc->inp_frame.f_height = pix_mp->height;
>> +               fimc->inp_frame.rect.top = 0;
>> +               fimc->inp_frame.rect.left = 0;
>> +               fimc->inp_frame.rect.width = pix_mp->width;
>> +               fimc->inp_frame.rect.height = pix_mp->height;
>> +       }
>> +
>> +       /* Try to match format at the host and the sensor */
>> +       if (!fimc->user_subdev_api) {
>> +               mf.code   = fmt->mbus_code;
>> +               mf.width  = pix_mp->width;
>> +               mf.height = pix_mp->height;
>> +               ret = fimc_pipeline_try_format(fimc,&mf,&s_fmt, true);
>>
>> +               if (ret)
>> +                       return ret;
>> +
>> +               pix_mp->width  = mf.width;
>> +               pix_mp->height = mf.height;
>> +       }
>> +
>>         fimc->fmt = fmt;
>> -       fimc->payload[0] = max((pixm->width * pixm->height *
>> fmt->depth[0]) / 8,
>> -                              pixm->plane_fmt[0].sizeimage);
>> -       frame->f_width = pixm->width;
>> -       frame->f_height = pixm->height;
>> +       fimc->payload[0] = max((pix_mp->width * pix_mp->height *
>> +                       fmt->depth[0]) / 8,
>> pix_mp->plane_fmt[0].sizeimage);
>> +       frame->f_width = pix_mp->width;
>> +       frame->f_height = pix_mp->height;
>>
>>         return 0;
>>   }
>>
>> +static int fimc_lite_s_fmt_mplane(struct file *file, void *priv,
>> +                                 struct v4l2_format *f)
>> +{
>> +       struct fimc_lite *fimc = video_drvdata(file);
>> +       int ret;
>> +
>> +       exynos_pipeline_graph_lock(fimc->pipeline_ops,&fimc->pipeline);
>>
>> +       /*
>> +        * The graph is walked within __fimc_lite_set_format() to set
>> +        * the format at subdevs thus the graph mutex needs to be held at
>> +        * this point and acquired before the video mutex, to avoid  AB-BA
>> +        * deadlock when fimc_md_link_notify() is called by other thread.
>> +        * Ideally the graph walking and setting format at the whole
>> pipeline
>> +        * should be removed from this driver and handled in userspace
>> only.
>
>
> You have copied this format setting code from fimc-capture.c, while I
> would rather remove it from there as well. But it have to stay for
> backward compatibility. Nevertheless on Exynos4x12 by default image
> format on whole pipeline will not be set in kernel by the video capture
> node driver, there is simply to many entities involved there and for
> full flexibility format setting/negotiation at the whole pipeline needs
> to be done by user space library. Or you can use media-ctl to configure
> the formats at each subdev.
>

Yes you are right. Even i was not sure to include this part. but just to align
with fimc-capture, i took this part of code. It would be great that individual
entity format settings can be set by the user space. Driver will only check
the source->sink formats at all links to see they are properly set by the user.

I will completely agree with you and will remove this code in the next posting.

>
>
>> --- a/drivers/media/platform/s5p-fimc/fimc-lite.h
>> +++ b/drivers/media/platform/s5p-fimc/fimc-lite.h
>> @@ -52,7 +52,6 @@ enum {
>>   #define FLITE_VER_EXYNOS4     0
>>   #define FLITE_VER_EXYNOS5     1
>>
>> -
>>   struct flite_variant {
>>         unsigned short max_width;
>>         unsigned short max_height;
>> @@ -175,6 +174,8 @@ struct fimc_lite {
>>         unsigned int            reqbufs_count;
>>         int                     ref_count;
>>
>> +       bool                    user_subdev_api;
>
>
> No, please don't copy this. I don't really want to see that sysfs
> workaround in other drivers.

Yes, as explained above, if user takes care of format settings for each entity
in the pipeline, we don't need to bother about having this workarounds.

Regards,
Shaik Ameer Basha

>
> --
>
> Regards,
> Sylwester

Re: [RFC 04/12] s5p-csis: Adding Exynos5250 compatibility

[Shaik Ameer Basha]

Hi Sylwester,

On Mon, Mar 11, 2013 at 2:10 AM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>
> Please don't leave the change log empty. I'll apply this patch.
> I'm just wondering, if there aren't any further changes needed
> to make the driver really working on exynos5250 ?
>

There was nothing from driver side to change for making it work
for Exynos5250. May be I need to update the S5P_INTMASK_EN_ALL
to include all interrupts.

Regards,
Shaik Ameer Basha

>
>> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
>> ---
>>   drivers/media/platform/s5p-fimc/mipi-csis.c |    1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/drivers/media/platform/s5p-fimc/mipi-csis.c
>> b/drivers/media/platform/s5p-fimc/mipi-csis.c
>> index df4411c..debda7c 100644
>> --- a/drivers/media/platform/s5p-fimc/mipi-csis.c
>> +++ b/drivers/media/platform/s5p-fimc/mipi-csis.c
>> @@ -1002,6 +1002,7 @@ static const struct dev_pm_ops s5pcsis_pm_ops = {
>>   static const struct of_device_id s5pcsis_of_match[] __devinitconst = {
>>         { .compatible = "samsung,exynos3110-csis" },
>>         { .compatible = "samsung,exynos4210-csis" },
>> +       { .compatible = "samsung,exynos5250-csis" },
>>         { /* sentinel */ },
>>   };
>>   MODULE_DEVICE_TABLE(of, s5pcsis_of_match);

Re: [RFC 05/12] ARM: EXYNOS: Add devicetree node for mipi-csis driver for exynos5

[Shaik Ameer Basha]

Hi Sylwester,

On Mon, Mar 11, 2013 at 2:24 AM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> This patch adds necessary source definations needed for mipi-csis
>> driver and adds devicetree node for exynos5250.
>>
>> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
>> ---
>>   arch/arm/boot/dts/exynos5250.dtsi       |   18 ++++++++++++++++++
>>   arch/arm/mach-exynos/clock-exynos5.c    |   16 ++++++++++++++--
>>   arch/arm/mach-exynos/include/mach/map.h |    3 +++
>>   arch/arm/mach-exynos/mach-exynos5-dt.c  |    4 ++++
>>   4 files changed, 39 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm/boot/dts/exynos5250.dtsi
>> b/arch/arm/boot/dts/exynos5250.dtsi
>> index 3a2cd9a..4fff98b 100644
>> --- a/arch/arm/boot/dts/exynos5250.dtsi
>> +++ b/arch/arm/boot/dts/exynos5250.dtsi
>> @@ -47,6 +47,8 @@
>>                 i2c6 =&i2c_6;
>>                 i2c7 =&i2c_7;
>>                 i2c8 =&i2c_8;
>> +               csis0 =&csis_0;
>> +               csis1 =&csis_1;
>>         };
>>
>>         gic:interrupt-controller@10481000 {
>> @@ -357,4 +359,20 @@
>>                 reg =<0x14450000 0x10000>;
>>                 interrupts =<0 94 0>;
>>         };
>> +
>> +       csis_0: csis@13C20000 {
>> +               compatible = "samsung,exynos5250-csis";
>> +               reg =<0x13C20000 0x4000>;
>> +               interrupts =<0 79 0>;
>> +               bus-width =<4>;
>> +               status = "disabled";
>> +       };
>> +
>> +       csis_1: csis@13C30000 {
>> +               compatible = "samsung,exynos5250-csis";
>> +               reg =<0x13C30000 0x4000>;
>> +               interrupts =<0 80 0>;
>> +               bus-width =<4>;
>
>
> Shouldn't this be 2 ? Anyway what's the point of adding this node here

Yes, It has to be 2. Seems some Ctrl+c/v issues. :)

> only to move it in a subsequent patch ? I guess you should first add
> 'camera' node and then have further patches adding relevant device nodes

Ok.. I even got the same comments just before postings. As it was a RFC patch,
I just posted this as it is.
Definitely, i will modify this in the upcoming version of patches.


Regards,
Shaik Ameer Basha

>
>
>> +               status = "disabled";
>> +       };
>>   };
>
>
>> diff --git a/arch/arm/mach-exynos/clock-exynos5.c
>> b/arch/arm/mach-exynos/clock-exynos5.c
>> index e9d7b80..34a22ff 100644
>> --- a/arch/arm/mach-exynos/clock-exynos5.c
>> +++ b/arch/arm/mach-exynos/clock-exynos5.c
>
>
> This file is already removed in Kukjin's for-next tree.
> And for dts changes I would start the patch summary line with "ARM: dts:".
>
>
>> @@ -859,6 +859,16 @@ static struct clk exynos5_init_clocks_off[] = {
>>                 .enable         = exynos5_clk_ip_gscl_ctrl,
>>                 .ctrlbit        = (1<<  3),
>>         }, {
>> +               .name           = "csis",
>> +               .devname        = "s5p-mipi-csis.0",
>> +               .enable         = exynos5_clk_ip_gscl_ctrl,
>> +               .ctrlbit        = (1<<  5),
>> +       }, {
>> +               .name           = "csis",
>> +               .devname        = "s5p-mipi-csis.1",
>> +               .enable         = exynos5_clk_ip_gscl_ctrl,
>> +               .ctrlbit        = (1<<  6),
>> +       }, {

Re: [RFC 05/12] ARM: EXYNOS: Add devicetree node for mipi-csis driver for exynos5

[Shaik Ameer Basha]

On Mon, Mar 11, 2013 at 2:27 AM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> --- a/arch/arm/boot/dts/exynos5250.dtsi
>> +++ b/arch/arm/boot/dts/exynos5250.dtsi
>> @@ -47,6 +47,8 @@
>>                 i2c6 =&i2c_6;
>>                 i2c7 =&i2c_7;
>>                 i2c8 =&i2c_8;
>> +               csis0 =&csis_0;
>> +               csis1 =&csis_1;
>
>
> You can drop these aliases if you use my latest patches as indicated
> in the comment to patch 00/12.
>

Ok.

>
>> diff --git a/arch/arm/mach-exynos/clock-exynos5.c
>> b/arch/arm/mach-exynos/clock-exynos5.c
>> index e9d7b80..34a22ff 100644
>> --- a/arch/arm/mach-exynos/clock-exynos5.c
>> +++ b/arch/arm/mach-exynos/clock-exynos5.c
>> @@ -859,6 +859,16 @@ static struct clk exynos5_init_clocks_off[] = {
>>                 .enable         = exynos5_clk_ip_gscl_ctrl,
>>                 .ctrlbit        = (1<<  3),
>>         }, {
>> +               .name           = "csis",
>> +               .devname        = "s5p-mipi-csis.0",
>> +               .enable         = exynos5_clk_ip_gscl_ctrl,
>> +               .ctrlbit        = (1<<  5),
>> +       }, {
>
>
> Instead you should add relevant clock definitions to the Samsung clocks
> driver,
> it's already merged in Kukjin's tree.
>

No Issues, the next version posting will be re-based on the Samsung
common clock driver.

Regards,
Shaik Ameer Basha

>
>
>
>

Re: [RFC 07/12] media: exynos5-is: Adding media device driver for exynos5

[Shaik Ameer Basha]

Hi Sylwester,

Thanks for the review.
I will address all you comments for this patch in the next version.

Thanks,
Shaik Ameer Basha

On Mon, Mar 11, 2013 at 3:58 AM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> This patch adds support for media device for EXYNOS5 SoCs.
>> The current media device supports the following ips to connect
>> through the media controller framework.
>>
>> * MIPI-CSIS
>>    Support interconnection(subdev interface) between devices
>>
>> * FIMC-LITE
>>    Support capture interface from device(Sensor, MIPI-CSIS) to memory
>>    Support interconnection(subdev interface) between devices
>>
>> G-Scaler will be added later to the current media device.
>>
>> * Gscaler: general scaler
>>    Support memory to memory interface
>>    Support output interface from memory to display device(LCD, TV)
>>    Support capture interface from device(FIMC-LITE, FIMD) to memory
>>
>> -->  media 0
>>    Camera Capture path consists of MIPI-CSIS, FIMC-LITE and G-Scaler
>>    +--------+     +-----------+     +-----------------+
>>    | Sensor | -->  | FIMC-LITE | -->  | G-Scaler-capture |
>>    +--------+     +-----------+     +-----------------+
>>
>>    +--------+     +-----------+     +-----------+     +-----------------+
>>    | Sensor | -->  | MIPI-CSIS | -->  | FIMC-LITE | -->  |
>> G-Scaler-capture |
>>    +--------+     +-----------+     +-----------+     +-----------------+
>>
>> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
>> ---
>>   drivers/media/platform/Kconfig                   |    1 +
>>   drivers/media/platform/Makefile                  |    1 +
>>   drivers/media/platform/exynos5-is/Kconfig        |    7 +
>>   drivers/media/platform/exynos5-is/Makefile       |    4 +
>>   drivers/media/platform/exynos5-is/exynos5-mdev.c | 1309
>> ++++++++++++++++++++++
>>   drivers/media/platform/exynos5-is/exynos5-mdev.h |  107 ++
>>   6 files changed, 1429 insertions(+)
>>   create mode 100644 drivers/media/platform/exynos5-is/Kconfig
>>   create mode 100644 drivers/media/platform/exynos5-is/Makefile
>>   create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.c
>>   create mode 100644 drivers/media/platform/exynos5-is/exynos5-mdev.h
>>
> ...
>
>> diff --git a/drivers/media/platform/exynos5-is/exynos5-mdev.c
>> b/drivers/media/platform/exynos5-is/exynos5-mdev.c
>> new file mode 100644
>> index 0000000..1158696
>> --- /dev/null
>> +++ b/drivers/media/platform/exynos5-is/exynos5-mdev.c
>> @@ -0,0 +1,1309 @@
>> +/*
>> + * S5P/EXYNOS4 SoC series camera host interface media device driver
>
>
> EXYNOS5
>
>
>> + * Copyright (C) 2011 - 2012 Samsung Electronics Co., Ltd.
>> + * Sylwester Nawrocki<s.nawrocki@samsung.com>
>
>
> This is incorrect too, you should add your authorship and a note it
> is based on other code, with a proper copyright notice.
>
>
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License as published
>> + * by the Free Software Foundation, either version 2 of the License,
>> + * or (at your option) any later version.
>> + */
>> +
>> +#include<linux/bug.h>
>> +#include<linux/device.h>
>> +#include<linux/errno.h>
>> +#include<linux/i2c.h>
>> +#include<linux/kernel.h>
>> +#include<linux/list.h>
>> +#include<linux/module.h>
>> +#include<linux/of.h>
>> +#include<linux/of_platform.h>
>> +#include<linux/of_device.h>
>> +#include<linux/of_i2c.h>
>> +#include<linux/pinctrl/consumer.h>
>> +#include<linux/platform_device.h>
>> +#include<linux/pm_runtime.h>
>> +#include<linux/types.h>
>> +#include<linux/slab.h>
>> +#include<media/v4l2-ctrls.h>
>> +#include<media/v4l2-of.h>
>> +#include<media/media-device.h>
>> +
>> +#include "exynos5-mdev.h"
>> +
>> +#define dbg(fmt, args...) \
>> +       pr_debug("%s:%d: " fmt "\n", __func__, __LINE__, ##args)
>> +
>> +static struct fimc_md *g_exynos_mdev;
>> +
>> +static int fimc_md_set_camclk(struct v4l2_subdev *sd, bool on);
>> +static int __fimc_md_set_camclk(struct fimc_md *fmd,
>> +                               struct fimc_sensor_info *s_info,
>> +                               bool on);
>> +/**
>> + * fimc_pipeline_prepare - update pipeline information with subdevice
>> pointers
>> + * @fimc: fimc device terminating the pipeline
>> + *
>> + * Caller holds the graph mutex.
>> + */
>> +static void fimc_pipeline_prepare(struct exynos5_pipeline0 *p,
>> +                                 struct media_entity *me)
>> +{
>> +       struct media_pad *pad =&me->pads[0];
>
>
> This will need to be changed to support subdevs with more than 2 pads.
> I should post relevant patch this week.
>
>> +       struct v4l2_subdev *sd;
>> +       int i;
>> +
>> +       for (i = 0; i<  IDX_MAX; i++)
>> +               p->subdevs[i] = NULL;
>> +
>> +       while (1) {
>> +
>> +               if (!(pad->flags&  MEDIA_PAD_FL_SINK))
>>
>> +                       break;
>> +
>> +               /* source pad */
>> +               pad = media_entity_remote_source(pad);
>> +
>> +               if (pad != NULL)
>> +                       pr_err("entity type: %d, entity name: %s\n",
>> +                           media_entity_type(pad->entity),
>> pad->entity->name);
>> +
>> +               if (pad == NULL ||
>> +                   media_entity_type(pad->entity) !=
>> MEDIA_ENT_T_V4L2_SUBDEV)
>> +                       break;
>> +
>> +               sd = media_entity_to_v4l2_subdev(pad->entity);
>> +
>> +               switch (sd->grp_id) {
>> +               case GRP_ID_FIMC_IS_SENSOR:
>> +               case GRP_ID_SENSOR:
>> +                       p->subdevs[IDX_SENSOR] = sd;
>> +                       break;
>> +               case GRP_ID_CSIS:
>> +                       p->subdevs[IDX_CSIS] = sd;
>> +                       break;
>> +               case GRP_ID_FLITE:
>> +                       p->subdevs[IDX_FLITE] = sd;
>> +                       break;
>> +               default:
>> +                       pr_warn("%s: Unknown subdev grp_id: %#x\n",
>> +                               __func__, sd->grp_id);
>> +               }
>> +
>> +               /* sink pad */
>> +               pad =&sd->entity.pads[0];
>> +       }
>> +}
>
> ...
>
>> +/*
>> + * Sensor subdevice helper functions
>> + */
>> +static struct v4l2_subdev *fimc_md_register_sensor(struct fimc_md *fmd,
>> +                                  struct fimc_sensor_info *s_info)
>> +{
>> +       struct i2c_adapter *adapter;
>> +       struct v4l2_subdev *sd = NULL;
>> +
>> +       if (!s_info || !fmd)
>> +               return NULL;
>> +
>> +       adapter = i2c_get_adapter(s_info->pdata.i2c_bus_num);
>> +       if (!adapter) {
>> +               v4l2_warn(&fmd->v4l2_dev,
>> +                         "Failed to get I2C adapter %d, deferring
>> probe\n",
>> +                         s_info->pdata.i2c_bus_num);
>> +               return ERR_PTR(-EPROBE_DEFER);
>> +       }
>> +       sd = v4l2_i2c_new_subdev_board(&fmd->v4l2_dev, adapter,
>> +                                      s_info->pdata.board_info, NULL);
>> +       if (IS_ERR_OR_NULL(sd)) {
>> +               i2c_put_adapter(adapter);
>> +               v4l2_warn(&fmd->v4l2_dev,
>> +                         "Failed to acquire subdev %s, deferring
>> probe\n",
>> +                         s_info->pdata.board_info->type);
>> +               return ERR_PTR(-EPROBE_DEFER);
>> +       }
>> +       v4l2_set_subdev_hostdata(sd, s_info);
>> +       sd->grp_id = GRP_ID_SENSOR;
>> +
>> +       v4l2_info(&fmd->v4l2_dev, "Registered sensor subdevice %s\n",
>> +                 sd->name);
>> +       return sd;
>> +}
>> +
>> +static void fimc_md_unregister_sensor(struct v4l2_subdev *sd)
>> +{
>> +       struct i2c_client *client = v4l2_get_subdevdata(sd);
>> +       struct i2c_adapter *adapter;
>> +
>> +       if (!client)
>> +               return;
>> +       v4l2_device_unregister_subdev(sd);
>> +
>> +       if (!client->dev.of_node) {
>> +               adapter = client->adapter;
>> +               i2c_unregister_device(client);
>> +               if (adapter)
>> +                       i2c_put_adapter(adapter);
>> +       }
>> +}
>
>
> You don't need the above code which is for !CONFIG_OF only.
>
>> +#ifdef CONFIG_OF
>
>
> Just add "depends on OF" in Kconfig for the whole driver and
> you can drop all #ifdef CONFIG_OF.
>
>> +/* Register I2C client subdev associated with @node. */
>> +static int fimc_md_of_add_sensor(struct fimc_md *fmd,
>> +                                struct device_node *node, int index)
>> +{
>> +       struct fimc_sensor_info *si;
>> +       struct i2c_client *client;
>> +       struct v4l2_subdev *sd;
>> +       int ret;
>> +
>> +       if (WARN_ON(index>= ARRAY_SIZE(fmd->sensor)))
>> +               return -EINVAL;
>> +
>> +       si =&fmd->sensor[index];
>>
>> +
>> +       client = of_find_i2c_device_by_node(node);
>> +       if (!client)
>> +               return -EPROBE_DEFER;
>> +
>> +       device_lock(&client->dev);
>> +
>> +       if (!client->driver ||
>> +           !try_module_get(client->driver->driver.owner)) {
>> +               ret = -EPROBE_DEFER;
>> +               goto dev_put;
>> +       }
>> +
>> +       /* Enable sensor's master clock */
>> +       ret = __fimc_md_set_camclk(fmd, si, true);
>> +       if (ret<  0)
>> +               goto mod_put;
>> +
>> +       sd = i2c_get_clientdata(client);
>> +
>> +       ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
>> +       __fimc_md_set_camclk(fmd, si, false);
>> +       if (ret<  0)
>> +               goto mod_put;
>> +
>> +       v4l2_set_subdev_hostdata(sd, si);
>> +       sd->grp_id = GRP_ID_SENSOR;
>> +       si->subdev = sd;
>> +       v4l2_info(&fmd->v4l2_dev, "Registered sensor subdevice: %s
>> (%d)\n",
>> +                 sd->name, fmd->num_sensors);
>> +       fmd->num_sensors++;
>> +
>> +mod_put:
>> +       module_put(client->driver->driver.owner);
>> +dev_put:
>> +       device_unlock(&client->dev);
>> +       put_device(&client->dev);
>> +       return ret;
>> +}
>
> ...
>
>> +#else
>> +#define fimc_md_of_sensors_register(fmd, np) (-ENOSYS)
>
>
> This would never be used since Exynos5 is a dt-only platform.
>
>
>> +#endif
>> +
>> +static int fimc_md_register_sensor_entities(struct fimc_md *fmd)
>> +{
>> +       struct s5p_platform_fimc *pdata = fmd->pdev->dev.platform_data;
>> +       struct device_node *of_node = fmd->pdev->dev.of_node;
>> +       int num_clients = 0;
>> +       int ret, i;
>> +
>> +       if (of_node) {
>> +               fmd->num_sensors = 0;
>> +               ret = fimc_md_of_sensors_register(fmd, of_node);
>> +       } else if (pdata) {
>
>
> The code below is for !CONFIG_OF only, no need to repeat it for Exynos5.
>
>> +               WARN_ON(pdata->num_clients>  ARRAY_SIZE(fmd->sensor));
>> +               num_clients = min_t(u32, pdata->num_clients,
>> +                                   ARRAY_SIZE(fmd->sensor));
>> +               fmd->num_sensors = num_clients;
>> +
>> +               fmd->num_sensors = num_clients;
>> +               for (i = 0; i<  num_clients; i++) {
>> +                       struct v4l2_subdev *sd;
>> +
>> +                       fmd->sensor[i].pdata = pdata->source_info[i];
>> +                       ret = __fimc_md_set_camclk(fmd,&fmd->sensor[i],
>> true);
>>
>> +                       if (ret)
>> +                               break;
>> +                       sd = fimc_md_register_sensor(fmd,&fmd->sensor[i]);
>> +                       ret = __fimc_md_set_camclk(fmd,&fmd->sensor[i],
>> false);
>>
>> +
>> +                       if (IS_ERR(sd)) {
>> +                               fmd->sensor[i].subdev = NULL;
>> +                               ret = PTR_ERR(sd);
>> +                               break;
>> +                       }
>> +                       fmd->sensor[i].subdev = sd;
>> +                       if (ret)
>> +                               break;
>> +               }
>> +       }
>> +
>> +       return ret;
>> +}
>> +
>> +/*
>> + * MIPI-CSIS, FIMC and FIMC-LITE platform devices registration.
>> + */
>> +
>> +static int register_fimc_lite_entity(struct fimc_md *fmd,
>> +                                    struct fimc_lite *fimc_lite)
>> +{
>> +       struct v4l2_subdev *sd;
>> +       int ret;
>> +
>> +       if (WARN_ON(fimc_lite->index>= FIMC_LITE_MAX_DEVS ||
>> +                   fmd->fimc_lite[fimc_lite->index]))
>> +               return -EBUSY;
>> +
>> +       sd =&fimc_lite->subdev;
>>
>> +       sd->grp_id = GRP_ID_FLITE;
>> +       v4l2_set_subdev_hostdata(sd, (void *)&exynos5_pipeline0_ops);
>> +
>> +       ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
>> +       if (!ret)
>> +               fmd->fimc_lite[fimc_lite->index] = fimc_lite;
>> +       else
>> +               v4l2_err(&fmd->v4l2_dev, "Failed to register
>> FIMC.LITE%d\n",
>> +                        fimc_lite->index);
>> +       return ret;
>> +}
>> +
>> +static int register_csis_entity(struct fimc_md *fmd,
>> +                               struct platform_device *pdev,
>> +                               struct v4l2_subdev *sd)
>> +{
>> +       struct device_node *node = pdev->dev.of_node;
>> +       int id, ret;
>> +
>> +       id = node ? of_alias_get_id(node, "csis") : max(0, pdev->id);
>> +
>> +       if (WARN_ON(id>= CSIS_MAX_ENTITIES || fmd->csis[id].sd))
>> +               return -EBUSY;
>> +
>> +       if (WARN_ON(id>= CSIS_MAX_ENTITIES))
>> +               return 0;
>> +
>> +       sd->grp_id = GRP_ID_CSIS;
>> +       ret = v4l2_device_register_subdev(&fmd->v4l2_dev, sd);
>> +       if (!ret)
>> +               fmd->csis[id].sd = sd;
>> +       else
>> +               v4l2_err(&fmd->v4l2_dev,
>> +                        "Failed to register MIPI-CSIS.%d (%d)\n", id,
>> ret);
>> +
>> +       return ret;
>> +}
>> +
>> +static int fimc_md_register_platform_entity(struct fimc_md *fmd,
>> +                                           struct platform_device *pdev,
>> +                                           int plat_entity)
>> +{
>> +       struct device *dev =&pdev->dev;
>>
>> +       int ret = -EPROBE_DEFER;
>> +       void *drvdata;
>> +
>> +       /* Lock to ensure dev->driver won't change. */
>> +       device_lock(dev);
>> +
>> +       if (!dev->driver || !try_module_get(dev->driver->owner))
>> +               goto dev_unlock;
>> +
>> +       drvdata = dev_get_drvdata(dev);
>> +       /* Some subdev didn't probe succesfully id drvdata is NULL */
>> +       if (drvdata) {
>> +               switch (plat_entity) {
>> +               case IDX_FLITE:
>> +                       ret = register_fimc_lite_entity(fmd, drvdata);
>> +                       break;
>> +               case IDX_CSIS:
>> +                       ret = register_csis_entity(fmd, pdev, drvdata);
>> +                       break;
>> +               default:
>> +                       ret = -ENODEV;
>> +               }
>> +       }
>> +
>> +       module_put(dev->driver->owner);
>> +dev_unlock:
>> +       device_unlock(dev);
>> +       if (ret == -EPROBE_DEFER)
>> +               dev_info(&fmd->pdev->dev, "deferring %s device
>> registration\n",
>> +                       dev_name(dev));
>> +       else if (ret<  0)
>> +               dev_err(&fmd->pdev->dev, "%s device registration failed
>> (%d)\n",
>> +                       dev_name(dev), ret);
>> +       return ret;
>> +}
>> +
>> +static int fimc_md_pdev_match(struct device *dev, void *data)
>
>
> This function is also useless in your case.
>
>
>> +{
>> +       struct platform_device *pdev = to_platform_device(dev);
>> +       int plat_entity = -1;
>> +       int ret;
>> +
>> +       if (!get_device(dev))
>> +               return -ENODEV;
>> +
>> +       if (!strcmp(pdev->name, CSIS_DRIVER_NAME))
>> +               plat_entity = IDX_CSIS;
>> +       else if (!strcmp(pdev->name, FIMC_LITE_DRV_NAME))
>> +               plat_entity = IDX_FLITE;
>> +
>> +       if (plat_entity>= 0)
>> +               ret = fimc_md_register_platform_entity(data, pdev,
>> +                                                      plat_entity);
>> +       put_device(dev);
>> +       return 0;
>> +}
>> +
>> +/* Register FIMC, FIMC-LITE and CSIS media entities */
>> +#ifdef CONFIG_OF
>> +static int fimc_md_register_of_platform_entities(struct fimc_md *fmd,
>> +                                                struct device_node
>> *parent)
>> +{
>> +       struct device_node *node;
>> +       int ret = 0;
>> +
>> +       for_each_available_child_of_node(parent, node) {
>> +               struct platform_device *pdev;
>> +               int plat_entity = -1;
>> +
>> +               pdev = of_find_device_by_node(node);
>> +               if (!pdev)
>> +                       continue;
>> +
>> +               /* If driver of any entity isn't ready try all again
>> later. */
>> +               if (!strcmp(node->name, CSIS_OF_NODE_NAME))
>> +                       plat_entity = IDX_CSIS;
>> +               else if (!strcmp(node->name, FIMC_LITE_OF_NODE_NAME))
>> +                       plat_entity = IDX_FLITE;
>> +
>> +               if (plat_entity>= 0)
>> +                       ret = fimc_md_register_platform_entity(fmd, pdev,
>> +                                                       plat_entity);
>> +               put_device(&pdev->dev);
>> +               if (ret<  0)
>> +                       break;
>> +       }
>> +
>> +       return ret;
>> +}
>> +#else
>> +#define fimc_md_register_platform_entities(fmd) (-ENOSYS)
>> +#endif
>
> ...
>>
>> +static int fimc_md_link_notify(struct media_pad *source,
>> +                              struct media_pad *sink, u32 flags)
>> +{
>> +       struct fimc_lite *fimc_lite = NULL;
>> +       struct exynos_pipeline *pipeline;
>> +       struct exynos5_pipeline0 *p;
>> +       struct v4l2_subdev *sd;
>> +       struct mutex *lock;
>> +       int ret = 0;
>> +       int ref_count;
>> +
>> +       if (media_entity_type(sink->entity) != MEDIA_ENT_T_V4L2_SUBDEV)
>> +               return 0;
>> +
>> +       sd = media_entity_to_v4l2_subdev(sink->entity);
>> +
>> +       switch (sd->grp_id) {
>> +       case GRP_ID_FLITE:
>> +               fimc_lite = v4l2_get_subdevdata(sd);
>> +               if (WARN_ON(fimc_lite == NULL))
>> +                       return 0;
>> +               pipeline =&fimc_lite->pipeline;
>> +               lock =&fimc_lite->lock;
>>
>> +               break;
>> +       default:
>> +               return 0;
>> +       }
>> +
>> +       p = (struct exynos5_pipeline0 *)pipeline->priv;
>> +       if (!p || !p->is_init)
>> +               return -EINVAL;
>> +
>> +       if (!(flags&  MEDIA_LNK_FL_ENABLED)) {
>>
>> +               int i;
>> +               mutex_lock(lock);
>
>
> ref_count needs to be checked here as well.
>
>
>> +               ret = __fimc_pipeline_close(pipeline);
>> +               for (i = 0; i<  IDX_MAX; i++)
>> +                       p->subdevs[i] = NULL;
>> +               mutex_unlock(lock);
>> +               return ret;
>> +       }
>> +       /*
>> +        * Link activation. Enable power of pipeline elements only if the
>> +        * pipeline is already in use, i.e. its video node is opened.
>> +        * Recreate the controls destroyed during the link deactivation.
>
>
> I think this line should be deleted.
>
>
>> +        */
>> +       mutex_lock(lock);
>> +
>> +       ref_count = fimc_lite->ref_count;
>> +       if (ref_count>  0)
>> +               ret = __fimc_pipeline_open(pipeline, source->entity,
>> true);
>> +
>> +       mutex_unlock(lock);
>> +       return ret ? -EPIPE : ret;
>> +}
>> +
>> +static ssize_t fimc_md_sysfs_show(struct device *dev,
>> +                                 struct device_attribute *attr, char
>> *buf)
>> +{
>> +       struct platform_device *pdev = to_platform_device(dev);
>> +       struct fimc_md *fmd = platform_get_drvdata(pdev);
>> +
>> +       if (fmd->user_subdev_api)
>> +               return strlcpy(buf, "Sub-device API (sub-dev)\n",
>> PAGE_SIZE);
>> +
>> +       return strlcpy(buf, "V4L2 video node only API (vid-dev)\n",
>> PAGE_SIZE);
>> +}
>> +
>> +static ssize_t fimc_md_sysfs_store(struct device *dev,
>> +                                  struct device_attribute *attr,
>> +                                  const char *buf, size_t count)
>> +{
>> +       struct platform_device *pdev = to_platform_device(dev);
>> +       struct fimc_md *fmd = platform_get_drvdata(pdev);
>> +       bool subdev_api;
>> +       int i;
>> +
>> +       if (!strcmp(buf, "vid-dev\n"))
>> +               subdev_api = false;
>> +       else if (!strcmp(buf, "sub-dev\n"))
>> +               subdev_api = true;
>> +       else
>> +               return count;
>> +
>> +       fmd->user_subdev_api = subdev_api;
>> +       for (i = 0; i<  FIMC_LITE_MAX_DEVS; i++)
>> +               if (fmd->fimc_lite[i])
>> +                       fmd->fimc_lite[i]->user_subdev_api = subdev_api;
>> +
>> +       return count;
>> +}
>> +/*
>> + * This device attribute is to select video pipeline configuration
>> method.
>> + * There are following valid values:
>> + *  vid-dev - for V4L2 video node API only, subdevice will be configured
>> + *  by the host driver.
>> + *  sub-dev - for media controller API, subdevs must be configured in
>> user
>> + *  space before starting streaming.
>> + */
>> +static DEVICE_ATTR(subdev_conf_mode, S_IWUSR | S_IRUGO,
>> +                  fimc_md_sysfs_show, fimc_md_sysfs_store);
>
>
> Please don't copy that. It is a not standard attribute that you should not
> need in this driver anyway.
>
>> +/**
>> + * fimc_md_sensor_notify - v4l2_device notification from a sensor subdev
>> + * @sd: pointer to a subdev generating the notification
>> + * @notification: the notification type, must be S5P_FIMC_TX_END_NOTIFY
>> + * @arg: pointer to an u32 type integer that stores the frame payload
>> value
>> + *
>> + * Passes the sensor notification to the capture device
>> + */
>> +static void fimc_md_sensor_notify(struct v4l2_subdev *sd,
>> +                               unsigned int notification, void *arg)
>> +{
>> +       struct fimc_md *fmd;
>> +       struct exynos_pipeline *ep;
>> +       struct exynos5_pipeline0 *p;
>> +       unsigned long flags;
>> +
>> +       if (sd == NULL)
>> +               return;
>> +
>> +       ep = media_pipe_to_exynos_pipeline(sd->entity.pipe);
>> +       p = (struct exynos5_pipeline0 *)ep->priv;
>> +
>> +       spin_lock_irqsave(&fmd->slock, flags);
>> +
>> +       if (p->sensor_notify)
>> +               p->sensor_notify(sd, notification, arg);
>> +
>> +       spin_unlock_irqrestore(&fmd->slock, flags);
>> +}
>> +
>> +static int fimc_md_probe(struct platform_device *pdev)
>> +{
>> +       struct device *dev =&pdev->dev;
>>
>> +       struct v4l2_device *v4l2_dev;
>> +       struct fimc_md *fmd;
>> +       int ret;
>> +
>> +       fmd = devm_kzalloc(dev, sizeof(*fmd), GFP_KERNEL);
>> +       if (!fmd)
>> +               return -ENOMEM;
>> +
>> +       spin_lock_init(&fmd->slock);
>> +       fmd->pdev = pdev;
>> +
>> +       strlcpy(fmd->media_dev.model, "SAMSUNG S5P FIMC",
>
>
> Please change this to "SAMSUNG EXYNOS5 IS" or something similar.
> This could confuse user space!
>
>> +               sizeof(fmd->media_dev.model));
>> +       fmd->media_dev.link_notify = fimc_md_link_notify;
>> +       fmd->media_dev.dev = dev;
>> +
>> +       v4l2_dev =&fmd->v4l2_dev;
>> +       v4l2_dev->mdev =&fmd->media_dev;
>>
>> +       v4l2_dev->notify = fimc_md_sensor_notify;
>> +       strlcpy(v4l2_dev->name, "s5p-fimc-md", sizeof(v4l2_dev->name));
>
>
> exynos-fimc-md ?
>
>> +
>> +       ret = v4l2_device_register(dev,&fmd->v4l2_dev);
>>
>> +       if (ret<  0) {
>> +               v4l2_err(v4l2_dev, "Failed to register v4l2_device: %d\n",
>> ret);
>> +
>> +               return ret;
>> +
>> +       }
>> +       ret = media_device_register(&fmd->media_dev);
>> +       if (ret<  0) {
>> +               v4l2_err(v4l2_dev, "Failed to register media dev: %d\n",
>> ret);
>> +               goto err_md;
>> +       }
>> +       ret = fimc_md_get_clocks(fmd);
>> +       if (ret)
>> +               goto err_clk;
>> +
>> +       fmd->user_subdev_api = (dev->of_node != NULL);
>> +       g_exynos_mdev = fmd;
>> +
>> +       /* Protect the media graph while we're registering entities */
>> +       mutex_lock(&fmd->media_dev.graph_mutex);
>> +
>> +       if (fmd->pdev->dev.of_node)
>> +               ret = fimc_md_register_of_platform_entities(fmd,
>> dev->of_node);
>> +       else
>> +               ret = bus_for_each_dev(&platform_bus_type, NULL, fmd,
>> +                                               fimc_md_pdev_match);
>> +
>> +       if (ret)
>> +               goto err_unlock;
>> +
>> +       if (dev->platform_data || dev->of_node) {
>
>
> platform_data will never be used, and dev->of_node would be always not NULL.
>
>
>> +               ret = fimc_md_register_sensor_entities(fmd);
>> +               if (ret)
>> +                       goto err_unlock;
>> +       }
>> +
>> +       ret = fimc_md_create_links(fmd);
>> +       if (ret)
>> +               goto err_unlock;
>> +
>> +       ret = v4l2_device_register_subdev_nodes(&fmd->v4l2_dev);
>> +       if (ret)
>> +               goto err_unlock;
>> +
>> +       ret = device_create_file(&pdev->dev,&dev_attr_subdev_conf_mode);
>> +       if (ret)
>> +               goto err_unlock;
>> +
>> +       platform_set_drvdata(pdev, fmd);
>> +       mutex_unlock(&fmd->media_dev.graph_mutex);
>> +       return 0;
>> +
>> +err_unlock:
>> +       mutex_unlock(&fmd->media_dev.graph_mutex);
>> +err_clk:
>> +       media_device_unregister(&fmd->media_dev);
>> +       fimc_md_put_clocks(fmd);
>> +       fimc_md_unregister_entities(fmd);
>> +err_md:
>> +       v4l2_device_unregister(&fmd->v4l2_dev);
>> +       return ret;
>> +}
>> +
>
> ...
>
>> +static struct platform_device_id fimc_driver_ids[] __always_unused = {
>> +       { .name = "exynos-fimc-md" },
>> +       { },
>> +};
>> +MODULE_DEVICE_TABLE(platform, fimc_driver_ids);
>> +
>> +static const struct of_device_id fimc_md_of_match[] __initconst = {
>
>
> The attribute is wrong. I would remove that "__initconst".
>
>
>> +       { .compatible = "samsung,exynos5-fimc" },
>> +       { },
>> +};
>> +MODULE_DEVICE_TABLE(of, fimc_md_of_match);
>> +
>
>
> --
>
> Regards,
> Sylwester

Re: [RFC 04/12] s5p-csis: Adding Exynos5250 compatibility

[Sylwester Nawrocki]

Hi Shaik,

On 03/11/2013 07:58 AM, Shaik Ameer Basha wrote:
> Hi Sylwester,
> 
> On Mon, Mar 11, 2013 at 2:10 AM, Sylwester Nawrocki
> <sylvester.nawrocki@gmail.com> wrote:
>> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> Please don't leave the change log empty. I'll apply this patch.
>> I'm just wondering, if there aren't any further changes needed
>> to make the driver really working on exynos5250 ?
>>
> 
> There was nothing from driver side to change for making it work
> for Exynos5250. May be I need to update the S5P_INTMASK_EN_ALL
> to include all interrupts.

Yes, it might be a good idea squash that change into this patch,
i.e. the last patch form Arun's exynos5 fimc-is series. BTW, I'll
try to find a time to review the fimc-is patches this week.

Regards,
Sylwester

Re: [RFC 01/12] media: s5p-fimc: modify existing mdev to use common pipeline

[Sylwester Nawrocki]

Hi Shaik,

On 03/11/2013 07:41 AM, Shaik Ameer Basha wrote:
> Hi Sylwester,
>
> Thanks for the review.
> Actually I know this is the important patch in this series and I
> wanted us to have
> enough time to discuss on this patch. That's why I posted this patch
> series in hurry.

Sure, I'm glad it was posted early.

> I will remove this patch from the exynos5-mdev series and will send this as a
> separate patch from next time.

OK.

> please find my review comments inline..
>
>
> On Mon, Mar 11, 2013 at 3:30 AM, Sylwester Nawrocki
> <sylvester.nawrocki@gmail.com>  wrote:
>> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>>
>>> This patch modifies the current fimc_pipeline to exynos_pipeline,
>>
>> I think we could leave it as fimc_pipeline, exynos_pipeline seems
>> too generic to me.
>
> no issues, if we are going to strict to this common pipeline implementation
> definitely we can retain fimc_pipeline or we can use some other name which
> is not too generic.
...
>>>    static int fimc_capture_hw_init(struct fimc_dev *fimc)
>>>    {
>>>          struct fimc_ctx *ctx = fimc->vid_cap.ctx;
>>> -       struct fimc_pipeline *p =&fimc->pipeline;
>>> +       struct exynos_pipeline *p =&fimc->pipeline;
>>>
>>>          struct fimc_sensor_info *sensor;
>>>          unsigned long flags;
>>> +       struct v4l2_subdev *sd;
>>>          int ret = 0;
>>>
>>> -       if (p->subdevs[IDX_SENSOR] == NULL || ctx == NULL)
>>> +       sd = exynos_pipeline_get_subdev(fimc->pipeline_ops,
>>> +                                       get_subdev_sensor, p);
>>
>>
>> Hmm, it feels it is going wrong path this way. I would keep changes
>> to the s5p-fimc driver as small as possible. And the modules that
>> are shared across the exynos4 and exynos5 driver should use generic
>> media graph walking routines where possible.
>
> The only problem here is, the fimc_subdev_index enum is specific to
> fimc-mdevice.
> and why should we expose one particular media-device driver specific
> enums to other drivers.

OK, I missed the point this enum is in a public header that would be
reused by other media drivers.

> My Idea was to remove all media device specific structures, macros
> from fimc, fimc-lite, mipi-csis and fimc-is drivers.

Of course we cannot remove them all, as video node drivers need be
able to control the pipeline. And I'm not very enthusiastic about
creating more dependencies between exynos4 and exynos5 drivers.
But I guess a common video pipeline object definition cannot be
avoided as FIMC-LITE is reused by exynos4 and exynos5 subsystems.

>>> +       if (sd == NULL || ctx == NULL)
>>>                  return -ENXIO;
>>>          if (ctx->s_frame.fmt == NULL)
>>>                  return -EINVAL;
>>>
>>> -       sensor = v4l2_get_subdev_hostdata(p->subdevs[IDX_SENSOR]);
>>> +       sensor = v4l2_get_subdev_hostdata(sd);
>>>
>>>          spin_lock_irqsave(&fimc->slock, flags);
>>>          fimc_prepare_dma_offset(ctx,&ctx->d_frame);
>>
>> ...
>>>
>>> @@ -486,9 +491,12 @@ static struct vb2_ops fimc_capture_qops = {
>>>    int fimc_capture_ctrls_create(struct fimc_dev *fimc)
>>>    {
>>>          struct fimc_vid_cap *vid_cap =&fimc->vid_cap;
>>>
>>> -       struct v4l2_subdev *sensor = fimc->pipeline.subdevs[IDX_SENSOR];
>>> +       struct v4l2_subdev *sensor;
>>>          int ret;
>>>
>>> +       sensor = exynos_pipeline_get_subdev(fimc->pipeline_ops,
>>> +
>>> get_subdev_sensor,&fimc->pipeline);
>>>
>>> +
>>>          if (WARN_ON(vid_cap->ctx == NULL))
>>>                  return -ENXIO;
>>>          if (vid_cap->ctx->ctrls.ready)
>>> @@ -513,7 +521,7 @@ static int fimc_capture_open(struct file *file)
>>>
>>>          dbg("pid: %d, state: 0x%lx", task_pid_nr(current), fimc->state);
>>>
>>> -       fimc_md_graph_lock(fimc);
>>> +       exynos_pipeline_graph_lock(fimc->pipeline_ops,&fimc->pipeline);
>>
>>
>> Hmm, this look pretty scary to me. But I suspect this change is not
>> needed at all. The graph lock is _not_ the pipeline lock. It protects
>> all media entities registered to the media device, and links between
>> them. Not only entities linked into specific video processing pipeline
>> at a moment.
>
> Sorry, here the function name doesn't suits its implementation.
> Actually  exynos_pipeline_graph_lock() does what exactly
> fimc_md_graph_lock() does.
> I thought of having one common function across all the drivers to use
> graph lock/unlock functionality.

I'm not convinced we need something like this at the exynos drivers
level. We could add something to the v4l2 core, e.g. functions taking
struct media_entity as an argument. No need for adding extra levels
of indirection.

>>> -/* Called with the media graph mutex held */
>>> -static struct v4l2_subdev *__find_remote_sensor(struct media_entity *me)
>>> -{
>>> -       struct media_pad *pad =&me->pads[0];
>>>
>>> -       struct v4l2_subdev *sd;
>>> -
>>> -       while (pad->flags&   MEDIA_PAD_FL_SINK) {
>>>
>>> -               /* source pad */
>>> -               pad = media_entity_remote_source(pad);
>>> -               if (pad == NULL ||
>>> -                   media_entity_type(pad->entity) !=
>>> MEDIA_ENT_T_V4L2_SUBDEV)
>>> -                       break;
>>> -
>>> -               sd = media_entity_to_v4l2_subdev(pad->entity);
>>> -
>>> -               if (sd->grp_id == GRP_ID_FIMC_IS_SENSOR)
>>> -                       return sd;
>>> -               /* sink pad */
>>> -               pad =&sd->entity.pads[0];
>>>
>>> -       }
>>> -       return NULL;
>>> -}
>>
>>
>> What's wrong with this function ? Why doesn't it work for you ?
>> I think we should drop direct usage of fimc->pipeline->subdevs[IDX_SENSOR]
>> instead. The sensor group IDs should probably be moved to some common
>> header file.
>>
>
> In case of exynos4, fimc-lite only used with fimc-is. That means it only
> uses the FIMC IS SENSOR. but in case of exynos5, fimc-lite can use both

Not really, FIMC-LITE does work without FIMC-IS. I tested it with external
sensors with an embedded ISP. So it is pretty much same as FIMC in terms
of video capture from external sensors.

The function above is only used when FIMC-LITE is used as a subdev, and
the FIMC video node driver creates and owns the fimc_pipeline object.

Accessing the pipeline->subdevs[] array from within FIMC-LITE would have
been racy.

> GRP_ID_SENSOR and GRP_ID_FIMC_IS_SENSOR. Even this group ID's are
> specific to individual media device drivers.

That's true. When a subdev is registered to v4l2_device its driver usually
assigns grp_id as it sees fit.

> Here the idea is driver need not know about which sensor is connected to the
> pipeline, it can be IS sensor or normal sensor. It can use the common pipeline
> api's to get the sensor sub-dev.
>
> Or as you suggested, we can move all this GRP_ID's to some common location
> and change the checking in this function to
>          -  if (sd->grp_id == GRP_ID_FIMC_IS_SENSOR)
>          + if ((sd->grp_id == GRP_ID_FIMC_IS_SENSOR) || (sd->grp_id ==
> GRP_ID_SENSOR))
>
> My only idea was to reuse the fimc pipeline structure (struct
> fimc_pipeline) across
> different media device drivers. Having a hard-coded max array index
> which is specific to
> one media device driver will not help it to be reused by multiple guys.
>
> I may be missing some details here. Can you please suggest a better
> way to remove this dependency.

OK, let's try with the get_subdev pipeline operation for now.

>>> diff --git a/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>>> b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>>> index fc93fad..938cc56 100644
>>> --- a/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>>> +++ b/drivers/media/platform/s5p-fimc/fimc-mdevice.c
>>> @@ -36,6 +36,8 @@
>>>    #include "fimc-mdevice.h"
>>>    #include "mipi-csis.h"
>>>
>>> +static struct fimc_md *g_fimc_mdev;
>>
>>
>> Hm, this is pretty ugly. Can you explain why it is needed ?
>
> I was expecting this.
> Actually i want to associate this fimc_mdev with this exynos_pipeline.
> But as the pipeline init happens from the driver context (fimc,
> fimc-lite), I should
> have a way to access struct fimc_md in pipeline init function. That's
> why I stored a
> global reference of struct fimc_md here.
> I think, I need to find a better way to achieve same functionality if needed. :)

The struct fimc_pipeline was meant as a collection of subdevs only.
Why do you need a back reference to struct fimc_md ? To be able to
control one pipeline from within the other ?

Single entities should be aware of the whole pipeline and the media
device driver internals as little as possible.

...
>>> +       struct v4l2_subdev *(*get_subdev_sensor)(struct exynos_pipeline *p);
>>> +       struct v4l2_subdev *(*get_subdev_csis)(struct exynos_pipeline *p);
>>
>> No, we should instead use generic media graph walking routines in the
>> shared drivers (FIMC-LITE, MIPI-CSIS). FIMC driver could be making
>> certain assumptions about it's related media device driver. These
>> drivers are contained in a single module anyway.
>
> Cant we expose some macros for subdevs in a common header and implement a
> get_subdev function in common pipeline to achieve this.
>
> enum in /include/media/s5p_fimc.h file. Something like
>
> EXYNOS_SD_SENSOR,
> EXYNOS_SD_MIPI_CSIS,
> ...
> ... etc.
>
> pipeline_ops->get_subdev(pipeline, EXYNOS_SD_SENSOR);
>
> actually, the implementation of get_subdev uses the media device specific
> structures which stored the subdev pointers internally by using  generic media
> graph walking routines itself. so there should not be any issues with using the
> common pipeline api's also.

This option might cause less issues WRT locking. Let's try this and see how
it looks. I suppose we could use GRP_IDs as the subdev identifiers, after
adding EXYNOS_SD_* prefix or something similar.

> [Or]
>
> as you already suggested, we can have some common group id's exposed
> in a common header file, and use the generic media graph walking
> routines in each driver to achieve this functionality.

Regards,
Sylwester

Re: [RFC 11/12] media: m5mols: Adding dt support to m5mols driver

[Sylwester Nawrocki]

Hi Shaik,

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> This patch adds the dt support to m5mols driver.
>
> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> ---
>   drivers/media/i2c/m5mols/m5mols_core.c |   54 +++++++++++++++++++++++++++++++-
>   1 file changed, 53 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/media/i2c/m5mols/m5mols_core.c b/drivers/media/i2c/m5mols/m5mols_core.c
> index d4e7567..21c66ef 100644
> --- a/drivers/media/i2c/m5mols/m5mols_core.c
> +++ b/drivers/media/i2c/m5mols/m5mols_core.c
> @@ -19,6 +19,8 @@
>   #include<linux/interrupt.h>
>   #include<linux/delay.h>
>   #include<linux/gpio.h>
> +#include<linux/of_gpio.h>
> +#include<linux/pinctrl/consumer.h>

What would you need pinctrl for ? In most cases this driver just needs 
one GPIO
(sensor RESET), which is normally passed in gpios property.

>   #include<linux/regulator/consumer.h>
>   #include<linux/videodev2.h>
>   #include<linux/module.h>
> @@ -926,13 +928,38 @@ static irqreturn_t m5mols_irq_handler(int irq, void *data)
>   	return IRQ_HANDLED;
>   }
>
> +static const struct of_device_id m5mols_match[];
> +
>   static int m5mols_probe(struct i2c_client *client,
>   			const struct i2c_device_id *id)
>   {
> -	const struct m5mols_platform_data *pdata = client->dev.platform_data;
> +	struct m5mols_platform_data *pdata;
>   	struct m5mols_info *info;
> +	const struct of_device_id *of_id;
>   	struct v4l2_subdev *sd;
>   	int ret;
> +	struct pinctrl *pctrl;
> +	int eint_gpio = 0;
> +
> +	if (client->dev.of_node) {
> +		of_id = of_match_node(m5mols_match, client->dev.of_node);
> +		if (of_id)
> +			pdata = (struct m5mols_platform_data *)of_id->data;
> +		client->dev.platform_data = pdata;

Oh, no. Probably best thing to do would be to get rid of struct
m5mols_platform_data pointer from struct m5mols_info and just add gpio_reset
field there. That's what we have currently in the driver's platform data
structure.

struct m5mols_platform_data {
	int gpio_reset;
	u8 reset_polarity;
	int (*set_power)(struct device *dev, int on);
};

gpio_reset and reset_polarity are already handled in the driver, and for
this we just need a single entry in 'gpios' property.

set_power callback can't be supported. Luckily there seems to be no board
that needs it any more. So we just drop it. One solution for more complex
power sequence could be the Runtime Interpreted Power Sequences. Once it
is available we might be able to describe what was previously in a board
file in set_power callback in the device tree.

> +	} else {
> +		pdata = client->dev.platform_data;
> +	}
> +
> +	if (!pdata)
> +		return -EINVAL;
> +
> +	pctrl = devm_pinctrl_get_select_default(&client->dev);

Two issues here:

1. m5mols DT node doesn't include pinctrl property, does it ?
2. default pinctrl state is now being handled in the driver core.

Hence this pinctrl set up could well be removed.

> +	if (client->dev.of_node) {
> +		eint_gpio = of_get_named_gpio(client->dev.of_node, "gpios", 0);
> +		client->irq = gpio_to_irq(eint_gpio);
> +		pdata->gpio_reset = of_get_named_gpio(client->dev.of_node,
> +								"gpios", 1);

Err, now when pinctrl and generic GPIO DT bindings are supported on Exynos5
this should not be needed at all. request_irq() should work when you add
relevant properties in this device DT node. Please see exynos4210-trats.dts,
mms114-touchscreen node for an example.

	mms114-touchscreen@48 {
		...	
		interrupt-parent = <&gpx0>;
		interrupts = <4 2>;
		...
	};

You specify GPIO bank in the 'interrupt-parent' property, and the GPIO
index within the bank in first cell of 'interrupts' property. The second
cell are interrupt flags as defined in /Documentation/devicetree/bindings/
interrupt-controller/interrupts.txt. I'm not sure how this interacts with
the interrupt flags passed to request_irq ATM.

It's the pinctrl driver's task to configure the GPIO pinmux into EINT
function, when the above properties are present in device DT node.

> +	}
>
>   	if (pdata == NULL) {
>   		dev_err(&client->dev, "No platform data\n");
> @@ -1040,9 +1067,34 @@ static const struct i2c_device_id m5mols_id[] = {
>   };
>   MODULE_DEVICE_TABLE(i2c, m5mols_id);
>
> +static int m5mols_set_power(struct device *dev, int on)
> +{
> +	struct m5mols_platform_data *pdata =
> +			(struct m5mols_platform_data *)dev->platform_data;
> +	gpio_set_value(pdata->gpio_reset, !on);
> +	gpio_set_value(pdata->gpio_reset, !!on);
> +	return 0;
> +}
> +
> +static struct m5mols_platform_data m5mols_drvdata = {
> +	.gpio_reset	= 0,
> +	.reset_polarity	= 0,
> +	.set_power	= m5mols_set_power,
> +};

Huh, you've got extra bonus points for creativity! :)

Do you have fixed voltage regulators, permanently turned on on your board ?
gpio_reset is already being configured in m5mols_sensor_power() function.

Does so short pulse on RESET line change anything in your case ? If needed
we could modify m5mols_sensor_power() function to properly cover more cases.
Or does it also work when you remove the above m5mols_set_power callback ?

> +static const struct of_device_id m5mols_match[] = {
> +	{
> +		.compatible = "fujitsu,m-5mols",
> +		.data =&m5mols_drvdata,
> +	},
> +	{},
> +};
> +MODULE_DEVICE_TABLE(of, m5mols_match);

We already have a patch adding DT support to this driver. I'll probably
send it out for 3.10 next week. An important issue your patch doesn't
address is that in most H/W configurations camera host device provides
master clock to the sensor. It is the case for all our boards. So we
need to ensure that the clocks is enabled for the sensor when it tries
to access hardware. This essentially boils down to performing I2C
communication with the sensor device.

We currently worked around this by moving initial sensor detection to
v4l2 subdev .registered callback. But the preferred approach is to expose
clocks in the host driver so the sensor sub-device drivers can get them
and control as they seem fit. This also allows more fine grained power
management in the sensor driver. That said the asynchronous probing and
clock handling API is still not settled in the media subsystem and I think
it's worth to push the above mentioned patch upstream, even though it's
just an interim solution.

That doesn't mean we don't need the DT binding documentation for this
device though. :-)


Regards,
Sylwester

Re: [RFC 12/12] ARM: dts: Add camera node to exynos5250-smdk5250.dts

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> Signed-off-by: Shaik Ameer Basha<shaik.ameer-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> ---
>   arch/arm/boot/dts/exynos5250-smdk5250.dts |   43 ++++++++++++++++++++++++++++-
>   1 file changed, 42 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/boot/dts/exynos5250-smdk5250.dts b/arch/arm/boot/dts/exynos5250-smdk5250.dts
> index 4b10744..7fbc236 100644
> --- a/arch/arm/boot/dts/exynos5250-smdk5250.dts
> +++ b/arch/arm/boot/dts/exynos5250-smdk5250.dts
> @@ -85,9 +85,26 @@
>   	};
>
>   	i2c@12CA0000 {
> -		status = "disabled";
> +		samsung,i2c-sda-delay =<100>;
> +		samsung,i2c-max-bus-freq =<100000>;
>   		pinctrl-0 =<&i2c4_bus>;
>   		pinctrl-names = "default";
> +
> +		m5mols@1f {
> +			compatible = "fujitsu,m-5mols";
> +			reg =<0x1F>;
> +			gpios =<&gpx3 3 0xf>,<&gpx1 2 1>;
> +			clock-frequency =<24000000>;
> +			pinctrl-0 =<&cam_port_a_clk_active>;
> +			pinctrl-names = "default";

Ah, so it's that way... I don't think that is correct. What you're doing
here is assigning an SoC clock output pin pinctrl node to pinctrl property
of an image sensor device that is external to an SoC. Why don't you put
this pinctrl properties in the common "camera" node ?

> +
> +			port {
> +				m5mols_ep: endpoint {
> +					remote-endpoint =<&csis0_ep>;
> +				};
> +			};
> +
> +		};
>   	};
>
>   	i2c@12CB0000 {
> @@ -214,4 +231,28 @@
>   		samsung,mfc-r =<0x43000000 0x800000>;
>   		samsung,mfc-l =<0x51000000 0x800000>;
>   	};
> +
> +	camera {
> +		compatible = "samsung,exynos5-fimc", "simple-bus";

Shouldn't it be "samsung,exynos5-is" (Imaging Subsystem), or something
more relevant to exynos5 ? Or what would be reasons to use "fimc" for
exynos5 ?

Re: [RFC 12/12] ARM: dts: Add camera node to exynos5250-smdk5250.dts

[Sylwester Nawrocki]

On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
> ---
>   arch/arm/boot/dts/exynos5250-smdk5250.dts |   43 ++++++++++++++++++++++++++++-
>   1 file changed, 42 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm/boot/dts/exynos5250-smdk5250.dts b/arch/arm/boot/dts/exynos5250-smdk5250.dts
> index 4b10744..7fbc236 100644
> --- a/arch/arm/boot/dts/exynos5250-smdk5250.dts
> +++ b/arch/arm/boot/dts/exynos5250-smdk5250.dts
> @@ -85,9 +85,26 @@
>   	};
>
>   	i2c@12CA0000 {
> -		status = "disabled";
> +		samsung,i2c-sda-delay =<100>;
> +		samsung,i2c-max-bus-freq =<100000>;
>   		pinctrl-0 =<&i2c4_bus>;
>   		pinctrl-names = "default";
> +
> +		m5mols@1f {
> +			compatible = "fujitsu,m-5mols";
> +			reg =<0x1F>;
> +			gpios =<&gpx3 3 0xf>,<&gpx1 2 1>;
> +			clock-frequency =<24000000>;
> +			pinctrl-0 =<&cam_port_a_clk_active>;
> +			pinctrl-names = "default";

Ah, so it's that way... I don't think that is correct. What you're doing
here is assigning an SoC clock output pin pinctrl node to pinctrl property
of an image sensor device that is external to an SoC. Why don't you put
this pinctrl properties in the common "camera" node ?

> +
> +			port {
> +				m5mols_ep: endpoint {
> +					remote-endpoint =<&csis0_ep>;
> +				};
> +			};
> +
> +		};
>   	};
>
>   	i2c@12CB0000 {
> @@ -214,4 +231,28 @@
>   		samsung,mfc-r =<0x43000000 0x800000>;
>   		samsung,mfc-l =<0x51000000 0x800000>;
>   	};
> +
> +	camera {
> +		compatible = "samsung,exynos5-fimc", "simple-bus";

Shouldn't it be "samsung,exynos5-is" (Imaging Subsystem), or something
more relevant to exynos5 ? Or what would be reasons to use "fimc" for
exynos5 ?

Re: [RFC 11/12] media: m5mols: Adding dt support to m5mols driver

[Shaik Ameer Basha]

Hi Sylwester,

Thanks for the review.
Actually I agree with all of your review comments.

This was just a temporary test patch, used to test exynos5-mdev.
I thought for some one to test exynos5-mdev series patches, i need to
provide one working m5mols dt driver.

Good to hear you already have one patch for this driver.
If you can able to post the dt patch for this driver, I will use that.

Regards,
Shaik Ameer Basha


On Sat, Mar 23, 2013 at 5:26 PM, Sylwester Nawrocki
<sylvester.nawrocki@gmail.com> wrote:
> Hi Shaik,
>
>
> On 03/06/2013 12:53 PM, Shaik Ameer Basha wrote:
>>
>> This patch adds the dt support to m5mols driver.
>>
>> Signed-off-by: Shaik Ameer Basha<shaik.ameer@samsung.com>
>> ---
>>   drivers/media/i2c/m5mols/m5mols_core.c |   54
>> +++++++++++++++++++++++++++++++-
>>   1 file changed, 53 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/media/i2c/m5mols/m5mols_core.c
>> b/drivers/media/i2c/m5mols/m5mols_core.c
>> index d4e7567..21c66ef 100644
>> --- a/drivers/media/i2c/m5mols/m5mols_core.c
>> +++ b/drivers/media/i2c/m5mols/m5mols_core.c
>> @@ -19,6 +19,8 @@
>>   #include<linux/interrupt.h>
>>   #include<linux/delay.h>
>>   #include<linux/gpio.h>
>> +#include<linux/of_gpio.h>
>> +#include<linux/pinctrl/consumer.h>
>
>
> What would you need pinctrl for ? In most cases this driver just needs one
> GPIO
> (sensor RESET), which is normally passed in gpios property.
>
>
>>   #include<linux/regulator/consumer.h>
>>   #include<linux/videodev2.h>
>>   #include<linux/module.h>
>> @@ -926,13 +928,38 @@ static irqreturn_t m5mols_irq_handler(int irq, void
>> *data)
>>         return IRQ_HANDLED;
>>   }
>>
>> +static const struct of_device_id m5mols_match[];
>> +
>>   static int m5mols_probe(struct i2c_client *client,
>>                         const struct i2c_device_id *id)
>>   {
>> -       const struct m5mols_platform_data *pdata =
>> client->dev.platform_data;
>> +       struct m5mols_platform_data *pdata;
>>         struct m5mols_info *info;
>> +       const struct of_device_id *of_id;
>>         struct v4l2_subdev *sd;
>>         int ret;
>> +       struct pinctrl *pctrl;
>> +       int eint_gpio = 0;
>> +
>> +       if (client->dev.of_node) {
>> +               of_id = of_match_node(m5mols_match, client->dev.of_node);
>> +               if (of_id)
>> +                       pdata = (struct m5mols_platform_data
>> *)of_id->data;
>> +               client->dev.platform_data = pdata;
>
>
> Oh, no. Probably best thing to do would be to get rid of struct
> m5mols_platform_data pointer from struct m5mols_info and just add gpio_reset
> field there. That's what we have currently in the driver's platform data
> structure.
>
> struct m5mols_platform_data {
>         int gpio_reset;
>         u8 reset_polarity;
>         int (*set_power)(struct device *dev, int on);
> };
>
> gpio_reset and reset_polarity are already handled in the driver, and for
> this we just need a single entry in 'gpios' property.
>
> set_power callback can't be supported. Luckily there seems to be no board
> that needs it any more. So we just drop it. One solution for more complex
> power sequence could be the Runtime Interpreted Power Sequences. Once it
> is available we might be able to describe what was previously in a board
> file in set_power callback in the device tree.
>
>
>> +       } else {
>> +               pdata = client->dev.platform_data;
>> +       }
>> +
>> +       if (!pdata)
>> +               return -EINVAL;
>> +
>> +       pctrl = devm_pinctrl_get_select_default(&client->dev);
>
>
> Two issues here:
>
> 1. m5mols DT node doesn't include pinctrl property, does it ?
> 2. default pinctrl state is now being handled in the driver core.
>
> Hence this pinctrl set up could well be removed.
>
>
>> +       if (client->dev.of_node) {
>> +               eint_gpio = of_get_named_gpio(client->dev.of_node,
>> "gpios", 0);
>> +               client->irq = gpio_to_irq(eint_gpio);
>> +               pdata->gpio_reset = of_get_named_gpio(client->dev.of_node,
>> +                                                               "gpios",
>> 1);
>
>
> Err, now when pinctrl and generic GPIO DT bindings are supported on Exynos5
> this should not be needed at all. request_irq() should work when you add
> relevant properties in this device DT node. Please see exynos4210-trats.dts,
> mms114-touchscreen node for an example.
>
>         mms114-touchscreen@48 {
>                 ...
>                 interrupt-parent = <&gpx0>;
>                 interrupts = <4 2>;
>                 ...
>         };
>
> You specify GPIO bank in the 'interrupt-parent' property, and the GPIO
> index within the bank in first cell of 'interrupts' property. The second
> cell are interrupt flags as defined in /Documentation/devicetree/bindings/
> interrupt-controller/interrupts.txt. I'm not sure how this interacts with
> the interrupt flags passed to request_irq ATM.
>
> It's the pinctrl driver's task to configure the GPIO pinmux into EINT
> function, when the above properties are present in device DT node.
>
>
>> +       }
>>
>>         if (pdata == NULL) {
>>                 dev_err(&client->dev, "No platform data\n");
>> @@ -1040,9 +1067,34 @@ static const struct i2c_device_id m5mols_id[] = {
>>   };
>>   MODULE_DEVICE_TABLE(i2c, m5mols_id);
>>
>> +static int m5mols_set_power(struct device *dev, int on)
>> +{
>> +       struct m5mols_platform_data *pdata =
>> +                       (struct m5mols_platform_data *)dev->platform_data;
>> +       gpio_set_value(pdata->gpio_reset, !on);
>> +       gpio_set_value(pdata->gpio_reset, !!on);
>> +       return 0;
>> +}
>> +
>> +static struct m5mols_platform_data m5mols_drvdata = {
>> +       .gpio_reset     = 0,
>> +       .reset_polarity = 0,
>> +       .set_power      = m5mols_set_power,
>> +};
>
>
> Huh, you've got extra bonus points for creativity! :)
>
> Do you have fixed voltage regulators, permanently turned on on your board ?
> gpio_reset is already being configured in m5mols_sensor_power() function.
>
> Does so short pulse on RESET line change anything in your case ? If needed
> we could modify m5mols_sensor_power() function to properly cover more cases.
> Or does it also work when you remove the above m5mols_set_power callback ?
>
>
>> +static const struct of_device_id m5mols_match[] = {
>> +       {
>> +               .compatible = "fujitsu,m-5mols",
>> +               .data =&m5mols_drvdata,
>> +       },
>> +       {},
>> +};
>> +MODULE_DEVICE_TABLE(of, m5mols_match);
>
>
> We already have a patch adding DT support to this driver. I'll probably
> send it out for 3.10 next week. An important issue your patch doesn't
> address is that in most H/W configurations camera host device provides
> master clock to the sensor. It is the case for all our boards. So we
> need to ensure that the clocks is enabled for the sensor when it tries
> to access hardware. This essentially boils down to performing I2C
> communication with the sensor device.
>
> We currently worked around this by moving initial sensor detection to
> v4l2 subdev .registered callback. But the preferred approach is to expose
> clocks in the host driver so the sensor sub-device drivers can get them
> and control as they seem fit. This also allows more fine grained power
> management in the sensor driver. That said the asynchronous probing and
> clock handling API is still not settled in the media subsystem and I think
> it's worth to push the above mentioned patch upstream, even though it's
> just an interim solution.
>
> That doesn't mean we don't need the DT binding documentation for this
> device though. :-)
>
>
> Regards,
> Sylwester