[PATCH 0/9] Media Controller for soc-camera

[PATCH 0/9] Media Controller for soc-camera

[Guennadi Liakhovetski]

This is the first attempt at extending soc-camera with Media Controller / 
pad-level APIs. Yes, I know, that Laurent wasn't quite happy with "V4L: 
add convenience macros to the subdevice / Media Controller API," maybe 
we'll remove it eventually, but so far my patches use it, so, I kept it 
for now. The general idea has been described in

http://article.gmane.org/gmane.linux.drivers.video-input-infrastructure/38083

In short: soc-camera implements a media controller device and two entities 
per camera host (bridge) instance, linked statically to each other and to 
the client. The host driver gets a chance to implement "local" only 
configuration, as opposed to the standard soc-camera way of propagating the 
configuration up the pipeline to the client (sensor / decoder) driver. An 
example implementation is provided for sh_mobile_ceu_camera and two sensor 
drivers. The whole machinery gets activated if the soc-camera core finds a 
client driver, that implements pad operations. In that case both the 
"standard" (V4L2) and the "new" (MC) ways of addressing the driver become 
available. I.e., it is possible to run both standard V4L2 applications and 
MC-aware ones.

Of course, applies on top of

git://linuxtv.org/gliakhovetski/v4l-dvb.git for-3.2

Deepthy: this is what I told you about in

http://article.gmane.org/gmane.linux.ports.arm.omap/64847

it just took me a bit longer, than I thought.

Guennadi Liakhovetski (9):
  V4L: soc-camera: add a function to lookup xlate by mediabus code
  sh_mobile_ceu_camera: simplify scaling and cropping algorithms
  V4L: soc-camera: remove redundant parameter from the .set_bus_param()
    method
  V4L: add convenience macros to the subdevice / Media Controller API
  V4L: soc-camera: move bus parameter configuration to
    .vidioc_streamon()
  V4L: soc-camera: prepare hooks for Media Controller wrapper
  V4L: soc-camera: add a Media Controller wrapper
  V4L: mt9t112: add pad level operations
  V4L: imx074: add pad level operations

 drivers/media/video/Makefile               |    6 +-
 drivers/media/video/atmel-isi.c            |   12 +-
 drivers/media/video/imx074.c               |   85 +++-
 drivers/media/video/mt9t112.c              |   97 +++-
 drivers/media/video/mx1_camera.c           |   12 +-
 drivers/media/video/mx2_camera.c           |   13 +-
 drivers/media/video/mx3_camera.c           |   13 +-
 drivers/media/video/omap1_camera.c         |   16 +-
 drivers/media/video/pxa_camera.c           |   13 +-
 drivers/media/video/sh_mobile_ceu_camera.c |  904 +++++++++++-----------------
 drivers/media/video/soc_camera.c           |  157 ++++-
 drivers/media/video/soc_entity.c           |  284 +++++++++
 drivers/media/video/soc_mediabus.c         |   16 -
 include/media/soc_camera.h                 |   34 +-
 include/media/soc_entity.h                 |   31 +
 include/media/v4l2-subdev.h                |   11 +
 16 files changed, 1064 insertions(+), 640 deletions(-)
 create mode 100644 drivers/media/video/soc_entity.c
 create mode 100644 include/media/soc_entity.h

-- 
1.7.2.5

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

[PATCH 1/9] V4L: soc-camera: add a function to lookup xlate by mediabus code

[Guennadi Liakhovetski]

In addition to a helper function, performing a format translation table
lookup by a fourcc value, a similar function is now needed to lookup
translation table entries by mediabus codes.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/soc_camera.c |   25 ++++++++++++++++++++++++-
 include/media/soc_camera.h       |    6 +++++-
 2 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/drivers/media/video/soc_camera.c b/drivers/media/video/soc_camera.c
index b72580c..ba409ac 100644
--- a/drivers/media/video/soc_camera.c
+++ b/drivers/media/video/soc_camera.c
@@ -112,7 +112,7 @@ static int soc_camera_power_off(struct soc_camera_device *icd,
 }
 
 const struct soc_camera_format_xlate *soc_camera_xlate_by_fourcc(
-	struct soc_camera_device *icd, unsigned int fourcc)
+	const struct soc_camera_device *icd, u32 fourcc)
 {
 	unsigned int i;
 
@@ -123,6 +123,29 @@ const struct soc_camera_format_xlate *soc_camera_xlate_by_fourcc(
 }
 EXPORT_SYMBOL(soc_camera_xlate_by_fourcc);
 
+/*
+ * Warning: the mediabus code -> fourcc mapping is not unique, this is why we
+ * need a hint of a preferred fourcc value. Use 0 if unknown.
+ */
+const struct soc_camera_format_xlate *soc_camera_xlate_by_mcode(
+	const struct soc_camera_device *icd, enum v4l2_mbus_pixelcode code,
+	u32 fourcc)
+{
+	unsigned int i;
+	struct soc_camera_format_xlate *xlate = NULL;
+
+	for (i = 0; i < icd->num_user_formats; i++)
+		if (icd->user_formats[i].code == code) {
+			if (icd->user_formats[i].host_fmt->fourcc == fourcc)
+				return icd->user_formats + i;
+			/* Prefer the first one */
+			if (!xlate)
+				xlate = icd->user_formats + i;
+		}
+	return xlate;
+}
+EXPORT_SYMBOL(soc_camera_xlate_by_mcode);
+
 /**
  * soc_camera_apply_board_flags() - apply platform SOCAM_SENSOR_INVERT_* flags
  * @icl:	camera platform parameters
diff --git a/include/media/soc_camera.h b/include/media/soc_camera.h
index b1377b9..22e4bee 100644
--- a/include/media/soc_camera.h
+++ b/include/media/soc_camera.h
@@ -174,7 +174,11 @@ int soc_camera_host_register(struct soc_camera_host *ici);
 void soc_camera_host_unregister(struct soc_camera_host *ici);
 
 const struct soc_camera_format_xlate *soc_camera_xlate_by_fourcc(
-	struct soc_camera_device *icd, unsigned int fourcc);
+	const struct soc_camera_device *icd, u32 fourcc);
+
+const struct soc_camera_format_xlate *soc_camera_xlate_by_mcode(
+	const struct soc_camera_device *icd, enum v4l2_mbus_pixelcode code,
+	u32 fourcc);
 
 /**
  * struct soc_camera_format_xlate - match between host and sensor formats
-- 
1.7.2.5

[PATCH 2/9] sh_mobile_ceu_camera: simplify scaling and cropping algorithms

[Guennadi Liakhovetski]

With the pad-level API scaling and cropping will be configured on each
entity separately. To prepare for the conversion remove all attempts
to optimise scaling and cropping on the host and clients, as has
previously been done by the sh_mobile_ceu_camera driver.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/sh_mobile_ceu_camera.c |  881 +++++++++++-----------------
 1 files changed, 355 insertions(+), 526 deletions(-)

diff --git a/drivers/media/video/sh_mobile_ceu_camera.c b/drivers/media/video/sh_mobile_ceu_camera.c
index 955947a..33ffc35 100644
--- a/drivers/media/video/sh_mobile_ceu_camera.c
+++ b/drivers/media/video/sh_mobile_ceu_camera.c
@@ -112,7 +112,6 @@ struct sh_mobile_ceu_dev {
 
 	u32 cflcr;
 
-	enum v4l2_field field;
 	int sequence;
 
 	unsigned int image_mode:1;
@@ -121,20 +120,16 @@ struct sh_mobile_ceu_dev {
 };
 
 struct sh_mobile_ceu_cam {
-	/* CEU offsets within the camera output, before the CEU scaler */
-	unsigned int ceu_left;
-	unsigned int ceu_top;
+	/* Client cropping rectangle */
+	struct v4l2_rect rect;
 	/* Client output, as seen by the CEU */
 	unsigned int width;
 	unsigned int height;
 	/*
-	 * User window from S_CROP / G_CROP, produced by client cropping and
-	 * scaling, CEU scaling and CEU cropping, mapped back onto the client
-	 * input window
+	 * CEU offsets and sizes within the camera output, before the CEU
+	 * scaling filter
 	 */
-	struct v4l2_rect subrect;
-	/* Camera cropping rectangle */
-	struct v4l2_rect rect;
+	struct v4l2_rect ceu_rect;
 	const struct soc_mbus_pixelfmt *extra_fmt;
 	enum v4l2_mbus_pixelcode code;
 };
@@ -303,7 +298,7 @@ static int sh_mobile_ceu_capture(struct sh_mobile_ceu_dev *pcdev)
 	if (!pcdev->active)
 		return ret;
 
-	if (V4L2_FIELD_INTERLACED_BT == pcdev->field) {
+	if (V4L2_FIELD_INTERLACED_BT == icd->field) {
 		top1	= CDBYR;
 		top2	= CDBCR;
 		bottom1	= CDAYR;
@@ -329,7 +324,7 @@ static int sh_mobile_ceu_capture(struct sh_mobile_ceu_dev *pcdev)
 	}
 
 	ceu_write(pcdev, top1, phys_addr_top);
-	if (V4L2_FIELD_NONE != pcdev->field) {
+	if (V4L2_FIELD_NONE != icd->field) {
 		if (planar)
 			phys_addr_bottom = phys_addr_top + icd->user_width;
 		else
@@ -343,7 +338,7 @@ static int sh_mobile_ceu_capture(struct sh_mobile_ceu_dev *pcdev)
 		phys_addr_top += icd->user_width *
 			icd->user_height;
 		ceu_write(pcdev, top2, phys_addr_top);
-		if (V4L2_FIELD_NONE != pcdev->field) {
+		if (V4L2_FIELD_NONE != icd->field) {
 			phys_addr_bottom = phys_addr_top + icd->user_width;
 			ceu_write(pcdev, bottom2, phys_addr_bottom);
 		}
@@ -517,7 +512,7 @@ static irqreturn_t sh_mobile_ceu_irq(int irq, void *data)
 	ret = sh_mobile_ceu_capture(pcdev);
 	do_gettimeofday(&vb->v4l2_buf.timestamp);
 	if (!ret) {
-		vb->v4l2_buf.field = pcdev->field;
+		vb->v4l2_buf.field = pcdev->icd->field;
 		vb->v4l2_buf.sequence = pcdev->sequence++;
 	}
 	vb2_buffer_done(vb, ret < 0 ? VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
@@ -659,17 +654,18 @@ static void sh_mobile_ceu_set_rect(struct soc_camera_device *icd)
 	u32 camor;
 
 	dev_geo(icd->parent, "Crop %ux%u@%u:%u\n",
-		icd->user_width, icd->user_height, cam->ceu_left, cam->ceu_top);
+		icd->user_width, icd->user_height, cam->ceu_rect.left, cam->ceu_rect.top);
 
-	left_offset	= cam->ceu_left;
-	top_offset	= cam->ceu_top;
+	left_offset	= cam->ceu_rect.left;
+	top_offset	= cam->ceu_rect.top;
 
 	WARN_ON(icd->user_width & 3 || icd->user_height & 3);
 
 	width = icd->user_width;
 
+	in_width = min(2560U, cam->width);
+
 	if (pcdev->image_mode) {
-		in_width = cam->width;
 		if (!pcdev->is_16bit) {
 			in_width *= 2;
 			left_offset *= 2;
@@ -688,7 +684,7 @@ static void sh_mobile_ceu_set_rect(struct soc_camera_device *icd)
 			w_factor = 1;
 		}
 
-		in_width = cam->width * w_factor;
+		in_width *= w_factor;
 		left_offset *= w_factor;
 
 		if (bytes_per_line < 0)
@@ -698,8 +694,8 @@ static void sh_mobile_ceu_set_rect(struct soc_camera_device *icd)
 	}
 
 	height = icd->user_height;
-	in_height = cam->height;
-	if (V4L2_FIELD_NONE != pcdev->field) {
+	in_height = min(1920U, cam->height);
+	if (V4L2_FIELD_NONE != icd->field) {
 		height = (height / 2) & ~3;
 		in_height /= 2;
 		top_offset /= 2;
@@ -721,6 +717,10 @@ static void sh_mobile_ceu_set_rect(struct soc_camera_device *icd)
 		cdwdr_width);
 
 	ceu_write(pcdev, CAMOR, camor);
+	/*
+	 * We made sure offset + length doesn't exceed client output for both
+	 * width and height
+	 */
 	ceu_write(pcdev, CAPWR, (in_height << 16) | in_width);
 	/* CFSZR clipping is applied _after_ the scaling filter (CFLCR) */
 	ceu_write(pcdev, CFSZR, (height << 16) | width);
@@ -884,7 +884,7 @@ static int sh_mobile_ceu_set_bus_param(struct soc_camera_device *icd,
 
 	ceu_write(pcdev, CAPCR, 0x00300000);
 
-	switch (pcdev->field) {
+	switch (icd->field) {
 	case V4L2_FIELD_INTERLACED_TB:
 		value = 0x101;
 		break;
@@ -1107,11 +1107,13 @@ static int sh_mobile_ceu_get_formats(struct soc_camera_device *icd, unsigned int
 			return -ENOMEM;
 
 		/* We are called with current camera crop, initialise subrect with it */
-		cam->rect	= rect;
-		cam->subrect	= rect;
-
-		cam->width	= mf.width;
-		cam->height	= mf.height;
+		cam->rect		= rect;
+		cam->ceu_rect.width	= mf.width;
+		cam->ceu_rect.height	= mf.height;
+		cam->ceu_rect.top	= 0;
+		cam->ceu_rect.left	= 0;
+		cam->width		= mf.width;
+		cam->height		= mf.height;
 
 		icd->host_priv = cam;
 	} else {
@@ -1181,24 +1183,6 @@ static bool is_smaller(struct v4l2_rect *r1, struct v4l2_rect *r2)
 	return r1->width < r2->width || r1->height < r2->height;
 }
 
-/* Check if r1 fails to cover r2 */
-static bool is_inside(struct v4l2_rect *r1, struct v4l2_rect *r2)
-{
-	return r1->left > r2->left || r1->top > r2->top ||
-		r1->left + r1->width < r2->left + r2->width ||
-		r1->top + r1->height < r2->top + r2->height;
-}
-
-static unsigned int scale_down(unsigned int size, unsigned int scale)
-{
-	return (size * 4096 + scale / 2) / scale;
-}
-
-static unsigned int calc_generic_scale(unsigned int input, unsigned int output)
-{
-	return (input * 4096 + output / 2) / output;
-}
-
 /* Get and store current client crop */
 static int client_g_rect(struct v4l2_subdev *sd, struct v4l2_rect *rect)
 {
@@ -1224,280 +1208,59 @@ static int client_g_rect(struct v4l2_subdev *sd, struct v4l2_rect *rect)
 	return ret;
 }
 
-/* Client crop has changed, update our sub-rectangle to remain within the area */
-static void update_subrect(struct sh_mobile_ceu_cam *cam)
+static void ceu_full_input(struct sh_mobile_ceu_cam *cam,
+			   const struct v4l2_mbus_framefmt *mf)
 {
-	struct v4l2_rect *rect = &cam->rect, *subrect = &cam->subrect;
-
-	if (rect->width < subrect->width)
-		subrect->width = rect->width;
-
-	if (rect->height < subrect->height)
-		subrect->height = rect->height;
-
-	if (rect->left > subrect->left)
-		subrect->left = rect->left;
-	else if (rect->left + rect->width >
-		 subrect->left + subrect->width)
-		subrect->left = rect->left + rect->width -
-			subrect->width;
-
-	if (rect->top > subrect->top)
-		subrect->top = rect->top;
-	else if (rect->top + rect->height >
-		 subrect->top + subrect->height)
-		subrect->top = rect->top + rect->height -
-			subrect->height;
+	cam->ceu_rect.width	= mf->width;
+	cam->ceu_rect.height	= mf->height;
+	cam->width		= mf->width;
+	cam->height		= mf->height;
 }
 
-/*
- * The common for both scaling and cropping iterative approach is:
- * 1. try if the client can produce exactly what requested by the user
- * 2. if (1) failed, try to double the client image until we get one big enough
- * 3. if (2) failed, try to request the maximum image
- */
-static int client_s_crop(struct soc_camera_device *icd, struct v4l2_crop *crop,
-			 struct v4l2_crop *cam_crop)
+static void ceu_non_native_scale(struct sh_mobile_ceu_dev *pcdev,
+				 struct sh_mobile_ceu_cam *cam,
+				 const struct v4l2_mbus_framefmt *mf)
 {
-	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
-	struct v4l2_rect *rect = &crop->c, *cam_rect = &cam_crop->c;
-	struct device *dev = sd->v4l2_dev->dev;
-	struct sh_mobile_ceu_cam *cam = icd->host_priv;
-	struct v4l2_cropcap cap;
-	int ret;
-	unsigned int width, height;
-
-	v4l2_subdev_call(sd, video, s_crop, crop);
-	ret = client_g_rect(sd, cam_rect);
-	if (ret < 0)
-		return ret;
+	/* .top == .left == 0 already */
+	ceu_full_input(cam, mf);
 
-	/*
-	 * Now cam_crop contains the current camera input rectangle, and it must
-	 * be within camera cropcap bounds
-	 */
-	if (!memcmp(rect, cam_rect, sizeof(*rect))) {
-		/* Even if camera S_CROP failed, but camera rectangle matches */
-		dev_dbg(dev, "Camera S_CROP successful for %dx%d@%d:%d\n",
-			rect->width, rect->height, rect->left, rect->top);
-		cam->rect = *cam_rect;
-		return 0;
-	}
-
-	/* Try to fix cropping, that camera hasn't managed to set */
-	dev_geo(dev, "Fix camera S_CROP for %dx%d@%d:%d to %dx%d@%d:%d\n",
-		cam_rect->width, cam_rect->height,
-		cam_rect->left, cam_rect->top,
-		rect->width, rect->height, rect->left, rect->top);
-
-	/* We need sensor maximum rectangle */
-	ret = v4l2_subdev_call(sd, video, cropcap, &cap);
-	if (ret < 0)
-		return ret;
-
-	/* Put user requested rectangle within sensor bounds */
-	soc_camera_limit_side(&rect->left, &rect->width, cap.bounds.left, 2,
-			      cap.bounds.width);
-	soc_camera_limit_side(&rect->top, &rect->height, cap.bounds.top, 4,
-			      cap.bounds.height);
-
-	/*
-	 * Popular special case - some cameras can only handle fixed sizes like
-	 * QVGA, VGA,... Take care to avoid infinite loop.
-	 */
-	width = max(cam_rect->width, 2);
-	height = max(cam_rect->height, 2);
-
-	/*
-	 * Loop as long as sensor is not covering the requested rectangle and
-	 * is still within its bounds
-	 */
-	while (!ret && (is_smaller(cam_rect, rect) ||
-			is_inside(cam_rect, rect)) &&
-	       (cap.bounds.width > width || cap.bounds.height > height)) {
-
-		width *= 2;
-		height *= 2;
-
-		cam_rect->width = width;
-		cam_rect->height = height;
-
-		/*
-		 * We do not know what capabilities the camera has to set up
-		 * left and top borders. We could try to be smarter in iterating
-		 * them, e.g., if camera current left is to the right of the
-		 * target left, set it to the middle point between the current
-		 * left and minimum left. But that would add too much
-		 * complexity: we would have to iterate each border separately.
-		 * Instead we just drop to the left and top bounds.
-		 */
-		if (cam_rect->left > rect->left)
-			cam_rect->left = cap.bounds.left;
-
-		if (cam_rect->left + cam_rect->width < rect->left + rect->width)
-			cam_rect->width = rect->left + rect->width -
-				cam_rect->left;
-
-		if (cam_rect->top > rect->top)
-			cam_rect->top = cap.bounds.top;
-
-		if (cam_rect->top + cam_rect->height < rect->top + rect->height)
-			cam_rect->height = rect->top + rect->height -
-				cam_rect->top;
-
-		v4l2_subdev_call(sd, video, s_crop, cam_crop);
-		ret = client_g_rect(sd, cam_rect);
-		dev_geo(dev, "Camera S_CROP %d for %dx%d@%d:%d\n", ret,
-			cam_rect->width, cam_rect->height,
-			cam_rect->left, cam_rect->top);
-	}
-
-	/* S_CROP must not modify the rectangle */
-	if (is_smaller(cam_rect, rect) || is_inside(cam_rect, rect)) {
-		/*
-		 * The camera failed to configure a suitable cropping,
-		 * we cannot use the current rectangle, set to max
-		 */
-		*cam_rect = cap.bounds;
-		v4l2_subdev_call(sd, video, s_crop, cam_crop);
-		ret = client_g_rect(sd, cam_rect);
-		dev_geo(dev, "Camera S_CROP %d for max %dx%d@%d:%d\n", ret,
-			cam_rect->width, cam_rect->height,
-			cam_rect->left, cam_rect->top);
-	}
-
-	if (!ret) {
-		cam->rect = *cam_rect;
-		update_subrect(cam);
-	}
-
-	return ret;
+	/* Possibly we were in a native mode and scaling */
+	pcdev->cflcr = 0;
 }
 
-/* Iterative s_mbus_fmt, also updates cached client crop on success */
-static int client_s_fmt(struct soc_camera_device *icd,
-			struct v4l2_mbus_framefmt *mf, bool ceu_can_scale)
+static void ceu_native_scale(struct sh_mobile_ceu_dev *pcdev,
+			     const struct sh_mobile_ceu_cam *cam,
+			     struct v4l2_pix_format *pix)
 {
-	struct sh_mobile_ceu_cam *cam = icd->host_priv;
-	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
-	struct device *dev = icd->parent;
-	unsigned int width = mf->width, height = mf->height, tmp_w, tmp_h;
-	unsigned int max_width, max_height;
-	struct v4l2_cropcap cap;
-	bool ceu_1to1;
-	int ret;
-
-	ret = v4l2_device_call_until_err(sd->v4l2_dev, (long)icd, video,
-					 s_mbus_fmt, mf);
-	if (ret < 0)
-		return ret;
-
-	dev_geo(dev, "camera scaled to %ux%u\n", mf->width, mf->height);
-
-	if (width == mf->width && height == mf->height) {
-		/* Perfect! The client has done it all. */
-		ceu_1to1 = true;
-		goto update_cache;
-	}
-
-	ceu_1to1 = false;
-	if (!ceu_can_scale)
-		goto update_cache;
-
-	cap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-
-	ret = v4l2_subdev_call(sd, video, cropcap, &cap);
-	if (ret < 0)
-		return ret;
-
-	max_width = min(cap.bounds.width, 2560);
-	max_height = min(cap.bounds.height, 1920);
+	u16 scale_v, scale_h;
 
-	/* Camera set a format, but geometry is not precise, try to improve */
-	tmp_w = mf->width;
-	tmp_h = mf->height;
+	/* No upscaling */
+	if (pix->width > cam->ceu_rect.width)
+		pix->width = cam->ceu_rect.width;
+	if (pix->height > cam->ceu_rect.height)
+		pix->height = cam->ceu_rect.height;
 
-	/* width <= max_width && height <= max_height - guaranteed by try_fmt */
-	while ((width > tmp_w || height > tmp_h) &&
-	       tmp_w < max_width && tmp_h < max_height) {
-		tmp_w = min(2 * tmp_w, max_width);
-		tmp_h = min(2 * tmp_h, max_height);
-		mf->width = tmp_w;
-		mf->height = tmp_h;
-		ret = v4l2_device_call_until_err(sd->v4l2_dev, (long)icd, video,
-						 s_mbus_fmt, mf);
-		dev_geo(dev, "Camera scaled to %ux%u\n",
-			mf->width, mf->height);
-		if (ret < 0) {
-			/* This shouldn't happen */
-			dev_err(dev, "Client failed to set format: %d\n", ret);
-			return ret;
-		}
-	}
-
-update_cache:
-	/* Update cache */
-	ret = client_g_rect(sd, &cam->rect);
-	if (ret < 0)
-		return ret;
+	/* Scale width x height down to pix->{width x height} */
+	scale_h = calc_scale(cam->ceu_rect.width, &pix->width);
+	scale_v = calc_scale(cam->ceu_rect.height, &pix->height);
 
-	if (ceu_1to1)
-		cam->subrect = cam->rect;
-	else
-		update_subrect(cam);
+	dev_geo(pcdev->icd->parent, "CEU W: %u : %u -> %u, H: %u : %u -> %u\n",
+		cam->ceu_rect.width, scale_h, pix->width,
+		cam->ceu_rect.height, scale_v, pix->height);
 
-	return 0;
+	pcdev->cflcr = scale_h | (scale_v << 16);
 }
 
-/**
- * @width	- on output: user width, mapped back to input
- * @height	- on output: user height, mapped back to input
- * @mf		- in- / output camera output window
- */
-static int client_scale(struct soc_camera_device *icd,
-			struct v4l2_mbus_framefmt *mf,
-			unsigned int *width, unsigned int *height,
-			bool ceu_can_scale)
+static bool ceu_fmt_can_scale(__u32 fourcc)
 {
-	struct sh_mobile_ceu_cam *cam = icd->host_priv;
-	struct device *dev = icd->parent;
-	struct v4l2_mbus_framefmt mf_tmp = *mf;
-	unsigned int scale_h, scale_v;
-	int ret;
-
-	/*
-	 * 5. Apply iterative camera S_FMT for camera user window (also updates
-	 *    client crop cache and the imaginary sub-rectangle).
-	 */
-	ret = client_s_fmt(icd, &mf_tmp, ceu_can_scale);
-	if (ret < 0)
-		return ret;
-
-	dev_geo(dev, "5: camera scaled to %ux%u\n",
-		mf_tmp.width, mf_tmp.height);
-
-	/* 6. Retrieve camera output window (g_fmt) */
-
-	/* unneeded - it is already in "mf_tmp" */
-
-	/* 7. Calculate new client scales. */
-	scale_h = calc_generic_scale(cam->rect.width, mf_tmp.width);
-	scale_v = calc_generic_scale(cam->rect.height, mf_tmp.height);
-
-	mf->width	= mf_tmp.width;
-	mf->height	= mf_tmp.height;
-	mf->colorspace	= mf_tmp.colorspace;
-
-	/*
-	 * 8. Calculate new CEU crop - apply camera scales to previously
-	 *    updated "effective" crop.
-	 */
-	*width = scale_down(cam->subrect.width, scale_h);
-	*height = scale_down(cam->subrect.height, scale_v);
-
-	dev_geo(dev, "8: new client sub-window %ux%u\n", *width, *height);
-
-	return 0;
+	switch (fourcc) {
+	case V4L2_PIX_FMT_NV12:
+	case V4L2_PIX_FMT_NV21:
+	case V4L2_PIX_FMT_NV16:
+	case V4L2_PIX_FMT_NV61:
+		return true;
+	}
+	return false;
 }
 
 /*
@@ -1513,125 +1276,132 @@ static int sh_mobile_ceu_set_crop(struct soc_camera_device *icd,
 	struct device *dev = icd->parent;
 	struct soc_camera_host *ici = to_soc_camera_host(dev);
 	struct sh_mobile_ceu_dev *pcdev = ici->priv;
-	struct v4l2_crop cam_crop;
 	struct sh_mobile_ceu_cam *cam = icd->host_priv;
-	struct v4l2_rect *cam_rect = &cam_crop.c;
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
+	struct v4l2_cropcap cap;
 	struct v4l2_mbus_framefmt mf;
-	unsigned int scale_cam_h, scale_cam_v, scale_ceu_h, scale_ceu_v,
-		out_width, out_height;
-	int interm_width, interm_height;
-	u32 capsr, cflcr;
+	u32 capsr, cflcr = pcdev->cflcr;
+	bool image_mode = ceu_fmt_can_scale(icd->current_fmt->host_fmt->fourcc);
 	int ret;
 
 	dev_geo(dev, "S_CROP(%ux%u@%u:%u)\n", rect->width, rect->height,
 		rect->left, rect->top);
 
+	ret = v4l2_subdev_call(sd, video, cropcap, &cap);
+	if (ret < 0)
+		return ret;
+
+	/* Put user requested rectangle within sensor bounds */
+	soc_camera_limit_side(&rect->left, &rect->width, cap.bounds.left, 2,
+			      cap.bounds.width);
+	soc_camera_limit_side(&rect->top, &rect->height, cap.bounds.top, 4,
+			      cap.bounds.height);
+
 	/* During camera cropping its output window can change too, stop CEU */
 	capsr = capture_save_reset(pcdev);
 	dev_dbg(dev, "CAPSR 0x%x, CFLCR 0x%x\n", capsr, pcdev->cflcr);
 
-	/*
-	 * 1. - 2. Apply iterative camera S_CROP for new input window, read back
-	 * actual camera rectangle.
-	 */
-	ret = client_s_crop(icd, a, &cam_crop);
+	/* Very similar to S_FMT */
+	v4l2_subdev_call(sd, video, s_crop, a);
+	ret = client_g_rect(sd, &cam->rect);
 	if (ret < 0)
 		return ret;
 
-	dev_geo(dev, "1-2: camera cropped to %ux%u@%u:%u\n",
-		cam_rect->width, cam_rect->height,
-		cam_rect->left, cam_rect->top);
-
-	/* On success cam_crop contains current camera crop */
-
-	/* 3. Retrieve camera output window */
+	/* re-read client output */
 	ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
 	if (ret < 0)
 		return ret;
 
-	if (mf.width > 2560 || mf.height > 1920)
-		return -EINVAL;
-
-	/* 4. Calculate camera scales */
-	scale_cam_h	= calc_generic_scale(cam_rect->width, mf.width);
-	scale_cam_v	= calc_generic_scale(cam_rect->height, mf.height);
-
-	/* Calculate intermediate window */
-	interm_width	= scale_down(rect->width, scale_cam_h);
-	interm_height	= scale_down(rect->height, scale_cam_v);
-
-	if (interm_width < icd->user_width) {
-		u32 new_scale_h;
-
-		new_scale_h = calc_generic_scale(rect->width, icd->user_width);
-
-		mf.width = scale_down(cam_rect->width, new_scale_h);
-	}
-
-	if (interm_height < icd->user_height) {
-		u32 new_scale_v;
-
-		new_scale_v = calc_generic_scale(rect->height, icd->user_height);
-
-		mf.height = scale_down(cam_rect->height, new_scale_v);
-	}
+	if (!image_mode) {
+		ceu_full_input(cam, &mf);
+		icd->user_width = mf.width & ~3;
+		icd->user_height = mf.height & ~3;
+	} else {
+		struct v4l2_pix_format pix = {
+			.pixelformat	= icd->current_fmt->host_fmt->fourcc,
+			.colorspace	= icd->colorspace,
+			.field		= icd->field,
+			.width		= icd->user_width & ~3,
+			.height		= icd->user_height & ~3,
+		};
+		/*
+		 * We do not care about client output frame, only check the
+		 * cropping result. If the client crop rectangle is smaller,
+		 * than what user has requested, request client crop = max,
+		 * scale = 1:1 and use CEU cropping instead or additionally
+		 */
+		if (is_smaller(&cam->rect, rect)) {
+			/* Request maximum crop rectangle */
+			*rect = cap.bounds;
+			v4l2_subdev_call(sd, video, s_crop, a);
+			ret = client_g_rect(sd, &cam->rect);
+			if (ret < 0)
+				return ret;
 
-	if (interm_width < icd->user_width || interm_height < icd->user_height) {
-		ret = v4l2_device_call_until_err(sd->v4l2_dev, (int)icd, video,
-						 s_mbus_fmt, &mf);
-		if (ret < 0)
-			return ret;
+			/* re-read client output */
+			ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
+			if (ret < 0)
+				return ret;
 
-		dev_geo(dev, "New camera output %ux%u\n", mf.width, mf.height);
-		scale_cam_h	= calc_generic_scale(cam_rect->width, mf.width);
-		scale_cam_v	= calc_generic_scale(cam_rect->height, mf.height);
-		interm_width	= scale_down(rect->width, scale_cam_h);
-		interm_height	= scale_down(rect->height, scale_cam_v);
-	}
+			if (!is_smaller(&cam->rect, rect)) {
+				/* Client crop sufficient, configure scale = 1:1 */
+				mf.width = rect->width;
+				mf.height = rect->height;
+				ret = v4l2_device_call_until_err(sd->v4l2_dev, (int)icd,
+								 video, s_mbus_fmt, &mf);
+				if (ret < 0)
+					return ret;
+			}
+		}
 
-	/* Cache camera output window */
-	cam->width	= mf.width;
-	cam->height	= mf.height;
+		/*
+		 * Is the client window bigger than requested and
+		 * scale == 1:1? If yes, apply CEU cropping
+		 */
+		if (cam->rect.width > rect->width &&
+		    cam->rect.height > rect->height &&
+		    cam->rect.width == mf.width &&
+		    cam->rect.height == mf.height) {
+			/* client crop can be improved: client scale == 1:1 */
+			cam->ceu_rect.width = min_t(u32, mf.width, rect->width);
+			cam->ceu_rect.height = min_t(u32, mf.height, rect->height);
+
+			cam->ceu_rect.left = rect->left <= cam->rect.left ? 0 :
+				min_t(u32, rect->left - cam->rect.left,
+				      mf.width - cam->ceu_rect.width);
+
+			cam->ceu_rect.top = rect->top <= cam->rect.top ? 0 :
+				min_t(u32, rect->top - cam->rect.top,
+				      mf.height - cam->ceu_rect.height);
+
+			cam->width = mf.width;
+			cam->height = mf.height;
+		} else {
+			/*
+			 * Either client cropping is perfect, or we cannot
+			 * improve it anyway. Do not apply CEU cropping, scale
+			 * the complete client output
+			 */
+			cam->ceu_rect.left = 0;
+			cam->ceu_rect.top = 0;
+			ceu_full_input(cam, &mf);
+		}
 
-	if (pcdev->image_mode) {
-		out_width	= min(interm_width, icd->user_width);
-		out_height	= min(interm_height, icd->user_height);
-	} else {
-		out_width	= interm_width;
-		out_height	= interm_height;
+		ceu_native_scale(pcdev, cam, &pix);
+		icd->user_width = pix.width;
+		icd->user_height = pix.height;
 	}
 
-	/*
-	 * 5. Calculate CEU scales from camera scales from results of (5) and
-	 *    the user window
-	 */
-	scale_ceu_h	= calc_scale(interm_width, &out_width);
-	scale_ceu_v	= calc_scale(interm_height, &out_height);
-
-	dev_geo(dev, "5: CEU scales %u:%u\n", scale_ceu_h, scale_ceu_v);
-
 	/* Apply CEU scales. */
-	cflcr = scale_ceu_h | (scale_ceu_v << 16);
-	if (cflcr != pcdev->cflcr) {
-		pcdev->cflcr = cflcr;
-		ceu_write(pcdev, CFLCR, cflcr);
-	}
-
-	icd->user_width	 = out_width & ~3;
-	icd->user_height = out_height & ~3;
-	/* Offsets are applied at the CEU scaling filter input */
-	cam->ceu_left	 = scale_down(rect->left - cam_rect->left, scale_cam_h) & ~1;
-	cam->ceu_top	 = scale_down(rect->top - cam_rect->top, scale_cam_v) & ~1;
+	if (cflcr != pcdev->cflcr)
+		ceu_write(pcdev, CFLCR, pcdev->cflcr);
 
-	/* 6. Use CEU cropping to crop to the new window. */
+	/* Use CEU cropping to crop to the new window. */
 	sh_mobile_ceu_set_rect(icd);
 
-	cam->subrect = *rect;
-
-	dev_geo(dev, "6: CEU cropped to %ux%u@%u:%u\n",
-		icd->user_width, icd->user_height,
-		cam->ceu_left, cam->ceu_top);
+	dev_geo(dev, "cropped to %ux%u@%u:%u\n",
+		cam->ceu_rect.width, cam->ceu_rect.height,
+		cam->rect.left, cam->rect.top);
 
 	/* Restore capture. The CE bit can be cleared by the hardware */
 	if (pcdev->active)
@@ -1648,183 +1418,233 @@ static int sh_mobile_ceu_get_crop(struct soc_camera_device *icd,
 	struct sh_mobile_ceu_cam *cam = icd->host_priv;
 
 	a->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-	a->c = cam->subrect;
+	a->c = cam->ceu_rect;
 
 	return 0;
 }
 
-/*
- * Calculate real client output window by applying new scales to the current
- * client crop. New scales are calculated from the requested output format and
- * CEU crop, mapped backed onto the client input (subrect).
- */
-static void calculate_client_output(struct soc_camera_device *icd,
-		const struct v4l2_pix_format *pix, struct v4l2_mbus_framefmt *mf)
+static enum v4l2_field ceu_supported_field(enum v4l2_field *user_field)
 {
-	struct sh_mobile_ceu_cam *cam = icd->host_priv;
-	struct device *dev = icd->parent;
-	struct v4l2_rect *cam_subrect = &cam->subrect;
-	unsigned int scale_v, scale_h;
-
-	if (cam_subrect->width == cam->rect.width &&
-	    cam_subrect->height == cam->rect.height) {
-		/* No sub-cropping */
-		mf->width	= pix->width;
-		mf->height	= pix->height;
-		return;
+	switch (*user_field) {
+	default:
+		*user_field = V4L2_FIELD_NONE;
+		/* fall-through */
+	case V4L2_FIELD_INTERLACED_TB:
+	case V4L2_FIELD_INTERLACED_BT:
+	case V4L2_FIELD_NONE:
+		return *user_field;
+	case V4L2_FIELD_INTERLACED:
+		return V4L2_FIELD_INTERLACED_TB;
 	}
+}
 
-	/* 1.-2. Current camera scales and subwin - cached. */
-
-	dev_geo(dev, "2: subwin %ux%u@%u:%u\n",
-		cam_subrect->width, cam_subrect->height,
-		cam_subrect->left, cam_subrect->top);
+static int ceu_client_crop(struct soc_camera_device *icd,
+			   struct v4l2_mbus_framefmt *mf)
+{
+	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
+	struct sh_mobile_ceu_cam *cam = icd->host_priv;
+	/* We were sub-cropping with a native format. Reset. */
+	struct v4l2_crop cam_crop = {
+		.type	= V4L2_BUF_TYPE_VIDEO_CAPTURE,
+		.c	= {
+			.width	= cam->ceu_rect.width,
+			.height	= cam->ceu_rect.height,
+			.top	= cam->ceu_rect.top + cam->rect.top,
+			.left	= cam->ceu_rect.left + cam->rect.left,
+		},
+	};
+	int ret;
 
 	/*
-	 * 3. Calculate new combined scales from input sub-window to requested
-	 *    user window.
+	 * re-request cropping - maybe the client can do better with the
+	 * new pixel code
 	 */
+	v4l2_subdev_call(sd, video, s_crop, &cam_crop);
+	ret = client_g_rect(sd, &cam->rect);
+	if (ret < 0)
+		return ret;
 
-	/*
-	 * TODO: CEU cannot scale images larger than VGA to smaller than SubQCIF
-	 * (128x96) or larger than VGA
-	 */
-	scale_h = calc_generic_scale(cam_subrect->width, pix->width);
-	scale_v = calc_generic_scale(cam_subrect->height, pix->height);
+	/* re-read client output */
+	ret = v4l2_subdev_call(sd, video, g_mbus_fmt, mf);
+	if (ret < 0)
+		return ret;
 
-	dev_geo(dev, "3: scales %u:%u\n", scale_h, scale_v);
+	/* Switching from a native format, offsets can be != 0 */
+	cam->ceu_rect.left	= 0;
+	cam->ceu_rect.top	= 0;
 
-	/*
-	 * 4. Calculate desired client output window by applying combined scales
-	 *    to client (real) input window.
-	 */
-	mf->width	= scale_down(cam->rect.width, scale_h);
-	mf->height	= scale_down(cam->rect.height, scale_v);
+	return 0;
+}
+
+/*
+ * Verify, whether the client has scaled into a window, smaller than requested
+ * by the user, and if the requested user output is smaller, than client
+ * cropping rectangle: we do not want to ask the client to upscale more, that
+ * what the user has requested, and we've already tried that.
+ */
+static int ceu_crop_full_window(struct soc_camera_device *icd,
+				struct v4l2_pix_format *pix,
+				struct v4l2_mbus_framefmt *mf)
+{
+	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
+	struct sh_mobile_ceu_cam *cam = icd->host_priv;
+	unsigned int max_width, max_height;
+	int ret;
+
+	max_width = min(2560, cam->rect.width);
+	max_height = min(1920, cam->rect.height);
+
+	if ((mf->width < pix->width && mf->width < max_width) ||
+	    (mf->height < pix->height && mf->height < max_height)) {
+		enum v4l2_mbus_pixelcode code = mf->code;
+
+		mf->width = max_width;
+		mf->height = max_height;
+		ret = v4l2_device_call_until_err(sd->v4l2_dev, (long)icd,
+						 video, s_mbus_fmt, mf);
+		if (ret < 0)
+			return ret;
+		BUG_ON(mf->code != code);
+	}
+
+	/* We were not cropping, i.e., .top == .left == 0 */
+	ceu_full_input(cam, mf);
+
+	return 0;
 }
 
 /* Similar to set_crop multistage iterative algorithm */
 static int sh_mobile_ceu_set_fmt(struct soc_camera_device *icd,
 				 struct v4l2_format *f)
 {
+	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct device *dev = icd->parent;
 	struct soc_camera_host *ici = to_soc_camera_host(dev);
 	struct sh_mobile_ceu_dev *pcdev = ici->priv;
 	struct sh_mobile_ceu_cam *cam = icd->host_priv;
 	struct v4l2_pix_format *pix = &f->fmt.pix;
-	struct v4l2_mbus_framefmt mf;
-	__u32 pixfmt = pix->pixelformat;
+	struct v4l2_mbus_framefmt mf = {
+		.width		= pix->width,
+		.height		= pix->height,
+	};
 	const struct soc_camera_format_xlate *xlate;
-	/* Keep Compiler Happy */
-	unsigned int ceu_sub_width = 0, ceu_sub_height = 0;
-	u16 scale_v, scale_h;
-	int ret;
-	bool image_mode;
+	bool ceu_crop = cam->ceu_rect.left || cam->ceu_rect.top ||
+		cam->ceu_rect.width != cam->rect.width ||
+		cam->ceu_rect.height != cam->rect.height;
+	__u32 pixfmt = pix->pixelformat;
 	enum v4l2_field field;
+	bool image_mode;
+	int ret;
 
-	switch (pix->field) {
-	default:
-		pix->field = V4L2_FIELD_NONE;
-		/* fall-through */
-	case V4L2_FIELD_INTERLACED_TB:
-	case V4L2_FIELD_INTERLACED_BT:
-	case V4L2_FIELD_NONE:
-		field = pix->field;
-		break;
-	case V4L2_FIELD_INTERLACED:
-		field = V4L2_FIELD_INTERLACED_TB;
-		break;
-	}
+	/* .try_fmt() has been called, size valid */
 
 	xlate = soc_camera_xlate_by_fourcc(icd, pixfmt);
 	if (!xlate) {
-		dev_warn(dev, "Format %x not found\n", pixfmt);
-		return -EINVAL;
+		xlate			= icd->current_fmt;
+		dev_warn(dev, "Format %x not found, staying with current %x\n",
+			 pixfmt, xlate->host_fmt->fourcc);
+		pixfmt			= xlate->host_fmt->fourcc;
+		pix->pixelformat	= pixfmt;
+		pix->colorspace		= icd->colorspace;
+		pix->field		= icd->field;
 	}
 
-	/* 1.-4. Calculate desired client output geometry */
-	calculate_client_output(icd, pix, &mf);
-	mf.field	= pix->field;
-	mf.colorspace	= pix->colorspace;
-	mf.code		= xlate->code;
-
-	switch (pixfmt) {
-	case V4L2_PIX_FMT_NV12:
-	case V4L2_PIX_FMT_NV21:
-	case V4L2_PIX_FMT_NV16:
-	case V4L2_PIX_FMT_NV61:
-		image_mode = true;
-		break;
-	default:
-		image_mode = false;
-	}
-
-	dev_geo(dev, "S_FMT(pix=0x%x, fld 0x%x, code 0x%x, %ux%u)\n", pixfmt, mf.field, mf.code,
-		pix->width, pix->height);
-
-	dev_geo(dev, "4: request camera output %ux%u\n", mf.width, mf.height);
-
-	/* 5. - 9. */
-	ret = client_scale(icd, &mf, &ceu_sub_width, &ceu_sub_height,
-			   image_mode && V4L2_FIELD_NONE == field);
-
-	dev_geo(dev, "5-9: client scale return %d\n", ret);
-
-	/* Done with the camera. Now see if we can improve the result */
-
-	dev_geo(dev, "fmt %ux%u, requested %ux%u\n",
-		mf.width, mf.height, pix->width, pix->height);
-	if (ret < 0)
-		return ret;
+	image_mode = ceu_fmt_can_scale(pixfmt);
 
-	if (mf.code != xlate->code)
-		return -EINVAL;
+	field = ceu_supported_field(&pix->field);
 
-	/* 9. Prepare CEU crop */
-	cam->width = mf.width;
-	cam->height = mf.height;
+	mf.colorspace	= pix->colorspace;
+	mf.field	= pix->field;
+	mf.code		= xlate->code;
 
-	/* 10. Use CEU scaling to scale to the requested user window. */
+	dev_geo(icd->parent, "S_FMT(pix=0x%x, fld 0x%x, code 0x%x, %ux%u)\n",
+		pixfmt, mf.field, mf.code, pix->width, pix->height);
 
-	/* We cannot scale up */
-	if (pix->width > ceu_sub_width)
-		ceu_sub_width = pix->width;
+	/*
+	 * We do not call client scaling if:
+	 * (1) pixelformat is the same; and
+	 * (2) it is a natively supported format; and
+	 * (3) CEU performs subcropping, i.e., client scales = 1:1
+	 */
+	if (pixfmt != icd->current_fmt->host_fmt->fourcc ||
+	    !image_mode || !ceu_crop) {
+		/*
+		 * first request client .s_fmt(): this will provide it with the new
+		 * pixel code
+		 */
+		ret = v4l2_device_call_until_err(sd->v4l2_dev, (long)icd, video,
+						 s_mbus_fmt, &mf);
+		if (ret < 0)
+			return ret;
 
-	if (pix->height > ceu_sub_height)
-		ceu_sub_height = pix->height;
+		if (xlate->code != mf.code)
+			xlate = soc_camera_xlate_by_mcode(icd, mf.code, pixfmt);
+		if (!xlate) {
+			xlate = icd->current_fmt;
+			dev_warn(dev,
+				 "Buggy client! Code %x not found, using current %x\n",
+				 mf.code, xlate->code);
+
+			mf.code = xlate->code;
+			mf.field = icd->field;
+			field = icd->field;
+			ret = v4l2_device_call_until_err(sd->v4l2_dev, (long)icd,
+							 video, s_mbus_fmt, &mf);
+			if (ret < 0)
+				return ret;
+			/* All tolerance has an end... */
+			BUG_ON(xlate->code != mf.code);
+		}
+		pixfmt			= xlate->host_fmt->fourcc;
+		pix->pixelformat	= pixfmt;
+		image_mode		= ceu_fmt_can_scale(pixfmt);
+	}
 
-	pix->colorspace = mf.colorspace;
+	icd->field = field;
 
-	if (image_mode) {
-		/* Scale pix->{width x height} down to width x height */
-		scale_h		= calc_scale(ceu_sub_width, &pix->width);
-		scale_v		= calc_scale(ceu_sub_height, &pix->height);
+	if (!image_mode) {
+		if (ceu_crop) {
+			/* switching from a native to a non-native format */
+			ret = ceu_client_crop(icd, &mf);
+			if (ret < 0)
+				return ret;
+		}
+		ceu_non_native_scale(pcdev, cam, &mf);
 	} else {
-		pix->width	= ceu_sub_width;
-		pix->height	= ceu_sub_height;
-		scale_h		= 0;
-		scale_v		= 0;
-	}
+		if (!ceu_crop) {
+			/* We were not cropping and we keep it that way */
+			ret = ceu_crop_full_window(icd, pix, &mf);
+			if (ret < 0)
+				return ret;
+		}
 
-	pcdev->cflcr = scale_h | (scale_v << 16);
+		/*
+		 * If we were cropping, we don't touch our input here, only
+		 * change the output
+		 */
+		ceu_native_scale(pcdev, cam, pix);
+	}
 
 	/*
 	 * We have calculated CFLCR, the actual configuration will be performed
 	 * in sh_mobile_ceu_set_bus_param()
 	 */
 
-	dev_geo(dev, "10: W: %u : 0x%x = %u, H: %u : 0x%x = %u\n",
-		ceu_sub_width, scale_h, pix->width,
-		ceu_sub_height, scale_v, pix->height);
+	dev_geo(dev, "Client: %ux%u@%u:%u -> %ux%u\n",
+		cam->rect.width, cam->rect.height, cam->rect.left, cam->rect.top,
+		cam->width, cam->height);
+	dev_geo(dev, "CEU: %ux%u@%u:%u -> %ux%u\n",
+		cam->ceu_rect.width, cam->ceu_rect.height, cam->ceu_rect.left, cam->ceu_rect.top,
+		pix->width, pix->height);
 
 	cam->code		= xlate->code;
 	icd->current_fmt	= xlate;
 
-	pcdev->field = field;
-	pcdev->image_mode = image_mode;
+	pcdev->image_mode	= image_mode;
 
 	/* CFSZR requirement */
-	pix->width	&= ~3;
-	pix->height	&= ~3;
+	pix->width		&= ~3;
+	pix->height		&= ~3;
 
 	return 0;
 }
@@ -1845,8 +1665,13 @@ static int sh_mobile_ceu_try_fmt(struct soc_camera_device *icd,
 
 	xlate = soc_camera_xlate_by_fourcc(icd, pixfmt);
 	if (!xlate) {
-		dev_warn(icd->parent, "Format %x not found\n", pixfmt);
-		return -EINVAL;
+		xlate			= icd->current_fmt;
+		dev_warn(icd->parent, "Format %x not found, using current %x\n",
+			 pixfmt, xlate->host_fmt->fourcc);
+		pixfmt			= xlate->host_fmt->fourcc;
+		pix->pixelformat	= pixfmt;
+		pix->colorspace		= icd->colorspace;
+		pix->field		= icd->field;
 	}
 
 	/* FIXME: calculate using depth and bus width */
@@ -1866,8 +1691,12 @@ static int sh_mobile_ceu_try_fmt(struct soc_camera_device *icd,
 	mf.colorspace	= pix->colorspace;
 
 	ret = v4l2_device_call_until_err(sd->v4l2_dev, (long)icd, video, try_mbus_fmt, &mf);
-	if (ret < 0)
+	if (ret < 0) {
+		/* Buggy client driver */
+		dev_err(icd->parent,
+			"FIXME: client try_fmt() = %d\n", ret);
 		return ret;
+	}
 
 	pix->width	= mf.width;
 	pix->height	= mf.height;
@@ -1892,7 +1721,7 @@ static int sh_mobile_ceu_try_fmt(struct soc_camera_device *icd,
 			ret = v4l2_device_call_until_err(sd->v4l2_dev, (long)icd, video,
 							 try_mbus_fmt, &mf);
 			if (ret < 0) {
-				/* Shouldn't actually happen... */
+				/* Shouldn't happen */
 				dev_err(icd->parent,
 					"FIXME: client try_fmt() = %d\n", ret);
 				return ret;
@@ -1908,10 +1737,10 @@ static int sh_mobile_ceu_try_fmt(struct soc_camera_device *icd,
 	pix->width	&= ~3;
 	pix->height	&= ~3;
 
-	dev_geo(icd->parent, "%s(): return %d, fmt 0x%x, %ux%u\n",
-		__func__, ret, pix->pixelformat, pix->width, pix->height);
+	dev_geo(icd->parent, "%s(): fmt 0x%x, %ux%u\n",
+		__func__, pix->pixelformat, pix->width, pix->height);
 
-	return ret;
+	return 0;
 }
 
 static int sh_mobile_ceu_set_livecrop(struct soc_camera_device *icd,
@@ -1945,7 +1774,7 @@ static int sh_mobile_ceu_set_livecrop(struct soc_camera_device *icd,
 				.width		= out_width,
 				.height		= out_height,
 				.pixelformat	= icd->current_fmt->host_fmt->fourcc,
-				.field		= pcdev->field,
+				.field		= icd->field,
 				.colorspace	= icd->colorspace,
 			},
 		};
-- 
1.7.2.5

[PATCH 3/9] V4L: soc-camera: remove redundant parameter from the .set_bus_param() method

[Guennadi Liakhovetski]

The "pixfmt" parameter of the struct soc_camera_host_ops::set_bus_param()
method is redundant, because at the time, when this method is called,
pixfmt is guaranteed to be equal to icd->current_fmt->host_fmt->fourcc.
Remove this parameter and update all drivers accordingly.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/atmel-isi.c            |    2 +-
 drivers/media/video/mx1_camera.c           |    2 +-
 drivers/media/video/mx2_camera.c           |    3 +--
 drivers/media/video/mx3_camera.c           |    3 ++-
 drivers/media/video/omap1_camera.c         |    4 ++--
 drivers/media/video/pxa_camera.c           |    3 ++-
 drivers/media/video/sh_mobile_ceu_camera.c |   11 ++---------
 drivers/media/video/soc_camera.c           |    2 +-
 include/media/soc_camera.h                 |    2 +-
 9 files changed, 13 insertions(+), 19 deletions(-)

diff --git a/drivers/media/video/atmel-isi.c b/drivers/media/video/atmel-isi.c
index 8c775c5..84d7a85 100644
--- a/drivers/media/video/atmel-isi.c
+++ b/drivers/media/video/atmel-isi.c
@@ -803,7 +803,7 @@ static int isi_camera_querycap(struct soc_camera_host *ici,
 	return 0;
 }
 
-static int isi_camera_set_bus_param(struct soc_camera_device *icd, u32 pixfmt)
+static int isi_camera_set_bus_param(struct soc_camera_device *icd)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
diff --git a/drivers/media/video/mx1_camera.c b/drivers/media/video/mx1_camera.c
index 18e94c7..055d11d 100644
--- a/drivers/media/video/mx1_camera.c
+++ b/drivers/media/video/mx1_camera.c
@@ -487,7 +487,7 @@ static int mx1_camera_set_crop(struct soc_camera_device *icd,
 	return v4l2_subdev_call(sd, video, s_crop, a);
 }
 
-static int mx1_camera_set_bus_param(struct soc_camera_device *icd, __u32 pixfmt)
+static int mx1_camera_set_bus_param(struct soc_camera_device *icd)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
diff --git a/drivers/media/video/mx2_camera.c b/drivers/media/video/mx2_camera.c
index a803d9e..ffbfbfe 100644
--- a/drivers/media/video/mx2_camera.c
+++ b/drivers/media/video/mx2_camera.c
@@ -766,8 +766,7 @@ static void mx27_camera_emma_buf_init(struct soc_camera_device *icd,
 			pcdev->base_emma + PRP_INTR_CNTL);
 }
 
-static int mx2_camera_set_bus_param(struct soc_camera_device *icd,
-		__u32 pixfmt)
+static int mx2_camera_set_bus_param(struct soc_camera_device *icd)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
diff --git a/drivers/media/video/mx3_camera.c b/drivers/media/video/mx3_camera.c
index fb38e22..6020061 100644
--- a/drivers/media/video/mx3_camera.c
+++ b/drivers/media/video/mx3_camera.c
@@ -980,12 +980,13 @@ static int mx3_camera_querycap(struct soc_camera_host *ici,
 	return 0;
 }
 
-static int mx3_camera_set_bus_param(struct soc_camera_device *icd, __u32 pixfmt)
+static int mx3_camera_set_bus_param(struct soc_camera_device *icd)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
 	struct mx3_camera_dev *mx3_cam = ici->priv;
 	struct v4l2_mbus_config cfg = {.type = V4L2_MBUS_PARALLEL,};
+	u32 pixfmt = icd->current_fmt->host_fmt->fourcc;
 	unsigned long bus_flags, common_flags;
 	u32 dw, sens_conf;
 	const struct soc_mbus_pixelfmt *fmt;
diff --git a/drivers/media/video/omap1_camera.c b/drivers/media/video/omap1_camera.c
index e87ae2f..e73a23e 100644
--- a/drivers/media/video/omap1_camera.c
+++ b/drivers/media/video/omap1_camera.c
@@ -1435,13 +1435,13 @@ static int omap1_cam_querycap(struct soc_camera_host *ici,
 	return 0;
 }
 
-static int omap1_cam_set_bus_param(struct soc_camera_device *icd,
-		__u32 pixfmt)
+static int omap1_cam_set_bus_param(struct soc_camera_device *icd)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct device *dev = icd->parent;
 	struct soc_camera_host *ici = to_soc_camera_host(dev);
 	struct omap1_cam_dev *pcdev = ici->priv;
+	u32 pixfmt = icd->current_fmt->host_fmt->fourcc;
 	const struct soc_camera_format_xlate *xlate;
 	const struct soc_mbus_pixelfmt *fmt;
 	struct v4l2_mbus_config cfg = {.type = V4L2_MBUS_PARALLEL,};
diff --git a/drivers/media/video/pxa_camera.c b/drivers/media/video/pxa_camera.c
index 79fb22c..2f9ae63 100644
--- a/drivers/media/video/pxa_camera.c
+++ b/drivers/media/video/pxa_camera.c
@@ -1133,12 +1133,13 @@ static void pxa_camera_setup_cicr(struct soc_camera_device *icd,
 	__raw_writel(cicr0, pcdev->base + CICR0);
 }
 
-static int pxa_camera_set_bus_param(struct soc_camera_device *icd, __u32 pixfmt)
+static int pxa_camera_set_bus_param(struct soc_camera_device *icd)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
 	struct pxa_camera_dev *pcdev = ici->priv;
 	struct v4l2_mbus_config cfg = {.type = V4L2_MBUS_PARALLEL,};
+	u32 pixfmt = icd->current_fmt->host_fmt->fourcc;
 	unsigned long bus_flags, common_flags;
 	int ret;
 	struct pxa_cam *cam = icd->host_priv;
diff --git a/drivers/media/video/sh_mobile_ceu_camera.c b/drivers/media/video/sh_mobile_ceu_camera.c
index 33ffc35..367dd43 100644
--- a/drivers/media/video/sh_mobile_ceu_camera.c
+++ b/drivers/media/video/sh_mobile_ceu_camera.c
@@ -782,8 +782,7 @@ static struct v4l2_subdev *find_bus_subdev(struct sh_mobile_ceu_dev *pcdev,
 		V4L2_MBUS_DATA_ACTIVE_HIGH)
 
 /* Capture is not running, no interrupts, no locking needed */
-static int sh_mobile_ceu_set_bus_param(struct soc_camera_device *icd,
-				       __u32 pixfmt)
+static int sh_mobile_ceu_set_bus_param(struct soc_camera_device *icd)
 {
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
 	struct sh_mobile_ceu_dev *pcdev = ici->priv;
@@ -921,11 +920,6 @@ static int sh_mobile_ceu_set_bus_param(struct soc_camera_device *icd,
 	ceu_write(pcdev, CDOCR, value);
 	ceu_write(pcdev, CFWCR, 0); /* keep "datafetch firewall" disabled */
 
-	dev_dbg(icd->parent, "S_FMT successful for %c%c%c%c %ux%u\n",
-		pixfmt & 0xff, (pixfmt >> 8) & 0xff,
-		(pixfmt >> 16) & 0xff, (pixfmt >> 24) & 0xff,
-		icd->user_width, icd->user_height);
-
 	capture_restore(pcdev, capsr);
 
 	/* not in bundle mode: skip CBDSR, CDAYR2, CDACR2, CDBYR2, CDBCR2 */
@@ -1785,8 +1779,7 @@ static int sh_mobile_ceu_set_livecrop(struct soc_camera_device *icd,
 		if (!ret) {
 			icd->user_width		= out_width & ~3;
 			icd->user_height	= out_height & ~3;
-			ret = sh_mobile_ceu_set_bus_param(icd,
-					icd->current_fmt->host_fmt->fourcc);
+			ret = sh_mobile_ceu_set_bus_param(icd);
 		}
 	}
 
diff --git a/drivers/media/video/soc_camera.c b/drivers/media/video/soc_camera.c
index ba409ac..5596688 100644
--- a/drivers/media/video/soc_camera.c
+++ b/drivers/media/video/soc_camera.c
@@ -510,7 +510,7 @@ static int soc_camera_set_fmt(struct soc_camera_device *icd,
 		icd->user_width, icd->user_height);
 
 	/* set physical bus parameters */
-	return ici->ops->set_bus_param(icd, pix->pixelformat);
+	return ici->ops->set_bus_param(icd);
 }
 
 static int soc_camera_open(struct file *file)
diff --git a/include/media/soc_camera.h b/include/media/soc_camera.h
index 22e4bee..d60bad4 100644
--- a/include/media/soc_camera.h
+++ b/include/media/soc_camera.h
@@ -94,7 +94,7 @@ struct soc_camera_host_ops {
 			      struct soc_camera_device *);
 	int (*reqbufs)(struct soc_camera_device *, struct v4l2_requestbuffers *);
 	int (*querycap)(struct soc_camera_host *, struct v4l2_capability *);
-	int (*set_bus_param)(struct soc_camera_device *, __u32);
+	int (*set_bus_param)(struct soc_camera_device *);
 	int (*get_parm)(struct soc_camera_device *, struct v4l2_streamparm *);
 	int (*set_parm)(struct soc_camera_device *, struct v4l2_streamparm *);
 	int (*enum_fsizes)(struct soc_camera_device *, struct v4l2_frmsizeenum *);
-- 
1.7.2.5

[PATCH 4/9] V4L: add convenience macros to the subdevice / Media Controller API

[Guennadi Liakhovetski]

Drivers, that can be built and work with and without
CONFIG_VIDEO_V4L2_SUBDEV_API, need the v4l2_subdev_get_try_format() and
v4l2_subdev_get_try_crop() functions, even though their return value
should never be dereferenced. Also add convenience macros to init and
clean up subdevice internal media entities.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 include/media/v4l2-subdev.h |   11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index f0f3358..4670506 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -569,6 +569,9 @@ v4l2_subdev_get_try_crop(struct v4l2_subdev_fh *fh, unsigned int pad)
 {
 	return &fh->try_crop[pad];
 }
+#else
+#define v4l2_subdev_get_try_format(arg...)	NULL
+#define v4l2_subdev_get_try_crop(arg...)	NULL
 #endif
 
 extern const struct v4l2_file_operations v4l2_subdev_fops;
@@ -610,4 +613,12 @@ void v4l2_subdev_init(struct v4l2_subdev *sd,
 	((!(sd) || !(sd)->v4l2_dev || !(sd)->v4l2_dev->notify) ? -ENODEV : \
 	 (sd)->v4l2_dev->notify((sd), (notification), (arg)))
 
+#if defined(CONFIG_MEDIA_CONTROLLER)
+#define subdev_media_entity_init(sd, n, p, e)	media_entity_init(&(sd)->entity, n, p, e)
+#define subdev_media_entity_cleanup(sd)		media_entity_cleanup(&(sd)->entity)
+#else
+#define subdev_media_entity_init(sd, n, p, e)	0
+#define subdev_media_entity_cleanup(sd)		do {} while (0)
+#endif
+
 #endif
-- 
1.7.2.5

[PATCH 5/9] V4L: soc-camera: move bus parameter configuration to .vidioc_streamon()

[Guennadi Liakhovetski]

With the Media Controller API various pipeline entities can be configured
independently and in unpredictable order. The only location, where we know
for sure, that the pipeline should be ready, is .vidioc_streamon(). This
makes it the only suitable location for the bus parameter configuration.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/soc_camera.c |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/video/soc_camera.c b/drivers/media/video/soc_camera.c
index 5596688..2905a88 100644
--- a/drivers/media/video/soc_camera.c
+++ b/drivers/media/video/soc_camera.c
@@ -509,8 +509,7 @@ static int soc_camera_set_fmt(struct soc_camera_device *icd,
 	dev_dbg(icd->pdev, "set width: %d height: %d\n",
 		icd->user_width, icd->user_height);
 
-	/* set physical bus parameters */
-	return ici->ops->set_bus_param(icd);
+	return 0;
 }
 
 static int soc_camera_open(struct file *file)
@@ -814,6 +813,11 @@ static int soc_camera_streamon(struct file *file, void *priv,
 	if (icd->streamer != file)
 		return -EBUSY;
 
+	/* set physical bus parameters */
+	ret = ici->ops->set_bus_param(icd);
+	if (ret < 0)
+		return ret;
+
 	/* This calls buf_queue from host driver's videobuf_queue_ops */
 	if (ici->ops->init_videobuf)
 		ret = videobuf_streamon(&icd->vb_vidq);
-- 
1.7.2.5

[PATCH 6/9] V4L: soc-camera: prepare hooks for Media Controller wrapper

[Guennadi Liakhovetski]

Extend soc-camera host operations with a target parameter to specify, whether
the operation should be propagated to subdevices or only applied to the host
itself.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/atmel-isi.c            |   10 +++-
 drivers/media/video/mx1_camera.c           |   10 +++-
 drivers/media/video/mx2_camera.c           |   10 +++-
 drivers/media/video/mx3_camera.c           |   10 +++-
 drivers/media/video/omap1_camera.c         |   12 +++-
 drivers/media/video/pxa_camera.c           |   10 +++-
 drivers/media/video/sh_mobile_ceu_camera.c |   12 +++-
 drivers/media/video/soc_camera.c           |   82 +++++++++++++++++++++-------
 include/media/soc_camera.h                 |   25 ++++++++-
 include/media/soc_entity.h                 |   19 +++++++
 10 files changed, 163 insertions(+), 37 deletions(-)
 create mode 100644 include/media/soc_entity.h

diff --git a/drivers/media/video/atmel-isi.c b/drivers/media/video/atmel-isi.c
index 84d7a85..6db8d43 100644
--- a/drivers/media/video/atmel-isi.c
+++ b/drivers/media/video/atmel-isi.c
@@ -527,7 +527,7 @@ static int isi_camera_init_videobuf(struct vb2_queue *q,
 }
 
 static int isi_camera_set_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
 	struct atmel_isi *isi = ici->priv;
@@ -537,6 +537,9 @@ static int isi_camera_set_fmt(struct soc_camera_device *icd,
 	struct v4l2_mbus_framefmt mf;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(icd->parent, "Format %x not found\n",
@@ -577,7 +580,7 @@ static int isi_camera_set_fmt(struct soc_camera_device *icd,
 }
 
 static int isi_camera_try_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -586,6 +589,9 @@ static int isi_camera_try_fmt(struct soc_camera_device *icd,
 	u32 pixfmt = pix->pixelformat;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pixfmt);
 	if (pixfmt && !xlate) {
 		dev_warn(icd->parent, "Format %x not found\n", pixfmt);
diff --git a/drivers/media/video/mx1_camera.c b/drivers/media/video/mx1_camera.c
index 055d11d..b63a163 100644
--- a/drivers/media/video/mx1_camera.c
+++ b/drivers/media/video/mx1_camera.c
@@ -564,7 +564,7 @@ static int mx1_camera_set_bus_param(struct soc_camera_device *icd)
 }
 
 static int mx1_camera_set_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -572,6 +572,9 @@ static int mx1_camera_set_fmt(struct soc_camera_device *icd,
 	struct v4l2_mbus_framefmt mf;
 	int ret, buswidth;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(icd->parent, "Format %x not found\n",
@@ -610,7 +613,7 @@ static int mx1_camera_set_fmt(struct soc_camera_device *icd,
 }
 
 static int mx1_camera_try_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -619,6 +622,9 @@ static int mx1_camera_try_fmt(struct soc_camera_device *icd,
 	int ret;
 	/* TODO: limit to mx1 hardware capabilities */
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(icd->parent, "Format %x not found\n",
diff --git a/drivers/media/video/mx2_camera.c b/drivers/media/video/mx2_camera.c
index ffbfbfe..614dd0a 100644
--- a/drivers/media/video/mx2_camera.c
+++ b/drivers/media/video/mx2_camera.c
@@ -911,7 +911,7 @@ static int mx2_camera_set_crop(struct soc_camera_device *icd,
 }
 
 static int mx2_camera_set_fmt(struct soc_camera_device *icd,
-			       struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -919,6 +919,9 @@ static int mx2_camera_set_fmt(struct soc_camera_device *icd,
 	struct v4l2_mbus_framefmt mf;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(icd->parent, "Format %x not found\n",
@@ -949,7 +952,7 @@ static int mx2_camera_set_fmt(struct soc_camera_device *icd,
 }
 
 static int mx2_camera_try_fmt(struct soc_camera_device *icd,
-				  struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -959,6 +962,9 @@ static int mx2_camera_try_fmt(struct soc_camera_device *icd,
 	unsigned int width_limit;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pixfmt);
 	if (pixfmt && !xlate) {
 		dev_warn(icd->parent, "Format %x not found\n", pixfmt);
diff --git a/drivers/media/video/mx3_camera.c b/drivers/media/video/mx3_camera.c
index 6020061..466c2cf 100644
--- a/drivers/media/video/mx3_camera.c
+++ b/drivers/media/video/mx3_camera.c
@@ -846,7 +846,7 @@ static int mx3_camera_set_crop(struct soc_camera_device *icd,
 }
 
 static int mx3_camera_set_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
 	struct mx3_camera_dev *mx3_cam = ici->priv;
@@ -856,6 +856,9 @@ static int mx3_camera_set_fmt(struct soc_camera_device *icd,
 	struct v4l2_mbus_framefmt mf;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(icd->parent, "Format %x not found\n",
@@ -906,7 +909,7 @@ static int mx3_camera_set_fmt(struct soc_camera_device *icd,
 }
 
 static int mx3_camera_try_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -915,6 +918,9 @@ static int mx3_camera_try_fmt(struct soc_camera_device *icd,
 	__u32 pixfmt = pix->pixelformat;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pixfmt);
 	if (pixfmt && !xlate) {
 		dev_warn(icd->parent, "Format %x not found\n", pixfmt);
diff --git a/drivers/media/video/omap1_camera.c b/drivers/media/video/omap1_camera.c
index e73a23e..967e32b 100644
--- a/drivers/media/video/omap1_camera.c
+++ b/drivers/media/video/omap1_camera.c
@@ -1216,7 +1216,7 @@ static int set_mbus_format(struct omap1_cam_dev *pcdev, struct device *dev,
 }
 
 static int omap1_cam_set_crop(struct soc_camera_device *icd,
-			       struct v4l2_crop *crop)
+			      struct v4l2_crop *crop)
 {
 	struct v4l2_rect *rect = &crop->c;
 	const struct soc_camera_format_xlate *xlate = icd->current_fmt;
@@ -1265,7 +1265,7 @@ static int omap1_cam_set_crop(struct soc_camera_device *icd,
 }
 
 static int omap1_cam_set_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			     struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -1276,6 +1276,9 @@ static int omap1_cam_set_fmt(struct soc_camera_device *icd,
 	struct v4l2_mbus_framefmt mf;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(dev, "%s: format %#x not found\n", __func__,
@@ -1314,7 +1317,7 @@ static int omap1_cam_set_fmt(struct soc_camera_device *icd,
 }
 
 static int omap1_cam_try_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			     struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -1323,6 +1326,9 @@ static int omap1_cam_try_fmt(struct soc_camera_device *icd,
 	int ret;
 	/* TODO: limit to mx1 hardware capabilities */
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(icd->parent, "Format %#x not found\n",
diff --git a/drivers/media/video/pxa_camera.c b/drivers/media/video/pxa_camera.c
index 2f9ae63..fc74788 100644
--- a/drivers/media/video/pxa_camera.c
+++ b/drivers/media/video/pxa_camera.c
@@ -1418,7 +1418,7 @@ static int pxa_camera_set_crop(struct soc_camera_device *icd,
 }
 
 static int pxa_camera_set_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct device *dev = icd->parent;
 	struct soc_camera_host *ici = to_soc_camera_host(dev);
@@ -1433,6 +1433,9 @@ static int pxa_camera_set_fmt(struct soc_camera_device *icd,
 	struct v4l2_mbus_framefmt mf;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pix->pixelformat);
 	if (!xlate) {
 		dev_warn(dev, "Format %x not found\n", pix->pixelformat);
@@ -1488,7 +1491,7 @@ static int pxa_camera_set_fmt(struct soc_camera_device *icd,
 }
 
 static int pxa_camera_try_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+			      struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	const struct soc_camera_format_xlate *xlate;
@@ -1497,6 +1500,9 @@ static int pxa_camera_try_fmt(struct soc_camera_device *icd,
 	__u32 pixfmt = pix->pixelformat;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	xlate = soc_camera_xlate_by_fourcc(icd, pixfmt);
 	if (!xlate) {
 		dev_warn(icd->parent, "Format %x not found\n", pixfmt);
diff --git a/drivers/media/video/sh_mobile_ceu_camera.c b/drivers/media/video/sh_mobile_ceu_camera.c
index 367dd43..3e7085f 100644
--- a/drivers/media/video/sh_mobile_ceu_camera.c
+++ b/drivers/media/video/sh_mobile_ceu_camera.c
@@ -1509,7 +1509,7 @@ static int ceu_crop_full_window(struct soc_camera_device *icd,
 
 /* Similar to set_crop multistage iterative algorithm */
 static int sh_mobile_ceu_set_fmt(struct soc_camera_device *icd,
-				 struct v4l2_format *f)
+				 struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct device *dev = icd->parent;
@@ -1530,6 +1530,9 @@ static int sh_mobile_ceu_set_fmt(struct soc_camera_device *icd,
 	bool image_mode;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	/* .try_fmt() has been called, size valid */
 
 	xlate = soc_camera_xlate_by_fourcc(icd, pixfmt);
@@ -1644,7 +1647,7 @@ static int sh_mobile_ceu_set_fmt(struct soc_camera_device *icd,
 }
 
 static int sh_mobile_ceu_try_fmt(struct soc_camera_device *icd,
-				 struct v4l2_format *f)
+				 struct v4l2_format *f, enum soc_camera_target tgt)
 {
 	const struct soc_camera_format_xlate *xlate;
 	struct v4l2_pix_format *pix = &f->fmt.pix;
@@ -1654,6 +1657,9 @@ static int sh_mobile_ceu_try_fmt(struct soc_camera_device *icd,
 	int width, height;
 	int ret;
 
+	if (tgt != SOCAM_TARGET_PIPELINE)
+		return -EINVAL;
+
 	dev_geo(icd->parent, "TRY_FMT(pix=0x%x, %ux%u)\n",
 		 pixfmt, pix->width, pix->height);
 
@@ -1772,7 +1778,7 @@ static int sh_mobile_ceu_set_livecrop(struct soc_camera_device *icd,
 				.colorspace	= icd->colorspace,
 			},
 		};
-		ret = sh_mobile_ceu_set_fmt(icd, &f);
+		ret = sh_mobile_ceu_set_fmt(icd, &f, SOCAM_TARGET_PIPELINE);
 		if (!ret && (out_width != f.fmt.pix.width ||
 			     out_height != f.fmt.pix.height))
 			ret = -EINVAL;
diff --git a/drivers/media/video/soc_camera.c b/drivers/media/video/soc_camera.c
index 2905a88..790c14c 100644
--- a/drivers/media/video/soc_camera.c
+++ b/drivers/media/video/soc_camera.c
@@ -30,6 +30,7 @@
 #include <linux/vmalloc.h>
 
 #include <media/soc_camera.h>
+#include <media/soc_entity.h>
 #include <media/v4l2-common.h>
 #include <media/v4l2-ioctl.h>
 #include <media/v4l2-dev.h>
@@ -183,8 +184,8 @@ EXPORT_SYMBOL(soc_camera_apply_board_flags);
 #define pixfmtstr(x) (x) & 0xff, ((x) >> 8) & 0xff, ((x) >> 16) & 0xff, \
 	((x) >> 24) & 0xff
 
-static int soc_camera_try_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+int soc_camera_try_fmt(struct soc_camera_device *icd, struct v4l2_format *f,
+		       enum soc_camera_target target)
 {
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
 	struct v4l2_pix_format *pix = &f->fmt.pix;
@@ -196,7 +197,7 @@ static int soc_camera_try_fmt(struct soc_camera_device *icd,
 	pix->bytesperline = 0;
 	pix->sizeimage = 0;
 
-	ret = ici->ops->try_fmt(icd, f);
+	ret = ici->ops->try_fmt(icd, f, target);
 	if (ret < 0)
 		return ret;
 
@@ -232,7 +233,7 @@ static int soc_camera_try_fmt_vid_cap(struct file *file, void *priv,
 		return -EINVAL;
 
 	/* limit format to hardware capabilities */
-	return soc_camera_try_fmt(icd, f);
+	return soc_camera_try_fmt(icd, f, SOCAM_TARGET_PIPELINE);
 }
 
 static int soc_camera_enum_input(struct file *file, void *priv,
@@ -472,22 +473,24 @@ static void soc_camera_free_user_formats(struct soc_camera_device *icd)
 }
 
 /* Called with .vb_lock held, or from the first open(2), see comment there */
-static int soc_camera_set_fmt(struct soc_camera_device *icd,
-			      struct v4l2_format *f)
+int soc_camera_set_fmt(struct soc_camera_device *icd, struct v4l2_format *f,
+		       enum soc_camera_target target)
 {
+	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
 	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
 	struct v4l2_pix_format *pix = &f->fmt.pix;
+	struct v4l2_mbus_framefmt mf;
 	int ret;
 
 	dev_dbg(icd->pdev, "S_FMT(%c%c%c%c, %ux%u)\n",
 		pixfmtstr(pix->pixelformat), pix->width, pix->height);
 
-	/* We always call try_fmt() before set_fmt() or set_crop() */
-	ret = soc_camera_try_fmt(icd, f);
+	/* We always call try_fmt() before set_fmt() */
+	ret = soc_camera_try_fmt(icd, f, target);
 	if (ret < 0)
 		return ret;
 
-	ret = ici->ops->set_fmt(icd, f);
+	ret = ici->ops->set_fmt(icd, f, target);
 	if (ret < 0) {
 		return ret;
 	} else if (!icd->current_fmt ||
@@ -497,8 +500,26 @@ static int soc_camera_set_fmt(struct soc_camera_device *icd,
 		return -EINVAL;
 	}
 
-	icd->user_width		= pix->width;
-	icd->user_height	= pix->height;
+	switch (target) {
+	case SOCAM_TARGET_PIPELINE:
+		ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
+		if (!ret) {
+			icd->host_input_width	= mf.width;
+			icd->host_input_height	= mf.height;
+		} else {
+			/* What shall we do with such a client? */
+			icd->host_input_width	= pix->width;
+			icd->host_input_height	= pix->height;
+		}
+		/* fall through */
+	case SOCAM_TARGET_HOST_OUT:
+		icd->user_width		= pix->width;
+		icd->user_height	= pix->height;
+		break;
+	case SOCAM_TARGET_HOST_IN:
+		icd->host_input_width	= pix->width;
+		icd->host_input_height	= pix->height;
+	}
 	icd->bytesperline	= pix->bytesperline;
 	icd->sizeimage		= pix->sizeimage;
 	icd->colorspace		= pix->colorspace;
@@ -515,8 +536,8 @@ static int soc_camera_set_fmt(struct soc_camera_device *icd,
 static int soc_camera_open(struct file *file)
 {
 	struct video_device *vdev = video_devdata(file);
-	struct soc_camera_device *icd = dev_get_drvdata(vdev->parent);
-	struct soc_camera_link *icl = to_soc_camera_link(icd);
+	struct soc_camera_device *icd = video_get_drvdata(vdev);
+	struct soc_camera_link *icl;
 	struct soc_camera_host *ici;
 	int ret;
 
@@ -524,6 +545,7 @@ static int soc_camera_open(struct file *file)
 		/* No device driver attached */
 		return -ENODEV;
 
+	icl = to_soc_camera_link(icd);
 	ici = to_soc_camera_host(icd->parent);
 
 	if (!try_module_get(ici->ops->owner)) {
@@ -573,7 +595,7 @@ static int soc_camera_open(struct file *file)
 		 * apart from someone else calling open() simultaneously, but
 		 * .video_lock is protecting us against it.
 		 */
-		ret = soc_camera_set_fmt(icd, &f);
+		ret = soc_camera_set_fmt(icd, &f, SOCAM_TARGET_PIPELINE);
 		if (ret < 0)
 			goto esfmt;
 
@@ -735,7 +757,7 @@ static int soc_camera_s_fmt_vid_cap(struct file *file, void *priv,
 		return -EBUSY;
 	}
 
-	ret = soc_camera_set_fmt(icd, f);
+	ret = soc_camera_set_fmt(icd, f, SOCAM_TARGET_PIPELINE);
 
 	if (!ret && !icd->streamer)
 		icd->streamer = file;
@@ -1132,6 +1154,17 @@ static int soc_camera_probe(struct soc_camera_device *icd)
 	sd = soc_camera_to_subdev(icd);
 	sd->grp_id = (long)icd;
 
+	if (sd->ops->pad) {
+		/*
+		 * This client driver implements the pad-level API. Build and
+		 * enable a Media Controller infrastructure. If the host driver
+		 * doesn't implement it, it will be emulated.
+		 */
+		ret = soc_camera_mc_install(icd);
+		if (ret < 0)
+			goto emce;
+	}
+
 	if (v4l2_ctrl_add_handler(&icd->ctrl_handler, sd->ctrl_handler))
 		goto ectrl;
 
@@ -1159,8 +1192,10 @@ static int soc_camera_probe(struct soc_camera_device *icd)
 
 	/* Try to improve our guess of a reasonable window format */
 	if (!v4l2_subdev_call(sd, video, g_mbus_fmt, &mf)) {
-		icd->user_width		= mf.width;
-		icd->user_height	= mf.height;
+		icd->host_input_width	= mf.width;
+		icd->host_input_height	= mf.height;
+		icd->user_width		= icd->host_input_width;
+		icd->user_height	= icd->host_input_height;
 		icd->colorspace		= mf.colorspace;
 		icd->field		= mf.field;
 	}
@@ -1180,6 +1215,8 @@ evidstart:
 	soc_camera_free_user_formats(icd);
 eiufmt:
 ectrl:
+	soc_camera_mc_free(icd);
+emce:
 	if (icl->board_info) {
 		soc_camera_free_i2c(icd);
 	} else {
@@ -1218,6 +1255,8 @@ static int soc_camera_remove(struct soc_camera_device *icd)
 		icd->vdev = NULL;
 	}
 
+	soc_camera_mc_free(icd);
+
 	if (icl->board_info) {
 		soc_camera_free_i2c(icd);
 	} else {
@@ -1336,6 +1375,8 @@ int soc_camera_host_register(struct soc_camera_host *ici)
 	if (ret < 0)
 		goto edevreg;
 
+	soc_camera_mc_register(ici);
+
 	list_add_tail(&ici->list, &hosts);
 	mutex_unlock(&list_lock);
 
@@ -1361,9 +1402,11 @@ void soc_camera_host_unregister(struct soc_camera_host *ici)
 		if (icd->iface == ici->nr && to_soc_camera_control(icd))
 			soc_camera_remove(icd);
 
-	mutex_unlock(&list_lock);
+	soc_camera_mc_unregister(ici);
 
 	v4l2_device_unregister(&ici->v4l2_dev);
+
+	mutex_unlock(&list_lock);
 }
 EXPORT_SYMBOL(soc_camera_host_unregister);
 
@@ -1443,7 +1486,6 @@ static int video_dev_create(struct soc_camera_device *icd)
 
 	strlcpy(vdev->name, ici->drv_name, sizeof(vdev->name));
 
-	vdev->parent		= icd->pdev;
 	vdev->current_norm	= V4L2_STD_UNKNOWN;
 	vdev->fops		= &soc_camera_fops;
 	vdev->ioctl_ops		= &soc_camera_ioctl_ops;
@@ -1451,6 +1493,8 @@ static int video_dev_create(struct soc_camera_device *icd)
 	vdev->tvnorms		= V4L2_STD_UNKNOWN;
 	vdev->ctrl_handler	= &icd->ctrl_handler;
 	vdev->lock		= &icd->video_lock;
+	vdev->v4l2_dev		= &ici->v4l2_dev;
+	video_set_drvdata(vdev, icd);
 
 	icd->vdev = vdev;
 
diff --git a/include/media/soc_camera.h b/include/media/soc_camera.h
index d60bad4..0a21ff1 100644
--- a/include/media/soc_camera.h
+++ b/include/media/soc_camera.h
@@ -54,6 +54,8 @@ struct soc_camera_device {
 		struct videobuf_queue vb_vidq;
 		struct vb2_queue vb2_vidq;
 	};
+	unsigned int host_input_width;
+	unsigned int host_input_height;
 };
 
 struct soc_camera_host {
@@ -63,6 +65,18 @@ struct soc_camera_host {
 	void *priv;
 	const char *drv_name;
 	struct soc_camera_host_ops *ops;
+#if defined(CONFIG_MEDIA_CONTROLLER)
+	struct media_device mdev;
+	struct v4l2_subdev bus_sd;
+	struct media_pad bus_pads[2];
+	struct media_pad vdev_pads[1];
+#endif
+};
+
+enum soc_camera_target {
+	SOCAM_TARGET_PIPELINE,
+	SOCAM_TARGET_HOST_IN,
+	SOCAM_TARGET_HOST_OUT,
 };
 
 struct soc_camera_host_ops {
@@ -86,8 +100,10 @@ struct soc_camera_host_ops {
 	 * to change the output sizes
 	 */
 	int (*set_livecrop)(struct soc_camera_device *, struct v4l2_crop *);
-	int (*set_fmt)(struct soc_camera_device *, struct v4l2_format *);
-	int (*try_fmt)(struct soc_camera_device *, struct v4l2_format *);
+	int (*set_fmt)(struct soc_camera_device *, struct v4l2_format *,
+		       enum soc_camera_target);
+	int (*try_fmt)(struct soc_camera_device *, struct v4l2_format *,
+		       enum soc_camera_target);
 	void (*init_videobuf)(struct videobuf_queue *,
 			      struct soc_camera_device *);
 	int (*init_videobuf2)(struct vb2_queue *,
@@ -173,6 +189,11 @@ static inline struct v4l2_subdev *soc_camera_to_subdev(
 int soc_camera_host_register(struct soc_camera_host *ici);
 void soc_camera_host_unregister(struct soc_camera_host *ici);
 
+int soc_camera_try_fmt(struct soc_camera_device *icd, struct v4l2_format *f,
+		       enum soc_camera_target target);
+int soc_camera_set_fmt(struct soc_camera_device *icd, struct v4l2_format *f,
+		       enum soc_camera_target target);
+
 const struct soc_camera_format_xlate *soc_camera_xlate_by_fourcc(
 	const struct soc_camera_device *icd, u32 fourcc);
 
diff --git a/include/media/soc_entity.h b/include/media/soc_entity.h
new file mode 100644
index 0000000..e461f5e
--- /dev/null
+++ b/include/media/soc_entity.h
@@ -0,0 +1,19 @@
+/*
+ * soc-camera Media Controller wrapper
+ *
+ * Copyright (C) 2011, Guennadi Liakhovetski <g.liakhovetski@gmx.de>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef SOC_ENTITY_H
+#define SOC_ENTITY_H
+
+#define soc_camera_mc_install(x) 0
+#define soc_camera_mc_free(x) do {} while (0)
+#define soc_camera_mc_register(x) do {} while (0)
+#define soc_camera_mc_unregister(x) do {} while (0)
+
+#endif
-- 
1.7.2.5

[PATCH 7/9] V4L: soc-camera: add a Media Controller wrapper

[Guennadi Liakhovetski]

This wrapper adds a Media Controller implementation to soc-camera drivers.
To really benefit from it individual host drivers should implement support
for values of enum soc_camera_target other than SOCAM_TARGET_PIPELINE in
their .set_fmt() and .try_fmt() methods.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/Makefile       |    6 +-
 drivers/media/video/soc_camera.c   |   46 ++++--
 drivers/media/video/soc_entity.c   |  284 ++++++++++++++++++++++++++++++++++++
 drivers/media/video/soc_mediabus.c |   16 --
 include/media/soc_camera.h         |    1 +
 include/media/soc_entity.h         |   12 ++
 6 files changed, 334 insertions(+), 31 deletions(-)
 create mode 100644 drivers/media/video/soc_entity.c

diff --git a/drivers/media/video/Makefile b/drivers/media/video/Makefile
index 11fff97..f4e3d52 100644
--- a/drivers/media/video/Makefile
+++ b/drivers/media/video/Makefile
@@ -162,7 +162,11 @@ obj-$(CONFIG_VIDEO_CX23885) += cx23885/
 obj-$(CONFIG_VIDEO_AK881X)		+= ak881x.o
 
 obj-$(CONFIG_VIDEO_OMAP2)		+= omap2cam.o
-obj-$(CONFIG_SOC_CAMERA)		+= soc_camera.o soc_mediabus.o
+obj-$(CONFIG_SOC_CAMERA)		+= soc_camera_core.o
+soc_camera_core-objs			:= soc_camera.o soc_mediabus.o
+ifeq ($(CONFIG_MEDIA_CONTROLLER),y)
+soc_camera_core-objs			+= soc_entity.o
+endif
 obj-$(CONFIG_SOC_CAMERA_PLATFORM)	+= soc_camera_platform.o
 # soc-camera host drivers have to be linked after camera drivers
 obj-$(CONFIG_VIDEO_MX1)			+= mx1_camera.o
diff --git a/drivers/media/video/soc_camera.c b/drivers/media/video/soc_camera.c
index 790c14c..9b4c3c0 100644
--- a/drivers/media/video/soc_camera.c
+++ b/drivers/media/video/soc_camera.c
@@ -515,17 +515,17 @@ int soc_camera_set_fmt(struct soc_camera_device *icd, struct v4l2_format *f,
 	case SOCAM_TARGET_HOST_OUT:
 		icd->user_width		= pix->width;
 		icd->user_height	= pix->height;
+		icd->bytesperline	= pix->bytesperline;
+		icd->sizeimage		= pix->sizeimage;
+		icd->colorspace		= pix->colorspace;
+		icd->field		= pix->field;
+		if (ici->ops->init_videobuf)
+			icd->vb_vidq.field = pix->field;
 		break;
 	case SOCAM_TARGET_HOST_IN:
 		icd->host_input_width	= pix->width;
 		icd->host_input_height	= pix->height;
 	}
-	icd->bytesperline	= pix->bytesperline;
-	icd->sizeimage		= pix->sizeimage;
-	icd->colorspace		= pix->colorspace;
-	icd->field		= pix->field;
-	if (ici->ops->init_videobuf)
-		icd->vb_vidq.field = pix->field;
 
 	dev_dbg(icd->pdev, "set width: %d height: %d\n",
 		icd->user_width, icd->user_height);
@@ -835,10 +835,14 @@ static int soc_camera_streamon(struct file *file, void *priv,
 	if (icd->streamer != file)
 		return -EBUSY;
 
+	ret = soc_camera_mc_streamon(icd);
+	if (ret < 0)
+		return ret;
+
 	/* set physical bus parameters */
 	ret = ici->ops->set_bus_param(icd);
 	if (ret < 0)
-		return ret;
+		goto ebusp;
 
 	/* This calls buf_queue from host driver's videobuf_queue_ops */
 	if (ici->ops->init_videobuf)
@@ -846,9 +850,23 @@ static int soc_camera_streamon(struct file *file, void *priv,
 	else
 		ret = vb2_streamon(&icd->vb2_vidq, i);
 
-	if (!ret)
-		v4l2_subdev_call(sd, video, s_stream, 1);
+	if (ret < 0)
+		goto estreamon;
+
+	ret = v4l2_subdev_call(sd, video, s_stream, 1);
+	if (ret < 0 && ret != -ENOIOCTLCMD)
+		goto esdstream;
+
+	return ret;
 
+esdstream:
+	if (ici->ops->init_videobuf)
+		videobuf_streamoff(&icd->vb_vidq);
+	else
+		vb2_streamoff(&icd->vb2_vidq, i);
+estreamon:
+ebusp:
+	soc_camera_mc_streamoff(icd);
 	return ret;
 }
 
@@ -877,6 +895,7 @@ static int soc_camera_streamoff(struct file *file, void *priv,
 		vb2_streamoff(&icd->vb2_vidq, i);
 
 	v4l2_subdev_call(sd, video, s_stream, 0);
+	soc_camera_mc_streamoff(icd);
 
 	return 0;
 }
@@ -1250,12 +1269,11 @@ static int soc_camera_remove(struct soc_camera_device *icd)
 	BUG_ON(!icd->parent);
 
 	v4l2_ctrl_handler_free(&icd->ctrl_handler);
-	if (vdev) {
+	soc_camera_mc_free(icd);
+	if (vdev)
 		video_unregister_device(vdev);
-		icd->vdev = NULL;
-	}
 
-	soc_camera_mc_free(icd);
+	icd->vdev = NULL;
 
 	if (icl->board_info) {
 		soc_camera_free_i2c(icd);
@@ -1484,7 +1502,7 @@ static int video_dev_create(struct soc_camera_device *icd)
 	if (!vdev)
 		return -ENOMEM;
 
-	strlcpy(vdev->name, ici->drv_name, sizeof(vdev->name));
+	snprintf(vdev->name, sizeof(vdev->name), "%s output", ici->drv_name);
 
 	vdev->current_norm	= V4L2_STD_UNKNOWN;
 	vdev->fops		= &soc_camera_fops;
diff --git a/drivers/media/video/soc_entity.c b/drivers/media/video/soc_entity.c
new file mode 100644
index 0000000..3a04700
--- /dev/null
+++ b/drivers/media/video/soc_entity.c
@@ -0,0 +1,284 @@
+/*
+ * soc-camera Media Controller wrapper
+ *
+ * Copyright (C) 2011, Guennadi Liakhovetski <kernel@pengutronix.de>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/media.h>
+#include <linux/string.h>
+#include <linux/v4l2-mediabus.h>
+
+#include <media/soc_camera.h>
+#include <media/soc_mediabus.h>
+#include <media/v4l2-common.h>
+#include <media/v4l2-subdev.h>
+
+#define soc_entity_native_mc(x) false
+
+enum {
+	SOC_HOST_BUS_PAD_SINK,
+	SOC_HOST_BUS_PAD_SOURCE,
+};
+
+enum {
+	SOC_HOST_VDEV_PAD_SINK,
+};
+
+static void se_v4l2_to_mbus(struct soc_camera_device *icd,
+			    struct v4l2_format *vf,
+			    struct v4l2_mbus_framefmt *mf)
+{
+	struct v4l2_pix_format *pfmt = &vf->fmt.pix;
+
+	mf->width	= pfmt->width;
+	mf->height	= pfmt->height;
+	mf->colorspace	= pfmt->colorspace;
+	mf->field	= pfmt->field;
+	mf->code	= soc_camera_xlate_by_fourcc(icd,
+						pfmt->pixelformat)->code;
+}
+
+static void se_mbus_to_v4l2(struct soc_camera_device *icd,
+			    struct v4l2_mbus_framefmt *mf,
+			    struct v4l2_format *vf)
+{
+	struct v4l2_pix_format *pfmt = &vf->fmt.pix;
+	u32 fourcc = icd->current_fmt->host_fmt->fourcc;
+
+	pfmt->width		= mf->width;
+	pfmt->height		= mf->height;
+	pfmt->colorspace	= mf->colorspace;
+	pfmt->field		= mf->field;
+	pfmt->pixelformat	= soc_camera_xlate_by_mcode(icd,
+					mf->code, fourcc)->host_fmt->fourcc;
+}
+
+static int bus_sd_pad_g_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+			    struct v4l2_subdev_format *sd_fmt)
+{
+	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
+	struct v4l2_mbus_framefmt *f = &sd_fmt->format;
+
+	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+		sd_fmt->format = *v4l2_subdev_get_try_format(fh, sd_fmt->pad);
+		return 0;
+	}
+
+	if (sd_fmt->pad == SOC_HOST_BUS_PAD_SINK) {
+		f->width	= icd->host_input_width;
+		f->height	= icd->host_input_height;
+	} else {
+		f->width	= icd->user_width;
+		f->height	= icd->user_height;
+	}
+	f->field	= icd->field;
+	f->code		= icd->current_fmt->code;
+	f->colorspace	= icd->colorspace;
+
+	return 0;
+}
+
+static int bus_sd_pad_s_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+			    struct v4l2_subdev_format *sd_fmt)
+{
+	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
+	struct v4l2_mbus_framefmt *mf = &sd_fmt->format;
+	struct v4l2_format vf = {
+		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE,
+	};
+	enum soc_camera_target tgt = sd_fmt->pad == SOC_HOST_BUS_PAD_SINK ?
+		SOCAM_TARGET_HOST_IN : SOCAM_TARGET_HOST_OUT;
+	int ret;
+
+	se_mbus_to_v4l2(icd, mf, &vf);
+
+	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+		struct v4l2_mbus_framefmt *try_fmt =
+			v4l2_subdev_get_try_format(fh, sd_fmt->pad);
+		ret = soc_camera_try_fmt(icd, &vf, tgt);
+		if (!ret) {
+			se_v4l2_to_mbus(icd, &vf, try_fmt);
+			sd_fmt->format = *try_fmt;
+		}
+		return ret;
+	}
+
+	ret = soc_camera_set_fmt(icd, &vf, tgt);
+	if (!ret)
+		se_v4l2_to_mbus(icd, &vf, &sd_fmt->format);
+
+	return ret;
+}
+
+static int bus_sd_pad_enum_mbus_code(struct v4l2_subdev *sd,
+				     struct v4l2_subdev_fh *fh,
+				     struct v4l2_subdev_mbus_code_enum *ce)
+{
+	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
+
+	if (ce->index >= icd->num_user_formats)
+		return -EINVAL;
+
+	ce->code = icd->user_formats[ce->index].code;
+	return 0;
+}
+
+static const struct v4l2_subdev_pad_ops se_bus_sd_pad_ops = {
+	.get_fmt	= bus_sd_pad_g_fmt,
+	.set_fmt	= bus_sd_pad_s_fmt,
+	.enum_mbus_code	= bus_sd_pad_enum_mbus_code,
+};
+
+static const struct v4l2_subdev_ops se_bus_sd_ops = {
+	.pad		= &se_bus_sd_pad_ops,
+};
+
+static const struct media_entity_operations se_bus_me_ops = {
+};
+
+static const struct media_entity_operations se_vdev_me_ops = {
+};
+
+int soc_camera_mc_streamon(struct soc_camera_device *icd)
+{
+	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
+	struct v4l2_subdev *bus_sd = &ici->bus_sd;
+	struct media_entity *bus_me = &bus_sd->entity;
+	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
+	struct v4l2_mbus_framefmt mf;
+	int ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
+	if (WARN_ON(ret < 0))
+		return ret;
+	if (icd->host_input_width != mf.width ||
+	    icd->host_input_height != mf.height ||
+	    icd->current_fmt->code != mf.code)
+		return -EINVAL;
+
+	media_entity_pipeline_start(bus_me, &ici->pipe);
+	return 0;
+}
+
+void soc_camera_mc_streamoff(struct soc_camera_device *icd)
+{
+	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
+	struct v4l2_subdev *bus_sd = &ici->bus_sd;
+	struct media_entity *bus_me = &bus_sd->entity;
+	media_entity_pipeline_stop(bus_me);
+}
+
+int soc_camera_mc_install(struct soc_camera_device *icd)
+{
+	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
+	struct v4l2_subdev *bus_sd = &ici->bus_sd;
+	struct media_entity *bus_me = &bus_sd->entity;
+	struct media_pad *bus_pads = ici->bus_pads;
+	struct media_pad *vdev_pads = ici->vdev_pads;
+	struct video_device *vdev = icd->vdev;
+	struct media_entity *vdev_me = &vdev->entity;
+	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
+	int ret;
+
+	if (!ici->v4l2_dev.mdev || soc_entity_native_mc(icd))
+		return 0;
+
+	/* Configure the video bus subdevice, entity, and pads */
+	v4l2_subdev_init(bus_sd, &se_bus_sd_ops);
+	v4l2_set_subdevdata(bus_sd, icd);
+	bus_sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+	snprintf(bus_sd->name, sizeof(bus_sd->name), "%s input", ici->drv_name);
+
+	bus_pads[SOC_HOST_BUS_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+	bus_pads[SOC_HOST_BUS_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+	bus_me->ops = &se_bus_me_ops;
+
+	ret = media_entity_init(bus_me, 2, bus_pads, 0);
+	if (ret < 0)
+		return ret;
+
+	/* Configure the video-device entity */
+	vdev_pads[SOC_HOST_VDEV_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+	vdev_me->ops = &se_vdev_me_ops;
+
+	ret = media_entity_init(vdev_me, 1, vdev_pads, 0);
+	if (ret < 0)
+		goto evmei;
+
+	/* Link the two entities */
+	ret = media_entity_create_link(bus_me, SOC_HOST_BUS_PAD_SOURCE,
+				vdev_me, SOC_HOST_VDEV_PAD_SINK,
+				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
+	if (ret < 0)
+		goto elink;
+
+	ret = v4l2_device_register_subdev(&ici->v4l2_dev, bus_sd);
+	if (ret < 0)
+		goto eregsd;
+
+	ret = v4l2_device_register_subdev_nodes(&ici->v4l2_dev);
+	if (ret < 0)
+		goto eregsdn;
+
+	/*
+	 * Link the client: make it immutable too for now, since there is no
+	 * meaningful mapping for the .link_setup() method to the soc-camera
+	 * API
+	 */
+	ret = media_entity_create_link(&sd->entity, 0,
+				bus_me, SOC_HOST_BUS_PAD_SINK,
+				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
+	if (ret < 0)
+		goto eclink;
+
+	return 0;
+
+eclink:
+eregsdn:
+	v4l2_device_unregister_subdev(bus_sd);
+eregsd:
+elink:
+	media_entity_cleanup(vdev_me);
+evmei:
+	media_entity_cleanup(bus_me);
+
+	return ret;
+}
+
+void soc_camera_mc_free(struct soc_camera_device *icd)
+{
+	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
+	struct v4l2_subdev *bus_sd = &ici->bus_sd;
+	struct media_entity *bus_me = &bus_sd->entity;
+	struct video_device *vdev = icd->vdev;
+	struct media_entity *vdev_me = &vdev->entity;
+
+	if (!ici->v4l2_dev.mdev || !soc_camera_to_subdev(icd)->ops->pad ||
+	    soc_entity_native_mc(icd))
+		return;
+
+	v4l2_device_unregister_subdev(bus_sd);
+
+	media_entity_cleanup(vdev_me);
+	media_entity_cleanup(bus_me);
+}
+
+void soc_camera_mc_register(struct soc_camera_host *ici)
+{
+	int ret;
+
+	/* The Big Moment: register the media device */
+	ici->mdev.dev = ici->v4l2_dev.dev;
+	ici->v4l2_dev.mdev = &ici->mdev;
+	strlcpy(ici->mdev.model, ici->drv_name, sizeof(ici->mdev.model));
+	ret = media_device_register(&ici->mdev);
+	if (ret < 0)
+		ici->v4l2_dev.mdev = NULL;
+}
+
+void soc_camera_mc_unregister(struct soc_camera_host *ici)
+{
+	if (ici->v4l2_dev.mdev)
+		media_device_unregister(&ici->mdev);
+}
diff --git a/drivers/media/video/soc_mediabus.c b/drivers/media/video/soc_mediabus.c
index cf7f219..9f84c5c 100644
--- a/drivers/media/video/soc_mediabus.c
+++ b/drivers/media/video/soc_mediabus.c
@@ -415,19 +415,3 @@ unsigned int soc_mbus_config_compatible(const struct v4l2_mbus_config *cfg,
 	return 0;
 }
 EXPORT_SYMBOL(soc_mbus_config_compatible);
-
-static int __init soc_mbus_init(void)
-{
-	return 0;
-}
-
-static void __exit soc_mbus_exit(void)
-{
-}
-
-module_init(soc_mbus_init);
-module_exit(soc_mbus_exit);
-
-MODULE_DESCRIPTION("soc-camera media bus interface");
-MODULE_AUTHOR("Guennadi Liakhovetski <g.liakhovetski@gmx.de>");
-MODULE_LICENSE("GPL v2");
diff --git a/include/media/soc_camera.h b/include/media/soc_camera.h
index 0a21ff1..8b2b4ee 100644
--- a/include/media/soc_camera.h
+++ b/include/media/soc_camera.h
@@ -70,6 +70,7 @@ struct soc_camera_host {
 	struct v4l2_subdev bus_sd;
 	struct media_pad bus_pads[2];
 	struct media_pad vdev_pads[1];
+	struct media_pipeline pipe;
 #endif
 };
 
diff --git a/include/media/soc_entity.h b/include/media/soc_entity.h
index e461f5e..e4c692a 100644
--- a/include/media/soc_entity.h
+++ b/include/media/soc_entity.h
@@ -11,9 +11,21 @@
 #ifndef SOC_ENTITY_H
 #define SOC_ENTITY_H
 
+#if defined(CONFIG_MEDIA_CONTROLLER)
+struct soc_camera_device;
+int soc_camera_mc_install(struct soc_camera_device *icd);
+void soc_camera_mc_free(struct soc_camera_device *icd);
+void soc_camera_mc_register(struct soc_camera_host *ici);
+void soc_camera_mc_unregister(struct soc_camera_host *ici);
+int soc_camera_mc_streamon(struct soc_camera_device *icd);
+int soc_camera_mc_streamoff(struct soc_camera_device *icd);
+#else
 #define soc_camera_mc_install(x) 0
 #define soc_camera_mc_free(x) do {} while (0)
 #define soc_camera_mc_register(x) do {} while (0)
 #define soc_camera_mc_unregister(x) do {} while (0)
+#define soc_camera_mc_streamon(x) 0
+#define soc_camera_mc_streamoff(x) do {} while (0)
+#endif
 
 #endif
-- 
1.7.2.5

[PATCH 8/9] V4L: mt9t112: add pad level operations

[Guennadi Liakhovetski]

On Media Controller enabled systems this patch allows the user to
communicate with the driver directly over /dev/v4l-subdev* device nodes
using VIDIOC_SUBDEV_* ioctl()s.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/mt9t112.c |   97 ++++++++++++++++++++++++++++++++++-------
 1 files changed, 81 insertions(+), 16 deletions(-)

diff --git a/drivers/media/video/mt9t112.c b/drivers/media/video/mt9t112.c
index 32114a3..bb95ad1 100644
--- a/drivers/media/video/mt9t112.c
+++ b/drivers/media/video/mt9t112.c
@@ -25,6 +25,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <linux/videodev2.h>
 
+#include <media/media-entity.h>
 #include <media/mt9t112.h>
 #include <media/soc_camera.h>
 #include <media/v4l2-chip-ident.h>
@@ -87,6 +88,7 @@ struct mt9t112_format {
 
 struct mt9t112_priv {
 	struct v4l2_subdev		 subdev;
+	struct media_pad		 pad;
 	struct mt9t112_camera_info	*info;
 	struct i2c_client		*client;
 	struct v4l2_rect		 frame;
@@ -739,8 +741,7 @@ static int mt9t112_init_camera(const struct i2c_client *client)
 static int mt9t112_g_chip_ident(struct v4l2_subdev *sd,
 				struct v4l2_dbg_chip_ident *id)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct mt9t112_priv *priv = to_mt9t112(client);
+	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv, subdev);
 
 	id->ident    = priv->model;
 	id->revision = 0;
@@ -790,7 +791,7 @@ static struct v4l2_subdev_core_ops mt9t112_subdev_core_ops = {
 static int mt9t112_s_stream(struct v4l2_subdev *sd, int enable)
 {
 	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct mt9t112_priv *priv = to_mt9t112(client);
+	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv, subdev);
 	int ret = 0;
 
 	if (!enable) {
@@ -888,8 +889,7 @@ static int mt9t112_cropcap(struct v4l2_subdev *sd, struct v4l2_cropcap *a)
 
 static int mt9t112_g_crop(struct v4l2_subdev *sd, struct v4l2_crop *a)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct mt9t112_priv *priv = to_mt9t112(client);
+	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv, subdev);
 
 	a->c	= priv->frame;
 	a->type	= V4L2_BUF_TYPE_VIDEO_CAPTURE;
@@ -899,8 +899,7 @@ static int mt9t112_g_crop(struct v4l2_subdev *sd, struct v4l2_crop *a)
 
 static int mt9t112_s_crop(struct v4l2_subdev *sd, struct v4l2_crop *a)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct mt9t112_priv *priv = to_mt9t112(client);
+	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv, subdev);
 	struct v4l2_rect *rect = &a->c;
 
 	return mt9t112_set_params(priv, rect, priv->format->code);
@@ -909,8 +908,7 @@ static int mt9t112_s_crop(struct v4l2_subdev *sd, struct v4l2_crop *a)
 static int mt9t112_g_fmt(struct v4l2_subdev *sd,
 			 struct v4l2_mbus_framefmt *mf)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct mt9t112_priv *priv = to_mt9t112(client);
+	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv, subdev);
 
 	mf->width	= priv->frame.width;
 	mf->height	= priv->frame.height;
@@ -924,8 +922,7 @@ static int mt9t112_g_fmt(struct v4l2_subdev *sd,
 static int mt9t112_s_fmt(struct v4l2_subdev *sd,
 			 struct v4l2_mbus_framefmt *mf)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct mt9t112_priv *priv = to_mt9t112(client);
+	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv, subdev);
 	struct v4l2_rect rect = {
 		.width = mf->width,
 		.height = mf->height,
@@ -996,8 +993,8 @@ static int mt9t112_s_mbus_config(struct v4l2_subdev *sd,
 				 const struct v4l2_mbus_config *cfg)
 {
 	struct i2c_client *client = v4l2_get_subdevdata(sd);
+	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv, subdev);
 	struct soc_camera_link *icl = soc_camera_i2c_to_link(client);
-	struct mt9t112_priv *priv = to_mt9t112(client);
 
 	if (soc_camera_apply_board_flags(icl, cfg) & V4L2_MBUS_PCLK_SAMPLE_RISING)
 		priv->flags |= PCLK_RISING;
@@ -1018,14 +1015,67 @@ static struct v4l2_subdev_video_ops mt9t112_subdev_video_ops = {
 	.s_mbus_config	= mt9t112_s_mbus_config,
 };
 
-/************************************************************************
-			i2c driver
-************************************************************************/
+static int mt9t112_enum_mbus_code(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+				  struct v4l2_subdev_mbus_code_enum *ce)
+{
+	if (ce->index >= ARRAY_SIZE(mt9t112_cfmts))
+		return -EINVAL;
+
+	ce->code = mt9t112_cfmts[ce->index].code;
+	return 0;
+}
+
+static int mt9t112_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+			   struct v4l2_subdev_format *sd_fmt)
+{
+	struct v4l2_mbus_framefmt *mf;
+
+	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE)
+		return mt9t112_g_fmt(sd, &sd_fmt->format);
+
+	mf = v4l2_subdev_get_try_format(fh, sd_fmt->pad);
+	sd_fmt->format = *mf;
+	return 0;
+}
+
+static int mt9t112_set_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+			   struct v4l2_subdev_format *sd_fmt)
+{
+	struct v4l2_mbus_framefmt *mf;
+
+	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE)
+		return mt9t112_s_fmt(sd, &sd_fmt->format);
+
+	mf = v4l2_subdev_get_try_format(fh, sd_fmt->pad);
+	*mf = sd_fmt->format;
+	return mt9t112_try_fmt(sd, mf);
+}
+
+struct v4l2_subdev_pad_ops mt9t112_subdev_pad_ops = {
+	.enum_mbus_code	= mt9t112_enum_mbus_code,
+	.get_fmt	= mt9t112_get_fmt,
+	.set_fmt	= mt9t112_set_fmt,
+};
+
 static struct v4l2_subdev_ops mt9t112_subdev_ops = {
 	.core	= &mt9t112_subdev_core_ops,
 	.video	= &mt9t112_subdev_video_ops,
+	.pad	= &mt9t112_subdev_pad_ops,
 };
 
+static int mt9t112_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
+{
+	struct v4l2_mbus_framefmt *mf = v4l2_subdev_get_try_format(fh, 0);
+	return mf ? mt9t112_try_fmt(sd, mf) : 0;
+}
+
+static const struct v4l2_subdev_internal_ops mt9t112_subdev_internal_ops = {
+	.open = mt9t112_open,
+};
+
+/************************************************************************
+			i2c driver
+************************************************************************/
 static int mt9t112_camera_probe(struct i2c_client *client)
 {
 	struct mt9t112_priv *priv = to_mt9t112(client);
@@ -1081,21 +1131,36 @@ static int mt9t112_probe(struct i2c_client *client,
 	priv->info = icl->priv;
 
 	v4l2_i2c_subdev_init(&priv->subdev, client, &mt9t112_subdev_ops);
+	priv->subdev.internal_ops = &mt9t112_subdev_internal_ops;
+	priv->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+	priv->pad.flags	= MEDIA_PAD_FL_SOURCE;
+	ret = subdev_media_entity_init(&priv->subdev, 1, &priv->pad, 0);
+	if (ret < 0)
+		goto emeinit;
 
 	ret = mt9t112_camera_probe(client);
 	if (ret)
-		kfree(priv);
+		goto evprobe;
 
 	/* Cannot fail: using the default supported pixel code */
 	mt9t112_set_params(priv, &rect, V4L2_MBUS_FMT_UYVY8_2X8);
 
 	return ret;
+
+evprobe:
+	subdev_media_entity_cleanup(&priv->subdev);
+emeinit:
+	kfree(priv);
+	return ret;
 }
 
 static int mt9t112_remove(struct i2c_client *client)
 {
 	struct mt9t112_priv *priv = to_mt9t112(client);
 
+	v4l2_device_unregister_subdev(&priv->subdev);
+	subdev_media_entity_cleanup(&priv->subdev);
 	kfree(priv);
 	return 0;
 }
-- 
1.7.2.5

[PATCH 9/9] V4L: imx074: add pad level operations

[Guennadi Liakhovetski]

On Media Controller enabled systems this patch allows the user to
communicate with the driver directly over /dev/v4l-subdev* device nodes
using VIDIOC_SUBDEV_* ioctl()s.

Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
---
 drivers/media/video/imx074.c |   85 +++++++++++++++++++++++++++++++++++++-----
 1 files changed, 75 insertions(+), 10 deletions(-)

diff --git a/drivers/media/video/imx074.c b/drivers/media/video/imx074.c
index 8775e26..9745887 100644
--- a/drivers/media/video/imx074.c
+++ b/drivers/media/video/imx074.c
@@ -16,6 +16,7 @@
 #include <linux/slab.h>
 #include <linux/videodev2.h>
 
+#include <media/media-entity.h>
 #include <media/soc_camera.h>
 #include <media/v4l2-subdev.h>
 #include <media/v4l2-chip-ident.h>
@@ -75,6 +76,7 @@ struct imx074_datafmt {
 
 struct imx074 {
 	struct v4l2_subdev		subdev;
+	struct media_pad		pad;
 	const struct imx074_datafmt	*fmt;
 };
 
@@ -172,8 +174,7 @@ static int imx074_try_fmt(struct v4l2_subdev *sd,
 static int imx074_s_fmt(struct v4l2_subdev *sd,
 			struct v4l2_mbus_framefmt *mf)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct imx074 *priv = to_imx074(client);
+	struct imx074 *priv = container_of(sd, struct imx074, subdev);
 
 	dev_dbg(sd->v4l2_dev->dev, "%s(%u)\n", __func__, mf->code);
 
@@ -191,9 +192,7 @@ static int imx074_s_fmt(struct v4l2_subdev *sd,
 static int imx074_g_fmt(struct v4l2_subdev *sd,
 			struct v4l2_mbus_framefmt *mf)
 {
-	struct i2c_client *client = v4l2_get_subdevdata(sd);
-	struct imx074 *priv = to_imx074(client);
-
+	struct imx074 *priv = container_of(sd, struct imx074, subdev);
 	const struct imx074_datafmt *fmt = priv->fmt;
 
 	mf->code	= fmt->code;
@@ -293,9 +292,62 @@ static struct v4l2_subdev_core_ops imx074_subdev_core_ops = {
 	.g_chip_ident	= imx074_g_chip_ident,
 };
 
+static int imx074_enum_mbus_code(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+				 struct v4l2_subdev_mbus_code_enum *ce)
+{
+	if (ce->index >= ARRAY_SIZE(imx074_colour_fmts))
+		return -EINVAL;
+
+	ce->code = imx074_colour_fmts[ce->index].code;
+	return 0;
+}
+
+static int imx074_get_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+			  struct v4l2_subdev_format *sd_fmt)
+{
+	struct v4l2_mbus_framefmt *mf;
+
+	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE)
+		return imx074_g_fmt(sd, &sd_fmt->format);
+
+	mf = v4l2_subdev_get_try_format(fh, sd_fmt->pad);
+	sd_fmt->format = *mf;
+	return 0;
+}
+
+static int imx074_set_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh,
+			  struct v4l2_subdev_format *sd_fmt)
+{
+	struct v4l2_mbus_framefmt *mf;
+
+	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE)
+		return imx074_s_fmt(sd, &sd_fmt->format);
+
+	mf = v4l2_subdev_get_try_format(fh, sd_fmt->pad);
+	*mf = sd_fmt->format;
+	return imx074_try_fmt(sd, mf);
+}
+
+struct v4l2_subdev_pad_ops imx074_subdev_pad_ops = {
+	.enum_mbus_code	= imx074_enum_mbus_code,
+	.get_fmt	= imx074_get_fmt,
+	.set_fmt	= imx074_set_fmt,
+};
+
 static struct v4l2_subdev_ops imx074_subdev_ops = {
 	.core	= &imx074_subdev_core_ops,
 	.video	= &imx074_subdev_video_ops,
+	.pad	= &imx074_subdev_pad_ops,
+};
+
+static int imx074_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
+{
+	struct v4l2_mbus_framefmt *mf = v4l2_subdev_get_try_format(fh, 0);
+	return mf ? imx074_try_fmt(sd, mf) : 0;
+}
+
+static const struct v4l2_subdev_internal_ops imx074_subdev_internal_ops = {
+	.open = imx074_open,
 };
 
 static int imx074_video_probe(struct i2c_client *client)
@@ -427,16 +479,27 @@ static int imx074_probe(struct i2c_client *client,
 	if (!priv)
 		return -ENOMEM;
 
+	priv->fmt = &imx074_colour_fmts[0];
+
 	v4l2_i2c_subdev_init(&priv->subdev, client, &imx074_subdev_ops);
+	priv->subdev.internal_ops = &imx074_subdev_internal_ops;
+	priv->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
 
-	priv->fmt	= &imx074_colour_fmts[0];
+	priv->pad.flags	= MEDIA_PAD_FL_SOURCE;
+	ret = subdev_media_entity_init(&priv->subdev, 1, &priv->pad, 0);
+	if (ret < 0)
+		goto emeinit;
 
 	ret = imx074_video_probe(client);
-	if (ret < 0) {
-		kfree(priv);
-		return ret;
-	}
+	if (ret < 0)
+		goto evprobe;
+
+	return ret;
 
+evprobe:
+	subdev_media_entity_cleanup(&priv->subdev);
+emeinit:
+	kfree(priv);
 	return ret;
 }
 
@@ -447,6 +510,8 @@ static int imx074_remove(struct i2c_client *client)
 
 	if (icl->free_bus)
 		icl->free_bus(icl);
+	v4l2_device_unregister_subdev(&priv->subdev);
+	subdev_media_entity_cleanup(&priv->subdev);
 	kfree(priv);
 
 	return 0;
-- 
1.7.2.5

Re: [PATCH 6/9] V4L: soc-camera: prepare hooks for Media Controller wrapper

[Laurent Pinchart]

Hi Guennadi,

Thanks for the patch. It's very nice to see you working on that :-)

I'm not a soc-camera expert, so my review is by no means extensive.

On Thursday 29 September 2011 18:18:54 Guennadi Liakhovetski wrote:

[snip]

> diff --git a/drivers/media/video/soc_camera.c
> b/drivers/media/video/soc_camera.c index 2905a88..790c14c 100644
> --- a/drivers/media/video/soc_camera.c
> +++ b/drivers/media/video/soc_camera.c

[snip]

> @@ -1361,9 +1402,11 @@ void soc_camera_host_unregister(struct
> soc_camera_host *ici) if (icd->iface == ici->nr &&
> to_soc_camera_control(icd))
>  			soc_camera_remove(icd);
> 
> -	mutex_unlock(&list_lock);
> +	soc_camera_mc_unregister(ici);
> 
>  	v4l2_device_unregister(&ici->v4l2_dev);
> +
> +	mutex_unlock(&list_lock);
>  }
>  EXPORT_SYMBOL(soc_camera_host_unregister);

Do soc_camera_mc_unregister() and v4l2_device_unregister() need to be 
protected by the mutex ?

> @@ -1443,7 +1486,6 @@ static int video_dev_create(struct soc_camera_device
> *icd)
> 
>  	strlcpy(vdev->name, ici->drv_name, sizeof(vdev->name));
> 
> -	vdev->parent		= icd->pdev;
>  	vdev->current_norm	= V4L2_STD_UNKNOWN;
>  	vdev->fops		= &soc_camera_fops;
>  	vdev->ioctl_ops		= &soc_camera_ioctl_ops;
> @@ -1451,6 +1493,8 @@ static int video_dev_create(struct soc_camera_device
> *icd) vdev->tvnorms		= V4L2_STD_UNKNOWN;
>  	vdev->ctrl_handler	= &icd->ctrl_handler;
>  	vdev->lock		= &icd->video_lock;
> +	vdev->v4l2_dev		= &ici->v4l2_dev;
> +	video_set_drvdata(vdev, icd);
> 
>  	icd->vdev = vdev;

This is an important change, maybe you can move it to a patch of its own.

> diff --git a/include/media/soc_camera.h b/include/media/soc_camera.h
> index d60bad4..0a21ff1 100644
> --- a/include/media/soc_camera.h
> +++ b/include/media/soc_camera.h

[snip]

> @@ -63,6 +65,18 @@ struct soc_camera_host {
>  	void *priv;
>  	const char *drv_name;
>  	struct soc_camera_host_ops *ops;
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> +	struct media_device mdev;
> +	struct v4l2_subdev bus_sd;
> +	struct media_pad bus_pads[2];
> +	struct media_pad vdev_pads[1];
> +#endif

Those fields are not used in this patch. Don't they belong to the next one ?

> +};
> +
> +enum soc_camera_target {
> +	SOCAM_TARGET_PIPELINE,
> +	SOCAM_TARGET_HOST_IN,
> +	SOCAM_TARGET_HOST_OUT,
>  };
> 
>  struct soc_camera_host_ops {

[snip]

> diff --git a/include/media/soc_entity.h b/include/media/soc_entity.h
> new file mode 100644
> index 0000000..e461f5e
> --- /dev/null
> +++ b/include/media/soc_entity.h
> @@ -0,0 +1,19 @@
> +/*
> + * soc-camera Media Controller wrapper
> + *
> + * Copyright (C) 2011, Guennadi Liakhovetski <g.liakhovetski@gmx.de>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +
> +#ifndef SOC_ENTITY_H
> +#define SOC_ENTITY_H
> +
> +#define soc_camera_mc_install(x) 0
> +#define soc_camera_mc_free(x) do {} while (0)
> +#define soc_camera_mc_register(x) do {} while (0)
> +#define soc_camera_mc_unregister(x) do {} while (0)

Doesn't this (and the corresponding changes to 
drivers/media/video/soc_camera.c) belong to the next patch ?

> +
> +#endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH 7/9] V4L: soc-camera: add a Media Controller wrapper

[Laurent Pinchart]

Hi Guennadi,

Thanks for the patch.

On Thursday 29 September 2011 18:18:55 Guennadi Liakhovetski wrote:
> This wrapper adds a Media Controller implementation to soc-camera drivers.
> To really benefit from it individual host drivers should implement support
> for values of enum soc_camera_target other than SOCAM_TARGET_PIPELINE in
> their .set_fmt() and .try_fmt() methods.

[snip]

> diff --git a/drivers/media/video/soc_entity.c
> b/drivers/media/video/soc_entity.c new file mode 100644
> index 0000000..3a04700
> --- /dev/null
> +++ b/drivers/media/video/soc_entity.c
> @@ -0,0 +1,284 @@

[snip]

> +static int bus_sd_pad_g_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh
> *fh,
> +			    struct v4l2_subdev_format *sd_fmt)
> +{
> +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> +	struct v4l2_mbus_framefmt *f = &sd_fmt->format;
> +
> +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> +		sd_fmt->format = *v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> +		return 0;
> +	}
> +
> +	if (sd_fmt->pad == SOC_HOST_BUS_PAD_SINK) {
> +		f->width	= icd->host_input_width;
> +		f->height	= icd->host_input_height;
> +	} else {
> +		f->width	= icd->user_width;
> +		f->height	= icd->user_height;
> +	}
> +	f->field	= icd->field;
> +	f->code		= icd->current_fmt->code;
> +	f->colorspace	= icd->colorspace;

Can soc-camera hosts perform format conversion ? If so you will likely need to 
store the mbus code for the input and output separately, possibly in 
v4l2_mbus_format fields. You could then simplify the [gs]_fmt functions by 
implementing similar to the __*_get_format functions in the OMAP3 ISP driver.

> +	return 0;
> +}
> +
> +static int bus_sd_pad_s_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh
> *fh,
> +			    struct v4l2_subdev_format *sd_fmt)
> +{
> +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> +	struct v4l2_mbus_framefmt *mf = &sd_fmt->format;
> +	struct v4l2_format vf = {
> +		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE,
> +	};
> +	enum soc_camera_target tgt = sd_fmt->pad == SOC_HOST_BUS_PAD_SINK ?
> +		SOCAM_TARGET_HOST_IN : SOCAM_TARGET_HOST_OUT;
> +	int ret;
> +
> +	se_mbus_to_v4l2(icd, mf, &vf);
> +
> +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> +		struct v4l2_mbus_framefmt *try_fmt =
> +			v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> +		ret = soc_camera_try_fmt(icd, &vf, tgt);
> +		if (!ret) {
> +			se_v4l2_to_mbus(icd, &vf, try_fmt);
> +			sd_fmt->format = *try_fmt;
> +		}
> +		return ret;
> +	}
> +
> +	ret = soc_camera_set_fmt(icd, &vf, tgt);
> +	if (!ret)
> +		se_v4l2_to_mbus(icd, &vf, &sd_fmt->format);
> +
> +	return ret;
> +}
> +
> +static int bus_sd_pad_enum_mbus_code(struct v4l2_subdev *sd,
> +				     struct v4l2_subdev_fh *fh,
> +				     struct v4l2_subdev_mbus_code_enum *ce)
> +{
> +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> +
> +	if (ce->index >= icd->num_user_formats)
> +		return -EINVAL;
> +
> +	ce->code = icd->user_formats[ce->index].code;
> +	return 0;
> +}
> +
> +static const struct v4l2_subdev_pad_ops se_bus_sd_pad_ops = {
> +	.get_fmt	= bus_sd_pad_g_fmt,
> +	.set_fmt	= bus_sd_pad_s_fmt,
> +	.enum_mbus_code	= bus_sd_pad_enum_mbus_code,
> +};
> +
> +static const struct v4l2_subdev_ops se_bus_sd_ops = {
> +	.pad		= &se_bus_sd_pad_ops,
> +};
> +
> +static const struct media_entity_operations se_bus_me_ops = {
> +};
> +
> +static const struct media_entity_operations se_vdev_me_ops = {
> +};

NULL operations are allowed, you don't have to use an empty structure.

> +
> +int soc_camera_mc_streamon(struct soc_camera_device *icd)
> +{
> +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> +	struct media_entity *bus_me = &bus_sd->entity;
> +	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
> +	struct v4l2_mbus_framefmt mf;
> +	int ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
> +	if (WARN_ON(ret < 0))
> +		return ret;
> +	if (icd->host_input_width != mf.width ||
> +	    icd->host_input_height != mf.height ||
> +	    icd->current_fmt->code != mf.code)
> +		return -EINVAL;

Shouldn't you also check that the source pad format matches the video node 
format ?

> +
> +	media_entity_pipeline_start(bus_me, &ici->pipe);
> +	return 0;
> +}
> +
> +void soc_camera_mc_streamoff(struct soc_camera_device *icd)
> +{
> +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> +	struct media_entity *bus_me = &bus_sd->entity;
> +	media_entity_pipeline_stop(bus_me);
> +}
> +
> +int soc_camera_mc_install(struct soc_camera_device *icd)
> +{
> +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> +	struct media_entity *bus_me = &bus_sd->entity;
> +	struct media_pad *bus_pads = ici->bus_pads;
> +	struct media_pad *vdev_pads = ici->vdev_pads;
> +	struct video_device *vdev = icd->vdev;
> +	struct media_entity *vdev_me = &vdev->entity;
> +	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
> +	int ret;
> +
> +	if (!ici->v4l2_dev.mdev || soc_entity_native_mc(icd))
> +		return 0;
> +
> +	/* Configure the video bus subdevice, entity, and pads */
> +	v4l2_subdev_init(bus_sd, &se_bus_sd_ops);
> +	v4l2_set_subdevdata(bus_sd, icd);
> +	bus_sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> +	snprintf(bus_sd->name, sizeof(bus_sd->name), "%s input", ici->drv_name);
> +
> +	bus_pads[SOC_HOST_BUS_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +	bus_pads[SOC_HOST_BUS_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +	bus_me->ops = &se_bus_me_ops;
> +
> +	ret = media_entity_init(bus_me, 2, bus_pads, 0);
> +	if (ret < 0)
> +		return ret;
> +
> +	/* Configure the video-device entity */
> +	vdev_pads[SOC_HOST_VDEV_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +	vdev_me->ops = &se_vdev_me_ops;
> +
> +	ret = media_entity_init(vdev_me, 1, vdev_pads, 0);
> +	if (ret < 0)
> +		goto evmei;
> +
> +	/* Link the two entities */
> +	ret = media_entity_create_link(bus_me, SOC_HOST_BUS_PAD_SOURCE,
> +				vdev_me, SOC_HOST_VDEV_PAD_SINK,
> +				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> +	if (ret < 0)
> +		goto elink;
> +
> +	ret = v4l2_device_register_subdev(&ici->v4l2_dev, bus_sd);
> +	if (ret < 0)
> +		goto eregsd;
> +
> +	ret = v4l2_device_register_subdev_nodes(&ici->v4l2_dev);
> +	if (ret < 0)
> +		goto eregsdn;
> +
> +	/*
> +	 * Link the client: make it immutable too for now, since there is no
> +	 * meaningful mapping for the .link_setup() method to the soc-camera
> +	 * API
> +	 */
> +	ret = media_entity_create_link(&sd->entity, 0,
> +				bus_me, SOC_HOST_BUS_PAD_SINK,
> +				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> +	if (ret < 0)
> +		goto eclink;

Qhat do you think about moving this above subdev registration ?

> +
> +	return 0;
> +
> +eclink:
> +eregsdn:
> +	v4l2_device_unregister_subdev(bus_sd);
> +eregsd:
> +elink:
> +	media_entity_cleanup(vdev_me);
> +evmei:
> +	media_entity_cleanup(bus_me);
> +
> +	return ret;
> +}
> +
> +void soc_camera_mc_free(struct soc_camera_device *icd)
> +{
> +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> +	struct media_entity *bus_me = &bus_sd->entity;
> +	struct video_device *vdev = icd->vdev;
> +	struct media_entity *vdev_me = &vdev->entity;
> +
> +	if (!ici->v4l2_dev.mdev || !soc_camera_to_subdev(icd)->ops->pad ||
> +	    soc_entity_native_mc(icd))
> +		return;
> +
> +	v4l2_device_unregister_subdev(bus_sd);
> +
> +	media_entity_cleanup(vdev_me);
> +	media_entity_cleanup(bus_me);
> +}
> +
> +void soc_camera_mc_register(struct soc_camera_host *ici)
> +{
> +	int ret;
> +
> +	/* The Big Moment: register the media device */
> +	ici->mdev.dev = ici->v4l2_dev.dev;
> +	ici->v4l2_dev.mdev = &ici->mdev;
> +	strlcpy(ici->mdev.model, ici->drv_name, sizeof(ici->mdev.model));
> +	ret = media_device_register(&ici->mdev);
> +	if (ret < 0)
> +		ici->v4l2_dev.mdev = NULL;
> +}
> +
> +void soc_camera_mc_unregister(struct soc_camera_host *ici)
> +{
> +	if (ici->v4l2_dev.mdev)
> +		media_device_unregister(&ici->mdev);
> +}
> diff --git a/drivers/media/video/soc_mediabus.c
> b/drivers/media/video/soc_mediabus.c index cf7f219..9f84c5c 100644
> --- a/drivers/media/video/soc_mediabus.c
> +++ b/drivers/media/video/soc_mediabus.c
> @@ -415,19 +415,3 @@ unsigned int soc_mbus_config_compatible(const struct
> v4l2_mbus_config *cfg, return 0;
>  }
>  EXPORT_SYMBOL(soc_mbus_config_compatible);
> -
> -static int __init soc_mbus_init(void)
> -{
> -	return 0;
> -}
> -
> -static void __exit soc_mbus_exit(void)
> -{
> -}
> -
> -module_init(soc_mbus_init);
> -module_exit(soc_mbus_exit);
> -
> -MODULE_DESCRIPTION("soc-camera media bus interface");
> -MODULE_AUTHOR("Guennadi Liakhovetski <g.liakhovetski@gmx.de>");
> -MODULE_LICENSE("GPL v2");
> diff --git a/include/media/soc_camera.h b/include/media/soc_camera.h
> index 0a21ff1..8b2b4ee 100644
> --- a/include/media/soc_camera.h
> +++ b/include/media/soc_camera.h
> @@ -70,6 +70,7 @@ struct soc_camera_host {
>  	struct v4l2_subdev bus_sd;
>  	struct media_pad bus_pads[2];
>  	struct media_pad vdev_pads[1];
> +	struct media_pipeline pipe;
>  #endif
>  };
> 
> diff --git a/include/media/soc_entity.h b/include/media/soc_entity.h
> index e461f5e..e4c692a 100644
> --- a/include/media/soc_entity.h
> +++ b/include/media/soc_entity.h
> @@ -11,9 +11,21 @@
>  #ifndef SOC_ENTITY_H
>  #define SOC_ENTITY_H
> 
> +#if defined(CONFIG_MEDIA_CONTROLLER)
> +struct soc_camera_device;
> +int soc_camera_mc_install(struct soc_camera_device *icd);
> +void soc_camera_mc_free(struct soc_camera_device *icd);
> +void soc_camera_mc_register(struct soc_camera_host *ici);
> +void soc_camera_mc_unregister(struct soc_camera_host *ici);
> +int soc_camera_mc_streamon(struct soc_camera_device *icd);
> +int soc_camera_mc_streamoff(struct soc_camera_device *icd);
> +#else
>  #define soc_camera_mc_install(x) 0
>  #define soc_camera_mc_free(x) do {} while (0)
>  #define soc_camera_mc_register(x) do {} while (0)
>  #define soc_camera_mc_unregister(x) do {} while (0)
> +#define soc_camera_mc_streamon(x) 0
> +#define soc_camera_mc_streamoff(x) do {} while (0)
> +#endif
> 
>  #endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH 8/9] V4L: mt9t112: add pad level operations

[Laurent Pinchart]

Hi Guennadi,

Thanks for the patch.

On Thursday 29 September 2011 18:18:56 Guennadi Liakhovetski wrote:
> On Media Controller enabled systems this patch allows the user to
> communicate with the driver directly over /dev/v4l-subdev* device nodes
> using VIDIOC_SUBDEV_* ioctl()s.
> 
> Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@gmx.de>

[snip]

> diff --git a/drivers/media/video/mt9t112.c b/drivers/media/video/mt9t112.c
> index 32114a3..bb95ad1 100644
> --- a/drivers/media/video/mt9t112.c
> +++ b/drivers/media/video/mt9t112.c

[snip]

> @@ -739,8 +741,7 @@ static int mt9t112_init_camera(const struct i2c_client
> *client) static int mt9t112_g_chip_ident(struct v4l2_subdev *sd,
>  				struct v4l2_dbg_chip_ident *id)
>  {
> -	struct i2c_client *client = v4l2_get_subdevdata(sd);
> -	struct mt9t112_priv *priv = to_mt9t112(client);
> +	struct mt9t112_priv *priv = container_of(sd, struct mt9t112_priv,
> subdev);

What about modifying to_mt9t112() to take a subdev pointer, or possibly 
creating a sd_to_mt9t112() ?

>  	id->ident    = priv->model;
>  	id->revision = 0;

[snip]

> @@ -1018,14 +1015,67 @@ static struct v4l2_subdev_video_ops

[snip]

> +static int mt9t112_set_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh
> *fh, +			   struct v4l2_subdev_format *sd_fmt)
> +{
> +	struct v4l2_mbus_framefmt *mf;
> +
> +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE)
> +		return mt9t112_s_fmt(sd, &sd_fmt->format);
> +
> +	mf = v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> +	*mf = sd_fmt->format;
> +	return mt9t112_try_fmt(sd, mf);

I think the code would be clea[nr]er if you split mt9t112_s_fmt() into try and 
set, and called try unconditionally in mt9t112_set_fmt().

> +}
> +
> +struct v4l2_subdev_pad_ops mt9t112_subdev_pad_ops = {
> +	.enum_mbus_code	= mt9t112_enum_mbus_code,
> +	.get_fmt	= mt9t112_get_fmt,
> +	.set_fmt	= mt9t112_set_fmt,

Having bot mt9t112_[gs]_fmt and mt9t112_[gs]et_fmt looks confusing to me. What 
about renaming the later mt9t112_[gs]et_pad_fmt ?

> +};
> +
>  static struct v4l2_subdev_ops mt9t112_subdev_ops = {
>  	.core	= &mt9t112_subdev_core_ops,
>  	.video	= &mt9t112_subdev_video_ops,
> +	.pad	= &mt9t112_subdev_pad_ops,
>  };
> 
> +static int mt9t112_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
> +{
> +	struct v4l2_mbus_framefmt *mf = v4l2_subdev_get_try_format(fh, 0);
> +	return mf ? mt9t112_try_fmt(sd, mf) : 0;

Can mf be NULL ?

> +}

-- 
Regards,

Laurent Pinchart

Re: [PATCH 0/9] Media Controller for soc-camera

[Laurent Pinchart]

Hi Guennadi,

Thanks for the patches. I'm glad to see soc-camera adopting the MC API :-)

On Thursday 29 September 2011 18:18:48 Guennadi Liakhovetski wrote:
> This is the first attempt at extending soc-camera with Media Controller /
> pad-level APIs. Yes, I know, that Laurent wasn't quite happy with "V4L:
> add convenience macros to the subdevice / Media Controller API," maybe
> we'll remove it eventually, but so far my patches use it, so, I kept it
> for now.

I'm fine with keeping it to allow the other patches to be reviewed already, 
but I still think we should drop it.

> The general idea has been described in
> 
> http://article.gmane.org/gmane.linux.drivers.video-input-infrastructure/380
> 83
> 
> In short: soc-camera implements a media controller device and two entities
> per camera host (bridge) instance, linked statically to each other and to
> the client. The host driver gets a chance to implement "local" only
> configuration, as opposed to the standard soc-camera way of propagating the
> configuration up the pipeline to the client (sensor / decoder) driver. An
> example implementation is provided for sh_mobile_ceu_camera and two sensor
> drivers. The whole machinery gets activated if the soc-camera core finds a
> client driver, that implements pad operations. In that case both the
> "standard" (V4L2) and the "new" (MC) ways of addressing the driver become
> available. I.e., it is possible to run both standard V4L2 applications and
> MC-aware ones.
>
> Of course, applies on top of
> 
> git://linuxtv.org/gliakhovetski/v4l-dvb.git for-3.2
> 
> Deepthy: this is what I told you about in
> 
> http://article.gmane.org/gmane.linux.ports.arm.omap/64847
> 
> it just took me a bit longer, than I thought.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 7/9] V4L: soc-camera: add a Media Controller wrapper

[Guennadi Liakhovetski]

Hi Laurent

Thanks for the reviews!

On Mon, 3 Oct 2011, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> Thanks for the patch.
> 
> On Thursday 29 September 2011 18:18:55 Guennadi Liakhovetski wrote:
> > This wrapper adds a Media Controller implementation to soc-camera drivers.
> > To really benefit from it individual host drivers should implement support
> > for values of enum soc_camera_target other than SOCAM_TARGET_PIPELINE in
> > their .set_fmt() and .try_fmt() methods.
> 
> [snip]
> 
> > diff --git a/drivers/media/video/soc_entity.c
> > b/drivers/media/video/soc_entity.c new file mode 100644
> > index 0000000..3a04700
> > --- /dev/null
> > +++ b/drivers/media/video/soc_entity.c
> > @@ -0,0 +1,284 @@
> 
> [snip]
> 
> > +static int bus_sd_pad_g_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh
> > *fh,
> > +			    struct v4l2_subdev_format *sd_fmt)
> > +{
> > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > +	struct v4l2_mbus_framefmt *f = &sd_fmt->format;
> > +
> > +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> > +		sd_fmt->format = *v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> > +		return 0;
> > +	}
> > +
> > +	if (sd_fmt->pad == SOC_HOST_BUS_PAD_SINK) {
> > +		f->width	= icd->host_input_width;
> > +		f->height	= icd->host_input_height;
> > +	} else {
> > +		f->width	= icd->user_width;
> > +		f->height	= icd->user_height;
> > +	}
> > +	f->field	= icd->field;
> > +	f->code		= icd->current_fmt->code;
> > +	f->colorspace	= icd->colorspace;
> 
> Can soc-camera hosts perform format conversion ? If so you will likely need to 
> store the mbus code for the input and output separately, possibly in 
> v4l2_mbus_format fields. You could then simplify the [gs]_fmt functions by 
> implementing similar to the __*_get_format functions in the OMAP3 ISP driver.

They can, yes. But, under soc-camera conversions are performed between 
mediabus codes and fourcc formats. Upon pipeline construction (probing) a 
table of format conversions is built, where hosts generate one or more 
translation entries for all client formats, that they support. The only 
example of a more complex translations so far is MIPI CSI-2, but even 
there we have decided to identify CSI-2 formats using the same media-bus 
codes, as what you "get" "between" the CSI-2 block and the DMA engine. For 
the only CSI-2 capable soc-camera host so far - the CEU driver - this is 
also a very natural representation, because there the CSI-2 block is 
indeed an additional pipeline stage, uniquely translating CSI-2 to 
media-bus codes, that are then fed to the CEU parallel port.

> 
> > +	return 0;
> > +}
> > +
> > +static int bus_sd_pad_s_fmt(struct v4l2_subdev *sd, struct v4l2_subdev_fh
> > *fh,
> > +			    struct v4l2_subdev_format *sd_fmt)
> > +{
> > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > +	struct v4l2_mbus_framefmt *mf = &sd_fmt->format;
> > +	struct v4l2_format vf = {
> > +		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE,
> > +	};
> > +	enum soc_camera_target tgt = sd_fmt->pad == SOC_HOST_BUS_PAD_SINK ?
> > +		SOCAM_TARGET_HOST_IN : SOCAM_TARGET_HOST_OUT;
> > +	int ret;
> > +
> > +	se_mbus_to_v4l2(icd, mf, &vf);
> > +
> > +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> > +		struct v4l2_mbus_framefmt *try_fmt =
> > +			v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> > +		ret = soc_camera_try_fmt(icd, &vf, tgt);
> > +		if (!ret) {
> > +			se_v4l2_to_mbus(icd, &vf, try_fmt);
> > +			sd_fmt->format = *try_fmt;
> > +		}
> > +		return ret;
> > +	}
> > +
> > +	ret = soc_camera_set_fmt(icd, &vf, tgt);
> > +	if (!ret)
> > +		se_v4l2_to_mbus(icd, &vf, &sd_fmt->format);
> > +
> > +	return ret;
> > +}
> > +
> > +static int bus_sd_pad_enum_mbus_code(struct v4l2_subdev *sd,
> > +				     struct v4l2_subdev_fh *fh,
> > +				     struct v4l2_subdev_mbus_code_enum *ce)
> > +{
> > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > +
> > +	if (ce->index >= icd->num_user_formats)
> > +		return -EINVAL;
> > +
> > +	ce->code = icd->user_formats[ce->index].code;
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_subdev_pad_ops se_bus_sd_pad_ops = {
> > +	.get_fmt	= bus_sd_pad_g_fmt,
> > +	.set_fmt	= bus_sd_pad_s_fmt,
> > +	.enum_mbus_code	= bus_sd_pad_enum_mbus_code,
> > +};
> > +
> > +static const struct v4l2_subdev_ops se_bus_sd_ops = {
> > +	.pad		= &se_bus_sd_pad_ops,
> > +};
> > +
> > +static const struct media_entity_operations se_bus_me_ops = {
> > +};
> > +
> > +static const struct media_entity_operations se_vdev_me_ops = {
> > +};
> 
> NULL operations are allowed, you don't have to use an empty structure.

Ok

> > +
> > +int soc_camera_mc_streamon(struct soc_camera_device *icd)
> > +{
> > +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> > +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> > +	struct media_entity *bus_me = &bus_sd->entity;
> > +	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
> > +	struct v4l2_mbus_framefmt mf;
> > +	int ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
> > +	if (WARN_ON(ret < 0))
> > +		return ret;
> > +	if (icd->host_input_width != mf.width ||
> > +	    icd->host_input_height != mf.height ||
> > +	    icd->current_fmt->code != mf.code)
> > +		return -EINVAL;
> 
> Shouldn't you also check that the source pad format matches the video node 
> format ?

I think, that's true by construction. It is already cheked in 
soc_camera_set_fmt():

	} else if (!icd->current_fmt ||
		   icd->current_fmt->host_fmt->fourcc != pix->pixelformat) {
		dev_err(icd->pdev,
			"Host driver hasn't set up current format correctly!\n");
		return -EINVAL;


> 
> > +
> > +	media_entity_pipeline_start(bus_me, &ici->pipe);
> > +	return 0;
> > +}
> > +
> > +void soc_camera_mc_streamoff(struct soc_camera_device *icd)
> > +{
> > +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> > +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> > +	struct media_entity *bus_me = &bus_sd->entity;
> > +	media_entity_pipeline_stop(bus_me);
> > +}
> > +
> > +int soc_camera_mc_install(struct soc_camera_device *icd)
> > +{
> > +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> > +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> > +	struct media_entity *bus_me = &bus_sd->entity;
> > +	struct media_pad *bus_pads = ici->bus_pads;
> > +	struct media_pad *vdev_pads = ici->vdev_pads;
> > +	struct video_device *vdev = icd->vdev;
> > +	struct media_entity *vdev_me = &vdev->entity;
> > +	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
> > +	int ret;
> > +
> > +	if (!ici->v4l2_dev.mdev || soc_entity_native_mc(icd))
> > +		return 0;
> > +
> > +	/* Configure the video bus subdevice, entity, and pads */
> > +	v4l2_subdev_init(bus_sd, &se_bus_sd_ops);
> > +	v4l2_set_subdevdata(bus_sd, icd);
> > +	bus_sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> > +	snprintf(bus_sd->name, sizeof(bus_sd->name), "%s input", ici->drv_name);
> > +
> > +	bus_pads[SOC_HOST_BUS_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +	bus_pads[SOC_HOST_BUS_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> > +	bus_me->ops = &se_bus_me_ops;
> > +
> > +	ret = media_entity_init(bus_me, 2, bus_pads, 0);
> > +	if (ret < 0)
> > +		return ret;
> > +
> > +	/* Configure the video-device entity */
> > +	vdev_pads[SOC_HOST_VDEV_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > +	vdev_me->ops = &se_vdev_me_ops;
> > +
> > +	ret = media_entity_init(vdev_me, 1, vdev_pads, 0);
> > +	if (ret < 0)
> > +		goto evmei;
> > +
> > +	/* Link the two entities */
> > +	ret = media_entity_create_link(bus_me, SOC_HOST_BUS_PAD_SOURCE,
> > +				vdev_me, SOC_HOST_VDEV_PAD_SINK,
> > +				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> > +	if (ret < 0)
> > +		goto elink;
> > +
> > +	ret = v4l2_device_register_subdev(&ici->v4l2_dev, bus_sd);
> > +	if (ret < 0)
> > +		goto eregsd;
> > +
> > +	ret = v4l2_device_register_subdev_nodes(&ici->v4l2_dev);
> > +	if (ret < 0)
> > +		goto eregsdn;
> > +
> > +	/*
> > +	 * Link the client: make it immutable too for now, since there is no
> > +	 * meaningful mapping for the .link_setup() method to the soc-camera
> > +	 * API
> > +	 */
> > +	ret = media_entity_create_link(&sd->entity, 0,
> > +				bus_me, SOC_HOST_BUS_PAD_SINK,
> > +				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> > +	if (ret < 0)
> > +		goto eclink;
> 
> Qhat do you think about moving this above subdev registration ?

I don't yet:-) But yes, perhaps, this can be done.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 7/9] V4L: soc-camera: add a Media Controller wrapper

[Laurent Pinchart]

Hi Guennadi,

On Monday 03 October 2011 17:29:23 Guennadi Liakhovetski wrote:
> Hi Laurent
> 
> Thanks for the reviews!

You're welcome.

> On Mon, 3 Oct 2011, Laurent Pinchart wrote:
> > On Thursday 29 September 2011 18:18:55 Guennadi Liakhovetski wrote:
> > > This wrapper adds a Media Controller implementation to soc-camera
> > > drivers. To really benefit from it individual host drivers should
> > > implement support for values of enum soc_camera_target other than
> > > SOCAM_TARGET_PIPELINE in their .set_fmt() and .try_fmt() methods.
> > 
> > [snip]
> > 
> > > diff --git a/drivers/media/video/soc_entity.c
> > > b/drivers/media/video/soc_entity.c new file mode 100644
> > > index 0000000..3a04700
> > > --- /dev/null
> > > +++ b/drivers/media/video/soc_entity.c
> > > @@ -0,0 +1,284 @@
> > 
> > [snip]
> > 
> > > +static int bus_sd_pad_g_fmt(struct v4l2_subdev *sd, struct
> > > v4l2_subdev_fh *fh,
> > > +			    struct v4l2_subdev_format *sd_fmt)
> > > +{
> > > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > > +	struct v4l2_mbus_framefmt *f = &sd_fmt->format;
> > > +
> > > +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> > > +		sd_fmt->format = *v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> > > +		return 0;
> > > +	}
> > > +
> > > +	if (sd_fmt->pad == SOC_HOST_BUS_PAD_SINK) {
> > > +		f->width	= icd->host_input_width;
> > > +		f->height	= icd->host_input_height;
> > > +	} else {
> > > +		f->width	= icd->user_width;
> > > +		f->height	= icd->user_height;
> > > +	}
> > > +	f->field	= icd->field;
> > > +	f->code		= icd->current_fmt->code;
> > > +	f->colorspace	= icd->colorspace;
> > 
> > Can soc-camera hosts perform format conversion ? If so you will likely
> > need to store the mbus code for the input and output separately,
> > possibly in v4l2_mbus_format fields. You could then simplify the
> > [gs]_fmt functions by implementing similar to the __*_get_format
> > functions in the OMAP3 ISP driver.
> 
> They can, yes. But, under soc-camera conversions are performed between
> mediabus codes and fourcc formats. Upon pipeline construction (probing) a
> table of format conversions is built, where hosts generate one or more
> translation entries for all client formats, that they support. The only
> example of a more complex translations so far is MIPI CSI-2, but even
> there we have decided to identify CSI-2 formats using the same media-bus
> codes, as what you "get" "between" the CSI-2 block and the DMA engine. For
> the only CSI-2 capable soc-camera host so far - the CEU driver - this is
> also a very natural representation, because there the CSI-2 block is
> indeed an additional pipeline stage, uniquely translating CSI-2 to
> media-bus codes, that are then fed to the CEU parallel port.

How does that work with the MC API then ? If the bridge can, let's say, 
convert between raw bayer and YUV, shouldn't the format at the bridge input be 
raw bayer and at the bridge output YUV ?

> > > +	return 0;
> > > +}
> > > +
> > > +static int bus_sd_pad_s_fmt(struct v4l2_subdev *sd, struct
> > > v4l2_subdev_fh *fh,
> > > +			    struct v4l2_subdev_format *sd_fmt)
> > > +{
> > > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > > +	struct v4l2_mbus_framefmt *mf = &sd_fmt->format;
> > > +	struct v4l2_format vf = {
> > > +		.type = V4L2_BUF_TYPE_VIDEO_CAPTURE,
> > > +	};
> > > +	enum soc_camera_target tgt = sd_fmt->pad == SOC_HOST_BUS_PAD_SINK ?
> > > +		SOCAM_TARGET_HOST_IN : SOCAM_TARGET_HOST_OUT;
> > > +	int ret;
> > > +
> > > +	se_mbus_to_v4l2(icd, mf, &vf);
> > > +
> > > +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> > > +		struct v4l2_mbus_framefmt *try_fmt =
> > > +			v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> > > +		ret = soc_camera_try_fmt(icd, &vf, tgt);
> > > +		if (!ret) {
> > > +			se_v4l2_to_mbus(icd, &vf, try_fmt);
> > > +			sd_fmt->format = *try_fmt;
> > > +		}
> > > +		return ret;
> > > +	}
> > > +
> > > +	ret = soc_camera_set_fmt(icd, &vf, tgt);
> > > +	if (!ret)
> > > +		se_v4l2_to_mbus(icd, &vf, &sd_fmt->format);
> > > +
> > > +	return ret;
> > > +}
> > > +
> > > +static int bus_sd_pad_enum_mbus_code(struct v4l2_subdev *sd,
> > > +				     struct v4l2_subdev_fh *fh,
> > > +				     struct v4l2_subdev_mbus_code_enum *ce)
> > > +{
> > > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > > +
> > > +	if (ce->index >= icd->num_user_formats)
> > > +		return -EINVAL;
> > > +
> > > +	ce->code = icd->user_formats[ce->index].code;
> > > +	return 0;
> > > +}
> > > +
> > > +static const struct v4l2_subdev_pad_ops se_bus_sd_pad_ops = {
> > > +	.get_fmt	= bus_sd_pad_g_fmt,
> > > +	.set_fmt	= bus_sd_pad_s_fmt,
> > > +	.enum_mbus_code	= bus_sd_pad_enum_mbus_code,
> > > +};
> > > +
> > > +static const struct v4l2_subdev_ops se_bus_sd_ops = {
> > > +	.pad		= &se_bus_sd_pad_ops,
> > > +};
> > > +
> > > +static const struct media_entity_operations se_bus_me_ops = {
> > > +};
> > > +
> > > +static const struct media_entity_operations se_vdev_me_ops = {
> > > +};
> > 
> > NULL operations are allowed, you don't have to use an empty structure.
> 
> Ok
> 
> > > +
> > > +int soc_camera_mc_streamon(struct soc_camera_device *icd)
> > > +{
> > > +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> > > +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> > > +	struct media_entity *bus_me = &bus_sd->entity;
> > > +	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
> > > +	struct v4l2_mbus_framefmt mf;
> > > +	int ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
> > > +	if (WARN_ON(ret < 0))
> > > +		return ret;
> > > +	if (icd->host_input_width != mf.width ||
> > > +	    icd->host_input_height != mf.height ||
> > > +	    icd->current_fmt->code != mf.code)
> > > +		return -EINVAL;
> > 
> > Shouldn't you also check that the source pad format matches the video
> > node format ?
> 
> I think, that's true by construction. It is already cheked in
> soc_camera_set_fmt():
> 
> 	} else if (!icd->current_fmt ||
> 		   icd->current_fmt->host_fmt->fourcc != pix->pixelformat) {
> 		dev_err(icd->pdev,
> 			"Host driver hasn't set up current format correctly!\n");
> 		return -EINVAL;

But aren't applications allowed to configure the format at the bridge output 
pad first ?

> > > +
> > > +	media_entity_pipeline_start(bus_me, &ici->pipe);
> > > +	return 0;
> > > +}
> > > +
> > > +void soc_camera_mc_streamoff(struct soc_camera_device *icd)
> > > +{
> > > +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> > > +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> > > +	struct media_entity *bus_me = &bus_sd->entity;
> > > +	media_entity_pipeline_stop(bus_me);
> > > +}
> > > +
> > > +int soc_camera_mc_install(struct soc_camera_device *icd)
> > > +{
> > > +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> > > +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> > > +	struct media_entity *bus_me = &bus_sd->entity;
> > > +	struct media_pad *bus_pads = ici->bus_pads;
> > > +	struct media_pad *vdev_pads = ici->vdev_pads;
> > > +	struct video_device *vdev = icd->vdev;
> > > +	struct media_entity *vdev_me = &vdev->entity;
> > > +	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
> > > +	int ret;
> > > +
> > > +	if (!ici->v4l2_dev.mdev || soc_entity_native_mc(icd))
> > > +		return 0;
> > > +
> > > +	/* Configure the video bus subdevice, entity, and pads */
> > > +	v4l2_subdev_init(bus_sd, &se_bus_sd_ops);
> > > +	v4l2_set_subdevdata(bus_sd, icd);
> > > +	bus_sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
> > > +	snprintf(bus_sd->name, sizeof(bus_sd->name), "%s input",
> > > ici->drv_name); +
> > > +	bus_pads[SOC_HOST_BUS_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > > +	bus_pads[SOC_HOST_BUS_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> > > +	bus_me->ops = &se_bus_me_ops;
> > > +
> > > +	ret = media_entity_init(bus_me, 2, bus_pads, 0);
> > > +	if (ret < 0)
> > > +		return ret;
> > > +
> > > +	/* Configure the video-device entity */
> > > +	vdev_pads[SOC_HOST_VDEV_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > > +	vdev_me->ops = &se_vdev_me_ops;
> > > +
> > > +	ret = media_entity_init(vdev_me, 1, vdev_pads, 0);
> > > +	if (ret < 0)
> > > +		goto evmei;
> > > +
> > > +	/* Link the two entities */
> > > +	ret = media_entity_create_link(bus_me, SOC_HOST_BUS_PAD_SOURCE,
> > > +				vdev_me, SOC_HOST_VDEV_PAD_SINK,
> > > +				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> > > +	if (ret < 0)
> > > +		goto elink;
> > > +
> > > +	ret = v4l2_device_register_subdev(&ici->v4l2_dev, bus_sd);
> > > +	if (ret < 0)
> > > +		goto eregsd;
> > > +
> > > +	ret = v4l2_device_register_subdev_nodes(&ici->v4l2_dev);
> > > +	if (ret < 0)
> > > +		goto eregsdn;
> > > +
> > > +	/*
> > > +	 * Link the client: make it immutable too for now, since there is no
> > > +	 * meaningful mapping for the .link_setup() method to the soc-camera
> > > +	 * API
> > > +	 */
> > > +	ret = media_entity_create_link(&sd->entity, 0,
> > > +				bus_me, SOC_HOST_BUS_PAD_SINK,
> > > +				MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> > > +	if (ret < 0)
> > > +		goto eclink;
> > 
> > Qhat do you think about moving this above subdev registration ?
> 
> I don't yet:-) But yes, perhaps, this can be done.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 7/9] V4L: soc-camera: add a Media Controller wrapper

[Guennadi Liakhovetski]

On Mon, 3 Oct 2011, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> On Monday 03 October 2011 17:29:23 Guennadi Liakhovetski wrote:
> > Hi Laurent
> > 
> > Thanks for the reviews!
> 
> You're welcome.
> 
> > On Mon, 3 Oct 2011, Laurent Pinchart wrote:
> > > On Thursday 29 September 2011 18:18:55 Guennadi Liakhovetski wrote:
> > > > This wrapper adds a Media Controller implementation to soc-camera
> > > > drivers. To really benefit from it individual host drivers should
> > > > implement support for values of enum soc_camera_target other than
> > > > SOCAM_TARGET_PIPELINE in their .set_fmt() and .try_fmt() methods.
> > > 
> > > [snip]
> > > 
> > > > diff --git a/drivers/media/video/soc_entity.c
> > > > b/drivers/media/video/soc_entity.c new file mode 100644
> > > > index 0000000..3a04700
> > > > --- /dev/null
> > > > +++ b/drivers/media/video/soc_entity.c
> > > > @@ -0,0 +1,284 @@
> > > 
> > > [snip]
> > > 
> > > > +static int bus_sd_pad_g_fmt(struct v4l2_subdev *sd, struct
> > > > v4l2_subdev_fh *fh,
> > > > +			    struct v4l2_subdev_format *sd_fmt)
> > > > +{
> > > > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > > > +	struct v4l2_mbus_framefmt *f = &sd_fmt->format;
> > > > +
> > > > +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> > > > +		sd_fmt->format = *v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> > > > +		return 0;
> > > > +	}
> > > > +
> > > > +	if (sd_fmt->pad == SOC_HOST_BUS_PAD_SINK) {
> > > > +		f->width	= icd->host_input_width;
> > > > +		f->height	= icd->host_input_height;
> > > > +	} else {
> > > > +		f->width	= icd->user_width;
> > > > +		f->height	= icd->user_height;
> > > > +	}
> > > > +	f->field	= icd->field;
> > > > +	f->code		= icd->current_fmt->code;
> > > > +	f->colorspace	= icd->colorspace;
> > > 
> > > Can soc-camera hosts perform format conversion ? If so you will likely
> > > need to store the mbus code for the input and output separately,
> > > possibly in v4l2_mbus_format fields. You could then simplify the
> > > [gs]_fmt functions by implementing similar to the __*_get_format
> > > functions in the OMAP3 ISP driver.
> > 
> > They can, yes. But, under soc-camera conversions are performed between
> > mediabus codes and fourcc formats. Upon pipeline construction (probing) a
> > table of format conversions is built, where hosts generate one or more
> > translation entries for all client formats, that they support. The only
> > example of a more complex translations so far is MIPI CSI-2, but even
> > there we have decided to identify CSI-2 formats using the same media-bus
> > codes, as what you "get" "between" the CSI-2 block and the DMA engine. For
> > the only CSI-2 capable soc-camera host so far - the CEU driver - this is
> > also a very natural representation, because there the CSI-2 block is
> > indeed an additional pipeline stage, uniquely translating CSI-2 to
> > media-bus codes, that are then fed to the CEU parallel port.
> 
> How does that work with the MC API then ? If the bridge can, let's say, 
> convert between raw bayer and YUV, shouldn't the format at the bridge input be 
> raw bayer and at the bridge output YUV ?

Doesn't it depend on your definition? I define the conversion as taking 
place on the "DMA-engine entity." I.e., a media-bus code is transferred 
unchanged all the way down to that entity and there it gets converted to 
one of fourcc formats for storage in the memory. Isn't what you are 
suggesting some kind of a t2o-stage conversion: first you convert one 
media-bus code to another one, then you convert the latter one to some 
fourcc, which is also not a one-to-one conversion.

[snip]

> > > > +int soc_camera_mc_streamon(struct soc_camera_device *icd)
> > > > +{
> > > > +	struct soc_camera_host *ici = to_soc_camera_host(icd->parent);
> > > > +	struct v4l2_subdev *bus_sd = &ici->bus_sd;
> > > > +	struct media_entity *bus_me = &bus_sd->entity;
> > > > +	struct v4l2_subdev *sd = soc_camera_to_subdev(icd);
> > > > +	struct v4l2_mbus_framefmt mf;
> > > > +	int ret = v4l2_subdev_call(sd, video, g_mbus_fmt, &mf);
> > > > +	if (WARN_ON(ret < 0))
> > > > +		return ret;
> > > > +	if (icd->host_input_width != mf.width ||
> > > > +	    icd->host_input_height != mf.height ||
> > > > +	    icd->current_fmt->code != mf.code)
> > > > +		return -EINVAL;
> > > 
> > > Shouldn't you also check that the source pad format matches the video
> > > node format ?
> > 
> > I think, that's true by construction. It is already cheked in
> > soc_camera_set_fmt():
> > 
> > 	} else if (!icd->current_fmt ||
> > 		   icd->current_fmt->host_fmt->fourcc != pix->pixelformat) {
> > 		dev_err(icd->pdev,
> > 			"Host driver hasn't set up current format correctly!\n");
> > 		return -EINVAL;
> 
> But aren't applications allowed to configure the format at the bridge output 
> pad first ?

They are. In either case an mbus -> fourcc translation is selected and the 
.current_fmt is filled.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/

Re: [PATCH 7/9] V4L: soc-camera: add a Media Controller wrapper

[Laurent Pinchart]

Hi Guennadi,

On Tuesday 04 October 2011 19:34:36 Guennadi Liakhovetski wrote:
> On Mon, 3 Oct 2011, Laurent Pinchart wrote:
> > On Monday 03 October 2011 17:29:23 Guennadi Liakhovetski wrote:
> > > On Mon, 3 Oct 2011, Laurent Pinchart wrote:
> > > > On Thursday 29 September 2011 18:18:55 Guennadi Liakhovetski wrote:
> > > > > This wrapper adds a Media Controller implementation to soc-camera
> > > > > drivers. To really benefit from it individual host drivers should
> > > > > implement support for values of enum soc_camera_target other than
> > > > > SOCAM_TARGET_PIPELINE in their .set_fmt() and .try_fmt() methods.
> > > > 
> > > > [snip]
> > > > 
> > > > > diff --git a/drivers/media/video/soc_entity.c
> > > > > b/drivers/media/video/soc_entity.c new file mode 100644
> > > > > index 0000000..3a04700
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/video/soc_entity.c
> > > > > @@ -0,0 +1,284 @@
> > > > 
> > > > [snip]
> > > > 
> > > > > +static int bus_sd_pad_g_fmt(struct v4l2_subdev *sd, struct
> > > > > v4l2_subdev_fh *fh,
> > > > > +			    struct v4l2_subdev_format *sd_fmt)
> > > > > +{
> > > > > +	struct soc_camera_device *icd = v4l2_get_subdevdata(sd);
> > > > > +	struct v4l2_mbus_framefmt *f = &sd_fmt->format;
> > > > > +
> > > > > +	if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
> > > > > +		sd_fmt->format = *v4l2_subdev_get_try_format(fh, sd_fmt->pad);
> > > > > +		return 0;
> > > > > +	}
> > > > > +
> > > > > +	if (sd_fmt->pad == SOC_HOST_BUS_PAD_SINK) {
> > > > > +		f->width	= icd->host_input_width;
> > > > > +		f->height	= icd->host_input_height;
> > > > > +	} else {
> > > > > +		f->width	= icd->user_width;
> > > > > +		f->height	= icd->user_height;
> > > > > +	}
> > > > > +	f->field	= icd->field;
> > > > > +	f->code		= icd->current_fmt->code;
> > > > > +	f->colorspace	= icd->colorspace;
> > > > 
> > > > Can soc-camera hosts perform format conversion ? If so you will
> > > > likely need to store the mbus code for the input and output
> > > > separately, possibly in v4l2_mbus_format fields. You could then
> > > > simplify the [gs]_fmt functions by implementing similar to the
> > > > __*_get_format functions in the OMAP3 ISP driver.
> > > 
> > > They can, yes. But, under soc-camera conversions are performed between
> > > mediabus codes and fourcc formats. Upon pipeline construction (probing)
> > > a table of format conversions is built, where hosts generate one or
> > > more translation entries for all client formats, that they support.
> > > The only example of a more complex translations so far is MIPI CSI-2,
> > > but even there we have decided to identify CSI-2 formats using the
> > > same media-bus codes, as what you "get" "between" the CSI-2 block and
> > > the DMA engine. For the only CSI-2 capable soc-camera host so far -
> > > the CEU driver - this is also a very natural representation, because
> > > there the CSI-2 block is indeed an additional pipeline stage, uniquely
> > > translating CSI-2 to media-bus codes, that are then fed to the CEU
> > > parallel port.
> > 
> > How does that work with the MC API then ? If the bridge can, let's say,
> > convert between raw bayer and YUV, shouldn't the format at the bridge
> > input be raw bayer and at the bridge output YUV ?
> 
> Doesn't it depend on your definition? I define the conversion as taking
> place on the "DMA-engine entity." I.e., a media-bus code is transferred
> unchanged all the way down to that entity and there it gets converted to
> one of fourcc formats for storage in the memory. Isn't what you are
> suggesting some kind of a t2o-stage conversion: first you convert one
> media-bus code to another one, then you convert the latter one to some
> fourcc, which is also not a one-to-one conversion.

t2o ?

An mbus code is converted to a fourcc by the DMA engine, but can't the bridge 
subdev also perform conversion, such as interpolating raw bayer to RGB, and 
possibly converting that to YUV ? In that case the bridge sink and source pads 
would have different mbus formats.

-- 
Regards,

Laurent Pinchart