[PATCH 0/8] media: vimc: remove media bus code limitation

[PATCH 0/8] media: vimc: remove media bus code limitation

[Helen Koike]

Hello,

This patch series has several vimc fixes (that I am sending in the same
series only for convenience, let me know if you prefer them to be
sent separately from the series).

The last commit removes the vimc_pix_map_list[] that was mapping
pixelformats with media bus formats, but it turns out they are not
1-to-1 equivalent and it is really painful to add other formats.
Also, for the userspace, media bus formats don't really matter as long
as they match between links. So this patch allows any media bus format
to be configured independently of the final expected pixelformat.

The series depends on
"[PATCH] media: Introduce helpers to fill pixel format structs "

Thanks,
Helen


Helen Koike (8):
  media: vimc: deb: fix default sink bayer format
  media: vimc: stream: fix thread state before sleep
  media: vimc: cap: fix step width/height in enum framesize
  media: v4l2-common: add bayer formats in v4l2_format_info
  media: vimc: stream: cleanup frame field from struct vimc_stream
  media: vimc: stream: add docs to struct vimc_stream
  media: vimc: stream: init/terminate the first entity
  media: vimc: propagate pixel format in the stream

 drivers/media/platform/vimc/vimc-capture.c  |  80 +++--
 drivers/media/platform/vimc/vimc-common.c   | 307 ++++++++------------
 drivers/media/platform/vimc/vimc-common.h   |  13 +
 drivers/media/platform/vimc/vimc-debayer.c  |  80 +++--
 drivers/media/platform/vimc/vimc-scaler.c   |  60 ++--
 drivers/media/platform/vimc/vimc-sensor.c   |  48 +--
 drivers/media/platform/vimc/vimc-streamer.c |  39 +--
 drivers/media/platform/vimc/vimc-streamer.h |  22 +-
 drivers/media/v4l2-core/v4l2-common.c       |  22 ++
 9 files changed, 341 insertions(+), 330 deletions(-)

-- 
2.20.1

[PATCH 1/8] media: vimc: deb: fix default sink bayer format

[Helen Koike]

The format of the sink pad should be a bayer mbus format.

This fixes a kernel NULL pointer dereference error that was caused when
the stream starts because the configured format was not found in the
pixelmap table.

Reported-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

 drivers/media/platform/vimc/vimc-debayer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
index eaed4233ad1b..20826f209731 100644
--- a/drivers/media/platform/vimc/vimc-debayer.c
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -66,7 +66,7 @@ struct vimc_deb_device {
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,
-	.code = MEDIA_BUS_FMT_RGB888_1X24,
+	.code = MEDIA_BUS_FMT_SRGGB8_1X8,
 	.field = V4L2_FIELD_NONE,
 	.colorspace = V4L2_COLORSPACE_DEFAULT,
 };
-- 
2.20.1

[PATCH 2/8] media: vimc: stream: fix thread state before sleep

[Helen Koike]

The state TASK_UNINTERRUPTIBLE should be set just before
schedule_timeout() call, so it knows the sleep mode it should enter.
There is no point in setting TASK_UNINTERRUPTIBLE at the initialization
of the thread as schedule_timeout() will set the state back to
TASK_RUNNING.

This fixes a warning in __might_sleep() call, as it's expecting the
task to be in TASK_RUNNING state just before changing the state to
a sleeping state.

Reported-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

 drivers/media/platform/vimc/vimc-streamer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
index fcc897fb247b..392754c18046 100644
--- a/drivers/media/platform/vimc/vimc-streamer.c
+++ b/drivers/media/platform/vimc/vimc-streamer.c
@@ -120,7 +120,6 @@ static int vimc_streamer_thread(void *data)
 	int i;
 
 	set_freezable();
-	set_current_state(TASK_UNINTERRUPTIBLE);
 
 	for (;;) {
 		try_to_freeze();
@@ -137,6 +136,7 @@ static int vimc_streamer_thread(void *data)
 				break;
 		}
 		//wait for 60hz
+		set_current_state(TASK_UNINTERRUPTIBLE);
 		schedule_timeout(HZ / 60);
 	}
 
-- 
2.20.1

[PATCH 3/8] media: vimc: cap: fix step width/height in enum framesize

[Helen Koike]

The type V4L2_FRMSIZE_TYPE_CONTINUOUS expects a step of 1.
This fixes v4l2-compliance test error:

        fail: v4l2-test-formats.cpp(184): invalid step_width/height for continuous framesize
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL

Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

 drivers/media/platform/vimc/vimc-capture.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
index 3d433361d297..e976a9d6b460 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -187,8 +187,8 @@ static int vimc_cap_enum_framesizes(struct file *file, void *fh,
 	fsize->stepwise.max_width = VIMC_FRAME_MAX_WIDTH;
 	fsize->stepwise.min_height = VIMC_FRAME_MIN_HEIGHT;
 	fsize->stepwise.max_height = VIMC_FRAME_MAX_HEIGHT;
-	fsize->stepwise.step_width = 2;
-	fsize->stepwise.step_height = 2;
+	fsize->stepwise.step_width = 1;
+	fsize->stepwise.step_height = 1;
 
 	return 0;
 }
-- 
2.20.1

[PATCH 4/8] media: v4l2-common: add bayer formats in v4l2_format_info

[Helen Koike]

Add bayer format information in struct v4l2_format_info table.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

 drivers/media/v4l2-core/v4l2-common.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c
index 11a16bb3efda..779e44d6db43 100644
--- a/drivers/media/v4l2-core/v4l2-common.c
+++ b/drivers/media/v4l2-core/v4l2-common.c
@@ -517,6 +517,28 @@ const struct v4l2_format_info *v4l2_format_info(u32 format)
 		{ .format = V4L2_PIX_FMT_NV21M,   .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 2 },
 		{ .format = V4L2_PIX_FMT_NV16M,   .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
 		{ .format = V4L2_PIX_FMT_NV61M,   .mem_planes = 2, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 },
+
+		/* Bayer RGB formats */
+		{ .format = V4L2_PIX_FMT_SBGGR8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGBRG8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGRBG8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SRGGB8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SBGGR10,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGBRG10,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGRBG10,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SRGGB10,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SBGGR10ALAW8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGBRG10ALAW8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGRBG10ALAW8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SRGGB10ALAW8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SBGGR10DPCM8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGBRG10DPCM8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGRBG10DPCM8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SRGGB10DPCM8,	.mem_planes = 1, .comp_planes = 1, .bpp = { 1, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SBGGR12,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGBRG12,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SGRBG12,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
+		{ .format = V4L2_PIX_FMT_SRGGB12,	.mem_planes = 1, .comp_planes = 1, .bpp = { 2, 0, 0, 0 }, .hdiv = 1, .vdiv = 1 },
 	};
 	unsigned int i;
 
-- 
2.20.1

[PATCH 5/8] media: vimc: stream: cleanup frame field from struct vimc_stream

[Helen Koike]

There is no need to have the frame field in the vimc_stream struct.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

 drivers/media/platform/vimc/vimc-streamer.c | 10 ++++------
 drivers/media/platform/vimc/vimc-streamer.h |  1 -
 2 files changed, 4 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
index 392754c18046..b7c1fdef5f0d 100644
--- a/drivers/media/platform/vimc/vimc-streamer.c
+++ b/drivers/media/platform/vimc/vimc-streamer.c
@@ -117,6 +117,7 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
 static int vimc_streamer_thread(void *data)
 {
 	struct vimc_stream *stream = data;
+	u8 *frame = NULL;
 	int i;
 
 	set_freezable();
@@ -127,12 +128,9 @@ static int vimc_streamer_thread(void *data)
 			break;
 
 		for (i = stream->pipe_size - 1; i >= 0; i--) {
-			stream->frame = stream->ved_pipeline[i]->process_frame(
-					stream->ved_pipeline[i],
-					stream->frame);
-			if (!stream->frame)
-				break;
-			if (IS_ERR(stream->frame))
+			frame = stream->ved_pipeline[i]->process_frame(
+					stream->ved_pipeline[i], frame);
+			if (!frame || IS_ERR(frame))
 				break;
 		}
 		//wait for 60hz
diff --git a/drivers/media/platform/vimc/vimc-streamer.h b/drivers/media/platform/vimc/vimc-streamer.h
index 752af2e2d5a2..dc1d0be431cb 100644
--- a/drivers/media/platform/vimc/vimc-streamer.h
+++ b/drivers/media/platform/vimc/vimc-streamer.h
@@ -19,7 +19,6 @@ struct vimc_stream {
 	struct media_pipeline pipe;
 	struct vimc_ent_device *ved_pipeline[VIMC_STREAMER_PIPELINE_MAX_SIZE];
 	unsigned int pipe_size;
-	u8 *frame;
 	struct task_struct *kthread;
 };
 
-- 
2.20.1

[PATCH 6/8] media: vimc: stream: add docs to struct vimc_stream

[Helen Koike]

Add missing documentation for struct vimc_stream

Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

 drivers/media/platform/vimc/vimc-streamer.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/media/platform/vimc/vimc-streamer.h b/drivers/media/platform/vimc/vimc-streamer.h
index dc1d0be431cb..a7c5ac5ace4f 100644
--- a/drivers/media/platform/vimc/vimc-streamer.h
+++ b/drivers/media/platform/vimc/vimc-streamer.h
@@ -15,6 +15,21 @@
 
 #define VIMC_STREAMER_PIPELINE_MAX_SIZE 16
 
+/**
+ * struct vimc_stream - struct that represents a stream in the pipeline
+ *
+ * @pipe:		the media pipeline object associated with this stream
+ * @ved_pipeline:	array containing all the entities participating in the
+ * stream. The order is from a video device (usually a capture device) where
+ * stream_on was called, to the entity generating the first base image to be
+ * processed in the pipeline.
+ * @pipe_size:		size of @ved_pipeline
+ * @kthread:		thread that generates the frames of the stream.
+ *
+ * When the user call stream_on in a video device, struct vimc_stream is
+ * used to keep track of all entities and subdevices that generates and
+ * process frames for the stream.
+ */
 struct vimc_stream {
 	struct media_pipeline pipe;
 	struct vimc_ent_device *ved_pipeline[VIMC_STREAMER_PIPELINE_MAX_SIZE];
-- 
2.20.1

[PATCH 7/8] media: vimc: stream: init/terminate the first entity

[Helen Koike]

The s_stream callback was not being called for the first entity in the
stream pipeline array.
Instead of verifying the type of the node (video or subdevice) and
calling s_stream from the second entity in the pipeline, do this process
for all the entities in the pipeline for consistency.

The previous code was not a problem because the first entity is a video
device and not a subdevice, but this patch prepares vimc to allow
setting some configuration in the entity before calling s_stream.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
---

 drivers/media/platform/vimc/vimc-streamer.c | 25 ++++++++++++---------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
index b7c1fdef5f0d..5a3bda62fbc8 100644
--- a/drivers/media/platform/vimc/vimc-streamer.c
+++ b/drivers/media/platform/vimc/vimc-streamer.c
@@ -46,19 +46,18 @@ static struct media_entity *vimc_get_source_entity(struct media_entity *ent)
  */
 static void vimc_streamer_pipeline_terminate(struct vimc_stream *stream)
 {
-	struct media_entity *entity;
+	struct vimc_ent_device *ved;
 	struct v4l2_subdev *sd;
 
 	while (stream->pipe_size) {
 		stream->pipe_size--;
-		entity = stream->ved_pipeline[stream->pipe_size]->ent;
-		entity = vimc_get_source_entity(entity);
+		ved = stream->ved_pipeline[stream->pipe_size];
 		stream->ved_pipeline[stream->pipe_size] = NULL;
 
-		if (!is_media_entity_v4l2_subdev(entity))
+		if (!is_media_entity_v4l2_subdev(ved->ent))
 			continue;
 
-		sd = media_entity_to_v4l2_subdev(entity);
+		sd = media_entity_to_v4l2_subdev(ved->ent);
 		v4l2_subdev_call(sd, video, s_stream, 0);
 	}
 }
@@ -89,18 +88,24 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
 		}
 		stream->ved_pipeline[stream->pipe_size++] = ved;
 
+		if (is_media_entity_v4l2_subdev(ved->ent)) {
+			sd = media_entity_to_v4l2_subdev(ved->ent);
+			ret = v4l2_subdev_call(sd, video, s_stream, 1);
+			if (ret && ret != -ENOIOCTLCMD) {
+				pr_err("subdev_call error %s\n", ved->ent->name);
+				vimc_streamer_pipeline_terminate(stream);
+				return ret;
+			}
+		}
+
 		entity = vimc_get_source_entity(ved->ent);
 		/* Check if the end of the pipeline was reached*/
 		if (!entity)
 			return 0;
 
+		/* Get the next device in the pipeline */
 		if (is_media_entity_v4l2_subdev(entity)) {
 			sd = media_entity_to_v4l2_subdev(entity);
-			ret = v4l2_subdev_call(sd, video, s_stream, 1);
-			if (ret && ret != -ENOIOCTLCMD) {
-				vimc_streamer_pipeline_terminate(stream);
-				return ret;
-			}
 			ved = v4l2_get_subdevdata(sd);
 		} else {
 			vdev = container_of(entity,
-- 
2.20.1

[PATCH 8/8] media: vimc: propagate pixel format in the stream

[Helen Koike]

Media bus codes were being mapped to pixelformats, which causes a
limitation on vimc because not all pixelformats can be mapped to media
bus codes.
Also, media bus codes are an internal configuration from the device.
Userspace only assures media bus codes matches between pads and expects
the image in a given pixelformat. So we can allow almost any media bus
format to be configured between pads, except for debayer that expects a
media bus code of type bayer in the sink pad.

Signed-off-by: Helen Koike <helen.koike@collabora.com>

---

 drivers/media/platform/vimc/vimc-capture.c  |  76 +++--
 drivers/media/platform/vimc/vimc-common.c   | 307 ++++++++------------
 drivers/media/platform/vimc/vimc-common.h   |  13 +
 drivers/media/platform/vimc/vimc-debayer.c  |  78 +++--
 drivers/media/platform/vimc/vimc-scaler.c   |  60 ++--
 drivers/media/platform/vimc/vimc-sensor.c   |  48 +--
 drivers/media/platform/vimc/vimc-streamer.c |   2 +
 drivers/media/platform/vimc/vimc-streamer.h |   6 +
 8 files changed, 281 insertions(+), 309 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
index e976a9d6b460..6377974879d7 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -28,6 +28,32 @@
 
 #define VIMC_CAP_DRV_NAME "vimc-capture"
 
+static const u32 vimc_cap_supported_pixftm[] = {
+	V4L2_PIX_FMT_BGR24,
+	V4L2_PIX_FMT_RGB24,
+	V4L2_PIX_FMT_ARGB32,
+	V4L2_PIX_FMT_SBGGR8,
+	V4L2_PIX_FMT_SGBRG8,
+	V4L2_PIX_FMT_SGRBG8,
+	V4L2_PIX_FMT_SRGGB8,
+	V4L2_PIX_FMT_SBGGR10,
+	V4L2_PIX_FMT_SGBRG10,
+	V4L2_PIX_FMT_SGRBG10,
+	V4L2_PIX_FMT_SRGGB10,
+	V4L2_PIX_FMT_SBGGR10ALAW8,
+	V4L2_PIX_FMT_SGBRG10ALAW8,
+	V4L2_PIX_FMT_SGRBG10ALAW8,
+	V4L2_PIX_FMT_SRGGB10ALAW8,
+	V4L2_PIX_FMT_SBGGR10DPCM8,
+	V4L2_PIX_FMT_SGBRG10DPCM8,
+	V4L2_PIX_FMT_SGRBG10DPCM8,
+	V4L2_PIX_FMT_SRGGB10DPCM8,
+	V4L2_PIX_FMT_SBGGR12,
+	V4L2_PIX_FMT_SGBRG12,
+	V4L2_PIX_FMT_SGRBG12,
+	V4L2_PIX_FMT_SRGGB12,
+};
+
 struct vimc_cap_device {
 	struct vimc_ent_device ved;
 	struct video_device vdev;
@@ -101,29 +127,25 @@ static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
 				    struct v4l2_format *f)
 {
 	struct v4l2_pix_format *format = &f->fmt.pix;
-	const struct vimc_pix_map *vpix;
 
 	format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
 				VIMC_FRAME_MAX_WIDTH) & ~1;
 	format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
 				 VIMC_FRAME_MAX_HEIGHT) & ~1;
 
-	/* Don't accept a pixelformat that is not on the table */
-	vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
-	if (!vpix) {
-		format->pixelformat = fmt_default.pixelformat;
-		vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
-	}
-	/* TODO: Add support for custom bytesperline values */
-	format->bytesperline = format->width * vpix->bpp;
-	format->sizeimage = format->bytesperline * format->height;
+	vimc_colorimetry_clamp(format);
 
 	if (format->field == V4L2_FIELD_ANY)
 		format->field = fmt_default.field;
 
-	vimc_colorimetry_clamp(format);
+	/* TODO: Add support for custom bytesperline values */
 
-	return 0;
+	/* Don't accept a pixelformat that is not on the table */
+	if (!v4l2_format_info(format->pixelformat))
+		format->pixelformat = fmt_default.pixelformat;
+
+	return v4l2_fill_pixfmt(format, format->pixelformat,
+				format->width, format->height);
 }
 
 static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
@@ -159,27 +181,31 @@ static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
 static int vimc_cap_enum_fmt_vid_cap(struct file *file, void *priv,
 				     struct v4l2_fmtdesc *f)
 {
-	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(f->index);
-
-	if (!vpix)
+	if (f->index >= ARRAY_SIZE(vimc_cap_supported_pixftm))
 		return -EINVAL;
 
-	f->pixelformat = vpix->pixelformat;
+	f->pixelformat = vimc_cap_supported_pixftm[f->index];
 
 	return 0;
 }
 
+static bool vimc_cap_is_pixfmt_supported(u32 pixelformat)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(vimc_cap_supported_pixftm); i++)
+		if (vimc_cap_supported_pixftm[i] == pixelformat)
+			return true;
+	return false;
+}
+
 static int vimc_cap_enum_framesizes(struct file *file, void *fh,
 				    struct v4l2_frmsizeenum *fsize)
 {
-	const struct vimc_pix_map *vpix;
-
 	if (fsize->index)
 		return -EINVAL;
 
-	/* Only accept code in the pix map table */
-	vpix = vimc_pix_map_by_code(fsize->pixel_format);
-	if (!vpix)
+	if (!vimc_cap_is_pixfmt_supported(fsize->pixel_format))
 		return -EINVAL;
 
 	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
@@ -253,6 +279,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 		return ret;
 	}
 
+	vcap->stream.producer_pixfmt = vcap->format.pixelformat;
 	ret = vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 1);
 	if (ret) {
 		media_pipeline_stop(entity);
@@ -403,7 +430,6 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 {
 	struct v4l2_device *v4l2_dev = master_data;
 	struct vimc_platform_data *pdata = comp->platform_data;
-	const struct vimc_pix_map *vpix;
 	struct vimc_cap_device *vcap;
 	struct video_device *vdev;
 	struct vb2_queue *q;
@@ -458,10 +484,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
 	/* Set default frame format */
 	vcap->format = fmt_default;
-	vpix = vimc_pix_map_by_pixelformat(vcap->format.pixelformat);
-	vcap->format.bytesperline = vcap->format.width * vpix->bpp;
-	vcap->format.sizeimage = vcap->format.bytesperline *
-				 vcap->format.height;
+	v4l2_fill_pixfmt(&vcap->format, vcap->format.pixelformat,
+			 vcap->format.width, vcap->format.height);
 
 	/* Fill the vimc_ent_device struct */
 	vcap->ved.ent = &vcap->vdev.entity;
diff --git a/drivers/media/platform/vimc/vimc-common.c b/drivers/media/platform/vimc/vimc-common.c
index 0adbfd8fd26d..b07bbf0564b9 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -20,192 +20,127 @@
 
 #include "vimc-common.h"
 
-/*
- * NOTE: non-bayer formats need to come first (necessary for enum_mbus_code
- * in the scaler)
- */
-static const struct vimc_pix_map vimc_pix_map_list[] = {
-	/* TODO: add all missing formats */
-
-	/* RGB formats */
-	{
-		.code = MEDIA_BUS_FMT_BGR888_1X24,
-		.pixelformat = V4L2_PIX_FMT_BGR24,
-		.bpp = 3,
-		.bayer = false,
-	},
-	{
-		.code = MEDIA_BUS_FMT_RGB888_1X24,
-		.pixelformat = V4L2_PIX_FMT_RGB24,
-		.bpp = 3,
-		.bayer = false,
-	},
-	{
-		.code = MEDIA_BUS_FMT_ARGB8888_1X32,
-		.pixelformat = V4L2_PIX_FMT_ARGB32,
-		.bpp = 4,
-		.bayer = false,
-	},
-
-	/* Bayer formats */
-	{
-		.code = MEDIA_BUS_FMT_SBGGR8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SBGGR8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGBRG8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SGBRG8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGRBG8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SGRBG8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SRGGB8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SRGGB8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SBGGR10_1X10,
-		.pixelformat = V4L2_PIX_FMT_SBGGR10,
-		.bpp = 2,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGBRG10_1X10,
-		.pixelformat = V4L2_PIX_FMT_SGBRG10,
-		.bpp = 2,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGRBG10_1X10,
-		.pixelformat = V4L2_PIX_FMT_SGRBG10,
-		.bpp = 2,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SRGGB10_1X10,
-		.pixelformat = V4L2_PIX_FMT_SRGGB10,
-		.bpp = 2,
-		.bayer = true,
-	},
-
-	/* 10bit raw bayer a-law compressed to 8 bits */
-	{
-		.code = MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SBGGR10ALAW8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SGBRG10ALAW8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SGRBG10ALAW8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SRGGB10_ALAW8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SRGGB10ALAW8,
-		.bpp = 1,
-		.bayer = true,
-	},
-
-	/* 10bit raw bayer DPCM compressed to 8 bits */
-	{
-		.code = MEDIA_BUS_FMT_SBGGR10_DPCM8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SBGGR10DPCM8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGBRG10_DPCM8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SGBRG10DPCM8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGRBG10_DPCM8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SGRBG10DPCM8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SRGGB10_DPCM8_1X8,
-		.pixelformat = V4L2_PIX_FMT_SRGGB10DPCM8,
-		.bpp = 1,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SBGGR12_1X12,
-		.pixelformat = V4L2_PIX_FMT_SBGGR12,
-		.bpp = 2,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGBRG12_1X12,
-		.pixelformat = V4L2_PIX_FMT_SGBRG12,
-		.bpp = 2,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SGRBG12_1X12,
-		.pixelformat = V4L2_PIX_FMT_SGRBG12,
-		.bpp = 2,
-		.bayer = true,
-	},
-	{
-		.code = MEDIA_BUS_FMT_SRGGB12_1X12,
-		.pixelformat = V4L2_PIX_FMT_SRGGB12,
-		.bpp = 2,
-		.bayer = true,
-	},
+const u32 vimc_mbus_list[] = {
+	MEDIA_BUS_FMT_FIXED,
+	MEDIA_BUS_FMT_RGB444_1X12,
+	MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE,
+	MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE,
+	MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
+	MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
+	MEDIA_BUS_FMT_RGB565_1X16,
+	MEDIA_BUS_FMT_BGR565_2X8_BE,
+	MEDIA_BUS_FMT_BGR565_2X8_LE,
+	MEDIA_BUS_FMT_RGB565_2X8_BE,
+	MEDIA_BUS_FMT_RGB565_2X8_LE,
+	MEDIA_BUS_FMT_RGB666_1X18,
+	MEDIA_BUS_FMT_RBG888_1X24,
+	MEDIA_BUS_FMT_RGB666_1X24_CPADHI,
+	MEDIA_BUS_FMT_RGB666_1X7X3_SPWG,
+	MEDIA_BUS_FMT_BGR888_1X24,
+	MEDIA_BUS_FMT_GBR888_1X24,
+	MEDIA_BUS_FMT_RGB888_1X24,
+	MEDIA_BUS_FMT_RGB888_2X12_BE,
+	MEDIA_BUS_FMT_RGB888_2X12_LE,
+	MEDIA_BUS_FMT_RGB888_1X7X4_SPWG,
+	MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA,
+	MEDIA_BUS_FMT_ARGB8888_1X32,
+	MEDIA_BUS_FMT_RGB888_1X32_PADHI,
+	MEDIA_BUS_FMT_RGB101010_1X30,
+	MEDIA_BUS_FMT_RGB121212_1X36,
+	MEDIA_BUS_FMT_RGB161616_1X48,
+	MEDIA_BUS_FMT_Y8_1X8,
+	MEDIA_BUS_FMT_UV8_1X8,
+	MEDIA_BUS_FMT_UYVY8_1_5X8,
+	MEDIA_BUS_FMT_VYUY8_1_5X8,
+	MEDIA_BUS_FMT_YUYV8_1_5X8,
+	MEDIA_BUS_FMT_YVYU8_1_5X8,
+	MEDIA_BUS_FMT_UYVY8_2X8,
+	MEDIA_BUS_FMT_VYUY8_2X8,
+	MEDIA_BUS_FMT_YUYV8_2X8,
+	MEDIA_BUS_FMT_YVYU8_2X8,
+	MEDIA_BUS_FMT_Y10_1X10,
+	MEDIA_BUS_FMT_Y10_2X8_PADHI_LE,
+	MEDIA_BUS_FMT_UYVY10_2X10,
+	MEDIA_BUS_FMT_VYUY10_2X10,
+	MEDIA_BUS_FMT_YUYV10_2X10,
+	MEDIA_BUS_FMT_YVYU10_2X10,
+	MEDIA_BUS_FMT_Y12_1X12,
+	MEDIA_BUS_FMT_UYVY12_2X12,
+	MEDIA_BUS_FMT_VYUY12_2X12,
+	MEDIA_BUS_FMT_YUYV12_2X12,
+	MEDIA_BUS_FMT_YVYU12_2X12,
+	MEDIA_BUS_FMT_UYVY8_1X16,
+	MEDIA_BUS_FMT_VYUY8_1X16,
+	MEDIA_BUS_FMT_YUYV8_1X16,
+	MEDIA_BUS_FMT_YVYU8_1X16,
+	MEDIA_BUS_FMT_YDYUYDYV8_1X16,
+	MEDIA_BUS_FMT_UYVY10_1X20,
+	MEDIA_BUS_FMT_VYUY10_1X20,
+	MEDIA_BUS_FMT_YUYV10_1X20,
+	MEDIA_BUS_FMT_YVYU10_1X20,
+	MEDIA_BUS_FMT_VUY8_1X24,
+	MEDIA_BUS_FMT_YUV8_1X24,
+	MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+	MEDIA_BUS_FMT_UYVY12_1X24,
+	MEDIA_BUS_FMT_VYUY12_1X24,
+	MEDIA_BUS_FMT_YUYV12_1X24,
+	MEDIA_BUS_FMT_YVYU12_1X24,
+	MEDIA_BUS_FMT_YUV10_1X30,
+	MEDIA_BUS_FMT_UYYVYY10_0_5X30,
+	MEDIA_BUS_FMT_AYUV8_1X32,
+	MEDIA_BUS_FMT_UYYVYY12_0_5X36,
+	MEDIA_BUS_FMT_YUV12_1X36,
+	MEDIA_BUS_FMT_YUV16_1X48,
+	MEDIA_BUS_FMT_UYYVYY16_0_5X48,
+	MEDIA_BUS_FMT_SBGGR8_1X8,
+	MEDIA_BUS_FMT_SGBRG8_1X8,
+	MEDIA_BUS_FMT_SGRBG8_1X8,
+	MEDIA_BUS_FMT_SRGGB8_1X8,
+	MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
+	MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
+	MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
+	MEDIA_BUS_FMT_SRGGB10_ALAW8_1X8,
+	MEDIA_BUS_FMT_SBGGR10_DPCM8_1X8,
+	MEDIA_BUS_FMT_SGBRG10_DPCM8_1X8,
+	MEDIA_BUS_FMT_SGRBG10_DPCM8_1X8,
+	MEDIA_BUS_FMT_SRGGB10_DPCM8_1X8,
+	MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE,
+	MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_LE,
+	MEDIA_BUS_FMT_SBGGR10_2X8_PADLO_BE,
+	MEDIA_BUS_FMT_SBGGR10_2X8_PADLO_LE,
+	MEDIA_BUS_FMT_SBGGR10_1X10,
+	MEDIA_BUS_FMT_SGBRG10_1X10,
+	MEDIA_BUS_FMT_SGRBG10_1X10,
+	MEDIA_BUS_FMT_SRGGB10_1X10,
+	MEDIA_BUS_FMT_SBGGR12_1X12,
+	MEDIA_BUS_FMT_SGBRG12_1X12,
+	MEDIA_BUS_FMT_SGRBG12_1X12,
+	MEDIA_BUS_FMT_SRGGB12_1X12,
+	MEDIA_BUS_FMT_SBGGR14_1X14,
+	MEDIA_BUS_FMT_SGBRG14_1X14,
+	MEDIA_BUS_FMT_SGRBG14_1X14,
+	MEDIA_BUS_FMT_SRGGB14_1X14,
+	MEDIA_BUS_FMT_SBGGR16_1X16,
+	MEDIA_BUS_FMT_SGBRG16_1X16,
+	MEDIA_BUS_FMT_SGRBG16_1X16,
+	MEDIA_BUS_FMT_SRGGB16_1X16,
+	MEDIA_BUS_FMT_JPEG_1X8,
+	MEDIA_BUS_FMT_S5C_UYVY_JPEG_1X8,
+	MEDIA_BUS_FMT_AHSV8888_1X32,
 };
 
-const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i)
+/* Helper function to enumerate mbus codes */
+int vimc_enum_mbus_code(struct v4l2_subdev *sd,
+			struct v4l2_subdev_pad_config *cfg,
+			struct v4l2_subdev_mbus_code_enum *code)
 {
-	if (i >= ARRAY_SIZE(vimc_pix_map_list))
-		return NULL;
-
-	return &vimc_pix_map_list[i];
-}
-EXPORT_SYMBOL_GPL(vimc_pix_map_by_index);
-
-const struct vimc_pix_map *vimc_pix_map_by_code(u32 code)
-{
-	unsigned int i;
-
-	for (i = 0; i < ARRAY_SIZE(vimc_pix_map_list); i++) {
-		if (vimc_pix_map_list[i].code == code)
-			return &vimc_pix_map_list[i];
-	}
-	return NULL;
-}
-EXPORT_SYMBOL_GPL(vimc_pix_map_by_code);
-
-const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat)
-{
-	unsigned int i;
+	if (code->index >= ARRAY_SIZE(vimc_mbus_list))
+		return -EINVAL;
 
-	for (i = 0; i < ARRAY_SIZE(vimc_pix_map_list); i++) {
-		if (vimc_pix_map_list[i].pixelformat == pixelformat)
-			return &vimc_pix_map_list[i];
-	}
-	return NULL;
+	code->code = vimc_mbus_list[code->index];
+	return 0;
 }
-EXPORT_SYMBOL_GPL(vimc_pix_map_by_pixelformat);
+EXPORT_SYMBOL_GPL(vimc_enum_mbus_code);
 
 /* Helper function to allocate and initialize pads */
 struct media_pad *vimc_pads_init(u16 num_pads, const unsigned long *pads_flag)
@@ -277,15 +212,13 @@ static int vimc_get_mbus_format(struct media_pad *pad,
 							 struct video_device,
 							 entity);
 		struct vimc_ent_device *ved = video_get_drvdata(vdev);
-		const struct vimc_pix_map *vpix;
 		struct v4l2_pix_format vdev_fmt;
 
 		if (!ved->vdev_get_format)
 			return -ENOIOCTLCMD;
 
 		ved->vdev_get_format(ved, &vdev_fmt);
-		vpix = vimc_pix_map_by_pixelformat(vdev_fmt.pixelformat);
-		v4l2_fill_mbus_format(&fmt->format, &vdev_fmt, vpix->code);
+		v4l2_fill_mbus_format(&fmt->format, &vdev_fmt, 0);
 	} else {
 		return -EINVAL;
 	}
@@ -325,8 +258,12 @@ int vimc_link_validate(struct media_link *link)
 	/* The width, height and code must match. */
 	if (source_fmt.format.width != sink_fmt.format.width
 	    || source_fmt.format.height != sink_fmt.format.height
-	    || source_fmt.format.code != sink_fmt.format.code)
+	    || (source_fmt.format.code && sink_fmt.format.code &&
+		source_fmt.format.code != sink_fmt.format.code)) {
+		pr_err("vimc: format doesn't match in link %s->%s\n",
+			link->source->entity->name, link->sink->entity->name);
 		return -EPIPE;
+	}
 
 	/*
 	 * The field order must match, or the sink field order must be NONE
diff --git a/drivers/media/platform/vimc/vimc-common.h b/drivers/media/platform/vimc/vimc-common.h
index 07987eab988f..30ce902ace4d 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -22,6 +22,8 @@
 #include <media/media-device.h>
 #include <media/v4l2-device.h>
 
+#include "vimc-streamer.h"
+
 #define VIMC_PDEV_NAME "vimc"
 
 /* VIMC-specific controls */
@@ -115,12 +117,23 @@ struct vimc_pix_map {
 struct vimc_ent_device {
 	struct media_entity *ent;
 	struct media_pad *pads;
+	struct vimc_stream *stream;
 	void * (*process_frame)(struct vimc_ent_device *ved,
 				const void *frame);
 	void (*vdev_get_format)(struct vimc_ent_device *ved,
 			      struct v4l2_pix_format *fmt);
 };
 
+/**
+ * vimc_enum_mbus_code - enumerate mbus codes
+ *
+ * Helper function to be pluged in .enum_mbus_code from
+ * struct v4l2_subdev_pad_ops.
+ */
+int vimc_enum_mbus_code(struct v4l2_subdev *sd,
+			struct v4l2_subdev_pad_config *cfg,
+			struct v4l2_subdev_mbus_code_enum *code);
+
 /**
  * vimc_pads_init - initialize pads
  *
diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
index 20826f209731..51b411739e27 100644
--- a/drivers/media/platform/vimc/vimc-debayer.c
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -26,6 +26,9 @@
 #include "vimc-common.h"
 
 #define VIMC_DEB_DRV_NAME "vimc-debayer"
+/* This module only supports tranforming a bayer format to V4L2_PIX_FMT_RGB24 */
+#define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
+#define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24
 
 static unsigned int deb_mean_win_size = 3;
 module_param(deb_mean_win_size, uint, 0000);
@@ -44,6 +47,7 @@ enum vimc_deb_rgb_colors {
 };
 
 struct vimc_deb_pix_map {
+	u32 pixelformat;
 	u32 code;
 	enum vimc_deb_rgb_colors order[2][2];
 };
@@ -73,61 +77,73 @@ static const struct v4l2_mbus_framefmt sink_fmt_default = {
 
 static const struct vimc_deb_pix_map vimc_deb_pix_map_list[] = {
 	{
+		.pixelformat = V4L2_PIX_FMT_SBGGR8,
 		.code = MEDIA_BUS_FMT_SBGGR8_1X8,
 		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
 			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SGBRG8,
 		.code = MEDIA_BUS_FMT_SGBRG8_1X8,
 		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
 			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SGRBG8,
 		.code = MEDIA_BUS_FMT_SGRBG8_1X8,
 		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
 			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SRGGB8,
 		.code = MEDIA_BUS_FMT_SRGGB8_1X8,
 		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
 			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SBGGR10,
 		.code = MEDIA_BUS_FMT_SBGGR10_1X10,
 		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
 			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SGBRG10,
 		.code = MEDIA_BUS_FMT_SGBRG10_1X10,
 		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
 			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SGRBG10,
 		.code = MEDIA_BUS_FMT_SGRBG10_1X10,
 		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
 			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SRGGB10,
 		.code = MEDIA_BUS_FMT_SRGGB10_1X10,
 		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
 			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SBGGR12,
 		.code = MEDIA_BUS_FMT_SBGGR12_1X12,
 		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
 			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SGBRG12,
 		.code = MEDIA_BUS_FMT_SGBRG12_1X12,
 		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
 			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SGRBG12,
 		.code = MEDIA_BUS_FMT_SGRBG12_1X12,
 		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
 			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
 	},
 	{
+		.pixelformat = V4L2_PIX_FMT_SRGGB12,
 		.code = MEDIA_BUS_FMT_SRGGB12_1X12,
 		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
 			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
@@ -168,41 +184,32 @@ static int vimc_deb_enum_mbus_code(struct v4l2_subdev *sd,
 				   struct v4l2_subdev_pad_config *cfg,
 				   struct v4l2_subdev_mbus_code_enum *code)
 {
-	/* We only support one format for source pads */
-	if (IS_SRC(code->pad)) {
-		struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
-
-		if (code->index)
-			return -EINVAL;
-
-		code->code = vdeb->src_code;
-	} else {
+	/* For the sink pad we only support codes in the map_list */
+	if (IS_SINK(code->pad)) {
 		if (code->index >= ARRAY_SIZE(vimc_deb_pix_map_list))
 			return -EINVAL;
 
 		code->code = vimc_deb_pix_map_list[code->index].code;
+		return 0;
 	}
 
-	return 0;
+	return vimc_enum_mbus_code(sd, cfg, code);
 }
 
 static int vimc_deb_enum_frame_size(struct v4l2_subdev *sd,
 				    struct v4l2_subdev_pad_config *cfg,
 				    struct v4l2_subdev_frame_size_enum *fse)
 {
-	struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
-
 	if (fse->index)
 		return -EINVAL;
 
+	/* For the sink pad we only support codes in the map_list */
 	if (IS_SINK(fse->pad)) {
 		const struct vimc_deb_pix_map *vpix =
 			vimc_deb_pix_map_by_code(fse->code);
 
 		if (!vpix)
 			return -EINVAL;
-	} else if (fse->code != vdeb->src_code) {
-		return -EINVAL;
 	}
 
 	fse->min_width = VIMC_FRAME_MIN_WIDTH;
@@ -270,11 +277,11 @@ static int vimc_deb_set_fmt(struct v4l2_subdev *sd,
 
 	/*
 	 * Do not change the format of the source pad,
-	 * it is propagated from the sink
+	 * it is propagated from the sink (except for the code)
 	 */
 	if (IS_SRC(fmt->pad)) {
+		vdeb->src_code = fmt->format.code;
 		fmt->format = *sink_fmt;
-		/* TODO: Add support for other formats */
 		fmt->format.code = vdeb->src_code;
 	} else {
 		/* Set the new format in the sink pad */
@@ -306,7 +313,7 @@ static const struct v4l2_subdev_pad_ops vimc_deb_pad_ops = {
 	.set_fmt		= vimc_deb_set_fmt,
 };
 
-static void vimc_deb_set_rgb_mbus_fmt_rgb888_1x24(struct vimc_deb_device *vdeb,
+static void vimc_deb_set_rgb_pix_rgb24(struct vimc_deb_device *vdeb,
 						  unsigned int lin,
 						  unsigned int col,
 						  unsigned int rgb[3])
@@ -323,25 +330,38 @@ static int vimc_deb_s_stream(struct v4l2_subdev *sd, int enable)
 	struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
 
 	if (enable) {
-		const struct vimc_pix_map *vpix;
+		u32 src_pixelformat = vdeb->ved.stream->producer_pixfmt;
+		const struct v4l2_format_info *pix_info;
 		unsigned int frame_size;
 
 		if (vdeb->src_frame)
 			return 0;
 
-		/* Calculate the frame size of the source pad */
-		vpix = vimc_pix_map_by_code(vdeb->src_code);
-		frame_size = vdeb->sink_fmt.width * vdeb->sink_fmt.height *
-				vpix->bpp;
-
-		/* Save the bytes per pixel of the sink */
-		vpix = vimc_pix_map_by_code(vdeb->sink_fmt.code);
-		vdeb->sink_bpp = vpix->bpp;
+		/* We only support translating bayer to RGB24 */
+		if (src_pixelformat != V4L2_PIX_FMT_RGB24) {
+			dev_err(vdeb->dev,
+				"translating to pixfmt (%s) is not supported\n",
+				v4l2_get_fourcc_name(src_pixelformat));
+			return -EINVAL;
+		}
 
 		/* Get the corresponding pixel map from the table */
 		vdeb->sink_pix_map =
 			vimc_deb_pix_map_by_code(vdeb->sink_fmt.code);
 
+		/* Request bayer format from the pipeline for the sink pad */
+		vdeb->ved.stream->producer_pixfmt =
+			vdeb->sink_pix_map->pixelformat;
+
+		/* Calculate frame_size of the source */
+		pix_info = v4l2_format_info(src_pixelformat);
+		frame_size = vdeb->sink_fmt.width * vdeb->sink_fmt.height *
+			     pix_info->bpp[0];
+
+		/* Get bpp from the sink */
+		pix_info = v4l2_format_info(vdeb->sink_pix_map->pixelformat);
+		vdeb->sink_bpp = pix_info->bpp[0];
+
 		/*
 		 * Allocate the frame buffer. Use vmalloc to be able to
 		 * allocate a large amount of memory
@@ -542,14 +562,14 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize the frame format */
 	vdeb->sink_fmt = sink_fmt_default;
+	vdeb->src_code = VIMC_DEB_SRC_MBUS_FMT_DEFAULT;
 	/*
 	 * TODO: Add support for more output formats, we only support
-	 * RGB888 for now
+	 * RGB24 for now.
 	 * NOTE: the src format is always the same as the sink, except
 	 * for the code
 	 */
-	vdeb->src_code = MEDIA_BUS_FMT_RGB888_1X24;
-	vdeb->set_rgb_src = vimc_deb_set_rgb_mbus_fmt_rgb888_1x24;
+	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
 
 	return 0;
 }
diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
index 2028afa4ef7a..245116b1c241 100644
--- a/drivers/media/platform/vimc/vimc-scaler.c
+++ b/drivers/media/platform/vimc/vimc-scaler.c
@@ -35,6 +35,12 @@ MODULE_PARM_DESC(sca_mult, " the image size multiplier");
 #define IS_SRC(pad)	(pad)
 #define MAX_ZOOM	8
 
+static const u32 vimc_sca_supported_pixftm[] = {
+	V4L2_PIX_FMT_BGR24,
+	V4L2_PIX_FMT_RGB24,
+	V4L2_PIX_FMT_ARGB32,
+};
+
 struct vimc_sca_device {
 	struct vimc_ent_device ved;
 	struct v4l2_subdev sd;
@@ -57,6 +63,16 @@ static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.colorspace = V4L2_COLORSPACE_DEFAULT,
 };
 
+static bool vimc_sca_is_pixfmt_supported(u32 pixelformat)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(vimc_sca_supported_pixftm); i++)
+		if (vimc_sca_supported_pixftm[i] == pixelformat)
+			return true;
+	return false;
+}
+
 static int vimc_sca_init_cfg(struct v4l2_subdev *sd,
 			     struct v4l2_subdev_pad_config *cfg)
 {
@@ -76,35 +92,13 @@ static int vimc_sca_init_cfg(struct v4l2_subdev *sd,
 	return 0;
 }
 
-static int vimc_sca_enum_mbus_code(struct v4l2_subdev *sd,
-				   struct v4l2_subdev_pad_config *cfg,
-				   struct v4l2_subdev_mbus_code_enum *code)
-{
-	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
-
-	/* We don't support bayer format */
-	if (!vpix || vpix->bayer)
-		return -EINVAL;
-
-	code->code = vpix->code;
-
-	return 0;
-}
-
 static int vimc_sca_enum_frame_size(struct v4l2_subdev *sd,
 				    struct v4l2_subdev_pad_config *cfg,
 				    struct v4l2_subdev_frame_size_enum *fse)
 {
-	const struct vimc_pix_map *vpix;
-
 	if (fse->index)
 		return -EINVAL;
 
-	/* Only accept code in the pix map table in non bayer format */
-	vpix = vimc_pix_map_by_code(fse->code);
-	if (!vpix || vpix->bayer)
-		return -EINVAL;
-
 	fse->min_width = VIMC_FRAME_MIN_WIDTH;
 	fse->min_height = VIMC_FRAME_MIN_HEIGHT;
 
@@ -141,13 +135,6 @@ static int vimc_sca_get_fmt(struct v4l2_subdev *sd,
 
 static void vimc_sca_adjust_sink_fmt(struct v4l2_mbus_framefmt *fmt)
 {
-	const struct vimc_pix_map *vpix;
-
-	/* Only accept code in the pix map table in non bayer format */
-	vpix = vimc_pix_map_by_code(fmt->code);
-	if (!vpix || vpix->bayer)
-		fmt->code = sink_fmt_default.code;
-
 	fmt->width = clamp_t(u32, fmt->width, VIMC_FRAME_MIN_WIDTH,
 			     VIMC_FRAME_MAX_WIDTH) & ~1;
 	fmt->height = clamp_t(u32, fmt->height, VIMC_FRAME_MIN_HEIGHT,
@@ -208,7 +195,7 @@ static int vimc_sca_set_fmt(struct v4l2_subdev *sd,
 
 static const struct v4l2_subdev_pad_ops vimc_sca_pad_ops = {
 	.init_cfg		= vimc_sca_init_cfg,
-	.enum_mbus_code		= vimc_sca_enum_mbus_code,
+	.enum_mbus_code		= vimc_enum_mbus_code,
 	.enum_frame_size	= vimc_sca_enum_frame_size,
 	.get_fmt		= vimc_sca_get_fmt,
 	.set_fmt		= vimc_sca_set_fmt,
@@ -219,15 +206,22 @@ static int vimc_sca_s_stream(struct v4l2_subdev *sd, int enable)
 	struct vimc_sca_device *vsca = v4l2_get_subdevdata(sd);
 
 	if (enable) {
-		const struct vimc_pix_map *vpix;
+		u32 pixelformat = vsca->ved.stream->producer_pixfmt;
+		const struct v4l2_format_info *pix_info;
 		unsigned int frame_size;
 
 		if (vsca->src_frame)
 			return 0;
 
+		if (!vimc_sca_is_pixfmt_supported(pixelformat)) {
+			dev_err(vsca->dev, "pixfmt (%s) is not supported\n",
+				v4l2_get_fourcc_name(pixelformat));
+			return -EINVAL;
+		}
+
 		/* Save the bytes per pixel of the sink */
-		vpix = vimc_pix_map_by_code(vsca->sink_fmt.code);
-		vsca->bpp = vpix->bpp;
+		pix_info = v4l2_format_info(pixelformat);
+		vsca->bpp = pix_info->bpp[0];
 
 		/* Calculate the width in bytes of the src frame */
 		vsca->src_line_size = vsca->sink_fmt.width *
diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
index d7891d3bbeaa..75df96cc5e39 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -65,34 +65,13 @@ static int vimc_sen_init_cfg(struct v4l2_subdev *sd,
 	return 0;
 }
 
-static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
-				   struct v4l2_subdev_pad_config *cfg,
-				   struct v4l2_subdev_mbus_code_enum *code)
-{
-	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
-
-	if (!vpix)
-		return -EINVAL;
-
-	code->code = vpix->code;
-
-	return 0;
-}
-
 static int vimc_sen_enum_frame_size(struct v4l2_subdev *sd,
 				    struct v4l2_subdev_pad_config *cfg,
 				    struct v4l2_subdev_frame_size_enum *fse)
 {
-	const struct vimc_pix_map *vpix;
-
 	if (fse->index)
 		return -EINVAL;
 
-	/* Only accept code in the pix map table */
-	vpix = vimc_pix_map_by_code(fse->code);
-	if (!vpix)
-		return -EINVAL;
-
 	fse->min_width = VIMC_FRAME_MIN_WIDTH;
 	fse->max_width = VIMC_FRAME_MAX_WIDTH;
 	fse->min_height = VIMC_FRAME_MIN_HEIGHT;
@@ -117,14 +96,17 @@ static int vimc_sen_get_fmt(struct v4l2_subdev *sd,
 
 static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
 {
-	const struct vimc_pix_map *vpix =
-				vimc_pix_map_by_code(vsen->mbus_format.code);
+	u32 pixelformat = vsen->ved.stream->producer_pixfmt;
+	const struct v4l2_format_info *pix_info;
+
+	pix_info = v4l2_format_info(pixelformat);
 
 	tpg_reset_source(&vsen->tpg, vsen->mbus_format.width,
 			 vsen->mbus_format.height, vsen->mbus_format.field);
-	tpg_s_bytesperline(&vsen->tpg, 0, vsen->mbus_format.width * vpix->bpp);
+	tpg_s_bytesperline(&vsen->tpg, 0,
+			   vsen->mbus_format.width * pix_info->bpp[0]);
 	tpg_s_buf_height(&vsen->tpg, vsen->mbus_format.height);
-	tpg_s_fourcc(&vsen->tpg, vpix->pixelformat);
+	tpg_s_fourcc(&vsen->tpg, pixelformat);
 	/* TODO: add support for V4L2_FIELD_ALTERNATE */
 	tpg_s_field(&vsen->tpg, vsen->mbus_format.field, false);
 	tpg_s_colorspace(&vsen->tpg, vsen->mbus_format.colorspace);
@@ -135,13 +117,6 @@ static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
 
 static void vimc_sen_adjust_fmt(struct v4l2_mbus_framefmt *fmt)
 {
-	const struct vimc_pix_map *vpix;
-
-	/* Only accept code in the pix map table */
-	vpix = vimc_pix_map_by_code(fmt->code);
-	if (!vpix)
-		fmt->code = fmt_default.code;
-
 	fmt->width = clamp_t(u32, fmt->width, VIMC_FRAME_MIN_WIDTH,
 			     VIMC_FRAME_MAX_WIDTH) & ~1;
 	fmt->height = clamp_t(u32, fmt->height, VIMC_FRAME_MIN_HEIGHT,
@@ -193,7 +168,7 @@ static int vimc_sen_set_fmt(struct v4l2_subdev *sd,
 
 static const struct v4l2_subdev_pad_ops vimc_sen_pad_ops = {
 	.init_cfg		= vimc_sen_init_cfg,
-	.enum_mbus_code		= vimc_sen_enum_mbus_code,
+	.enum_mbus_code		= vimc_enum_mbus_code,
 	.enum_frame_size	= vimc_sen_enum_frame_size,
 	.get_fmt		= vimc_sen_get_fmt,
 	.set_fmt		= vimc_sen_set_fmt,
@@ -215,7 +190,8 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)
 				container_of(sd, struct vimc_sen_device, sd);
 
 	if (enable) {
-		const struct vimc_pix_map *vpix;
+		u32 pixelformat = vsen->ved.stream->producer_pixfmt;
+		const struct v4l2_format_info *pix_info;
 		unsigned int frame_size;
 
 		if (vsen->kthread_sen)
@@ -223,8 +199,8 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)
 			return 0;
 
 		/* Calculate the frame size */
-		vpix = vimc_pix_map_by_code(vsen->mbus_format.code);
-		frame_size = vsen->mbus_format.width * vpix->bpp *
+		pix_info = v4l2_format_info(pixelformat);
+		frame_size = vsen->mbus_format.width * pix_info->bpp[0] *
 			     vsen->mbus_format.height;
 
 		/*
diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
index 5a3bda62fbc8..c19093b6c787 100644
--- a/drivers/media/platform/vimc/vimc-streamer.c
+++ b/drivers/media/platform/vimc/vimc-streamer.c
@@ -52,6 +52,7 @@ static void vimc_streamer_pipeline_terminate(struct vimc_stream *stream)
 	while (stream->pipe_size) {
 		stream->pipe_size--;
 		ved = stream->ved_pipeline[stream->pipe_size];
+		ved->stream = NULL;
 		stream->ved_pipeline[stream->pipe_size] = NULL;
 
 		if (!is_media_entity_v4l2_subdev(ved->ent))
@@ -87,6 +88,7 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
 			return -EINVAL;
 		}
 		stream->ved_pipeline[stream->pipe_size++] = ved;
+		ved->stream = stream;
 
 		if (is_media_entity_v4l2_subdev(ved->ent)) {
 			sd = media_entity_to_v4l2_subdev(ved->ent);
diff --git a/drivers/media/platform/vimc/vimc-streamer.h b/drivers/media/platform/vimc/vimc-streamer.h
index a7c5ac5ace4f..2b3667408794 100644
--- a/drivers/media/platform/vimc/vimc-streamer.h
+++ b/drivers/media/platform/vimc/vimc-streamer.h
@@ -25,6 +25,11 @@
  * processed in the pipeline.
  * @pipe_size:		size of @ved_pipeline
  * @kthread:		thread that generates the frames of the stream.
+ * @producer_pixfmt:	the pixel format requested from the pipeline. This must
+ * be set just before calling vimc_streamer_s_stream(ent, 1). This value is
+ * propagated up to the source of the base image (usually a sensor node) and
+ * can be modified by entities during s_stream callback to request a different
+ * format from rest of the pipeline.
  *
  * When the user call stream_on in a video device, struct vimc_stream is
  * used to keep track of all entities and subdevices that generates and
@@ -35,6 +40,7 @@ struct vimc_stream {
 	struct vimc_ent_device *ved_pipeline[VIMC_STREAMER_PIPELINE_MAX_SIZE];
 	unsigned int pipe_size;
 	struct task_struct *kthread;
+	u32 producer_pixfmt;
 };
 
 /**
-- 
2.20.1

Re: [PATCH 5/8] media: vimc: stream: cleanup frame field from struct vimc_stream

[André Almeida]

Hello,

I've tested the stream (with the default media bus format and using an 
alternative one) using a custom user space application and tested all 
capture entities using qv4l2 -d /dev/videoX (where X is each of the 
capture entities). Also, I have checked v4l2-compliance -d /dev/videoX 
and with v4l2-compliance --streaming=5 -d /dev/videoX and no regressions 
where found.

On 3/6/19 7:42 PM, Helen Koike wrote:
> There is no need to have the frame field in the vimc_stream struct.
>
> Signed-off-by: Helen Koike <helen.koike@collabora.com>
Tested-by: André Almeida <andre.almeida@collabora.com>
> ---
>
>   drivers/media/platform/vimc/vimc-streamer.c | 10 ++++------
>   drivers/media/platform/vimc/vimc-streamer.h |  1 -
>   2 files changed, 4 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
> index 392754c18046..b7c1fdef5f0d 100644
> --- a/drivers/media/platform/vimc/vimc-streamer.c
> +++ b/drivers/media/platform/vimc/vimc-streamer.c
> @@ -117,6 +117,7 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
>   static int vimc_streamer_thread(void *data)
>   {
>   	struct vimc_stream *stream = data;
> +	u8 *frame = NULL;
>   	int i;
>   
>   	set_freezable();
> @@ -127,12 +128,9 @@ static int vimc_streamer_thread(void *data)
>   			break;
>   
>   		for (i = stream->pipe_size - 1; i >= 0; i--) {
> -			stream->frame = stream->ved_pipeline[i]->process_frame(
> -					stream->ved_pipeline[i],
> -					stream->frame);
> -			if (!stream->frame)
> -				break;
> -			if (IS_ERR(stream->frame))
> +			frame = stream->ved_pipeline[i]->process_frame(
> +					stream->ved_pipeline[i], frame);
> +			if (!frame || IS_ERR(frame))
>   				break;
>   		}
>   		//wait for 60hz
> diff --git a/drivers/media/platform/vimc/vimc-streamer.h b/drivers/media/platform/vimc/vimc-streamer.h
> index 752af2e2d5a2..dc1d0be431cb 100644
> --- a/drivers/media/platform/vimc/vimc-streamer.h
> +++ b/drivers/media/platform/vimc/vimc-streamer.h
> @@ -19,7 +19,6 @@ struct vimc_stream {
>   	struct media_pipeline pipe;
>   	struct vimc_ent_device *ved_pipeline[VIMC_STREAMER_PIPELINE_MAX_SIZE];
>   	unsigned int pipe_size;
> -	u8 *frame;
>   	struct task_struct *kthread;
>   };
>   

Re: [PATCH 7/8] media: vimc: stream: init/terminate the first entity

[André Almeida]

On 3/6/19 7:42 PM, Helen Koike wrote:
> The s_stream callback was not being called for the first entity in the
> stream pipeline array.
> Instead of verifying the type of the node (video or subdevice) and
> calling s_stream from the second entity in the pipeline, do this process
> for all the entities in the pipeline for consistency.
>
> The previous code was not a problem because the first entity is a video
> device and not a subdevice, but this patch prepares vimc to allow
> setting some configuration in the entity before calling s_stream.
>
> Signed-off-by: Helen Koike <helen.koike@collabora.com>
Hello,

I've tested the stream (with the default media bus format and using an 
alternative one) using a custom user space application and tested all 
capture entities using qv4l2 -d /dev/videoX (where X is each of the 
capture entities). Also, I have checked v4l2-compliance -d /dev/videoX 
and with v4l2-compliance --streaming=5 -d /dev/videoX and no regressions 
where found.

Tested-by: André Almeida <andre.almeida@collabora.com>
> ---
>
>   drivers/media/platform/vimc/vimc-streamer.c | 25 ++++++++++++---------
>   1 file changed, 15 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
> index b7c1fdef5f0d..5a3bda62fbc8 100644
> --- a/drivers/media/platform/vimc/vimc-streamer.c
> +++ b/drivers/media/platform/vimc/vimc-streamer.c
> @@ -46,19 +46,18 @@ static struct media_entity *vimc_get_source_entity(struct media_entity *ent)
>    */
>   static void vimc_streamer_pipeline_terminate(struct vimc_stream *stream)
>   {
> -	struct media_entity *entity;
> +	struct vimc_ent_device *ved;
>   	struct v4l2_subdev *sd;
>   
>   	while (stream->pipe_size) {
>   		stream->pipe_size--;
> -		entity = stream->ved_pipeline[stream->pipe_size]->ent;
> -		entity = vimc_get_source_entity(entity);
> +		ved = stream->ved_pipeline[stream->pipe_size];
>   		stream->ved_pipeline[stream->pipe_size] = NULL;
>   
> -		if (!is_media_entity_v4l2_subdev(entity))
> +		if (!is_media_entity_v4l2_subdev(ved->ent))
>   			continue;
>   
> -		sd = media_entity_to_v4l2_subdev(entity);
> +		sd = media_entity_to_v4l2_subdev(ved->ent);
>   		v4l2_subdev_call(sd, video, s_stream, 0);
>   	}
>   }
> @@ -89,18 +88,24 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
>   		}
>   		stream->ved_pipeline[stream->pipe_size++] = ved;
>   
> +		if (is_media_entity_v4l2_subdev(ved->ent)) {
> +			sd = media_entity_to_v4l2_subdev(ved->ent);
> +			ret = v4l2_subdev_call(sd, video, s_stream, 1);
> +			if (ret && ret != -ENOIOCTLCMD) {
> +				pr_err("subdev_call error %s\n", ved->ent->name);
> +				vimc_streamer_pipeline_terminate(stream);
> +				return ret;
> +			}
> +		}
> +
>   		entity = vimc_get_source_entity(ved->ent);
>   		/* Check if the end of the pipeline was reached*/
>   		if (!entity)
>   			return 0;
>   
> +		/* Get the next device in the pipeline */
>   		if (is_media_entity_v4l2_subdev(entity)) {
>   			sd = media_entity_to_v4l2_subdev(entity);
> -			ret = v4l2_subdev_call(sd, video, s_stream, 1);
> -			if (ret && ret != -ENOIOCTLCMD) {
> -				vimc_streamer_pipeline_terminate(stream);
> -				return ret;
> -			}
>   			ved = v4l2_get_subdevdata(sd);
>   		} else {
>   			vdev = container_of(entity,

Re: [PATCH 8/8] media: vimc: propagate pixel format in the stream

[André Almeida]

On 3/6/19 7:42 PM, Helen Koike wrote:
> Media bus codes were being mapped to pixelformats, which causes a
> limitation on vimc because not all pixelformats can be mapped to media
> bus codes.
> Also, media bus codes are an internal configuration from the device.
> Userspace only assures media bus codes matches between pads and expects
> the image in a given pixelformat. So we can allow almost any media bus
> format to be configured between pads, except for debayer that expects a
> media bus code of type bayer in the sink pad.
>
> Signed-off-by: Helen Koike <helen.koike@collabora.com>
Hello,

I've tested the stream (with the default media bus format and using an 
alternative one) using a custom user space application and tested all 
capture entities using qv4l2 -d /dev/videoX (where X is each of the 
capture entities). Also, I have checked v4l2-compliance -d /dev/videoX 
and with v4l2-compliance --streaming=5 -d /dev/videoX and no regressions 
where found.

Tested-by: André Almeida <andre.almeida@collabora.com>
>
> ---
>
>   drivers/media/platform/vimc/vimc-capture.c  |  76 +++--
>   drivers/media/platform/vimc/vimc-common.c   | 307 ++++++++------------
>   drivers/media/platform/vimc/vimc-common.h   |  13 +
>   drivers/media/platform/vimc/vimc-debayer.c  |  78 +++--
>   drivers/media/platform/vimc/vimc-scaler.c   |  60 ++--
>   drivers/media/platform/vimc/vimc-sensor.c   |  48 +--
>   drivers/media/platform/vimc/vimc-streamer.c |   2 +
>   drivers/media/platform/vimc/vimc-streamer.h |   6 +
>   8 files changed, 281 insertions(+), 309 deletions(-)
>
> diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
> index e976a9d6b460..6377974879d7 100644
> --- a/drivers/media/platform/vimc/vimc-capture.c
> +++ b/drivers/media/platform/vimc/vimc-capture.c
> @@ -28,6 +28,32 @@
>   
>   #define VIMC_CAP_DRV_NAME "vimc-capture"
>   
> +static const u32 vimc_cap_supported_pixftm[] = {
> +	V4L2_PIX_FMT_BGR24,
> +	V4L2_PIX_FMT_RGB24,
> +	V4L2_PIX_FMT_ARGB32,
> +	V4L2_PIX_FMT_SBGGR8,
> +	V4L2_PIX_FMT_SGBRG8,
> +	V4L2_PIX_FMT_SGRBG8,
> +	V4L2_PIX_FMT_SRGGB8,
> +	V4L2_PIX_FMT_SBGGR10,
> +	V4L2_PIX_FMT_SGBRG10,
> +	V4L2_PIX_FMT_SGRBG10,
> +	V4L2_PIX_FMT_SRGGB10,
> +	V4L2_PIX_FMT_SBGGR10ALAW8,
> +	V4L2_PIX_FMT_SGBRG10ALAW8,
> +	V4L2_PIX_FMT_SGRBG10ALAW8,
> +	V4L2_PIX_FMT_SRGGB10ALAW8,
> +	V4L2_PIX_FMT_SBGGR10DPCM8,
> +	V4L2_PIX_FMT_SGBRG10DPCM8,
> +	V4L2_PIX_FMT_SGRBG10DPCM8,
> +	V4L2_PIX_FMT_SRGGB10DPCM8,
> +	V4L2_PIX_FMT_SBGGR12,
> +	V4L2_PIX_FMT_SGBRG12,
> +	V4L2_PIX_FMT_SGRBG12,
> +	V4L2_PIX_FMT_SRGGB12,
> +};
> +
>   struct vimc_cap_device {
>   	struct vimc_ent_device ved;
>   	struct video_device vdev;
> @@ -101,29 +127,25 @@ static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
>   				    struct v4l2_format *f)
>   {
>   	struct v4l2_pix_format *format = &f->fmt.pix;
> -	const struct vimc_pix_map *vpix;
>   
>   	format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
>   				VIMC_FRAME_MAX_WIDTH) & ~1;
>   	format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
>   				 VIMC_FRAME_MAX_HEIGHT) & ~1;
>   
> -	/* Don't accept a pixelformat that is not on the table */
> -	vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
> -	if (!vpix) {
> -		format->pixelformat = fmt_default.pixelformat;
> -		vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
> -	}
> -	/* TODO: Add support for custom bytesperline values */
> -	format->bytesperline = format->width * vpix->bpp;
> -	format->sizeimage = format->bytesperline * format->height;
> +	vimc_colorimetry_clamp(format);
>   
>   	if (format->field == V4L2_FIELD_ANY)
>   		format->field = fmt_default.field;
>   
> -	vimc_colorimetry_clamp(format);
> +	/* TODO: Add support for custom bytesperline values */
>   
> -	return 0;
> +	/* Don't accept a pixelformat that is not on the table */
> +	if (!v4l2_format_info(format->pixelformat))
> +		format->pixelformat = fmt_default.pixelformat;
> +
> +	return v4l2_fill_pixfmt(format, format->pixelformat,
> +				format->width, format->height);
>   }
>   
>   static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
> @@ -159,27 +181,31 @@ static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
>   static int vimc_cap_enum_fmt_vid_cap(struct file *file, void *priv,
>   				     struct v4l2_fmtdesc *f)
>   {
> -	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(f->index);
> -
> -	if (!vpix)
> +	if (f->index >= ARRAY_SIZE(vimc_cap_supported_pixftm))
>   		return -EINVAL;
>   
> -	f->pixelformat = vpix->pixelformat;
> +	f->pixelformat = vimc_cap_supported_pixftm[f->index];
>   
>   	return 0;
>   }
>   
> +static bool vimc_cap_is_pixfmt_supported(u32 pixelformat)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(vimc_cap_supported_pixftm); i++)
> +		if (vimc_cap_supported_pixftm[i] == pixelformat)
> +			return true;
> +	return false;
> +}
> +
>   static int vimc_cap_enum_framesizes(struct file *file, void *fh,
>   				    struct v4l2_frmsizeenum *fsize)
>   {
> -	const struct vimc_pix_map *vpix;
> -
>   	if (fsize->index)
>   		return -EINVAL;
>   
> -	/* Only accept code in the pix map table */
> -	vpix = vimc_pix_map_by_code(fsize->pixel_format);
> -	if (!vpix)
> +	if (!vimc_cap_is_pixfmt_supported(fsize->pixel_format))
>   		return -EINVAL;
>   
>   	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> @@ -253,6 +279,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
>   		return ret;
>   	}
>   
> +	vcap->stream.producer_pixfmt = vcap->format.pixelformat;
>   	ret = vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 1);
>   	if (ret) {
>   		media_pipeline_stop(entity);
> @@ -403,7 +430,6 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>   {
>   	struct v4l2_device *v4l2_dev = master_data;
>   	struct vimc_platform_data *pdata = comp->platform_data;
> -	const struct vimc_pix_map *vpix;
>   	struct vimc_cap_device *vcap;
>   	struct video_device *vdev;
>   	struct vb2_queue *q;
> @@ -458,10 +484,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>   
>   	/* Set default frame format */
>   	vcap->format = fmt_default;
> -	vpix = vimc_pix_map_by_pixelformat(vcap->format.pixelformat);
> -	vcap->format.bytesperline = vcap->format.width * vpix->bpp;
> -	vcap->format.sizeimage = vcap->format.bytesperline *
> -				 vcap->format.height;
> +	v4l2_fill_pixfmt(&vcap->format, vcap->format.pixelformat,
> +			 vcap->format.width, vcap->format.height);
>   
>   	/* Fill the vimc_ent_device struct */
>   	vcap->ved.ent = &vcap->vdev.entity;
> diff --git a/drivers/media/platform/vimc/vimc-common.c b/drivers/media/platform/vimc/vimc-common.c
> index 0adbfd8fd26d..b07bbf0564b9 100644
> --- a/drivers/media/platform/vimc/vimc-common.c
> +++ b/drivers/media/platform/vimc/vimc-common.c
> @@ -20,192 +20,127 @@
>   
>   #include "vimc-common.h"
>   
> -/*
> - * NOTE: non-bayer formats need to come first (necessary for enum_mbus_code
> - * in the scaler)
> - */
> -static const struct vimc_pix_map vimc_pix_map_list[] = {
> -	/* TODO: add all missing formats */
> -
> -	/* RGB formats */
> -	{
> -		.code = MEDIA_BUS_FMT_BGR888_1X24,
> -		.pixelformat = V4L2_PIX_FMT_BGR24,
> -		.bpp = 3,
> -		.bayer = false,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_RGB888_1X24,
> -		.pixelformat = V4L2_PIX_FMT_RGB24,
> -		.bpp = 3,
> -		.bayer = false,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_ARGB8888_1X32,
> -		.pixelformat = V4L2_PIX_FMT_ARGB32,
> -		.bpp = 4,
> -		.bayer = false,
> -	},
> -
> -	/* Bayer formats */
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -
> -	/* 10bit raw bayer a-law compressed to 8 bits */
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -
> -	/* 10bit raw bayer DPCM compressed to 8 bits */
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> +const u32 vimc_mbus_list[] = {
> +	MEDIA_BUS_FMT_FIXED,
> +	MEDIA_BUS_FMT_RGB444_1X12,
> +	MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE,
> +	MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
> +	MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_RGB565_1X16,
> +	MEDIA_BUS_FMT_BGR565_2X8_BE,
> +	MEDIA_BUS_FMT_BGR565_2X8_LE,
> +	MEDIA_BUS_FMT_RGB565_2X8_BE,
> +	MEDIA_BUS_FMT_RGB565_2X8_LE,
> +	MEDIA_BUS_FMT_RGB666_1X18,
> +	MEDIA_BUS_FMT_RBG888_1X24,
> +	MEDIA_BUS_FMT_RGB666_1X24_CPADHI,
> +	MEDIA_BUS_FMT_RGB666_1X7X3_SPWG,
> +	MEDIA_BUS_FMT_BGR888_1X24,
> +	MEDIA_BUS_FMT_GBR888_1X24,
> +	MEDIA_BUS_FMT_RGB888_1X24,
> +	MEDIA_BUS_FMT_RGB888_2X12_BE,
> +	MEDIA_BUS_FMT_RGB888_2X12_LE,
> +	MEDIA_BUS_FMT_RGB888_1X7X4_SPWG,
> +	MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA,
> +	MEDIA_BUS_FMT_ARGB8888_1X32,
> +	MEDIA_BUS_FMT_RGB888_1X32_PADHI,
> +	MEDIA_BUS_FMT_RGB101010_1X30,
> +	MEDIA_BUS_FMT_RGB121212_1X36,
> +	MEDIA_BUS_FMT_RGB161616_1X48,
> +	MEDIA_BUS_FMT_Y8_1X8,
> +	MEDIA_BUS_FMT_UV8_1X8,
> +	MEDIA_BUS_FMT_UYVY8_1_5X8,
> +	MEDIA_BUS_FMT_VYUY8_1_5X8,
> +	MEDIA_BUS_FMT_YUYV8_1_5X8,
> +	MEDIA_BUS_FMT_YVYU8_1_5X8,
> +	MEDIA_BUS_FMT_UYVY8_2X8,
> +	MEDIA_BUS_FMT_VYUY8_2X8,
> +	MEDIA_BUS_FMT_YUYV8_2X8,
> +	MEDIA_BUS_FMT_YVYU8_2X8,
> +	MEDIA_BUS_FMT_Y10_1X10,
> +	MEDIA_BUS_FMT_Y10_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_UYVY10_2X10,
> +	MEDIA_BUS_FMT_VYUY10_2X10,
> +	MEDIA_BUS_FMT_YUYV10_2X10,
> +	MEDIA_BUS_FMT_YVYU10_2X10,
> +	MEDIA_BUS_FMT_Y12_1X12,
> +	MEDIA_BUS_FMT_UYVY12_2X12,
> +	MEDIA_BUS_FMT_VYUY12_2X12,
> +	MEDIA_BUS_FMT_YUYV12_2X12,
> +	MEDIA_BUS_FMT_YVYU12_2X12,
> +	MEDIA_BUS_FMT_UYVY8_1X16,
> +	MEDIA_BUS_FMT_VYUY8_1X16,
> +	MEDIA_BUS_FMT_YUYV8_1X16,
> +	MEDIA_BUS_FMT_YVYU8_1X16,
> +	MEDIA_BUS_FMT_YDYUYDYV8_1X16,
> +	MEDIA_BUS_FMT_UYVY10_1X20,
> +	MEDIA_BUS_FMT_VYUY10_1X20,
> +	MEDIA_BUS_FMT_YUYV10_1X20,
> +	MEDIA_BUS_FMT_YVYU10_1X20,
> +	MEDIA_BUS_FMT_VUY8_1X24,
> +	MEDIA_BUS_FMT_YUV8_1X24,
> +	MEDIA_BUS_FMT_UYYVYY8_0_5X24,
> +	MEDIA_BUS_FMT_UYVY12_1X24,
> +	MEDIA_BUS_FMT_VYUY12_1X24,
> +	MEDIA_BUS_FMT_YUYV12_1X24,
> +	MEDIA_BUS_FMT_YVYU12_1X24,
> +	MEDIA_BUS_FMT_YUV10_1X30,
> +	MEDIA_BUS_FMT_UYYVYY10_0_5X30,
> +	MEDIA_BUS_FMT_AYUV8_1X32,
> +	MEDIA_BUS_FMT_UYYVYY12_0_5X36,
> +	MEDIA_BUS_FMT_YUV12_1X36,
> +	MEDIA_BUS_FMT_YUV16_1X48,
> +	MEDIA_BUS_FMT_UYYVYY16_0_5X48,
> +	MEDIA_BUS_FMT_SBGGR8_1X8,
> +	MEDIA_BUS_FMT_SGBRG8_1X8,
> +	MEDIA_BUS_FMT_SGRBG8_1X8,
> +	MEDIA_BUS_FMT_SRGGB8_1X8,
> +	MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SRGGB10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SBGGR10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SGBRG10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SGRBG10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SRGGB10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADLO_BE,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADLO_LE,
> +	MEDIA_BUS_FMT_SBGGR10_1X10,
> +	MEDIA_BUS_FMT_SGBRG10_1X10,
> +	MEDIA_BUS_FMT_SGRBG10_1X10,
> +	MEDIA_BUS_FMT_SRGGB10_1X10,
> +	MEDIA_BUS_FMT_SBGGR12_1X12,
> +	MEDIA_BUS_FMT_SGBRG12_1X12,
> +	MEDIA_BUS_FMT_SGRBG12_1X12,
> +	MEDIA_BUS_FMT_SRGGB12_1X12,
> +	MEDIA_BUS_FMT_SBGGR14_1X14,
> +	MEDIA_BUS_FMT_SGBRG14_1X14,
> +	MEDIA_BUS_FMT_SGRBG14_1X14,
> +	MEDIA_BUS_FMT_SRGGB14_1X14,
> +	MEDIA_BUS_FMT_SBGGR16_1X16,
> +	MEDIA_BUS_FMT_SGBRG16_1X16,
> +	MEDIA_BUS_FMT_SGRBG16_1X16,
> +	MEDIA_BUS_FMT_SRGGB16_1X16,
> +	MEDIA_BUS_FMT_JPEG_1X8,
> +	MEDIA_BUS_FMT_S5C_UYVY_JPEG_1X8,
> +	MEDIA_BUS_FMT_AHSV8888_1X32,
>   };
>   
> -const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i)
> +/* Helper function to enumerate mbus codes */
> +int vimc_enum_mbus_code(struct v4l2_subdev *sd,
> +			struct v4l2_subdev_pad_config *cfg,
> +			struct v4l2_subdev_mbus_code_enum *code)
>   {
> -	if (i >= ARRAY_SIZE(vimc_pix_map_list))
> -		return NULL;
> -
> -	return &vimc_pix_map_list[i];
> -}
> -EXPORT_SYMBOL_GPL(vimc_pix_map_by_index);
> -
> -const struct vimc_pix_map *vimc_pix_map_by_code(u32 code)
> -{
> -	unsigned int i;
> -
> -	for (i = 0; i < ARRAY_SIZE(vimc_pix_map_list); i++) {
> -		if (vimc_pix_map_list[i].code == code)
> -			return &vimc_pix_map_list[i];
> -	}
> -	return NULL;
> -}
> -EXPORT_SYMBOL_GPL(vimc_pix_map_by_code);
> -
> -const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat)
> -{
> -	unsigned int i;
> +	if (code->index >= ARRAY_SIZE(vimc_mbus_list))
> +		return -EINVAL;
>   
> -	for (i = 0; i < ARRAY_SIZE(vimc_pix_map_list); i++) {
> -		if (vimc_pix_map_list[i].pixelformat == pixelformat)
> -			return &vimc_pix_map_list[i];
> -	}
> -	return NULL;
> +	code->code = vimc_mbus_list[code->index];
> +	return 0;
>   }
> -EXPORT_SYMBOL_GPL(vimc_pix_map_by_pixelformat);
> +EXPORT_SYMBOL_GPL(vimc_enum_mbus_code);
>   
>   /* Helper function to allocate and initialize pads */
>   struct media_pad *vimc_pads_init(u16 num_pads, const unsigned long *pads_flag)
> @@ -277,15 +212,13 @@ static int vimc_get_mbus_format(struct media_pad *pad,
>   							 struct video_device,
>   							 entity);
>   		struct vimc_ent_device *ved = video_get_drvdata(vdev);
> -		const struct vimc_pix_map *vpix;
>   		struct v4l2_pix_format vdev_fmt;
>   
>   		if (!ved->vdev_get_format)
>   			return -ENOIOCTLCMD;
>   
>   		ved->vdev_get_format(ved, &vdev_fmt);
> -		vpix = vimc_pix_map_by_pixelformat(vdev_fmt.pixelformat);
> -		v4l2_fill_mbus_format(&fmt->format, &vdev_fmt, vpix->code);
> +		v4l2_fill_mbus_format(&fmt->format, &vdev_fmt, 0);
>   	} else {
>   		return -EINVAL;
>   	}
> @@ -325,8 +258,12 @@ int vimc_link_validate(struct media_link *link)
>   	/* The width, height and code must match. */
>   	if (source_fmt.format.width != sink_fmt.format.width
>   	    || source_fmt.format.height != sink_fmt.format.height
> -	    || source_fmt.format.code != sink_fmt.format.code)
> +	    || (source_fmt.format.code && sink_fmt.format.code &&
> +		source_fmt.format.code != sink_fmt.format.code)) {
> +		pr_err("vimc: format doesn't match in link %s->%s\n",
> +			link->source->entity->name, link->sink->entity->name);
>   		return -EPIPE;
> +	}
>   
>   	/*
>   	 * The field order must match, or the sink field order must be NONE
> diff --git a/drivers/media/platform/vimc/vimc-common.h b/drivers/media/platform/vimc/vimc-common.h
> index 07987eab988f..30ce902ace4d 100644
> --- a/drivers/media/platform/vimc/vimc-common.h
> +++ b/drivers/media/platform/vimc/vimc-common.h
> @@ -22,6 +22,8 @@
>   #include <media/media-device.h>
>   #include <media/v4l2-device.h>
>   
> +#include "vimc-streamer.h"
> +
>   #define VIMC_PDEV_NAME "vimc"
>   
>   /* VIMC-specific controls */
> @@ -115,12 +117,23 @@ struct vimc_pix_map {
>   struct vimc_ent_device {
>   	struct media_entity *ent;
>   	struct media_pad *pads;
> +	struct vimc_stream *stream;
>   	void * (*process_frame)(struct vimc_ent_device *ved,
>   				const void *frame);
>   	void (*vdev_get_format)(struct vimc_ent_device *ved,
>   			      struct v4l2_pix_format *fmt);
>   };
>   
> +/**
> + * vimc_enum_mbus_code - enumerate mbus codes
> + *
> + * Helper function to be pluged in .enum_mbus_code from
> + * struct v4l2_subdev_pad_ops.
> + */
> +int vimc_enum_mbus_code(struct v4l2_subdev *sd,
> +			struct v4l2_subdev_pad_config *cfg,
> +			struct v4l2_subdev_mbus_code_enum *code);
> +
>   /**
>    * vimc_pads_init - initialize pads
>    *
> diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
> index 20826f209731..51b411739e27 100644
> --- a/drivers/media/platform/vimc/vimc-debayer.c
> +++ b/drivers/media/platform/vimc/vimc-debayer.c
> @@ -26,6 +26,9 @@
>   #include "vimc-common.h"
>   
>   #define VIMC_DEB_DRV_NAME "vimc-debayer"
> +/* This module only supports tranforming a bayer format to V4L2_PIX_FMT_RGB24 */
> +#define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
> +#define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24
>   
>   static unsigned int deb_mean_win_size = 3;
>   module_param(deb_mean_win_size, uint, 0000);
> @@ -44,6 +47,7 @@ enum vimc_deb_rgb_colors {
>   };
>   
>   struct vimc_deb_pix_map {
> +	u32 pixelformat;
>   	u32 code;
>   	enum vimc_deb_rgb_colors order[2][2];
>   };
> @@ -73,61 +77,73 @@ static const struct v4l2_mbus_framefmt sink_fmt_default = {
>   
>   static const struct vimc_deb_pix_map vimc_deb_pix_map_list[] = {
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SBGGR8,
>   		.code = MEDIA_BUS_FMT_SBGGR8_1X8,
>   		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
>   			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SGBRG8,
>   		.code = MEDIA_BUS_FMT_SGBRG8_1X8,
>   		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
>   			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SGRBG8,
>   		.code = MEDIA_BUS_FMT_SGRBG8_1X8,
>   		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
>   			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SRGGB8,
>   		.code = MEDIA_BUS_FMT_SRGGB8_1X8,
>   		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
>   			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SBGGR10,
>   		.code = MEDIA_BUS_FMT_SBGGR10_1X10,
>   		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
>   			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SGBRG10,
>   		.code = MEDIA_BUS_FMT_SGBRG10_1X10,
>   		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
>   			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SGRBG10,
>   		.code = MEDIA_BUS_FMT_SGRBG10_1X10,
>   		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
>   			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SRGGB10,
>   		.code = MEDIA_BUS_FMT_SRGGB10_1X10,
>   		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
>   			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SBGGR12,
>   		.code = MEDIA_BUS_FMT_SBGGR12_1X12,
>   		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
>   			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SGBRG12,
>   		.code = MEDIA_BUS_FMT_SGBRG12_1X12,
>   		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
>   			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SGRBG12,
>   		.code = MEDIA_BUS_FMT_SGRBG12_1X12,
>   		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
>   			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
>   	},
>   	{
> +		.pixelformat = V4L2_PIX_FMT_SRGGB12,
>   		.code = MEDIA_BUS_FMT_SRGGB12_1X12,
>   		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
>   			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
> @@ -168,41 +184,32 @@ static int vimc_deb_enum_mbus_code(struct v4l2_subdev *sd,
>   				   struct v4l2_subdev_pad_config *cfg,
>   				   struct v4l2_subdev_mbus_code_enum *code)
>   {
> -	/* We only support one format for source pads */
> -	if (IS_SRC(code->pad)) {
> -		struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
> -
> -		if (code->index)
> -			return -EINVAL;
> -
> -		code->code = vdeb->src_code;
> -	} else {
> +	/* For the sink pad we only support codes in the map_list */
> +	if (IS_SINK(code->pad)) {
>   		if (code->index >= ARRAY_SIZE(vimc_deb_pix_map_list))
>   			return -EINVAL;
>   
>   		code->code = vimc_deb_pix_map_list[code->index].code;
> +		return 0;
>   	}
>   
> -	return 0;
> +	return vimc_enum_mbus_code(sd, cfg, code);
>   }
>   
>   static int vimc_deb_enum_frame_size(struct v4l2_subdev *sd,
>   				    struct v4l2_subdev_pad_config *cfg,
>   				    struct v4l2_subdev_frame_size_enum *fse)
>   {
> -	struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
> -
>   	if (fse->index)
>   		return -EINVAL;
>   
> +	/* For the sink pad we only support codes in the map_list */
>   	if (IS_SINK(fse->pad)) {
>   		const struct vimc_deb_pix_map *vpix =
>   			vimc_deb_pix_map_by_code(fse->code);
>   
>   		if (!vpix)
>   			return -EINVAL;
> -	} else if (fse->code != vdeb->src_code) {
> -		return -EINVAL;
>   	}
>   
>   	fse->min_width = VIMC_FRAME_MIN_WIDTH;
> @@ -270,11 +277,11 @@ static int vimc_deb_set_fmt(struct v4l2_subdev *sd,
>   
>   	/*
>   	 * Do not change the format of the source pad,
> -	 * it is propagated from the sink
> +	 * it is propagated from the sink (except for the code)
>   	 */
>   	if (IS_SRC(fmt->pad)) {
> +		vdeb->src_code = fmt->format.code;
>   		fmt->format = *sink_fmt;
> -		/* TODO: Add support for other formats */
>   		fmt->format.code = vdeb->src_code;
>   	} else {
>   		/* Set the new format in the sink pad */
> @@ -306,7 +313,7 @@ static const struct v4l2_subdev_pad_ops vimc_deb_pad_ops = {
>   	.set_fmt		= vimc_deb_set_fmt,
>   };
>   
> -static void vimc_deb_set_rgb_mbus_fmt_rgb888_1x24(struct vimc_deb_device *vdeb,
> +static void vimc_deb_set_rgb_pix_rgb24(struct vimc_deb_device *vdeb,
>   						  unsigned int lin,
>   						  unsigned int col,
>   						  unsigned int rgb[3])
> @@ -323,25 +330,38 @@ static int vimc_deb_s_stream(struct v4l2_subdev *sd, int enable)
>   	struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
>   
>   	if (enable) {
> -		const struct vimc_pix_map *vpix;
> +		u32 src_pixelformat = vdeb->ved.stream->producer_pixfmt;
> +		const struct v4l2_format_info *pix_info;
>   		unsigned int frame_size;
>   
>   		if (vdeb->src_frame)
>   			return 0;
>   
> -		/* Calculate the frame size of the source pad */
> -		vpix = vimc_pix_map_by_code(vdeb->src_code);
> -		frame_size = vdeb->sink_fmt.width * vdeb->sink_fmt.height *
> -				vpix->bpp;
> -
> -		/* Save the bytes per pixel of the sink */
> -		vpix = vimc_pix_map_by_code(vdeb->sink_fmt.code);
> -		vdeb->sink_bpp = vpix->bpp;
> +		/* We only support translating bayer to RGB24 */
> +		if (src_pixelformat != V4L2_PIX_FMT_RGB24) {
> +			dev_err(vdeb->dev,
> +				"translating to pixfmt (%s) is not supported\n",
> +				v4l2_get_fourcc_name(src_pixelformat));
> +			return -EINVAL;
> +		}
>   
>   		/* Get the corresponding pixel map from the table */
>   		vdeb->sink_pix_map =
>   			vimc_deb_pix_map_by_code(vdeb->sink_fmt.code);
>   
> +		/* Request bayer format from the pipeline for the sink pad */
> +		vdeb->ved.stream->producer_pixfmt =
> +			vdeb->sink_pix_map->pixelformat;
> +
> +		/* Calculate frame_size of the source */
> +		pix_info = v4l2_format_info(src_pixelformat);
> +		frame_size = vdeb->sink_fmt.width * vdeb->sink_fmt.height *
> +			     pix_info->bpp[0];
> +
> +		/* Get bpp from the sink */
> +		pix_info = v4l2_format_info(vdeb->sink_pix_map->pixelformat);
> +		vdeb->sink_bpp = pix_info->bpp[0];
> +
>   		/*
>   		 * Allocate the frame buffer. Use vmalloc to be able to
>   		 * allocate a large amount of memory
> @@ -542,14 +562,14 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>   
>   	/* Initialize the frame format */
>   	vdeb->sink_fmt = sink_fmt_default;
> +	vdeb->src_code = VIMC_DEB_SRC_MBUS_FMT_DEFAULT;
>   	/*
>   	 * TODO: Add support for more output formats, we only support
> -	 * RGB888 for now
> +	 * RGB24 for now.
>   	 * NOTE: the src format is always the same as the sink, except
>   	 * for the code
>   	 */
> -	vdeb->src_code = MEDIA_BUS_FMT_RGB888_1X24;
> -	vdeb->set_rgb_src = vimc_deb_set_rgb_mbus_fmt_rgb888_1x24;
> +	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
>   
>   	return 0;
>   }
> diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
> index 2028afa4ef7a..245116b1c241 100644
> --- a/drivers/media/platform/vimc/vimc-scaler.c
> +++ b/drivers/media/platform/vimc/vimc-scaler.c
> @@ -35,6 +35,12 @@ MODULE_PARM_DESC(sca_mult, " the image size multiplier");
>   #define IS_SRC(pad)	(pad)
>   #define MAX_ZOOM	8
>   
> +static const u32 vimc_sca_supported_pixftm[] = {
> +	V4L2_PIX_FMT_BGR24,
> +	V4L2_PIX_FMT_RGB24,
> +	V4L2_PIX_FMT_ARGB32,
> +};
> +
>   struct vimc_sca_device {
>   	struct vimc_ent_device ved;
>   	struct v4l2_subdev sd;
> @@ -57,6 +63,16 @@ static const struct v4l2_mbus_framefmt sink_fmt_default = {
>   	.colorspace = V4L2_COLORSPACE_DEFAULT,
>   };
>   
> +static bool vimc_sca_is_pixfmt_supported(u32 pixelformat)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(vimc_sca_supported_pixftm); i++)
> +		if (vimc_sca_supported_pixftm[i] == pixelformat)
> +			return true;
> +	return false;
> +}
> +
>   static int vimc_sca_init_cfg(struct v4l2_subdev *sd,
>   			     struct v4l2_subdev_pad_config *cfg)
>   {
> @@ -76,35 +92,13 @@ static int vimc_sca_init_cfg(struct v4l2_subdev *sd,
>   	return 0;
>   }
>   
> -static int vimc_sca_enum_mbus_code(struct v4l2_subdev *sd,
> -				   struct v4l2_subdev_pad_config *cfg,
> -				   struct v4l2_subdev_mbus_code_enum *code)
> -{
> -	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
> -
> -	/* We don't support bayer format */
> -	if (!vpix || vpix->bayer)
> -		return -EINVAL;
> -
> -	code->code = vpix->code;
> -
> -	return 0;
> -}
> -
>   static int vimc_sca_enum_frame_size(struct v4l2_subdev *sd,
>   				    struct v4l2_subdev_pad_config *cfg,
>   				    struct v4l2_subdev_frame_size_enum *fse)
>   {
> -	const struct vimc_pix_map *vpix;
> -
>   	if (fse->index)
>   		return -EINVAL;
>   
> -	/* Only accept code in the pix map table in non bayer format */
> -	vpix = vimc_pix_map_by_code(fse->code);
> -	if (!vpix || vpix->bayer)
> -		return -EINVAL;
> -
>   	fse->min_width = VIMC_FRAME_MIN_WIDTH;
>   	fse->min_height = VIMC_FRAME_MIN_HEIGHT;
>   
> @@ -141,13 +135,6 @@ static int vimc_sca_get_fmt(struct v4l2_subdev *sd,
>   
>   static void vimc_sca_adjust_sink_fmt(struct v4l2_mbus_framefmt *fmt)
>   {
> -	const struct vimc_pix_map *vpix;
> -
> -	/* Only accept code in the pix map table in non bayer format */
> -	vpix = vimc_pix_map_by_code(fmt->code);
> -	if (!vpix || vpix->bayer)
> -		fmt->code = sink_fmt_default.code;
> -
>   	fmt->width = clamp_t(u32, fmt->width, VIMC_FRAME_MIN_WIDTH,
>   			     VIMC_FRAME_MAX_WIDTH) & ~1;
>   	fmt->height = clamp_t(u32, fmt->height, VIMC_FRAME_MIN_HEIGHT,
> @@ -208,7 +195,7 @@ static int vimc_sca_set_fmt(struct v4l2_subdev *sd,
>   
>   static const struct v4l2_subdev_pad_ops vimc_sca_pad_ops = {
>   	.init_cfg		= vimc_sca_init_cfg,
> -	.enum_mbus_code		= vimc_sca_enum_mbus_code,
> +	.enum_mbus_code		= vimc_enum_mbus_code,
>   	.enum_frame_size	= vimc_sca_enum_frame_size,
>   	.get_fmt		= vimc_sca_get_fmt,
>   	.set_fmt		= vimc_sca_set_fmt,
> @@ -219,15 +206,22 @@ static int vimc_sca_s_stream(struct v4l2_subdev *sd, int enable)
>   	struct vimc_sca_device *vsca = v4l2_get_subdevdata(sd);
>   
>   	if (enable) {
> -		const struct vimc_pix_map *vpix;
> +		u32 pixelformat = vsca->ved.stream->producer_pixfmt;
> +		const struct v4l2_format_info *pix_info;
>   		unsigned int frame_size;
>   
>   		if (vsca->src_frame)
>   			return 0;
>   
> +		if (!vimc_sca_is_pixfmt_supported(pixelformat)) {
> +			dev_err(vsca->dev, "pixfmt (%s) is not supported\n",
> +				v4l2_get_fourcc_name(pixelformat));
> +			return -EINVAL;
> +		}
> +
>   		/* Save the bytes per pixel of the sink */
> -		vpix = vimc_pix_map_by_code(vsca->sink_fmt.code);
> -		vsca->bpp = vpix->bpp;
> +		pix_info = v4l2_format_info(pixelformat);
> +		vsca->bpp = pix_info->bpp[0];
>   
>   		/* Calculate the width in bytes of the src frame */
>   		vsca->src_line_size = vsca->sink_fmt.width *
> diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
> index d7891d3bbeaa..75df96cc5e39 100644
> --- a/drivers/media/platform/vimc/vimc-sensor.c
> +++ b/drivers/media/platform/vimc/vimc-sensor.c
> @@ -65,34 +65,13 @@ static int vimc_sen_init_cfg(struct v4l2_subdev *sd,
>   	return 0;
>   }
>   
> -static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
> -				   struct v4l2_subdev_pad_config *cfg,
> -				   struct v4l2_subdev_mbus_code_enum *code)
> -{
> -	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
> -
> -	if (!vpix)
> -		return -EINVAL;
> -
> -	code->code = vpix->code;
> -
> -	return 0;
> -}
> -
>   static int vimc_sen_enum_frame_size(struct v4l2_subdev *sd,
>   				    struct v4l2_subdev_pad_config *cfg,
>   				    struct v4l2_subdev_frame_size_enum *fse)
>   {
> -	const struct vimc_pix_map *vpix;
> -
>   	if (fse->index)
>   		return -EINVAL;
>   
> -	/* Only accept code in the pix map table */
> -	vpix = vimc_pix_map_by_code(fse->code);
> -	if (!vpix)
> -		return -EINVAL;
> -
>   	fse->min_width = VIMC_FRAME_MIN_WIDTH;
>   	fse->max_width = VIMC_FRAME_MAX_WIDTH;
>   	fse->min_height = VIMC_FRAME_MIN_HEIGHT;
> @@ -117,14 +96,17 @@ static int vimc_sen_get_fmt(struct v4l2_subdev *sd,
>   
>   static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
>   {
> -	const struct vimc_pix_map *vpix =
> -				vimc_pix_map_by_code(vsen->mbus_format.code);
> +	u32 pixelformat = vsen->ved.stream->producer_pixfmt;
> +	const struct v4l2_format_info *pix_info;
> +
> +	pix_info = v4l2_format_info(pixelformat);
>   
>   	tpg_reset_source(&vsen->tpg, vsen->mbus_format.width,
>   			 vsen->mbus_format.height, vsen->mbus_format.field);
> -	tpg_s_bytesperline(&vsen->tpg, 0, vsen->mbus_format.width * vpix->bpp);
> +	tpg_s_bytesperline(&vsen->tpg, 0,
> +			   vsen->mbus_format.width * pix_info->bpp[0]);
>   	tpg_s_buf_height(&vsen->tpg, vsen->mbus_format.height);
> -	tpg_s_fourcc(&vsen->tpg, vpix->pixelformat);
> +	tpg_s_fourcc(&vsen->tpg, pixelformat);
>   	/* TODO: add support for V4L2_FIELD_ALTERNATE */
>   	tpg_s_field(&vsen->tpg, vsen->mbus_format.field, false);
>   	tpg_s_colorspace(&vsen->tpg, vsen->mbus_format.colorspace);
> @@ -135,13 +117,6 @@ static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
>   
>   static void vimc_sen_adjust_fmt(struct v4l2_mbus_framefmt *fmt)
>   {
> -	const struct vimc_pix_map *vpix;
> -
> -	/* Only accept code in the pix map table */
> -	vpix = vimc_pix_map_by_code(fmt->code);
> -	if (!vpix)
> -		fmt->code = fmt_default.code;
> -
>   	fmt->width = clamp_t(u32, fmt->width, VIMC_FRAME_MIN_WIDTH,
>   			     VIMC_FRAME_MAX_WIDTH) & ~1;
>   	fmt->height = clamp_t(u32, fmt->height, VIMC_FRAME_MIN_HEIGHT,
> @@ -193,7 +168,7 @@ static int vimc_sen_set_fmt(struct v4l2_subdev *sd,
>   
>   static const struct v4l2_subdev_pad_ops vimc_sen_pad_ops = {
>   	.init_cfg		= vimc_sen_init_cfg,
> -	.enum_mbus_code		= vimc_sen_enum_mbus_code,
> +	.enum_mbus_code		= vimc_enum_mbus_code,
>   	.enum_frame_size	= vimc_sen_enum_frame_size,
>   	.get_fmt		= vimc_sen_get_fmt,
>   	.set_fmt		= vimc_sen_set_fmt,
> @@ -215,7 +190,8 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)
>   				container_of(sd, struct vimc_sen_device, sd);
>   
>   	if (enable) {
> -		const struct vimc_pix_map *vpix;
> +		u32 pixelformat = vsen->ved.stream->producer_pixfmt;
> +		const struct v4l2_format_info *pix_info;
>   		unsigned int frame_size;
>   
>   		if (vsen->kthread_sen)
> @@ -223,8 +199,8 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)
>   			return 0;
>   
>   		/* Calculate the frame size */
> -		vpix = vimc_pix_map_by_code(vsen->mbus_format.code);
> -		frame_size = vsen->mbus_format.width * vpix->bpp *
> +		pix_info = v4l2_format_info(pixelformat);
> +		frame_size = vsen->mbus_format.width * pix_info->bpp[0] *
>   			     vsen->mbus_format.height;
>   
>   		/*
> diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
> index 5a3bda62fbc8..c19093b6c787 100644
> --- a/drivers/media/platform/vimc/vimc-streamer.c
> +++ b/drivers/media/platform/vimc/vimc-streamer.c
> @@ -52,6 +52,7 @@ static void vimc_streamer_pipeline_terminate(struct vimc_stream *stream)
>   	while (stream->pipe_size) {
>   		stream->pipe_size--;
>   		ved = stream->ved_pipeline[stream->pipe_size];
> +		ved->stream = NULL;
>   		stream->ved_pipeline[stream->pipe_size] = NULL;
>   
>   		if (!is_media_entity_v4l2_subdev(ved->ent))
> @@ -87,6 +88,7 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
>   			return -EINVAL;
>   		}
>   		stream->ved_pipeline[stream->pipe_size++] = ved;
> +		ved->stream = stream;
>   
>   		if (is_media_entity_v4l2_subdev(ved->ent)) {
>   			sd = media_entity_to_v4l2_subdev(ved->ent);
> diff --git a/drivers/media/platform/vimc/vimc-streamer.h b/drivers/media/platform/vimc/vimc-streamer.h
> index a7c5ac5ace4f..2b3667408794 100644
> --- a/drivers/media/platform/vimc/vimc-streamer.h
> +++ b/drivers/media/platform/vimc/vimc-streamer.h
> @@ -25,6 +25,11 @@
>    * processed in the pipeline.
>    * @pipe_size:		size of @ved_pipeline
>    * @kthread:		thread that generates the frames of the stream.
> + * @producer_pixfmt:	the pixel format requested from the pipeline. This must
> + * be set just before calling vimc_streamer_s_stream(ent, 1). This value is
> + * propagated up to the source of the base image (usually a sensor node) and
> + * can be modified by entities during s_stream callback to request a different
> + * format from rest of the pipeline.
>    *
>    * When the user call stream_on in a video device, struct vimc_stream is
>    * used to keep track of all entities and subdevices that generates and
> @@ -35,6 +40,7 @@ struct vimc_stream {
>   	struct vimc_ent_device *ved_pipeline[VIMC_STREAMER_PIPELINE_MAX_SIZE];
>   	unsigned int pipe_size;
>   	struct task_struct *kthread;
> +	u32 producer_pixfmt;
>   };
>   
>   /**

Re: [PATCH 8/8] media: vimc: propagate pixel format in the stream

[Hans Verkuil]

Hi Helen,

On 3/6/19 11:42 PM, Helen Koike wrote:
> Media bus codes were being mapped to pixelformats, which causes a
> limitation on vimc because not all pixelformats can be mapped to media
> bus codes.
> Also, media bus codes are an internal configuration from the device.
> Userspace only assures media bus codes matches between pads and expects
> the image in a given pixelformat. So we can allow almost any media bus
> format to be configured between pads, except for debayer that expects a
> media bus code of type bayer in the sink pad.
> 
> Signed-off-by: Helen Koike <helen.koike@collabora.com>

This patch introduces new failures with v4l2-compliance.

Just load vimc and run 'v4l2-compliance -m /dev/mediaX' and you'll get the
fails.

It's always the same failures:

Sub-Device ioctls (Source Pad 1):
        test Try VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
                fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
                fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
        test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
        test Try VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
        test Active VIDIOC_SUBDEV_ENUM_MBUS_CODE/FRAME_SIZE/FRAME_INTERVAL: OK
                fail: v4l2-test-subdevs.cpp(313): fmt.code == 0 || fmt.code == ~0U
                fail: v4l2-test-subdevs.cpp(356): checkMBusFrameFmt(node, fmt.format)
        test Active VIDIOC_SUBDEV_G/S_FMT: FAIL
        test Active VIDIOC_SUBDEV_G/S_SELECTION/CROP: OK (Not Supported)
        test VIDIOC_SUBDEV_G/S_FRAME_INTERVAL: OK (Not Supported)

The cause seems to be due to changes in vimc_sen_adjust_fmt (and similar functions
in other subdevs) where it no longer checks if the media_bus code is known, and if
not, replaces it with a default code.

Regards,

	Hans

> 
> ---
> 
>  drivers/media/platform/vimc/vimc-capture.c  |  76 +++--
>  drivers/media/platform/vimc/vimc-common.c   | 307 ++++++++------------
>  drivers/media/platform/vimc/vimc-common.h   |  13 +
>  drivers/media/platform/vimc/vimc-debayer.c  |  78 +++--
>  drivers/media/platform/vimc/vimc-scaler.c   |  60 ++--
>  drivers/media/platform/vimc/vimc-sensor.c   |  48 +--
>  drivers/media/platform/vimc/vimc-streamer.c |   2 +
>  drivers/media/platform/vimc/vimc-streamer.h |   6 +
>  8 files changed, 281 insertions(+), 309 deletions(-)
> 
> diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
> index e976a9d6b460..6377974879d7 100644
> --- a/drivers/media/platform/vimc/vimc-capture.c
> +++ b/drivers/media/platform/vimc/vimc-capture.c
> @@ -28,6 +28,32 @@
>  
>  #define VIMC_CAP_DRV_NAME "vimc-capture"
>  
> +static const u32 vimc_cap_supported_pixftm[] = {
> +	V4L2_PIX_FMT_BGR24,
> +	V4L2_PIX_FMT_RGB24,
> +	V4L2_PIX_FMT_ARGB32,
> +	V4L2_PIX_FMT_SBGGR8,
> +	V4L2_PIX_FMT_SGBRG8,
> +	V4L2_PIX_FMT_SGRBG8,
> +	V4L2_PIX_FMT_SRGGB8,
> +	V4L2_PIX_FMT_SBGGR10,
> +	V4L2_PIX_FMT_SGBRG10,
> +	V4L2_PIX_FMT_SGRBG10,
> +	V4L2_PIX_FMT_SRGGB10,
> +	V4L2_PIX_FMT_SBGGR10ALAW8,
> +	V4L2_PIX_FMT_SGBRG10ALAW8,
> +	V4L2_PIX_FMT_SGRBG10ALAW8,
> +	V4L2_PIX_FMT_SRGGB10ALAW8,
> +	V4L2_PIX_FMT_SBGGR10DPCM8,
> +	V4L2_PIX_FMT_SGBRG10DPCM8,
> +	V4L2_PIX_FMT_SGRBG10DPCM8,
> +	V4L2_PIX_FMT_SRGGB10DPCM8,
> +	V4L2_PIX_FMT_SBGGR12,
> +	V4L2_PIX_FMT_SGBRG12,
> +	V4L2_PIX_FMT_SGRBG12,
> +	V4L2_PIX_FMT_SRGGB12,
> +};
> +
>  struct vimc_cap_device {
>  	struct vimc_ent_device ved;
>  	struct video_device vdev;
> @@ -101,29 +127,25 @@ static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
>  				    struct v4l2_format *f)
>  {
>  	struct v4l2_pix_format *format = &f->fmt.pix;
> -	const struct vimc_pix_map *vpix;
>  
>  	format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
>  				VIMC_FRAME_MAX_WIDTH) & ~1;
>  	format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
>  				 VIMC_FRAME_MAX_HEIGHT) & ~1;
>  
> -	/* Don't accept a pixelformat that is not on the table */
> -	vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
> -	if (!vpix) {
> -		format->pixelformat = fmt_default.pixelformat;
> -		vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
> -	}
> -	/* TODO: Add support for custom bytesperline values */
> -	format->bytesperline = format->width * vpix->bpp;
> -	format->sizeimage = format->bytesperline * format->height;
> +	vimc_colorimetry_clamp(format);
>  
>  	if (format->field == V4L2_FIELD_ANY)
>  		format->field = fmt_default.field;
>  
> -	vimc_colorimetry_clamp(format);
> +	/* TODO: Add support for custom bytesperline values */
>  
> -	return 0;
> +	/* Don't accept a pixelformat that is not on the table */
> +	if (!v4l2_format_info(format->pixelformat))
> +		format->pixelformat = fmt_default.pixelformat;
> +
> +	return v4l2_fill_pixfmt(format, format->pixelformat,
> +				format->width, format->height);
>  }
>  
>  static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
> @@ -159,27 +181,31 @@ static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
>  static int vimc_cap_enum_fmt_vid_cap(struct file *file, void *priv,
>  				     struct v4l2_fmtdesc *f)
>  {
> -	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(f->index);
> -
> -	if (!vpix)
> +	if (f->index >= ARRAY_SIZE(vimc_cap_supported_pixftm))
>  		return -EINVAL;
>  
> -	f->pixelformat = vpix->pixelformat;
> +	f->pixelformat = vimc_cap_supported_pixftm[f->index];
>  
>  	return 0;
>  }
>  
> +static bool vimc_cap_is_pixfmt_supported(u32 pixelformat)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(vimc_cap_supported_pixftm); i++)
> +		if (vimc_cap_supported_pixftm[i] == pixelformat)
> +			return true;
> +	return false;
> +}
> +
>  static int vimc_cap_enum_framesizes(struct file *file, void *fh,
>  				    struct v4l2_frmsizeenum *fsize)
>  {
> -	const struct vimc_pix_map *vpix;
> -
>  	if (fsize->index)
>  		return -EINVAL;
>  
> -	/* Only accept code in the pix map table */
> -	vpix = vimc_pix_map_by_code(fsize->pixel_format);
> -	if (!vpix)
> +	if (!vimc_cap_is_pixfmt_supported(fsize->pixel_format))
>  		return -EINVAL;
>  
>  	fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
> @@ -253,6 +279,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
>  		return ret;
>  	}
>  
> +	vcap->stream.producer_pixfmt = vcap->format.pixelformat;
>  	ret = vimc_streamer_s_stream(&vcap->stream, &vcap->ved, 1);
>  	if (ret) {
>  		media_pipeline_stop(entity);
> @@ -403,7 +430,6 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>  {
>  	struct v4l2_device *v4l2_dev = master_data;
>  	struct vimc_platform_data *pdata = comp->platform_data;
> -	const struct vimc_pix_map *vpix;
>  	struct vimc_cap_device *vcap;
>  	struct video_device *vdev;
>  	struct vb2_queue *q;
> @@ -458,10 +484,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>  
>  	/* Set default frame format */
>  	vcap->format = fmt_default;
> -	vpix = vimc_pix_map_by_pixelformat(vcap->format.pixelformat);
> -	vcap->format.bytesperline = vcap->format.width * vpix->bpp;
> -	vcap->format.sizeimage = vcap->format.bytesperline *
> -				 vcap->format.height;
> +	v4l2_fill_pixfmt(&vcap->format, vcap->format.pixelformat,
> +			 vcap->format.width, vcap->format.height);
>  
>  	/* Fill the vimc_ent_device struct */
>  	vcap->ved.ent = &vcap->vdev.entity;
> diff --git a/drivers/media/platform/vimc/vimc-common.c b/drivers/media/platform/vimc/vimc-common.c
> index 0adbfd8fd26d..b07bbf0564b9 100644
> --- a/drivers/media/platform/vimc/vimc-common.c
> +++ b/drivers/media/platform/vimc/vimc-common.c
> @@ -20,192 +20,127 @@
>  
>  #include "vimc-common.h"
>  
> -/*
> - * NOTE: non-bayer formats need to come first (necessary for enum_mbus_code
> - * in the scaler)
> - */
> -static const struct vimc_pix_map vimc_pix_map_list[] = {
> -	/* TODO: add all missing formats */
> -
> -	/* RGB formats */
> -	{
> -		.code = MEDIA_BUS_FMT_BGR888_1X24,
> -		.pixelformat = V4L2_PIX_FMT_BGR24,
> -		.bpp = 3,
> -		.bayer = false,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_RGB888_1X24,
> -		.pixelformat = V4L2_PIX_FMT_RGB24,
> -		.bpp = 3,
> -		.bayer = false,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_ARGB8888_1X32,
> -		.pixelformat = V4L2_PIX_FMT_ARGB32,
> -		.bpp = 4,
> -		.bayer = false,
> -	},
> -
> -	/* Bayer formats */
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB10_1X10,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB10,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -
> -	/* 10bit raw bayer a-law compressed to 8 bits */
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB10_ALAW8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB10ALAW8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -
> -	/* 10bit raw bayer DPCM compressed to 8 bits */
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB10_DPCM8_1X8,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB10DPCM8,
> -		.bpp = 1,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SBGGR12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SBGGR12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGBRG12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SGBRG12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SGRBG12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SGRBG12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> -	{
> -		.code = MEDIA_BUS_FMT_SRGGB12_1X12,
> -		.pixelformat = V4L2_PIX_FMT_SRGGB12,
> -		.bpp = 2,
> -		.bayer = true,
> -	},
> +const u32 vimc_mbus_list[] = {
> +	MEDIA_BUS_FMT_FIXED,
> +	MEDIA_BUS_FMT_RGB444_1X12,
> +	MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE,
> +	MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
> +	MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_RGB565_1X16,
> +	MEDIA_BUS_FMT_BGR565_2X8_BE,
> +	MEDIA_BUS_FMT_BGR565_2X8_LE,
> +	MEDIA_BUS_FMT_RGB565_2X8_BE,
> +	MEDIA_BUS_FMT_RGB565_2X8_LE,
> +	MEDIA_BUS_FMT_RGB666_1X18,
> +	MEDIA_BUS_FMT_RBG888_1X24,
> +	MEDIA_BUS_FMT_RGB666_1X24_CPADHI,
> +	MEDIA_BUS_FMT_RGB666_1X7X3_SPWG,
> +	MEDIA_BUS_FMT_BGR888_1X24,
> +	MEDIA_BUS_FMT_GBR888_1X24,
> +	MEDIA_BUS_FMT_RGB888_1X24,
> +	MEDIA_BUS_FMT_RGB888_2X12_BE,
> +	MEDIA_BUS_FMT_RGB888_2X12_LE,
> +	MEDIA_BUS_FMT_RGB888_1X7X4_SPWG,
> +	MEDIA_BUS_FMT_RGB888_1X7X4_JEIDA,
> +	MEDIA_BUS_FMT_ARGB8888_1X32,
> +	MEDIA_BUS_FMT_RGB888_1X32_PADHI,
> +	MEDIA_BUS_FMT_RGB101010_1X30,
> +	MEDIA_BUS_FMT_RGB121212_1X36,
> +	MEDIA_BUS_FMT_RGB161616_1X48,
> +	MEDIA_BUS_FMT_Y8_1X8,
> +	MEDIA_BUS_FMT_UV8_1X8,
> +	MEDIA_BUS_FMT_UYVY8_1_5X8,
> +	MEDIA_BUS_FMT_VYUY8_1_5X8,
> +	MEDIA_BUS_FMT_YUYV8_1_5X8,
> +	MEDIA_BUS_FMT_YVYU8_1_5X8,
> +	MEDIA_BUS_FMT_UYVY8_2X8,
> +	MEDIA_BUS_FMT_VYUY8_2X8,
> +	MEDIA_BUS_FMT_YUYV8_2X8,
> +	MEDIA_BUS_FMT_YVYU8_2X8,
> +	MEDIA_BUS_FMT_Y10_1X10,
> +	MEDIA_BUS_FMT_Y10_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_UYVY10_2X10,
> +	MEDIA_BUS_FMT_VYUY10_2X10,
> +	MEDIA_BUS_FMT_YUYV10_2X10,
> +	MEDIA_BUS_FMT_YVYU10_2X10,
> +	MEDIA_BUS_FMT_Y12_1X12,
> +	MEDIA_BUS_FMT_UYVY12_2X12,
> +	MEDIA_BUS_FMT_VYUY12_2X12,
> +	MEDIA_BUS_FMT_YUYV12_2X12,
> +	MEDIA_BUS_FMT_YVYU12_2X12,
> +	MEDIA_BUS_FMT_UYVY8_1X16,
> +	MEDIA_BUS_FMT_VYUY8_1X16,
> +	MEDIA_BUS_FMT_YUYV8_1X16,
> +	MEDIA_BUS_FMT_YVYU8_1X16,
> +	MEDIA_BUS_FMT_YDYUYDYV8_1X16,
> +	MEDIA_BUS_FMT_UYVY10_1X20,
> +	MEDIA_BUS_FMT_VYUY10_1X20,
> +	MEDIA_BUS_FMT_YUYV10_1X20,
> +	MEDIA_BUS_FMT_YVYU10_1X20,
> +	MEDIA_BUS_FMT_VUY8_1X24,
> +	MEDIA_BUS_FMT_YUV8_1X24,
> +	MEDIA_BUS_FMT_UYYVYY8_0_5X24,
> +	MEDIA_BUS_FMT_UYVY12_1X24,
> +	MEDIA_BUS_FMT_VYUY12_1X24,
> +	MEDIA_BUS_FMT_YUYV12_1X24,
> +	MEDIA_BUS_FMT_YVYU12_1X24,
> +	MEDIA_BUS_FMT_YUV10_1X30,
> +	MEDIA_BUS_FMT_UYYVYY10_0_5X30,
> +	MEDIA_BUS_FMT_AYUV8_1X32,
> +	MEDIA_BUS_FMT_UYYVYY12_0_5X36,
> +	MEDIA_BUS_FMT_YUV12_1X36,
> +	MEDIA_BUS_FMT_YUV16_1X48,
> +	MEDIA_BUS_FMT_UYYVYY16_0_5X48,
> +	MEDIA_BUS_FMT_SBGGR8_1X8,
> +	MEDIA_BUS_FMT_SGBRG8_1X8,
> +	MEDIA_BUS_FMT_SGRBG8_1X8,
> +	MEDIA_BUS_FMT_SRGGB8_1X8,
> +	MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SRGGB10_ALAW8_1X8,
> +	MEDIA_BUS_FMT_SBGGR10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SGBRG10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SGRBG10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SRGGB10_DPCM8_1X8,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_LE,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADLO_BE,
> +	MEDIA_BUS_FMT_SBGGR10_2X8_PADLO_LE,
> +	MEDIA_BUS_FMT_SBGGR10_1X10,
> +	MEDIA_BUS_FMT_SGBRG10_1X10,
> +	MEDIA_BUS_FMT_SGRBG10_1X10,
> +	MEDIA_BUS_FMT_SRGGB10_1X10,
> +	MEDIA_BUS_FMT_SBGGR12_1X12,
> +	MEDIA_BUS_FMT_SGBRG12_1X12,
> +	MEDIA_BUS_FMT_SGRBG12_1X12,
> +	MEDIA_BUS_FMT_SRGGB12_1X12,
> +	MEDIA_BUS_FMT_SBGGR14_1X14,
> +	MEDIA_BUS_FMT_SGBRG14_1X14,
> +	MEDIA_BUS_FMT_SGRBG14_1X14,
> +	MEDIA_BUS_FMT_SRGGB14_1X14,
> +	MEDIA_BUS_FMT_SBGGR16_1X16,
> +	MEDIA_BUS_FMT_SGBRG16_1X16,
> +	MEDIA_BUS_FMT_SGRBG16_1X16,
> +	MEDIA_BUS_FMT_SRGGB16_1X16,
> +	MEDIA_BUS_FMT_JPEG_1X8,
> +	MEDIA_BUS_FMT_S5C_UYVY_JPEG_1X8,
> +	MEDIA_BUS_FMT_AHSV8888_1X32,
>  };
>  
> -const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i)
> +/* Helper function to enumerate mbus codes */
> +int vimc_enum_mbus_code(struct v4l2_subdev *sd,
> +			struct v4l2_subdev_pad_config *cfg,
> +			struct v4l2_subdev_mbus_code_enum *code)
>  {
> -	if (i >= ARRAY_SIZE(vimc_pix_map_list))
> -		return NULL;
> -
> -	return &vimc_pix_map_list[i];
> -}
> -EXPORT_SYMBOL_GPL(vimc_pix_map_by_index);
> -
> -const struct vimc_pix_map *vimc_pix_map_by_code(u32 code)
> -{
> -	unsigned int i;
> -
> -	for (i = 0; i < ARRAY_SIZE(vimc_pix_map_list); i++) {
> -		if (vimc_pix_map_list[i].code == code)
> -			return &vimc_pix_map_list[i];
> -	}
> -	return NULL;
> -}
> -EXPORT_SYMBOL_GPL(vimc_pix_map_by_code);
> -
> -const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat)
> -{
> -	unsigned int i;
> +	if (code->index >= ARRAY_SIZE(vimc_mbus_list))
> +		return -EINVAL;
>  
> -	for (i = 0; i < ARRAY_SIZE(vimc_pix_map_list); i++) {
> -		if (vimc_pix_map_list[i].pixelformat == pixelformat)
> -			return &vimc_pix_map_list[i];
> -	}
> -	return NULL;
> +	code->code = vimc_mbus_list[code->index];
> +	return 0;
>  }
> -EXPORT_SYMBOL_GPL(vimc_pix_map_by_pixelformat);
> +EXPORT_SYMBOL_GPL(vimc_enum_mbus_code);
>  
>  /* Helper function to allocate and initialize pads */
>  struct media_pad *vimc_pads_init(u16 num_pads, const unsigned long *pads_flag)
> @@ -277,15 +212,13 @@ static int vimc_get_mbus_format(struct media_pad *pad,
>  							 struct video_device,
>  							 entity);
>  		struct vimc_ent_device *ved = video_get_drvdata(vdev);
> -		const struct vimc_pix_map *vpix;
>  		struct v4l2_pix_format vdev_fmt;
>  
>  		if (!ved->vdev_get_format)
>  			return -ENOIOCTLCMD;
>  
>  		ved->vdev_get_format(ved, &vdev_fmt);
> -		vpix = vimc_pix_map_by_pixelformat(vdev_fmt.pixelformat);
> -		v4l2_fill_mbus_format(&fmt->format, &vdev_fmt, vpix->code);
> +		v4l2_fill_mbus_format(&fmt->format, &vdev_fmt, 0);
>  	} else {
>  		return -EINVAL;
>  	}
> @@ -325,8 +258,12 @@ int vimc_link_validate(struct media_link *link)
>  	/* The width, height and code must match. */
>  	if (source_fmt.format.width != sink_fmt.format.width
>  	    || source_fmt.format.height != sink_fmt.format.height
> -	    || source_fmt.format.code != sink_fmt.format.code)
> +	    || (source_fmt.format.code && sink_fmt.format.code &&
> +		source_fmt.format.code != sink_fmt.format.code)) {
> +		pr_err("vimc: format doesn't match in link %s->%s\n",
> +			link->source->entity->name, link->sink->entity->name);
>  		return -EPIPE;
> +	}
>  
>  	/*
>  	 * The field order must match, or the sink field order must be NONE
> diff --git a/drivers/media/platform/vimc/vimc-common.h b/drivers/media/platform/vimc/vimc-common.h
> index 07987eab988f..30ce902ace4d 100644
> --- a/drivers/media/platform/vimc/vimc-common.h
> +++ b/drivers/media/platform/vimc/vimc-common.h
> @@ -22,6 +22,8 @@
>  #include <media/media-device.h>
>  #include <media/v4l2-device.h>
>  
> +#include "vimc-streamer.h"
> +
>  #define VIMC_PDEV_NAME "vimc"
>  
>  /* VIMC-specific controls */
> @@ -115,12 +117,23 @@ struct vimc_pix_map {
>  struct vimc_ent_device {
>  	struct media_entity *ent;
>  	struct media_pad *pads;
> +	struct vimc_stream *stream;
>  	void * (*process_frame)(struct vimc_ent_device *ved,
>  				const void *frame);
>  	void (*vdev_get_format)(struct vimc_ent_device *ved,
>  			      struct v4l2_pix_format *fmt);
>  };
>  
> +/**
> + * vimc_enum_mbus_code - enumerate mbus codes
> + *
> + * Helper function to be pluged in .enum_mbus_code from
> + * struct v4l2_subdev_pad_ops.
> + */
> +int vimc_enum_mbus_code(struct v4l2_subdev *sd,
> +			struct v4l2_subdev_pad_config *cfg,
> +			struct v4l2_subdev_mbus_code_enum *code);
> +
>  /**
>   * vimc_pads_init - initialize pads
>   *
> diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
> index 20826f209731..51b411739e27 100644
> --- a/drivers/media/platform/vimc/vimc-debayer.c
> +++ b/drivers/media/platform/vimc/vimc-debayer.c
> @@ -26,6 +26,9 @@
>  #include "vimc-common.h"
>  
>  #define VIMC_DEB_DRV_NAME "vimc-debayer"
> +/* This module only supports tranforming a bayer format to V4L2_PIX_FMT_RGB24 */
> +#define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
> +#define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24
>  
>  static unsigned int deb_mean_win_size = 3;
>  module_param(deb_mean_win_size, uint, 0000);
> @@ -44,6 +47,7 @@ enum vimc_deb_rgb_colors {
>  };
>  
>  struct vimc_deb_pix_map {
> +	u32 pixelformat;
>  	u32 code;
>  	enum vimc_deb_rgb_colors order[2][2];
>  };
> @@ -73,61 +77,73 @@ static const struct v4l2_mbus_framefmt sink_fmt_default = {
>  
>  static const struct vimc_deb_pix_map vimc_deb_pix_map_list[] = {
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SBGGR8,
>  		.code = MEDIA_BUS_FMT_SBGGR8_1X8,
>  		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
>  			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SGBRG8,
>  		.code = MEDIA_BUS_FMT_SGBRG8_1X8,
>  		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
>  			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SGRBG8,
>  		.code = MEDIA_BUS_FMT_SGRBG8_1X8,
>  		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
>  			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SRGGB8,
>  		.code = MEDIA_BUS_FMT_SRGGB8_1X8,
>  		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
>  			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SBGGR10,
>  		.code = MEDIA_BUS_FMT_SBGGR10_1X10,
>  		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
>  			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SGBRG10,
>  		.code = MEDIA_BUS_FMT_SGBRG10_1X10,
>  		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
>  			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SGRBG10,
>  		.code = MEDIA_BUS_FMT_SGRBG10_1X10,
>  		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
>  			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SRGGB10,
>  		.code = MEDIA_BUS_FMT_SRGGB10_1X10,
>  		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
>  			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SBGGR12,
>  		.code = MEDIA_BUS_FMT_SBGGR12_1X12,
>  		.order = { { VIMC_DEB_BLUE, VIMC_DEB_GREEN },
>  			   { VIMC_DEB_GREEN, VIMC_DEB_RED } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SGBRG12,
>  		.code = MEDIA_BUS_FMT_SGBRG12_1X12,
>  		.order = { { VIMC_DEB_GREEN, VIMC_DEB_BLUE },
>  			   { VIMC_DEB_RED, VIMC_DEB_GREEN } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SGRBG12,
>  		.code = MEDIA_BUS_FMT_SGRBG12_1X12,
>  		.order = { { VIMC_DEB_GREEN, VIMC_DEB_RED },
>  			   { VIMC_DEB_BLUE, VIMC_DEB_GREEN } }
>  	},
>  	{
> +		.pixelformat = V4L2_PIX_FMT_SRGGB12,
>  		.code = MEDIA_BUS_FMT_SRGGB12_1X12,
>  		.order = { { VIMC_DEB_RED, VIMC_DEB_GREEN },
>  			   { VIMC_DEB_GREEN, VIMC_DEB_BLUE } }
> @@ -168,41 +184,32 @@ static int vimc_deb_enum_mbus_code(struct v4l2_subdev *sd,
>  				   struct v4l2_subdev_pad_config *cfg,
>  				   struct v4l2_subdev_mbus_code_enum *code)
>  {
> -	/* We only support one format for source pads */
> -	if (IS_SRC(code->pad)) {
> -		struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
> -
> -		if (code->index)
> -			return -EINVAL;
> -
> -		code->code = vdeb->src_code;
> -	} else {
> +	/* For the sink pad we only support codes in the map_list */
> +	if (IS_SINK(code->pad)) {
>  		if (code->index >= ARRAY_SIZE(vimc_deb_pix_map_list))
>  			return -EINVAL;
>  
>  		code->code = vimc_deb_pix_map_list[code->index].code;
> +		return 0;
>  	}
>  
> -	return 0;
> +	return vimc_enum_mbus_code(sd, cfg, code);
>  }
>  
>  static int vimc_deb_enum_frame_size(struct v4l2_subdev *sd,
>  				    struct v4l2_subdev_pad_config *cfg,
>  				    struct v4l2_subdev_frame_size_enum *fse)
>  {
> -	struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
> -
>  	if (fse->index)
>  		return -EINVAL;
>  
> +	/* For the sink pad we only support codes in the map_list */
>  	if (IS_SINK(fse->pad)) {
>  		const struct vimc_deb_pix_map *vpix =
>  			vimc_deb_pix_map_by_code(fse->code);
>  
>  		if (!vpix)
>  			return -EINVAL;
> -	} else if (fse->code != vdeb->src_code) {
> -		return -EINVAL;
>  	}
>  
>  	fse->min_width = VIMC_FRAME_MIN_WIDTH;
> @@ -270,11 +277,11 @@ static int vimc_deb_set_fmt(struct v4l2_subdev *sd,
>  
>  	/*
>  	 * Do not change the format of the source pad,
> -	 * it is propagated from the sink
> +	 * it is propagated from the sink (except for the code)
>  	 */
>  	if (IS_SRC(fmt->pad)) {
> +		vdeb->src_code = fmt->format.code;
>  		fmt->format = *sink_fmt;
> -		/* TODO: Add support for other formats */
>  		fmt->format.code = vdeb->src_code;
>  	} else {
>  		/* Set the new format in the sink pad */
> @@ -306,7 +313,7 @@ static const struct v4l2_subdev_pad_ops vimc_deb_pad_ops = {
>  	.set_fmt		= vimc_deb_set_fmt,
>  };
>  
> -static void vimc_deb_set_rgb_mbus_fmt_rgb888_1x24(struct vimc_deb_device *vdeb,
> +static void vimc_deb_set_rgb_pix_rgb24(struct vimc_deb_device *vdeb,
>  						  unsigned int lin,
>  						  unsigned int col,
>  						  unsigned int rgb[3])
> @@ -323,25 +330,38 @@ static int vimc_deb_s_stream(struct v4l2_subdev *sd, int enable)
>  	struct vimc_deb_device *vdeb = v4l2_get_subdevdata(sd);
>  
>  	if (enable) {
> -		const struct vimc_pix_map *vpix;
> +		u32 src_pixelformat = vdeb->ved.stream->producer_pixfmt;
> +		const struct v4l2_format_info *pix_info;
>  		unsigned int frame_size;
>  
>  		if (vdeb->src_frame)
>  			return 0;
>  
> -		/* Calculate the frame size of the source pad */
> -		vpix = vimc_pix_map_by_code(vdeb->src_code);
> -		frame_size = vdeb->sink_fmt.width * vdeb->sink_fmt.height *
> -				vpix->bpp;
> -
> -		/* Save the bytes per pixel of the sink */
> -		vpix = vimc_pix_map_by_code(vdeb->sink_fmt.code);
> -		vdeb->sink_bpp = vpix->bpp;
> +		/* We only support translating bayer to RGB24 */
> +		if (src_pixelformat != V4L2_PIX_FMT_RGB24) {
> +			dev_err(vdeb->dev,
> +				"translating to pixfmt (%s) is not supported\n",
> +				v4l2_get_fourcc_name(src_pixelformat));
> +			return -EINVAL;
> +		}
>  
>  		/* Get the corresponding pixel map from the table */
>  		vdeb->sink_pix_map =
>  			vimc_deb_pix_map_by_code(vdeb->sink_fmt.code);
>  
> +		/* Request bayer format from the pipeline for the sink pad */
> +		vdeb->ved.stream->producer_pixfmt =
> +			vdeb->sink_pix_map->pixelformat;
> +
> +		/* Calculate frame_size of the source */
> +		pix_info = v4l2_format_info(src_pixelformat);
> +		frame_size = vdeb->sink_fmt.width * vdeb->sink_fmt.height *
> +			     pix_info->bpp[0];
> +
> +		/* Get bpp from the sink */
> +		pix_info = v4l2_format_info(vdeb->sink_pix_map->pixelformat);
> +		vdeb->sink_bpp = pix_info->bpp[0];
> +
>  		/*
>  		 * Allocate the frame buffer. Use vmalloc to be able to
>  		 * allocate a large amount of memory
> @@ -542,14 +562,14 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>  
>  	/* Initialize the frame format */
>  	vdeb->sink_fmt = sink_fmt_default;
> +	vdeb->src_code = VIMC_DEB_SRC_MBUS_FMT_DEFAULT;
>  	/*
>  	 * TODO: Add support for more output formats, we only support
> -	 * RGB888 for now
> +	 * RGB24 for now.
>  	 * NOTE: the src format is always the same as the sink, except
>  	 * for the code
>  	 */
> -	vdeb->src_code = MEDIA_BUS_FMT_RGB888_1X24;
> -	vdeb->set_rgb_src = vimc_deb_set_rgb_mbus_fmt_rgb888_1x24;
> +	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
>  
>  	return 0;
>  }
> diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
> index 2028afa4ef7a..245116b1c241 100644
> --- a/drivers/media/platform/vimc/vimc-scaler.c
> +++ b/drivers/media/platform/vimc/vimc-scaler.c
> @@ -35,6 +35,12 @@ MODULE_PARM_DESC(sca_mult, " the image size multiplier");
>  #define IS_SRC(pad)	(pad)
>  #define MAX_ZOOM	8
>  
> +static const u32 vimc_sca_supported_pixftm[] = {
> +	V4L2_PIX_FMT_BGR24,
> +	V4L2_PIX_FMT_RGB24,
> +	V4L2_PIX_FMT_ARGB32,
> +};
> +
>  struct vimc_sca_device {
>  	struct vimc_ent_device ved;
>  	struct v4l2_subdev sd;
> @@ -57,6 +63,16 @@ static const struct v4l2_mbus_framefmt sink_fmt_default = {
>  	.colorspace = V4L2_COLORSPACE_DEFAULT,
>  };
>  
> +static bool vimc_sca_is_pixfmt_supported(u32 pixelformat)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < ARRAY_SIZE(vimc_sca_supported_pixftm); i++)
> +		if (vimc_sca_supported_pixftm[i] == pixelformat)
> +			return true;
> +	return false;
> +}
> +
>  static int vimc_sca_init_cfg(struct v4l2_subdev *sd,
>  			     struct v4l2_subdev_pad_config *cfg)
>  {
> @@ -76,35 +92,13 @@ static int vimc_sca_init_cfg(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> -static int vimc_sca_enum_mbus_code(struct v4l2_subdev *sd,
> -				   struct v4l2_subdev_pad_config *cfg,
> -				   struct v4l2_subdev_mbus_code_enum *code)
> -{
> -	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
> -
> -	/* We don't support bayer format */
> -	if (!vpix || vpix->bayer)
> -		return -EINVAL;
> -
> -	code->code = vpix->code;
> -
> -	return 0;
> -}
> -
>  static int vimc_sca_enum_frame_size(struct v4l2_subdev *sd,
>  				    struct v4l2_subdev_pad_config *cfg,
>  				    struct v4l2_subdev_frame_size_enum *fse)
>  {
> -	const struct vimc_pix_map *vpix;
> -
>  	if (fse->index)
>  		return -EINVAL;
>  
> -	/* Only accept code in the pix map table in non bayer format */
> -	vpix = vimc_pix_map_by_code(fse->code);
> -	if (!vpix || vpix->bayer)
> -		return -EINVAL;
> -
>  	fse->min_width = VIMC_FRAME_MIN_WIDTH;
>  	fse->min_height = VIMC_FRAME_MIN_HEIGHT;
>  
> @@ -141,13 +135,6 @@ static int vimc_sca_get_fmt(struct v4l2_subdev *sd,
>  
>  static void vimc_sca_adjust_sink_fmt(struct v4l2_mbus_framefmt *fmt)
>  {
> -	const struct vimc_pix_map *vpix;
> -
> -	/* Only accept code in the pix map table in non bayer format */
> -	vpix = vimc_pix_map_by_code(fmt->code);
> -	if (!vpix || vpix->bayer)
> -		fmt->code = sink_fmt_default.code;
> -
>  	fmt->width = clamp_t(u32, fmt->width, VIMC_FRAME_MIN_WIDTH,
>  			     VIMC_FRAME_MAX_WIDTH) & ~1;
>  	fmt->height = clamp_t(u32, fmt->height, VIMC_FRAME_MIN_HEIGHT,
> @@ -208,7 +195,7 @@ static int vimc_sca_set_fmt(struct v4l2_subdev *sd,
>  
>  static const struct v4l2_subdev_pad_ops vimc_sca_pad_ops = {
>  	.init_cfg		= vimc_sca_init_cfg,
> -	.enum_mbus_code		= vimc_sca_enum_mbus_code,
> +	.enum_mbus_code		= vimc_enum_mbus_code,
>  	.enum_frame_size	= vimc_sca_enum_frame_size,
>  	.get_fmt		= vimc_sca_get_fmt,
>  	.set_fmt		= vimc_sca_set_fmt,
> @@ -219,15 +206,22 @@ static int vimc_sca_s_stream(struct v4l2_subdev *sd, int enable)
>  	struct vimc_sca_device *vsca = v4l2_get_subdevdata(sd);
>  
>  	if (enable) {
> -		const struct vimc_pix_map *vpix;
> +		u32 pixelformat = vsca->ved.stream->producer_pixfmt;
> +		const struct v4l2_format_info *pix_info;
>  		unsigned int frame_size;
>  
>  		if (vsca->src_frame)
>  			return 0;
>  
> +		if (!vimc_sca_is_pixfmt_supported(pixelformat)) {
> +			dev_err(vsca->dev, "pixfmt (%s) is not supported\n",
> +				v4l2_get_fourcc_name(pixelformat));
> +			return -EINVAL;
> +		}
> +
>  		/* Save the bytes per pixel of the sink */
> -		vpix = vimc_pix_map_by_code(vsca->sink_fmt.code);
> -		vsca->bpp = vpix->bpp;
> +		pix_info = v4l2_format_info(pixelformat);
> +		vsca->bpp = pix_info->bpp[0];
>  
>  		/* Calculate the width in bytes of the src frame */
>  		vsca->src_line_size = vsca->sink_fmt.width *
> diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
> index d7891d3bbeaa..75df96cc5e39 100644
> --- a/drivers/media/platform/vimc/vimc-sensor.c
> +++ b/drivers/media/platform/vimc/vimc-sensor.c
> @@ -65,34 +65,13 @@ static int vimc_sen_init_cfg(struct v4l2_subdev *sd,
>  	return 0;
>  }
>  
> -static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
> -				   struct v4l2_subdev_pad_config *cfg,
> -				   struct v4l2_subdev_mbus_code_enum *code)
> -{
> -	const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
> -
> -	if (!vpix)
> -		return -EINVAL;
> -
> -	code->code = vpix->code;
> -
> -	return 0;
> -}
> -
>  static int vimc_sen_enum_frame_size(struct v4l2_subdev *sd,
>  				    struct v4l2_subdev_pad_config *cfg,
>  				    struct v4l2_subdev_frame_size_enum *fse)
>  {
> -	const struct vimc_pix_map *vpix;
> -
>  	if (fse->index)
>  		return -EINVAL;
>  
> -	/* Only accept code in the pix map table */
> -	vpix = vimc_pix_map_by_code(fse->code);
> -	if (!vpix)
> -		return -EINVAL;
> -
>  	fse->min_width = VIMC_FRAME_MIN_WIDTH;
>  	fse->max_width = VIMC_FRAME_MAX_WIDTH;
>  	fse->min_height = VIMC_FRAME_MIN_HEIGHT;
> @@ -117,14 +96,17 @@ static int vimc_sen_get_fmt(struct v4l2_subdev *sd,
>  
>  static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
>  {
> -	const struct vimc_pix_map *vpix =
> -				vimc_pix_map_by_code(vsen->mbus_format.code);
> +	u32 pixelformat = vsen->ved.stream->producer_pixfmt;
> +	const struct v4l2_format_info *pix_info;
> +
> +	pix_info = v4l2_format_info(pixelformat);
>  
>  	tpg_reset_source(&vsen->tpg, vsen->mbus_format.width,
>  			 vsen->mbus_format.height, vsen->mbus_format.field);
> -	tpg_s_bytesperline(&vsen->tpg, 0, vsen->mbus_format.width * vpix->bpp);
> +	tpg_s_bytesperline(&vsen->tpg, 0,
> +			   vsen->mbus_format.width * pix_info->bpp[0]);
>  	tpg_s_buf_height(&vsen->tpg, vsen->mbus_format.height);
> -	tpg_s_fourcc(&vsen->tpg, vpix->pixelformat);
> +	tpg_s_fourcc(&vsen->tpg, pixelformat);
>  	/* TODO: add support for V4L2_FIELD_ALTERNATE */
>  	tpg_s_field(&vsen->tpg, vsen->mbus_format.field, false);
>  	tpg_s_colorspace(&vsen->tpg, vsen->mbus_format.colorspace);
> @@ -135,13 +117,6 @@ static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
>  
>  static void vimc_sen_adjust_fmt(struct v4l2_mbus_framefmt *fmt)
>  {
> -	const struct vimc_pix_map *vpix;
> -
> -	/* Only accept code in the pix map table */
> -	vpix = vimc_pix_map_by_code(fmt->code);
> -	if (!vpix)
> -		fmt->code = fmt_default.code;
> -
>  	fmt->width = clamp_t(u32, fmt->width, VIMC_FRAME_MIN_WIDTH,
>  			     VIMC_FRAME_MAX_WIDTH) & ~1;
>  	fmt->height = clamp_t(u32, fmt->height, VIMC_FRAME_MIN_HEIGHT,
> @@ -193,7 +168,7 @@ static int vimc_sen_set_fmt(struct v4l2_subdev *sd,
>  
>  static const struct v4l2_subdev_pad_ops vimc_sen_pad_ops = {
>  	.init_cfg		= vimc_sen_init_cfg,
> -	.enum_mbus_code		= vimc_sen_enum_mbus_code,
> +	.enum_mbus_code		= vimc_enum_mbus_code,
>  	.enum_frame_size	= vimc_sen_enum_frame_size,
>  	.get_fmt		= vimc_sen_get_fmt,
>  	.set_fmt		= vimc_sen_set_fmt,
> @@ -215,7 +190,8 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)
>  				container_of(sd, struct vimc_sen_device, sd);
>  
>  	if (enable) {
> -		const struct vimc_pix_map *vpix;
> +		u32 pixelformat = vsen->ved.stream->producer_pixfmt;
> +		const struct v4l2_format_info *pix_info;
>  		unsigned int frame_size;
>  
>  		if (vsen->kthread_sen)
> @@ -223,8 +199,8 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int enable)
>  			return 0;
>  
>  		/* Calculate the frame size */
> -		vpix = vimc_pix_map_by_code(vsen->mbus_format.code);
> -		frame_size = vsen->mbus_format.width * vpix->bpp *
> +		pix_info = v4l2_format_info(pixelformat);
> +		frame_size = vsen->mbus_format.width * pix_info->bpp[0] *
>  			     vsen->mbus_format.height;
>  
>  		/*
> diff --git a/drivers/media/platform/vimc/vimc-streamer.c b/drivers/media/platform/vimc/vimc-streamer.c
> index 5a3bda62fbc8..c19093b6c787 100644
> --- a/drivers/media/platform/vimc/vimc-streamer.c
> +++ b/drivers/media/platform/vimc/vimc-streamer.c
> @@ -52,6 +52,7 @@ static void vimc_streamer_pipeline_terminate(struct vimc_stream *stream)
>  	while (stream->pipe_size) {
>  		stream->pipe_size--;
>  		ved = stream->ved_pipeline[stream->pipe_size];
> +		ved->stream = NULL;
>  		stream->ved_pipeline[stream->pipe_size] = NULL;
>  
>  		if (!is_media_entity_v4l2_subdev(ved->ent))
> @@ -87,6 +88,7 @@ static int vimc_streamer_pipeline_init(struct vimc_stream *stream,
>  			return -EINVAL;
>  		}
>  		stream->ved_pipeline[stream->pipe_size++] = ved;
> +		ved->stream = stream;
>  
>  		if (is_media_entity_v4l2_subdev(ved->ent)) {
>  			sd = media_entity_to_v4l2_subdev(ved->ent);
> diff --git a/drivers/media/platform/vimc/vimc-streamer.h b/drivers/media/platform/vimc/vimc-streamer.h
> index a7c5ac5ace4f..2b3667408794 100644
> --- a/drivers/media/platform/vimc/vimc-streamer.h
> +++ b/drivers/media/platform/vimc/vimc-streamer.h
> @@ -25,6 +25,11 @@
>   * processed in the pipeline.
>   * @pipe_size:		size of @ved_pipeline
>   * @kthread:		thread that generates the frames of the stream.
> + * @producer_pixfmt:	the pixel format requested from the pipeline. This must
> + * be set just before calling vimc_streamer_s_stream(ent, 1). This value is
> + * propagated up to the source of the base image (usually a sensor node) and
> + * can be modified by entities during s_stream callback to request a different
> + * format from rest of the pipeline.
>   *
>   * When the user call stream_on in a video device, struct vimc_stream is
>   * used to keep track of all entities and subdevices that generates and
> @@ -35,6 +40,7 @@ struct vimc_stream {
>  	struct vimc_ent_device *ved_pipeline[VIMC_STREAMER_PIPELINE_MAX_SIZE];
>  	unsigned int pipe_size;
>  	struct task_struct *kthread;
> +	u32 producer_pixfmt;
>  };
>  
>  /**
>