[PATCH v5 0/8] Add support for multi-planar formats and 10 bit formats

[PATCH v5 0/8] Add support for multi-planar formats and 10 bit formats

[Satish Kumar Nagireddy]

 The patches are for xilinx v4l. The patcheset enable support to handle multiplanar
 formats and 10 bit formats. Single planar implementation is removed as mplane can
 handle both.

 Patch-set has downstream changes and bug fixes. Added new media bus format
 MEDIA_BUS_FMT_VYYUYY8_1X24, new pixel format V4L2_PIX_FMT_XV15 and rst
 documentation.

Jeffrey Mouroux (1):
  uapi: media: New fourcc code and rst for 10 bit format

Radhey Shyam Pandey (1):
  v4l: xilinx: dma: Remove colorspace check in xvip_dma_verify_format

Rohit Athavale (1):
  xilinx: v4l: dma: Update driver to allow for probe defer

Satish Kumar Nagireddy (4):
  media-bus: uapi: Add YCrCb 420 media bus format and rst
  v4l: xilinx: dma: Update video format descriptor
  v4l: xilinx: dma: Add multi-planar support
  v4l: xilinx: dma: Add support for 10 bit formats

Vishal Sagar (1):
  xilinx: v4l: dma: Terminate DMA when media pipeline fail to start

 Documentation/media/uapi/v4l/pixfmt-xv15.rst    | 134 +++++++++++++++++++
 Documentation/media/uapi/v4l/subdev-formats.rst |  38 +++++-
 Documentation/media/uapi/v4l/yuv-formats.rst    |   1 +
 drivers/media/platform/xilinx/xilinx-dma.c      | 170 +++++++++++++++---------
 drivers/media/platform/xilinx/xilinx-dma.h      |   4 +-
 drivers/media/platform/xilinx/xilinx-vip.c      |  37 ++++--
 drivers/media/platform/xilinx/xilinx-vip.h      |  15 ++-
 drivers/media/platform/xilinx/xilinx-vipp.c     |  16 +--
 include/uapi/linux/media-bus-format.h           |   3 +-
 include/uapi/linux/videodev2.h                  |   1 +
 10 files changed, 333 insertions(+), 86 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-xv15.rst

-- 
2.7.4

[PATCH v5 1/8] v4l: xilinx: dma: Remove colorspace check in xvip_dma_verify_format

[Satish Kumar Nagireddy]

From: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>

In current implementation driver only checks the colorspace
between the last subdev in the pipeline and the connected video node,
the pipeline could be configured with wrong colorspace information
until the very end. It thus makes little sense to check the
colorspace only at the video node. So check can be dropped until
we find a better solution to carry colorspace information
through pipelines and to userspace.

Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
 drivers/media/platform/xilinx/xilinx-dma.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 522cdfd..cb20ada 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -75,8 +75,7 @@ static int xvip_dma_verify_format(struct xvip_dma *dma)
 
 	if (dma->fmtinfo->code != fmt.format.code ||
 	    dma->format.height != fmt.format.height ||
-	    dma->format.width != fmt.format.width ||
-	    dma->format.colorspace != fmt.format.colorspace)
+	    dma->format.width != fmt.format.width)
 		return -EINVAL;
 
 	return 0;
-- 
2.7.4

[PATCH v5 2/8] xilinx: v4l: dma: Update driver to allow for probe defer

[Satish Kumar Nagireddy]

From: Rohit Athavale <rathaval@xilinx.com>

Update xvip_dma_init() to use dma_request_chan(), enabling probe
deferral. Also update the cleanup routine to prevent dereferencing
an ERR_PTR().

Signed-off-by: Rohit Athavale <rathaval@xilinx.com>
Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
 drivers/media/platform/xilinx/xilinx-dma.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index cb20ada..5426efe 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -729,10 +729,13 @@ int xvip_dma_init(struct xvip_composite_device *xdev, struct xvip_dma *dma,
 
 	/* ... and the DMA channel. */
 	snprintf(name, sizeof(name), "port%u", port);
-	dma->dma = dma_request_slave_channel(dma->xdev->dev, name);
-	if (dma->dma == NULL) {
-		dev_err(dma->xdev->dev, "no VDMA channel found\n");
-		ret = -ENODEV;
+	dma->dma = dma_request_chan(dma->xdev->dev, name);
+	if (IS_ERR(dma->dma)) {
+		ret = PTR_ERR(dma->dma);
+		if (ret != -EPROBE_DEFER)
+			dev_err(dma->xdev->dev,
+				"No Video DMA channel found");
+
 		goto error;
 	}
 
@@ -756,7 +759,7 @@ void xvip_dma_cleanup(struct xvip_dma *dma)
 	if (video_is_registered(&dma->video))
 		video_unregister_device(&dma->video);
 
-	if (dma->dma)
+	if (!IS_ERR(dma->dma))
 		dma_release_channel(dma->dma);
 
 	media_entity_cleanup(&dma->video.entity);
-- 
2.7.4

[PATCH v5 3/8] xilinx: v4l: dma: Terminate DMA when media pipeline fail to start

[Satish Kumar Nagireddy]

From: Vishal Sagar <vishal.sagar@xilinx.com>

If an incorrectly configured media pipeline is started, the allocated
dma descriptors aren't freed. This leads to kernel oops when pipeline
is configured correctly and run subsequently.

This patch also replaces dmaengine_terminate_all() with
dmaengine_terminate_sync() as the former one is deprecated.

Signed-off-by: Vishal Sagar <vishal.sagar@xilinx.com>
Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
 drivers/media/platform/xilinx/xilinx-dma.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 5426efe..727dc6e 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -437,6 +437,7 @@ static int xvip_dma_start_streaming(struct vb2_queue *vq, unsigned int count)
 	media_pipeline_stop(&dma->video.entity);
 
 error:
+	dmaengine_terminate_sync(dma->dma);
 	/* Give back all queued buffers to videobuf2. */
 	spin_lock_irq(&dma->queued_lock);
 	list_for_each_entry_safe(buf, nbuf, &dma->queued_bufs, queue) {
@@ -458,7 +459,7 @@ static void xvip_dma_stop_streaming(struct vb2_queue *vq)
 	xvip_pipeline_set_stream(pipe, false);
 
 	/* Stop and reset the DMA engine. */
-	dmaengine_terminate_all(dma->dma);
+	dmaengine_terminate_sync(dma->dma);
 
 	/* Cleanup the pipeline and mark it as being stopped. */
 	xvip_pipeline_cleanup(pipe);
-- 
2.7.4

[PATCH v5 4/8] media-bus: uapi: Add YCrCb 420 media bus format and rst

[Satish Kumar Nagireddy]

This commit adds YUV 420 media bus format. VYYUYY8_1X24
is an approximate way to descrive the pixels sent over
the bus.

This patch also contain rst documentation for media bus format.

Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
Changes in v3:
 - Fixed table alignment issue in rst file. Ensured the output is proper uisng
   'make pdfdocs'

Changes in v2:
 - Added rst documentation for MEDIA_BUS_FMT_VYYUYY8_1X24

 Documentation/media/uapi/v4l/subdev-formats.rst | 38 ++++++++++++++++++++++++-
 include/uapi/linux/media-bus-format.h           |  3 +-
 2 files changed, 39 insertions(+), 2 deletions(-)

diff --git a/Documentation/media/uapi/v4l/subdev-formats.rst b/Documentation/media/uapi/v4l/subdev-formats.rst
index 9fcabe7..904c52b 100644
--- a/Documentation/media/uapi/v4l/subdev-formats.rst
+++ b/Documentation/media/uapi/v4l/subdev-formats.rst
@@ -6640,6 +6640,43 @@ the following codes.
       - u\ :sub:`2`
       - u\ :sub:`1`
       - u\ :sub:`0`
+    * .. _MEDIA-BUS-FMT-VYYUYY8-1X24:
+
+      - MEDIA_BUS_FMT_VYYUYY8_1X24
+      - 0x202c
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      -
+      - v\ :sub:`3`
+      - v\ :sub:`2`
+      - v\ :sub:`1`
+      - v\ :sub:`0`
+      - y\ :sub:`7`
+      - y\ :sub:`6`
+      - y\ :sub:`5`
+      - y\ :sub:`4`
+      - y\ :sub:`3`
+      - y\ :sub:`2`
+      - y\ :sub:`1`
+      - y\ :sub:`0`
+      - u\ :sub:`3`
+      - u\ :sub:`2`
+      - u\ :sub:`1`
+      - u\ :sub:`0`
+      - y\ :sub:`7`
+      - y\ :sub:`6`
+      - y\ :sub:`5`
+      - y\ :sub:`4`
+      - y\ :sub:`3`
+      - y\ :sub:`2`
+      - y\ :sub:`1`
+      - y\ :sub:`0`
     * .. _MEDIA-BUS-FMT-YUV10-1X30:
 
       - MEDIA_BUS_FMT_YUV10_1X30
@@ -7287,7 +7324,6 @@ The following table list existing packed 48bit wide YUV formats.
       - y\ :sub:`1`
       - y\ :sub:`0`
 
-
 .. raw:: latex
 
 	\endgroup
diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
index 9e35117..ade7e9d 100644
--- a/include/uapi/linux/media-bus-format.h
+++ b/include/uapi/linux/media-bus-format.h
@@ -62,7 +62,7 @@
 #define MEDIA_BUS_FMT_RGB121212_1X36		0x1019
 #define MEDIA_BUS_FMT_RGB161616_1X48		0x101a
 
-/* YUV (including grey) - next is	0x202c */
+/* YUV (including grey) - next is	0x202d */
 #define MEDIA_BUS_FMT_Y8_1X8			0x2001
 #define MEDIA_BUS_FMT_UV8_1X8			0x2015
 #define MEDIA_BUS_FMT_UYVY8_1_5X8		0x2002
@@ -106,6 +106,7 @@
 #define MEDIA_BUS_FMT_YUV12_1X36		0x2029
 #define MEDIA_BUS_FMT_YUV16_1X48		0x202a
 #define MEDIA_BUS_FMT_UYYVYY16_0_5X48		0x202b
+#define MEDIA_BUS_FMT_VYYUYY8_1X24		0x202c
 
 /* Bayer - next is	0x3021 */
 #define MEDIA_BUS_FMT_SBGGR8_1X8		0x3001
-- 
2.7.4

[PATCH v5 5/8] v4l: xilinx: dma: Update video format descriptor

[Satish Kumar Nagireddy]

This patch updates video format descriptor to help information
viz., number of planes per color format and chroma sub sampling
factors.

Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
Changes in v5:
 - Added YUV420 10 bit format to video descriptor table

Changes in v4:
 - Introduced bpp (bits per pixel) per plane

 drivers/media/platform/xilinx/xilinx-dma.c | 12 ++++++------
 drivers/media/platform/xilinx/xilinx-vip.c | 18 ++++++++++--------
 drivers/media/platform/xilinx/xilinx-vip.h | 10 ++++++++--
 3 files changed, 24 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 727dc6e..518d572 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -366,7 +366,7 @@ static void xvip_dma_buffer_queue(struct vb2_buffer *vb)
 	}
 
 	dma->xt.frame_size = 1;
-	dma->sgl[0].size = dma->format.width * dma->fmtinfo->bpp;
+	dma->sgl[0].size = dma->format.width * dma->fmtinfo->bpp[0];
 	dma->sgl[0].icg = dma->format.bytesperline - dma->sgl[0].size;
 	dma->xt.numf = dma->format.height;
 
@@ -569,12 +569,12 @@ __xvip_dma_try_format(struct xvip_dma *dma, struct v4l2_pix_format *pix,
 	 * the minimum and maximum values, clamp the requested width and convert
 	 * it back to pixels.
 	 */
-	align = lcm(dma->align, info->bpp);
+	align = lcm(dma->align, info->bpp[0]);
 	min_width = roundup(XVIP_DMA_MIN_WIDTH, align);
 	max_width = rounddown(XVIP_DMA_MAX_WIDTH, align);
-	width = rounddown(pix->width * info->bpp, align);
+	width = rounddown(pix->width * info->bpp[0], align);
 
-	pix->width = clamp(width, min_width, max_width) / info->bpp;
+	pix->width = clamp(width, min_width, max_width) / info->bpp[0];
 	pix->height = clamp(pix->height, XVIP_DMA_MIN_HEIGHT,
 			    XVIP_DMA_MAX_HEIGHT);
 
@@ -582,7 +582,7 @@ __xvip_dma_try_format(struct xvip_dma *dma, struct v4l2_pix_format *pix,
 	 * line value is zero, the module doesn't support user configurable line
 	 * sizes. Override the requested value with the minimum in that case.
 	 */
-	min_bpl = pix->width * info->bpp;
+	min_bpl = pix->width * info->bpp[0];
 	max_bpl = rounddown(XVIP_DMA_MAX_WIDTH, dma->align);
 	bpl = rounddown(pix->bytesperline, dma->align);
 
@@ -676,7 +676,7 @@ int xvip_dma_init(struct xvip_composite_device *xdev, struct xvip_dma *dma,
 	dma->format.field = V4L2_FIELD_NONE;
 	dma->format.width = XVIP_DMA_DEF_WIDTH;
 	dma->format.height = XVIP_DMA_DEF_HEIGHT;
-	dma->format.bytesperline = dma->format.width * dma->fmtinfo->bpp;
+	dma->format.bytesperline = dma->format.width * dma->fmtinfo->bpp[0];
 	dma->format.sizeimage = dma->format.bytesperline * dma->format.height;
 
 	/* Initialize the media entity... */
diff --git a/drivers/media/platform/xilinx/xilinx-vip.c b/drivers/media/platform/xilinx/xilinx-vip.c
index 3112591..fb1a08f 100644
--- a/drivers/media/platform/xilinx/xilinx-vip.c
+++ b/drivers/media/platform/xilinx/xilinx-vip.c
@@ -27,22 +27,24 @@
  */
 
 static const struct xvip_video_format xvip_video_formats[] = {
+	{ XVIP_VF_YUV_420, 8, NULL, MEDIA_BUS_FMT_VYYUYY8_1X24,
+	  {1, 2, 0}, V4L2_PIX_FMT_NV12, 2, 2, 2, "4:2:0, semi-planar, YUV" },
 	{ XVIP_VF_YUV_422, 8, NULL, MEDIA_BUS_FMT_UYVY8_1X16,
-	  2, V4L2_PIX_FMT_YUYV, "4:2:2, packed, YUYV" },
+	  {2, 0, 0}, V4L2_PIX_FMT_YUYV, 1, 2, 1, "4:2:2, packed, YUYV" },
 	{ XVIP_VF_YUV_444, 8, NULL, MEDIA_BUS_FMT_VUY8_1X24,
-	  3, V4L2_PIX_FMT_YUV444, "4:4:4, packed, YUYV" },
+	  {3, 0, 0}, V4L2_PIX_FMT_YUV444, 1, 1, 1, "4:4:4, packed, YUYV" },
 	{ XVIP_VF_RBG, 8, NULL, MEDIA_BUS_FMT_RBG888_1X24,
-	  3, 0, NULL },
+	  {3, 0, 0}, V4L2_PIX_FMT_RGB24, 1, 1, 1, "24-bit RGB" },
 	{ XVIP_VF_MONO_SENSOR, 8, "mono", MEDIA_BUS_FMT_Y8_1X8,
-	  1, V4L2_PIX_FMT_GREY, "Greyscale 8-bit" },
+	  {1, 0, 0}, V4L2_PIX_FMT_GREY, 1, 1, 1, "Greyscale 8-bit" },
 	{ XVIP_VF_MONO_SENSOR, 8, "rggb", MEDIA_BUS_FMT_SRGGB8_1X8,
-	  1, V4L2_PIX_FMT_SGRBG8, "Bayer 8-bit RGGB" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SGRBG8, 1, 1, 1, "Bayer 8-bit RGGB" },
 	{ XVIP_VF_MONO_SENSOR, 8, "grbg", MEDIA_BUS_FMT_SGRBG8_1X8,
-	  1, V4L2_PIX_FMT_SGRBG8, "Bayer 8-bit GRBG" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SGRBG8, 1, 1, 1, "Bayer 8-bit GRBG" },
 	{ XVIP_VF_MONO_SENSOR, 8, "gbrg", MEDIA_BUS_FMT_SGBRG8_1X8,
-	  1, V4L2_PIX_FMT_SGBRG8, "Bayer 8-bit GBRG" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SGBRG8, 1, 1, 1, "Bayer 8-bit GBRG" },
 	{ XVIP_VF_MONO_SENSOR, 8, "bggr", MEDIA_BUS_FMT_SBGGR8_1X8,
-	  1, V4L2_PIX_FMT_SBGGR8, "Bayer 8-bit BGGR" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SBGGR8, 1, 1, 1, "Bayer 8-bit BGGR" },
 };
 
 /**
diff --git a/drivers/media/platform/xilinx/xilinx-vip.h b/drivers/media/platform/xilinx/xilinx-vip.h
index 42fee20..256efa2 100644
--- a/drivers/media/platform/xilinx/xilinx-vip.h
+++ b/drivers/media/platform/xilinx/xilinx-vip.h
@@ -109,8 +109,11 @@ struct xvip_device {
  * @width: AXI4 format width in bits per component
  * @pattern: CFA pattern for Mono/Sensor formats
  * @code: media bus format code
- * @bpp: bytes per pixel (when stored in memory)
+ * @bpp: bytes per pixel is per plane
  * @fourcc: V4L2 pixel format FCC identifier
+ * @num_planes: number of planes w.r.t. color format
+ * @hsub: Horizontal sampling factor of Chroma
+ * @vsub: Vertical sampling factor of Chroma
  * @description: format description, suitable for userspace
  */
 struct xvip_video_format {
@@ -118,8 +121,11 @@ struct xvip_video_format {
 	unsigned int width;
 	const char *pattern;
 	unsigned int code;
-	unsigned int bpp;
+	unsigned int bpp[3];
 	u32 fourcc;
+	u8 num_planes;
+	u8 hsub;
+	u8 vsub;
 	const char *description;
 };
 
-- 
2.7.4

[PATCH v5 6/8] v4l: xilinx: dma: Add multi-planar support

[Satish Kumar Nagireddy]

The current v4l driver supports single plane formats. This patch
adds support to handle multi-planar formats. Driver can handle
both single and multi-planar formats.

Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
Changes in v5:
 - Added default height
 - Corrected sizeimage declaration with u32

Changes in v4:
 - Single plane implementation is removed as multi-plane supports both
 - num_buffers and bpl_factor parameters are removed to have clean
   implementation

 drivers/media/platform/xilinx/xilinx-dma.c  | 150 +++++++++++++++++-----------
 drivers/media/platform/xilinx/xilinx-dma.h  |   4 +-
 drivers/media/platform/xilinx/xilinx-vipp.c |  16 +--
 3 files changed, 104 insertions(+), 66 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 518d572..2ffc276 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -74,8 +74,8 @@ static int xvip_dma_verify_format(struct xvip_dma *dma)
 		return ret == -ENOIOCTLCMD ? -EINVAL : ret;
 
 	if (dma->fmtinfo->code != fmt.format.code ||
-	    dma->format.height != fmt.format.height ||
-	    dma->format.width != fmt.format.width)
+	    dma->format.fmt.pix_mp.width != fmt.format.width ||
+	    dma->format.fmt.pix_mp.height != fmt.format.height)
 		return -EINVAL;
 
 	return 0;
@@ -310,7 +310,8 @@ static void xvip_dma_complete(void *param)
 	buf->buf.field = V4L2_FIELD_NONE;
 	buf->buf.sequence = dma->sequence++;
 	buf->buf.vb2_buf.timestamp = ktime_get_ns();
-	vb2_set_plane_payload(&buf->buf.vb2_buf, 0, dma->format.sizeimage);
+	vb2_set_plane_payload(&buf->buf.vb2_buf, 0,
+			      dma->format.fmt.pix_mp.plane_fmt[0].sizeimage);
 	vb2_buffer_done(&buf->buf.vb2_buf, VB2_BUF_STATE_DONE);
 }
 
@@ -320,13 +321,15 @@ xvip_dma_queue_setup(struct vb2_queue *vq,
 		     unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct xvip_dma *dma = vb2_get_drv_priv(vq);
+	u32 sizeimage;
 
 	/* Make sure the image size is large enough. */
+	sizeimage = dma->format.fmt.pix_mp.plane_fmt[0].sizeimage;
 	if (*nplanes)
-		return sizes[0] < dma->format.sizeimage ? -EINVAL : 0;
+		return sizes[0] < sizeimage ? -EINVAL : 0;
 
 	*nplanes = 1;
-	sizes[0] = dma->format.sizeimage;
+	sizes[0] = sizeimage;
 
 	return 0;
 }
@@ -350,8 +353,9 @@ static void xvip_dma_buffer_queue(struct vb2_buffer *vb)
 	struct dma_async_tx_descriptor *desc;
 	dma_addr_t addr = vb2_dma_contig_plane_dma_addr(vb, 0);
 	u32 flags;
+	struct v4l2_pix_format_mplane *pix_mp = &dma->format.fmt.pix_mp;
 
-	if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
+	if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
 		flags = DMA_PREP_INTERRUPT | DMA_CTRL_ACK;
 		dma->xt.dir = DMA_DEV_TO_MEM;
 		dma->xt.src_sgl = false;
@@ -365,10 +369,11 @@ static void xvip_dma_buffer_queue(struct vb2_buffer *vb)
 		dma->xt.src_start = addr;
 	}
 
-	dma->xt.frame_size = 1;
-	dma->sgl[0].size = dma->format.width * dma->fmtinfo->bpp[0];
-	dma->sgl[0].icg = dma->format.bytesperline - dma->sgl[0].size;
-	dma->xt.numf = dma->format.height;
+	dma->xt.frame_size = dma->fmtinfo->num_planes;
+	dma->sgl[0].size = pix_mp->width * dma->fmtinfo->bpp[0];
+	dma->sgl[0].icg = pix_mp->plane_fmt[0].bytesperline - dma->sgl[0].size;
+	dma->xt.numf = pix_mp->height;
+	dma->sgl[0].dst_icg = 0;
 
 	desc = dmaengine_prep_interleaved_dma(dma->dma, &dma->xt, flags);
 	if (!desc) {
@@ -496,11 +501,12 @@ xvip_dma_querycap(struct file *file, void *fh, struct v4l2_capability *cap)
 
 	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
 			  | dma->xdev->v4l2_caps;
+	cap->device_caps = V4L2_CAP_STREAMING;
 
-	if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-		cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+	if (dma->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		cap->device_caps |= V4L2_CAP_VIDEO_CAPTURE_MPLANE;
 	else
-		cap->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+		cap->device_caps |= V4L2_CAP_VIDEO_OUTPUT_MPLANE;
 
 	strlcpy(cap->driver, "xilinx-vipp", sizeof(cap->driver));
 	strlcpy(cap->card, dma->video.name, sizeof(cap->card));
@@ -524,7 +530,7 @@ xvip_dma_enum_format(struct file *file, void *fh, struct v4l2_fmtdesc *f)
 	if (f->index > 0)
 		return -EINVAL;
 
-	f->pixelformat = dma->format.pixelformat;
+	f->pixelformat = dma->format.fmt.pix_mp.pixelformat;
 	strlcpy(f->description, dma->fmtinfo->description,
 		sizeof(f->description));
 
@@ -537,13 +543,14 @@ xvip_dma_get_format(struct file *file, void *fh, struct v4l2_format *format)
 	struct v4l2_fh *vfh = file->private_data;
 	struct xvip_dma *dma = to_xvip_dma(vfh->vdev);
 
-	format->fmt.pix = dma->format;
+	format->fmt.pix_mp = dma->format.fmt.pix_mp;
 
 	return 0;
 }
 
 static void
-__xvip_dma_try_format(struct xvip_dma *dma, struct v4l2_pix_format *pix,
+__xvip_dma_try_format(struct xvip_dma *dma,
+		      struct v4l2_format *format,
 		      const struct xvip_video_format **fmtinfo)
 {
 	const struct xvip_video_format *info;
@@ -551,19 +558,20 @@ __xvip_dma_try_format(struct xvip_dma *dma, struct v4l2_pix_format *pix,
 	unsigned int max_width;
 	unsigned int min_bpl;
 	unsigned int max_bpl;
-	unsigned int width;
+	unsigned int width, height;
 	unsigned int align;
 	unsigned int bpl;
+	struct v4l2_pix_format_mplane *pix_mp = &format->fmt.pix_mp;
+	struct v4l2_plane_pix_format *plane_fmt = pix_mp->plane_fmt;
 
 	/* Retrieve format information and select the default format if the
 	 * requested format isn't supported.
 	 */
-	info = xvip_get_format_by_fourcc(pix->pixelformat);
+	info = xvip_get_format_by_fourcc(pix_mp->pixelformat);
 	if (IS_ERR(info))
 		info = xvip_get_format_by_fourcc(XVIP_DMA_DEF_FORMAT);
 
-	pix->pixelformat = info->fourcc;
-	pix->field = V4L2_FIELD_NONE;
+	pix_mp->field = V4L2_FIELD_NONE;
 
 	/* The transfer alignment requirements are expressed in bytes. Compute
 	 * the minimum and maximum values, clamp the requested width and convert
@@ -572,22 +580,41 @@ __xvip_dma_try_format(struct xvip_dma *dma, struct v4l2_pix_format *pix,
 	align = lcm(dma->align, info->bpp[0]);
 	min_width = roundup(XVIP_DMA_MIN_WIDTH, align);
 	max_width = rounddown(XVIP_DMA_MAX_WIDTH, align);
-	width = rounddown(pix->width * info->bpp[0], align);
+	pix_mp->width = clamp(pix_mp->width, min_width, max_width);
+	pix_mp->height = clamp(pix_mp->height, XVIP_DMA_MIN_HEIGHT,
+			       XVIP_DMA_MAX_HEIGHT);
 
-	pix->width = clamp(width, min_width, max_width) / info->bpp[0];
-	pix->height = clamp(pix->height, XVIP_DMA_MIN_HEIGHT,
-			    XVIP_DMA_MAX_HEIGHT);
-
-	/* Clamp the requested bytes per line value. If the maximum bytes per
-	 * line value is zero, the module doesn't support user configurable line
-	 * sizes. Override the requested value with the minimum in that case.
+	/*
+	 * Clamp the requested bytes per line value. If the maximum
+	 * bytes per line value is zero, the module doesn't support
+	 * user configurable line sizes. Override the requested value
+	 * with the minimum in that case.
 	 */
-	min_bpl = pix->width * info->bpp[0];
-	max_bpl = rounddown(XVIP_DMA_MAX_WIDTH, dma->align);
-	bpl = rounddown(pix->bytesperline, dma->align);
-
-	pix->bytesperline = clamp(bpl, min_bpl, max_bpl);
-	pix->sizeimage = pix->bytesperline * pix->height;
+	max_bpl = rounddown(XVIP_DMA_MAX_WIDTH, align);
+	min_bpl = pix_mp->width * info->bpp[0];
+	min_bpl = roundup(min_bpl, align);
+	bpl = roundup(plane_fmt[0].bytesperline, align);
+	plane_fmt[0].bytesperline = clamp(bpl, min_bpl, max_bpl);
+
+	if (info->num_planes == 1) {
+		/* Single plane formats */
+		plane_fmt[0].sizeimage = plane_fmt[0].bytesperline *
+					 pix_mp->height;
+	} else {
+		/*
+		 * Supports Multi plane formats in a contiguous buffer,
+		 * so we need only one buffer
+		 */
+		unsigned int i;
+
+		plane_fmt[0].sizeimage = 0;
+		for (i = 0; i < info->num_planes; i++) {
+			width = plane_fmt[0].bytesperline /
+				(i ? info->hsub : 1);
+			height = pix_mp->height / (i ? info->vsub : 1);
+			plane_fmt[0].sizeimage += width * info->bpp[i] * height;
+		}
+	}
 
 	if (fmtinfo)
 		*fmtinfo = info;
@@ -599,7 +626,7 @@ xvip_dma_try_format(struct file *file, void *fh, struct v4l2_format *format)
 	struct v4l2_fh *vfh = file->private_data;
 	struct xvip_dma *dma = to_xvip_dma(vfh->vdev);
 
-	__xvip_dma_try_format(dma, &format->fmt.pix, NULL);
+	__xvip_dma_try_format(dma, format, NULL);
 	return 0;
 }
 
@@ -610,12 +637,13 @@ xvip_dma_set_format(struct file *file, void *fh, struct v4l2_format *format)
 	struct xvip_dma *dma = to_xvip_dma(vfh->vdev);
 	const struct xvip_video_format *info;
 
-	__xvip_dma_try_format(dma, &format->fmt.pix, &info);
+	__xvip_dma_try_format(dma, format, &info);
 
 	if (vb2_is_busy(&dma->queue))
 		return -EBUSY;
 
-	dma->format = format->fmt.pix;
+	dma->format.fmt.pix_mp = format->fmt.pix_mp;
+
 	dma->fmtinfo = info;
 
 	return 0;
@@ -623,13 +651,14 @@ xvip_dma_set_format(struct file *file, void *fh, struct v4l2_format *format)
 
 static const struct v4l2_ioctl_ops xvip_dma_ioctl_ops = {
 	.vidioc_querycap		= xvip_dma_querycap,
-	.vidioc_enum_fmt_vid_cap	= xvip_dma_enum_format,
-	.vidioc_g_fmt_vid_cap		= xvip_dma_get_format,
-	.vidioc_g_fmt_vid_out		= xvip_dma_get_format,
-	.vidioc_s_fmt_vid_cap		= xvip_dma_set_format,
-	.vidioc_s_fmt_vid_out		= xvip_dma_set_format,
-	.vidioc_try_fmt_vid_cap		= xvip_dma_try_format,
-	.vidioc_try_fmt_vid_out		= xvip_dma_try_format,
+	.vidioc_enum_fmt_vid_cap_mplane	= xvip_dma_enum_format,
+	.vidioc_enum_fmt_vid_out_mplane	= xvip_dma_enum_format,
+	.vidioc_g_fmt_vid_cap_mplane	= xvip_dma_get_format,
+	.vidioc_g_fmt_vid_out_mplane	= xvip_dma_get_format,
+	.vidioc_s_fmt_vid_cap_mplane	= xvip_dma_set_format,
+	.vidioc_s_fmt_vid_out_mplane	= xvip_dma_set_format,
+	.vidioc_try_fmt_vid_cap_mplane	= xvip_dma_try_format,
+	.vidioc_try_fmt_vid_out_mplane	= xvip_dma_try_format,
 	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
 	.vidioc_querybuf		= vb2_ioctl_querybuf,
 	.vidioc_qbuf			= vb2_ioctl_qbuf,
@@ -662,6 +691,7 @@ int xvip_dma_init(struct xvip_composite_device *xdev, struct xvip_dma *dma,
 {
 	char name[16];
 	int ret;
+	struct v4l2_pix_format_mplane *pix_mp;
 
 	dma->xdev = xdev;
 	dma->port = port;
@@ -671,17 +701,22 @@ int xvip_dma_init(struct xvip_composite_device *xdev, struct xvip_dma *dma,
 	spin_lock_init(&dma->queued_lock);
 
 	dma->fmtinfo = xvip_get_format_by_fourcc(XVIP_DMA_DEF_FORMAT);
-	dma->format.pixelformat = dma->fmtinfo->fourcc;
-	dma->format.colorspace = V4L2_COLORSPACE_SRGB;
-	dma->format.field = V4L2_FIELD_NONE;
-	dma->format.width = XVIP_DMA_DEF_WIDTH;
-	dma->format.height = XVIP_DMA_DEF_HEIGHT;
-	dma->format.bytesperline = dma->format.width * dma->fmtinfo->bpp[0];
-	dma->format.sizeimage = dma->format.bytesperline * dma->format.height;
+	dma->format.type = type;
+
+	pix_mp = &dma->format.fmt.pix_mp;
+	pix_mp->pixelformat = dma->fmtinfo->fourcc;
+	pix_mp->colorspace = V4L2_COLORSPACE_SRGB;
+	pix_mp->field = V4L2_FIELD_NONE;
+	pix_mp->width = XVIP_DMA_DEF_WIDTH;
+	pix_mp->height = XVIP_DMA_DEF_HEIGHT;
+	pix_mp->plane_fmt[0].bytesperline = pix_mp->width *
+					    dma->fmtinfo->bpp[0];
+	pix_mp->plane_fmt[0].sizeimage = pix_mp->plane_fmt[0].bytesperline *
+					 pix_mp->height;
 
 	/* Initialize the media entity... */
-	dma->pad.flags = type == V4L2_BUF_TYPE_VIDEO_CAPTURE
-		       ? MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+	dma->pad.flags = type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
+			? MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
 
 	ret = media_entity_pads_init(&dma->video.entity, 1, &dma->pad);
 	if (ret < 0)
@@ -693,11 +728,14 @@ int xvip_dma_init(struct xvip_composite_device *xdev, struct xvip_dma *dma,
 	dma->video.queue = &dma->queue;
 	snprintf(dma->video.name, sizeof(dma->video.name), "%s %s %u",
 		 xdev->dev->of_node->name,
-		 type == V4L2_BUF_TYPE_VIDEO_CAPTURE ? "output" : "input",
+		 type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
+					? "output" : "input",
 		 port);
+
 	dma->video.vfl_type = VFL_TYPE_GRABBER;
-	dma->video.vfl_dir = type == V4L2_BUF_TYPE_VIDEO_CAPTURE
-			   ? VFL_DIR_RX : VFL_DIR_TX;
+	dma->video.vfl_dir = type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
+				? VFL_DIR_RX : VFL_DIR_TX;
+
 	dma->video.release = video_device_release_empty;
 	dma->video.ioctl_ops = &xvip_dma_ioctl_ops;
 	dma->video.lock = &dma->lock;
diff --git a/drivers/media/platform/xilinx/xilinx-dma.h b/drivers/media/platform/xilinx/xilinx-dma.h
index e95d136..96933ed 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.h
+++ b/drivers/media/platform/xilinx/xilinx-dma.h
@@ -62,7 +62,7 @@ static inline struct xvip_pipeline *to_xvip_pipeline(struct media_entity *e)
  * @pipe: pipeline belonging to the DMA channel
  * @port: composite device DT node port number for the DMA channel
  * @lock: protects the @format, @fmtinfo and @queue fields
- * @format: active V4L2 pixel format
+ * @format: V4L2 format
  * @fmtinfo: format information corresponding to the active @format
  * @queue: vb2 buffers queue
  * @sequence: V4L2 buffers sequence number
@@ -83,7 +83,7 @@ struct xvip_dma {
 	unsigned int port;
 
 	struct mutex lock;
-	struct v4l2_pix_format format;
+	struct v4l2_format format;
 	const struct xvip_video_format *fmtinfo;
 
 	struct vb2_queue queue;
diff --git a/drivers/media/platform/xilinx/xilinx-vipp.c b/drivers/media/platform/xilinx/xilinx-vipp.c
index 6bb28cd..509b50f 100644
--- a/drivers/media/platform/xilinx/xilinx-vipp.c
+++ b/drivers/media/platform/xilinx/xilinx-vipp.c
@@ -433,12 +433,15 @@ static int xvip_graph_dma_init_one(struct xvip_composite_device *xdev,
 	if (ret < 0)
 		return ret;
 
-	if (strcmp(direction, "input") == 0)
-		type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-	else if (strcmp(direction, "output") == 0)
-		type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
-	else
+	if (strcmp(direction, "input") == 0) {
+		type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+		xdev->v4l2_caps |= V4L2_CAP_VIDEO_CAPTURE_MPLANE;
+	} else if (strcmp(direction, "output") == 0) {
+		type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+		xdev->v4l2_caps |= V4L2_CAP_VIDEO_OUTPUT_MPLANE;
+	} else {
 		return -EINVAL;
+	}
 
 	of_property_read_u32(node, "reg", &index);
 
@@ -454,9 +457,6 @@ static int xvip_graph_dma_init_one(struct xvip_composite_device *xdev,
 
 	list_add_tail(&dma->list, &xdev->dmas);
 
-	xdev->v4l2_caps |= type == V4L2_BUF_TYPE_VIDEO_CAPTURE
-			 ? V4L2_CAP_VIDEO_CAPTURE : V4L2_CAP_VIDEO_OUTPUT;
-
 	return 0;
 }
 
-- 
2.7.4

[PATCH v5 7/8] uapi: media: New fourcc code and rst for 10 bit format

[Satish Kumar Nagireddy]

From: Jeffrey Mouroux <jmouroux@xilinx.com>

This patch adds new fourcc code and rst documentation for
YUV420 10 bit format.

Signed-off-by: Jeffrey Mouroux <jmouroux@xilinx.com>
Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
Changes in v5:
 - Squashed rst documentation and new pixel format of YUV420 10 bit into single patch

Changes in v4:
 - Added rst documentation for YUV420 10 bit format

 Documentation/media/uapi/v4l/pixfmt-xv15.rst | 134 +++++++++++++++++++++++++++
 Documentation/media/uapi/v4l/yuv-formats.rst |   1 +
 include/uapi/linux/videodev2.h               |   1 +
 3 files changed, 136 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-xv15.rst

diff --git a/Documentation/media/uapi/v4l/pixfmt-xv15.rst b/Documentation/media/uapi/v4l/pixfmt-xv15.rst
new file mode 100644
index 0000000..fc829c3
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-xv15.rst
@@ -0,0 +1,134 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _V4L2-PIX-FMT-XV15:
+
+**************************
+V4L2_PIX_FMT_XV15 ('XV15')
+**************************
+
+
+Semi-planar YUV 420 10-bit
+
+
+Description
+===========
+
+This is the 10-bit version of YUV 420 semi-planar format. XV15 is the one
+where chroma plane is contiguous with the luma plane in memory.
+
+Each pixel of YUV 420 contains a single luma component of 10-bits in length.
+Three luma components are stored per word with the remaining two bits serving
+as padding.
+
+The chroma plane is subsampled and is only 1/2 the size of the luma plane.  A
+single chroma component serves two pixels on a given row and is re-used on the
+adjacent row of luma data.
+
+**Data Layout of Luma Plane**
+Each cell is one 32-bit word.
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - word + 0:
+      - X'\ :sub:`[31:30]`
+      - Y'\ :sub:`02 [29:20]`
+      - Y'\ :sub:`01 [19:10]`
+      - Y'\ :sub:`00 [09:00]`
+      -
+    * - word + 1:
+      - X'\ :sub:`[31:30]`
+      - Y'\ :sub:`05 [29:20]`
+      - Y'\ :sub:`04 [19:10]`
+      - Y'\ :sub:`03 [09:00]`
+      -
+    * - word + 2:
+      - X'\ :sub:`[31:30]`
+      - Y'\ :sub:`08 [29:20]`
+      - Y'\ :sub:`07 [19:10]`
+      - Y'\ :sub:`06 [09:00]`
+      -
+
+
+**Data Layout of Chroma Plane**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * - word + 0:
+      - X'\ :sub:`[31:30]`
+      - U'\ :sub:`02 [29:20]`
+      - V'\ :sub:`01 [19:10]`
+      - U'\ :sub:`00 [09:00]`
+      -
+    * - word + 1:
+      - X'\ :sub:`[31:30]`
+      - V'\ :sub:`05 [29:20]`
+      - U'\ :sub:`04 [19:10]`
+      - V'\ :sub:`03 [09:00]`
+      -
+    * - word + 2:
+      - X'\ :sub:`[31:30]`
+      - U'\ :sub:`08 [29:20]`
+      - V'\ :sub:`07 [19:10]`
+      - U'\ :sub:`06 [09:00]`
+      -
+
+**Color Sample Location**
+
+.. flat-table::
+    :header-rows:  0
+    :stub-columns: 0
+
+    * -
+      - 0
+      -
+      - 1
+      - 2
+      -
+      - 3
+    * - 0
+      - Y
+      -
+      - Y
+      - Y
+      -
+      - Y
+    * -
+      -
+      - C
+      -
+      -
+      - C
+      -
+    * - 1
+      - Y
+      -
+      - Y
+      - Y
+      -
+      - Y
+    * -
+    * - 2
+      - Y
+      -
+      - Y
+      - Y
+      -
+      - Y
+    * -
+      -
+      - C
+      -
+      -
+      - C
+      -
+    * - 3
+      - Y
+      -
+      - Y
+      - Y
+      -
+      - Y
diff --git a/Documentation/media/uapi/v4l/yuv-formats.rst b/Documentation/media/uapi/v4l/yuv-formats.rst
index 3334ea4..c500bc1 100644
--- a/Documentation/media/uapi/v4l/yuv-formats.rst
+++ b/Documentation/media/uapi/v4l/yuv-formats.rst
@@ -49,6 +49,7 @@ to brightness information.
     pixfmt-nv12
     pixfmt-nv12m
     pixfmt-nv12mt
+    pixfmt-xv15
     pixfmt-nv16
     pixfmt-nv16m
     pixfmt-nv24
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877b..873bafa 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -551,6 +551,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_NV61    v4l2_fourcc('N', 'V', '6', '1') /* 16  Y/CrCb 4:2:2  */
 #define V4L2_PIX_FMT_NV24    v4l2_fourcc('N', 'V', '2', '4') /* 24  Y/CbCr 4:4:4  */
 #define V4L2_PIX_FMT_NV42    v4l2_fourcc('N', 'V', '4', '2') /* 24  Y/CrCb 4:4:4  */
+#define V4L2_PIX_FMT_XV15    v4l2_fourcc('X', 'V', '1', '5') /* 32  XY/UV 4:2:0 10-bit */
 
 /* two non contiguous planes - one Y, one Cr + Cb interleaved  */
 #define V4L2_PIX_FMT_NV12M   v4l2_fourcc('N', 'M', '1', '2') /* 12  Y/CbCr 4:2:0  */
-- 
2.7.4

[PATCH v5 8/8] v4l: xilinx: dma: Add support for 10 bit formats

[Satish Kumar Nagireddy]

This patch adds xvip_format_plane_width_bytes function to
calculate number of bytes for a macropixel formats and also
adds new 10 bit pixel formats to video descriptor table.

Signed-off-by: Satish Kumar Nagireddy <satish.nagireddy.nagireddy@xilinx.com>
---
Changes in v4:
 - Introduced macropixel concept to calculate bytes for a given width for 10 bit formats

 drivers/media/platform/xilinx/xilinx-dma.c |  5 ++--
 drivers/media/platform/xilinx/xilinx-vip.c | 37 ++++++++++++++++++++++--------
 drivers/media/platform/xilinx/xilinx-vip.h |  5 ++++
 3 files changed, 36 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/xilinx/xilinx-dma.c b/drivers/media/platform/xilinx/xilinx-dma.c
index 2ffc276..89afa24 100644
--- a/drivers/media/platform/xilinx/xilinx-dma.c
+++ b/drivers/media/platform/xilinx/xilinx-dma.c
@@ -370,7 +370,8 @@ static void xvip_dma_buffer_queue(struct vb2_buffer *vb)
 	}
 
 	dma->xt.frame_size = dma->fmtinfo->num_planes;
-	dma->sgl[0].size = pix_mp->width * dma->fmtinfo->bpp[0];
+	dma->sgl[0].size = xvip_fmt_plane_width_bytes(dma->fmtinfo,
+						      pix_mp->width);
 	dma->sgl[0].icg = pix_mp->plane_fmt[0].bytesperline - dma->sgl[0].size;
 	dma->xt.numf = pix_mp->height;
 	dma->sgl[0].dst_icg = 0;
@@ -591,7 +592,7 @@ __xvip_dma_try_format(struct xvip_dma *dma,
 	 * with the minimum in that case.
 	 */
 	max_bpl = rounddown(XVIP_DMA_MAX_WIDTH, align);
-	min_bpl = pix_mp->width * info->bpp[0];
+	min_bpl = xvip_fmt_plane_width_bytes(info, pix_mp->width);
 	min_bpl = roundup(min_bpl, align);
 	bpl = roundup(plane_fmt[0].bytesperline, align);
 	plane_fmt[0].bytesperline = clamp(bpl, min_bpl, max_bpl);
diff --git a/drivers/media/platform/xilinx/xilinx-vip.c b/drivers/media/platform/xilinx/xilinx-vip.c
index fb1a08f..7569f05 100644
--- a/drivers/media/platform/xilinx/xilinx-vip.c
+++ b/drivers/media/platform/xilinx/xilinx-vip.c
@@ -28,23 +28,25 @@
 
 static const struct xvip_video_format xvip_video_formats[] = {
 	{ XVIP_VF_YUV_420, 8, NULL, MEDIA_BUS_FMT_VYYUYY8_1X24,
-	  {1, 2, 0}, V4L2_PIX_FMT_NV12, 2, 2, 2, "4:2:0, semi-planar, YUV" },
+	  {1, 2, 0}, V4L2_PIX_FMT_NV12, 2, 2, 2, 1, 1, "4:2:0, semi-planar, YUV" },
 	{ XVIP_VF_YUV_422, 8, NULL, MEDIA_BUS_FMT_UYVY8_1X16,
-	  {2, 0, 0}, V4L2_PIX_FMT_YUYV, 1, 2, 1, "4:2:2, packed, YUYV" },
+	  {2, 0, 0}, V4L2_PIX_FMT_YUYV, 1, 2, 1, 1, 1, "4:2:2, packed, YUYV" },
 	{ XVIP_VF_YUV_444, 8, NULL, MEDIA_BUS_FMT_VUY8_1X24,
-	  {3, 0, 0}, V4L2_PIX_FMT_YUV444, 1, 1, 1, "4:4:4, packed, YUYV" },
+	  {3, 0, 0}, V4L2_PIX_FMT_YUV444, 1, 1, 1, 1, 1, "4:4:4, packed, YUYV" },
+	{ XVIP_VF_YUV_420, 10, NULL, MEDIA_BUS_FMT_VYYUYY8_1X24,
+	  {1, 2, 0}, V4L2_PIX_FMT_XV15, 2, 2, 2, 4, 3, "4:2:0, 10-bit 2-plane cont" },
 	{ XVIP_VF_RBG, 8, NULL, MEDIA_BUS_FMT_RBG888_1X24,
-	  {3, 0, 0}, V4L2_PIX_FMT_RGB24, 1, 1, 1, "24-bit RGB" },
+	  {3, 0, 0}, V4L2_PIX_FMT_RGB24, 1, 1, 1, 1, 1, "24-bit RGB" },
 	{ XVIP_VF_MONO_SENSOR, 8, "mono", MEDIA_BUS_FMT_Y8_1X8,
-	  {1, 0, 0}, V4L2_PIX_FMT_GREY, 1, 1, 1, "Greyscale 8-bit" },
+	  {1, 0, 0}, V4L2_PIX_FMT_GREY, 1, 1, 1, 1, 1, "Greyscale 8-bit" },
 	{ XVIP_VF_MONO_SENSOR, 8, "rggb", MEDIA_BUS_FMT_SRGGB8_1X8,
-	  {1, 0, 0}, V4L2_PIX_FMT_SGRBG8, 1, 1, 1, "Bayer 8-bit RGGB" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SGRBG8, 1, 1, 1, 1, 1, "Bayer 8-bit RGGB" },
 	{ XVIP_VF_MONO_SENSOR, 8, "grbg", MEDIA_BUS_FMT_SGRBG8_1X8,
-	  {1, 0, 0}, V4L2_PIX_FMT_SGRBG8, 1, 1, 1, "Bayer 8-bit GRBG" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SGRBG8, 1, 1, 1, 1, 1, "Bayer 8-bit GRBG" },
 	{ XVIP_VF_MONO_SENSOR, 8, "gbrg", MEDIA_BUS_FMT_SGBRG8_1X8,
-	  {1, 0, 0}, V4L2_PIX_FMT_SGBRG8, 1, 1, 1, "Bayer 8-bit GBRG" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SGBRG8, 1, 1, 1, 1, 1, "Bayer 8-bit GBRG" },
 	{ XVIP_VF_MONO_SENSOR, 8, "bggr", MEDIA_BUS_FMT_SBGGR8_1X8,
-	  {1, 0, 0}, V4L2_PIX_FMT_SBGGR8, 1, 1, 1, "Bayer 8-bit BGGR" },
+	  {1, 0, 0}, V4L2_PIX_FMT_SBGGR8, 1, 1, 1, 1, 1, "Bayer 8-bit BGGR" },
 };
 
 /**
@@ -94,6 +96,23 @@ const struct xvip_video_format *xvip_get_format_by_fourcc(u32 fourcc)
 EXPORT_SYMBOL_GPL(xvip_get_format_by_fourcc);
 
 /**
+ * xvip_fmt_plane_width_bytes - bytes of the given width of the plane
+ * @info: VIP format description
+ * @width: width
+ *
+ * Return: Returns the number of bytes for given @width
+ */
+int xvip_fmt_plane_width_bytes(const struct xvip_video_format *info, u32 width)
+{
+	if (!info)
+		return 0;
+
+	return DIV_ROUND_UP(width * info->bytes_per_macropixel * info->bpp[0],
+			    info->pixels_per_macropixel);
+}
+EXPORT_SYMBOL_GPL(xvip_fmt_plane_width_bytes);
+
+/**
  * xvip_of_get_format - Parse a device tree node and return format information
  * @node: the device tree node
  *
diff --git a/drivers/media/platform/xilinx/xilinx-vip.h b/drivers/media/platform/xilinx/xilinx-vip.h
index 256efa2..f216e18 100644
--- a/drivers/media/platform/xilinx/xilinx-vip.h
+++ b/drivers/media/platform/xilinx/xilinx-vip.h
@@ -114,6 +114,8 @@ struct xvip_device {
  * @num_planes: number of planes w.r.t. color format
  * @hsub: Horizontal sampling factor of Chroma
  * @vsub: Vertical sampling factor of Chroma
+ * @bytes_per_macropixel: Number of bytes per macro-pixel
+ * @pixels_per_macropixel: Number of pixels per macro-pixel
  * @description: format description, suitable for userspace
  */
 struct xvip_video_format {
@@ -126,12 +128,15 @@ struct xvip_video_format {
 	u8 num_planes;
 	u8 hsub;
 	u8 vsub;
+	u32 bytes_per_macropixel;
+	u32 pixels_per_macropixel;
 	const char *description;
 };
 
 const struct xvip_video_format *xvip_get_format_by_code(unsigned int code);
 const struct xvip_video_format *xvip_get_format_by_fourcc(u32 fourcc);
 const struct xvip_video_format *xvip_of_get_format(struct device_node *node);
+int xvip_fmt_plane_width_bytes(const struct xvip_video_format *info, u32 width);
 void xvip_set_format_size(struct v4l2_mbus_framefmt *format,
 			  const struct v4l2_subdev_format *fmt);
 int xvip_enum_mbus_code(struct v4l2_subdev *subdev,
-- 
2.7.4

Re: [PATCH v5 0/8] Add support for multi-planar formats and 10 bit formats

[Hans Verkuil]

Hi Satish,

On 03/05/18 04:42, Satish Kumar Nagireddy wrote:
>  The patches are for xilinx v4l. The patcheset enable support to handle multiplanar
>  formats and 10 bit formats. Single planar implementation is removed as mplane can
>  handle both.

If I understand the format correctly, then the planes are contiguous in memory,
i.e. it is a single buffer.

You do not need to switch to the _MPLANE API for that: that API is meant for the
case where the planes are not contiguous in memory but each plane has its own
buffer. And yes, we should have called it the _MBUFFER API or something :-(

https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-nv12.html

Switching to the _MPLANE API will actually break userspace, so that's another
reason why you shouldn't do this. But from what I can tell, it really isn't
needed at all.

Regards,

	Hans

> 
>  Patch-set has downstream changes and bug fixes. Added new media bus format
>  MEDIA_BUS_FMT_VYYUYY8_1X24, new pixel format V4L2_PIX_FMT_XV15 and rst
>  documentation.
> 
> Jeffrey Mouroux (1):
>   uapi: media: New fourcc code and rst for 10 bit format
> 
> Radhey Shyam Pandey (1):
>   v4l: xilinx: dma: Remove colorspace check in xvip_dma_verify_format
> 
> Rohit Athavale (1):
>   xilinx: v4l: dma: Update driver to allow for probe defer
> 
> Satish Kumar Nagireddy (4):
>   media-bus: uapi: Add YCrCb 420 media bus format and rst
>   v4l: xilinx: dma: Update video format descriptor
>   v4l: xilinx: dma: Add multi-planar support
>   v4l: xilinx: dma: Add support for 10 bit formats
> 
> Vishal Sagar (1):
>   xilinx: v4l: dma: Terminate DMA when media pipeline fail to start
> 
>  Documentation/media/uapi/v4l/pixfmt-xv15.rst    | 134 +++++++++++++++++++
>  Documentation/media/uapi/v4l/subdev-formats.rst |  38 +++++-
>  Documentation/media/uapi/v4l/yuv-formats.rst    |   1 +
>  drivers/media/platform/xilinx/xilinx-dma.c      | 170 +++++++++++++++---------
>  drivers/media/platform/xilinx/xilinx-dma.h      |   4 +-
>  drivers/media/platform/xilinx/xilinx-vip.c      |  37 ++++--
>  drivers/media/platform/xilinx/xilinx-vip.h      |  15 ++-
>  drivers/media/platform/xilinx/xilinx-vipp.c     |  16 +--
>  include/uapi/linux/media-bus-format.h           |   3 +-
>  include/uapi/linux/videodev2.h                  |   1 +
>  10 files changed, 333 insertions(+), 86 deletions(-)
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-xv15.rst
> 

Re: [PATCH v5 0/8] Add support for multi-planar formats and 10 bit formats

[Hyun Kwon]

Hi Hans,

Thanks for the comment.

On Mon, 2018-05-07 at 05:59:39 -0700, Hans Verkuil wrote:
> Hi Satish,
> 
> On 03/05/18 04:42, Satish Kumar Nagireddy wrote:
> >  The patches are for xilinx v4l. The patcheset enable support to handle multiplanar
> >  formats and 10 bit formats. Single planar implementation is removed as mplane can
> >  handle both.
> 
> If I understand the format correctly, then the planes are contiguous in memory,
> i.e. it is a single buffer.
> 
> You do not need to switch to the _MPLANE API for that: that API is meant for the
> case where the planes are not contiguous in memory but each plane has its own
> buffer. And yes, we should have called it the _MBUFFER API or something :-(
> 
> https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-nv12.html
> 
> Switching to the _MPLANE API will actually break userspace, so that's another
> reason why you shouldn't do this. But from what I can tell, it really isn't
> needed at all.
> 

Sharing some background to get your further input. :-)

The Xilinx V4L driver is currently only for the soft IPs, which are
programmable on FPGA, and those IPs are constantly updated. Initially, IPs
didn't support _MPLANE formats, so it started with single buffer type format.
Now, the IPs support _MPLANE formats, even though those formats are not part of
this patch. Those formats are in downstream vendor tree and will be upstreamed
at some point[1]. While implementing the multi-buffer formats, we had similar
concern regarding UAPI and ended up having the module param[2]. It was there
for a couple of Xilinx release cycles to migrate internal applications to
_MPLANE formats and to get report if that breaks any external applications. Now
we thought it's good time to hard-switch the driver to _MPLANE completely
rather than keeping single buffer code, especially because it seems legal to
support single buffer formats with _MPLANE type. If this is not the case and
the applications with single buffer formats but without mplane formats should
be supported, we can revive the single buffer code in one way or another.

Thanks,
-hyun

[1] https://github.com/Xilinx/linux-xlnx/blob/xilinx-v2018.1/drivers/media/platform/xilinx/xilinx-vip.c#L33
[2] https://github.com/Xilinx/linux-xlnx/blob/xilinx-v2018.1/drivers/media/platform/xilinx/xilinx-vipp.c#L40

> Regards,
> 
> 	Hans
> 
> > 
> >  Patch-set has downstream changes and bug fixes. Added new media bus format
> >  MEDIA_BUS_FMT_VYYUYY8_1X24, new pixel format V4L2_PIX_FMT_XV15 and rst
> >  documentation.
> > 
> > Jeffrey Mouroux (1):
> >   uapi: media: New fourcc code and rst for 10 bit format
> > 
> > Radhey Shyam Pandey (1):
> >   v4l: xilinx: dma: Remove colorspace check in xvip_dma_verify_format
> > 
> > Rohit Athavale (1):
> >   xilinx: v4l: dma: Update driver to allow for probe defer
> > 
> > Satish Kumar Nagireddy (4):
> >   media-bus: uapi: Add YCrCb 420 media bus format and rst
> >   v4l: xilinx: dma: Update video format descriptor
> >   v4l: xilinx: dma: Add multi-planar support
> >   v4l: xilinx: dma: Add support for 10 bit formats
> > 
> > Vishal Sagar (1):
> >   xilinx: v4l: dma: Terminate DMA when media pipeline fail to start
> > 
> >  Documentation/media/uapi/v4l/pixfmt-xv15.rst    | 134 +++++++++++++++++++
> >  Documentation/media/uapi/v4l/subdev-formats.rst |  38 +++++-
> >  Documentation/media/uapi/v4l/yuv-formats.rst    |   1 +
> >  drivers/media/platform/xilinx/xilinx-dma.c      | 170 +++++++++++++++---------
> >  drivers/media/platform/xilinx/xilinx-dma.h      |   4 +-
> >  drivers/media/platform/xilinx/xilinx-vip.c      |  37 ++++--
> >  drivers/media/platform/xilinx/xilinx-vip.h      |  15 ++-
> >  drivers/media/platform/xilinx/xilinx-vipp.c     |  16 +--
> >  include/uapi/linux/media-bus-format.h           |   3 +-
> >  include/uapi/linux/videodev2.h                  |   1 +
> >  10 files changed, 333 insertions(+), 86 deletions(-)
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-xv15.rst
> > 
> 

Re: [PATCH v5 0/8] Add support for multi-planar formats and 10 bit formats

[Hans Verkuil]

On 05/07/2018 07:45 PM, Hyun Kwon wrote:
> Hi Hans,
> 
> Thanks for the comment.
> 
> On Mon, 2018-05-07 at 05:59:39 -0700, Hans Verkuil wrote:
>> Hi Satish,
>>
>> On 03/05/18 04:42, Satish Kumar Nagireddy wrote:
>>>  The patches are for xilinx v4l. The patcheset enable support to handle multiplanar
>>>  formats and 10 bit formats. Single planar implementation is removed as mplane can
>>>  handle both.
>>
>> If I understand the format correctly, then the planes are contiguous in memory,
>> i.e. it is a single buffer.
>>
>> You do not need to switch to the _MPLANE API for that: that API is meant for the
>> case where the planes are not contiguous in memory but each plane has its own
>> buffer. And yes, we should have called it the _MBUFFER API or something :-(
>>
>> https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-nv12.html
>>
>> Switching to the _MPLANE API will actually break userspace, so that's another
>> reason why you shouldn't do this. But from what I can tell, it really isn't
>> needed at all.
>>
> 
> Sharing some background to get your further input. :-)
> 
> The Xilinx V4L driver is currently only for the soft IPs, which are
> programmable on FPGA, and those IPs are constantly updated. Initially, IPs
> didn't support _MPLANE formats, so it started with single buffer type format.
> Now, the IPs support _MPLANE formats, even though those formats are not part of
> this patch. Those formats are in downstream vendor tree and will be upstreamed
> at some point[1]. While implementing the multi-buffer formats, we had similar
> concern regarding UAPI and ended up having the module param[2]. It was there
> for a couple of Xilinx release cycles to migrate internal applications to
> _MPLANE formats and to get report if that breaks any external applications. Now
> we thought it's good time to hard-switch the driver to _MPLANE completely
> rather than keeping single buffer code, especially because it seems legal to
> support single buffer formats with _MPLANE type. If this is not the case and
> the applications with single buffer formats but without mplane formats should
> be supported, we can revive the single buffer code in one way or another.

In that case you need to split off the _MPLANE API change into a separate patch.
Switching to the MPLANE API has nothing to do with supporting this format, it is
done for a different reason (future support for real multiplane formats).

Since this also breaks userspace applications you will need to clearly state in
the commit log why this is a reasonable thing to do.

Regards,

	Hans

> 
> Thanks,
> -hyun
> 
> [1] https://github.com/Xilinx/linux-xlnx/blob/xilinx-v2018.1/drivers/media/platform/xilinx/xilinx-vip.c#L33
> [2] https://github.com/Xilinx/linux-xlnx/blob/xilinx-v2018.1/drivers/media/platform/xilinx/xilinx-vipp.c#L40
> 
>> Regards,
>>
>> 	Hans
>>
>>>
>>>  Patch-set has downstream changes and bug fixes. Added new media bus format
>>>  MEDIA_BUS_FMT_VYYUYY8_1X24, new pixel format V4L2_PIX_FMT_XV15 and rst
>>>  documentation.
>>>
>>> Jeffrey Mouroux (1):
>>>   uapi: media: New fourcc code and rst for 10 bit format
>>>
>>> Radhey Shyam Pandey (1):
>>>   v4l: xilinx: dma: Remove colorspace check in xvip_dma_verify_format
>>>
>>> Rohit Athavale (1):
>>>   xilinx: v4l: dma: Update driver to allow for probe defer
>>>
>>> Satish Kumar Nagireddy (4):
>>>   media-bus: uapi: Add YCrCb 420 media bus format and rst
>>>   v4l: xilinx: dma: Update video format descriptor
>>>   v4l: xilinx: dma: Add multi-planar support
>>>   v4l: xilinx: dma: Add support for 10 bit formats
>>>
>>> Vishal Sagar (1):
>>>   xilinx: v4l: dma: Terminate DMA when media pipeline fail to start
>>>
>>>  Documentation/media/uapi/v4l/pixfmt-xv15.rst    | 134 +++++++++++++++++++
>>>  Documentation/media/uapi/v4l/subdev-formats.rst |  38 +++++-
>>>  Documentation/media/uapi/v4l/yuv-formats.rst    |   1 +
>>>  drivers/media/platform/xilinx/xilinx-dma.c      | 170 +++++++++++++++---------
>>>  drivers/media/platform/xilinx/xilinx-dma.h      |   4 +-
>>>  drivers/media/platform/xilinx/xilinx-vip.c      |  37 ++++--
>>>  drivers/media/platform/xilinx/xilinx-vip.h      |  15 ++-
>>>  drivers/media/platform/xilinx/xilinx-vipp.c     |  16 +--
>>>  include/uapi/linux/media-bus-format.h           |   3 +-
>>>  include/uapi/linux/videodev2.h                  |   1 +
>>>  10 files changed, 333 insertions(+), 86 deletions(-)
>>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-xv15.rst
>>>
>>

Re: [PATCH v5 0/8] Add support for multi-planar formats and 10 bit formats

[Hyun Kwon]

Hi Hans,

On Tue, 2018-05-08 at 00:57:25 -0700, Hans Verkuil wrote:
> On 05/07/2018 07:45 PM, Hyun Kwon wrote:
> > Hi Hans,
> > 
> > Thanks for the comment.
> > 
> > On Mon, 2018-05-07 at 05:59:39 -0700, Hans Verkuil wrote:
> >> Hi Satish,
> >>
> >> On 03/05/18 04:42, Satish Kumar Nagireddy wrote:
> >>>  The patches are for xilinx v4l. The patcheset enable support to handle multiplanar
> >>>  formats and 10 bit formats. Single planar implementation is removed as mplane can
> >>>  handle both.
> >>
> >> If I understand the format correctly, then the planes are contiguous in memory,
> >> i.e. it is a single buffer.
> >>
> >> You do not need to switch to the _MPLANE API for that: that API is meant for the
> >> case where the planes are not contiguous in memory but each plane has its own
> >> buffer. And yes, we should have called it the _MBUFFER API or something :-(
> >>
> >> https://hverkuil.home.xs4all.nl/spec/uapi/v4l/pixfmt-nv12.html
> >>
> >> Switching to the _MPLANE API will actually break userspace, so that's another
> >> reason why you shouldn't do this. But from what I can tell, it really isn't
> >> needed at all.
> >>
> > 
> > Sharing some background to get your further input. :-)
> > 
> > The Xilinx V4L driver is currently only for the soft IPs, which are
> > programmable on FPGA, and those IPs are constantly updated. Initially, IPs
> > didn't support _MPLANE formats, so it started with single buffer type format.
> > Now, the IPs support _MPLANE formats, even though those formats are not part of
> > this patch. Those formats are in downstream vendor tree and will be upstreamed
> > at some point[1]. While implementing the multi-buffer formats, we had similar
> > concern regarding UAPI and ended up having the module param[2]. It was there
> > for a couple of Xilinx release cycles to migrate internal applications to
> > _MPLANE formats and to get report if that breaks any external applications. Now
> > we thought it's good time to hard-switch the driver to _MPLANE completely
> > rather than keeping single buffer code, especially because it seems legal to
> > support single buffer formats with _MPLANE type. If this is not the case and
> > the applications with single buffer formats but without mplane formats should
> > be supported, we can revive the single buffer code in one way or another.
> 
> In that case you need to split off the _MPLANE API change into a separate patch.
> Switching to the MPLANE API has nothing to do with supporting this format, it is
> done for a different reason (future support for real multiplane formats).
> 
> Since this also breaks userspace applications you will need to clearly state in
> the commit log why this is a reasonable thing to do.
> 

Sure. I agree, and we will re-organize patches.

Thanks,
-hyun

> Regards,
> 
> 	Hans
> 
> > 
> > Thanks,
> > -hyun
> > 
> > [1] https://github.com/Xilinx/linux-xlnx/blob/xilinx-v2018.1/drivers/media/platform/xilinx/xilinx-vip.c#L33
> > [2] https://github.com/Xilinx/linux-xlnx/blob/xilinx-v2018.1/drivers/media/platform/xilinx/xilinx-vipp.c#L40
> > 
> >> Regards,
> >>
> >> 	Hans
> >>
> >>>
> >>>  Patch-set has downstream changes and bug fixes. Added new media bus format
> >>>  MEDIA_BUS_FMT_VYYUYY8_1X24, new pixel format V4L2_PIX_FMT_XV15 and rst
> >>>  documentation.
> >>>
> >>> Jeffrey Mouroux (1):
> >>>   uapi: media: New fourcc code and rst for 10 bit format
> >>>
> >>> Radhey Shyam Pandey (1):
> >>>   v4l: xilinx: dma: Remove colorspace check in xvip_dma_verify_format
> >>>
> >>> Rohit Athavale (1):
> >>>   xilinx: v4l: dma: Update driver to allow for probe defer
> >>>
> >>> Satish Kumar Nagireddy (4):
> >>>   media-bus: uapi: Add YCrCb 420 media bus format and rst
> >>>   v4l: xilinx: dma: Update video format descriptor
> >>>   v4l: xilinx: dma: Add multi-planar support
> >>>   v4l: xilinx: dma: Add support for 10 bit formats
> >>>
> >>> Vishal Sagar (1):
> >>>   xilinx: v4l: dma: Terminate DMA when media pipeline fail to start
> >>>
> >>>  Documentation/media/uapi/v4l/pixfmt-xv15.rst    | 134 +++++++++++++++++++
> >>>  Documentation/media/uapi/v4l/subdev-formats.rst |  38 +++++-
> >>>  Documentation/media/uapi/v4l/yuv-formats.rst    |   1 +
> >>>  drivers/media/platform/xilinx/xilinx-dma.c      | 170 +++++++++++++++---------
> >>>  drivers/media/platform/xilinx/xilinx-dma.h      |   4 +-
> >>>  drivers/media/platform/xilinx/xilinx-vip.c      |  37 ++++--
> >>>  drivers/media/platform/xilinx/xilinx-vip.h      |  15 ++-
> >>>  drivers/media/platform/xilinx/xilinx-vipp.c     |  16 +--
> >>>  include/uapi/linux/media-bus-format.h           |   3 +-
> >>>  include/uapi/linux/videodev2.h                  |   1 +
> >>>  10 files changed, 333 insertions(+), 86 deletions(-)
> >>>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-xv15.rst
> >>>
> >>
>