[PATCH v3 00/24] media: ti-vpe: cal: Add media controller support

[PATCH v3 00/24] media: ti-vpe: cal: Add media controller support

[Laurent Pinchart]

Hello,

This patch series adds media controller support to the ti-cal driver. It
incorporates the review feedback received on v2, and has been rebased on
top of the media tree, dropping about three quarters of the patches from
v2 that have already been merged.

The end goal is to support pipelines where the image source is more
complex than a single subdev (for instance a SMIA++-compliant sensor, or
an FPD-Link source made of a camera, a serializer and a deserializer),
and to prepare for support of multiple hardware contexts, needed for
CSI-2 virtual channel and data type multiplexing support. These use
cases require configuration of the external pipeline from userspace, and
thus need support for the media controller and V4L2 subdev userspace
APIs.

The series starts with refactoring CAMERARX instances support by
splitting them to sudbevs in patch 01/24. Patches 02/24 to 06/24 perform
small cleanups, patch 07/24 implements the subdev operations and patch
08/24 uses them to configure and control the CAMERARX.

More refactoring and cleanup follow in patches 09/24 to 23/24, until
patch 24/24 that adds media controller support. As the existing video
node centric API needs to be preserved to avoid breaking userspace
applications, a module parameter is added to enable the media controller
API. Changes are otherwise fairly limited, and mostly involve the
implementation of MC-centric V4L2 ioctl handlers (which are far simpler
than their video node centric equivalents), additional pipeline
validation in the vb2 start streaming handler, and disabling control
handling in the video nodes.

Addition of a new Kconfig option to select the default value of the MC
API control module parameter has been left out from patch 24/24 until
the related discussions complete. This can easily be implemented on top,
without introducing any regression. Alternatively patch 24/24 could be
left out for the time being if desired.

The patches have been tested on an AM65x EVM with an OV5640 camera
module with the yavta test application, both in MC mode and in legacy
mode. As the ov5640 driver sets a default format identical to the
default of the ti-cal driver, capture in both modes can simply be tested
with

        yavta -f UYVY -s 640x480 -c10 --requeue-last /dev/video0

without requiring manual pipeline setup through the media controller
API. Usage of other formats or resolutions require pipeline
configuration before capture.

Laurent Pinchart (24):
  media: ti-vpe: cal: Create subdev for CAMERARX
  media: ti-vpe: cal: Drop cal_ctx m_fmt field
  media: ti-vpe: cal: Move format handling to cal.c and expose helpers
  media: ti-vpe: cal: Rename MAX_(WIDTH|HEIGHT)_* macros with CAL_
    prefix
  media: ti-vpe: cal: Replace hardcoded BIT() value with macro
  media: ti-vpe: cal: Iterate over correct number of CAMERARX instances
  media: ti-vpe: cal: Implement subdev ops for CAMERARX
  media: ti-vpe: cal: Use CAMERARX subdev s_stream op in video device
    code
  media: ti-vpe: cal: Don't pass format to cal_ctx_wr_dma_config()
  media: ti-vpe: cal: Rename struct cal_fmt to cal_format_info
  media: ti-vpe: cal: Refactor interrupt enable/disable
  media: ti-vpe: cal: Fold PPI enable in CAMERARX .s_stream()
  media: ti-vpe: cal: Stop write DMA without disabling PPI
  media: ti-vpe: cal: Use spin_lock_irq() when starting or stopping
    stream
  media: ti-vpe: cal: Share buffer release code between start and stop
  media: ti-vpe: cal: Drop V4L2_CAP_READWRITE
  media: ti-vpe: cal: Drop unneeded check in cal_calc_format_size()
  media: ti-vpe: cal: Remove DMA queue empty check at start streaming
    time
  media: ti-vpe: cal: Use list_first_entry()
  media: ti-vpe: cal: Group all DMA queue fields in struct cal_dmaqueue
  media: ti-vpe: cal: Set cal_dmaqueue.pending to NULL when no pending
    buffer
  media: ti-vpe: cal: Store buffer DMA address in dma_addr_t
  media: ti-vpe: cal: Simplify the context API
  media: ti-vpe: cal: Implement media controller centric API

 drivers/media/platform/ti-vpe/cal-camerarx.c | 374 ++++++++--
 drivers/media/platform/ti-vpe/cal-video.c    | 720 ++++++++++---------
 drivers/media/platform/ti-vpe/cal.c          | 395 ++++++++--
 drivers/media/platform/ti-vpe/cal.h          | 105 ++-
 4 files changed, 1082 insertions(+), 512 deletions(-)

-- 
Regards,

Laurent Pinchart

[PATCH v3 01/24] media: ti-vpe: cal: Create subdev for CAMERARX

[Laurent Pinchart]

Create and register V4L2 subdevs for the CAMERARX instances, and link
them in the media graph to the sensors and video nodes. The subdev API
is not exposed to userspace at this point, and no subdev operation is
implemented, but the media controller graph is visible to applications.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---
Changes since v1:

- Fix order of cleanup steps in cal_probe()
- Don't of_node_put(phy->sensor_ep_node) on success in
  cal_camerarx_parse_dt()
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 42 +++++++++++++++++++-
 drivers/media/platform/ti-vpe/cal-video.c    | 12 ++++++
 drivers/media/platform/ti-vpe/cal.c          | 35 ++++++++++++----
 drivers/media/platform/ti-vpe/cal.h          |  9 ++++-
 4 files changed, 88 insertions(+), 10 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 806cbf175d39..1c9ce30eb7af 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -533,8 +533,8 @@ static int cal_camerarx_regmap_init(struct cal_dev *cal,
 static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
 {
 	struct v4l2_fwnode_endpoint *endpoint = &phy->endpoint;
-	struct device_node *ep_node;
 	char data_lanes[V4L2_FWNODE_CSI2_MAX_DATA_LANES * 2];
+	struct device_node *ep_node;
 	unsigned int i;
 	int ret;
 
@@ -582,9 +582,11 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
 		endpoint->bus.mipi_csi2.flags);
 
 	/* Retrieve the connected device and store it for later use. */
-	phy->sensor_node = of_graph_get_remote_port_parent(ep_node);
+	phy->sensor_ep_node = of_graph_get_remote_endpoint(ep_node);
+	phy->sensor_node = of_graph_get_port_parent(phy->sensor_ep_node);
 	if (!phy->sensor_node) {
 		phy_dbg(3, phy, "Can't get remote parent\n");
+		of_node_put(phy->sensor_ep_node);
 		ret = -EINVAL;
 		goto done;
 	}
@@ -596,11 +598,25 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
 	return ret;
 }
 
+/* ------------------------------------------------------------------
+ *	V4L2 Subdev Operations
+ * ------------------------------------------------------------------
+ */
+
+static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
+};
+
+/* ------------------------------------------------------------------
+ *	Create and Destroy
+ * ------------------------------------------------------------------
+ */
+
 struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 					 unsigned int instance)
 {
 	struct platform_device *pdev = to_platform_device(cal->dev);
 	struct cal_camerarx *phy;
+	struct v4l2_subdev *sd;
 	int ret;
 
 	phy = kzalloc(sizeof(*phy), GFP_KERNEL);
@@ -632,9 +648,28 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	if (ret)
 		goto error;
 
+	/* Initialize the V4L2 subdev and media entity. */
+	sd = &phy->subdev;
+	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
+	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
+	sd->dev = cal->dev;
+
+	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+	phy->pads[CAL_CAMERARX_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(phy->pads),
+				     phy->pads);
+	if (ret)
+		goto error;
+
+	ret = v4l2_device_register_subdev(&cal->v4l2_dev, sd);
+	if (ret)
+		goto error;
+
 	return phy;
 
 error:
+	media_entity_cleanup(&phy->subdev.entity);
 	kfree(phy);
 	return ERR_PTR(ret);
 }
@@ -644,6 +679,9 @@ void cal_camerarx_destroy(struct cal_camerarx *phy)
 	if (!phy)
 		return;
 
+	v4l2_device_unregister_subdev(&phy->subdev);
+	media_entity_cleanup(&phy->subdev.entity);
+	of_node_put(phy->sensor_ep_node);
 	of_node_put(phy->sensor_node);
 	kfree(phy);
 }
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index df472a175e83..0a1a11692208 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -809,6 +809,18 @@ int cal_ctx_v4l2_register(struct cal_ctx *ctx)
 		return ret;
 	}
 
+	ret = media_create_pad_link(&ctx->phy->subdev.entity,
+				    CAL_CAMERARX_PAD_SOURCE,
+				    &vfd->entity, 0,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		ctx_err(ctx, "Failed to create media link for context %u\n",
+			ctx->index);
+		video_unregister_device(vfd);
+		return ret;
+	}
+
 	ctx_info(ctx, "V4L2 device registered as %s\n",
 		 video_device_node_name(vfd));
 
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 59a0266b1f39..f6e42b2c022a 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -421,6 +421,8 @@ static int cal_async_notifier_bound(struct v4l2_async_notifier *notifier,
 				    struct v4l2_async_subdev *asd)
 {
 	struct cal_camerarx *phy = to_cal_asd(asd)->phy;
+	int pad;
+	int ret;
 
 	if (phy->sensor) {
 		phy_info(phy, "Rejecting subdev %s (Already set!!)",
@@ -431,6 +433,25 @@ static int cal_async_notifier_bound(struct v4l2_async_notifier *notifier,
 	phy->sensor = subdev;
 	phy_dbg(1, phy, "Using sensor %s for capture\n", subdev->name);
 
+	pad = media_entity_get_fwnode_pad(&subdev->entity,
+					  of_fwnode_handle(phy->sensor_ep_node),
+					  MEDIA_PAD_FL_SOURCE);
+	if (pad < 0) {
+		phy_err(phy, "Sensor %s has no connected source pad\n",
+			subdev->name);
+		return pad;
+	}
+
+	ret = media_create_pad_link(&subdev->entity, pad,
+				    &phy->subdev.entity, CAL_CAMERARX_PAD_SINK,
+				    MEDIA_LNK_FL_IMMUTABLE |
+				    MEDIA_LNK_FL_ENABLED);
+	if (ret) {
+		phy_err(phy, "Failed to create media link for sensor %s\n",
+			subdev->name);
+		return ret;
+	}
+
 	return 0;
 }
 
@@ -797,6 +818,11 @@ static int cal_probe(struct platform_device *pdev)
 	cal_get_hwinfo(cal);
 	pm_runtime_put_sync(&pdev->dev);
 
+	/* Initialize the media device. */
+	ret = cal_media_init(cal);
+	if (ret < 0)
+		goto error_pm_runtime;
+
 	/* Create CAMERARX PHYs. */
 	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
 		cal->phy[i] = cal_camerarx_create(cal, i);
@@ -816,11 +842,6 @@ static int cal_probe(struct platform_device *pdev)
 		goto error_camerarx;
 	}
 
-	/* Initialize the media device. */
-	ret = cal_media_init(cal);
-	if (ret < 0)
-		goto error_camerarx;
-
 	/* Create contexts. */
 	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
 		if (!cal->phy[i]->sensor_node)
@@ -848,12 +869,12 @@ static int cal_probe(struct platform_device *pdev)
 			cal_ctx_v4l2_cleanup(ctx);
 	}
 
-	cal_media_cleanup(cal);
-
 error_camerarx:
 	for (i = 0; i < ARRAY_SIZE(cal->phy); i++)
 		cal_camerarx_destroy(cal->phy[i]);
 
+	cal_media_cleanup(cal);
+
 error_pm_runtime:
 	pm_runtime_disable(&pdev->dev);
 
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 4123405ee0cf..9bb6cc1bdbcc 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -24,6 +24,7 @@
 #include <media/v4l2-dev.h>
 #include <media/v4l2-device.h>
 #include <media/v4l2-fwnode.h>
+#include <media/v4l2-subdev.h>
 #include <media/videobuf2-v4l2.h>
 
 #define CAL_MODULE_NAME			"cal"
@@ -33,12 +34,14 @@
 #define MAX_WIDTH_BYTES			(8192 * 8)
 #define MAX_HEIGHT_LINES		16383
 
+#define CAL_CAMERARX_PAD_SINK		0
+#define CAL_CAMERARX_PAD_SOURCE		1
+
 struct device;
 struct device_node;
 struct resource;
 struct regmap;
 struct regmap_fied;
-struct v4l2_subdev;
 
 /* CTRL_CORE_CAMERRX_CONTROL register field id */
 enum cal_camerarx_field {
@@ -108,8 +111,12 @@ struct cal_camerarx {
 	unsigned int		instance;
 
 	struct v4l2_fwnode_endpoint	endpoint;
+	struct device_node	*sensor_ep_node;
 	struct device_node	*sensor_node;
 	struct v4l2_subdev	*sensor;
+
+	struct v4l2_subdev	subdev;
+	struct media_pad	pads[2];
 };
 
 struct cal_dev {
-- 
Regards,

Laurent Pinchart

[PATCH v3 02/24] media: ti-vpe: cal: Drop cal_ctx m_fmt field

[Laurent Pinchart]

The struct cal_ctx m_fmt field stores the media bus format for the
context input. Only the format 'field' field is used, store it in the
video format instead, and drop m_fmt.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 10 +++++-----
 drivers/media/platform/ti-vpe/cal.c       |  2 +-
 drivers/media/platform/ti-vpe/cal.h       |  4 +---
 3 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 0a1a11692208..f57767e79ca5 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -387,10 +387,10 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 
 	v4l2_fill_pix_format(&ctx->v_fmt.fmt.pix, &mbus_fmt);
 	ctx->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-	ctx->v_fmt.fmt.pix.pixelformat  = fmt->fourcc;
+	ctx->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
+	ctx->v_fmt.fmt.pix.field = mbus_fmt.field;
 	cal_calc_format_size(ctx, fmt, &ctx->v_fmt);
 	ctx->fmt = fmt;
-	ctx->m_fmt = mbus_fmt;
 	*f = ctx->v_fmt;
 
 	return 0;
@@ -775,13 +775,13 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 		return -EINVAL;
 	}
 
-	/* Save current subdev format */
+	/* Save current format */
 	v4l2_fill_pix_format(&ctx->v_fmt.fmt.pix, &mbus_fmt);
 	ctx->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-	ctx->v_fmt.fmt.pix.pixelformat  = fmt->fourcc;
+	ctx->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
+	ctx->v_fmt.fmt.pix.field = mbus_fmt.field;
 	cal_calc_format_size(ctx, fmt, &ctx->v_fmt);
 	ctx->fmt = fmt;
-	ctx->m_fmt = mbus_fmt;
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index f6e42b2c022a..34a344b7f08a 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -314,7 +314,7 @@ static inline void cal_schedule_next_buffer(struct cal_ctx *ctx)
 static inline void cal_process_buffer_complete(struct cal_ctx *ctx)
 {
 	ctx->cur_frm->vb.vb2_buf.timestamp = ktime_get_ns();
-	ctx->cur_frm->vb.field = ctx->m_fmt.field;
+	ctx->cur_frm->vb.field = ctx->v_fmt.fmt.pix.field;
 	ctx->cur_frm->vb.sequence = ctx->sequence++;
 
 	vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 9bb6cc1bdbcc..955dacd878e7 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -164,9 +164,7 @@ struct cal_ctx {
 	/* video capture */
 	const struct cal_fmt	*fmt;
 	/* Used to store current pixel format */
-	struct v4l2_format		v_fmt;
-	/* Used to store current mbus frame format */
-	struct v4l2_mbus_framefmt	m_fmt;
+	struct v4l2_format	v_fmt;
 
 	/* Current subdev enumerated format */
 	const struct cal_fmt	**active_fmt;
-- 
Regards,

Laurent Pinchart

[PATCH v3 03/24] media: ti-vpe: cal: Move format handling to cal.c and expose helpers

[Laurent Pinchart]

The cal_formats array contain the description of all formats supported
by the hardware. It's currently used by the V4L2 video device operations
only, but will be needed by the CAMERARX subdev code too. Move it from
cal-video.c to cal.c and add helper functions to access it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 105 +-----------------
 drivers/media/platform/ti-vpe/cal.c       | 127 ++++++++++++++++++++++
 drivers/media/platform/ti-vpe/cal.h       |   5 +
 3 files changed, 134 insertions(+), 103 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index f57767e79ca5..355bb365daf0 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -26,107 +26,6 @@
 
 #include "cal.h"
 
-/* ------------------------------------------------------------------
- *	Format Handling
- * ------------------------------------------------------------------
- */
-
-static const struct cal_fmt cal_formats[] = {
-	{
-		.fourcc		= V4L2_PIX_FMT_YUYV,
-		.code		= MEDIA_BUS_FMT_YUYV8_2X8,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_UYVY,
-		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_YVYU,
-		.code		= MEDIA_BUS_FMT_YVYU8_2X8,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_VYUY,
-		.code		= MEDIA_BUS_FMT_VYUY8_2X8,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
-		.code		= MEDIA_BUS_FMT_RGB565_2X8_LE,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_RGB565X, /* rrrrrggg gggbbbbb */
-		.code		= MEDIA_BUS_FMT_RGB565_2X8_BE,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_RGB555, /* gggbbbbb arrrrrgg */
-		.code		= MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_RGB555X, /* arrrrrgg gggbbbbb */
-		.code		= MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
-		.bpp		= 16,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_RGB24, /* rgb */
-		.code		= MEDIA_BUS_FMT_RGB888_2X12_LE,
-		.bpp		= 24,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_BGR24, /* bgr */
-		.code		= MEDIA_BUS_FMT_RGB888_2X12_BE,
-		.bpp		= 24,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_RGB32, /* argb */
-		.code		= MEDIA_BUS_FMT_ARGB8888_1X32,
-		.bpp		= 32,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SBGGR8,
-		.code		= MEDIA_BUS_FMT_SBGGR8_1X8,
-		.bpp		= 8,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SGBRG8,
-		.code		= MEDIA_BUS_FMT_SGBRG8_1X8,
-		.bpp		= 8,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SGRBG8,
-		.code		= MEDIA_BUS_FMT_SGRBG8_1X8,
-		.bpp		= 8,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SRGGB8,
-		.code		= MEDIA_BUS_FMT_SRGGB8_1X8,
-		.bpp		= 8,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SBGGR10,
-		.code		= MEDIA_BUS_FMT_SBGGR10_1X10,
-		.bpp		= 10,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SGBRG10,
-		.code		= MEDIA_BUS_FMT_SGBRG10_1X10,
-		.bpp		= 10,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SGRBG10,
-		.code		= MEDIA_BUS_FMT_SGRBG10_1X10,
-		.bpp		= 10,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SRGGB10,
-		.code		= MEDIA_BUS_FMT_SRGGB10_1X10,
-		.bpp		= 10,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SBGGR12,
-		.code		= MEDIA_BUS_FMT_SBGGR12_1X12,
-		.bpp		= 12,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SGBRG12,
-		.code		= MEDIA_BUS_FMT_SGBRG12_1X12,
-		.bpp		= 12,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SGRBG12,
-		.code		= MEDIA_BUS_FMT_SGRBG12_1X12,
-		.bpp		= 12,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_SRGGB12,
-		.code		= MEDIA_BUS_FMT_SRGGB12_1X12,
-		.bpp		= 12,
-	},
-};
-
 /*  Print Four-character-code (FOURCC) */
 static char *fourcc_to_str(u32 fmt)
 {
@@ -726,7 +625,7 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 	int ret = 0;
 
 	/* Enumerate sub device formats and enable all matching local formats */
-	ctx->active_fmt = devm_kcalloc(ctx->cal->dev, ARRAY_SIZE(cal_formats),
+	ctx->active_fmt = devm_kcalloc(ctx->cal->dev, cal_num_formats,
 				       sizeof(*ctx->active_fmt), GFP_KERNEL);
 	ctx->num_active_fmt = 0;
 
@@ -744,7 +643,7 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 			"subdev %s: code: %04x idx: %u\n",
 			ctx->phy->sensor->name, mbus_code.code, j);
 
-		for (k = 0; k < ARRAY_SIZE(cal_formats); k++) {
+		for (k = 0; k < cal_num_formats; k++) {
 			const struct cal_fmt *fmt = &cal_formats[k];
 
 			if (mbus_code.code == fmt->code) {
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 34a344b7f08a..b9ee535513e8 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -43,6 +43,133 @@ unsigned int cal_debug;
 module_param_named(debug, cal_debug, uint, 0644);
 MODULE_PARM_DESC(debug, "activates debug info");
 
+/* ------------------------------------------------------------------
+ *	Format Handling
+ * ------------------------------------------------------------------
+ */
+
+const struct cal_fmt cal_formats[] = {
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.code		= MEDIA_BUS_FMT_YUYV8_2X8,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_UYVY,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.code		= MEDIA_BUS_FMT_YVYU8_2X8,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_VYUY,
+		.code		= MEDIA_BUS_FMT_VYUY8_2X8,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
+		.code		= MEDIA_BUS_FMT_RGB565_2X8_LE,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_RGB565X, /* rrrrrggg gggbbbbb */
+		.code		= MEDIA_BUS_FMT_RGB565_2X8_BE,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_RGB555, /* gggbbbbb arrrrrgg */
+		.code		= MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_RGB555X, /* arrrrrgg gggbbbbb */
+		.code		= MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
+		.bpp		= 16,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_RGB24, /* rgb */
+		.code		= MEDIA_BUS_FMT_RGB888_2X12_LE,
+		.bpp		= 24,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_BGR24, /* bgr */
+		.code		= MEDIA_BUS_FMT_RGB888_2X12_BE,
+		.bpp		= 24,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_RGB32, /* argb */
+		.code		= MEDIA_BUS_FMT_ARGB8888_1X32,
+		.bpp		= 32,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SBGGR8,
+		.code		= MEDIA_BUS_FMT_SBGGR8_1X8,
+		.bpp		= 8,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGBRG8,
+		.code		= MEDIA_BUS_FMT_SGBRG8_1X8,
+		.bpp		= 8,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGRBG8,
+		.code		= MEDIA_BUS_FMT_SGRBG8_1X8,
+		.bpp		= 8,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SRGGB8,
+		.code		= MEDIA_BUS_FMT_SRGGB8_1X8,
+		.bpp		= 8,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SBGGR10,
+		.code		= MEDIA_BUS_FMT_SBGGR10_1X10,
+		.bpp		= 10,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGBRG10,
+		.code		= MEDIA_BUS_FMT_SGBRG10_1X10,
+		.bpp		= 10,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGRBG10,
+		.code		= MEDIA_BUS_FMT_SGRBG10_1X10,
+		.bpp		= 10,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SRGGB10,
+		.code		= MEDIA_BUS_FMT_SRGGB10_1X10,
+		.bpp		= 10,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SBGGR12,
+		.code		= MEDIA_BUS_FMT_SBGGR12_1X12,
+		.bpp		= 12,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGBRG12,
+		.code		= MEDIA_BUS_FMT_SGBRG12_1X12,
+		.bpp		= 12,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SGRBG12,
+		.code		= MEDIA_BUS_FMT_SGRBG12_1X12,
+		.bpp		= 12,
+	}, {
+		.fourcc		= V4L2_PIX_FMT_SRGGB12,
+		.code		= MEDIA_BUS_FMT_SRGGB12_1X12,
+		.bpp		= 12,
+	},
+};
+
+const unsigned int cal_num_formats = ARRAY_SIZE(cal_formats);
+
+const struct cal_fmt *cal_format_by_fourcc(u32 fourcc)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(cal_formats); ++i) {
+		if (cal_formats[i].fourcc == fourcc)
+			return &cal_formats[i];
+	}
+
+	return NULL;
+}
+
+const struct cal_fmt *cal_format_by_code(u32 code)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(cal_formats); ++i) {
+		if (cal_formats[i].code == code)
+			return &cal_formats[i];
+	}
+
+	return NULL;
+}
+
 /* ------------------------------------------------------------------
  *	Platform Data
  * ------------------------------------------------------------------
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 955dacd878e7..6e6bdf8af3d0 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -244,6 +244,11 @@ static inline void cal_set_field(u32 *valp, u32 field, u32 mask)
 	*valp = val;
 }
 
+extern const struct cal_fmt cal_formats[];
+extern const unsigned int cal_num_formats;
+const struct cal_fmt *cal_format_by_fourcc(u32 fourcc);
+const struct cal_fmt *cal_format_by_code(u32 code);
+
 void cal_quickdump_regs(struct cal_dev *cal);
 
 void cal_camerarx_disable(struct cal_camerarx *phy);
-- 
Regards,

Laurent Pinchart

[PATCH v3 04/24] media: ti-vpe: cal: Rename MAX_(WIDTH|HEIGHT)_* macros with CAL_ prefix

[Laurent Pinchart]

The MAX_WIDTH_BYTES and MAX_HEIGHT_LINES macros have a generic name that
is prone to namespace clashes. Rename them with a CAL_ prefix. While at
it, expand their documentation and add CAL_MIN_(WIDTH|HEIGHT)_* macros
that will be used to implement CAMERARX subdev operations.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c |  5 +++--
 drivers/media/platform/ti-vpe/cal.h       | 12 ++++++++++--
 2 files changed, 13 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 355bb365daf0..42e750925e8b 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -163,9 +163,10 @@ static int cal_calc_format_size(struct cal_ctx *ctx,
 	 * We need to recalculate the actual maxi width depending on the
 	 * number of bytes per pixels required.
 	 */
-	max_width = MAX_WIDTH_BYTES / (ALIGN(fmt->bpp, 8) >> 3);
+	max_width = CAL_MAX_WIDTH_BYTES / (ALIGN(fmt->bpp, 8) >> 3);
 	v4l_bound_align_image(&f->fmt.pix.width, 48, max_width, 2,
-			      &f->fmt.pix.height, 32, MAX_HEIGHT_LINES, 0, 0);
+			      &f->fmt.pix.height, 32, CAL_MAX_HEIGHT_LINES,
+			      0, 0);
 
 	bpl = (f->fmt.pix.width * ALIGN(fmt->bpp, 8)) >> 3;
 	f->fmt.pix.bytesperline = ALIGN(bpl, 16);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 6e6bdf8af3d0..d42d381d928f 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -31,8 +31,16 @@
 #define CAL_NUM_CONTEXT			2
 #define CAL_NUM_CSI2_PORTS		2
 
-#define MAX_WIDTH_BYTES			(8192 * 8)
-#define MAX_HEIGHT_LINES		16383
+/*
+ * The width is limited by the size of the CAL_WR_DMA_XSIZE_j.XSIZE field,
+ * expressed in multiples of 64 bits. The height is limited by the size of the
+ * CAL_CSI2_CTXi_j.CTXi_LINES and CAL_WR_DMA_CTRL_j.YSIZE fields, expressed in
+ * lines.
+ */
+#define CAL_MIN_WIDTH_BYTES		16
+#define CAL_MAX_WIDTH_BYTES		(8192 * 8)
+#define CAL_MIN_HEIGHT_LINES		1
+#define CAL_MAX_HEIGHT_LINES		16383
 
 #define CAL_CAMERARX_PAD_SINK		0
 #define CAL_CAMERARX_PAD_SOURCE		1
-- 
Regards,

Laurent Pinchart

[PATCH v3 05/24] media: ti-vpe: cal: Replace hardcoded BIT() value with macro

[Laurent Pinchart]

Replace the hardcoded BIT(3) value with CAL_CSI2_PPI_CTRL_FRAME_MASK to
increase readability.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 1c9ce30eb7af..6af09d4c0049 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -485,7 +485,8 @@ void cal_camerarx_disable_irqs(struct cal_camerarx *phy)
 
 void cal_camerarx_ppi_enable(struct cal_camerarx *phy)
 {
-	cal_write(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance), BIT(3));
+	cal_write(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
+		  CAL_CSI2_PPI_CTRL_FRAME_MASK);
 	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
 			1, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
 }
-- 
Regards,

Laurent Pinchart

[PATCH v3 06/24] media: ti-vpe: cal: Iterate over correct number of CAMERARX instances

[Laurent Pinchart]

When performing operations on all CAMERARX instances, code usually
iterates over all cal->phy[] entries and skips the NULL pointers. The
number of available CAMERARX instances is however available through
cal->data->num_csi2_phy. Use it instead.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index b9ee535513e8..5d073d9cd8b5 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -263,12 +263,9 @@ void cal_quickdump_regs(struct cal_dev *cal)
 		       (__force const void *)cal->base,
 		       resource_size(cal->res), false);
 
-	for (i = 0; i < ARRAY_SIZE(cal->phy); ++i) {
+	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
 		struct cal_camerarx *phy = cal->phy[i];
 
-		if (!phy)
-			continue;
-
 		cal_info(cal, "CSI2 Core %u Registers @ %pa:\n", i,
 			 &phy->res->start);
 		print_hex_dump(KERN_INFO, "", DUMP_PREFIX_OFFSET, 16, 4,
@@ -464,7 +461,7 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 		if (status & CAL_HL_IRQ_OCPO_ERR_MASK)
 			dev_err_ratelimited(cal->dev, "OCPO ERROR\n");
 
-		for (i = 0; i < CAL_NUM_CSI2_PORTS; ++i) {
+		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
 			if (status & CAL_HL_IRQ_CIO_MASK(i)) {
 				u32 cio_stat = cal_read(cal,
 							CAL_CSI2_COMPLEXIO_IRQSTATUS(i));
@@ -608,13 +605,13 @@ static int cal_async_notifier_register(struct cal_dev *cal)
 	v4l2_async_notifier_init(&cal->notifier);
 	cal->notifier.ops = &cal_async_notifier_ops;
 
-	for (i = 0; i < ARRAY_SIZE(cal->phy); ++i) {
+	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
 		struct cal_camerarx *phy = cal->phy[i];
 		struct cal_v4l2_async_subdev *casd;
 		struct v4l2_async_subdev *asd;
 		struct fwnode_handle *fwnode;
 
-		if (!phy || !phy->sensor_node)
+		if (!phy->sensor_node)
 			continue;
 
 		fwnode = of_fwnode_handle(phy->sensor_node);
@@ -997,7 +994,7 @@ static int cal_probe(struct platform_device *pdev)
 	}
 
 error_camerarx:
-	for (i = 0; i < ARRAY_SIZE(cal->phy); i++)
+	for (i = 0; i < cal->data->num_csi2_phy; i++)
 		cal_camerarx_destroy(cal->phy[i]);
 
 	cal_media_cleanup(cal);
@@ -1026,7 +1023,7 @@ static int cal_remove(struct platform_device *pdev)
 
 	cal_media_cleanup(cal);
 
-	for (i = 0; i < ARRAY_SIZE(cal->phy); i++)
+	for (i = 0; i < cal->data->num_csi2_phy; i++)
 		cal_camerarx_destroy(cal->phy[i]);
 
 	pm_runtime_put_sync(&pdev->dev);
@@ -1038,14 +1035,15 @@ static int cal_remove(struct platform_device *pdev)
 static int cal_runtime_resume(struct device *dev)
 {
 	struct cal_dev *cal = dev_get_drvdata(dev);
+	unsigned int i;
 
 	if (cal->data->flags & DRA72_CAL_PRE_ES2_LDO_DISABLE) {
 		/*
 		 * Apply errata on both port everytime we (re-)enable
 		 * the clock
 		 */
-		cal_camerarx_i913_errata(cal->phy[0]);
-		cal_camerarx_i913_errata(cal->phy[1]);
+		for (i = 0; i < cal->data->num_csi2_phy; i++)
+			cal_camerarx_i913_errata(cal->phy[i]);
 	}
 
 	return 0;
-- 
Regards,

Laurent Pinchart

[PATCH v3 07/24] media: ti-vpe: cal: Implement subdev ops for CAMERARX

[Laurent Pinchart]

Implement subdev operations for the CAMERARX. They will be used to
replace calls to custom CAMERARX functions in the V4L2 video device
code, and will be exposed to userspace.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 205 +++++++++++++++++++
 drivers/media/platform/ti-vpe/cal.h          |   2 +
 2 files changed, 207 insertions(+)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 6af09d4c0049..ca941e3e72e9 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -604,7 +604,209 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
  * ------------------------------------------------------------------
  */
 
+static inline struct cal_camerarx *to_cal_camerarx(struct v4l2_subdev *sd)
+{
+	return container_of(sd, struct cal_camerarx, subdev);
+}
+
+static struct v4l2_mbus_framefmt *
+cal_camerarx_get_pad_format(struct cal_camerarx *phy,
+			    struct v4l2_subdev_pad_config *cfg,
+			    unsigned int pad, u32 which)
+{
+	switch (which) {
+	case V4L2_SUBDEV_FORMAT_TRY:
+		return v4l2_subdev_get_try_format(&phy->subdev, cfg, pad);
+	case V4L2_SUBDEV_FORMAT_ACTIVE:
+		return &phy->formats[pad];
+	default:
+		return NULL;
+	}
+}
+
+static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+
+	if (enable)
+		return cal_camerarx_start(phy, NULL);
+
+	cal_camerarx_stop(phy);
+	return 0;
+}
+
+static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
+					  struct v4l2_subdev_pad_config *cfg,
+					  struct v4l2_subdev_mbus_code_enum *code)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+
+	/* No transcoding, source and sink codes must match. */
+	if (code->pad == CAL_CAMERARX_PAD_SOURCE) {
+		struct v4l2_mbus_framefmt *fmt;
+
+		if (code->index > 0)
+			return -EINVAL;
+
+		fmt = cal_camerarx_get_pad_format(phy, cfg,
+						  CAL_CAMERARX_PAD_SINK,
+						  code->which);
+		code->code = fmt->code;
+		return 0;
+	}
+
+	if (code->index >= cal_num_formats)
+		return -EINVAL;
+
+	code->code = cal_formats[code->index].code;
+
+	return 0;
+}
+
+static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
+					   struct v4l2_subdev_pad_config *cfg,
+					   struct v4l2_subdev_frame_size_enum *fse)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+	const struct cal_fmt *fmtinfo;
+
+	if (fse->index > 0)
+		return -EINVAL;
+
+	/* No transcoding, source and sink formats must match. */
+	if (fse->pad == CAL_CAMERARX_PAD_SOURCE) {
+		struct v4l2_mbus_framefmt *fmt;
+
+		fmt = cal_camerarx_get_pad_format(phy, cfg,
+						  CAL_CAMERARX_PAD_SINK,
+						  fse->which);
+		if (fse->code != fmt->code)
+			return -EINVAL;
+
+		fse->min_width = fmt->width;
+		fse->max_width = fmt->width;
+		fse->min_height = fmt->height;
+		fse->max_height = fmt->height;
+
+		return 0;
+	}
+
+	fmtinfo = cal_format_by_code(fse->code);
+	if (!fmtinfo)
+		return -EINVAL;
+
+	fse->min_width = CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
+	fse->max_width = CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
+	fse->min_height = CAL_MIN_HEIGHT_LINES;
+	fse->max_height = CAL_MAX_HEIGHT_LINES;
+
+	return 0;
+}
+
+static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_pad_config *cfg,
+				   struct v4l2_subdev_format *format)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+	struct v4l2_mbus_framefmt *fmt;
+
+	fmt = cal_camerarx_get_pad_format(phy, cfg, format->pad, format->which);
+	format->format = *fmt;
+
+	return 0;
+}
+
+static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
+				   struct v4l2_subdev_pad_config *cfg,
+				   struct v4l2_subdev_format *format)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+	const struct cal_fmt *fmtinfo;
+	struct v4l2_mbus_framefmt *fmt;
+	unsigned int bpp;
+
+	/* No transcoding, source and sink formats must match. */
+	if (format->pad == CAL_CAMERARX_PAD_SOURCE)
+		return cal_camerarx_sd_get_fmt(sd, cfg, format);
+
+	/*
+	 * Default to the first format is the requested media bus code isn't
+	 * supported.
+	 */
+	fmtinfo = cal_format_by_code(format->format.code);
+	if (!fmtinfo)
+		fmtinfo = &cal_formats[0];
+
+	/*
+	 * Clamp the size, update the code. The field and colorspace are
+	 * accepted as-is.
+	 */
+	bpp = ALIGN(fmtinfo->bpp, 8);
+
+	format->format.width = clamp_t(unsigned int, format->format.width,
+				       CAL_MIN_WIDTH_BYTES * 8 / bpp,
+				       CAL_MAX_WIDTH_BYTES * 8 / bpp);
+	format->format.height = clamp_t(unsigned int, format->format.height,
+					CAL_MIN_HEIGHT_LINES,
+					CAL_MAX_HEIGHT_LINES);
+	format->format.code = fmtinfo->code;
+
+	/* Store the format and propagate it to the source pad. */
+	fmt = cal_camerarx_get_pad_format(phy, cfg, CAL_CAMERARX_PAD_SINK,
+					  format->which);
+	*fmt = format->format;
+
+	fmt = cal_camerarx_get_pad_format(phy, cfg, CAL_CAMERARX_PAD_SOURCE,
+					  format->which);
+	*fmt = format->format;
+
+	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
+		phy->fmtinfo = fmtinfo;
+
+	return 0;
+}
+
+static int cal_camerarx_sd_init_cfg(struct v4l2_subdev *sd,
+				    struct v4l2_subdev_pad_config *cfg)
+{
+	struct v4l2_subdev_format format = {
+		.which = cfg ? V4L2_SUBDEV_FORMAT_TRY
+		       : V4L2_SUBDEV_FORMAT_ACTIVE,
+		.pad = CAL_CAMERARX_PAD_SINK,
+		.format = {
+			.width = 640,
+			.height = 480,
+			.code = MEDIA_BUS_FMT_UYVY8_2X8,
+			.field = V4L2_FIELD_NONE,
+			.colorspace = V4L2_COLORSPACE_SRGB,
+			.ycbcr_enc = V4L2_YCBCR_ENC_601,
+			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+			.xfer_func = V4L2_XFER_FUNC_SRGB,
+		},
+	};
+
+	return cal_camerarx_sd_set_fmt(sd, cfg, &format);
+}
+
+static const struct v4l2_subdev_video_ops cal_camerarx_video_ops = {
+	.s_stream = cal_camerarx_sd_s_stream,
+};
+
+static const struct v4l2_subdev_pad_ops cal_camerarx_pad_ops = {
+	.init_cfg = cal_camerarx_sd_init_cfg,
+	.enum_mbus_code = cal_camerarx_sd_enum_mbus_code,
+	.enum_frame_size = cal_camerarx_sd_enum_frame_size,
+	.get_fmt = cal_camerarx_sd_get_fmt,
+	.set_fmt = cal_camerarx_sd_set_fmt,
+};
+
 static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
+	.video = &cal_camerarx_video_ops,
+	.pad = &cal_camerarx_pad_ops,
+};
+
+struct media_entity_operations cal_camerarx_media_ops = {
+	.link_validate = v4l2_subdev_link_validate,
 };
 
 /* ------------------------------------------------------------------
@@ -658,11 +860,14 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 
 	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
 	phy->pads[CAL_CAMERARX_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+	sd->entity.ops = &cal_camerarx_media_ops;
 	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(phy->pads),
 				     phy->pads);
 	if (ret)
 		goto error;
 
+	cal_camerarx_sd_init_cfg(sd, NULL);
+
 	ret = v4l2_device_register_subdev(&cal->v4l2_dev, sd);
 	if (ret)
 		goto error;
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index d42d381d928f..241edd8b7536 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -125,6 +125,8 @@ struct cal_camerarx {
 
 	struct v4l2_subdev	subdev;
 	struct media_pad	pads[2];
+	struct v4l2_mbus_framefmt	formats[2];
+	const struct cal_fmt	*fmtinfo;
 };
 
 struct cal_dev {
-- 
Regards,

Laurent Pinchart

[PATCH v3 08/24] media: ti-vpe: cal: Use CAMERARX subdev s_stream op in video device code

[Laurent Pinchart]

Replace calls to cal_camerarx_start() and cal_camerarx_stop() with usage
of the .s_stream() subdev operation. This requires calling the
.set_fmt() operation as the CAMERARX now relies on the format
information set through the subdev API instead of receiving it in the
cal_camerarx_start() function.

This change prepare for exposing the CAMERARX subdev operations to
userspace by using the same API within the kernel.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 15 ++++++------
 drivers/media/platform/ti-vpe/cal-video.c    | 25 +++++++++++---------
 drivers/media/platform/ti-vpe/cal.h          |  2 --
 3 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index ca941e3e72e9..584dc548504d 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -116,8 +116,7 @@ void cal_camerarx_disable(struct cal_camerarx *phy)
 #define TCLK_MISS	1
 #define TCLK_SETTLE	14
 
-static void cal_camerarx_config(struct cal_camerarx *phy, s64 external_rate,
-				const struct cal_fmt *fmt)
+static void cal_camerarx_config(struct cal_camerarx *phy, s64 external_rate)
 {
 	unsigned int reg0, reg1;
 	unsigned int ths_term, ths_settle;
@@ -132,9 +131,9 @@ static void cal_camerarx_config(struct cal_camerarx *phy, s64 external_rate,
 	 * CSI-2 is DDR and we only count used lanes.
 	 *
 	 * csi2_ddrclk_khz = external_rate / 1000
-	 *		   / (2 * num_lanes) * fmt->bpp;
+	 *		   / (2 * num_lanes) * phy->fmtinfo->bpp;
 	 */
-	csi2_ddrclk_khz = div_s64(external_rate * fmt->bpp,
+	csi2_ddrclk_khz = div_s64(external_rate * phy->fmtinfo->bpp,
 				  2 * num_lanes * 1000);
 
 	phy_dbg(1, phy, "csi2_ddrclk_khz: %d\n", csi2_ddrclk_khz);
@@ -234,7 +233,7 @@ static void cal_camerarx_wait_stop_state(struct cal_camerarx *phy)
 		phy_err(phy, "Timeout waiting for stop state\n");
 }
 
-int cal_camerarx_start(struct cal_camerarx *phy, const struct cal_fmt *fmt)
+static int cal_camerarx_start(struct cal_camerarx *phy)
 {
 	s64 external_rate;
 	u32 sscounter;
@@ -289,7 +288,7 @@ int cal_camerarx_start(struct cal_camerarx *phy, const struct cal_fmt *fmt)
 	camerarx_read(phy, CAL_CSI2_PHY_REG0);
 
 	/* Program the PHY timing parameters. */
-	cal_camerarx_config(phy, external_rate, fmt);
+	cal_camerarx_config(phy, external_rate);
 
 	/*
 	 *    b. Assert the FORCERXMODE signal.
@@ -362,7 +361,7 @@ int cal_camerarx_start(struct cal_camerarx *phy, const struct cal_fmt *fmt)
 	return 0;
 }
 
-void cal_camerarx_stop(struct cal_camerarx *phy)
+static void cal_camerarx_stop(struct cal_camerarx *phy)
 {
 	unsigned int i;
 	int ret;
@@ -629,7 +628,7 @@ static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
 	struct cal_camerarx *phy = to_cal_camerarx(sd);
 
 	if (enable)
-		return cal_camerarx_start(phy, NULL);
+		return cal_camerarx_start(phy);
 
 	cal_camerarx_stop(phy);
 	return 0;
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 42e750925e8b..1ada27d42da1 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -256,8 +256,11 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 	struct vb2_queue *q = &ctx->vb_vidq;
+	struct v4l2_subdev_format sd_fmt = {
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+		.pad = CAL_CAMERARX_PAD_SINK,
+	};
 	const struct cal_fmt *fmt;
-	struct v4l2_mbus_framefmt mbus_fmt;
 	int ret;
 
 	if (vb2_is_busy(q)) {
@@ -271,25 +274,28 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 
 	fmt = find_format_by_pix(ctx, f->fmt.pix.pixelformat);
 
-	v4l2_fill_mbus_format(&mbus_fmt, &f->fmt.pix, fmt->code);
+	v4l2_fill_mbus_format(&sd_fmt.format, &f->fmt.pix, fmt->code);
 
-	ret = __subdev_set_format(ctx, &mbus_fmt);
+	ret = __subdev_set_format(ctx, &sd_fmt.format);
 	if (ret)
 		return ret;
 
 	/* Just double check nothing has gone wrong */
-	if (mbus_fmt.code != fmt->code) {
+	if (sd_fmt.format.code != fmt->code) {
 		ctx_dbg(3, ctx,
 			"%s subdev changed format on us, this should not happen\n",
 			__func__);
 		return -EINVAL;
 	}
 
-	v4l2_fill_pix_format(&ctx->v_fmt.fmt.pix, &mbus_fmt);
+	v4l2_fill_pix_format(&ctx->v_fmt.fmt.pix, &sd_fmt.format);
 	ctx->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	ctx->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
-	ctx->v_fmt.fmt.pix.field = mbus_fmt.field;
+	ctx->v_fmt.fmt.pix.field = sd_fmt.format.field;
 	cal_calc_format_size(ctx, fmt, &ctx->v_fmt);
+
+	v4l2_subdev_call(&ctx->phy->subdev, pad, set_fmt, NULL, &sd_fmt);
+
 	ctx->fmt = fmt;
 	*f = ctx->v_fmt;
 
@@ -455,9 +461,6 @@ static int cal_buffer_prepare(struct vb2_buffer *vb)
 					      vb.vb2_buf);
 	unsigned long size;
 
-	if (WARN_ON(!ctx->fmt))
-		return -EINVAL;
-
 	size = ctx->v_fmt.fmt.pix.sizeimage;
 	if (vb2_plane_size(vb, 0) < size) {
 		ctx_err(ctx,
@@ -518,7 +521,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	cal_camerarx_enable_irqs(ctx->phy);
 
-	ret = cal_camerarx_start(ctx->phy, ctx->fmt);
+	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
 	if (ret)
 		goto err;
 
@@ -569,7 +572,7 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 		ctx_err(ctx, "failed to disable dma cleanly\n");
 
 	cal_camerarx_disable_irqs(ctx->phy);
-	cal_camerarx_stop(ctx->phy);
+	v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 0);
 
 	/* Release all active buffers */
 	spin_lock_irqsave(&ctx->slock, flags);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 241edd8b7536..45f65ebf7aa9 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -262,8 +262,6 @@ const struct cal_fmt *cal_format_by_code(u32 code);
 void cal_quickdump_regs(struct cal_dev *cal);
 
 void cal_camerarx_disable(struct cal_camerarx *phy);
-int cal_camerarx_start(struct cal_camerarx *phy, const struct cal_fmt *fmt);
-void cal_camerarx_stop(struct cal_camerarx *phy);
 void cal_camerarx_enable_irqs(struct cal_camerarx *phy);
 void cal_camerarx_disable_irqs(struct cal_camerarx *phy);
 void cal_camerarx_ppi_enable(struct cal_camerarx *phy);
-- 
Regards,

Laurent Pinchart

[PATCH v3 09/24] media: ti-vpe: cal: Don't pass format to cal_ctx_wr_dma_config()

[Laurent Pinchart]

The cal_ctx_wr_dma_config() function has access to the context, there's
no need to give it format-related values retrieved from the context.
Access the values internally, and reword internal comments. The comment
regarding the CAL_WR_DMA_OFST register not being well understood is
dropped, as the datasheet explicitly documents it as "Offset between two
consecutive line starts".

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c |  3 +--
 drivers/media/platform/ti-vpe/cal.c       | 25 +++++++++--------------
 drivers/media/platform/ti-vpe/cal.h       |  3 +--
 3 files changed, 12 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 1ada27d42da1..083389635269 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -516,8 +516,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	cal_ctx_csi2_config(ctx);
 	cal_ctx_pix_proc_config(ctx);
-	cal_ctx_wr_dma_config(ctx, ctx->v_fmt.fmt.pix.bytesperline,
-			      ctx->v_fmt.fmt.pix.height);
+	cal_ctx_wr_dma_config(ctx);
 
 	cal_camerarx_enable_irqs(ctx->phy);
 
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 5d073d9cd8b5..4c938ca1f1a9 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -356,14 +356,15 @@ void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
 		cal_read(ctx->cal, CAL_PIX_PROC(ctx->index)));
 }
 
-void cal_ctx_wr_dma_config(struct cal_ctx *ctx, unsigned int width,
-			    unsigned int height)
+void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 {
+	unsigned int stride = ctx->v_fmt.fmt.pix.bytesperline;
 	u32 val;
 
 	val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->index));
 	cal_set_field(&val, ctx->cport, CAL_WR_DMA_CTRL_CPORT_MASK);
-	cal_set_field(&val, height, CAL_WR_DMA_CTRL_YSIZE_MASK);
+	cal_set_field(&val, ctx->v_fmt.fmt.pix.height,
+		      CAL_WR_DMA_CTRL_YSIZE_MASK);
 	cal_set_field(&val, CAL_WR_DMA_CTRL_DTAG_PIX_DAT,
 		      CAL_WR_DMA_CTRL_DTAG_MASK);
 	cal_set_field(&val, CAL_WR_DMA_CTRL_MODE_CONST,
@@ -375,14 +376,8 @@ void cal_ctx_wr_dma_config(struct cal_ctx *ctx, unsigned int width,
 	ctx_dbg(3, ctx, "CAL_WR_DMA_CTRL(%d) = 0x%08x\n", ctx->index,
 		cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->index)));
 
-	/*
-	 * width/16 not sure but giving it a whirl.
-	 * zero does not work right
-	 */
-	cal_write_field(ctx->cal,
-			CAL_WR_DMA_OFST(ctx->index),
-			(width / 16),
-			CAL_WR_DMA_OFST_MASK);
+	cal_write_field(ctx->cal, CAL_WR_DMA_OFST(ctx->index),
+			stride / 16, CAL_WR_DMA_OFST_MASK);
 	ctx_dbg(3, ctx, "CAL_WR_DMA_OFST(%d) = 0x%08x\n", ctx->index,
 		cal_read(ctx->cal, CAL_WR_DMA_OFST(ctx->index)));
 
@@ -390,11 +385,11 @@ void cal_ctx_wr_dma_config(struct cal_ctx *ctx, unsigned int width,
 	/* 64 bit word means no skipping */
 	cal_set_field(&val, 0, CAL_WR_DMA_XSIZE_XSKIP_MASK);
 	/*
-	 * (width*8)/64 this should be size of an entire line
-	 * in 64bit word but 0 means all data until the end
-	 * is detected automagically
+	 * The XSIZE field is expressed in 64-bit units and prevents overflows
+	 * in case of synchronization issues by limiting the number of bytes
+	 * written per line.
 	 */
-	cal_set_field(&val, (width / 8), CAL_WR_DMA_XSIZE_MASK);
+	cal_set_field(&val, stride / 8, CAL_WR_DMA_XSIZE_MASK);
 	cal_write(ctx->cal, CAL_WR_DMA_XSIZE(ctx->index), val);
 	ctx_dbg(3, ctx, "CAL_WR_DMA_XSIZE(%d) = 0x%08x\n", ctx->index,
 		cal_read(ctx->cal, CAL_WR_DMA_XSIZE(ctx->index)));
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 45f65ebf7aa9..d8e7f0c586f6 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -273,8 +273,7 @@ void cal_camerarx_destroy(struct cal_camerarx *phy);
 
 void cal_ctx_csi2_config(struct cal_ctx *ctx);
 void cal_ctx_pix_proc_config(struct cal_ctx *ctx);
-void cal_ctx_wr_dma_config(struct cal_ctx *ctx, unsigned int width,
-			   unsigned int height);
+void cal_ctx_wr_dma_config(struct cal_ctx *ctx);
 void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, unsigned int dmaaddr);
 
 int cal_ctx_v4l2_register(struct cal_ctx *ctx);
-- 
Regards,

Laurent Pinchart

[PATCH v3 10/24] media: ti-vpe: cal: Rename struct cal_fmt to cal_format_info

[Laurent Pinchart]

The cal_fmt structure stores information about a format. Its name is
ambiguous, as it could be understood as storing a format. Furthermore,
instances of the structure are called 'fmt' through the code, which
leads to confusion with the 'format' variables. Rename the structure to
cal_format_info, and the corresponding 'fmt' variables to 'fmtinfo'.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c |   4 +-
 drivers/media/platform/ti-vpe/cal-video.c    | 104 +++++++++----------
 drivers/media/platform/ti-vpe/cal.c          |  10 +-
 drivers/media/platform/ti-vpe/cal.h          |  14 +--
 4 files changed, 66 insertions(+), 66 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 584dc548504d..fd37ce209461 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -667,7 +667,7 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
 					   struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct cal_camerarx *phy = to_cal_camerarx(sd);
-	const struct cal_fmt *fmtinfo;
+	const struct cal_format_info *fmtinfo;
 
 	if (fse->index > 0)
 		return -EINVAL;
@@ -720,7 +720,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 				   struct v4l2_subdev_format *format)
 {
 	struct cal_camerarx *phy = to_cal_camerarx(sd);
-	const struct cal_fmt *fmtinfo;
+	const struct cal_format_info *fmtinfo;
 	struct v4l2_mbus_framefmt *fmt;
 	unsigned int bpp;
 
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 083389635269..e7ad0b93fc59 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -45,31 +45,31 @@ static char *fourcc_to_str(u32 fmt)
  * ------------------------------------------------------------------
  */
 
-static const struct cal_fmt *find_format_by_pix(struct cal_ctx *ctx,
-						u32 pixelformat)
+static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
+							u32 pixelformat)
 {
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 	unsigned int k;
 
 	for (k = 0; k < ctx->num_active_fmt; k++) {
-		fmt = ctx->active_fmt[k];
-		if (fmt->fourcc == pixelformat)
-			return fmt;
+		fmtinfo = ctx->active_fmt[k];
+		if (fmtinfo->fourcc == pixelformat)
+			return fmtinfo;
 	}
 
 	return NULL;
 }
 
-static const struct cal_fmt *find_format_by_code(struct cal_ctx *ctx,
-						 u32 code)
+static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
+							 u32 code)
 {
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 	unsigned int k;
 
 	for (k = 0; k < ctx->num_active_fmt; k++) {
-		fmt = ctx->active_fmt[k];
-		if (fmt->code == code)
-			return fmt;
+		fmtinfo = ctx->active_fmt[k];
+		if (fmtinfo->code == code)
+			return fmtinfo;
 	}
 
 	return NULL;
@@ -92,14 +92,14 @@ static int cal_enum_fmt_vid_cap(struct file *file, void  *priv,
 				struct v4l2_fmtdesc *f)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 
 	if (f->index >= ctx->num_active_fmt)
 		return -EINVAL;
 
-	fmt = ctx->active_fmt[f->index];
+	fmtinfo = ctx->active_fmt[f->index];
 
-	f->pixelformat = fmt->fourcc;
+	f->pixelformat = fmtinfo->fourcc;
 	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	return 0;
 }
@@ -148,12 +148,12 @@ static int __subdev_set_format(struct cal_ctx *ctx,
 }
 
 static int cal_calc_format_size(struct cal_ctx *ctx,
-				const struct cal_fmt *fmt,
+				const struct cal_format_info *fmtinfo,
 				struct v4l2_format *f)
 {
 	u32 bpl, max_width;
 
-	if (!fmt) {
+	if (!fmtinfo) {
 		ctx_dbg(3, ctx, "No cal_fmt provided!\n");
 		return -EINVAL;
 	}
@@ -163,12 +163,12 @@ static int cal_calc_format_size(struct cal_ctx *ctx,
 	 * We need to recalculate the actual maxi width depending on the
 	 * number of bytes per pixels required.
 	 */
-	max_width = CAL_MAX_WIDTH_BYTES / (ALIGN(fmt->bpp, 8) >> 3);
+	max_width = CAL_MAX_WIDTH_BYTES / (ALIGN(fmtinfo->bpp, 8) >> 3);
 	v4l_bound_align_image(&f->fmt.pix.width, 48, max_width, 2,
 			      &f->fmt.pix.height, 32, CAL_MAX_HEIGHT_LINES,
 			      0, 0);
 
-	bpl = (f->fmt.pix.width * ALIGN(fmt->bpp, 8)) >> 3;
+	bpl = (f->fmt.pix.width * ALIGN(fmtinfo->bpp, 8)) >> 3;
 	f->fmt.pix.bytesperline = ALIGN(bpl, 16);
 
 	f->fmt.pix.sizeimage = f->fmt.pix.height *
@@ -196,18 +196,18 @@ static int cal_try_fmt_vid_cap(struct file *file, void *priv,
 			       struct v4l2_format *f)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 	struct v4l2_subdev_frame_size_enum fse;
 	int ret, found;
 
-	fmt = find_format_by_pix(ctx, f->fmt.pix.pixelformat);
-	if (!fmt) {
+	fmtinfo = find_format_by_pix(ctx, f->fmt.pix.pixelformat);
+	if (!fmtinfo) {
 		ctx_dbg(3, ctx, "Fourcc format (0x%08x) not found.\n",
 			f->fmt.pix.pixelformat);
 
 		/* Just get the first one enumerated */
-		fmt = ctx->active_fmt[0];
-		f->fmt.pix.pixelformat = fmt->fourcc;
+		fmtinfo = ctx->active_fmt[0];
+		f->fmt.pix.pixelformat = fmtinfo->fourcc;
 	}
 
 	f->fmt.pix.field = ctx->v_fmt.fmt.pix.field;
@@ -216,7 +216,7 @@ static int cal_try_fmt_vid_cap(struct file *file, void *priv,
 	ret = 0;
 	found = false;
 	fse.pad = 0;
-	fse.code = fmt->code;
+	fse.code = fmtinfo->code;
 	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	for (fse.index = 0; ; fse.index++) {
 		ret = v4l2_subdev_call(ctx->phy->sensor, pad, enum_frame_size,
@@ -248,7 +248,7 @@ static int cal_try_fmt_vid_cap(struct file *file, void *priv,
 	 * updated properly during s_fmt
 	 */
 	f->fmt.pix.colorspace = ctx->v_fmt.fmt.pix.colorspace;
-	return cal_calc_format_size(ctx, fmt, f);
+	return cal_calc_format_size(ctx, fmtinfo, f);
 }
 
 static int cal_s_fmt_vid_cap(struct file *file, void *priv,
@@ -260,7 +260,7 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
 		.pad = CAL_CAMERARX_PAD_SINK,
 	};
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 	int ret;
 
 	if (vb2_is_busy(q)) {
@@ -272,16 +272,16 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 	if (ret < 0)
 		return ret;
 
-	fmt = find_format_by_pix(ctx, f->fmt.pix.pixelformat);
+	fmtinfo = find_format_by_pix(ctx, f->fmt.pix.pixelformat);
 
-	v4l2_fill_mbus_format(&sd_fmt.format, &f->fmt.pix, fmt->code);
+	v4l2_fill_mbus_format(&sd_fmt.format, &f->fmt.pix, fmtinfo->code);
 
 	ret = __subdev_set_format(ctx, &sd_fmt.format);
 	if (ret)
 		return ret;
 
 	/* Just double check nothing has gone wrong */
-	if (sd_fmt.format.code != fmt->code) {
+	if (sd_fmt.format.code != fmtinfo->code) {
 		ctx_dbg(3, ctx,
 			"%s subdev changed format on us, this should not happen\n",
 			__func__);
@@ -290,13 +290,13 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 
 	v4l2_fill_pix_format(&ctx->v_fmt.fmt.pix, &sd_fmt.format);
 	ctx->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-	ctx->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
+	ctx->v_fmt.fmt.pix.pixelformat = fmtinfo->fourcc;
 	ctx->v_fmt.fmt.pix.field = sd_fmt.format.field;
-	cal_calc_format_size(ctx, fmt, &ctx->v_fmt);
+	cal_calc_format_size(ctx, fmtinfo, &ctx->v_fmt);
 
 	v4l2_subdev_call(&ctx->phy->subdev, pad, set_fmt, NULL, &sd_fmt);
 
-	ctx->fmt = fmt;
+	ctx->fmtinfo = fmtinfo;
 	*f = ctx->v_fmt;
 
 	return 0;
@@ -306,13 +306,13 @@ static int cal_enum_framesizes(struct file *file, void *fh,
 			       struct v4l2_frmsizeenum *fsize)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 	struct v4l2_subdev_frame_size_enum fse;
 	int ret;
 
 	/* check for valid format */
-	fmt = find_format_by_pix(ctx, fsize->pixel_format);
-	if (!fmt) {
+	fmtinfo = find_format_by_pix(ctx, fsize->pixel_format);
+	if (!fmtinfo) {
 		ctx_dbg(3, ctx, "Invalid pixel code: %x\n",
 			fsize->pixel_format);
 		return -EINVAL;
@@ -320,7 +320,7 @@ static int cal_enum_framesizes(struct file *file, void *fh,
 
 	fse.index = fsize->index;
 	fse.pad = 0;
-	fse.code = fmt->code;
+	fse.code = fmtinfo->code;
 	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 
 	ret = v4l2_subdev_call(ctx->phy->sensor, pad, enum_frame_size, NULL,
@@ -366,7 +366,7 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
 				   struct v4l2_frmivalenum *fival)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 	struct v4l2_subdev_frame_interval_enum fie = {
 		.index = fival->index,
 		.width = fival->width,
@@ -375,11 +375,11 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
 	};
 	int ret;
 
-	fmt = find_format_by_pix(ctx, fival->pixel_format);
-	if (!fmt)
+	fmtinfo = find_format_by_pix(ctx, fival->pixel_format);
+	if (!fmtinfo)
 		return -EINVAL;
 
-	fie.code = fmt->code;
+	fie.code = fmtinfo->code;
 	ret = v4l2_subdev_call(ctx->phy->sensor, pad, enum_frame_interval,
 			       NULL, &fie);
 	if (ret)
@@ -623,7 +623,7 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 {
 	struct v4l2_subdev_mbus_code_enum mbus_code;
 	struct v4l2_mbus_framefmt mbus_fmt;
-	const struct cal_fmt *fmt;
+	const struct cal_format_info *fmtinfo;
 	unsigned int i, j, k;
 	int ret = 0;
 
@@ -647,14 +647,14 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 			ctx->phy->sensor->name, mbus_code.code, j);
 
 		for (k = 0; k < cal_num_formats; k++) {
-			const struct cal_fmt *fmt = &cal_formats[k];
+			fmtinfo = &cal_formats[k];
 
-			if (mbus_code.code == fmt->code) {
-				ctx->active_fmt[i] = fmt;
+			if (mbus_code.code == fmtinfo->code) {
+				ctx->active_fmt[i] = fmtinfo;
 				ctx_dbg(2, ctx,
 					"matched fourcc: %s: code: %04x idx: %u\n",
-					fourcc_to_str(fmt->fourcc),
-					fmt->code, i);
+					fourcc_to_str(fmtinfo->fourcc),
+					fmtinfo->code, i);
 				ctx->num_active_fmt = ++i;
 			}
 		}
@@ -670,8 +670,8 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 	if (ret)
 		return ret;
 
-	fmt = find_format_by_code(ctx, mbus_fmt.code);
-	if (!fmt) {
+	fmtinfo = find_format_by_code(ctx, mbus_fmt.code);
+	if (!fmtinfo) {
 		ctx_dbg(3, ctx, "mbus code format (0x%08x) not found.\n",
 			mbus_fmt.code);
 		return -EINVAL;
@@ -680,10 +680,10 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 	/* Save current format */
 	v4l2_fill_pix_format(&ctx->v_fmt.fmt.pix, &mbus_fmt);
 	ctx->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-	ctx->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
+	ctx->v_fmt.fmt.pix.pixelformat = fmtinfo->fourcc;
 	ctx->v_fmt.fmt.pix.field = mbus_fmt.field;
-	cal_calc_format_size(ctx, fmt, &ctx->v_fmt);
-	ctx->fmt = fmt;
+	cal_calc_format_size(ctx, fmtinfo, &ctx->v_fmt);
+	ctx->fmtinfo = fmtinfo;
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 4c938ca1f1a9..213381b8ddfa 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -48,7 +48,7 @@ MODULE_PARM_DESC(debug, "activates debug info");
  * ------------------------------------------------------------------
  */
 
-const struct cal_fmt cal_formats[] = {
+const struct cal_format_info cal_formats[] = {
 	{
 		.fourcc		= V4L2_PIX_FMT_YUYV,
 		.code		= MEDIA_BUS_FMT_YUYV8_2X8,
@@ -146,7 +146,7 @@ const struct cal_fmt cal_formats[] = {
 
 const unsigned int cal_num_formats = ARRAY_SIZE(cal_formats);
 
-const struct cal_fmt *cal_format_by_fourcc(u32 fourcc)
+const struct cal_format_info *cal_format_by_fourcc(u32 fourcc)
 {
 	unsigned int i;
 
@@ -158,7 +158,7 @@ const struct cal_fmt *cal_format_by_fourcc(u32 fourcc)
 	return NULL;
 }
 
-const struct cal_fmt *cal_format_by_code(u32 code)
+const struct cal_format_info *cal_format_by_code(u32 code)
 {
 	unsigned int i;
 
@@ -309,7 +309,7 @@ void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
 {
 	u32 val, extract, pack;
 
-	switch (ctx->fmt->bpp) {
+	switch (ctx->fmtinfo->bpp) {
 	case 8:
 		extract = CAL_PIX_PROC_EXTRACT_B8;
 		pack = CAL_PIX_PROC_PACK_B8;
@@ -338,7 +338,7 @@ void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
 		 */
 		dev_warn_once(ctx->cal->dev,
 			      "%s:%d:%s: bpp:%d unsupported! Overwritten with 8.\n",
-			      __FILE__, __LINE__, __func__, ctx->fmt->bpp);
+			      __FILE__, __LINE__, __func__, ctx->fmtinfo->bpp);
 		extract = CAL_PIX_PROC_EXTRACT_B8;
 		pack = CAL_PIX_PROC_PACK_B8;
 		break;
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index d8e7f0c586f6..be8224942120 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -60,7 +60,7 @@ enum cal_camerarx_field {
 	F_MAX_FIELDS,
 };
 
-struct cal_fmt {
+struct cal_format_info {
 	u32	fourcc;
 	u32	code;
 	/* Bits per pixel */
@@ -126,7 +126,7 @@ struct cal_camerarx {
 	struct v4l2_subdev	subdev;
 	struct media_pad	pads[2];
 	struct v4l2_mbus_framefmt	formats[2];
-	const struct cal_fmt	*fmtinfo;
+	const struct cal_format_info	*fmtinfo;
 };
 
 struct cal_dev {
@@ -172,12 +172,12 @@ struct cal_ctx {
 	struct cal_dmaqueue	vidq;
 
 	/* video capture */
-	const struct cal_fmt	*fmt;
+	const struct cal_format_info	*fmtinfo;
 	/* Used to store current pixel format */
 	struct v4l2_format	v_fmt;
 
 	/* Current subdev enumerated format */
-	const struct cal_fmt	**active_fmt;
+	const struct cal_format_info	**active_fmt;
 	unsigned int		num_active_fmt;
 
 	unsigned int		sequence;
@@ -254,10 +254,10 @@ static inline void cal_set_field(u32 *valp, u32 field, u32 mask)
 	*valp = val;
 }
 
-extern const struct cal_fmt cal_formats[];
+extern const struct cal_format_info cal_formats[];
 extern const unsigned int cal_num_formats;
-const struct cal_fmt *cal_format_by_fourcc(u32 fourcc);
-const struct cal_fmt *cal_format_by_code(u32 code);
+const struct cal_format_info *cal_format_by_fourcc(u32 fourcc);
+const struct cal_format_info *cal_format_by_code(u32 code);
 
 void cal_quickdump_regs(struct cal_dev *cal);
 
-- 
Regards,

Laurent Pinchart

[PATCH v3 11/24] media: ti-vpe: cal: Refactor interrupt enable/disable

[Laurent Pinchart]

Interrupts are enabled and disabled by the cal_camerarx_enable_irqs()
and cal_camerarx_disable_irqs(). Despite their name, they deal with all
interrupts, not just the CAMERARX interrupts, and they hardcode the
assumption that the context index is identical to the CAMERARX index.

Split the context-related interrupt management to two new functions,
cal_ctx_enable_irqs() and cal_ctx_disable_irqs(), called from the
cal_start_streaming() and cal_stop_streaming() functions. The explicit
calls to cal_camerarx_enable_irqs() and cal_camerarx_disable_irqs() are
folded with the CAMERARX .s_stream() operation to simplify the CAMERARX
API.

Enabling the OCPO error interrupt is moved to the PM runtime resume
operation, as it's global to the device, not related to a CAMERARX or
context. The VC IRQ enable and disable are removed as they're not used,
the parent interrupt bit (CAL_HL_IRQ_VC_MASK) never being set.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 83 +++++++-------------
 drivers/media/platform/ti-vpe/cal-video.c    |  6 +-
 drivers/media/platform/ti-vpe/cal.c          | 24 ++++++
 drivers/media/platform/ti-vpe/cal.h          |  4 +-
 4 files changed, 57 insertions(+), 60 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index fd37ce209461..ce46046c9ba4 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -233,6 +233,29 @@ static void cal_camerarx_wait_stop_state(struct cal_camerarx *phy)
 		phy_err(phy, "Timeout waiting for stop state\n");
 }
 
+static void cal_camerarx_enable_irqs(struct cal_camerarx *phy)
+{
+	const u32 cio_err_mask =
+		CAL_CSI2_COMPLEXIO_IRQ_LANE_ERRORS_MASK |
+		CAL_CSI2_COMPLEXIO_IRQ_FIFO_OVR_MASK |
+		CAL_CSI2_COMPLEXIO_IRQ_SHORT_PACKET_MASK |
+		CAL_CSI2_COMPLEXIO_IRQ_ECC_NO_CORRECTION_MASK;
+
+	/* Enable CIO error IRQs. */
+	cal_write(phy->cal, CAL_HL_IRQENABLE_SET(0),
+		  CAL_HL_IRQ_CIO_MASK(phy->instance));
+	cal_write(phy->cal, CAL_CSI2_COMPLEXIO_IRQENABLE(phy->instance),
+		  cio_err_mask);
+}
+
+static void cal_camerarx_disable_irqs(struct cal_camerarx *phy)
+{
+	/* Disable CIO error irqs */
+	cal_write(phy->cal, CAL_HL_IRQENABLE_CLR(0),
+		  CAL_HL_IRQ_CIO_MASK(phy->instance));
+	cal_write(phy->cal, CAL_CSI2_COMPLEXIO_IRQENABLE(phy->instance), 0);
+}
+
 static int cal_camerarx_start(struct cal_camerarx *phy)
 {
 	s64 external_rate;
@@ -250,6 +273,8 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 		return ret;
 	}
 
+	cal_camerarx_enable_irqs(phy);
+
 	/*
 	 * CSI-2 PHY Link Initialization Sequence, according to the DRA74xP /
 	 * DRA75xP / DRA76xP / DRA77xP TRM. The DRA71x / DRA72x and the AM65x /
@@ -339,6 +364,7 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	ret = v4l2_subdev_call(phy->sensor, video, s_stream, 1);
 	if (ret) {
 		v4l2_subdev_call(phy->sensor, core, s_power, 0);
+		cal_camerarx_disable_irqs(phy);
 		phy_err(phy, "stream on failed in subdev\n");
 		return ret;
 	}
@@ -366,6 +392,8 @@ static void cal_camerarx_stop(struct cal_camerarx *phy)
 	unsigned int i;
 	int ret;
 
+	cal_camerarx_disable_irqs(phy);
+
 	cal_camerarx_power(phy, false);
 
 	/* Assert Complex IO Reset */
@@ -427,61 +455,6 @@ void cal_camerarx_i913_errata(struct cal_camerarx *phy)
 	camerarx_write(phy, CAL_CSI2_PHY_REG10, reg10);
 }
 
-/*
- * Enable the expected IRQ sources
- */
-void cal_camerarx_enable_irqs(struct cal_camerarx *phy)
-{
-	u32 val;
-
-	const u32 cio_err_mask =
-		CAL_CSI2_COMPLEXIO_IRQ_LANE_ERRORS_MASK |
-		CAL_CSI2_COMPLEXIO_IRQ_FIFO_OVR_MASK |
-		CAL_CSI2_COMPLEXIO_IRQ_SHORT_PACKET_MASK |
-		CAL_CSI2_COMPLEXIO_IRQ_ECC_NO_CORRECTION_MASK;
-
-	/* Enable CIO error irqs */
-	cal_write(phy->cal, CAL_HL_IRQENABLE_SET(0),
-		  CAL_HL_IRQ_CIO_MASK(phy->instance));
-	cal_write(phy->cal, CAL_CSI2_COMPLEXIO_IRQENABLE(phy->instance),
-		  cio_err_mask);
-
-	/* Always enable OCPO error */
-	cal_write(phy->cal, CAL_HL_IRQENABLE_SET(0), CAL_HL_IRQ_OCPO_ERR_MASK);
-
-	/* Enable IRQ_WDMA_END 0/1 */
-	val = 0;
-	cal_set_field(&val, 1, CAL_HL_IRQ_MASK(phy->instance));
-	cal_write(phy->cal, CAL_HL_IRQENABLE_SET(1), val);
-	/* Enable IRQ_WDMA_START 0/1 */
-	val = 0;
-	cal_set_field(&val, 1, CAL_HL_IRQ_MASK(phy->instance));
-	cal_write(phy->cal, CAL_HL_IRQENABLE_SET(2), val);
-	/* Todo: Add VC_IRQ and CSI2_COMPLEXIO_IRQ handling */
-	cal_write(phy->cal, CAL_CSI2_VC_IRQENABLE(0), 0xFF000000);
-}
-
-void cal_camerarx_disable_irqs(struct cal_camerarx *phy)
-{
-	u32 val;
-
-	/* Disable CIO error irqs */
-	cal_write(phy->cal, CAL_HL_IRQENABLE_CLR(0),
-		  CAL_HL_IRQ_CIO_MASK(phy->instance));
-	cal_write(phy->cal, CAL_CSI2_COMPLEXIO_IRQENABLE(phy->instance), 0);
-
-	/* Disable IRQ_WDMA_END 0/1 */
-	val = 0;
-	cal_set_field(&val, 1, CAL_HL_IRQ_MASK(phy->instance));
-	cal_write(phy->cal, CAL_HL_IRQENABLE_CLR(1), val);
-	/* Disable IRQ_WDMA_START 0/1 */
-	val = 0;
-	cal_set_field(&val, 1, CAL_HL_IRQ_MASK(phy->instance));
-	cal_write(phy->cal, CAL_HL_IRQENABLE_CLR(2), val);
-	/* Todo: Add VC_IRQ and CSI2_COMPLEXIO_IRQ handling */
-	cal_write(phy->cal, CAL_CSI2_VC_IRQENABLE(0), 0);
-}
-
 void cal_camerarx_ppi_enable(struct cal_camerarx *phy)
 {
 	cal_write(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index e7ad0b93fc59..3807d91f0392 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -517,8 +517,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	cal_ctx_csi2_config(ctx);
 	cal_ctx_pix_proc_config(ctx);
 	cal_ctx_wr_dma_config(ctx);
-
-	cal_camerarx_enable_irqs(ctx->phy);
+	cal_ctx_enable_irqs(ctx);
 
 	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
 	if (ret)
@@ -570,7 +569,8 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 	if (dma_act)
 		ctx_err(ctx, "failed to disable dma cleanly\n");
 
-	cal_camerarx_disable_irqs(ctx->phy);
+	cal_ctx_disable_irqs(ctx);
+
 	v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 0);
 
 	/* Release all active buffers */
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 213381b8ddfa..785ce4171d40 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -411,6 +411,24 @@ void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, unsigned int dmaaddr)
 	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->index), dmaaddr);
 }
 
+void cal_ctx_enable_irqs(struct cal_ctx *ctx)
+{
+	/* Enable IRQ_WDMA_END and IRQ_WDMA_START. */
+	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(1),
+		  CAL_HL_IRQ_MASK(ctx->index));
+	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(2),
+		  CAL_HL_IRQ_MASK(ctx->index));
+}
+
+void cal_ctx_disable_irqs(struct cal_ctx *ctx)
+{
+	/* Disable IRQ_WDMA_END and IRQ_WDMA_START. */
+	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(1),
+		  CAL_HL_IRQ_MASK(ctx->index));
+	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(2),
+		  CAL_HL_IRQ_MASK(ctx->index));
+}
+
 /* ------------------------------------------------------------------
  *	IRQ Handling
  * ------------------------------------------------------------------
@@ -1041,6 +1059,12 @@ static int cal_runtime_resume(struct device *dev)
 			cal_camerarx_i913_errata(cal->phy[i]);
 	}
 
+	/*
+	 * Enable global interrupts that are not related to a particular
+	 * CAMERARAX or context.
+	 */
+	cal_write(cal, CAL_HL_IRQENABLE_SET(0), CAL_HL_IRQ_OCPO_ERR_MASK);
+
 	return 0;
 }
 
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index be8224942120..1c6fbee211d2 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -262,8 +262,6 @@ const struct cal_format_info *cal_format_by_code(u32 code);
 void cal_quickdump_regs(struct cal_dev *cal);
 
 void cal_camerarx_disable(struct cal_camerarx *phy);
-void cal_camerarx_enable_irqs(struct cal_camerarx *phy);
-void cal_camerarx_disable_irqs(struct cal_camerarx *phy);
 void cal_camerarx_ppi_enable(struct cal_camerarx *phy);
 void cal_camerarx_ppi_disable(struct cal_camerarx *phy);
 void cal_camerarx_i913_errata(struct cal_camerarx *phy);
@@ -275,6 +273,8 @@ void cal_ctx_csi2_config(struct cal_ctx *ctx);
 void cal_ctx_pix_proc_config(struct cal_ctx *ctx);
 void cal_ctx_wr_dma_config(struct cal_ctx *ctx);
 void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, unsigned int dmaaddr);
+void cal_ctx_enable_irqs(struct cal_ctx *ctx);
+void cal_ctx_disable_irqs(struct cal_ctx *ctx);
 
 int cal_ctx_v4l2_register(struct cal_ctx *ctx);
 void cal_ctx_v4l2_unregister(struct cal_ctx *ctx);
-- 
Regards,

Laurent Pinchart

[PATCH v3 12/24] media: ti-vpe: cal: Fold PPI enable in CAMERARX .s_stream()

[Laurent Pinchart]

To further decouple the context and CAMERARX components, move the call
to cal_camerarx_ppi_enable() from cal_start_streaming() to the CAMERARX
.s_stream() operation. The DMA destination address has to be set before
starting the CAMERARX, which is desirable anyway.

cal_camerarx_ppi_disable() will be addressed separately.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 31 +++++++++++---------
 drivers/media/platform/ti-vpe/cal-video.c    |  4 +--
 drivers/media/platform/ti-vpe/cal.h          |  1 -
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index ce46046c9ba4..941efa99e3b5 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -256,6 +256,20 @@ static void cal_camerarx_disable_irqs(struct cal_camerarx *phy)
 	cal_write(phy->cal, CAL_CSI2_COMPLEXIO_IRQENABLE(phy->instance), 0);
 }
 
+static void cal_camerarx_ppi_enable(struct cal_camerarx *phy)
+{
+	cal_write(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
+		  CAL_CSI2_PPI_CTRL_FRAME_MASK);
+	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
+			1, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
+}
+
+void cal_camerarx_ppi_disable(struct cal_camerarx *phy)
+{
+	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
+			0, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
+}
+
 static int cal_camerarx_start(struct cal_camerarx *phy)
 {
 	s64 external_rate;
@@ -384,6 +398,9 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	 * implemented.
 	 */
 
+	/* Finally, enable the PHY Protocol Interface (PPI). */
+	cal_camerarx_ppi_enable(phy);
+
 	return 0;
 }
 
@@ -455,20 +472,6 @@ void cal_camerarx_i913_errata(struct cal_camerarx *phy)
 	camerarx_write(phy, CAL_CSI2_PHY_REG10, reg10);
 }
 
-void cal_camerarx_ppi_enable(struct cal_camerarx *phy)
-{
-	cal_write(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
-		  CAL_CSI2_PPI_CTRL_FRAME_MASK);
-	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
-			1, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
-}
-
-void cal_camerarx_ppi_disable(struct cal_camerarx *phy)
-{
-	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
-			0, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
-}
-
 static int cal_camerarx_regmap_init(struct cal_dev *cal,
 				    struct cal_camerarx *phy)
 {
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 3807d91f0392..627d816548b8 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -517,15 +517,13 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	cal_ctx_csi2_config(ctx);
 	cal_ctx_pix_proc_config(ctx);
 	cal_ctx_wr_dma_config(ctx);
+	cal_ctx_wr_dma_addr(ctx, addr);
 	cal_ctx_enable_irqs(ctx);
 
 	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
 	if (ret)
 		goto err;
 
-	cal_ctx_wr_dma_addr(ctx, addr);
-	cal_camerarx_ppi_enable(ctx->phy);
-
 	if (cal_debug >= 4)
 		cal_quickdump_regs(ctx->cal);
 
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 1c6fbee211d2..501700fc4955 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -262,7 +262,6 @@ const struct cal_format_info *cal_format_by_code(u32 code);
 void cal_quickdump_regs(struct cal_dev *cal);
 
 void cal_camerarx_disable(struct cal_camerarx *phy);
-void cal_camerarx_ppi_enable(struct cal_camerarx *phy);
 void cal_camerarx_ppi_disable(struct cal_camerarx *phy);
 void cal_camerarx_i913_errata(struct cal_camerarx *phy);
 struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
-- 
Regards,

Laurent Pinchart

[PATCH v3 13/24] media: ti-vpe: cal: Stop write DMA without disabling PPI

[Laurent Pinchart]

When stopping the stream, the driver needs to ensure that ongoing DMA
completes and that no new DMA is started. It does so using a feature of
the PPI that can be stopped on a frame boundary. The downside of this
mechanism is that the DMA can't be stopped independently of the source,
which prevents usage of multiple contexts for the same source (to handle
CSI-2 virtual channels or data types).

Rework the stream stop mechanism to stop the write DMA without disabling
the PPI first. The new mechanism relies on the combination of a state
machine in the driver and shadowing of the CAL_WR_DMA_CTRL_j.MODE field
in the hardware. When a stop is requested, the DMA start interrupt
handler will request the hardware to stop at the end of the current
frame by disabling the write DMA context in the shadowed register, and
flag that a stop is in progress. The next DMA end interrupt will then
report that the stop is complete.

This makes it possible to stop the PPI after stopping the DMA, and fold
the cal_camerarx_ppi_disable() call into cal_camerarx_stop().

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c |   6 +-
 drivers/media/platform/ti-vpe/cal-video.c    |  26 +---
 drivers/media/platform/ti-vpe/cal.c          | 133 +++++++++++++------
 drivers/media/platform/ti-vpe/cal.h          |  14 +-
 4 files changed, 117 insertions(+), 62 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 941efa99e3b5..1920f36137b8 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -258,13 +258,11 @@ static void cal_camerarx_disable_irqs(struct cal_camerarx *phy)
 
 static void cal_camerarx_ppi_enable(struct cal_camerarx *phy)
 {
-	cal_write(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
-		  CAL_CSI2_PPI_CTRL_FRAME_MASK);
 	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
 			1, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
 }
 
-void cal_camerarx_ppi_disable(struct cal_camerarx *phy)
+static void cal_camerarx_ppi_disable(struct cal_camerarx *phy)
 {
 	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
 			0, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
@@ -409,6 +407,8 @@ static void cal_camerarx_stop(struct cal_camerarx *phy)
 	unsigned int i;
 	int ret;
 
+	cal_camerarx_ppi_disable(phy);
+
 	cal_camerarx_disable_irqs(phy);
 
 	cal_camerarx_power(phy, false);
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 627d816548b8..d3f805a512c0 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -9,7 +9,6 @@
  *	Laurent Pinchart <laurent.pinchart@ideasonboard.com>
  */
 
-#include <linux/delay.h>
 #include <linux/ioctl.h>
 #include <linux/pm_runtime.h>
 #include <linux/videodev2.h>
@@ -511,6 +510,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	addr = vb2_dma_contig_plane_dma_addr(&ctx->cur_frm->vb.vb2_buf, 0);
 	ctx->sequence = 0;
+	ctx->dma_state = CAL_DMA_RUNNING;
 
 	pm_runtime_get_sync(ctx->cal->dev);
 
@@ -530,6 +530,10 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 err:
+	cal_ctx_wr_dma_disable(ctx);
+	cal_ctx_disable_irqs(ctx);
+	ctx->dma_state = CAL_DMA_STOPPED;
+
 	spin_lock_irqsave(&ctx->slock, flags);
 	vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, VB2_BUF_STATE_QUEUED);
 	ctx->cur_frm = NULL;
@@ -547,26 +551,9 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
 	struct cal_dmaqueue *dma_q = &ctx->vidq;
 	struct cal_buffer *buf, *tmp;
-	unsigned long timeout;
 	unsigned long flags;
-	bool dma_act;
-
-	cal_camerarx_ppi_disable(ctx->phy);
-
-	/* wait for stream and dma to finish */
-	dma_act = true;
-	timeout = jiffies + msecs_to_jiffies(500);
-	while (dma_act && time_before(jiffies, timeout)) {
-		msleep(50);
-
-		spin_lock_irqsave(&ctx->slock, flags);
-		dma_act = ctx->dma_act;
-		spin_unlock_irqrestore(&ctx->slock, flags);
-	}
-
-	if (dma_act)
-		ctx_err(ctx, "failed to disable dma cleanly\n");
 
+	cal_ctx_wr_dma_stop(ctx);
 	cal_ctx_disable_irqs(ctx);
 
 	v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 0);
@@ -745,6 +732,7 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 	INIT_LIST_HEAD(&ctx->vidq.active);
 	spin_lock_init(&ctx->slock);
 	mutex_init(&ctx->mutex);
+	init_waitqueue_head(&ctx->dma_wait);
 
 	/* Initialize the vb2 queue. */
 	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 785ce4171d40..f1c2b8bc28f4 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -411,6 +411,45 @@ void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, unsigned int dmaaddr)
 	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->index), dmaaddr);
 }
 
+void cal_ctx_wr_dma_disable(struct cal_ctx *ctx)
+{
+	u32 val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->index));
+
+	cal_set_field(&val, CAL_WR_DMA_CTRL_MODE_DIS,
+		      CAL_WR_DMA_CTRL_MODE_MASK);
+	cal_write(ctx->cal, CAL_WR_DMA_CTRL(ctx->index), val);
+}
+
+static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
+{
+	bool stopped;
+
+	spin_lock_irq(&ctx->slock);
+	stopped = ctx->dma_state == CAL_DMA_STOPPED;
+	spin_unlock_irq(&ctx->slock);
+
+	return stopped;
+}
+
+int cal_ctx_wr_dma_stop(struct cal_ctx *ctx)
+{
+	long timeout;
+
+	/* Request DMA stop and wait until it completes. */
+	spin_lock_irq(&ctx->slock);
+	ctx->dma_state = CAL_DMA_STOP_REQUESTED;
+	spin_unlock_irq(&ctx->slock);
+
+	timeout = wait_event_timeout(ctx->dma_wait, cal_ctx_wr_dma_stopped(ctx),
+				     msecs_to_jiffies(500));
+	if (!timeout) {
+		ctx_err(ctx, "failed to disable dma cleanly\n");
+		return -ETIMEDOUT;
+	}
+
+	return 0;
+}
+
 void cal_ctx_enable_irqs(struct cal_ctx *ctx)
 {
 	/* Enable IRQ_WDMA_END and IRQ_WDMA_START. */
@@ -434,35 +473,71 @@ void cal_ctx_disable_irqs(struct cal_ctx *ctx)
  * ------------------------------------------------------------------
  */
 
-static inline void cal_schedule_next_buffer(struct cal_ctx *ctx)
+static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 {
 	struct cal_dmaqueue *dma_q = &ctx->vidq;
-	struct cal_buffer *buf;
-	unsigned long addr;
 
-	buf = list_entry(dma_q->active.next, struct cal_buffer, list);
-	ctx->next_frm = buf;
-	list_del(&buf->list);
+	spin_lock(&ctx->slock);
 
-	addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
-	cal_ctx_wr_dma_addr(ctx, addr);
+	if (ctx->dma_state == CAL_DMA_STOP_REQUESTED) {
+		/*
+		 * If a stop is requested, disable the write DMA context
+		 * immediately. The CAL_WR_DMA_CTRL_j.MODE field is shadowed,
+		 * the current frame will complete and the DMA will then stop.
+		 */
+		cal_ctx_wr_dma_disable(ctx);
+		ctx->dma_state = CAL_DMA_STOP_PENDING;
+	} else if (!list_empty(&dma_q->active) &&
+		   ctx->cur_frm == ctx->next_frm) {
+		/*
+		 * Otherwise, if a new buffer is available, queue it to the
+		 * hardware.
+		 */
+		struct cal_buffer *buf;
+		unsigned long addr;
+
+		buf = list_entry(dma_q->active.next, struct cal_buffer, list);
+		addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
+		cal_ctx_wr_dma_addr(ctx, addr);
+
+		ctx->next_frm = buf;
+		list_del(&buf->list);
+	}
+
+	spin_unlock(&ctx->slock);
 }
 
-static inline void cal_process_buffer_complete(struct cal_ctx *ctx)
+static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
 {
-	ctx->cur_frm->vb.vb2_buf.timestamp = ktime_get_ns();
-	ctx->cur_frm->vb.field = ctx->v_fmt.fmt.pix.field;
-	ctx->cur_frm->vb.sequence = ctx->sequence++;
+	struct cal_buffer *buf = NULL;
 
-	vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
-	ctx->cur_frm = ctx->next_frm;
+	spin_lock(&ctx->slock);
+
+	/* If the DMA context was stopping, it is now stopped. */
+	if (ctx->dma_state == CAL_DMA_STOP_PENDING) {
+		ctx->dma_state = CAL_DMA_STOPPED;
+		wake_up(&ctx->dma_wait);
+	}
+
+	/* If a new buffer was queued, complete the current buffer. */
+	if (ctx->cur_frm != ctx->next_frm) {
+		buf = ctx->cur_frm;
+		ctx->cur_frm = ctx->next_frm;
+	}
+
+	spin_unlock(&ctx->slock);
+
+	if (buf) {
+		buf->vb.vb2_buf.timestamp = ktime_get_ns();
+		buf->vb.field = ctx->v_fmt.fmt.pix.field;
+		buf->vb.sequence = ctx->sequence++;
+		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+	}
 }
 
 static irqreturn_t cal_irq(int irq_cal, void *data)
 {
 	struct cal_dev *cal = data;
-	struct cal_ctx *ctx;
-	struct cal_dmaqueue *dma_q;
 	u32 status;
 
 	status = cal_read(cal, CAL_HL_IRQSTATUS(0));
@@ -497,17 +572,8 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 		cal_write(cal, CAL_HL_IRQSTATUS(1), status);
 
 		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
-			if (status & CAL_HL_IRQ_MASK(i)) {
-				ctx = cal->ctx[i];
-
-				spin_lock(&ctx->slock);
-				ctx->dma_act = false;
-
-				if (ctx->cur_frm != ctx->next_frm)
-					cal_process_buffer_complete(ctx);
-
-				spin_unlock(&ctx->slock);
-			}
+			if (status & CAL_HL_IRQ_MASK(i))
+				cal_irq_wdma_end(cal->ctx[i]);
 		}
 	}
 
@@ -520,17 +586,8 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 		cal_write(cal, CAL_HL_IRQSTATUS(2), status);
 
 		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
-			if (status & CAL_HL_IRQ_MASK(i)) {
-				ctx = cal->ctx[i];
-				dma_q = &ctx->vidq;
-
-				spin_lock(&ctx->slock);
-				ctx->dma_act = true;
-				if (!list_empty(&dma_q->active) &&
-				    ctx->cur_frm == ctx->next_frm)
-					cal_schedule_next_buffer(ctx);
-				spin_unlock(&ctx->slock);
-			}
+			if (status & CAL_HL_IRQ_MASK(i))
+				cal_irq_wdma_start(cal->ctx[i]);
 		}
 	}
 
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 501700fc4955..5ee0e05c3305 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -17,6 +17,7 @@
 #include <linux/mutex.h>
 #include <linux/spinlock.h>
 #include <linux/videodev2.h>
+#include <linux/wait.h>
 
 #include <media/media-device.h>
 #include <media/v4l2-async.h>
@@ -60,6 +61,13 @@ enum cal_camerarx_field {
 	F_MAX_FIELDS,
 };
 
+enum cal_dma_state {
+	CAL_DMA_RUNNING,
+	CAL_DMA_STOP_REQUESTED,
+	CAL_DMA_STOP_PENDING,
+	CAL_DMA_STOPPED,
+};
+
 struct cal_format_info {
 	u32	fourcc;
 	u32	code;
@@ -190,7 +198,8 @@ struct cal_ctx {
 	/* Pointer pointing to next v4l2_buffer */
 	struct cal_buffer	*next_frm;
 
-	bool dma_act;
+	enum cal_dma_state	dma_state;
+	struct wait_queue_head	dma_wait;
 };
 
 extern unsigned int cal_debug;
@@ -262,7 +271,6 @@ const struct cal_format_info *cal_format_by_code(u32 code);
 void cal_quickdump_regs(struct cal_dev *cal);
 
 void cal_camerarx_disable(struct cal_camerarx *phy);
-void cal_camerarx_ppi_disable(struct cal_camerarx *phy);
 void cal_camerarx_i913_errata(struct cal_camerarx *phy);
 struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 					 unsigned int instance);
@@ -272,6 +280,8 @@ void cal_ctx_csi2_config(struct cal_ctx *ctx);
 void cal_ctx_pix_proc_config(struct cal_ctx *ctx);
 void cal_ctx_wr_dma_config(struct cal_ctx *ctx);
 void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, unsigned int dmaaddr);
+void cal_ctx_wr_dma_disable(struct cal_ctx *ctx);
+int cal_ctx_wr_dma_stop(struct cal_ctx *ctx);
 void cal_ctx_enable_irqs(struct cal_ctx *ctx);
 void cal_ctx_disable_irqs(struct cal_ctx *ctx);
 
-- 
Regards,

Laurent Pinchart

[PATCH v3 14/24] media: ti-vpe: cal: Use spin_lock_irq() when starting or stopping stream

[Laurent Pinchart]

The cal_start_streaming() and cal_stop_streaming() functions are called
with interrupts enabled. spin_lock_irq() can thus be used instead of
spin_lock_irqsave().

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index d3f805a512c0..de0ba6128715 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -492,12 +492,11 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	struct cal_dmaqueue *dma_q = &ctx->vidq;
 	struct cal_buffer *buf, *tmp;
 	unsigned long addr;
-	unsigned long flags;
 	int ret;
 
-	spin_lock_irqsave(&ctx->slock, flags);
+	spin_lock_irq(&ctx->slock);
 	if (list_empty(&dma_q->active)) {
-		spin_unlock_irqrestore(&ctx->slock, flags);
+		spin_unlock_irq(&ctx->slock);
 		ctx_dbg(3, ctx, "buffer queue is empty\n");
 		return -EIO;
 	}
@@ -506,7 +505,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	ctx->cur_frm = buf;
 	ctx->next_frm = buf;
 	list_del(&buf->list);
-	spin_unlock_irqrestore(&ctx->slock, flags);
+	spin_unlock_irq(&ctx->slock);
 
 	addr = vb2_dma_contig_plane_dma_addr(&ctx->cur_frm->vb.vb2_buf, 0);
 	ctx->sequence = 0;
@@ -534,7 +533,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	cal_ctx_disable_irqs(ctx);
 	ctx->dma_state = CAL_DMA_STOPPED;
 
-	spin_lock_irqsave(&ctx->slock, flags);
+	spin_lock_irq(&ctx->slock);
 	vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, VB2_BUF_STATE_QUEUED);
 	ctx->cur_frm = NULL;
 	ctx->next_frm = NULL;
@@ -542,7 +541,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED);
 	}
-	spin_unlock_irqrestore(&ctx->slock, flags);
+	spin_unlock_irq(&ctx->slock);
 	return ret;
 }
 
@@ -551,7 +550,6 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
 	struct cal_dmaqueue *dma_q = &ctx->vidq;
 	struct cal_buffer *buf, *tmp;
-	unsigned long flags;
 
 	cal_ctx_wr_dma_stop(ctx);
 	cal_ctx_disable_irqs(ctx);
@@ -559,7 +557,7 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 	v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 0);
 
 	/* Release all active buffers */
-	spin_lock_irqsave(&ctx->slock, flags);
+	spin_lock_irq(&ctx->slock);
 	list_for_each_entry_safe(buf, tmp, &dma_q->active, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
@@ -574,7 +572,7 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 	}
 	ctx->cur_frm = NULL;
 	ctx->next_frm = NULL;
-	spin_unlock_irqrestore(&ctx->slock, flags);
+	spin_unlock_irq(&ctx->slock);
 
 	pm_runtime_put_sync(ctx->cal->dev);
 }
-- 
Regards,

Laurent Pinchart

[PATCH v3 15/24] media: ti-vpe: cal: Share buffer release code between start and stop

[Laurent Pinchart]

The cal_start_streaming() and cal_stop_streaming() functions duplicate
the same buffer release logic. split it to a separate function to share
the code.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 55 +++++++++++------------
 1 file changed, 26 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index de0ba6128715..7eec0a57b141 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -486,11 +486,34 @@ static void cal_buffer_queue(struct vb2_buffer *vb)
 	spin_unlock_irqrestore(&ctx->slock, flags);
 }
 
+static void cal_release_buffers(struct cal_ctx *ctx,
+				enum vb2_buffer_state state)
+{
+	struct cal_buffer *buf, *tmp;
+
+	/* Release all active buffers. */
+	spin_lock_irq(&ctx->slock);
+
+	list_for_each_entry_safe(buf, tmp, &ctx->vidq.active, list) {
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vb.vb2_buf, state);
+	}
+
+	if (ctx->next_frm != ctx->cur_frm)
+		vb2_buffer_done(&ctx->next_frm->vb.vb2_buf, state);
+	vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, state);
+
+	ctx->cur_frm = NULL;
+	ctx->next_frm = NULL;
+
+	spin_unlock_irq(&ctx->slock);
+}
+
 static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
 	struct cal_dmaqueue *dma_q = &ctx->vidq;
-	struct cal_buffer *buf, *tmp;
+	struct cal_buffer *buf;
 	unsigned long addr;
 	int ret;
 
@@ -533,46 +556,20 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	cal_ctx_disable_irqs(ctx);
 	ctx->dma_state = CAL_DMA_STOPPED;
 
-	spin_lock_irq(&ctx->slock);
-	vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, VB2_BUF_STATE_QUEUED);
-	ctx->cur_frm = NULL;
-	ctx->next_frm = NULL;
-	list_for_each_entry_safe(buf, tmp, &dma_q->active, list) {
-		list_del(&buf->list);
-		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED);
-	}
-	spin_unlock_irq(&ctx->slock);
+	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
 }
 
 static void cal_stop_streaming(struct vb2_queue *vq)
 {
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
-	struct cal_dmaqueue *dma_q = &ctx->vidq;
-	struct cal_buffer *buf, *tmp;
 
 	cal_ctx_wr_dma_stop(ctx);
 	cal_ctx_disable_irqs(ctx);
 
 	v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 0);
 
-	/* Release all active buffers */
-	spin_lock_irq(&ctx->slock);
-	list_for_each_entry_safe(buf, tmp, &dma_q->active, list) {
-		list_del(&buf->list);
-		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
-	}
-
-	if (ctx->cur_frm == ctx->next_frm) {
-		vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, VB2_BUF_STATE_ERROR);
-	} else {
-		vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, VB2_BUF_STATE_ERROR);
-		vb2_buffer_done(&ctx->next_frm->vb.vb2_buf,
-				VB2_BUF_STATE_ERROR);
-	}
-	ctx->cur_frm = NULL;
-	ctx->next_frm = NULL;
-	spin_unlock_irq(&ctx->slock);
+	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
 
 	pm_runtime_put_sync(ctx->cal->dev);
 }
-- 
Regards,

Laurent Pinchart

[PATCH v3 16/24] media: ti-vpe: cal: Drop V4L2_CAP_READWRITE

[Laurent Pinchart]

The V4L2 read/write API is inefficient and makes little sense for
devices that handle frame-based formats. Applications shouldn't use it,
drop its support from the driver.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
Changes since v2:

- Drop .read handler
---
 drivers/media/platform/ti-vpe/cal-video.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 7eec0a57b141..32dd4d9ca212 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -393,7 +393,6 @@ static const struct v4l2_file_operations cal_fops = {
 	.owner		= THIS_MODULE,
 	.open           = v4l2_fh_open,
 	.release        = vb2_fop_release,
-	.read           = vb2_fop_read,
 	.poll		= vb2_fop_poll,
 	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
 	.mmap           = vb2_fop_mmap,
@@ -595,8 +594,7 @@ static const struct video_device cal_videodev = {
 	.ioctl_ops	= &cal_ioctl_ops,
 	.minor		= -1,
 	.release	= video_device_release_empty,
-	.device_caps	= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING |
-			  V4L2_CAP_READWRITE,
+	.device_caps	= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
 };
 
 static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
@@ -731,7 +729,7 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 
 	/* Initialize the vb2 queue. */
 	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
-	q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;
+	q->io_modes = VB2_MMAP | VB2_DMABUF;
 	q->drv_priv = ctx;
 	q->buf_struct_size = sizeof(struct cal_buffer);
 	q->ops = &cal_video_qops;
-- 
Regards,

Laurent Pinchart

[PATCH v3 17/24] media: ti-vpe: cal: Drop unneeded check in cal_calc_format_size()

[Laurent Pinchart]

The cal_calc_format_size() function checks that the passed fmtinfo
argument is not NULL. All callers ensure that the pointer is valid. Drop
the check.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 16 +++++-----------
 1 file changed, 5 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 32dd4d9ca212..ad1e85189831 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -146,17 +146,12 @@ static int __subdev_set_format(struct cal_ctx *ctx,
 	return 0;
 }
 
-static int cal_calc_format_size(struct cal_ctx *ctx,
-				const struct cal_format_info *fmtinfo,
-				struct v4l2_format *f)
+static void cal_calc_format_size(struct cal_ctx *ctx,
+				 const struct cal_format_info *fmtinfo,
+				 struct v4l2_format *f)
 {
 	u32 bpl, max_width;
 
-	if (!fmtinfo) {
-		ctx_dbg(3, ctx, "No cal_fmt provided!\n");
-		return -EINVAL;
-	}
-
 	/*
 	 * Maximum width is bound by the DMA max width in bytes.
 	 * We need to recalculate the actual maxi width depending on the
@@ -177,8 +172,6 @@ static int cal_calc_format_size(struct cal_ctx *ctx,
 		__func__, fourcc_to_str(f->fmt.pix.pixelformat),
 		f->fmt.pix.width, f->fmt.pix.height,
 		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
-
-	return 0;
 }
 
 static int cal_g_fmt_vid_cap(struct file *file, void *priv,
@@ -247,7 +240,8 @@ static int cal_try_fmt_vid_cap(struct file *file, void *priv,
 	 * updated properly during s_fmt
 	 */
 	f->fmt.pix.colorspace = ctx->v_fmt.fmt.pix.colorspace;
-	return cal_calc_format_size(ctx, fmtinfo, f);
+	cal_calc_format_size(ctx, fmtinfo, f);
+	return 0;
 }
 
 static int cal_s_fmt_vid_cap(struct file *file, void *priv,
-- 
Regards,

Laurent Pinchart

[PATCH v3 18/24] media: ti-vpe: cal: Remove DMA queue empty check at start streaming time

[Laurent Pinchart]

The vb2 queue ensures that the start streaming operation will only be
called with a minimal number of buffers queued to the driver. There's
thus no need to manually check if the DMA queue is empty. Remove the
check.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index ad1e85189831..bfc3ba4a96af 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -511,12 +511,6 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	int ret;
 
 	spin_lock_irq(&ctx->slock);
-	if (list_empty(&dma_q->active)) {
-		spin_unlock_irq(&ctx->slock);
-		ctx_dbg(3, ctx, "buffer queue is empty\n");
-		return -EIO;
-	}
-
 	buf = list_entry(dma_q->active.next, struct cal_buffer, list);
 	ctx->cur_frm = buf;
 	ctx->next_frm = buf;
-- 
Regards,

Laurent Pinchart

[PATCH v3 19/24] media: ti-vpe: cal: Use list_first_entry()

[Laurent Pinchart]

Use the list_first_entry() macro where appropriate to replace manual
usage of list_entry(head.next).

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 2 +-
 drivers/media/platform/ti-vpe/cal.c       | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index bfc3ba4a96af..c80c4643d8ed 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -511,7 +511,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	int ret;
 
 	spin_lock_irq(&ctx->slock);
-	buf = list_entry(dma_q->active.next, struct cal_buffer, list);
+	buf = list_first_entry(&dma_q->active, struct cal_buffer, list);
 	ctx->cur_frm = buf;
 	ctx->next_frm = buf;
 	list_del(&buf->list);
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index f1c2b8bc28f4..8f25e7a6f5e8 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -496,7 +496,7 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 		struct cal_buffer *buf;
 		unsigned long addr;
 
-		buf = list_entry(dma_q->active.next, struct cal_buffer, list);
+		buf = list_first_entry(&dma_q->active, struct cal_buffer, list);
 		addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
 		cal_ctx_wr_dma_addr(ctx, addr);
 
-- 
Regards,

Laurent Pinchart

[PATCH v3 20/24] media: ti-vpe: cal: Group all DMA queue fields in struct cal_dmaqueue

[Laurent Pinchart]

The cal_dmaqueue structure only contains the list of queued buffers.
Move the other fields that are logically related to the DMA queue
(current and next buffer points, state, wait queue and lock) from
cal_ctx to cal_dmaqueue.

Take this as an opportunity to document the fields usage and to give
them more appropriate names. The 'active' field stored the list of all
queued buffers, not the active buffers, so rename it to 'queue'. The
'cur_frm' and 'next_frm' are respectively renamed to 'active' and
'pending' to better explain their purpose. The 'dma_state' and
'dma_wait' fields are stripped of their 'dma_' prefix as they're now
part of cal_dmaqueue. Finally, 'slock' is renamed to 'lock'.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 48 +++++++++++-----------
 drivers/media/platform/ti-vpe/cal.c       | 49 +++++++++++------------
 drivers/media/platform/ti-vpe/cal.h       | 44 ++++++++++++++------
 3 files changed, 79 insertions(+), 62 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index c80c4643d8ed..34dfe38dc960 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -470,13 +470,12 @@ static void cal_buffer_queue(struct vb2_buffer *vb)
 	struct cal_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 	struct cal_buffer *buf = container_of(vb, struct cal_buffer,
 					      vb.vb2_buf);
-	struct cal_dmaqueue *vidq = &ctx->vidq;
 	unsigned long flags;
 
 	/* recheck locking */
-	spin_lock_irqsave(&ctx->slock, flags);
-	list_add_tail(&buf->list, &vidq->active);
-	spin_unlock_irqrestore(&ctx->slock, flags);
+	spin_lock_irqsave(&ctx->dma.lock, flags);
+	list_add_tail(&buf->list, &ctx->dma.queue);
+	spin_unlock_irqrestore(&ctx->dma.lock, flags);
 }
 
 static void cal_release_buffers(struct cal_ctx *ctx,
@@ -484,42 +483,41 @@ static void cal_release_buffers(struct cal_ctx *ctx,
 {
 	struct cal_buffer *buf, *tmp;
 
-	/* Release all active buffers. */
-	spin_lock_irq(&ctx->slock);
+	/* Release all queued buffers. */
+	spin_lock_irq(&ctx->dma.lock);
 
-	list_for_each_entry_safe(buf, tmp, &ctx->vidq.active, list) {
+	list_for_each_entry_safe(buf, tmp, &ctx->dma.queue, list) {
 		list_del(&buf->list);
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
 
-	if (ctx->next_frm != ctx->cur_frm)
-		vb2_buffer_done(&ctx->next_frm->vb.vb2_buf, state);
-	vb2_buffer_done(&ctx->cur_frm->vb.vb2_buf, state);
+	if (ctx->dma.pending != ctx->dma.active)
+		vb2_buffer_done(&ctx->dma.pending->vb.vb2_buf, state);
+	vb2_buffer_done(&ctx->dma.active->vb.vb2_buf, state);
 
-	ctx->cur_frm = NULL;
-	ctx->next_frm = NULL;
+	ctx->dma.active = NULL;
+	ctx->dma.pending = NULL;
 
-	spin_unlock_irq(&ctx->slock);
+	spin_unlock_irq(&ctx->dma.lock);
 }
 
 static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
-	struct cal_dmaqueue *dma_q = &ctx->vidq;
 	struct cal_buffer *buf;
 	unsigned long addr;
 	int ret;
 
-	spin_lock_irq(&ctx->slock);
-	buf = list_first_entry(&dma_q->active, struct cal_buffer, list);
-	ctx->cur_frm = buf;
-	ctx->next_frm = buf;
+	spin_lock_irq(&ctx->dma.lock);
+	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
+	ctx->dma.active = buf;
+	ctx->dma.pending = buf;
 	list_del(&buf->list);
-	spin_unlock_irq(&ctx->slock);
+	spin_unlock_irq(&ctx->dma.lock);
 
-	addr = vb2_dma_contig_plane_dma_addr(&ctx->cur_frm->vb.vb2_buf, 0);
+	addr = vb2_dma_contig_plane_dma_addr(&ctx->dma.active->vb.vb2_buf, 0);
 	ctx->sequence = 0;
-	ctx->dma_state = CAL_DMA_RUNNING;
+	ctx->dma.state = CAL_DMA_RUNNING;
 
 	pm_runtime_get_sync(ctx->cal->dev);
 
@@ -541,7 +539,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 err:
 	cal_ctx_wr_dma_disable(ctx);
 	cal_ctx_disable_irqs(ctx);
-	ctx->dma_state = CAL_DMA_STOPPED;
+	ctx->dma.state = CAL_DMA_STOPPED;
 
 	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
@@ -710,10 +708,10 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 	struct vb2_queue *q = &ctx->vb_vidq;
 	int ret;
 
-	INIT_LIST_HEAD(&ctx->vidq.active);
-	spin_lock_init(&ctx->slock);
+	INIT_LIST_HEAD(&ctx->dma.queue);
+	spin_lock_init(&ctx->dma.lock);
 	mutex_init(&ctx->mutex);
-	init_waitqueue_head(&ctx->dma_wait);
+	init_waitqueue_head(&ctx->dma.wait);
 
 	/* Initialize the vb2 queue. */
 	q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 8f25e7a6f5e8..3e0a69bb7fe5 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -424,9 +424,9 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
 {
 	bool stopped;
 
-	spin_lock_irq(&ctx->slock);
-	stopped = ctx->dma_state == CAL_DMA_STOPPED;
-	spin_unlock_irq(&ctx->slock);
+	spin_lock_irq(&ctx->dma.lock);
+	stopped = ctx->dma.state == CAL_DMA_STOPPED;
+	spin_unlock_irq(&ctx->dma.lock);
 
 	return stopped;
 }
@@ -436,11 +436,11 @@ int cal_ctx_wr_dma_stop(struct cal_ctx *ctx)
 	long timeout;
 
 	/* Request DMA stop and wait until it completes. */
-	spin_lock_irq(&ctx->slock);
-	ctx->dma_state = CAL_DMA_STOP_REQUESTED;
-	spin_unlock_irq(&ctx->slock);
+	spin_lock_irq(&ctx->dma.lock);
+	ctx->dma.state = CAL_DMA_STOP_REQUESTED;
+	spin_unlock_irq(&ctx->dma.lock);
 
-	timeout = wait_event_timeout(ctx->dma_wait, cal_ctx_wr_dma_stopped(ctx),
+	timeout = wait_event_timeout(ctx->dma.wait, cal_ctx_wr_dma_stopped(ctx),
 				     msecs_to_jiffies(500));
 	if (!timeout) {
 		ctx_err(ctx, "failed to disable dma cleanly\n");
@@ -475,20 +475,18 @@ void cal_ctx_disable_irqs(struct cal_ctx *ctx)
 
 static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 {
-	struct cal_dmaqueue *dma_q = &ctx->vidq;
+	spin_lock(&ctx->dma.lock);
 
-	spin_lock(&ctx->slock);
-
-	if (ctx->dma_state == CAL_DMA_STOP_REQUESTED) {
+	if (ctx->dma.state == CAL_DMA_STOP_REQUESTED) {
 		/*
 		 * If a stop is requested, disable the write DMA context
 		 * immediately. The CAL_WR_DMA_CTRL_j.MODE field is shadowed,
 		 * the current frame will complete and the DMA will then stop.
 		 */
 		cal_ctx_wr_dma_disable(ctx);
-		ctx->dma_state = CAL_DMA_STOP_PENDING;
-	} else if (!list_empty(&dma_q->active) &&
-		   ctx->cur_frm == ctx->next_frm) {
+		ctx->dma.state = CAL_DMA_STOP_PENDING;
+	} else if (!list_empty(&ctx->dma.queue) &&
+		   ctx->dma.active == ctx->dma.pending) {
 		/*
 		 * Otherwise, if a new buffer is available, queue it to the
 		 * hardware.
@@ -496,36 +494,37 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 		struct cal_buffer *buf;
 		unsigned long addr;
 
-		buf = list_first_entry(&dma_q->active, struct cal_buffer, list);
+		buf = list_first_entry(&ctx->dma.queue, struct cal_buffer,
+				       list);
 		addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
 		cal_ctx_wr_dma_addr(ctx, addr);
 
-		ctx->next_frm = buf;
+		ctx->dma.pending = buf;
 		list_del(&buf->list);
 	}
 
-	spin_unlock(&ctx->slock);
+	spin_unlock(&ctx->dma.lock);
 }
 
 static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
 {
 	struct cal_buffer *buf = NULL;
 
-	spin_lock(&ctx->slock);
+	spin_lock(&ctx->dma.lock);
 
 	/* If the DMA context was stopping, it is now stopped. */
-	if (ctx->dma_state == CAL_DMA_STOP_PENDING) {
-		ctx->dma_state = CAL_DMA_STOPPED;
-		wake_up(&ctx->dma_wait);
+	if (ctx->dma.state == CAL_DMA_STOP_PENDING) {
+		ctx->dma.state = CAL_DMA_STOPPED;
+		wake_up(&ctx->dma.wait);
 	}
 
 	/* If a new buffer was queued, complete the current buffer. */
-	if (ctx->cur_frm != ctx->next_frm) {
-		buf = ctx->cur_frm;
-		ctx->cur_frm = ctx->next_frm;
+	if (ctx->dma.active != ctx->dma.pending) {
+		buf = ctx->dma.active;
+		ctx->dma.active = ctx->dma.pending;
 	}
 
-	spin_unlock(&ctx->slock);
+	spin_unlock(&ctx->dma.lock);
 
 	if (buf) {
 		buf->vb.vb2_buf.timestamp = ktime_get_ns();
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 5ee0e05c3305..9d3cbc13f915 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -82,8 +82,38 @@ struct cal_buffer {
 	struct list_head	list;
 };
 
+/**
+ * struct cal_dmaqueue - Queue of DMA buffers
+ * @active: Buffer being DMA'ed to for the current frame
+ */
 struct cal_dmaqueue {
-	struct list_head	active;
+	/**
+	 * Protects all fields in the cal_dmaqueue.
+	 */
+	spinlock_t		lock;
+
+	/**
+	 * Buffers queued to the driver and waiting for DMA processing.
+	 * Buffers are added to the list by the vb2 .buffer_queue() operation,
+	 * and move to @pending when they are scheduled for the next frame.
+	 */
+	struct list_head	queue;
+	/**
+	 * Buffer provided to the hardware to DMA the next frame. Will move to
+	 * @active at the end of the current frame.
+	 */
+	struct cal_buffer	*pending;
+	/**
+	 * Buffer being DMA'ed to for the current frame. Will be retired and
+	 * given back to vb2 at the end of the current frame if a @pending
+	 * buffer has been scheduled to replace it.
+	 */
+	struct cal_buffer	*active;
+
+	/** State of the DMA engine. */
+	enum cal_dma_state	state;
+	/** Wait queue to signal a @state transition to CAL_DMA_STOPPED. */
+	struct wait_queue_head	wait;
 };
 
 struct cal_camerarx_data {
@@ -174,10 +204,8 @@ struct cal_ctx {
 
 	/* v4l2_ioctl mutex */
 	struct mutex		mutex;
-	/* v4l2 buffers lock */
-	spinlock_t		slock;
 
-	struct cal_dmaqueue	vidq;
+	struct cal_dmaqueue	dma;
 
 	/* video capture */
 	const struct cal_format_info	*fmtinfo;
@@ -192,14 +220,6 @@ struct cal_ctx {
 	struct vb2_queue	vb_vidq;
 	unsigned int		index;
 	unsigned int		cport;
-
-	/* Pointer pointing to current v4l2_buffer */
-	struct cal_buffer	*cur_frm;
-	/* Pointer pointing to next v4l2_buffer */
-	struct cal_buffer	*next_frm;
-
-	enum cal_dma_state	dma_state;
-	struct wait_queue_head	dma_wait;
 };
 
 extern unsigned int cal_debug;
-- 
Regards,

Laurent Pinchart

[PATCH v3 21/24] media: ti-vpe: cal: Set cal_dmaqueue.pending to NULL when no pending buffer

[Laurent Pinchart]

When a pending buffer becomes active, the cal_dmaqueue.active field is
updated, but the pending field keeps the same value until a new buffer
becomes pending. This requires handling the special case of
pending == active in different places. Simplify the code by setting the
pending field to NULL when the pending buffer becomes active. Buffers
are now simply moved from queue to pending and from pending to active.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 14 ++++++++------
 drivers/media/platform/ti-vpe/cal.c       |  6 +++---
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 34dfe38dc960..438447728b46 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -491,12 +491,15 @@ static void cal_release_buffers(struct cal_ctx *ctx,
 		vb2_buffer_done(&buf->vb.vb2_buf, state);
 	}
 
-	if (ctx->dma.pending != ctx->dma.active)
+	if (ctx->dma.pending) {
 		vb2_buffer_done(&ctx->dma.pending->vb.vb2_buf, state);
-	vb2_buffer_done(&ctx->dma.active->vb.vb2_buf, state);
+		ctx->dma.pending = NULL;
+	}
 
-	ctx->dma.active = NULL;
-	ctx->dma.pending = NULL;
+	if (ctx->dma.active) {
+		vb2_buffer_done(&ctx->dma.active->vb.vb2_buf, state);
+		ctx->dma.active = NULL;
+	}
 
 	spin_unlock_irq(&ctx->dma.lock);
 }
@@ -510,12 +513,11 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	spin_lock_irq(&ctx->dma.lock);
 	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
-	ctx->dma.active = buf;
 	ctx->dma.pending = buf;
 	list_del(&buf->list);
 	spin_unlock_irq(&ctx->dma.lock);
 
-	addr = vb2_dma_contig_plane_dma_addr(&ctx->dma.active->vb.vb2_buf, 0);
+	addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
 	ctx->sequence = 0;
 	ctx->dma.state = CAL_DMA_RUNNING;
 
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 3e0a69bb7fe5..547dffcfe68f 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -485,8 +485,7 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 		 */
 		cal_ctx_wr_dma_disable(ctx);
 		ctx->dma.state = CAL_DMA_STOP_PENDING;
-	} else if (!list_empty(&ctx->dma.queue) &&
-		   ctx->dma.active == ctx->dma.pending) {
+	} else if (!list_empty(&ctx->dma.queue) && !ctx->dma.pending) {
 		/*
 		 * Otherwise, if a new buffer is available, queue it to the
 		 * hardware.
@@ -519,9 +518,10 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
 	}
 
 	/* If a new buffer was queued, complete the current buffer. */
-	if (ctx->dma.active != ctx->dma.pending) {
+	if (ctx->dma.pending) {
 		buf = ctx->dma.active;
 		ctx->dma.active = ctx->dma.pending;
+		ctx->dma.pending = NULL;
 	}
 
 	spin_unlock(&ctx->dma.lock);
-- 
Regards,

Laurent Pinchart

[PATCH v3 22/24] media: ti-vpe: cal: Store buffer DMA address in dma_addr_t

[Laurent Pinchart]

dma_addr_t is the correct type to store DMA addresses. Replace incorrect
usage of unsigned long and unsigned int.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 2 +-
 drivers/media/platform/ti-vpe/cal.c       | 6 +++---
 drivers/media/platform/ti-vpe/cal.h       | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 438447728b46..511767dd69bd 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -508,7 +508,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
 	struct cal_buffer *buf;
-	unsigned long addr;
+	dma_addr_t addr;
 	int ret;
 
 	spin_lock_irq(&ctx->dma.lock);
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 547dffcfe68f..3cf625262d32 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -406,9 +406,9 @@ void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 	ctx_dbg(3, ctx, "CAL_CTRL = 0x%08x\n", cal_read(ctx->cal, CAL_CTRL));
 }
 
-void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, unsigned int dmaaddr)
+void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, dma_addr_t addr)
 {
-	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->index), dmaaddr);
+	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->index), addr);
 }
 
 void cal_ctx_wr_dma_disable(struct cal_ctx *ctx)
@@ -491,7 +491,7 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 		 * hardware.
 		 */
 		struct cal_buffer *buf;
-		unsigned long addr;
+		dma_addr_t addr;
 
 		buf = list_first_entry(&ctx->dma.queue, struct cal_buffer,
 				       list);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 9d3cbc13f915..26916f72fd60 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -299,7 +299,7 @@ void cal_camerarx_destroy(struct cal_camerarx *phy);
 void cal_ctx_csi2_config(struct cal_ctx *ctx);
 void cal_ctx_pix_proc_config(struct cal_ctx *ctx);
 void cal_ctx_wr_dma_config(struct cal_ctx *ctx);
-void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, unsigned int dmaaddr);
+void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, dma_addr_t addr);
 void cal_ctx_wr_dma_disable(struct cal_ctx *ctx);
 int cal_ctx_wr_dma_stop(struct cal_ctx *ctx);
 void cal_ctx_enable_irqs(struct cal_ctx *ctx);
-- 
Regards,

Laurent Pinchart

[PATCH v3 23/24] media: ti-vpe: cal: Simplify the context API

[Laurent Pinchart]

Rework the context API exposed to cal-video.c to simplify it. The
configuration and enable steps are all grouped in a single
cal_ctx_start() function, and the DMA stop and IRQ disable are similarly
groupd in cal_ctx_stop(). The cal_ctx_wr_dma_addr() function is renamed
to cal_ctx_set_dma_addr() for consistency with the cal_ctx_ prefix of
the start and stop functions.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Benoit Parrot <bparrot@ti.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 21 +++------
 drivers/media/platform/ti-vpe/cal.c       | 54 +++++++++++++----------
 drivers/media/platform/ti-vpe/cal.h       | 11 ++---
 3 files changed, 41 insertions(+), 45 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 511767dd69bd..779f1e1bc529 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -518,16 +518,11 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	spin_unlock_irq(&ctx->dma.lock);
 
 	addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
-	ctx->sequence = 0;
-	ctx->dma.state = CAL_DMA_RUNNING;
 
 	pm_runtime_get_sync(ctx->cal->dev);
 
-	cal_ctx_csi2_config(ctx);
-	cal_ctx_pix_proc_config(ctx);
-	cal_ctx_wr_dma_config(ctx);
-	cal_ctx_wr_dma_addr(ctx, addr);
-	cal_ctx_enable_irqs(ctx);
+	cal_ctx_set_dma_addr(ctx, addr);
+	cal_ctx_start(ctx);
 
 	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
 	if (ret)
@@ -539,9 +534,8 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	return 0;
 
 err:
-	cal_ctx_wr_dma_disable(ctx);
-	cal_ctx_disable_irqs(ctx);
-	ctx->dma.state = CAL_DMA_STOPPED;
+	cal_ctx_stop(ctx);
+	pm_runtime_put_sync(ctx->cal->dev);
 
 	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
 	return ret;
@@ -551,14 +545,13 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 {
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
 
-	cal_ctx_wr_dma_stop(ctx);
-	cal_ctx_disable_irqs(ctx);
+	cal_ctx_stop(ctx);
 
 	v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 0);
 
+	pm_runtime_put_sync(ctx->cal->dev);
+
 	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
-
-	pm_runtime_put_sync(ctx->cal->dev);
 }
 
 static const struct vb2_ops cal_video_qops = {
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 3cf625262d32..293cbac905b3 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -280,7 +280,7 @@ void cal_quickdump_regs(struct cal_dev *cal)
  * ------------------------------------------------------------------
  */
 
-void cal_ctx_csi2_config(struct cal_ctx *ctx)
+static void cal_ctx_csi2_config(struct cal_ctx *ctx)
 {
 	u32 val;
 
@@ -305,7 +305,7 @@ void cal_ctx_csi2_config(struct cal_ctx *ctx)
 		cal_read(ctx->cal, CAL_CSI2_CTX0(ctx->index)));
 }
 
-void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
+static void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
 {
 	u32 val, extract, pack;
 
@@ -356,7 +356,7 @@ void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
 		cal_read(ctx->cal, CAL_PIX_PROC(ctx->index)));
 }
 
-void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
+static void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 {
 	unsigned int stride = ctx->v_fmt.fmt.pix.bytesperline;
 	u32 val;
@@ -406,12 +406,12 @@ void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 	ctx_dbg(3, ctx, "CAL_CTRL = 0x%08x\n", cal_read(ctx->cal, CAL_CTRL));
 }
 
-void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, dma_addr_t addr)
+void cal_ctx_set_dma_addr(struct cal_ctx *ctx, dma_addr_t addr)
 {
 	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->index), addr);
 }
 
-void cal_ctx_wr_dma_disable(struct cal_ctx *ctx)
+static void cal_ctx_wr_dma_disable(struct cal_ctx *ctx)
 {
 	u32 val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->index));
 
@@ -431,11 +431,31 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
 	return stopped;
 }
 
-int cal_ctx_wr_dma_stop(struct cal_ctx *ctx)
+void cal_ctx_start(struct cal_ctx *ctx)
+{
+	ctx->sequence = 0;
+	ctx->dma.state = CAL_DMA_RUNNING;
+
+	/* Configure the CSI-2, pixel processing and write DMA contexts. */
+	cal_ctx_csi2_config(ctx);
+	cal_ctx_pix_proc_config(ctx);
+	cal_ctx_wr_dma_config(ctx);
+
+	/* Enable IRQ_WDMA_END and IRQ_WDMA_START. */
+	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(1),
+		  CAL_HL_IRQ_MASK(ctx->index));
+	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(2),
+		  CAL_HL_IRQ_MASK(ctx->index));
+}
+
+void cal_ctx_stop(struct cal_ctx *ctx)
 {
 	long timeout;
 
-	/* Request DMA stop and wait until it completes. */
+	/*
+	 * Request DMA stop and wait until it completes. If completion times
+	 * out, forcefully disable the DMA.
+	 */
 	spin_lock_irq(&ctx->dma.lock);
 	ctx->dma.state = CAL_DMA_STOP_REQUESTED;
 	spin_unlock_irq(&ctx->dma.lock);
@@ -444,28 +464,16 @@ int cal_ctx_wr_dma_stop(struct cal_ctx *ctx)
 				     msecs_to_jiffies(500));
 	if (!timeout) {
 		ctx_err(ctx, "failed to disable dma cleanly\n");
-		return -ETIMEDOUT;
+		cal_ctx_wr_dma_disable(ctx);
 	}
 
-	return 0;
-}
-
-void cal_ctx_enable_irqs(struct cal_ctx *ctx)
-{
-	/* Enable IRQ_WDMA_END and IRQ_WDMA_START. */
-	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(1),
-		  CAL_HL_IRQ_MASK(ctx->index));
-	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(2),
-		  CAL_HL_IRQ_MASK(ctx->index));
-}
-
-void cal_ctx_disable_irqs(struct cal_ctx *ctx)
-{
 	/* Disable IRQ_WDMA_END and IRQ_WDMA_START. */
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(1),
 		  CAL_HL_IRQ_MASK(ctx->index));
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(2),
 		  CAL_HL_IRQ_MASK(ctx->index));
+
+	ctx->dma.state = CAL_DMA_STOPPED;
 }
 
 /* ------------------------------------------------------------------
@@ -496,7 +504,7 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 		buf = list_first_entry(&ctx->dma.queue, struct cal_buffer,
 				       list);
 		addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
-		cal_ctx_wr_dma_addr(ctx, addr);
+		cal_ctx_set_dma_addr(ctx, addr);
 
 		ctx->dma.pending = buf;
 		list_del(&buf->list);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 26916f72fd60..60f5f7480b17 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -296,14 +296,9 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 					 unsigned int instance);
 void cal_camerarx_destroy(struct cal_camerarx *phy);
 
-void cal_ctx_csi2_config(struct cal_ctx *ctx);
-void cal_ctx_pix_proc_config(struct cal_ctx *ctx);
-void cal_ctx_wr_dma_config(struct cal_ctx *ctx);
-void cal_ctx_wr_dma_addr(struct cal_ctx *ctx, dma_addr_t addr);
-void cal_ctx_wr_dma_disable(struct cal_ctx *ctx);
-int cal_ctx_wr_dma_stop(struct cal_ctx *ctx);
-void cal_ctx_enable_irqs(struct cal_ctx *ctx);
-void cal_ctx_disable_irqs(struct cal_ctx *ctx);
+void cal_ctx_set_dma_addr(struct cal_ctx *ctx, dma_addr_t addr);
+void cal_ctx_start(struct cal_ctx *ctx);
+void cal_ctx_stop(struct cal_ctx *ctx);
 
 int cal_ctx_v4l2_register(struct cal_ctx *ctx);
 void cal_ctx_v4l2_unregister(struct cal_ctx *ctx);
-- 
Regards,

Laurent Pinchart

[PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Laurent Pinchart]

The CAL driver is video node centric, it controls the whole device
through the video device nodes. This limits the possible use cases as it
can't support sources that are more complex than a single subdev. To
support more complex hardware pipelines, implement support for the media
controller centric API. The exposed API can be selected through a module
parameter.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v1:

- Fix parameter name and description in MODULE_PARM_DESC()
---
 drivers/media/platform/ti-vpe/cal-camerarx.c |   1 +
 drivers/media/platform/ti-vpe/cal-video.c    | 356 ++++++++++++++-----
 drivers/media/platform/ti-vpe/cal.c          |  10 +-
 drivers/media/platform/ti-vpe/cal.h          |   2 +
 4 files changed, 288 insertions(+), 81 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 1920f36137b8..8abf887c47d6 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -830,6 +830,7 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	sd = &phy->subdev;
 	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
 	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
 	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
 	sd->dev = cal->dev;
 
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 779f1e1bc529..119a17fb97bc 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -40,40 +40,10 @@ static char *fourcc_to_str(u32 fmt)
 }
 
 /* ------------------------------------------------------------------
- *	V4L2 Video IOCTLs
+ *	V4L2 Common IOCTLs
  * ------------------------------------------------------------------
  */
 
-static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
-							u32 pixelformat)
-{
-	const struct cal_format_info *fmtinfo;
-	unsigned int k;
-
-	for (k = 0; k < ctx->num_active_fmt; k++) {
-		fmtinfo = ctx->active_fmt[k];
-		if (fmtinfo->fourcc == pixelformat)
-			return fmtinfo;
-	}
-
-	return NULL;
-}
-
-static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
-							 u32 code)
-{
-	const struct cal_format_info *fmtinfo;
-	unsigned int k;
-
-	for (k = 0; k < ctx->num_active_fmt; k++) {
-		fmtinfo = ctx->active_fmt[k];
-		if (fmtinfo->code == code)
-			return fmtinfo;
-	}
-
-	return NULL;
-}
-
 static int cal_querycap(struct file *file, void *priv,
 			struct v4l2_capability *cap)
 {
@@ -87,6 +57,51 @@ static int cal_querycap(struct file *file, void *priv,
 	return 0;
 }
 
+static int cal_g_fmt_vid_cap(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct cal_ctx *ctx = video_drvdata(file);
+
+	*f = ctx->v_fmt;
+
+	return 0;
+}
+
+/* ------------------------------------------------------------------
+ *	V4L2 Video Node Centric IOCTLs
+ * ------------------------------------------------------------------
+ */
+
+static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
+							u32 pixelformat)
+{
+	const struct cal_format_info *fmtinfo;
+	unsigned int k;
+
+	for (k = 0; k < ctx->num_active_fmt; k++) {
+		fmtinfo = ctx->active_fmt[k];
+		if (fmtinfo->fourcc == pixelformat)
+			return fmtinfo;
+	}
+
+	return NULL;
+}
+
+static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
+							 u32 code)
+{
+	const struct cal_format_info *fmtinfo;
+	unsigned int k;
+
+	for (k = 0; k < ctx->num_active_fmt; k++) {
+		fmtinfo = ctx->active_fmt[k];
+		if (fmtinfo->code == code)
+			return fmtinfo;
+	}
+
+	return NULL;
+}
+
 static int cal_enum_fmt_vid_cap(struct file *file, void  *priv,
 				struct v4l2_fmtdesc *f)
 {
@@ -174,16 +189,6 @@ static void cal_calc_format_size(struct cal_ctx *ctx,
 		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
 }
 
-static int cal_g_fmt_vid_cap(struct file *file, void *priv,
-			     struct v4l2_format *f)
-{
-	struct cal_ctx *ctx = video_drvdata(file);
-
-	*f = ctx->v_fmt;
-
-	return 0;
-}
-
 static int cal_try_fmt_vid_cap(struct file *file, void *priv,
 			       struct v4l2_format *f)
 {
@@ -383,16 +388,7 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
 	return 0;
 }
 
-static const struct v4l2_file_operations cal_fops = {
-	.owner		= THIS_MODULE,
-	.open           = v4l2_fh_open,
-	.release        = vb2_fop_release,
-	.poll		= vb2_fop_poll,
-	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
-	.mmap           = vb2_fop_mmap,
-};
-
-static const struct v4l2_ioctl_ops cal_ioctl_ops = {
+static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
 	.vidioc_querycap      = cal_querycap,
 	.vidioc_enum_fmt_vid_cap  = cal_enum_fmt_vid_cap,
 	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
@@ -418,7 +414,153 @@ static const struct v4l2_ioctl_ops cal_ioctl_ops = {
 };
 
 /* ------------------------------------------------------------------
- *	videobuf2 Operations
+ *	V4L2 Media Controller Centric IOCTLs
+ * ------------------------------------------------------------------
+ */
+
+static int cal_mc_enum_fmt_vid_cap(struct file *file, void  *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	if (f->index >= cal_num_formats)
+		return -EINVAL;
+
+	f->pixelformat = cal_formats[f->index].fourcc;
+	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+
+	return 0;
+}
+
+static void cal_mc_try_fmt(struct cal_ctx *ctx, struct v4l2_format *f,
+			   const struct cal_format_info **info)
+{
+	struct v4l2_pix_format *format = &f->fmt.pix;
+	const struct cal_format_info *fmtinfo;
+	unsigned int bpp;
+
+	/*
+	 * Default to the first format is the requested pixel format code isn't
+	 * supported.
+	 */
+	fmtinfo = cal_format_by_fourcc(f->fmt.pix.pixelformat);
+	if (!fmtinfo)
+		fmtinfo = &cal_formats[0];
+
+	/*
+	 * Clamp the size, update the pixel format. The field and colorspace are
+	 * accepted as-is, except for V4L2_FIELD_ANY that is turned into
+	 * V4L2_FIELD_NONE.
+	 */
+	bpp = ALIGN(fmtinfo->bpp, 8);
+
+	format->width = clamp_t(unsigned int, format->width,
+				CAL_MIN_WIDTH_BYTES * 8 / bpp,
+				CAL_MAX_WIDTH_BYTES * 8 / bpp);
+	format->height = clamp_t(unsigned int, format->height,
+				 CAL_MIN_HEIGHT_LINES, CAL_MAX_HEIGHT_LINES);
+	format->pixelformat = fmtinfo->fourcc;
+
+	if (format->field == V4L2_FIELD_ANY)
+		format->field = V4L2_FIELD_NONE;
+
+	/*
+	 * Calculate the number of bytes per line and the image size. The
+	 * hardware stores the stride as a number of 16 bytes words, in a
+	 * signed 15-bit value. Only 14 bits are thus usable.
+	 */
+	format->bytesperline = ALIGN(clamp(format->bytesperline,
+					   format->width * bpp / 8,
+					   ((1U << 14) - 1) * 16), 16);
+
+	format->sizeimage = format->height * format->bytesperline;
+
+	if (info)
+		*info = fmtinfo;
+
+	ctx_dbg(3, ctx, "%s: %s %ux%u (bytesperline %u sizeimage %u)\n",
+		__func__, fourcc_to_str(format->pixelformat),
+		format->width, format->height,
+		format->bytesperline, format->sizeimage);
+}
+
+static int cal_mc_try_fmt_vid_cap(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct cal_ctx *ctx = video_drvdata(file);
+
+	cal_mc_try_fmt(ctx, f, NULL);
+	return 0;
+}
+
+static int cal_mc_s_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	struct cal_ctx *ctx = video_drvdata(file);
+	const struct cal_format_info *fmtinfo;
+
+	if (vb2_is_busy(&ctx->vb_vidq)) {
+		ctx_dbg(3, ctx, "%s device busy\n", __func__);
+		return -EBUSY;
+	}
+
+	cal_mc_try_fmt(ctx, f, &fmtinfo);
+
+	ctx->v_fmt = *f;
+	ctx->fmtinfo = fmtinfo;
+
+	return 0;
+}
+
+static int cal_mc_enum_framesizes(struct file *file, void *fh,
+				  struct v4l2_frmsizeenum *fsize)
+{
+	struct cal_ctx *ctx = video_drvdata(file);
+	const struct cal_format_info *fmtinfo;
+	unsigned int bpp;
+
+	if (fsize->index > 0)
+		return -EINVAL;
+
+	fmtinfo = cal_format_by_fourcc(fsize->pixel_format);
+	if (!fmtinfo) {
+		ctx_dbg(3, ctx, "Invalid pixel format 0x%08x\n",
+			fsize->pixel_format);
+		return -EINVAL;
+	}
+
+	bpp = ALIGN(fmtinfo->bpp, 8);
+
+	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+	fsize->stepwise.min_width = CAL_MIN_WIDTH_BYTES * 8 / bpp;
+	fsize->stepwise.max_width = CAL_MAX_WIDTH_BYTES * 8 / bpp;
+	fsize->stepwise.step_width = 64 / bpp;
+	fsize->stepwise.min_height = CAL_MIN_HEIGHT_LINES;
+	fsize->stepwise.max_height = CAL_MAX_HEIGHT_LINES;
+	fsize->stepwise.step_height = 1;
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops cal_ioctl_mc_ops = {
+	.vidioc_querycap      = cal_querycap,
+	.vidioc_enum_fmt_vid_cap  = cal_mc_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap   = cal_mc_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap     = cal_mc_s_fmt_vid_cap,
+	.vidioc_enum_framesizes   = cal_mc_enum_framesizes,
+	.vidioc_reqbufs       = vb2_ioctl_reqbufs,
+	.vidioc_create_bufs   = vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf   = vb2_ioctl_prepare_buf,
+	.vidioc_querybuf      = vb2_ioctl_querybuf,
+	.vidioc_qbuf          = vb2_ioctl_qbuf,
+	.vidioc_dqbuf         = vb2_ioctl_dqbuf,
+	.vidioc_expbuf        = vb2_ioctl_expbuf,
+	.vidioc_streamon      = vb2_ioctl_streamon,
+	.vidioc_streamoff     = vb2_ioctl_streamoff,
+	.vidioc_log_status    = v4l2_ctrl_log_status,
+};
+
+/* ------------------------------------------------------------------
+ *	videobuf2 Common Operations
  * ------------------------------------------------------------------
  */
 
@@ -504,6 +646,26 @@ static void cal_release_buffers(struct cal_ctx *ctx,
 	spin_unlock_irq(&ctx->dma.lock);
 }
 
+/* ------------------------------------------------------------------
+ *	videobuf2 Operations
+ * ------------------------------------------------------------------
+ */
+
+static int cal_video_check_format(struct cal_ctx *ctx)
+{
+	const struct v4l2_mbus_framefmt *format;
+
+	format = &ctx->phy->formats[CAL_CAMERARX_PAD_SOURCE];
+
+	if (ctx->fmtinfo->code != format->code ||
+	    ctx->v_fmt.fmt.pix.height != format->height ||
+	    ctx->v_fmt.fmt.pix.width != format->width ||
+	    ctx->v_fmt.fmt.pix.field != format->field)
+		return -EPIPE;
+
+	return 0;
+}
+
 static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
@@ -511,6 +673,23 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma_addr_t addr;
 	int ret;
 
+	ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
+	if (ret < 0) {
+		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
+		return ret;
+	}
+
+	/*
+	 * Verify that the currently configured format matches the output of
+	 * the connected CAMERARX.
+	 */
+	ret = cal_video_check_format(ctx);
+	if (ret < 0) {
+		ctx_dbg(3, ctx,
+			"Format mismatch between CAMERARX and video node\n");
+		goto error_pipeline;
+	}
+
 	spin_lock_irq(&ctx->dma.lock);
 	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
 	ctx->dma.pending = buf;
@@ -526,18 +705,21 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
 	if (ret)
-		goto err;
+		goto error_stop;
 
 	if (cal_debug >= 4)
 		cal_quickdump_regs(ctx->cal);
 
 	return 0;
 
-err:
+error_stop:
 	cal_ctx_stop(ctx);
 	pm_runtime_put_sync(ctx->cal->dev);
 
+error_pipeline:
+	media_pipeline_stop(&ctx->vdev.entity);
 	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
+
 	return ret;
 }
 
@@ -552,6 +734,8 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 	pm_runtime_put_sync(ctx->cal->dev);
 
 	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
+
+	media_pipeline_stop(&ctx->vdev.entity);
 }
 
 static const struct vb2_ops cal_video_qops = {
@@ -569,13 +753,13 @@ static const struct vb2_ops cal_video_qops = {
  * ------------------------------------------------------------------
  */
 
-static const struct video_device cal_videodev = {
-	.name		= CAL_MODULE_NAME,
-	.fops		= &cal_fops,
-	.ioctl_ops	= &cal_ioctl_ops,
-	.minor		= -1,
-	.release	= video_device_release_empty,
-	.device_caps	= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
+static const struct v4l2_file_operations cal_fops = {
+	.owner		= THIS_MODULE,
+	.open           = v4l2_fh_open,
+	.release        = vb2_fop_release,
+	.poll		= vb2_fop_poll,
+	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
+	.mmap           = vb2_fop_mmap,
 };
 
 static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
@@ -649,19 +833,22 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 
 int cal_ctx_v4l2_register(struct cal_ctx *ctx)
 {
-	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
 	struct video_device *vfd = &ctx->vdev;
 	int ret;
 
-	ret = cal_ctx_v4l2_init_formats(ctx);
-	if (ret)
-		return ret;
+	if (!cal_mc_api) {
+		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
 
-	ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler, NULL,
-				    true);
-	if (ret < 0) {
-		ctx_err(ctx, "Failed to add sensor ctrl handler\n");
-		return ret;
+		ret = cal_ctx_v4l2_init_formats(ctx);
+		if (ret)
+			return ret;
+
+		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler,
+					    NULL, true);
+		if (ret < 0) {
+			ctx_err(ctx, "Failed to add sensor ctrl handler\n");
+			return ret;
+		}
 	}
 
 	ret = video_register_device(vfd, VFL_TYPE_VIDEO, cal_video_nr);
@@ -698,7 +885,6 @@ void cal_ctx_v4l2_unregister(struct cal_ctx *ctx)
 
 int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 {
-	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
 	struct video_device *vfd = &ctx->vdev;
 	struct vb2_queue *q = &ctx->vb_vidq;
 	int ret;
@@ -725,10 +911,14 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 		return ret;
 
 	/* Initialize the video device and media entity. */
-	*vfd = cal_videodev;
+	vfd->fops = &cal_fops;
+	vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING
+			 | (cal_mc_api ? V4L2_CAP_IO_MC : 0);
 	vfd->v4l2_dev = &ctx->cal->v4l2_dev;
 	vfd->queue = q;
 	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->index);
+	vfd->release = video_device_release_empty;
+	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_video_ops;
 	vfd->lock = &ctx->mutex;
 	video_set_drvdata(vfd, ctx);
 
@@ -737,15 +927,19 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 	if (ret < 0)
 		return ret;
 
-	/* Initialize the control handler. */
-	ret = v4l2_ctrl_handler_init(hdl, 11);
-	if (ret < 0) {
-		ctx_err(ctx, "Failed to init ctrl handler\n");
-		goto error;
+	if (!cal_mc_api) {
+		/* Initialize the control handler. */
+		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
+
+		ret = v4l2_ctrl_handler_init(hdl, 11);
+		if (ret < 0) {
+			ctx_err(ctx, "Failed to init ctrl handler\n");
+			goto error;
+		}
+
+		vfd->ctrl_handler = hdl;
 	}
 
-	vfd->ctrl_handler = hdl;
-
 	return 0;
 
 error:
@@ -755,6 +949,8 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 
 void cal_ctx_v4l2_cleanup(struct cal_ctx *ctx)
 {
-	v4l2_ctrl_handler_free(&ctx->ctrl_handler);
+	if (!cal_mc_api)
+		v4l2_ctrl_handler_free(&ctx->ctrl_handler);
+
 	media_entity_cleanup(&ctx->vdev.entity);
 }
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 293cbac905b3..2ce2b6404c92 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -43,6 +43,10 @@ unsigned int cal_debug;
 module_param_named(debug, cal_debug, uint, 0644);
 MODULE_PARM_DESC(debug, "activates debug info");
 
+bool cal_mc_api;
+module_param_named(mc_api, cal_mc_api, bool, 0444);
+MODULE_PARM_DESC(mc_api, "activates the MC API");
+
 /* ------------------------------------------------------------------
  *	Format Handling
  * ------------------------------------------------------------------
@@ -660,13 +664,17 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
 {
 	struct cal_dev *cal = container_of(notifier, struct cal_dev, notifier);
 	unsigned int i;
+	int ret = 0;
 
 	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
 		if (cal->ctx[i])
 			cal_ctx_v4l2_register(cal->ctx[i]);
 	}
 
-	return 0;
+	if (cal_mc_api)
+		ret = v4l2_device_register_subdev_nodes(&cal->v4l2_dev);
+
+	return ret;
 }
 
 static const struct v4l2_async_notifier_operations cal_async_notifier_ops = {
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 60f5f7480b17..199692610fb9 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -160,6 +160,7 @@ struct cal_camerarx {
 	struct device_node	*sensor_ep_node;
 	struct device_node	*sensor_node;
 	struct v4l2_subdev	*sensor;
+	struct media_pipeline	pipe;
 
 	struct v4l2_subdev	subdev;
 	struct media_pad	pads[2];
@@ -224,6 +225,7 @@ struct cal_ctx {
 
 extern unsigned int cal_debug;
 extern int cal_video_nr;
+extern bool cal_mc_api;
 
 #define cal_dbg(level, cal, fmt, arg...)				\
 	do {								\
-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Hans Verkuil]

On 07/12/2020 00:53, Laurent Pinchart wrote:
> The CAL driver is video node centric, it controls the whole device
> through the video device nodes. This limits the possible use cases as it
> can't support sources that are more complex than a single subdev. To
> support more complex hardware pipelines, implement support for the media
> controller centric API. The exposed API can be selected through a module
> parameter.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v1:
> 
> - Fix parameter name and description in MODULE_PARM_DESC()
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c |   1 +
>  drivers/media/platform/ti-vpe/cal-video.c    | 356 ++++++++++++++-----
>  drivers/media/platform/ti-vpe/cal.c          |  10 +-
>  drivers/media/platform/ti-vpe/cal.h          |   2 +
>  4 files changed, 288 insertions(+), 81 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index 1920f36137b8..8abf887c47d6 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -830,6 +830,7 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	sd = &phy->subdev;
>  	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
>  	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
>  	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
>  	sd->dev = cal->dev;
>  
> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> index 779f1e1bc529..119a17fb97bc 100644
> --- a/drivers/media/platform/ti-vpe/cal-video.c
> +++ b/drivers/media/platform/ti-vpe/cal-video.c
> @@ -40,40 +40,10 @@ static char *fourcc_to_str(u32 fmt)
>  }
>  
>  /* ------------------------------------------------------------------
> - *	V4L2 Video IOCTLs
> + *	V4L2 Common IOCTLs
>   * ------------------------------------------------------------------
>   */
>  
> -static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
> -							u32 pixelformat)
> -{
> -	const struct cal_format_info *fmtinfo;
> -	unsigned int k;
> -
> -	for (k = 0; k < ctx->num_active_fmt; k++) {
> -		fmtinfo = ctx->active_fmt[k];
> -		if (fmtinfo->fourcc == pixelformat)
> -			return fmtinfo;
> -	}
> -
> -	return NULL;
> -}
> -
> -static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
> -							 u32 code)
> -{
> -	const struct cal_format_info *fmtinfo;
> -	unsigned int k;
> -
> -	for (k = 0; k < ctx->num_active_fmt; k++) {
> -		fmtinfo = ctx->active_fmt[k];
> -		if (fmtinfo->code == code)
> -			return fmtinfo;
> -	}
> -
> -	return NULL;
> -}
> -
>  static int cal_querycap(struct file *file, void *priv,
>  			struct v4l2_capability *cap)
>  {
> @@ -87,6 +57,51 @@ static int cal_querycap(struct file *file, void *priv,
>  	return 0;
>  }
>  
> +static int cal_g_fmt_vid_cap(struct file *file, void *priv,
> +			     struct v4l2_format *f)
> +{
> +	struct cal_ctx *ctx = video_drvdata(file);
> +
> +	*f = ctx->v_fmt;
> +
> +	return 0;
> +}
> +
> +/* ------------------------------------------------------------------
> + *	V4L2 Video Node Centric IOCTLs
> + * ------------------------------------------------------------------
> + */
> +
> +static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
> +							u32 pixelformat)
> +{
> +	const struct cal_format_info *fmtinfo;
> +	unsigned int k;
> +
> +	for (k = 0; k < ctx->num_active_fmt; k++) {
> +		fmtinfo = ctx->active_fmt[k];
> +		if (fmtinfo->fourcc == pixelformat)
> +			return fmtinfo;
> +	}
> +
> +	return NULL;
> +}
> +
> +static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
> +							 u32 code)
> +{
> +	const struct cal_format_info *fmtinfo;
> +	unsigned int k;
> +
> +	for (k = 0; k < ctx->num_active_fmt; k++) {
> +		fmtinfo = ctx->active_fmt[k];
> +		if (fmtinfo->code == code)
> +			return fmtinfo;
> +	}
> +
> +	return NULL;
> +}
> +
>  static int cal_enum_fmt_vid_cap(struct file *file, void  *priv,
>  				struct v4l2_fmtdesc *f)
>  {
> @@ -174,16 +189,6 @@ static void cal_calc_format_size(struct cal_ctx *ctx,
>  		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
>  }
>  
> -static int cal_g_fmt_vid_cap(struct file *file, void *priv,
> -			     struct v4l2_format *f)
> -{
> -	struct cal_ctx *ctx = video_drvdata(file);
> -
> -	*f = ctx->v_fmt;
> -
> -	return 0;
> -}
> -
>  static int cal_try_fmt_vid_cap(struct file *file, void *priv,
>  			       struct v4l2_format *f)
>  {
> @@ -383,16 +388,7 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
>  	return 0;
>  }
>  
> -static const struct v4l2_file_operations cal_fops = {
> -	.owner		= THIS_MODULE,
> -	.open           = v4l2_fh_open,
> -	.release        = vb2_fop_release,
> -	.poll		= vb2_fop_poll,
> -	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
> -	.mmap           = vb2_fop_mmap,
> -};
> -
> -static const struct v4l2_ioctl_ops cal_ioctl_ops = {
> +static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
>  	.vidioc_querycap      = cal_querycap,
>  	.vidioc_enum_fmt_vid_cap  = cal_enum_fmt_vid_cap,
>  	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
> @@ -418,7 +414,153 @@ static const struct v4l2_ioctl_ops cal_ioctl_ops = {
>  };
>  
>  /* ------------------------------------------------------------------
> - *	videobuf2 Operations
> + *	V4L2 Media Controller Centric IOCTLs
> + * ------------------------------------------------------------------
> + */
> +
> +static int cal_mc_enum_fmt_vid_cap(struct file *file, void  *priv,
> +				   struct v4l2_fmtdesc *f)
> +{
> +	if (f->index >= cal_num_formats)
> +		return -EINVAL;
> +
> +	f->pixelformat = cal_formats[f->index].fourcc;
> +	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +
> +	return 0;
> +}
> +
> +static void cal_mc_try_fmt(struct cal_ctx *ctx, struct v4l2_format *f,
> +			   const struct cal_format_info **info)
> +{
> +	struct v4l2_pix_format *format = &f->fmt.pix;
> +	const struct cal_format_info *fmtinfo;
> +	unsigned int bpp;
> +
> +	/*
> +	 * Default to the first format is the requested pixel format code isn't
> +	 * supported.
> +	 */
> +	fmtinfo = cal_format_by_fourcc(f->fmt.pix.pixelformat);
> +	if (!fmtinfo)
> +		fmtinfo = &cal_formats[0];
> +
> +	/*
> +	 * Clamp the size, update the pixel format. The field and colorspace are
> +	 * accepted as-is, except for V4L2_FIELD_ANY that is turned into
> +	 * V4L2_FIELD_NONE.
> +	 */
> +	bpp = ALIGN(fmtinfo->bpp, 8);
> +
> +	format->width = clamp_t(unsigned int, format->width,
> +				CAL_MIN_WIDTH_BYTES * 8 / bpp,
> +				CAL_MAX_WIDTH_BYTES * 8 / bpp);
> +	format->height = clamp_t(unsigned int, format->height,
> +				 CAL_MIN_HEIGHT_LINES, CAL_MAX_HEIGHT_LINES);
> +	format->pixelformat = fmtinfo->fourcc;
> +
> +	if (format->field == V4L2_FIELD_ANY)
> +		format->field = V4L2_FIELD_NONE;
> +
> +	/*
> +	 * Calculate the number of bytes per line and the image size. The
> +	 * hardware stores the stride as a number of 16 bytes words, in a
> +	 * signed 15-bit value. Only 14 bits are thus usable.
> +	 */
> +	format->bytesperline = ALIGN(clamp(format->bytesperline,
> +					   format->width * bpp / 8,
> +					   ((1U << 14) - 1) * 16), 16);
> +
> +	format->sizeimage = format->height * format->bytesperline;
> +
> +	if (info)
> +		*info = fmtinfo;
> +
> +	ctx_dbg(3, ctx, "%s: %s %ux%u (bytesperline %u sizeimage %u)\n",
> +		__func__, fourcc_to_str(format->pixelformat),
> +		format->width, format->height,
> +		format->bytesperline, format->sizeimage);
> +}
> +
> +static int cal_mc_try_fmt_vid_cap(struct file *file, void *priv,
> +				  struct v4l2_format *f)
> +{
> +	struct cal_ctx *ctx = video_drvdata(file);
> +
> +	cal_mc_try_fmt(ctx, f, NULL);
> +	return 0;
> +}
> +
> +static int cal_mc_s_fmt_vid_cap(struct file *file, void *priv,
> +				struct v4l2_format *f)
> +{
> +	struct cal_ctx *ctx = video_drvdata(file);
> +	const struct cal_format_info *fmtinfo;
> +
> +	if (vb2_is_busy(&ctx->vb_vidq)) {
> +		ctx_dbg(3, ctx, "%s device busy\n", __func__);
> +		return -EBUSY;
> +	}
> +
> +	cal_mc_try_fmt(ctx, f, &fmtinfo);
> +
> +	ctx->v_fmt = *f;
> +	ctx->fmtinfo = fmtinfo;
> +
> +	return 0;
> +}
> +
> +static int cal_mc_enum_framesizes(struct file *file, void *fh,
> +				  struct v4l2_frmsizeenum *fsize)
> +{
> +	struct cal_ctx *ctx = video_drvdata(file);
> +	const struct cal_format_info *fmtinfo;
> +	unsigned int bpp;
> +
> +	if (fsize->index > 0)
> +		return -EINVAL;
> +
> +	fmtinfo = cal_format_by_fourcc(fsize->pixel_format);
> +	if (!fmtinfo) {
> +		ctx_dbg(3, ctx, "Invalid pixel format 0x%08x\n",
> +			fsize->pixel_format);
> +		return -EINVAL;
> +	}
> +
> +	bpp = ALIGN(fmtinfo->bpp, 8);
> +
> +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +	fsize->stepwise.min_width = CAL_MIN_WIDTH_BYTES * 8 / bpp;
> +	fsize->stepwise.max_width = CAL_MAX_WIDTH_BYTES * 8 / bpp;
> +	fsize->stepwise.step_width = 64 / bpp;
> +	fsize->stepwise.min_height = CAL_MIN_HEIGHT_LINES;
> +	fsize->stepwise.max_height = CAL_MAX_HEIGHT_LINES;
> +	fsize->stepwise.step_height = 1;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops cal_ioctl_mc_ops = {
> +	.vidioc_querycap      = cal_querycap,
> +	.vidioc_enum_fmt_vid_cap  = cal_mc_enum_fmt_vid_cap,
> +	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
> +	.vidioc_try_fmt_vid_cap   = cal_mc_try_fmt_vid_cap,
> +	.vidioc_s_fmt_vid_cap     = cal_mc_s_fmt_vid_cap,
> +	.vidioc_enum_framesizes   = cal_mc_enum_framesizes,
> +	.vidioc_reqbufs       = vb2_ioctl_reqbufs,
> +	.vidioc_create_bufs   = vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf   = vb2_ioctl_prepare_buf,
> +	.vidioc_querybuf      = vb2_ioctl_querybuf,
> +	.vidioc_qbuf          = vb2_ioctl_qbuf,
> +	.vidioc_dqbuf         = vb2_ioctl_dqbuf,
> +	.vidioc_expbuf        = vb2_ioctl_expbuf,
> +	.vidioc_streamon      = vb2_ioctl_streamon,
> +	.vidioc_streamoff     = vb2_ioctl_streamoff,
> +	.vidioc_log_status    = v4l2_ctrl_log_status,
> +};
> +
> +/* ------------------------------------------------------------------
> + *	videobuf2 Common Operations
>   * ------------------------------------------------------------------
>   */
>  
> @@ -504,6 +646,26 @@ static void cal_release_buffers(struct cal_ctx *ctx,
>  	spin_unlock_irq(&ctx->dma.lock);
>  }
>  
> +/* ------------------------------------------------------------------
> + *	videobuf2 Operations
> + * ------------------------------------------------------------------
> + */
> +
> +static int cal_video_check_format(struct cal_ctx *ctx)
> +{
> +	const struct v4l2_mbus_framefmt *format;
> +
> +	format = &ctx->phy->formats[CAL_CAMERARX_PAD_SOURCE];
> +
> +	if (ctx->fmtinfo->code != format->code ||
> +	    ctx->v_fmt.fmt.pix.height != format->height ||
> +	    ctx->v_fmt.fmt.pix.width != format->width ||
> +	    ctx->v_fmt.fmt.pix.field != format->field)
> +		return -EPIPE;
> +
> +	return 0;
> +}
> +
>  static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>  {
>  	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
> @@ -511,6 +673,23 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>  	dma_addr_t addr;
>  	int ret;
>  
> +	ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
> +	if (ret < 0) {
> +		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
> +		return ret;
> +	}
> +
> +	/*
> +	 * Verify that the currently configured format matches the output of
> +	 * the connected CAMERARX.
> +	 */
> +	ret = cal_video_check_format(ctx);
> +	if (ret < 0) {
> +		ctx_dbg(3, ctx,
> +			"Format mismatch between CAMERARX and video node\n");
> +		goto error_pipeline;
> +	}
> +
>  	spin_lock_irq(&ctx->dma.lock);
>  	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
>  	ctx->dma.pending = buf;
> @@ -526,18 +705,21 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>  
>  	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
>  	if (ret)
> -		goto err;
> +		goto error_stop;
>  
>  	if (cal_debug >= 4)
>  		cal_quickdump_regs(ctx->cal);
>  
>  	return 0;
>  
> -err:
> +error_stop:
>  	cal_ctx_stop(ctx);
>  	pm_runtime_put_sync(ctx->cal->dev);
>  
> +error_pipeline:
> +	media_pipeline_stop(&ctx->vdev.entity);
>  	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
> +
>  	return ret;
>  }
>  
> @@ -552,6 +734,8 @@ static void cal_stop_streaming(struct vb2_queue *vq)
>  	pm_runtime_put_sync(ctx->cal->dev);
>  
>  	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
> +
> +	media_pipeline_stop(&ctx->vdev.entity);
>  }
>  
>  static const struct vb2_ops cal_video_qops = {
> @@ -569,13 +753,13 @@ static const struct vb2_ops cal_video_qops = {
>   * ------------------------------------------------------------------
>   */
>  
> -static const struct video_device cal_videodev = {
> -	.name		= CAL_MODULE_NAME,
> -	.fops		= &cal_fops,
> -	.ioctl_ops	= &cal_ioctl_ops,
> -	.minor		= -1,
> -	.release	= video_device_release_empty,
> -	.device_caps	= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
> +static const struct v4l2_file_operations cal_fops = {
> +	.owner		= THIS_MODULE,
> +	.open           = v4l2_fh_open,
> +	.release        = vb2_fop_release,
> +	.poll		= vb2_fop_poll,
> +	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
> +	.mmap           = vb2_fop_mmap,
>  };
>  
>  static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
> @@ -649,19 +833,22 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
>  
>  int cal_ctx_v4l2_register(struct cal_ctx *ctx)
>  {
> -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>  	struct video_device *vfd = &ctx->vdev;
>  	int ret;
>  
> -	ret = cal_ctx_v4l2_init_formats(ctx);
> -	if (ret)
> -		return ret;
> +	if (!cal_mc_api) {
> +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>  
> -	ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler, NULL,
> -				    true);
> -	if (ret < 0) {
> -		ctx_err(ctx, "Failed to add sensor ctrl handler\n");
> -		return ret;
> +		ret = cal_ctx_v4l2_init_formats(ctx);
> +		if (ret)
> +			return ret;
> +
> +		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler,
> +					    NULL, true);
> +		if (ret < 0) {
> +			ctx_err(ctx, "Failed to add sensor ctrl handler\n");
> +			return ret;
> +		}
>  	}
>  
>  	ret = video_register_device(vfd, VFL_TYPE_VIDEO, cal_video_nr);
> @@ -698,7 +885,6 @@ void cal_ctx_v4l2_unregister(struct cal_ctx *ctx)
>  
>  int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>  {
> -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>  	struct video_device *vfd = &ctx->vdev;
>  	struct vb2_queue *q = &ctx->vb_vidq;
>  	int ret;
> @@ -725,10 +911,14 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>  		return ret;
>  
>  	/* Initialize the video device and media entity. */
> -	*vfd = cal_videodev;
> +	vfd->fops = &cal_fops;
> +	vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING
> +			 | (cal_mc_api ? V4L2_CAP_IO_MC : 0);
>  	vfd->v4l2_dev = &ctx->cal->v4l2_dev;
>  	vfd->queue = q;
>  	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->index);
> +	vfd->release = video_device_release_empty;
> +	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_video_ops;
>  	vfd->lock = &ctx->mutex;
>  	video_set_drvdata(vfd, ctx);
>  
> @@ -737,15 +927,19 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>  	if (ret < 0)
>  		return ret;
>  
> -	/* Initialize the control handler. */
> -	ret = v4l2_ctrl_handler_init(hdl, 11);
> -	if (ret < 0) {
> -		ctx_err(ctx, "Failed to init ctrl handler\n");
> -		goto error;
> +	if (!cal_mc_api) {
> +		/* Initialize the control handler. */
> +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> +
> +		ret = v4l2_ctrl_handler_init(hdl, 11);
> +		if (ret < 0) {
> +			ctx_err(ctx, "Failed to init ctrl handler\n");
> +			goto error;
> +		}
> +
> +		vfd->ctrl_handler = hdl;
>  	}

I think it is a good idea if the probe() function will log whether this
driver is in MC or video centric mode.

>  
> -	vfd->ctrl_handler = hdl;
> -
>  	return 0;
>  
>  error:
> @@ -755,6 +949,8 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>  
>  void cal_ctx_v4l2_cleanup(struct cal_ctx *ctx)
>  {
> -	v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> +	if (!cal_mc_api)
> +		v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> +
>  	media_entity_cleanup(&ctx->vdev.entity);
>  }
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index 293cbac905b3..2ce2b6404c92 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -43,6 +43,10 @@ unsigned int cal_debug;
>  module_param_named(debug, cal_debug, uint, 0644);
>  MODULE_PARM_DESC(debug, "activates debug info");
>  
> +bool cal_mc_api;
> +module_param_named(mc_api, cal_mc_api, bool, 0444);
> +MODULE_PARM_DESC(mc_api, "activates the MC API");
> +

I noticed that this defaults to video centric.

To come back to the discussion of the v2 of this patch, I believe we
need to decide what to do here so we have a good template for future
drivers that need this.

My opinion is that you want a Kconfig option to set the default for
this, so this becomes something like this:

bool cal_mc_api = CONFIG_TI_CAL_MC_API;

What do you think?

I will make a PR for v5.12 for patches 1-23, but I would like to have this
remaining issue resolved before merging this final patch.

I do think that a Kconfig option is very desirable, but whether the default
of this option should be y or n is less clear. Since this driver has always
been video-centric I can imagine that it makes sense to set it to n. But
for e.g. a new driver like the tegra-video driver (currently in staging),
it would make sense to set it to y since it is a new driver. Ditto for the
rpi camera driver.

In that case the rule would be that for new mainline drivers the default
should always be y (MC-centric), but if the driver was already in mainline
and MC support is added (like for this driver), then the default remains n
for backwards compatibility.

Regards,

	Hans

>  /* ------------------------------------------------------------------
>   *	Format Handling
>   * ------------------------------------------------------------------
> @@ -660,13 +664,17 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>  {
>  	struct cal_dev *cal = container_of(notifier, struct cal_dev, notifier);
>  	unsigned int i;
> +	int ret = 0;
>  
>  	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
>  		if (cal->ctx[i])
>  			cal_ctx_v4l2_register(cal->ctx[i]);
>  	}
>  
> -	return 0;
> +	if (cal_mc_api)
> +		ret = v4l2_device_register_subdev_nodes(&cal->v4l2_dev);
> +
> +	return ret;
>  }
>  
>  static const struct v4l2_async_notifier_operations cal_async_notifier_ops = {
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 60f5f7480b17..199692610fb9 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -160,6 +160,7 @@ struct cal_camerarx {
>  	struct device_node	*sensor_ep_node;
>  	struct device_node	*sensor_node;
>  	struct v4l2_subdev	*sensor;
> +	struct media_pipeline	pipe;
>  
>  	struct v4l2_subdev	subdev;
>  	struct media_pad	pads[2];
> @@ -224,6 +225,7 @@ struct cal_ctx {
>  
>  extern unsigned int cal_debug;
>  extern int cal_video_nr;
> +extern bool cal_mc_api;
>  
>  #define cal_dbg(level, cal, fmt, arg...)				\
>  	do {								\
> 

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Laurent Pinchart]

Hi Hans,

On Mon, Dec 07, 2020 at 11:11:33AM +0100, Hans Verkuil wrote:
> On 07/12/2020 00:53, Laurent Pinchart wrote:
> > The CAL driver is video node centric, it controls the whole device
> > through the video device nodes. This limits the possible use cases as it
> > can't support sources that are more complex than a single subdev. To
> > support more complex hardware pipelines, implement support for the media
> > controller centric API. The exposed API can be selected through a module
> > parameter.
> > 
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v1:
> > 
> > - Fix parameter name and description in MODULE_PARM_DESC()
> > ---
> >  drivers/media/platform/ti-vpe/cal-camerarx.c |   1 +
> >  drivers/media/platform/ti-vpe/cal-video.c    | 356 ++++++++++++++-----
> >  drivers/media/platform/ti-vpe/cal.c          |  10 +-
> >  drivers/media/platform/ti-vpe/cal.h          |   2 +
> >  4 files changed, 288 insertions(+), 81 deletions(-)
> > 
> > diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> > index 1920f36137b8..8abf887c47d6 100644
> > --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> > +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> > @@ -830,6 +830,7 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
> >  	sd = &phy->subdev;
> >  	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
> >  	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> >  	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
> >  	sd->dev = cal->dev;
> >  
> > diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> > index 779f1e1bc529..119a17fb97bc 100644
> > --- a/drivers/media/platform/ti-vpe/cal-video.c
> > +++ b/drivers/media/platform/ti-vpe/cal-video.c
> > @@ -40,40 +40,10 @@ static char *fourcc_to_str(u32 fmt)
> >  }
> >  
> >  /* ------------------------------------------------------------------
> > - *	V4L2 Video IOCTLs
> > + *	V4L2 Common IOCTLs
> >   * ------------------------------------------------------------------
> >   */
> >  
> > -static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
> > -							u32 pixelformat)
> > -{
> > -	const struct cal_format_info *fmtinfo;
> > -	unsigned int k;
> > -
> > -	for (k = 0; k < ctx->num_active_fmt; k++) {
> > -		fmtinfo = ctx->active_fmt[k];
> > -		if (fmtinfo->fourcc == pixelformat)
> > -			return fmtinfo;
> > -	}
> > -
> > -	return NULL;
> > -}
> > -
> > -static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
> > -							 u32 code)
> > -{
> > -	const struct cal_format_info *fmtinfo;
> > -	unsigned int k;
> > -
> > -	for (k = 0; k < ctx->num_active_fmt; k++) {
> > -		fmtinfo = ctx->active_fmt[k];
> > -		if (fmtinfo->code == code)
> > -			return fmtinfo;
> > -	}
> > -
> > -	return NULL;
> > -}
> > -
> >  static int cal_querycap(struct file *file, void *priv,
> >  			struct v4l2_capability *cap)
> >  {
> > @@ -87,6 +57,51 @@ static int cal_querycap(struct file *file, void *priv,
> >  	return 0;
> >  }
> >  
> > +static int cal_g_fmt_vid_cap(struct file *file, void *priv,
> > +			     struct v4l2_format *f)
> > +{
> > +	struct cal_ctx *ctx = video_drvdata(file);
> > +
> > +	*f = ctx->v_fmt;
> > +
> > +	return 0;
> > +}
> > +
> > +/* ------------------------------------------------------------------
> > + *	V4L2 Video Node Centric IOCTLs
> > + * ------------------------------------------------------------------
> > + */
> > +
> > +static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
> > +							u32 pixelformat)
> > +{
> > +	const struct cal_format_info *fmtinfo;
> > +	unsigned int k;
> > +
> > +	for (k = 0; k < ctx->num_active_fmt; k++) {
> > +		fmtinfo = ctx->active_fmt[k];
> > +		if (fmtinfo->fourcc == pixelformat)
> > +			return fmtinfo;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> > +static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
> > +							 u32 code)
> > +{
> > +	const struct cal_format_info *fmtinfo;
> > +	unsigned int k;
> > +
> > +	for (k = 0; k < ctx->num_active_fmt; k++) {
> > +		fmtinfo = ctx->active_fmt[k];
> > +		if (fmtinfo->code == code)
> > +			return fmtinfo;
> > +	}
> > +
> > +	return NULL;
> > +}
> > +
> >  static int cal_enum_fmt_vid_cap(struct file *file, void  *priv,
> >  				struct v4l2_fmtdesc *f)
> >  {
> > @@ -174,16 +189,6 @@ static void cal_calc_format_size(struct cal_ctx *ctx,
> >  		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
> >  }
> >  
> > -static int cal_g_fmt_vid_cap(struct file *file, void *priv,
> > -			     struct v4l2_format *f)
> > -{
> > -	struct cal_ctx *ctx = video_drvdata(file);
> > -
> > -	*f = ctx->v_fmt;
> > -
> > -	return 0;
> > -}
> > -
> >  static int cal_try_fmt_vid_cap(struct file *file, void *priv,
> >  			       struct v4l2_format *f)
> >  {
> > @@ -383,16 +388,7 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
> >  	return 0;
> >  }
> >  
> > -static const struct v4l2_file_operations cal_fops = {
> > -	.owner		= THIS_MODULE,
> > -	.open           = v4l2_fh_open,
> > -	.release        = vb2_fop_release,
> > -	.poll		= vb2_fop_poll,
> > -	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
> > -	.mmap           = vb2_fop_mmap,
> > -};
> > -
> > -static const struct v4l2_ioctl_ops cal_ioctl_ops = {
> > +static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
> >  	.vidioc_querycap      = cal_querycap,
> >  	.vidioc_enum_fmt_vid_cap  = cal_enum_fmt_vid_cap,
> >  	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
> > @@ -418,7 +414,153 @@ static const struct v4l2_ioctl_ops cal_ioctl_ops = {
> >  };
> >  
> >  /* ------------------------------------------------------------------
> > - *	videobuf2 Operations
> > + *	V4L2 Media Controller Centric IOCTLs
> > + * ------------------------------------------------------------------
> > + */
> > +
> > +static int cal_mc_enum_fmt_vid_cap(struct file *file, void  *priv,
> > +				   struct v4l2_fmtdesc *f)
> > +{
> > +	if (f->index >= cal_num_formats)
> > +		return -EINVAL;
> > +
> > +	f->pixelformat = cal_formats[f->index].fourcc;
> > +	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > +
> > +	return 0;
> > +}
> > +
> > +static void cal_mc_try_fmt(struct cal_ctx *ctx, struct v4l2_format *f,
> > +			   const struct cal_format_info **info)
> > +{
> > +	struct v4l2_pix_format *format = &f->fmt.pix;
> > +	const struct cal_format_info *fmtinfo;
> > +	unsigned int bpp;
> > +
> > +	/*
> > +	 * Default to the first format is the requested pixel format code isn't
> > +	 * supported.
> > +	 */
> > +	fmtinfo = cal_format_by_fourcc(f->fmt.pix.pixelformat);
> > +	if (!fmtinfo)
> > +		fmtinfo = &cal_formats[0];
> > +
> > +	/*
> > +	 * Clamp the size, update the pixel format. The field and colorspace are
> > +	 * accepted as-is, except for V4L2_FIELD_ANY that is turned into
> > +	 * V4L2_FIELD_NONE.
> > +	 */
> > +	bpp = ALIGN(fmtinfo->bpp, 8);
> > +
> > +	format->width = clamp_t(unsigned int, format->width,
> > +				CAL_MIN_WIDTH_BYTES * 8 / bpp,
> > +				CAL_MAX_WIDTH_BYTES * 8 / bpp);
> > +	format->height = clamp_t(unsigned int, format->height,
> > +				 CAL_MIN_HEIGHT_LINES, CAL_MAX_HEIGHT_LINES);
> > +	format->pixelformat = fmtinfo->fourcc;
> > +
> > +	if (format->field == V4L2_FIELD_ANY)
> > +		format->field = V4L2_FIELD_NONE;
> > +
> > +	/*
> > +	 * Calculate the number of bytes per line and the image size. The
> > +	 * hardware stores the stride as a number of 16 bytes words, in a
> > +	 * signed 15-bit value. Only 14 bits are thus usable.
> > +	 */
> > +	format->bytesperline = ALIGN(clamp(format->bytesperline,
> > +					   format->width * bpp / 8,
> > +					   ((1U << 14) - 1) * 16), 16);
> > +
> > +	format->sizeimage = format->height * format->bytesperline;
> > +
> > +	if (info)
> > +		*info = fmtinfo;
> > +
> > +	ctx_dbg(3, ctx, "%s: %s %ux%u (bytesperline %u sizeimage %u)\n",
> > +		__func__, fourcc_to_str(format->pixelformat),
> > +		format->width, format->height,
> > +		format->bytesperline, format->sizeimage);
> > +}
> > +
> > +static int cal_mc_try_fmt_vid_cap(struct file *file, void *priv,
> > +				  struct v4l2_format *f)
> > +{
> > +	struct cal_ctx *ctx = video_drvdata(file);
> > +
> > +	cal_mc_try_fmt(ctx, f, NULL);
> > +	return 0;
> > +}
> > +
> > +static int cal_mc_s_fmt_vid_cap(struct file *file, void *priv,
> > +				struct v4l2_format *f)
> > +{
> > +	struct cal_ctx *ctx = video_drvdata(file);
> > +	const struct cal_format_info *fmtinfo;
> > +
> > +	if (vb2_is_busy(&ctx->vb_vidq)) {
> > +		ctx_dbg(3, ctx, "%s device busy\n", __func__);
> > +		return -EBUSY;
> > +	}
> > +
> > +	cal_mc_try_fmt(ctx, f, &fmtinfo);
> > +
> > +	ctx->v_fmt = *f;
> > +	ctx->fmtinfo = fmtinfo;
> > +
> > +	return 0;
> > +}
> > +
> > +static int cal_mc_enum_framesizes(struct file *file, void *fh,
> > +				  struct v4l2_frmsizeenum *fsize)
> > +{
> > +	struct cal_ctx *ctx = video_drvdata(file);
> > +	const struct cal_format_info *fmtinfo;
> > +	unsigned int bpp;
> > +
> > +	if (fsize->index > 0)
> > +		return -EINVAL;
> > +
> > +	fmtinfo = cal_format_by_fourcc(fsize->pixel_format);
> > +	if (!fmtinfo) {
> > +		ctx_dbg(3, ctx, "Invalid pixel format 0x%08x\n",
> > +			fsize->pixel_format);
> > +		return -EINVAL;
> > +	}
> > +
> > +	bpp = ALIGN(fmtinfo->bpp, 8);
> > +
> > +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > +	fsize->stepwise.min_width = CAL_MIN_WIDTH_BYTES * 8 / bpp;
> > +	fsize->stepwise.max_width = CAL_MAX_WIDTH_BYTES * 8 / bpp;
> > +	fsize->stepwise.step_width = 64 / bpp;
> > +	fsize->stepwise.min_height = CAL_MIN_HEIGHT_LINES;
> > +	fsize->stepwise.max_height = CAL_MAX_HEIGHT_LINES;
> > +	fsize->stepwise.step_height = 1;
> > +
> > +	return 0;
> > +}
> > +
> > +static const struct v4l2_ioctl_ops cal_ioctl_mc_ops = {
> > +	.vidioc_querycap      = cal_querycap,
> > +	.vidioc_enum_fmt_vid_cap  = cal_mc_enum_fmt_vid_cap,
> > +	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
> > +	.vidioc_try_fmt_vid_cap   = cal_mc_try_fmt_vid_cap,
> > +	.vidioc_s_fmt_vid_cap     = cal_mc_s_fmt_vid_cap,
> > +	.vidioc_enum_framesizes   = cal_mc_enum_framesizes,
> > +	.vidioc_reqbufs       = vb2_ioctl_reqbufs,
> > +	.vidioc_create_bufs   = vb2_ioctl_create_bufs,
> > +	.vidioc_prepare_buf   = vb2_ioctl_prepare_buf,
> > +	.vidioc_querybuf      = vb2_ioctl_querybuf,
> > +	.vidioc_qbuf          = vb2_ioctl_qbuf,
> > +	.vidioc_dqbuf         = vb2_ioctl_dqbuf,
> > +	.vidioc_expbuf        = vb2_ioctl_expbuf,
> > +	.vidioc_streamon      = vb2_ioctl_streamon,
> > +	.vidioc_streamoff     = vb2_ioctl_streamoff,
> > +	.vidioc_log_status    = v4l2_ctrl_log_status,
> > +};
> > +
> > +/* ------------------------------------------------------------------
> > + *	videobuf2 Common Operations
> >   * ------------------------------------------------------------------
> >   */
> >  
> > @@ -504,6 +646,26 @@ static void cal_release_buffers(struct cal_ctx *ctx,
> >  	spin_unlock_irq(&ctx->dma.lock);
> >  }
> >  
> > +/* ------------------------------------------------------------------
> > + *	videobuf2 Operations
> > + * ------------------------------------------------------------------
> > + */
> > +
> > +static int cal_video_check_format(struct cal_ctx *ctx)
> > +{
> > +	const struct v4l2_mbus_framefmt *format;
> > +
> > +	format = &ctx->phy->formats[CAL_CAMERARX_PAD_SOURCE];
> > +
> > +	if (ctx->fmtinfo->code != format->code ||
> > +	    ctx->v_fmt.fmt.pix.height != format->height ||
> > +	    ctx->v_fmt.fmt.pix.width != format->width ||
> > +	    ctx->v_fmt.fmt.pix.field != format->field)
> > +		return -EPIPE;
> > +
> > +	return 0;
> > +}
> > +
> >  static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
> >  {
> >  	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
> > @@ -511,6 +673,23 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
> >  	dma_addr_t addr;
> >  	int ret;
> >  
> > +	ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
> > +	if (ret < 0) {
> > +		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
> > +		return ret;
> > +	}
> > +
> > +	/*
> > +	 * Verify that the currently configured format matches the output of
> > +	 * the connected CAMERARX.
> > +	 */
> > +	ret = cal_video_check_format(ctx);
> > +	if (ret < 0) {
> > +		ctx_dbg(3, ctx,
> > +			"Format mismatch between CAMERARX and video node\n");
> > +		goto error_pipeline;
> > +	}
> > +
> >  	spin_lock_irq(&ctx->dma.lock);
> >  	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
> >  	ctx->dma.pending = buf;
> > @@ -526,18 +705,21 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
> >  
> >  	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
> >  	if (ret)
> > -		goto err;
> > +		goto error_stop;
> >  
> >  	if (cal_debug >= 4)
> >  		cal_quickdump_regs(ctx->cal);
> >  
> >  	return 0;
> >  
> > -err:
> > +error_stop:
> >  	cal_ctx_stop(ctx);
> >  	pm_runtime_put_sync(ctx->cal->dev);
> >  
> > +error_pipeline:
> > +	media_pipeline_stop(&ctx->vdev.entity);
> >  	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
> > +
> >  	return ret;
> >  }
> >  
> > @@ -552,6 +734,8 @@ static void cal_stop_streaming(struct vb2_queue *vq)
> >  	pm_runtime_put_sync(ctx->cal->dev);
> >  
> >  	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
> > +
> > +	media_pipeline_stop(&ctx->vdev.entity);
> >  }
> >  
> >  static const struct vb2_ops cal_video_qops = {
> > @@ -569,13 +753,13 @@ static const struct vb2_ops cal_video_qops = {
> >   * ------------------------------------------------------------------
> >   */
> >  
> > -static const struct video_device cal_videodev = {
> > -	.name		= CAL_MODULE_NAME,
> > -	.fops		= &cal_fops,
> > -	.ioctl_ops	= &cal_ioctl_ops,
> > -	.minor		= -1,
> > -	.release	= video_device_release_empty,
> > -	.device_caps	= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
> > +static const struct v4l2_file_operations cal_fops = {
> > +	.owner		= THIS_MODULE,
> > +	.open           = v4l2_fh_open,
> > +	.release        = vb2_fop_release,
> > +	.poll		= vb2_fop_poll,
> > +	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
> > +	.mmap           = vb2_fop_mmap,
> >  };
> >  
> >  static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
> > @@ -649,19 +833,22 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
> >  
> >  int cal_ctx_v4l2_register(struct cal_ctx *ctx)
> >  {
> > -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> >  	struct video_device *vfd = &ctx->vdev;
> >  	int ret;
> >  
> > -	ret = cal_ctx_v4l2_init_formats(ctx);
> > -	if (ret)
> > -		return ret;
> > +	if (!cal_mc_api) {
> > +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> >  
> > -	ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler, NULL,
> > -				    true);
> > -	if (ret < 0) {
> > -		ctx_err(ctx, "Failed to add sensor ctrl handler\n");
> > -		return ret;
> > +		ret = cal_ctx_v4l2_init_formats(ctx);
> > +		if (ret)
> > +			return ret;
> > +
> > +		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler,
> > +					    NULL, true);
> > +		if (ret < 0) {
> > +			ctx_err(ctx, "Failed to add sensor ctrl handler\n");
> > +			return ret;
> > +		}
> >  	}
> >  
> >  	ret = video_register_device(vfd, VFL_TYPE_VIDEO, cal_video_nr);
> > @@ -698,7 +885,6 @@ void cal_ctx_v4l2_unregister(struct cal_ctx *ctx)
> >  
> >  int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >  {
> > -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> >  	struct video_device *vfd = &ctx->vdev;
> >  	struct vb2_queue *q = &ctx->vb_vidq;
> >  	int ret;
> > @@ -725,10 +911,14 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >  		return ret;
> >  
> >  	/* Initialize the video device and media entity. */
> > -	*vfd = cal_videodev;
> > +	vfd->fops = &cal_fops;
> > +	vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING
> > +			 | (cal_mc_api ? V4L2_CAP_IO_MC : 0);
> >  	vfd->v4l2_dev = &ctx->cal->v4l2_dev;
> >  	vfd->queue = q;
> >  	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->index);
> > +	vfd->release = video_device_release_empty;
> > +	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_video_ops;
> >  	vfd->lock = &ctx->mutex;
> >  	video_set_drvdata(vfd, ctx);
> >  
> > @@ -737,15 +927,19 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >  	if (ret < 0)
> >  		return ret;
> >  
> > -	/* Initialize the control handler. */
> > -	ret = v4l2_ctrl_handler_init(hdl, 11);
> > -	if (ret < 0) {
> > -		ctx_err(ctx, "Failed to init ctrl handler\n");
> > -		goto error;
> > +	if (!cal_mc_api) {
> > +		/* Initialize the control handler. */
> > +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> > +
> > +		ret = v4l2_ctrl_handler_init(hdl, 11);
> > +		if (ret < 0) {
> > +			ctx_err(ctx, "Failed to init ctrl handler\n");
> > +			goto error;
> > +		}
> > +
> > +		vfd->ctrl_handler = hdl;
> >  	}
> 
> I think it is a good idea if the probe() function will log whether this
> driver is in MC or video centric mode.

Would a debug message be fine ? I usually try to make the probe function
silent by default to avoid adding to the already large boot time log.

> >  
> > -	vfd->ctrl_handler = hdl;
> > -
> >  	return 0;
> >  
> >  error:
> > @@ -755,6 +949,8 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >  
> >  void cal_ctx_v4l2_cleanup(struct cal_ctx *ctx)
> >  {
> > -	v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> > +	if (!cal_mc_api)
> > +		v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> > +
> >  	media_entity_cleanup(&ctx->vdev.entity);
> >  }
> > diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> > index 293cbac905b3..2ce2b6404c92 100644
> > --- a/drivers/media/platform/ti-vpe/cal.c
> > +++ b/drivers/media/platform/ti-vpe/cal.c
> > @@ -43,6 +43,10 @@ unsigned int cal_debug;
> >  module_param_named(debug, cal_debug, uint, 0644);
> >  MODULE_PARM_DESC(debug, "activates debug info");
> >  
> > +bool cal_mc_api;
> > +module_param_named(mc_api, cal_mc_api, bool, 0444);
> > +MODULE_PARM_DESC(mc_api, "activates the MC API");
> > +
> 
> I noticed that this defaults to video centric.
> 
> To come back to the discussion of the v2 of this patch, I believe we
> need to decide what to do here so we have a good template for future
> drivers that need this.
> 
> My opinion is that you want a Kconfig option to set the default for
> this, so this becomes something like this:
> 
> bool cal_mc_api = CONFIG_TI_CAL_MC_API;
> 
> What do you think?
> 
> I will make a PR for v5.12 for patches 1-23, but I would like to have this
> remaining issue resolved before merging this final patch.
> 
> I do think that a Kconfig option is very desirable, but whether the default
> of this option should be y or n is less clear. Since this driver has always
> been video-centric I can imagine that it makes sense to set it to n. But
> for e.g. a new driver like the tegra-video driver (currently in staging),
> it would make sense to set it to y since it is a new driver. Ditto for the
> rpi camera driver.

For this driver I think video-centric mode is the best default to start
with, to avoid changing the behaviour all of a sudden. We can switch it
to MC-centric by default later if desired, after userspace gets a chance
to adapt.

> In that case the rule would be that for new mainline drivers the default
> should always be y (MC-centric), but if the driver was already in mainline
> and MC support is added (like for this driver), then the default remains n
> for backwards compatibility.

I think that for new drivers we shouldn't support video-centric mode at
all. It should only be supported in downstream (vendor) kernels, and
only if backward compatibility with existing userspace needs to be
ensured. The unicam driver, for instance, fits in that category. Drivers
whose development is ongoing (or hasn't started) should only use the MC
API. Whether the option should be y or n by default would then be a
vendor decision, it wouldn't affect upstream.

> >  /* ------------------------------------------------------------------
> >   *	Format Handling
> >   * ------------------------------------------------------------------
> > @@ -660,13 +664,17 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
> >  {
> >  	struct cal_dev *cal = container_of(notifier, struct cal_dev, notifier);
> >  	unsigned int i;
> > +	int ret = 0;
> >  
> >  	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> >  		if (cal->ctx[i])
> >  			cal_ctx_v4l2_register(cal->ctx[i]);
> >  	}
> >  
> > -	return 0;
> > +	if (cal_mc_api)
> > +		ret = v4l2_device_register_subdev_nodes(&cal->v4l2_dev);
> > +
> > +	return ret;
> >  }
> >  
> >  static const struct v4l2_async_notifier_operations cal_async_notifier_ops = {
> > diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> > index 60f5f7480b17..199692610fb9 100644
> > --- a/drivers/media/platform/ti-vpe/cal.h
> > +++ b/drivers/media/platform/ti-vpe/cal.h
> > @@ -160,6 +160,7 @@ struct cal_camerarx {
> >  	struct device_node	*sensor_ep_node;
> >  	struct device_node	*sensor_node;
> >  	struct v4l2_subdev	*sensor;
> > +	struct media_pipeline	pipe;
> >  
> >  	struct v4l2_subdev	subdev;
> >  	struct media_pad	pads[2];
> > @@ -224,6 +225,7 @@ struct cal_ctx {
> >  
> >  extern unsigned int cal_debug;
> >  extern int cal_video_nr;
> > +extern bool cal_mc_api;
> >  
> >  #define cal_dbg(level, cal, fmt, arg...)				\
> >  	do {								\

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Hans Verkuil]

On 08/12/2020 00:51, Laurent Pinchart wrote:
> Hi Hans,
> 
> On Mon, Dec 07, 2020 at 11:11:33AM +0100, Hans Verkuil wrote:
>> On 07/12/2020 00:53, Laurent Pinchart wrote:
>>> The CAL driver is video node centric, it controls the whole device
>>> through the video device nodes. This limits the possible use cases as it
>>> can't support sources that are more complex than a single subdev. To
>>> support more complex hardware pipelines, implement support for the media
>>> controller centric API. The exposed API can be selected through a module
>>> parameter.
>>>
>>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>> ---
>>> Changes since v1:
>>>
>>> - Fix parameter name and description in MODULE_PARM_DESC()
>>> ---
>>>  drivers/media/platform/ti-vpe/cal-camerarx.c |   1 +
>>>  drivers/media/platform/ti-vpe/cal-video.c    | 356 ++++++++++++++-----
>>>  drivers/media/platform/ti-vpe/cal.c          |  10 +-
>>>  drivers/media/platform/ti-vpe/cal.h          |   2 +
>>>  4 files changed, 288 insertions(+), 81 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
>>> index 1920f36137b8..8abf887c47d6 100644
>>> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
>>> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
>>> @@ -830,6 +830,7 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>>>  	sd = &phy->subdev;
>>>  	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
>>>  	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
>>> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
>>>  	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
>>>  	sd->dev = cal->dev;
>>>  
>>> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
>>> index 779f1e1bc529..119a17fb97bc 100644
>>> --- a/drivers/media/platform/ti-vpe/cal-video.c
>>> +++ b/drivers/media/platform/ti-vpe/cal-video.c
>>> @@ -40,40 +40,10 @@ static char *fourcc_to_str(u32 fmt)
>>>  }
>>>  
>>>  /* ------------------------------------------------------------------
>>> - *	V4L2 Video IOCTLs
>>> + *	V4L2 Common IOCTLs
>>>   * ------------------------------------------------------------------
>>>   */
>>>  
>>> -static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
>>> -							u32 pixelformat)
>>> -{
>>> -	const struct cal_format_info *fmtinfo;
>>> -	unsigned int k;
>>> -
>>> -	for (k = 0; k < ctx->num_active_fmt; k++) {
>>> -		fmtinfo = ctx->active_fmt[k];
>>> -		if (fmtinfo->fourcc == pixelformat)
>>> -			return fmtinfo;
>>> -	}
>>> -
>>> -	return NULL;
>>> -}
>>> -
>>> -static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
>>> -							 u32 code)
>>> -{
>>> -	const struct cal_format_info *fmtinfo;
>>> -	unsigned int k;
>>> -
>>> -	for (k = 0; k < ctx->num_active_fmt; k++) {
>>> -		fmtinfo = ctx->active_fmt[k];
>>> -		if (fmtinfo->code == code)
>>> -			return fmtinfo;
>>> -	}
>>> -
>>> -	return NULL;
>>> -}
>>> -
>>>  static int cal_querycap(struct file *file, void *priv,
>>>  			struct v4l2_capability *cap)
>>>  {
>>> @@ -87,6 +57,51 @@ static int cal_querycap(struct file *file, void *priv,
>>>  	return 0;
>>>  }
>>>  
>>> +static int cal_g_fmt_vid_cap(struct file *file, void *priv,
>>> +			     struct v4l2_format *f)
>>> +{
>>> +	struct cal_ctx *ctx = video_drvdata(file);
>>> +
>>> +	*f = ctx->v_fmt;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +/* ------------------------------------------------------------------
>>> + *	V4L2 Video Node Centric IOCTLs
>>> + * ------------------------------------------------------------------
>>> + */
>>> +
>>> +static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
>>> +							u32 pixelformat)
>>> +{
>>> +	const struct cal_format_info *fmtinfo;
>>> +	unsigned int k;
>>> +
>>> +	for (k = 0; k < ctx->num_active_fmt; k++) {
>>> +		fmtinfo = ctx->active_fmt[k];
>>> +		if (fmtinfo->fourcc == pixelformat)
>>> +			return fmtinfo;
>>> +	}
>>> +
>>> +	return NULL;
>>> +}
>>> +
>>> +static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
>>> +							 u32 code)
>>> +{
>>> +	const struct cal_format_info *fmtinfo;
>>> +	unsigned int k;
>>> +
>>> +	for (k = 0; k < ctx->num_active_fmt; k++) {
>>> +		fmtinfo = ctx->active_fmt[k];
>>> +		if (fmtinfo->code == code)
>>> +			return fmtinfo;
>>> +	}
>>> +
>>> +	return NULL;
>>> +}
>>> +
>>>  static int cal_enum_fmt_vid_cap(struct file *file, void  *priv,
>>>  				struct v4l2_fmtdesc *f)
>>>  {
>>> @@ -174,16 +189,6 @@ static void cal_calc_format_size(struct cal_ctx *ctx,
>>>  		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
>>>  }
>>>  
>>> -static int cal_g_fmt_vid_cap(struct file *file, void *priv,
>>> -			     struct v4l2_format *f)
>>> -{
>>> -	struct cal_ctx *ctx = video_drvdata(file);
>>> -
>>> -	*f = ctx->v_fmt;
>>> -
>>> -	return 0;
>>> -}
>>> -
>>>  static int cal_try_fmt_vid_cap(struct file *file, void *priv,
>>>  			       struct v4l2_format *f)
>>>  {
>>> @@ -383,16 +388,7 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
>>>  	return 0;
>>>  }
>>>  
>>> -static const struct v4l2_file_operations cal_fops = {
>>> -	.owner		= THIS_MODULE,
>>> -	.open           = v4l2_fh_open,
>>> -	.release        = vb2_fop_release,
>>> -	.poll		= vb2_fop_poll,
>>> -	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
>>> -	.mmap           = vb2_fop_mmap,
>>> -};
>>> -
>>> -static const struct v4l2_ioctl_ops cal_ioctl_ops = {
>>> +static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
>>>  	.vidioc_querycap      = cal_querycap,
>>>  	.vidioc_enum_fmt_vid_cap  = cal_enum_fmt_vid_cap,
>>>  	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
>>> @@ -418,7 +414,153 @@ static const struct v4l2_ioctl_ops cal_ioctl_ops = {
>>>  };
>>>  
>>>  /* ------------------------------------------------------------------
>>> - *	videobuf2 Operations
>>> + *	V4L2 Media Controller Centric IOCTLs
>>> + * ------------------------------------------------------------------
>>> + */
>>> +
>>> +static int cal_mc_enum_fmt_vid_cap(struct file *file, void  *priv,
>>> +				   struct v4l2_fmtdesc *f)
>>> +{
>>> +	if (f->index >= cal_num_formats)
>>> +		return -EINVAL;
>>> +
>>> +	f->pixelformat = cal_formats[f->index].fourcc;
>>> +	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static void cal_mc_try_fmt(struct cal_ctx *ctx, struct v4l2_format *f,
>>> +			   const struct cal_format_info **info)
>>> +{
>>> +	struct v4l2_pix_format *format = &f->fmt.pix;
>>> +	const struct cal_format_info *fmtinfo;
>>> +	unsigned int bpp;
>>> +
>>> +	/*
>>> +	 * Default to the first format is the requested pixel format code isn't
>>> +	 * supported.
>>> +	 */
>>> +	fmtinfo = cal_format_by_fourcc(f->fmt.pix.pixelformat);
>>> +	if (!fmtinfo)
>>> +		fmtinfo = &cal_formats[0];
>>> +
>>> +	/*
>>> +	 * Clamp the size, update the pixel format. The field and colorspace are
>>> +	 * accepted as-is, except for V4L2_FIELD_ANY that is turned into
>>> +	 * V4L2_FIELD_NONE.
>>> +	 */
>>> +	bpp = ALIGN(fmtinfo->bpp, 8);
>>> +
>>> +	format->width = clamp_t(unsigned int, format->width,
>>> +				CAL_MIN_WIDTH_BYTES * 8 / bpp,
>>> +				CAL_MAX_WIDTH_BYTES * 8 / bpp);
>>> +	format->height = clamp_t(unsigned int, format->height,
>>> +				 CAL_MIN_HEIGHT_LINES, CAL_MAX_HEIGHT_LINES);
>>> +	format->pixelformat = fmtinfo->fourcc;
>>> +
>>> +	if (format->field == V4L2_FIELD_ANY)
>>> +		format->field = V4L2_FIELD_NONE;
>>> +
>>> +	/*
>>> +	 * Calculate the number of bytes per line and the image size. The
>>> +	 * hardware stores the stride as a number of 16 bytes words, in a
>>> +	 * signed 15-bit value. Only 14 bits are thus usable.
>>> +	 */
>>> +	format->bytesperline = ALIGN(clamp(format->bytesperline,
>>> +					   format->width * bpp / 8,
>>> +					   ((1U << 14) - 1) * 16), 16);
>>> +
>>> +	format->sizeimage = format->height * format->bytesperline;
>>> +
>>> +	if (info)
>>> +		*info = fmtinfo;
>>> +
>>> +	ctx_dbg(3, ctx, "%s: %s %ux%u (bytesperline %u sizeimage %u)\n",
>>> +		__func__, fourcc_to_str(format->pixelformat),
>>> +		format->width, format->height,
>>> +		format->bytesperline, format->sizeimage);
>>> +}
>>> +
>>> +static int cal_mc_try_fmt_vid_cap(struct file *file, void *priv,
>>> +				  struct v4l2_format *f)
>>> +{
>>> +	struct cal_ctx *ctx = video_drvdata(file);
>>> +
>>> +	cal_mc_try_fmt(ctx, f, NULL);
>>> +	return 0;
>>> +}
>>> +
>>> +static int cal_mc_s_fmt_vid_cap(struct file *file, void *priv,
>>> +				struct v4l2_format *f)
>>> +{
>>> +	struct cal_ctx *ctx = video_drvdata(file);
>>> +	const struct cal_format_info *fmtinfo;
>>> +
>>> +	if (vb2_is_busy(&ctx->vb_vidq)) {
>>> +		ctx_dbg(3, ctx, "%s device busy\n", __func__);
>>> +		return -EBUSY;
>>> +	}
>>> +
>>> +	cal_mc_try_fmt(ctx, f, &fmtinfo);
>>> +
>>> +	ctx->v_fmt = *f;
>>> +	ctx->fmtinfo = fmtinfo;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static int cal_mc_enum_framesizes(struct file *file, void *fh,
>>> +				  struct v4l2_frmsizeenum *fsize)
>>> +{
>>> +	struct cal_ctx *ctx = video_drvdata(file);
>>> +	const struct cal_format_info *fmtinfo;
>>> +	unsigned int bpp;
>>> +
>>> +	if (fsize->index > 0)
>>> +		return -EINVAL;
>>> +
>>> +	fmtinfo = cal_format_by_fourcc(fsize->pixel_format);
>>> +	if (!fmtinfo) {
>>> +		ctx_dbg(3, ctx, "Invalid pixel format 0x%08x\n",
>>> +			fsize->pixel_format);
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	bpp = ALIGN(fmtinfo->bpp, 8);
>>> +
>>> +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
>>> +	fsize->stepwise.min_width = CAL_MIN_WIDTH_BYTES * 8 / bpp;
>>> +	fsize->stepwise.max_width = CAL_MAX_WIDTH_BYTES * 8 / bpp;
>>> +	fsize->stepwise.step_width = 64 / bpp;
>>> +	fsize->stepwise.min_height = CAL_MIN_HEIGHT_LINES;
>>> +	fsize->stepwise.max_height = CAL_MAX_HEIGHT_LINES;
>>> +	fsize->stepwise.step_height = 1;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>> +static const struct v4l2_ioctl_ops cal_ioctl_mc_ops = {
>>> +	.vidioc_querycap      = cal_querycap,
>>> +	.vidioc_enum_fmt_vid_cap  = cal_mc_enum_fmt_vid_cap,
>>> +	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
>>> +	.vidioc_try_fmt_vid_cap   = cal_mc_try_fmt_vid_cap,
>>> +	.vidioc_s_fmt_vid_cap     = cal_mc_s_fmt_vid_cap,
>>> +	.vidioc_enum_framesizes   = cal_mc_enum_framesizes,
>>> +	.vidioc_reqbufs       = vb2_ioctl_reqbufs,
>>> +	.vidioc_create_bufs   = vb2_ioctl_create_bufs,
>>> +	.vidioc_prepare_buf   = vb2_ioctl_prepare_buf,
>>> +	.vidioc_querybuf      = vb2_ioctl_querybuf,
>>> +	.vidioc_qbuf          = vb2_ioctl_qbuf,
>>> +	.vidioc_dqbuf         = vb2_ioctl_dqbuf,
>>> +	.vidioc_expbuf        = vb2_ioctl_expbuf,
>>> +	.vidioc_streamon      = vb2_ioctl_streamon,
>>> +	.vidioc_streamoff     = vb2_ioctl_streamoff,
>>> +	.vidioc_log_status    = v4l2_ctrl_log_status,
>>> +};
>>> +
>>> +/* ------------------------------------------------------------------
>>> + *	videobuf2 Common Operations
>>>   * ------------------------------------------------------------------
>>>   */
>>>  
>>> @@ -504,6 +646,26 @@ static void cal_release_buffers(struct cal_ctx *ctx,
>>>  	spin_unlock_irq(&ctx->dma.lock);
>>>  }
>>>  
>>> +/* ------------------------------------------------------------------
>>> + *	videobuf2 Operations
>>> + * ------------------------------------------------------------------
>>> + */
>>> +
>>> +static int cal_video_check_format(struct cal_ctx *ctx)
>>> +{
>>> +	const struct v4l2_mbus_framefmt *format;
>>> +
>>> +	format = &ctx->phy->formats[CAL_CAMERARX_PAD_SOURCE];
>>> +
>>> +	if (ctx->fmtinfo->code != format->code ||
>>> +	    ctx->v_fmt.fmt.pix.height != format->height ||
>>> +	    ctx->v_fmt.fmt.pix.width != format->width ||
>>> +	    ctx->v_fmt.fmt.pix.field != format->field)
>>> +		return -EPIPE;
>>> +
>>> +	return 0;
>>> +}
>>> +
>>>  static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>>>  {
>>>  	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
>>> @@ -511,6 +673,23 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>>>  	dma_addr_t addr;
>>>  	int ret;
>>>  
>>> +	ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
>>> +	if (ret < 0) {
>>> +		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
>>> +		return ret;
>>> +	}
>>> +
>>> +	/*
>>> +	 * Verify that the currently configured format matches the output of
>>> +	 * the connected CAMERARX.
>>> +	 */
>>> +	ret = cal_video_check_format(ctx);
>>> +	if (ret < 0) {
>>> +		ctx_dbg(3, ctx,
>>> +			"Format mismatch between CAMERARX and video node\n");
>>> +		goto error_pipeline;
>>> +	}
>>> +
>>>  	spin_lock_irq(&ctx->dma.lock);
>>>  	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
>>>  	ctx->dma.pending = buf;
>>> @@ -526,18 +705,21 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>>>  
>>>  	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
>>>  	if (ret)
>>> -		goto err;
>>> +		goto error_stop;
>>>  
>>>  	if (cal_debug >= 4)
>>>  		cal_quickdump_regs(ctx->cal);
>>>  
>>>  	return 0;
>>>  
>>> -err:
>>> +error_stop:
>>>  	cal_ctx_stop(ctx);
>>>  	pm_runtime_put_sync(ctx->cal->dev);
>>>  
>>> +error_pipeline:
>>> +	media_pipeline_stop(&ctx->vdev.entity);
>>>  	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
>>> +
>>>  	return ret;
>>>  }
>>>  
>>> @@ -552,6 +734,8 @@ static void cal_stop_streaming(struct vb2_queue *vq)
>>>  	pm_runtime_put_sync(ctx->cal->dev);
>>>  
>>>  	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
>>> +
>>> +	media_pipeline_stop(&ctx->vdev.entity);
>>>  }
>>>  
>>>  static const struct vb2_ops cal_video_qops = {
>>> @@ -569,13 +753,13 @@ static const struct vb2_ops cal_video_qops = {
>>>   * ------------------------------------------------------------------
>>>   */
>>>  
>>> -static const struct video_device cal_videodev = {
>>> -	.name		= CAL_MODULE_NAME,
>>> -	.fops		= &cal_fops,
>>> -	.ioctl_ops	= &cal_ioctl_ops,
>>> -	.minor		= -1,
>>> -	.release	= video_device_release_empty,
>>> -	.device_caps	= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
>>> +static const struct v4l2_file_operations cal_fops = {
>>> +	.owner		= THIS_MODULE,
>>> +	.open           = v4l2_fh_open,
>>> +	.release        = vb2_fop_release,
>>> +	.poll		= vb2_fop_poll,
>>> +	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
>>> +	.mmap           = vb2_fop_mmap,
>>>  };
>>>  
>>>  static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
>>> @@ -649,19 +833,22 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
>>>  
>>>  int cal_ctx_v4l2_register(struct cal_ctx *ctx)
>>>  {
>>> -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>>>  	struct video_device *vfd = &ctx->vdev;
>>>  	int ret;
>>>  
>>> -	ret = cal_ctx_v4l2_init_formats(ctx);
>>> -	if (ret)
>>> -		return ret;
>>> +	if (!cal_mc_api) {
>>> +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>>>  
>>> -	ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler, NULL,
>>> -				    true);
>>> -	if (ret < 0) {
>>> -		ctx_err(ctx, "Failed to add sensor ctrl handler\n");
>>> -		return ret;
>>> +		ret = cal_ctx_v4l2_init_formats(ctx);
>>> +		if (ret)
>>> +			return ret;
>>> +
>>> +		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler,
>>> +					    NULL, true);
>>> +		if (ret < 0) {
>>> +			ctx_err(ctx, "Failed to add sensor ctrl handler\n");
>>> +			return ret;
>>> +		}
>>>  	}
>>>  
>>>  	ret = video_register_device(vfd, VFL_TYPE_VIDEO, cal_video_nr);
>>> @@ -698,7 +885,6 @@ void cal_ctx_v4l2_unregister(struct cal_ctx *ctx)
>>>  
>>>  int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>>>  {
>>> -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>>>  	struct video_device *vfd = &ctx->vdev;
>>>  	struct vb2_queue *q = &ctx->vb_vidq;
>>>  	int ret;
>>> @@ -725,10 +911,14 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>>>  		return ret;
>>>  
>>>  	/* Initialize the video device and media entity. */
>>> -	*vfd = cal_videodev;
>>> +	vfd->fops = &cal_fops;
>>> +	vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING
>>> +			 | (cal_mc_api ? V4L2_CAP_IO_MC : 0);
>>>  	vfd->v4l2_dev = &ctx->cal->v4l2_dev;
>>>  	vfd->queue = q;
>>>  	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->index);
>>> +	vfd->release = video_device_release_empty;
>>> +	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_video_ops;
>>>  	vfd->lock = &ctx->mutex;
>>>  	video_set_drvdata(vfd, ctx);
>>>  
>>> @@ -737,15 +927,19 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>>>  	if (ret < 0)
>>>  		return ret;
>>>  
>>> -	/* Initialize the control handler. */
>>> -	ret = v4l2_ctrl_handler_init(hdl, 11);
>>> -	if (ret < 0) {
>>> -		ctx_err(ctx, "Failed to init ctrl handler\n");
>>> -		goto error;
>>> +	if (!cal_mc_api) {
>>> +		/* Initialize the control handler. */
>>> +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>>> +
>>> +		ret = v4l2_ctrl_handler_init(hdl, 11);
>>> +		if (ret < 0) {
>>> +			ctx_err(ctx, "Failed to init ctrl handler\n");
>>> +			goto error;
>>> +		}
>>> +
>>> +		vfd->ctrl_handler = hdl;
>>>  	}
>>
>> I think it is a good idea if the probe() function will log whether this
>> driver is in MC or video centric mode.
> 
> Would a debug message be fine ? I usually try to make the probe function
> silent by default to avoid adding to the already large boot time log.

I now realized that you set V4L2_CAP_IO_MC in the MC case, and that's easy
to check with v4l2-ctl -D, so userspace does have an easy method to see
whether or not it is MC-centric. So just ignore my comment, the code is fine
as it is now.

> 
>>>  
>>> -	vfd->ctrl_handler = hdl;
>>> -
>>>  	return 0;
>>>  
>>>  error:
>>> @@ -755,6 +949,8 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
>>>  
>>>  void cal_ctx_v4l2_cleanup(struct cal_ctx *ctx)
>>>  {
>>> -	v4l2_ctrl_handler_free(&ctx->ctrl_handler);
>>> +	if (!cal_mc_api)
>>> +		v4l2_ctrl_handler_free(&ctx->ctrl_handler);
>>> +
>>>  	media_entity_cleanup(&ctx->vdev.entity);
>>>  }
>>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>>> index 293cbac905b3..2ce2b6404c92 100644
>>> --- a/drivers/media/platform/ti-vpe/cal.c
>>> +++ b/drivers/media/platform/ti-vpe/cal.c
>>> @@ -43,6 +43,10 @@ unsigned int cal_debug;
>>>  module_param_named(debug, cal_debug, uint, 0644);
>>>  MODULE_PARM_DESC(debug, "activates debug info");
>>>  
>>> +bool cal_mc_api;
>>> +module_param_named(mc_api, cal_mc_api, bool, 0444);
>>> +MODULE_PARM_DESC(mc_api, "activates the MC API");
>>> +
>>
>> I noticed that this defaults to video centric.
>>
>> To come back to the discussion of the v2 of this patch, I believe we
>> need to decide what to do here so we have a good template for future
>> drivers that need this.
>>
>> My opinion is that you want a Kconfig option to set the default for
>> this, so this becomes something like this:
>>
>> bool cal_mc_api = CONFIG_TI_CAL_MC_API;
>>
>> What do you think?
>>
>> I will make a PR for v5.12 for patches 1-23, but I would like to have this
>> remaining issue resolved before merging this final patch.
>>
>> I do think that a Kconfig option is very desirable, but whether the default
>> of this option should be y or n is less clear. Since this driver has always
>> been video-centric I can imagine that it makes sense to set it to n. But
>> for e.g. a new driver like the tegra-video driver (currently in staging),
>> it would make sense to set it to y since it is a new driver. Ditto for the
>> rpi camera driver.
> 
> For this driver I think video-centric mode is the best default to start
> with, to avoid changing the behaviour all of a sudden. We can switch it
> to MC-centric by default later if desired, after userspace gets a chance
> to adapt.

Fair enough.

> 
>> In that case the rule would be that for new mainline drivers the default
>> should always be y (MC-centric), but if the driver was already in mainline
>> and MC support is added (like for this driver), then the default remains n
>> for backwards compatibility.
> 
> I think that for new drivers we shouldn't support video-centric mode at
> all. It should only be supported in downstream (vendor) kernels, and
> only if backward compatibility with existing userspace needs to be
> ensured. The unicam driver, for instance, fits in that category. Drivers
> whose development is ongoing (or hasn't started) should only use the MC
> API. Whether the option should be y or n by default would then be a
> vendor decision, it wouldn't affect upstream.

No, that I strongly disagree with. Vendors would have to carry those patches
for a long time, and if past experience is any guide, they will mess it up.
Or even refuse to upgrade to the mainline code because it is too much hassle
and instead keep using their own driver.

In my opinion the mainline driver should be MC-centric, and it is up to the
vendor to decide whether video-centric is also supported: this should only
be done if there is a long history of video-centric behavior in the past.
In that case a Kconfig option is needed to select MC, and in the mainline
kernel this should default to y for such new drivers.

In both Raspbian and Linux4Tegra video-centric has been the norm for many
years, so there are many userspace applications that expect that behavior.
You want those distros to use the mainline driver (eventually...) since
those distros are widely used so you also get a large installed base and
(hopefully) bug reports and bug fixes for the driver. If you decide to
require the distro to carry a patch to turn a driver into a video-centric
variant, then I am afraid they will not bother upgrading to the mainline
driver and just keep their own driver.

In any case, I really like your approach, all I want is a Kconfig option
and it is good to go.

Regards,

	Hans

> 
>>>  /* ------------------------------------------------------------------
>>>   *	Format Handling
>>>   * ------------------------------------------------------------------
>>> @@ -660,13 +664,17 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>>>  {
>>>  	struct cal_dev *cal = container_of(notifier, struct cal_dev, notifier);
>>>  	unsigned int i;
>>> +	int ret = 0;
>>>  
>>>  	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
>>>  		if (cal->ctx[i])
>>>  			cal_ctx_v4l2_register(cal->ctx[i]);
>>>  	}
>>>  
>>> -	return 0;
>>> +	if (cal_mc_api)
>>> +		ret = v4l2_device_register_subdev_nodes(&cal->v4l2_dev);
>>> +
>>> +	return ret;
>>>  }
>>>  
>>>  static const struct v4l2_async_notifier_operations cal_async_notifier_ops = {
>>> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
>>> index 60f5f7480b17..199692610fb9 100644
>>> --- a/drivers/media/platform/ti-vpe/cal.h
>>> +++ b/drivers/media/platform/ti-vpe/cal.h
>>> @@ -160,6 +160,7 @@ struct cal_camerarx {
>>>  	struct device_node	*sensor_ep_node;
>>>  	struct device_node	*sensor_node;
>>>  	struct v4l2_subdev	*sensor;
>>> +	struct media_pipeline	pipe;
>>>  
>>>  	struct v4l2_subdev	subdev;
>>>  	struct media_pad	pads[2];
>>> @@ -224,6 +225,7 @@ struct cal_ctx {
>>>  
>>>  extern unsigned int cal_debug;
>>>  extern int cal_video_nr;
>>> +extern bool cal_mc_api;
>>>  
>>>  #define cal_dbg(level, cal, fmt, arg...)				\
>>>  	do {								\
> 

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Laurent Pinchart]

Hi Hans,

On Tue, Dec 08, 2020 at 09:58:39AM +0100, Hans Verkuil wrote:
> On 08/12/2020 00:51, Laurent Pinchart wrote:
> > On Mon, Dec 07, 2020 at 11:11:33AM +0100, Hans Verkuil wrote:
> >> On 07/12/2020 00:53, Laurent Pinchart wrote:
> >>> The CAL driver is video node centric, it controls the whole device
> >>> through the video device nodes. This limits the possible use cases as it
> >>> can't support sources that are more complex than a single subdev. To
> >>> support more complex hardware pipelines, implement support for the media
> >>> controller centric API. The exposed API can be selected through a module
> >>> parameter.
> >>>
> >>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>> ---
> >>> Changes since v1:
> >>>
> >>> - Fix parameter name and description in MODULE_PARM_DESC()
> >>> ---
> >>>  drivers/media/platform/ti-vpe/cal-camerarx.c |   1 +
> >>>  drivers/media/platform/ti-vpe/cal-video.c    | 356 ++++++++++++++-----
> >>>  drivers/media/platform/ti-vpe/cal.c          |  10 +-
> >>>  drivers/media/platform/ti-vpe/cal.h          |   2 +
> >>>  4 files changed, 288 insertions(+), 81 deletions(-)
> >>>
> >>> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> >>> index 1920f36137b8..8abf887c47d6 100644
> >>> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> >>> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> >>> @@ -830,6 +830,7 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
> >>>  	sd = &phy->subdev;
> >>>  	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
> >>>  	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> >>> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> >>>  	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
> >>>  	sd->dev = cal->dev;
> >>>  
> >>> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> >>> index 779f1e1bc529..119a17fb97bc 100644
> >>> --- a/drivers/media/platform/ti-vpe/cal-video.c
> >>> +++ b/drivers/media/platform/ti-vpe/cal-video.c
> >>> @@ -40,40 +40,10 @@ static char *fourcc_to_str(u32 fmt)
> >>>  }
> >>>  
> >>>  /* ------------------------------------------------------------------
> >>> - *	V4L2 Video IOCTLs
> >>> + *	V4L2 Common IOCTLs
> >>>   * ------------------------------------------------------------------
> >>>   */
> >>>  
> >>> -static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
> >>> -							u32 pixelformat)
> >>> -{
> >>> -	const struct cal_format_info *fmtinfo;
> >>> -	unsigned int k;
> >>> -
> >>> -	for (k = 0; k < ctx->num_active_fmt; k++) {
> >>> -		fmtinfo = ctx->active_fmt[k];
> >>> -		if (fmtinfo->fourcc == pixelformat)
> >>> -			return fmtinfo;
> >>> -	}
> >>> -
> >>> -	return NULL;
> >>> -}
> >>> -
> >>> -static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
> >>> -							 u32 code)
> >>> -{
> >>> -	const struct cal_format_info *fmtinfo;
> >>> -	unsigned int k;
> >>> -
> >>> -	for (k = 0; k < ctx->num_active_fmt; k++) {
> >>> -		fmtinfo = ctx->active_fmt[k];
> >>> -		if (fmtinfo->code == code)
> >>> -			return fmtinfo;
> >>> -	}
> >>> -
> >>> -	return NULL;
> >>> -}
> >>> -
> >>>  static int cal_querycap(struct file *file, void *priv,
> >>>  			struct v4l2_capability *cap)
> >>>  {
> >>> @@ -87,6 +57,51 @@ static int cal_querycap(struct file *file, void *priv,
> >>>  	return 0;
> >>>  }
> >>>  
> >>> +static int cal_g_fmt_vid_cap(struct file *file, void *priv,
> >>> +			     struct v4l2_format *f)
> >>> +{
> >>> +	struct cal_ctx *ctx = video_drvdata(file);
> >>> +
> >>> +	*f = ctx->v_fmt;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +/* ------------------------------------------------------------------
> >>> + *	V4L2 Video Node Centric IOCTLs
> >>> + * ------------------------------------------------------------------
> >>> + */
> >>> +
> >>> +static const struct cal_format_info *find_format_by_pix(struct cal_ctx *ctx,
> >>> +							u32 pixelformat)
> >>> +{
> >>> +	const struct cal_format_info *fmtinfo;
> >>> +	unsigned int k;
> >>> +
> >>> +	for (k = 0; k < ctx->num_active_fmt; k++) {
> >>> +		fmtinfo = ctx->active_fmt[k];
> >>> +		if (fmtinfo->fourcc == pixelformat)
> >>> +			return fmtinfo;
> >>> +	}
> >>> +
> >>> +	return NULL;
> >>> +}
> >>> +
> >>> +static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
> >>> +							 u32 code)
> >>> +{
> >>> +	const struct cal_format_info *fmtinfo;
> >>> +	unsigned int k;
> >>> +
> >>> +	for (k = 0; k < ctx->num_active_fmt; k++) {
> >>> +		fmtinfo = ctx->active_fmt[k];
> >>> +		if (fmtinfo->code == code)
> >>> +			return fmtinfo;
> >>> +	}
> >>> +
> >>> +	return NULL;
> >>> +}
> >>> +
> >>>  static int cal_enum_fmt_vid_cap(struct file *file, void  *priv,
> >>>  				struct v4l2_fmtdesc *f)
> >>>  {
> >>> @@ -174,16 +189,6 @@ static void cal_calc_format_size(struct cal_ctx *ctx,
> >>>  		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
> >>>  }
> >>>  
> >>> -static int cal_g_fmt_vid_cap(struct file *file, void *priv,
> >>> -			     struct v4l2_format *f)
> >>> -{
> >>> -	struct cal_ctx *ctx = video_drvdata(file);
> >>> -
> >>> -	*f = ctx->v_fmt;
> >>> -
> >>> -	return 0;
> >>> -}
> >>> -
> >>>  static int cal_try_fmt_vid_cap(struct file *file, void *priv,
> >>>  			       struct v4l2_format *f)
> >>>  {
> >>> @@ -383,16 +388,7 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
> >>>  	return 0;
> >>>  }
> >>>  
> >>> -static const struct v4l2_file_operations cal_fops = {
> >>> -	.owner		= THIS_MODULE,
> >>> -	.open           = v4l2_fh_open,
> >>> -	.release        = vb2_fop_release,
> >>> -	.poll		= vb2_fop_poll,
> >>> -	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
> >>> -	.mmap           = vb2_fop_mmap,
> >>> -};
> >>> -
> >>> -static const struct v4l2_ioctl_ops cal_ioctl_ops = {
> >>> +static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
> >>>  	.vidioc_querycap      = cal_querycap,
> >>>  	.vidioc_enum_fmt_vid_cap  = cal_enum_fmt_vid_cap,
> >>>  	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
> >>> @@ -418,7 +414,153 @@ static const struct v4l2_ioctl_ops cal_ioctl_ops = {
> >>>  };
> >>>  
> >>>  /* ------------------------------------------------------------------
> >>> - *	videobuf2 Operations
> >>> + *	V4L2 Media Controller Centric IOCTLs
> >>> + * ------------------------------------------------------------------
> >>> + */
> >>> +
> >>> +static int cal_mc_enum_fmt_vid_cap(struct file *file, void  *priv,
> >>> +				   struct v4l2_fmtdesc *f)
> >>> +{
> >>> +	if (f->index >= cal_num_formats)
> >>> +		return -EINVAL;
> >>> +
> >>> +	f->pixelformat = cal_formats[f->index].fourcc;
> >>> +	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static void cal_mc_try_fmt(struct cal_ctx *ctx, struct v4l2_format *f,
> >>> +			   const struct cal_format_info **info)
> >>> +{
> >>> +	struct v4l2_pix_format *format = &f->fmt.pix;
> >>> +	const struct cal_format_info *fmtinfo;
> >>> +	unsigned int bpp;
> >>> +
> >>> +	/*
> >>> +	 * Default to the first format is the requested pixel format code isn't
> >>> +	 * supported.
> >>> +	 */
> >>> +	fmtinfo = cal_format_by_fourcc(f->fmt.pix.pixelformat);
> >>> +	if (!fmtinfo)
> >>> +		fmtinfo = &cal_formats[0];
> >>> +
> >>> +	/*
> >>> +	 * Clamp the size, update the pixel format. The field and colorspace are
> >>> +	 * accepted as-is, except for V4L2_FIELD_ANY that is turned into
> >>> +	 * V4L2_FIELD_NONE.
> >>> +	 */
> >>> +	bpp = ALIGN(fmtinfo->bpp, 8);
> >>> +
> >>> +	format->width = clamp_t(unsigned int, format->width,
> >>> +				CAL_MIN_WIDTH_BYTES * 8 / bpp,
> >>> +				CAL_MAX_WIDTH_BYTES * 8 / bpp);
> >>> +	format->height = clamp_t(unsigned int, format->height,
> >>> +				 CAL_MIN_HEIGHT_LINES, CAL_MAX_HEIGHT_LINES);
> >>> +	format->pixelformat = fmtinfo->fourcc;
> >>> +
> >>> +	if (format->field == V4L2_FIELD_ANY)
> >>> +		format->field = V4L2_FIELD_NONE;
> >>> +
> >>> +	/*
> >>> +	 * Calculate the number of bytes per line and the image size. The
> >>> +	 * hardware stores the stride as a number of 16 bytes words, in a
> >>> +	 * signed 15-bit value. Only 14 bits are thus usable.
> >>> +	 */
> >>> +	format->bytesperline = ALIGN(clamp(format->bytesperline,
> >>> +					   format->width * bpp / 8,
> >>> +					   ((1U << 14) - 1) * 16), 16);
> >>> +
> >>> +	format->sizeimage = format->height * format->bytesperline;
> >>> +
> >>> +	if (info)
> >>> +		*info = fmtinfo;
> >>> +
> >>> +	ctx_dbg(3, ctx, "%s: %s %ux%u (bytesperline %u sizeimage %u)\n",
> >>> +		__func__, fourcc_to_str(format->pixelformat),
> >>> +		format->width, format->height,
> >>> +		format->bytesperline, format->sizeimage);
> >>> +}
> >>> +
> >>> +static int cal_mc_try_fmt_vid_cap(struct file *file, void *priv,
> >>> +				  struct v4l2_format *f)
> >>> +{
> >>> +	struct cal_ctx *ctx = video_drvdata(file);
> >>> +
> >>> +	cal_mc_try_fmt(ctx, f, NULL);
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static int cal_mc_s_fmt_vid_cap(struct file *file, void *priv,
> >>> +				struct v4l2_format *f)
> >>> +{
> >>> +	struct cal_ctx *ctx = video_drvdata(file);
> >>> +	const struct cal_format_info *fmtinfo;
> >>> +
> >>> +	if (vb2_is_busy(&ctx->vb_vidq)) {
> >>> +		ctx_dbg(3, ctx, "%s device busy\n", __func__);
> >>> +		return -EBUSY;
> >>> +	}
> >>> +
> >>> +	cal_mc_try_fmt(ctx, f, &fmtinfo);
> >>> +
> >>> +	ctx->v_fmt = *f;
> >>> +	ctx->fmtinfo = fmtinfo;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static int cal_mc_enum_framesizes(struct file *file, void *fh,
> >>> +				  struct v4l2_frmsizeenum *fsize)
> >>> +{
> >>> +	struct cal_ctx *ctx = video_drvdata(file);
> >>> +	const struct cal_format_info *fmtinfo;
> >>> +	unsigned int bpp;
> >>> +
> >>> +	if (fsize->index > 0)
> >>> +		return -EINVAL;
> >>> +
> >>> +	fmtinfo = cal_format_by_fourcc(fsize->pixel_format);
> >>> +	if (!fmtinfo) {
> >>> +		ctx_dbg(3, ctx, "Invalid pixel format 0x%08x\n",
> >>> +			fsize->pixel_format);
> >>> +		return -EINVAL;
> >>> +	}
> >>> +
> >>> +	bpp = ALIGN(fmtinfo->bpp, 8);
> >>> +
> >>> +	fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> >>> +	fsize->stepwise.min_width = CAL_MIN_WIDTH_BYTES * 8 / bpp;
> >>> +	fsize->stepwise.max_width = CAL_MAX_WIDTH_BYTES * 8 / bpp;
> >>> +	fsize->stepwise.step_width = 64 / bpp;
> >>> +	fsize->stepwise.min_height = CAL_MIN_HEIGHT_LINES;
> >>> +	fsize->stepwise.max_height = CAL_MAX_HEIGHT_LINES;
> >>> +	fsize->stepwise.step_height = 1;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>> +static const struct v4l2_ioctl_ops cal_ioctl_mc_ops = {
> >>> +	.vidioc_querycap      = cal_querycap,
> >>> +	.vidioc_enum_fmt_vid_cap  = cal_mc_enum_fmt_vid_cap,
> >>> +	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
> >>> +	.vidioc_try_fmt_vid_cap   = cal_mc_try_fmt_vid_cap,
> >>> +	.vidioc_s_fmt_vid_cap     = cal_mc_s_fmt_vid_cap,
> >>> +	.vidioc_enum_framesizes   = cal_mc_enum_framesizes,
> >>> +	.vidioc_reqbufs       = vb2_ioctl_reqbufs,
> >>> +	.vidioc_create_bufs   = vb2_ioctl_create_bufs,
> >>> +	.vidioc_prepare_buf   = vb2_ioctl_prepare_buf,
> >>> +	.vidioc_querybuf      = vb2_ioctl_querybuf,
> >>> +	.vidioc_qbuf          = vb2_ioctl_qbuf,
> >>> +	.vidioc_dqbuf         = vb2_ioctl_dqbuf,
> >>> +	.vidioc_expbuf        = vb2_ioctl_expbuf,
> >>> +	.vidioc_streamon      = vb2_ioctl_streamon,
> >>> +	.vidioc_streamoff     = vb2_ioctl_streamoff,
> >>> +	.vidioc_log_status    = v4l2_ctrl_log_status,
> >>> +};
> >>> +
> >>> +/* ------------------------------------------------------------------
> >>> + *	videobuf2 Common Operations
> >>>   * ------------------------------------------------------------------
> >>>   */
> >>>  
> >>> @@ -504,6 +646,26 @@ static void cal_release_buffers(struct cal_ctx *ctx,
> >>>  	spin_unlock_irq(&ctx->dma.lock);
> >>>  }
> >>>  
> >>> +/* ------------------------------------------------------------------
> >>> + *	videobuf2 Operations
> >>> + * ------------------------------------------------------------------
> >>> + */
> >>> +
> >>> +static int cal_video_check_format(struct cal_ctx *ctx)
> >>> +{
> >>> +	const struct v4l2_mbus_framefmt *format;
> >>> +
> >>> +	format = &ctx->phy->formats[CAL_CAMERARX_PAD_SOURCE];
> >>> +
> >>> +	if (ctx->fmtinfo->code != format->code ||
> >>> +	    ctx->v_fmt.fmt.pix.height != format->height ||
> >>> +	    ctx->v_fmt.fmt.pix.width != format->width ||
> >>> +	    ctx->v_fmt.fmt.pix.field != format->field)
> >>> +		return -EPIPE;
> >>> +
> >>> +	return 0;
> >>> +}
> >>> +
> >>>  static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
> >>>  {
> >>>  	struct cal_ctx *ctx = vb2_get_drv_priv(vq);
> >>> @@ -511,6 +673,23 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
> >>>  	dma_addr_t addr;
> >>>  	int ret;
> >>>  
> >>> +	ret = media_pipeline_start(&ctx->vdev.entity, &ctx->phy->pipe);
> >>> +	if (ret < 0) {
> >>> +		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
> >>> +		return ret;
> >>> +	}
> >>> +
> >>> +	/*
> >>> +	 * Verify that the currently configured format matches the output of
> >>> +	 * the connected CAMERARX.
> >>> +	 */
> >>> +	ret = cal_video_check_format(ctx);
> >>> +	if (ret < 0) {
> >>> +		ctx_dbg(3, ctx,
> >>> +			"Format mismatch between CAMERARX and video node\n");
> >>> +		goto error_pipeline;
> >>> +	}
> >>> +
> >>>  	spin_lock_irq(&ctx->dma.lock);
> >>>  	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
> >>>  	ctx->dma.pending = buf;
> >>> @@ -526,18 +705,21 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
> >>>  
> >>>  	ret = v4l2_subdev_call(&ctx->phy->subdev, video, s_stream, 1);
> >>>  	if (ret)
> >>> -		goto err;
> >>> +		goto error_stop;
> >>>  
> >>>  	if (cal_debug >= 4)
> >>>  		cal_quickdump_regs(ctx->cal);
> >>>  
> >>>  	return 0;
> >>>  
> >>> -err:
> >>> +error_stop:
> >>>  	cal_ctx_stop(ctx);
> >>>  	pm_runtime_put_sync(ctx->cal->dev);
> >>>  
> >>> +error_pipeline:
> >>> +	media_pipeline_stop(&ctx->vdev.entity);
> >>>  	cal_release_buffers(ctx, VB2_BUF_STATE_QUEUED);
> >>> +
> >>>  	return ret;
> >>>  }
> >>>  
> >>> @@ -552,6 +734,8 @@ static void cal_stop_streaming(struct vb2_queue *vq)
> >>>  	pm_runtime_put_sync(ctx->cal->dev);
> >>>  
> >>>  	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
> >>> +
> >>> +	media_pipeline_stop(&ctx->vdev.entity);
> >>>  }
> >>>  
> >>>  static const struct vb2_ops cal_video_qops = {
> >>> @@ -569,13 +753,13 @@ static const struct vb2_ops cal_video_qops = {
> >>>   * ------------------------------------------------------------------
> >>>   */
> >>>  
> >>> -static const struct video_device cal_videodev = {
> >>> -	.name		= CAL_MODULE_NAME,
> >>> -	.fops		= &cal_fops,
> >>> -	.ioctl_ops	= &cal_ioctl_ops,
> >>> -	.minor		= -1,
> >>> -	.release	= video_device_release_empty,
> >>> -	.device_caps	= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING,
> >>> +static const struct v4l2_file_operations cal_fops = {
> >>> +	.owner		= THIS_MODULE,
> >>> +	.open           = v4l2_fh_open,
> >>> +	.release        = vb2_fop_release,
> >>> +	.poll		= vb2_fop_poll,
> >>> +	.unlocked_ioctl = video_ioctl2, /* V4L2 ioctl handler */
> >>> +	.mmap           = vb2_fop_mmap,
> >>>  };
> >>>  
> >>>  static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
> >>> @@ -649,19 +833,22 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
> >>>  
> >>>  int cal_ctx_v4l2_register(struct cal_ctx *ctx)
> >>>  {
> >>> -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> >>>  	struct video_device *vfd = &ctx->vdev;
> >>>  	int ret;
> >>>  
> >>> -	ret = cal_ctx_v4l2_init_formats(ctx);
> >>> -	if (ret)
> >>> -		return ret;
> >>> +	if (!cal_mc_api) {
> >>> +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> >>>  
> >>> -	ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler, NULL,
> >>> -				    true);
> >>> -	if (ret < 0) {
> >>> -		ctx_err(ctx, "Failed to add sensor ctrl handler\n");
> >>> -		return ret;
> >>> +		ret = cal_ctx_v4l2_init_formats(ctx);
> >>> +		if (ret)
> >>> +			return ret;
> >>> +
> >>> +		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler,
> >>> +					    NULL, true);
> >>> +		if (ret < 0) {
> >>> +			ctx_err(ctx, "Failed to add sensor ctrl handler\n");
> >>> +			return ret;
> >>> +		}
> >>>  	}
> >>>  
> >>>  	ret = video_register_device(vfd, VFL_TYPE_VIDEO, cal_video_nr);
> >>> @@ -698,7 +885,6 @@ void cal_ctx_v4l2_unregister(struct cal_ctx *ctx)
> >>>  
> >>>  int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >>>  {
> >>> -	struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> >>>  	struct video_device *vfd = &ctx->vdev;
> >>>  	struct vb2_queue *q = &ctx->vb_vidq;
> >>>  	int ret;
> >>> @@ -725,10 +911,14 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >>>  		return ret;
> >>>  
> >>>  	/* Initialize the video device and media entity. */
> >>> -	*vfd = cal_videodev;
> >>> +	vfd->fops = &cal_fops;
> >>> +	vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING
> >>> +			 | (cal_mc_api ? V4L2_CAP_IO_MC : 0);
> >>>  	vfd->v4l2_dev = &ctx->cal->v4l2_dev;
> >>>  	vfd->queue = q;
> >>>  	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->index);
> >>> +	vfd->release = video_device_release_empty;
> >>> +	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_video_ops;
> >>>  	vfd->lock = &ctx->mutex;
> >>>  	video_set_drvdata(vfd, ctx);
> >>>  
> >>> @@ -737,15 +927,19 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >>>  	if (ret < 0)
> >>>  		return ret;
> >>>  
> >>> -	/* Initialize the control handler. */
> >>> -	ret = v4l2_ctrl_handler_init(hdl, 11);
> >>> -	if (ret < 0) {
> >>> -		ctx_err(ctx, "Failed to init ctrl handler\n");
> >>> -		goto error;
> >>> +	if (!cal_mc_api) {
> >>> +		/* Initialize the control handler. */
> >>> +		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
> >>> +
> >>> +		ret = v4l2_ctrl_handler_init(hdl, 11);
> >>> +		if (ret < 0) {
> >>> +			ctx_err(ctx, "Failed to init ctrl handler\n");
> >>> +			goto error;
> >>> +		}
> >>> +
> >>> +		vfd->ctrl_handler = hdl;
> >>>  	}
> >>
> >> I think it is a good idea if the probe() function will log whether this
> >> driver is in MC or video centric mode.
> > 
> > Would a debug message be fine ? I usually try to make the probe function
> > silent by default to avoid adding to the already large boot time log.
> 
> I now realized that you set V4L2_CAP_IO_MC in the MC case, and that's easy
> to check with v4l2-ctl -D, so userspace does have an easy method to see
> whether or not it is MC-centric. So just ignore my comment, the code is fine
> as it is now.
> 
> > 
> >>>  
> >>> -	vfd->ctrl_handler = hdl;
> >>> -
> >>>  	return 0;
> >>>  
> >>>  error:
> >>> @@ -755,6 +949,8 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
> >>>  
> >>>  void cal_ctx_v4l2_cleanup(struct cal_ctx *ctx)
> >>>  {
> >>> -	v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> >>> +	if (!cal_mc_api)
> >>> +		v4l2_ctrl_handler_free(&ctx->ctrl_handler);
> >>> +
> >>>  	media_entity_cleanup(&ctx->vdev.entity);
> >>>  }
> >>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> >>> index 293cbac905b3..2ce2b6404c92 100644
> >>> --- a/drivers/media/platform/ti-vpe/cal.c
> >>> +++ b/drivers/media/platform/ti-vpe/cal.c
> >>> @@ -43,6 +43,10 @@ unsigned int cal_debug;
> >>>  module_param_named(debug, cal_debug, uint, 0644);
> >>>  MODULE_PARM_DESC(debug, "activates debug info");
> >>>  
> >>> +bool cal_mc_api;
> >>> +module_param_named(mc_api, cal_mc_api, bool, 0444);
> >>> +MODULE_PARM_DESC(mc_api, "activates the MC API");
> >>> +
> >>
> >> I noticed that this defaults to video centric.
> >>
> >> To come back to the discussion of the v2 of this patch, I believe we
> >> need to decide what to do here so we have a good template for future
> >> drivers that need this.
> >>
> >> My opinion is that you want a Kconfig option to set the default for
> >> this, so this becomes something like this:
> >>
> >> bool cal_mc_api = CONFIG_TI_CAL_MC_API;
> >>
> >> What do you think?
> >>
> >> I will make a PR for v5.12 for patches 1-23, but I would like to have this
> >> remaining issue resolved before merging this final patch.
> >>
> >> I do think that a Kconfig option is very desirable, but whether the default
> >> of this option should be y or n is less clear. Since this driver has always
> >> been video-centric I can imagine that it makes sense to set it to n. But
> >> for e.g. a new driver like the tegra-video driver (currently in staging),
> >> it would make sense to set it to y since it is a new driver. Ditto for the
> >> rpi camera driver.
> > 
> > For this driver I think video-centric mode is the best default to start
> > with, to avoid changing the behaviour all of a sudden. We can switch it
> > to MC-centric by default later if desired, after userspace gets a chance
> > to adapt.
> 
> Fair enough.
> 
> > 
> >> In that case the rule would be that for new mainline drivers the default
> >> should always be y (MC-centric), but if the driver was already in mainline
> >> and MC support is added (like for this driver), then the default remains n
> >> for backwards compatibility.
> > 
> > I think that for new drivers we shouldn't support video-centric mode at
> > all. It should only be supported in downstream (vendor) kernels, and
> > only if backward compatibility with existing userspace needs to be
> > ensured. The unicam driver, for instance, fits in that category. Drivers
> > whose development is ongoing (or hasn't started) should only use the MC
> > API. Whether the option should be y or n by default would then be a
> > vendor decision, it wouldn't affect upstream.
> 
> No, that I strongly disagree with. Vendors would have to carry those patches
> for a long time, and if past experience is any guide, they will mess it up.
> Or even refuse to upgrade to the mainline code because it is too much hassle
> and instead keep using their own driver.
> 
> In my opinion the mainline driver should be MC-centric, and it is up to the
> vendor to decide whether video-centric is also supported: this should only
> be done if there is a long history of video-centric behavior in the past.
> In that case a Kconfig option is needed to select MC, and in the mainline
> kernel this should default to y for such new drivers.
> 
> In both Raspbian and Linux4Tegra video-centric has been the norm for many
> years, so there are many userspace applications that expect that behavior.
> You want those distros to use the mainline driver (eventually...) since
> those distros are widely used so you also get a large installed base and
> (hopefully) bug reports and bug fixes for the driver. If you decide to
> require the distro to carry a patch to turn a driver into a video-centric
> variant, then I am afraid they will not bother upgrading to the mainline
> driver and just keep their own driver.

For Raspberry Pi, and the Unicam driver in particular, that won't be
possible. A video-centric API will require quite a few hacks that
shouldn't be upstreamed, in particular to support multiple CSI-2 data
types. The current implementation uses two sink pad in the CSI-2
receiver subdevs to model the image and embedded data multiplexed over
the CSI-2 virtual channel. This requires corresponding changes to sensor
drivers to use two source pads. Sakari has reviewed this, and the
implementation will need to move to the V4L2 multiplexed streams support
API (which has been proposed but not merged yet), and I can't see this
working well with a video-centric approach.

I suspect the same would apply to any CSI-2 receiver, and thus to Tegra
as well, but I can't comment on that as I'm not familiar with the
hardware and driver.

> In any case, I really like your approach, all I want is a Kconfig option
> and it is good to go.
> 
> >>>  /* ------------------------------------------------------------------
> >>>   *	Format Handling
> >>>   * ------------------------------------------------------------------
> >>> @@ -660,13 +664,17 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
> >>>  {
> >>>  	struct cal_dev *cal = container_of(notifier, struct cal_dev, notifier);
> >>>  	unsigned int i;
> >>> +	int ret = 0;
> >>>  
> >>>  	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> >>>  		if (cal->ctx[i])
> >>>  			cal_ctx_v4l2_register(cal->ctx[i]);
> >>>  	}
> >>>  
> >>> -	return 0;
> >>> +	if (cal_mc_api)
> >>> +		ret = v4l2_device_register_subdev_nodes(&cal->v4l2_dev);
> >>> +
> >>> +	return ret;
> >>>  }
> >>>  
> >>>  static const struct v4l2_async_notifier_operations cal_async_notifier_ops = {
> >>> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> >>> index 60f5f7480b17..199692610fb9 100644
> >>> --- a/drivers/media/platform/ti-vpe/cal.h
> >>> +++ b/drivers/media/platform/ti-vpe/cal.h
> >>> @@ -160,6 +160,7 @@ struct cal_camerarx {
> >>>  	struct device_node	*sensor_ep_node;
> >>>  	struct device_node	*sensor_node;
> >>>  	struct v4l2_subdev	*sensor;
> >>> +	struct media_pipeline	pipe;
> >>>  
> >>>  	struct v4l2_subdev	subdev;
> >>>  	struct media_pad	pads[2];
> >>> @@ -224,6 +225,7 @@ struct cal_ctx {
> >>>  
> >>>  extern unsigned int cal_debug;
> >>>  extern int cal_video_nr;
> >>> +extern bool cal_mc_api;
> >>>  
> >>>  #define cal_dbg(level, cal, fmt, arg...)				\
> >>>  	do {								\

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Tomi Valkeinen]

On 08/12/2020 18:15, Laurent Pinchart wrote:

>>>> I noticed that this defaults to video centric.
>>>>
>>>> To come back to the discussion of the v2 of this patch, I believe we
>>>> need to decide what to do here so we have a good template for future
>>>> drivers that need this.
>>>>
>>>> My opinion is that you want a Kconfig option to set the default for
>>>> this, so this becomes something like this:
>>>>
>>>> bool cal_mc_api = CONFIG_TI_CAL_MC_API;
>>>>
>>>> What do you think?
>>>>
>>>> I will make a PR for v5.12 for patches 1-23, but I would like to have this
>>>> remaining issue resolved before merging this final patch.
>>>>
>>>> I do think that a Kconfig option is very desirable, but whether the default
>>>> of this option should be y or n is less clear. Since this driver has always
>>>> been video-centric I can imagine that it makes sense to set it to n. But
>>>> for e.g. a new driver like the tegra-video driver (currently in staging),
>>>> it would make sense to set it to y since it is a new driver. Ditto for the
>>>> rpi camera driver.
>>>
>>> For this driver I think video-centric mode is the best default to start
>>> with, to avoid changing the behaviour all of a sudden. We can switch it
>>> to MC-centric by default later if desired, after userspace gets a chance
>>> to adapt.
>>
>> Fair enough.
>>
>>>
>>>> In that case the rule would be that for new mainline drivers the default
>>>> should always be y (MC-centric), but if the driver was already in mainline
>>>> and MC support is added (like for this driver), then the default remains n
>>>> for backwards compatibility.
>>>
>>> I think that for new drivers we shouldn't support video-centric mode at
>>> all. It should only be supported in downstream (vendor) kernels, and
>>> only if backward compatibility with existing userspace needs to be
>>> ensured. The unicam driver, for instance, fits in that category. Drivers
>>> whose development is ongoing (or hasn't started) should only use the MC
>>> API. Whether the option should be y or n by default would then be a
>>> vendor decision, it wouldn't affect upstream.
>>
>> No, that I strongly disagree with. Vendors would have to carry those patches
>> for a long time, and if past experience is any guide, they will mess it up.
>> Or even refuse to upgrade to the mainline code because it is too much hassle
>> and instead keep using their own driver.
>>
>> In my opinion the mainline driver should be MC-centric, and it is up to the
>> vendor to decide whether video-centric is also supported: this should only
>> be done if there is a long history of video-centric behavior in the past.
>> In that case a Kconfig option is needed to select MC, and in the mainline
>> kernel this should default to y for such new drivers.
>>
>> In both Raspbian and Linux4Tegra video-centric has been the norm for many
>> years, so there are many userspace applications that expect that behavior.
>> You want those distros to use the mainline driver (eventually...) since
>> those distros are widely used so you also get a large installed base and
>> (hopefully) bug reports and bug fixes for the driver. If you decide to
>> require the distro to carry a patch to turn a driver into a video-centric
>> variant, then I am afraid they will not bother upgrading to the mainline
>> driver and just keep their own driver.
> 
> For Raspberry Pi, and the Unicam driver in particular, that won't be
> possible. A video-centric API will require quite a few hacks that
> shouldn't be upstreamed, in particular to support multiple CSI-2 data
> types. The current implementation uses two sink pad in the CSI-2
> receiver subdevs to model the image and embedded data multiplexed over
> the CSI-2 virtual channel. This requires corresponding changes to sensor
> drivers to use two source pads. Sakari has reviewed this, and the
> implementation will need to move to the V4L2 multiplexed streams support
> API (which has been proposed but not merged yet), and I can't see this
> working well with a video-centric approach.
> 
> I suspect the same would apply to any CSI-2 receiver, and thus to Tegra
> as well, but I can't comment on that as I'm not familiar with the
> hardware and driver.
> 
>> In any case, I really like your approach, all I want is a Kconfig option
>> and it is good to go.

Waking up this thread, as I'm writing new patches based on these =).

For this series, afaiu there are no open questions. We can add a kconfig
option to choose the default option (in addition to the module
parameter), and as discussed, this one should default to video mode.

For new drivers, I think we should require support for MC (and default
to MC), but leave the decision about video support to the
vendor/developer. I have the same concerns as Hans if we reject new
drivers with video support by default.

Then again, I think it's sensible to require the video support to be...
well, "sensible". The code for video support should be quite
straightforward and simple. It should be a valid reason to reject the
driver if the driver tries to support complex HW setups with video model
and ends up creating all kinds of hacks which are not needed with MC.
(the Unicam case Laurent described above sounds like this).

 Tomi

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Hans Verkuil]

On 15/02/2021 16:23, Tomi Valkeinen wrote:
> On 08/12/2020 18:15, Laurent Pinchart wrote:
> 
>>>>> I noticed that this defaults to video centric.
>>>>>
>>>>> To come back to the discussion of the v2 of this patch, I believe we
>>>>> need to decide what to do here so we have a good template for future
>>>>> drivers that need this.
>>>>>
>>>>> My opinion is that you want a Kconfig option to set the default for
>>>>> this, so this becomes something like this:
>>>>>
>>>>> bool cal_mc_api = CONFIG_TI_CAL_MC_API;
>>>>>
>>>>> What do you think?
>>>>>
>>>>> I will make a PR for v5.12 for patches 1-23, but I would like to have this
>>>>> remaining issue resolved before merging this final patch.
>>>>>
>>>>> I do think that a Kconfig option is very desirable, but whether the default
>>>>> of this option should be y or n is less clear. Since this driver has always
>>>>> been video-centric I can imagine that it makes sense to set it to n. But
>>>>> for e.g. a new driver like the tegra-video driver (currently in staging),
>>>>> it would make sense to set it to y since it is a new driver. Ditto for the
>>>>> rpi camera driver.
>>>>
>>>> For this driver I think video-centric mode is the best default to start
>>>> with, to avoid changing the behaviour all of a sudden. We can switch it
>>>> to MC-centric by default later if desired, after userspace gets a chance
>>>> to adapt.
>>>
>>> Fair enough.
>>>
>>>>
>>>>> In that case the rule would be that for new mainline drivers the default
>>>>> should always be y (MC-centric), but if the driver was already in mainline
>>>>> and MC support is added (like for this driver), then the default remains n
>>>>> for backwards compatibility.
>>>>
>>>> I think that for new drivers we shouldn't support video-centric mode at
>>>> all. It should only be supported in downstream (vendor) kernels, and
>>>> only if backward compatibility with existing userspace needs to be
>>>> ensured. The unicam driver, for instance, fits in that category. Drivers
>>>> whose development is ongoing (or hasn't started) should only use the MC
>>>> API. Whether the option should be y or n by default would then be a
>>>> vendor decision, it wouldn't affect upstream.
>>>
>>> No, that I strongly disagree with. Vendors would have to carry those patches
>>> for a long time, and if past experience is any guide, they will mess it up.
>>> Or even refuse to upgrade to the mainline code because it is too much hassle
>>> and instead keep using their own driver.
>>>
>>> In my opinion the mainline driver should be MC-centric, and it is up to the
>>> vendor to decide whether video-centric is also supported: this should only
>>> be done if there is a long history of video-centric behavior in the past.
>>> In that case a Kconfig option is needed to select MC, and in the mainline
>>> kernel this should default to y for such new drivers.
>>>
>>> In both Raspbian and Linux4Tegra video-centric has been the norm for many
>>> years, so there are many userspace applications that expect that behavior.
>>> You want those distros to use the mainline driver (eventually...) since
>>> those distros are widely used so you also get a large installed base and
>>> (hopefully) bug reports and bug fixes for the driver. If you decide to
>>> require the distro to carry a patch to turn a driver into a video-centric
>>> variant, then I am afraid they will not bother upgrading to the mainline
>>> driver and just keep their own driver.
>>
>> For Raspberry Pi, and the Unicam driver in particular, that won't be
>> possible. A video-centric API will require quite a few hacks that
>> shouldn't be upstreamed, in particular to support multiple CSI-2 data
>> types. The current implementation uses two sink pad in the CSI-2
>> receiver subdevs to model the image and embedded data multiplexed over
>> the CSI-2 virtual channel. This requires corresponding changes to sensor
>> drivers to use two source pads. Sakari has reviewed this, and the
>> implementation will need to move to the V4L2 multiplexed streams support
>> API (which has been proposed but not merged yet), and I can't see this
>> working well with a video-centric approach.
>>
>> I suspect the same would apply to any CSI-2 receiver, and thus to Tegra
>> as well, but I can't comment on that as I'm not familiar with the
>> hardware and driver.
>>
>>> In any case, I really like your approach, all I want is a Kconfig option
>>> and it is good to go.
> 
> Waking up this thread, as I'm writing new patches based on these =).
> 
> For this series, afaiu there are no open questions. We can add a kconfig
> option to choose the default option (in addition to the module
> parameter), and as discussed, this one should default to video mode.

Can someone make a v4 of this patch? It would be nice to get this last
remaining patch merged.

> 
> For new drivers, I think we should require support for MC (and default
> to MC), but leave the decision about video support to the
> vendor/developer. 

Makes sense.

> I have the same concerns as Hans if we reject new
> drivers with video support by default.
> Then again, I think it's sensible to require the video support to be...
> well, "sensible". The code for video support should be quite
> straightforward and simple. It should be a valid reason to reject the
> driver if the driver tries to support complex HW setups with video model
> and ends up creating all kinds of hacks which are not needed with MC.
> (the Unicam case Laurent described above sounds like this).

And there is a grey area between 'sensible' and 'not sensible'. If there
is already a large video-centric ecosystem, then there is a good reason
to allow for more complexity to avoid distros forking the driver.

I guess I would have to see the complexity to be able to say if it is
worth it in my opinion (which I am sure differs from other people's
opinion!).

Regards,

	Hans

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Laurent Pinchart]

Hi Hans,

On Wed, Mar 03, 2021 at 04:15:37PM +0100, Hans Verkuil wrote:
> On 15/02/2021 16:23, Tomi Valkeinen wrote:
> > On 08/12/2020 18:15, Laurent Pinchart wrote:
> > 
> >>>>> I noticed that this defaults to video centric.
> >>>>>
> >>>>> To come back to the discussion of the v2 of this patch, I believe we
> >>>>> need to decide what to do here so we have a good template for future
> >>>>> drivers that need this.
> >>>>>
> >>>>> My opinion is that you want a Kconfig option to set the default for
> >>>>> this, so this becomes something like this:
> >>>>>
> >>>>> bool cal_mc_api = CONFIG_TI_CAL_MC_API;
> >>>>>
> >>>>> What do you think?
> >>>>>
> >>>>> I will make a PR for v5.12 for patches 1-23, but I would like to have this
> >>>>> remaining issue resolved before merging this final patch.
> >>>>>
> >>>>> I do think that a Kconfig option is very desirable, but whether the default
> >>>>> of this option should be y or n is less clear. Since this driver has always
> >>>>> been video-centric I can imagine that it makes sense to set it to n. But
> >>>>> for e.g. a new driver like the tegra-video driver (currently in staging),
> >>>>> it would make sense to set it to y since it is a new driver. Ditto for the
> >>>>> rpi camera driver.
> >>>>
> >>>> For this driver I think video-centric mode is the best default to start
> >>>> with, to avoid changing the behaviour all of a sudden. We can switch it
> >>>> to MC-centric by default later if desired, after userspace gets a chance
> >>>> to adapt.
> >>>
> >>> Fair enough.
> >>>
> >>>>> In that case the rule would be that for new mainline drivers the default
> >>>>> should always be y (MC-centric), but if the driver was already in mainline
> >>>>> and MC support is added (like for this driver), then the default remains n
> >>>>> for backwards compatibility.
> >>>>
> >>>> I think that for new drivers we shouldn't support video-centric mode at
> >>>> all. It should only be supported in downstream (vendor) kernels, and
> >>>> only if backward compatibility with existing userspace needs to be
> >>>> ensured. The unicam driver, for instance, fits in that category. Drivers
> >>>> whose development is ongoing (or hasn't started) should only use the MC
> >>>> API. Whether the option should be y or n by default would then be a
> >>>> vendor decision, it wouldn't affect upstream.
> >>>
> >>> No, that I strongly disagree with. Vendors would have to carry those patches
> >>> for a long time, and if past experience is any guide, they will mess it up.
> >>> Or even refuse to upgrade to the mainline code because it is too much hassle
> >>> and instead keep using their own driver.
> >>>
> >>> In my opinion the mainline driver should be MC-centric, and it is up to the
> >>> vendor to decide whether video-centric is also supported: this should only
> >>> be done if there is a long history of video-centric behavior in the past.
> >>> In that case a Kconfig option is needed to select MC, and in the mainline
> >>> kernel this should default to y for such new drivers.
> >>>
> >>> In both Raspbian and Linux4Tegra video-centric has been the norm for many
> >>> years, so there are many userspace applications that expect that behavior.
> >>> You want those distros to use the mainline driver (eventually...) since
> >>> those distros are widely used so you also get a large installed base and
> >>> (hopefully) bug reports and bug fixes for the driver. If you decide to
> >>> require the distro to carry a patch to turn a driver into a video-centric
> >>> variant, then I am afraid they will not bother upgrading to the mainline
> >>> driver and just keep their own driver.
> >>
> >> For Raspberry Pi, and the Unicam driver in particular, that won't be
> >> possible. A video-centric API will require quite a few hacks that
> >> shouldn't be upstreamed, in particular to support multiple CSI-2 data
> >> types. The current implementation uses two sink pad in the CSI-2
> >> receiver subdevs to model the image and embedded data multiplexed over
> >> the CSI-2 virtual channel. This requires corresponding changes to sensor
> >> drivers to use two source pads. Sakari has reviewed this, and the
> >> implementation will need to move to the V4L2 multiplexed streams support
> >> API (which has been proposed but not merged yet), and I can't see this
> >> working well with a video-centric approach.
> >>
> >> I suspect the same would apply to any CSI-2 receiver, and thus to Tegra
> >> as well, but I can't comment on that as I'm not familiar with the
> >> hardware and driver.
> >>
> >>> In any case, I really like your approach, all I want is a Kconfig option
> >>> and it is good to go.
> > 
> > Waking up this thread, as I'm writing new patches based on these =).
> > 
> > For this series, afaiu there are no open questions. We can add a kconfig
> > option to choose the default option (in addition to the module
> > parameter), and as discussed, this one should default to video mode.
> 
> Can someone make a v4 of this patch? It would be nice to get this last
> remaining patch merged.

I'll work on this.

> > For new drivers, I think we should require support for MC (and default
> > to MC), but leave the decision about video support to the
> > vendor/developer. 
> 
> Makes sense.
> 
> > I have the same concerns as Hans if we reject new
> > drivers with video support by default.
> > Then again, I think it's sensible to require the video support to be...
> > well, "sensible". The code for video support should be quite
> > straightforward and simple. It should be a valid reason to reject the
> > driver if the driver tries to support complex HW setups with video model
> > and ends up creating all kinds of hacks which are not needed with MC.
> > (the Unicam case Laurent described above sounds like this).
> 
> And there is a grey area between 'sensible' and 'not sensible'. If there
> is already a large video-centric ecosystem, then there is a good reason
> to allow for more complexity to avoid distros forking the driver.

I don't think generic distros will care. We're talking about SoC drivers
here, coming from a downstream kernel, so it's only the vendor's
ecosystem that matters, and whether they want to support both in their
camera stack or move it to MC-only.

> I guess I would have to see the complexity to be able to say if it is
> worth it in my opinion (which I am sure differs from other people's
> opinion!).

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 24/24] media: ti-vpe: cal: Implement media controller centric API

[Hans Verkuil]

On 03/03/2021 16:22, Laurent Pinchart wrote:
> Hi Hans,
> 
> On Wed, Mar 03, 2021 at 04:15:37PM +0100, Hans Verkuil wrote:
>> On 15/02/2021 16:23, Tomi Valkeinen wrote:
>>> On 08/12/2020 18:15, Laurent Pinchart wrote:
>>>
>>>>>>> I noticed that this defaults to video centric.
>>>>>>>
>>>>>>> To come back to the discussion of the v2 of this patch, I believe we
>>>>>>> need to decide what to do here so we have a good template for future
>>>>>>> drivers that need this.
>>>>>>>
>>>>>>> My opinion is that you want a Kconfig option to set the default for
>>>>>>> this, so this becomes something like this:
>>>>>>>
>>>>>>> bool cal_mc_api = CONFIG_TI_CAL_MC_API;
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>> I will make a PR for v5.12 for patches 1-23, but I would like to have this
>>>>>>> remaining issue resolved before merging this final patch.
>>>>>>>
>>>>>>> I do think that a Kconfig option is very desirable, but whether the default
>>>>>>> of this option should be y or n is less clear. Since this driver has always
>>>>>>> been video-centric I can imagine that it makes sense to set it to n. But
>>>>>>> for e.g. a new driver like the tegra-video driver (currently in staging),
>>>>>>> it would make sense to set it to y since it is a new driver. Ditto for the
>>>>>>> rpi camera driver.
>>>>>>
>>>>>> For this driver I think video-centric mode is the best default to start
>>>>>> with, to avoid changing the behaviour all of a sudden. We can switch it
>>>>>> to MC-centric by default later if desired, after userspace gets a chance
>>>>>> to adapt.
>>>>>
>>>>> Fair enough.
>>>>>
>>>>>>> In that case the rule would be that for new mainline drivers the default
>>>>>>> should always be y (MC-centric), but if the driver was already in mainline
>>>>>>> and MC support is added (like for this driver), then the default remains n
>>>>>>> for backwards compatibility.
>>>>>>
>>>>>> I think that for new drivers we shouldn't support video-centric mode at
>>>>>> all. It should only be supported in downstream (vendor) kernels, and
>>>>>> only if backward compatibility with existing userspace needs to be
>>>>>> ensured. The unicam driver, for instance, fits in that category. Drivers
>>>>>> whose development is ongoing (or hasn't started) should only use the MC
>>>>>> API. Whether the option should be y or n by default would then be a
>>>>>> vendor decision, it wouldn't affect upstream.
>>>>>
>>>>> No, that I strongly disagree with. Vendors would have to carry those patches
>>>>> for a long time, and if past experience is any guide, they will mess it up.
>>>>> Or even refuse to upgrade to the mainline code because it is too much hassle
>>>>> and instead keep using their own driver.
>>>>>
>>>>> In my opinion the mainline driver should be MC-centric, and it is up to the
>>>>> vendor to decide whether video-centric is also supported: this should only
>>>>> be done if there is a long history of video-centric behavior in the past.
>>>>> In that case a Kconfig option is needed to select MC, and in the mainline
>>>>> kernel this should default to y for such new drivers.
>>>>>
>>>>> In both Raspbian and Linux4Tegra video-centric has been the norm for many
>>>>> years, so there are many userspace applications that expect that behavior.
>>>>> You want those distros to use the mainline driver (eventually...) since
>>>>> those distros are widely used so you also get a large installed base and
>>>>> (hopefully) bug reports and bug fixes for the driver. If you decide to
>>>>> require the distro to carry a patch to turn a driver into a video-centric
>>>>> variant, then I am afraid they will not bother upgrading to the mainline
>>>>> driver and just keep their own driver.
>>>>
>>>> For Raspberry Pi, and the Unicam driver in particular, that won't be
>>>> possible. A video-centric API will require quite a few hacks that
>>>> shouldn't be upstreamed, in particular to support multiple CSI-2 data
>>>> types. The current implementation uses two sink pad in the CSI-2
>>>> receiver subdevs to model the image and embedded data multiplexed over
>>>> the CSI-2 virtual channel. This requires corresponding changes to sensor
>>>> drivers to use two source pads. Sakari has reviewed this, and the
>>>> implementation will need to move to the V4L2 multiplexed streams support
>>>> API (which has been proposed but not merged yet), and I can't see this
>>>> working well with a video-centric approach.
>>>>
>>>> I suspect the same would apply to any CSI-2 receiver, and thus to Tegra
>>>> as well, but I can't comment on that as I'm not familiar with the
>>>> hardware and driver.
>>>>
>>>>> In any case, I really like your approach, all I want is a Kconfig option
>>>>> and it is good to go.
>>>
>>> Waking up this thread, as I'm writing new patches based on these =).
>>>
>>> For this series, afaiu there are no open questions. We can add a kconfig
>>> option to choose the default option (in addition to the module
>>> parameter), and as discussed, this one should default to video mode.
>>
>> Can someone make a v4 of this patch? It would be nice to get this last
>> remaining patch merged.
> 
> I'll work on this.
> 
>>> For new drivers, I think we should require support for MC (and default
>>> to MC), but leave the decision about video support to the
>>> vendor/developer. 
>>
>> Makes sense.
>>
>>> I have the same concerns as Hans if we reject new
>>> drivers with video support by default.
>>> Then again, I think it's sensible to require the video support to be...
>>> well, "sensible". The code for video support should be quite
>>> straightforward and simple. It should be a valid reason to reject the
>>> driver if the driver tries to support complex HW setups with video model
>>> and ends up creating all kinds of hacks which are not needed with MC.
>>> (the Unicam case Laurent described above sounds like this).
>>
>> And there is a grey area between 'sensible' and 'not sensible'. If there
>> is already a large video-centric ecosystem, then there is a good reason
>> to allow for more complexity to avoid distros forking the driver.
> 
> I don't think generic distros will care. We're talking about SoC drivers
> here, coming from a downstream kernel, so it's only the vendor's
> ecosystem that matters, and whether they want to support both in their
> camera stack or move it to MC-only.

Sorry, I wasn't clear: I meant SoC vendor distros. Now most of these have
few users and/or are awful and often very quickly abandoned. I don't care
about those.

But Raspbian and to a lesser extent Linux4Tegra are widely used by hobbyists
as well as professionals, and Raspbian in particular is using a fairly recent
kernel. Making a mainline driver that doesn't fit that ecosystem will almost
certainly cause it to be forked and I think that is something that should be
avoided, even at the expense of additional complexity. It's one area where
we should be willing to go the extra mile.

How many extra miles we are willing to go is another matter, and I don't
know enough to come to a conclusion on that.

In any case, a mainline RPi driver should start out as an MC driver, while
keeping in mind that a video-centric mode might be added later.

If adding a video centric mode turns out to be too much work, then so be it.

Since the vast majority of RPi users will just use Raspbian, I'm afraid that
creating an MC-only RPi driver will just mean that almost nobody will actually
use it, because everyone will stick to the Raspbian video-centric driver.

Anyway, this is just my opinion.

Regards,

	Hans