[PATCH v3 00/38] media: ti-vpe: cal: multistream & embedded data support

[PATCH v3 00/38] media: ti-vpe: cal: multistream & embedded data support

[Tomi Valkeinen]

Hi,

This is v3 of the CAL series to add multiplexed streams and embedded
data support.

This is based on the v7 of the v4l2 core changes to add multiplexed
streams:

https://lore.kernel.org/linux-media/20210524104408.599645-1-tomi.valkeinen@ideasonboard.com/

I didn't receive any comments on the v2 of the series, so the patches up
to and including "media: ti-vpe: cal: allow more than 1 source pads"
have very little changes. The patches after that add the multiplexed
streams and embedded data support, which are now based on the new
approach in the above mentioned v7 series

 Tomi

Tomi Valkeinen (38):
  media: ti-vpe: cal: add g/s_parm for legacy API
  media: ti-vpe: cal: fix error handling in cal_camerarx_create
  media: ti-vpe: cal: remove unused cal_camerarx->dev field
  media: ti-vpe: cal: rename "sensor" to "source"
  media: ti-vpe: cal: move global config from cal_ctx_wr_dma_config to
    runtime resume
  media: ti-vpe: cal: use v4l2_get_link_freq
  media: ti-vpe: cal: add cal_ctx_prepare/unprepare
  media: ti-vpe: cal: change index and cport to u8
  media: ti-vpe: cal: Add CSI2 context
  media: ti-vpe: cal: Add pixel processing context
  media: ti-vpe: cal: rename cal_ctx->index to dma_ctx
  media: ti-vpe: cal: rename CAL_HL_IRQ_MASK
  media: ti-vpe: cal: clean up CAL_CSI2_VC_IRQ_* macros
  media: ti-vpe: cal: catch VC errors
  media: ti-vpe: cal: remove wait when stopping camerarx
  media: ti-vpe: cal: disable csi2 ctx and pix proc at ctx_stop
  media: ti-vpe: cal: allocate pix proc dynamically
  media: ti-vpe: cal: add 'use_pix_proc' field
  media: ti-vpe: cal: add cal_ctx_wr_dma_enable and fix a race
  media: ti-vpe: cal: add vc and datatype fields to cal_ctx
  media: ti-vpe: cal: handle cal_ctx_v4l2_register error
  media: ti-vpe: cal: set field always to V4L2_FIELD_NONE
  media: ti-vpe: cal: fix typo in a comment
  media: ti-vpe: cal: add mbus_code support to cal_mc_enum_fmt_vid_cap
  media: ti-vpe: cal: rename non-MC funcs to cal_legacy_*
  media: ti-vpe: cal: init ctx->v_fmt correctly in MC mode
  media: ti-vpe: cal: remove cal_camerarx->fmtinfo
  media: ti-vpe: cal: support 8 DMA contexts
  media: ti-vpe: cal: cleanup phy iteration in cal_remove
  media: ti-vpe: cal: fix ctx uninitialization
  media: ti-vpe: cal: fix queuing of the initial buffer
  media: ti-vpe: cal: use CSI-2 frame number
  media: ti-vpe: cal: add camerarx locking
  media: ti-vpe: cal: add camerarx enable/disable refcounting
  media: ti-vpe: cal: allow more than 1 source pads
  media: ti-vpe: cal: add embedded data support
  media: ti-vpe: cal: use frame desc to get vc and dt
  media: ti-vpe: cal: add multiplexed streams support

 drivers/media/platform/ti-vpe/cal-camerarx.c | 565 ++++++++++++++-----
 drivers/media/platform/ti-vpe/cal-video.c    | 273 +++++++--
 drivers/media/platform/ti-vpe/cal.c          | 334 ++++++++---
 drivers/media/platform/ti-vpe/cal.h          |  74 ++-
 drivers/media/platform/ti-vpe/cal_regs.h     |  53 +-
 5 files changed, 951 insertions(+), 348 deletions(-)

-- 
2.25.1

[PATCH v3 01/38] media: ti-vpe: cal: add g/s_parm for legacy API

[Tomi Valkeinen]

v4l2-compliance complains about g/s_parm when using the non-MC API. Fix
it by adding the functions and just call v4l2_s/g_parm_cap for the
phy subdev.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index ca75b54311a8..c49ee169b019 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -388,6 +388,20 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
 	return 0;
 }
 
+static int cal_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+	struct cal_ctx *ctx = video_drvdata(file);
+
+	return v4l2_g_parm_cap(video_devdata(file), ctx->phy->sensor, a);
+}
+
+static int cal_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+	struct cal_ctx *ctx = video_drvdata(file);
+
+	return v4l2_s_parm_cap(video_devdata(file), ctx->phy->sensor, a);
+}
+
 static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
 	.vidioc_querycap      = cal_querycap,
 	.vidioc_enum_fmt_vid_cap  = cal_enum_fmt_vid_cap,
@@ -411,6 +425,8 @@ static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
 	.vidioc_log_status    = v4l2_ctrl_log_status,
 	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
 	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+	.vidioc_g_parm		= cal_g_parm,
+	.vidioc_s_parm		= cal_s_parm,
 };
 
 /* ------------------------------------------------------------------
-- 
2.25.1

[PATCH v3 02/38] media: ti-vpe: cal: fix error handling in cal_camerarx_create

[Tomi Valkeinen]

cal_camerarx_create() doesn't handle error returned from
cal_camerarx_sd_init_cfg(). Fix this.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 124a4e2bdefe..e2e384a887ac 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -845,7 +845,9 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	if (ret)
 		goto error;
 
-	cal_camerarx_sd_init_cfg(sd, NULL);
+	ret = cal_camerarx_sd_init_cfg(sd, NULL);
+	if (ret)
+		goto error;
 
 	ret = v4l2_device_register_subdev(&cal->v4l2_dev, sd);
 	if (ret)
-- 
2.25.1

Re: [PATCH v3 02/38] media: ti-vpe: cal: fix error handling in cal_camerarx_create

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:08:33PM +0300, Tomi Valkeinen wrote:
> cal_camerarx_create() doesn't handle error returned from
> cal_camerarx_sd_init_cfg(). Fix this.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index 124a4e2bdefe..e2e384a887ac 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -845,7 +845,9 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	if (ret)
>  		goto error;
>  
> -	cal_camerarx_sd_init_cfg(sd, NULL);
> +	ret = cal_camerarx_sd_init_cfg(sd, NULL);
> +	if (ret)
> +		goto error;
>  
>  	ret = v4l2_device_register_subdev(&cal->v4l2_dev, sd);
>  	if (ret)

-- 
Regards,

Laurent Pinchart

[PATCH v3 03/38] media: ti-vpe: cal: remove unused cal_camerarx->dev field

[Tomi Valkeinen]

cal_camerarx->dev field is not used, remove it.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index db0e408eaa94..e079c6a9f93f 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -149,7 +149,6 @@ struct cal_data {
 struct cal_camerarx {
 	void __iomem		*base;
 	struct resource		*res;
-	struct device		*dev;
 	struct regmap_field	*fields[F_MAX_FIELDS];
 
 	struct cal_dev		*cal;
-- 
2.25.1

[PATCH v3 04/38] media: ti-vpe: cal: rename "sensor" to "source"

[Tomi Valkeinen]

CAL driver uses "sensor" name to refer to the subdev connected to CAL.
As the subdev can also be a bridge, the naming is misleading and might
cause the reader to think it refers to the actual sensor at the end of
the pipeline.

Rename "sensor" to "source".

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 34 ++++++++++----------
 drivers/media/platform/ti-vpe/cal-video.c    | 26 +++++++--------
 drivers/media/platform/ti-vpe/cal.c          | 20 ++++++------
 drivers/media/platform/ti-vpe/cal.h          |  6 ++--
 4 files changed, 43 insertions(+), 43 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index e2e384a887ac..92d39ef45625 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -50,15 +50,15 @@ static s64 cal_camerarx_get_external_rate(struct cal_camerarx *phy)
 	struct v4l2_ctrl *ctrl;
 	s64 rate;
 
-	ctrl = v4l2_ctrl_find(phy->sensor->ctrl_handler, V4L2_CID_PIXEL_RATE);
+	ctrl = v4l2_ctrl_find(phy->source->ctrl_handler, V4L2_CID_PIXEL_RATE);
 	if (!ctrl) {
 		phy_err(phy, "no pixel rate control in subdev: %s\n",
-			phy->sensor->name);
+			phy->source->name);
 		return -EPIPE;
 	}
 
 	rate = v4l2_ctrl_g_ctrl_int64(ctrl);
-	phy_dbg(3, phy, "sensor Pixel Rate: %llu\n", rate);
+	phy_dbg(3, phy, "Source Pixel Rate: %llu\n", rate);
 
 	return rate;
 }
@@ -279,7 +279,7 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	if (external_rate < 0)
 		return external_rate;
 
-	ret = v4l2_subdev_call(phy->sensor, core, s_power, 1);
+	ret = v4l2_subdev_call(phy->source, core, s_power, 1);
 	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV) {
 		phy_err(phy, "power on failed in subdev\n");
 		return ret;
@@ -311,7 +311,7 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	 * 2. CSI PHY and link initialization sequence.
 	 *
 	 *    a. Deassert the CSI-2 PHY reset. Do not wait for reset completion
-	 *       at this point, as it requires the external sensor to send the
+	 *       at this point, as it requires the external source to send the
 	 *       CSI-2 HS clock.
 	 */
 	cal_write_field(phy->cal, CAL_CSI2_COMPLEXIO_CFG(phy->instance),
@@ -370,12 +370,12 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	cal_camerarx_power(phy, true);
 
 	/*
-	 * Start the sensor to enable the CSI-2 HS clock. We can now wait for
+	 * Start the source to enable the CSI-2 HS clock. We can now wait for
 	 * CSI-2 PHY reset to complete.
 	 */
-	ret = v4l2_subdev_call(phy->sensor, video, s_stream, 1);
+	ret = v4l2_subdev_call(phy->source, video, s_stream, 1);
 	if (ret) {
-		v4l2_subdev_call(phy->sensor, core, s_power, 0);
+		v4l2_subdev_call(phy->source, core, s_power, 0);
 		cal_camerarx_disable_irqs(phy);
 		phy_err(phy, "stream on failed in subdev\n");
 		return ret;
@@ -435,10 +435,10 @@ static void cal_camerarx_stop(struct cal_camerarx *phy)
 	/* Disable the phy */
 	cal_camerarx_disable(phy);
 
-	if (v4l2_subdev_call(phy->sensor, video, s_stream, 0))
+	if (v4l2_subdev_call(phy->source, video, s_stream, 0))
 		phy_err(phy, "stream off failed in subdev\n");
 
-	ret = v4l2_subdev_call(phy->sensor, core, s_power, 0);
+	ret = v4l2_subdev_call(phy->source, core, s_power, 0);
 	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV)
 		phy_err(phy, "power off failed in subdev\n");
 }
@@ -558,16 +558,16 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
 		endpoint->bus.mipi_csi2.flags);
 
 	/* Retrieve the connected device and store it for later use. */
-	phy->sensor_ep_node = of_graph_get_remote_endpoint(ep_node);
-	phy->sensor_node = of_graph_get_port_parent(phy->sensor_ep_node);
-	if (!phy->sensor_node) {
+	phy->source_ep_node = of_graph_get_remote_endpoint(ep_node);
+	phy->source_node = of_graph_get_port_parent(phy->source_ep_node);
+	if (!phy->source_node) {
 		phy_dbg(3, phy, "Can't get remote parent\n");
-		of_node_put(phy->sensor_ep_node);
+		of_node_put(phy->source_ep_node);
 		ret = -EINVAL;
 		goto done;
 	}
 
-	phy_dbg(1, phy, "Found connected device %pOFn\n", phy->sensor_node);
+	phy_dbg(1, phy, "Found connected device %pOFn\n", phy->source_node);
 
 done:
 	of_node_put(ep_node);
@@ -868,7 +868,7 @@ void cal_camerarx_destroy(struct cal_camerarx *phy)
 
 	v4l2_device_unregister_subdev(&phy->subdev);
 	media_entity_cleanup(&phy->subdev.entity);
-	of_node_put(phy->sensor_ep_node);
-	of_node_put(phy->sensor_node);
+	of_node_put(phy->source_ep_node);
+	of_node_put(phy->source_node);
 	kfree(phy);
 }
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index c49ee169b019..777413793d86 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -128,7 +128,7 @@ static int __subdev_get_format(struct cal_ctx *ctx,
 	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	sd_fmt.pad = 0;
 
-	ret = v4l2_subdev_call(ctx->phy->sensor, pad, get_fmt, NULL, &sd_fmt);
+	ret = v4l2_subdev_call(ctx->phy->source, pad, get_fmt, NULL, &sd_fmt);
 	if (ret)
 		return ret;
 
@@ -151,7 +151,7 @@ static int __subdev_set_format(struct cal_ctx *ctx,
 	sd_fmt.pad = 0;
 	*mbus_fmt = *fmt;
 
-	ret = v4l2_subdev_call(ctx->phy->sensor, pad, set_fmt, NULL, &sd_fmt);
+	ret = v4l2_subdev_call(ctx->phy->source, pad, set_fmt, NULL, &sd_fmt);
 	if (ret)
 		return ret;
 
@@ -216,7 +216,7 @@ static int cal_try_fmt_vid_cap(struct file *file, void *priv,
 	fse.code = fmtinfo->code;
 	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	for (fse.index = 0; ; fse.index++) {
-		ret = v4l2_subdev_call(ctx->phy->sensor, pad, enum_frame_size,
+		ret = v4l2_subdev_call(ctx->phy->source, pad, enum_frame_size,
 				       NULL, &fse);
 		if (ret)
 			break;
@@ -321,7 +321,7 @@ static int cal_enum_framesizes(struct file *file, void *fh,
 	fse.code = fmtinfo->code;
 	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 
-	ret = v4l2_subdev_call(ctx->phy->sensor, pad, enum_frame_size, NULL,
+	ret = v4l2_subdev_call(ctx->phy->source, pad, enum_frame_size, NULL,
 			       &fse);
 	if (ret)
 		return ret;
@@ -378,7 +378,7 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
 		return -EINVAL;
 
 	fie.code = fmtinfo->code;
-	ret = v4l2_subdev_call(ctx->phy->sensor, pad, enum_frame_interval,
+	ret = v4l2_subdev_call(ctx->phy->source, pad, enum_frame_interval,
 			       NULL, &fie);
 	if (ret)
 		return ret;
@@ -392,14 +392,14 @@ static int cal_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 
-	return v4l2_g_parm_cap(video_devdata(file), ctx->phy->sensor, a);
+	return v4l2_g_parm_cap(video_devdata(file), ctx->phy->source, a);
 }
 
 static int cal_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 
-	return v4l2_s_parm_cap(video_devdata(file), ctx->phy->sensor, a);
+	return v4l2_s_parm_cap(video_devdata(file), ctx->phy->source, a);
 }
 
 static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
@@ -799,20 +799,20 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 		memset(&mbus_code, 0, sizeof(mbus_code));
 		mbus_code.index = j;
 		mbus_code.which = V4L2_SUBDEV_FORMAT_ACTIVE;
-		ret = v4l2_subdev_call(ctx->phy->sensor, pad, enum_mbus_code,
+		ret = v4l2_subdev_call(ctx->phy->source, pad, enum_mbus_code,
 				       NULL, &mbus_code);
 		if (ret == -EINVAL)
 			break;
 
 		if (ret) {
 			ctx_err(ctx, "Error enumerating mbus codes in subdev %s: %d\n",
-				ctx->phy->sensor->name, ret);
+				ctx->phy->source->name, ret);
 			return ret;
 		}
 
 		ctx_dbg(2, ctx,
 			"subdev %s: code: %04x idx: %u\n",
-			ctx->phy->sensor->name, mbus_code.code, j);
+			ctx->phy->source->name, mbus_code.code, j);
 
 		for (k = 0; k < cal_num_formats; k++) {
 			fmtinfo = &cal_formats[k];
@@ -830,7 +830,7 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 
 	if (i == 0) {
 		ctx_err(ctx, "No suitable format reported by subdev %s\n",
-			ctx->phy->sensor->name);
+			ctx->phy->source->name);
 		return -EINVAL;
 	}
 
@@ -867,10 +867,10 @@ int cal_ctx_v4l2_register(struct cal_ctx *ctx)
 	if (!cal_mc_api) {
 		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
 
-		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->sensor->ctrl_handler,
+		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->source->ctrl_handler,
 					    NULL, true);
 		if (ret < 0) {
-			ctx_err(ctx, "Failed to add sensor ctrl handler\n");
+			ctx_err(ctx, "Failed to add source ctrl handler\n");
 			return ret;
 		}
 	}
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 2e2bef91b2b0..a5340b583592 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -635,20 +635,20 @@ static int cal_async_notifier_bound(struct v4l2_async_notifier *notifier,
 	int pad;
 	int ret;
 
-	if (phy->sensor) {
+	if (phy->source) {
 		phy_info(phy, "Rejecting subdev %s (Already set!!)",
 			 subdev->name);
 		return 0;
 	}
 
-	phy->sensor = subdev;
-	phy_dbg(1, phy, "Using sensor %s for capture\n", subdev->name);
+	phy->source = subdev;
+	phy_dbg(1, phy, "Using source %s for capture\n", subdev->name);
 
 	pad = media_entity_get_fwnode_pad(&subdev->entity,
-					  of_fwnode_handle(phy->sensor_ep_node),
+					  of_fwnode_handle(phy->source_ep_node),
 					  MEDIA_PAD_FL_SOURCE);
 	if (pad < 0) {
-		phy_err(phy, "Sensor %s has no connected source pad\n",
+		phy_err(phy, "Source %s has no connected source pad\n",
 			subdev->name);
 		return pad;
 	}
@@ -658,7 +658,7 @@ static int cal_async_notifier_bound(struct v4l2_async_notifier *notifier,
 				    MEDIA_LNK_FL_IMMUTABLE |
 				    MEDIA_LNK_FL_ENABLED);
 	if (ret) {
-		phy_err(phy, "Failed to create media link for sensor %s\n",
+		phy_err(phy, "Failed to create media link for source %s\n",
 			subdev->name);
 		return ret;
 	}
@@ -701,10 +701,10 @@ static int cal_async_notifier_register(struct cal_dev *cal)
 		struct cal_v4l2_async_subdev *casd;
 		struct fwnode_handle *fwnode;
 
-		if (!phy->sensor_node)
+		if (!phy->source_node)
 			continue;
 
-		fwnode = of_fwnode_handle(phy->sensor_node);
+		fwnode = of_fwnode_handle(phy->source_node);
 		casd = v4l2_async_notifier_add_fwnode_subdev(&cal->notifier,
 							     fwnode,
 							     struct cal_v4l2_async_subdev);
@@ -1045,7 +1045,7 @@ static int cal_probe(struct platform_device *pdev)
 			goto error_camerarx;
 		}
 
-		if (cal->phy[i]->sensor_node)
+		if (cal->phy[i]->source_node)
 			connected = true;
 	}
 
@@ -1057,7 +1057,7 @@ static int cal_probe(struct platform_device *pdev)
 
 	/* Create contexts. */
 	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
-		if (!cal->phy[i]->sensor_node)
+		if (!cal->phy[i]->source_node)
 			continue;
 
 		cal->ctx[i] = cal_ctx_create(cal, i);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index e079c6a9f93f..af46084580bd 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -155,9 +155,9 @@ struct cal_camerarx {
 	unsigned int		instance;
 
 	struct v4l2_fwnode_endpoint	endpoint;
-	struct device_node	*sensor_ep_node;
-	struct device_node	*sensor_node;
-	struct v4l2_subdev	*sensor;
+	struct device_node	*source_ep_node;
+	struct device_node	*source_node;
+	struct v4l2_subdev	*source;
 	struct media_pipeline	pipe;
 
 	struct v4l2_subdev	subdev;
-- 
2.25.1

[PATCH v3 05/38] media: ti-vpe: cal: move global config from cal_ctx_wr_dma_config to runtime resume

[Tomi Valkeinen]

For some reason CAL_CTRL is written at the end of cal_ctx_wr_dma_config.
CAL_CTRL is a global (for CAL) register, so it should be independent of
contexts.

Move the code to cal_runtime_resume().

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index a5340b583592..d24f1918f264 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -403,17 +403,6 @@ static void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 	cal_write(ctx->cal, CAL_WR_DMA_XSIZE(ctx->index), val);
 	ctx_dbg(3, ctx, "CAL_WR_DMA_XSIZE(%d) = 0x%08x\n", ctx->index,
 		cal_read(ctx->cal, CAL_WR_DMA_XSIZE(ctx->index)));
-
-	val = cal_read(ctx->cal, CAL_CTRL);
-	cal_set_field(&val, CAL_CTRL_BURSTSIZE_BURST128,
-		      CAL_CTRL_BURSTSIZE_MASK);
-	cal_set_field(&val, 0xF, CAL_CTRL_TAGCNT_MASK);
-	cal_set_field(&val, CAL_CTRL_POSTED_WRITES_NONPOSTED,
-		      CAL_CTRL_POSTED_WRITES_MASK);
-	cal_set_field(&val, 0xFF, CAL_CTRL_MFLAGL_MASK);
-	cal_set_field(&val, 0xFF, CAL_CTRL_MFLAGH_MASK);
-	cal_write(ctx->cal, CAL_CTRL, val);
-	ctx_dbg(3, ctx, "CAL_CTRL = 0x%08x\n", cal_read(ctx->cal, CAL_CTRL));
 }
 
 void cal_ctx_set_dma_addr(struct cal_ctx *ctx, dma_addr_t addr)
@@ -1125,6 +1114,7 @@ static int cal_runtime_resume(struct device *dev)
 {
 	struct cal_dev *cal = dev_get_drvdata(dev);
 	unsigned int i;
+	u32 val;
 
 	if (cal->data->flags & DRA72_CAL_PRE_ES2_LDO_DISABLE) {
 		/*
@@ -1141,6 +1131,17 @@ static int cal_runtime_resume(struct device *dev)
 	 */
 	cal_write(cal, CAL_HL_IRQENABLE_SET(0), CAL_HL_IRQ_OCPO_ERR_MASK);
 
+	val = cal_read(cal, CAL_CTRL);
+	cal_set_field(&val, CAL_CTRL_BURSTSIZE_BURST128,
+		      CAL_CTRL_BURSTSIZE_MASK);
+	cal_set_field(&val, 0xf, CAL_CTRL_TAGCNT_MASK);
+	cal_set_field(&val, CAL_CTRL_POSTED_WRITES_NONPOSTED,
+		      CAL_CTRL_POSTED_WRITES_MASK);
+	cal_set_field(&val, 0xff, CAL_CTRL_MFLAGL_MASK);
+	cal_set_field(&val, 0xff, CAL_CTRL_MFLAGH_MASK);
+	cal_write(cal, CAL_CTRL, val);
+	cal_dbg(3, cal, "CAL_CTRL = 0x%08x\n", cal_read(cal, CAL_CTRL));
+
 	return 0;
 }
 
-- 
2.25.1

[PATCH v3 06/38] media: ti-vpe: cal: use v4l2_get_link_freq

[Tomi Valkeinen]

CAL driver uses V4L2_CID_PIXEL_RATE to get the required pixel rate, and
then changes that value to link rate before configuring the registers.

Rewrite the code to use v4l2_get_link_freq(), which simplifies the code
as we get the link rate directly, and it also adds support for
V4L2_CID_LINK_FREQ.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 52 +++++++-------------
 1 file changed, 19 insertions(+), 33 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 92d39ef45625..3bc63a5e9317 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -45,22 +45,23 @@ static inline void camerarx_write(struct cal_camerarx *phy, u32 offset, u32 val)
  * ------------------------------------------------------------------
  */
 
-static s64 cal_camerarx_get_external_rate(struct cal_camerarx *phy)
+static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
 {
-	struct v4l2_ctrl *ctrl;
-	s64 rate;
+	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
+	u32 num_lanes = mipi_csi2->num_data_lanes;
+	u32 bpp = phy->fmtinfo->bpp;
+	s64 freq;
 
-	ctrl = v4l2_ctrl_find(phy->source->ctrl_handler, V4L2_CID_PIXEL_RATE);
-	if (!ctrl) {
-		phy_err(phy, "no pixel rate control in subdev: %s\n",
+	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
+	if (freq < 0) {
+		phy_err(phy, "failed to get link freq for subdev '%s'\n",
 			phy->source->name);
-		return -EPIPE;
+		return freq;
 	}
 
-	rate = v4l2_ctrl_g_ctrl_int64(ctrl);
-	phy_dbg(3, phy, "Source Pixel Rate: %llu\n", rate);
+	phy_dbg(3, phy, "Source Link Freq: %llu\n", freq);
 
-	return rate;
+	return freq;
 }
 
 static void cal_camerarx_lane_config(struct cal_camerarx *phy)
@@ -116,34 +117,19 @@ void cal_camerarx_disable(struct cal_camerarx *phy)
 #define TCLK_MISS	1
 #define TCLK_SETTLE	14
 
-static void cal_camerarx_config(struct cal_camerarx *phy, s64 external_rate)
+static void cal_camerarx_config(struct cal_camerarx *phy, s64 link_freq)
 {
 	unsigned int reg0, reg1;
 	unsigned int ths_term, ths_settle;
-	unsigned int csi2_ddrclk_khz;
-	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 =
-			&phy->endpoint.bus.mipi_csi2;
-	u32 num_lanes = mipi_csi2->num_data_lanes;
 
 	/* DPHY timing configuration */
 
-	/*
-	 * CSI-2 is DDR and we only count used lanes.
-	 *
-	 * csi2_ddrclk_khz = external_rate / 1000
-	 *		   / (2 * num_lanes) * phy->fmtinfo->bpp;
-	 */
-	csi2_ddrclk_khz = div_s64(external_rate * phy->fmtinfo->bpp,
-				  2 * num_lanes * 1000);
-
-	phy_dbg(1, phy, "csi2_ddrclk_khz: %d\n", csi2_ddrclk_khz);
-
 	/* THS_TERM: Programmed value = floor(20 ns/DDRClk period) */
-	ths_term = 20 * csi2_ddrclk_khz / 1000000;
+	ths_term = div_s64(20 * link_freq, 1000 * 1000 * 1000);
 	phy_dbg(1, phy, "ths_term: %d (0x%02x)\n", ths_term, ths_term);
 
 	/* THS_SETTLE: Programmed value = floor(105 ns/DDRClk period) + 4 */
-	ths_settle = (105 * csi2_ddrclk_khz / 1000000) + 4;
+	ths_settle = div_s64(105 * link_freq, 1000 * 1000 * 1000) + 4;
 	phy_dbg(1, phy, "ths_settle: %d (0x%02x)\n", ths_settle, ths_settle);
 
 	reg0 = camerarx_read(phy, CAL_CSI2_PHY_REG0);
@@ -270,14 +256,14 @@ static void cal_camerarx_ppi_disable(struct cal_camerarx *phy)
 
 static int cal_camerarx_start(struct cal_camerarx *phy)
 {
-	s64 external_rate;
+	s64 link_freq;
 	u32 sscounter;
 	u32 val;
 	int ret;
 
-	external_rate = cal_camerarx_get_external_rate(phy);
-	if (external_rate < 0)
-		return external_rate;
+	link_freq = cal_camerarx_get_ext_link_freq(phy);
+	if (link_freq < 0)
+		return link_freq;
 
 	ret = v4l2_subdev_call(phy->source, core, s_power, 1);
 	if (ret < 0 && ret != -ENOIOCTLCMD && ret != -ENODEV) {
@@ -325,7 +311,7 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	camerarx_read(phy, CAL_CSI2_PHY_REG0);
 
 	/* Program the PHY timing parameters. */
-	cal_camerarx_config(phy, external_rate);
+	cal_camerarx_config(phy, link_freq);
 
 	/*
 	 *    b. Assert the FORCERXMODE signal.
-- 
2.25.1

[PATCH v3 07/38] media: ti-vpe: cal: add cal_ctx_prepare/unprepare

[Tomi Valkeinen]

In the following patches we need to do context configuration which might
fail. Add new functions, cal_ctx_prepare and cal_ctx_unprepare, to
handle such configuration.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-video.c |  9 +++++++++
 drivers/media/platform/ti-vpe/cal.c       | 10 ++++++++++
 drivers/media/platform/ti-vpe/cal.h       |  2 ++
 3 files changed, 21 insertions(+)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 777413793d86..9f0eebd92aac 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -708,6 +708,12 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 		goto error_pipeline;
 	}
 
+	ret = cal_ctx_prepare(ctx);
+	if (ret) {
+		ctx_err(ctx, "Failed to prepare context: %d\n", ret);
+		goto error_pipeline;
+	}
+
 	spin_lock_irq(&ctx->dma.lock);
 	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
 	ctx->dma.pending = buf;
@@ -733,6 +739,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 error_stop:
 	cal_ctx_stop(ctx);
 	pm_runtime_put_sync(ctx->cal->dev);
+	cal_ctx_unprepare(ctx);
 
 error_pipeline:
 	media_pipeline_stop(ctx->vdev.entity.pads);
@@ -752,6 +759,8 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 
 	pm_runtime_put_sync(ctx->cal->dev);
 
+	cal_ctx_unprepare(ctx);
+
 	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
 
 	media_pipeline_stop(ctx->vdev.entity.pads);
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index d24f1918f264..6d6dce8001b2 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -430,6 +430,16 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
 	return stopped;
 }
 
+int cal_ctx_prepare(struct cal_ctx *ctx)
+{
+	return 0;
+}
+
+void cal_ctx_unprepare(struct cal_ctx *ctx)
+{
+
+}
+
 void cal_ctx_start(struct cal_ctx *ctx)
 {
 	ctx->sequence = 0;
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index af46084580bd..09ad20faa9e1 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -296,6 +296,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 					 unsigned int instance);
 void cal_camerarx_destroy(struct cal_camerarx *phy);
 
+int cal_ctx_prepare(struct cal_ctx *ctx);
+void cal_ctx_unprepare(struct cal_ctx *ctx);
 void cal_ctx_set_dma_addr(struct cal_ctx *ctx, dma_addr_t addr);
 void cal_ctx_start(struct cal_ctx *ctx);
 void cal_ctx_stop(struct cal_ctx *ctx);
-- 
2.25.1

[PATCH v3 08/38] media: ti-vpe: cal: change index and cport to u8

[Tomi Valkeinen]

cal_ctx's index and cport fields are numbers less than 8. In the
following patches we will get a bunch of new fields, all of which are
similar small numbers, so lets change the type to u8.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 09ad20faa9e1..251bb0ba7b3b 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -217,8 +217,8 @@ struct cal_ctx {
 
 	unsigned int		sequence;
 	struct vb2_queue	vb_vidq;
-	unsigned int		index;
-	unsigned int		cport;
+	u8			index;
+	u8			cport;
 };
 
 extern unsigned int cal_debug;
-- 
2.25.1

[PATCH v3 09/38] media: ti-vpe: cal: Add CSI2 context

[Tomi Valkeinen]

CAL has 8 CSI2 contexts per PHY, which are used to tag the incoming
data.  The current driver only uses the first context, but we need to
support all of them to implement multi-stream support.

Add a csi2_ctx field to cal_ctx, which indicates which of the 8 CSI2
contexts is used for the particular cal_ctx. Also clean up the context
register macros to take the CSI2 context number as a parameter.

Note that before this patch the CSI2 context used for both PHYs was
always 0. This patch always uses cal_ctx index number as the CSI2
context. There is no functional difference, but this approach will work
also in the future when we use more than 1 CSI2 context per PHY.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c      | 10 ++++++----
 drivers/media/platform/ti-vpe/cal.h      |  1 +
 drivers/media/platform/ti-vpe/cal_regs.h | 18 ++----------------
 3 files changed, 9 insertions(+), 20 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 6d6dce8001b2..98739f9200ff 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -294,7 +294,7 @@ static void cal_ctx_csi2_config(struct cal_ctx *ctx)
 {
 	u32 val;
 
-	val = cal_read(ctx->cal, CAL_CSI2_CTX0(ctx->index));
+	val = cal_read(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx));
 	cal_set_field(&val, ctx->cport, CAL_CSI2_CTX_CPORT_MASK);
 	/*
 	 * DT type: MIPI CSI-2 Specs
@@ -310,9 +310,10 @@ static void cal_ctx_csi2_config(struct cal_ctx *ctx)
 	cal_set_field(&val, CAL_CSI2_CTX_ATT_PIX, CAL_CSI2_CTX_ATT_MASK);
 	cal_set_field(&val, CAL_CSI2_CTX_PACK_MODE_LINE,
 		      CAL_CSI2_CTX_PACK_MODE_MASK);
-	cal_write(ctx->cal, CAL_CSI2_CTX0(ctx->index), val);
-	ctx_dbg(3, ctx, "CAL_CSI2_CTX0(%d) = 0x%08x\n", ctx->index,
-		cal_read(ctx->cal, CAL_CSI2_CTX0(ctx->index)));
+	cal_write(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx), val);
+	ctx_dbg(3, ctx, "CAL_CSI2_CTX(%u, %u) = 0x%08x\n",
+		ctx->phy->instance, ctx->csi2_ctx,
+		cal_read(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx)));
 }
 
 static void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
@@ -854,6 +855,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 	ctx->cal = cal;
 	ctx->phy = cal->phy[inst];
 	ctx->index = inst;
+	ctx->csi2_ctx = inst;
 	ctx->cport = inst;
 
 	ret = cal_ctx_v4l2_init(ctx);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 251bb0ba7b3b..bcc3378b6b41 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -219,6 +219,7 @@ struct cal_ctx {
 	struct vb2_queue	vb_vidq;
 	u8			index;
 	u8			cport;
+	u8			csi2_ctx;
 };
 
 extern unsigned int cal_debug;
diff --git a/drivers/media/platform/ti-vpe/cal_regs.h b/drivers/media/platform/ti-vpe/cal_regs.h
index f752096dcf7f..bf937919a1e9 100644
--- a/drivers/media/platform/ti-vpe/cal_regs.h
+++ b/drivers/media/platform/ti-vpe/cal_regs.h
@@ -72,22 +72,8 @@
 #define CAL_CSI2_TIMING(m)		(0x314U + (m) * 0x80U)
 #define CAL_CSI2_VC_IRQENABLE(m)	(0x318U + (m) * 0x80U)
 #define CAL_CSI2_VC_IRQSTATUS(m)	(0x328U + (m) * 0x80U)
-#define CAL_CSI2_CTX0(m)		(0x330U + (m) * 0x80U)
-#define CAL_CSI2_CTX1(m)		(0x334U + (m) * 0x80U)
-#define CAL_CSI2_CTX2(m)		(0x338U + (m) * 0x80U)
-#define CAL_CSI2_CTX3(m)		(0x33cU + (m) * 0x80U)
-#define CAL_CSI2_CTX4(m)		(0x340U + (m) * 0x80U)
-#define CAL_CSI2_CTX5(m)		(0x344U + (m) * 0x80U)
-#define CAL_CSI2_CTX6(m)		(0x348U + (m) * 0x80U)
-#define CAL_CSI2_CTX7(m)		(0x34cU + (m) * 0x80U)
-#define CAL_CSI2_STATUS0(m)		(0x350U + (m) * 0x80U)
-#define CAL_CSI2_STATUS1(m)		(0x354U + (m) * 0x80U)
-#define CAL_CSI2_STATUS2(m)		(0x358U + (m) * 0x80U)
-#define CAL_CSI2_STATUS3(m)		(0x35cU + (m) * 0x80U)
-#define CAL_CSI2_STATUS4(m)		(0x360U + (m) * 0x80U)
-#define CAL_CSI2_STATUS5(m)		(0x364U + (m) * 0x80U)
-#define CAL_CSI2_STATUS6(m)		(0x368U + (m) * 0x80U)
-#define CAL_CSI2_STATUS7(m)		(0x36cU + (m) * 0x80U)
+#define CAL_CSI2_CTX(phy, csi2_ctx)	(0x330U + (phy) * 0x80U + (csi2_ctx) * 4)
+#define CAL_CSI2_STATUS(phy, csi2_ctx)	(0x350U + (phy) * 0x80U + (csi2_ctx) * 4)
 
 /* CAL CSI2 PHY register offsets */
 #define CAL_CSI2_PHY_REG0		0x000
-- 
2.25.1

Re: [PATCH v3 09/38] media: ti-vpe: cal: Add CSI2 context

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:08:40PM +0300, Tomi Valkeinen wrote:
> CAL has 8 CSI2 contexts per PHY, which are used to tag the incoming
> data.  The current driver only uses the first context, but we need to
> support all of them to implement multi-stream support.
> 
> Add a csi2_ctx field to cal_ctx, which indicates which of the 8 CSI2
> contexts is used for the particular cal_ctx. Also clean up the context
> register macros to take the CSI2 context number as a parameter.
> 
> Note that before this patch the CSI2 context used for both PHYs was
> always 0. This patch always uses cal_ctx index number as the CSI2
> context. There is no functional difference, but this approach will work
> also in the future when we use more than 1 CSI2 context per PHY.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/ti-vpe/cal.c      | 10 ++++++----
>  drivers/media/platform/ti-vpe/cal.h      |  1 +
>  drivers/media/platform/ti-vpe/cal_regs.h | 18 ++----------------
>  3 files changed, 9 insertions(+), 20 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index 6d6dce8001b2..98739f9200ff 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -294,7 +294,7 @@ static void cal_ctx_csi2_config(struct cal_ctx *ctx)
>  {
>  	u32 val;
>  
> -	val = cal_read(ctx->cal, CAL_CSI2_CTX0(ctx->index));
> +	val = cal_read(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx));
>  	cal_set_field(&val, ctx->cport, CAL_CSI2_CTX_CPORT_MASK);
>  	/*
>  	 * DT type: MIPI CSI-2 Specs
> @@ -310,9 +310,10 @@ static void cal_ctx_csi2_config(struct cal_ctx *ctx)
>  	cal_set_field(&val, CAL_CSI2_CTX_ATT_PIX, CAL_CSI2_CTX_ATT_MASK);
>  	cal_set_field(&val, CAL_CSI2_CTX_PACK_MODE_LINE,
>  		      CAL_CSI2_CTX_PACK_MODE_MASK);
> -	cal_write(ctx->cal, CAL_CSI2_CTX0(ctx->index), val);
> -	ctx_dbg(3, ctx, "CAL_CSI2_CTX0(%d) = 0x%08x\n", ctx->index,
> -		cal_read(ctx->cal, CAL_CSI2_CTX0(ctx->index)));
> +	cal_write(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx), val);
> +	ctx_dbg(3, ctx, "CAL_CSI2_CTX(%u, %u) = 0x%08x\n",
> +		ctx->phy->instance, ctx->csi2_ctx,
> +		cal_read(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx)));
>  }
>  
>  static void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
> @@ -854,6 +855,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
>  	ctx->cal = cal;
>  	ctx->phy = cal->phy[inst];
>  	ctx->index = inst;
> +	ctx->csi2_ctx = inst;
>  	ctx->cport = inst;
>  
>  	ret = cal_ctx_v4l2_init(ctx);
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 251bb0ba7b3b..bcc3378b6b41 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -219,6 +219,7 @@ struct cal_ctx {
>  	struct vb2_queue	vb_vidq;
>  	u8			index;
>  	u8			cport;
> +	u8			csi2_ctx;
>  };
>  
>  extern unsigned int cal_debug;
> diff --git a/drivers/media/platform/ti-vpe/cal_regs.h b/drivers/media/platform/ti-vpe/cal_regs.h
> index f752096dcf7f..bf937919a1e9 100644
> --- a/drivers/media/platform/ti-vpe/cal_regs.h
> +++ b/drivers/media/platform/ti-vpe/cal_regs.h
> @@ -72,22 +72,8 @@
>  #define CAL_CSI2_TIMING(m)		(0x314U + (m) * 0x80U)
>  #define CAL_CSI2_VC_IRQENABLE(m)	(0x318U + (m) * 0x80U)
>  #define CAL_CSI2_VC_IRQSTATUS(m)	(0x328U + (m) * 0x80U)
> -#define CAL_CSI2_CTX0(m)		(0x330U + (m) * 0x80U)
> -#define CAL_CSI2_CTX1(m)		(0x334U + (m) * 0x80U)
> -#define CAL_CSI2_CTX2(m)		(0x338U + (m) * 0x80U)
> -#define CAL_CSI2_CTX3(m)		(0x33cU + (m) * 0x80U)
> -#define CAL_CSI2_CTX4(m)		(0x340U + (m) * 0x80U)
> -#define CAL_CSI2_CTX5(m)		(0x344U + (m) * 0x80U)
> -#define CAL_CSI2_CTX6(m)		(0x348U + (m) * 0x80U)
> -#define CAL_CSI2_CTX7(m)		(0x34cU + (m) * 0x80U)
> -#define CAL_CSI2_STATUS0(m)		(0x350U + (m) * 0x80U)
> -#define CAL_CSI2_STATUS1(m)		(0x354U + (m) * 0x80U)
> -#define CAL_CSI2_STATUS2(m)		(0x358U + (m) * 0x80U)
> -#define CAL_CSI2_STATUS3(m)		(0x35cU + (m) * 0x80U)
> -#define CAL_CSI2_STATUS4(m)		(0x360U + (m) * 0x80U)
> -#define CAL_CSI2_STATUS5(m)		(0x364U + (m) * 0x80U)
> -#define CAL_CSI2_STATUS6(m)		(0x368U + (m) * 0x80U)
> -#define CAL_CSI2_STATUS7(m)		(0x36cU + (m) * 0x80U)
> +#define CAL_CSI2_CTX(phy, csi2_ctx)	(0x330U + (phy) * 0x80U + (csi2_ctx) * 4)
> +#define CAL_CSI2_STATUS(phy, csi2_ctx)	(0x350U + (phy) * 0x80U + (csi2_ctx) * 4)
>  
>  /* CAL CSI2 PHY register offsets */
>  #define CAL_CSI2_PHY_REG0		0x000

-- 
Regards,

Laurent Pinchart

[PATCH v3 10/38] media: ti-vpe: cal: Add pixel processing context

[Tomi Valkeinen]

CAL has 4 pixel processing contexts (PIX PROC) of which the driver
currently uses pix proc 0 for PHY0, and pix proc 1 for PHY1 (as the
driver supports only a single source per PHY).

Add a pix_proc field to cal_ctx to allow us to use different pix proc
contexts in the future

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 9 +++++----
 drivers/media/platform/ti-vpe/cal.h | 1 +
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 98739f9200ff..7f5ce6f9d874 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -355,16 +355,16 @@ static void cal_ctx_pix_proc_config(struct cal_ctx *ctx)
 		break;
 	}
 
-	val = cal_read(ctx->cal, CAL_PIX_PROC(ctx->index));
+	val = cal_read(ctx->cal, CAL_PIX_PROC(ctx->pix_proc));
 	cal_set_field(&val, extract, CAL_PIX_PROC_EXTRACT_MASK);
 	cal_set_field(&val, CAL_PIX_PROC_DPCMD_BYPASS, CAL_PIX_PROC_DPCMD_MASK);
 	cal_set_field(&val, CAL_PIX_PROC_DPCME_BYPASS, CAL_PIX_PROC_DPCME_MASK);
 	cal_set_field(&val, pack, CAL_PIX_PROC_PACK_MASK);
 	cal_set_field(&val, ctx->cport, CAL_PIX_PROC_CPORT_MASK);
 	cal_set_field(&val, 1, CAL_PIX_PROC_EN_MASK);
-	cal_write(ctx->cal, CAL_PIX_PROC(ctx->index), val);
-	ctx_dbg(3, ctx, "CAL_PIX_PROC(%d) = 0x%08x\n", ctx->index,
-		cal_read(ctx->cal, CAL_PIX_PROC(ctx->index)));
+	cal_write(ctx->cal, CAL_PIX_PROC(ctx->pix_proc), val);
+	ctx_dbg(3, ctx, "CAL_PIX_PROC(%u) = 0x%08x\n", ctx->pix_proc,
+		cal_read(ctx->cal, CAL_PIX_PROC(ctx->pix_proc)));
 }
 
 static void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
@@ -857,6 +857,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 	ctx->index = inst;
 	ctx->csi2_ctx = inst;
 	ctx->cport = inst;
+	ctx->pix_proc = inst;
 
 	ret = cal_ctx_v4l2_init(ctx);
 	if (ret)
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index bcc3378b6b41..9475dc80559b 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -220,6 +220,7 @@ struct cal_ctx {
 	u8			index;
 	u8			cport;
 	u8			csi2_ctx;
+	u8			pix_proc;
 };
 
 extern unsigned int cal_debug;
-- 
2.25.1

[PATCH v3 11/38] media: ti-vpe: cal: rename cal_ctx->index to dma_ctx

[Tomi Valkeinen]

Rename cal_ctx->index to dma_ctx to clarify that the field refers to the
DMA context number.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-video.c |  4 +--
 drivers/media/platform/ti-vpe/cal.c       | 38 +++++++++++------------
 drivers/media/platform/ti-vpe/cal.h       |  8 ++---
 3 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 9f0eebd92aac..1311adfb35a7 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -897,7 +897,7 @@ int cal_ctx_v4l2_register(struct cal_ctx *ctx)
 				    MEDIA_LNK_FL_ENABLED);
 	if (ret) {
 		ctx_err(ctx, "Failed to create media link for context %u\n",
-			ctx->index);
+			ctx->dma_ctx);
 		video_unregister_device(vfd);
 		return ret;
 	}
@@ -949,7 +949,7 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 			 | (cal_mc_api ? V4L2_CAP_IO_MC : 0);
 	vfd->v4l2_dev = &ctx->cal->v4l2_dev;
 	vfd->queue = q;
-	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->index);
+	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->dma_ctx);
 	vfd->release = video_device_release_empty;
 	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_video_ops;
 	vfd->lock = &ctx->mutex;
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 7f5ce6f9d874..b9b533a4497f 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -372,7 +372,7 @@ static void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 	unsigned int stride = ctx->v_fmt.fmt.pix.bytesperline;
 	u32 val;
 
-	val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->index));
+	val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx));
 	cal_set_field(&val, ctx->cport, CAL_WR_DMA_CTRL_CPORT_MASK);
 	cal_set_field(&val, ctx->v_fmt.fmt.pix.height,
 		      CAL_WR_DMA_CTRL_YSIZE_MASK);
@@ -383,16 +383,16 @@ static void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 	cal_set_field(&val, CAL_WR_DMA_CTRL_PATTERN_LINEAR,
 		      CAL_WR_DMA_CTRL_PATTERN_MASK);
 	cal_set_field(&val, 1, CAL_WR_DMA_CTRL_STALL_RD_MASK);
-	cal_write(ctx->cal, CAL_WR_DMA_CTRL(ctx->index), val);
-	ctx_dbg(3, ctx, "CAL_WR_DMA_CTRL(%d) = 0x%08x\n", ctx->index,
-		cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->index)));
+	cal_write(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx), val);
+	ctx_dbg(3, ctx, "CAL_WR_DMA_CTRL(%d) = 0x%08x\n", ctx->dma_ctx,
+		cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx)));
 
-	cal_write_field(ctx->cal, CAL_WR_DMA_OFST(ctx->index),
+	cal_write_field(ctx->cal, CAL_WR_DMA_OFST(ctx->dma_ctx),
 			stride / 16, CAL_WR_DMA_OFST_MASK);
-	ctx_dbg(3, ctx, "CAL_WR_DMA_OFST(%d) = 0x%08x\n", ctx->index,
-		cal_read(ctx->cal, CAL_WR_DMA_OFST(ctx->index)));
+	ctx_dbg(3, ctx, "CAL_WR_DMA_OFST(%d) = 0x%08x\n", ctx->dma_ctx,
+		cal_read(ctx->cal, CAL_WR_DMA_OFST(ctx->dma_ctx)));
 
-	val = cal_read(ctx->cal, CAL_WR_DMA_XSIZE(ctx->index));
+	val = cal_read(ctx->cal, CAL_WR_DMA_XSIZE(ctx->dma_ctx));
 	/* 64 bit word means no skipping */
 	cal_set_field(&val, 0, CAL_WR_DMA_XSIZE_XSKIP_MASK);
 	/*
@@ -401,23 +401,23 @@ static void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 	 * written per line.
 	 */
 	cal_set_field(&val, stride / 8, CAL_WR_DMA_XSIZE_MASK);
-	cal_write(ctx->cal, CAL_WR_DMA_XSIZE(ctx->index), val);
-	ctx_dbg(3, ctx, "CAL_WR_DMA_XSIZE(%d) = 0x%08x\n", ctx->index,
-		cal_read(ctx->cal, CAL_WR_DMA_XSIZE(ctx->index)));
+	cal_write(ctx->cal, CAL_WR_DMA_XSIZE(ctx->dma_ctx), val);
+	ctx_dbg(3, ctx, "CAL_WR_DMA_XSIZE(%d) = 0x%08x\n", ctx->dma_ctx,
+		cal_read(ctx->cal, CAL_WR_DMA_XSIZE(ctx->dma_ctx)));
 }
 
 void cal_ctx_set_dma_addr(struct cal_ctx *ctx, dma_addr_t addr)
 {
-	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->index), addr);
+	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->dma_ctx), addr);
 }
 
 static void cal_ctx_wr_dma_disable(struct cal_ctx *ctx)
 {
-	u32 val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->index));
+	u32 val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx));
 
 	cal_set_field(&val, CAL_WR_DMA_CTRL_MODE_DIS,
 		      CAL_WR_DMA_CTRL_MODE_MASK);
-	cal_write(ctx->cal, CAL_WR_DMA_CTRL(ctx->index), val);
+	cal_write(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx), val);
 }
 
 static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
@@ -453,9 +453,9 @@ void cal_ctx_start(struct cal_ctx *ctx)
 
 	/* Enable IRQ_WDMA_END and IRQ_WDMA_START. */
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(1),
-		  CAL_HL_IRQ_MASK(ctx->index));
+		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(2),
-		  CAL_HL_IRQ_MASK(ctx->index));
+		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
 }
 
 void cal_ctx_stop(struct cal_ctx *ctx)
@@ -479,9 +479,9 @@ void cal_ctx_stop(struct cal_ctx *ctx)
 
 	/* Disable IRQ_WDMA_END and IRQ_WDMA_START. */
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(1),
-		  CAL_HL_IRQ_MASK(ctx->index));
+		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(2),
-		  CAL_HL_IRQ_MASK(ctx->index));
+		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
 
 	ctx->dma.state = CAL_DMA_STOPPED;
 }
@@ -854,7 +854,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 
 	ctx->cal = cal;
 	ctx->phy = cal->phy[inst];
-	ctx->index = inst;
+	ctx->dma_ctx = inst;
 	ctx->csi2_ctx = inst;
 	ctx->cport = inst;
 	ctx->pix_proc = inst;
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 9475dc80559b..e4db2a905f68 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -217,7 +217,7 @@ struct cal_ctx {
 
 	unsigned int		sequence;
 	struct vb2_queue	vb_vidq;
-	u8			index;
+	u8			dma_ctx;
 	u8			cport;
 	u8			csi2_ctx;
 	u8			pix_proc;
@@ -238,11 +238,11 @@ extern bool cal_mc_api;
 	dev_err((cal)->dev, fmt, ##arg)
 
 #define ctx_dbg(level, ctx, fmt, arg...)				\
-	cal_dbg(level, (ctx)->cal, "ctx%u: " fmt, (ctx)->index, ##arg)
+	cal_dbg(level, (ctx)->cal, "ctx%u: " fmt, (ctx)->dma_ctx, ##arg)
 #define ctx_info(ctx, fmt, arg...)					\
-	cal_info((ctx)->cal, "ctx%u: " fmt, (ctx)->index, ##arg)
+	cal_info((ctx)->cal, "ctx%u: " fmt, (ctx)->dma_ctx, ##arg)
 #define ctx_err(ctx, fmt, arg...)					\
-	cal_err((ctx)->cal, "ctx%u: " fmt, (ctx)->index, ##arg)
+	cal_err((ctx)->cal, "ctx%u: " fmt, (ctx)->dma_ctx, ##arg)
 
 #define phy_dbg(level, phy, fmt, arg...)				\
 	cal_dbg(level, (phy)->cal, "phy%u: " fmt, (phy)->instance, ##arg)
-- 
2.25.1

[PATCH v3 12/38] media: ti-vpe: cal: rename CAL_HL_IRQ_MASK

[Tomi Valkeinen]

CAL_HL_IRQ_MASK macro is used for both WDMA start and end status bits.
For clarity, rename CAL_HL_IRQ_MASK macro to CAL_HL_IRQ_WDMA_END_MASK
and CAL_HL_IRQ_WDMA_START_MASK.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c      | 12 ++++++------
 drivers/media/platform/ti-vpe/cal_regs.h |  3 ++-
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index b9b533a4497f..01363294b882 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -453,9 +453,9 @@ void cal_ctx_start(struct cal_ctx *ctx)
 
 	/* Enable IRQ_WDMA_END and IRQ_WDMA_START. */
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(1),
-		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
+		  CAL_HL_IRQ_WDMA_END_MASK(ctx->dma_ctx));
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(2),
-		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
+		  CAL_HL_IRQ_WDMA_START_MASK(ctx->dma_ctx));
 }
 
 void cal_ctx_stop(struct cal_ctx *ctx)
@@ -479,9 +479,9 @@ void cal_ctx_stop(struct cal_ctx *ctx)
 
 	/* Disable IRQ_WDMA_END and IRQ_WDMA_START. */
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(1),
-		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
+		  CAL_HL_IRQ_WDMA_END_MASK(ctx->dma_ctx));
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_CLR(2),
-		  CAL_HL_IRQ_MASK(ctx->dma_ctx));
+		  CAL_HL_IRQ_WDMA_START_MASK(ctx->dma_ctx));
 
 	ctx->dma.state = CAL_DMA_STOPPED;
 }
@@ -589,7 +589,7 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 		cal_write(cal, CAL_HL_IRQSTATUS(1), status);
 
 		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
-			if (status & CAL_HL_IRQ_MASK(i))
+			if (status & CAL_HL_IRQ_WDMA_END_MASK(i))
 				cal_irq_wdma_end(cal->ctx[i]);
 		}
 	}
@@ -603,7 +603,7 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 		cal_write(cal, CAL_HL_IRQSTATUS(2), status);
 
 		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
-			if (status & CAL_HL_IRQ_MASK(i))
+			if (status & CAL_HL_IRQ_WDMA_START_MASK(i))
 				cal_irq_wdma_start(cal->ctx[i]);
 		}
 	}
diff --git a/drivers/media/platform/ti-vpe/cal_regs.h b/drivers/media/platform/ti-vpe/cal_regs.h
index bf937919a1e9..94cb4f329cf3 100644
--- a/drivers/media/platform/ti-vpe/cal_regs.h
+++ b/drivers/media/platform/ti-vpe/cal_regs.h
@@ -125,7 +125,8 @@
 #define CAL_HL_IRQ_EOI_LINE_NUMBER_READ0		0
 #define CAL_HL_IRQ_EOI_LINE_NUMBER_EOI0			0
 
-#define CAL_HL_IRQ_MASK(m)			BIT(m)
+#define CAL_HL_IRQ_WDMA_END_MASK(m)		BIT(m)
+#define CAL_HL_IRQ_WDMA_START_MASK(m)		BIT(m)
 
 #define CAL_HL_IRQ_OCPO_ERR_MASK		BIT(6)
 
-- 
2.25.1

[PATCH v3 13/38] media: ti-vpe: cal: clean up CAL_CSI2_VC_IRQ_* macros

[Tomi Valkeinen]

The macros related to CAL_CSI2_VC_IRQ can be handled better by having
the VC number as a macro parameter.

Note that the macros are not used anywhere yet, so no other changes are
needed.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal_regs.h | 30 +++++-------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal_regs.h b/drivers/media/platform/ti-vpe/cal_regs.h
index 94cb4f329cf3..7eeceeeb303e 100644
--- a/drivers/media/platform/ti-vpe/cal_regs.h
+++ b/drivers/media/platform/ti-vpe/cal_regs.h
@@ -406,30 +406,12 @@
 #define CAL_CSI2_TIMING_STOP_STATE_X16_IO1_MASK		BIT(14)
 #define CAL_CSI2_TIMING_FORCE_RX_MODE_IO1_MASK		BIT(15)
 
-#define CAL_CSI2_VC_IRQ_FS_IRQ_0_MASK			BIT(0)
-#define CAL_CSI2_VC_IRQ_FE_IRQ_0_MASK			BIT(1)
-#define CAL_CSI2_VC_IRQ_LS_IRQ_0_MASK			BIT(2)
-#define CAL_CSI2_VC_IRQ_LE_IRQ_0_MASK			BIT(3)
-#define CAL_CSI2_VC_IRQ_CS_IRQ_0_MASK			BIT(4)
-#define CAL_CSI2_VC_IRQ_ECC_CORRECTION0_IRQ_0_MASK	BIT(5)
-#define CAL_CSI2_VC_IRQ_FS_IRQ_1_MASK			BIT(8)
-#define CAL_CSI2_VC_IRQ_FE_IRQ_1_MASK			BIT(9)
-#define CAL_CSI2_VC_IRQ_LS_IRQ_1_MASK			BIT(10)
-#define CAL_CSI2_VC_IRQ_LE_IRQ_1_MASK			BIT(11)
-#define CAL_CSI2_VC_IRQ_CS_IRQ_1_MASK			BIT(12)
-#define CAL_CSI2_VC_IRQ_ECC_CORRECTION0_IRQ_1_MASK	BIT(13)
-#define CAL_CSI2_VC_IRQ_FS_IRQ_2_MASK			BIT(16)
-#define CAL_CSI2_VC_IRQ_FE_IRQ_2_MASK			BIT(17)
-#define CAL_CSI2_VC_IRQ_LS_IRQ_2_MASK			BIT(18)
-#define CAL_CSI2_VC_IRQ_LE_IRQ_2_MASK			BIT(19)
-#define CAL_CSI2_VC_IRQ_CS_IRQ_2_MASK			BIT(20)
-#define CAL_CSI2_VC_IRQ_ECC_CORRECTION0_IRQ_2_MASK	BIT(21)
-#define CAL_CSI2_VC_IRQ_FS_IRQ_3_MASK			BIT(24)
-#define CAL_CSI2_VC_IRQ_FE_IRQ_3_MASK			BIT(25)
-#define CAL_CSI2_VC_IRQ_LS_IRQ_3_MASK			BIT(26)
-#define CAL_CSI2_VC_IRQ_LE_IRQ_3_MASK			BIT(27)
-#define CAL_CSI2_VC_IRQ_CS_IRQ_3_MASK			BIT(28)
-#define CAL_CSI2_VC_IRQ_ECC_CORRECTION0_IRQ_3_MASK	BIT(29)
+#define CAL_CSI2_VC_IRQ_FS_IRQ_MASK(n)			BIT(0 + ((n) * 8))
+#define CAL_CSI2_VC_IRQ_FE_IRQ_MASK(n)			BIT(1 + ((n) * 8))
+#define CAL_CSI2_VC_IRQ_LS_IRQ_MASK(n)			BIT(2 + ((n) * 8))
+#define CAL_CSI2_VC_IRQ_LE_IRQ_MASK(n)			BIT(3 + ((n) * 8))
+#define CAL_CSI2_VC_IRQ_CS_IRQ_MASK(n)			BIT(4 + ((n) * 8))
+#define CAL_CSI2_VC_IRQ_ECC_CORRECTION_IRQ_MASK(n)	BIT(5 + ((n) * 8))
 
 #define CAL_CSI2_CTX_DT_MASK		GENMASK(5, 0)
 #define CAL_CSI2_CTX_VC_MASK		GENMASK(7, 6)
-- 
2.25.1

[PATCH v3 14/38] media: ti-vpe: cal: catch VC errors

[Tomi Valkeinen]

CAL driver currently ignores VC related errors. To help catch error
conditions, enable all the VC error interrupts and handle them in the
interrupt handler by printing an error.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 25 ++++++++++++++++----
 drivers/media/platform/ti-vpe/cal.c          | 10 ++++++++
 2 files changed, 31 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 3bc63a5e9317..b36e55b63718 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -226,24 +226,41 @@ static void cal_camerarx_enable_irqs(struct cal_camerarx *phy)
 		CAL_CSI2_COMPLEXIO_IRQ_FIFO_OVR_MASK |
 		CAL_CSI2_COMPLEXIO_IRQ_SHORT_PACKET_MASK |
 		CAL_CSI2_COMPLEXIO_IRQ_ECC_NO_CORRECTION_MASK;
-
-	/* Enable CIO error IRQs. */
+	const u32 vc_err_mask =
+		CAL_CSI2_VC_IRQ_CS_IRQ_MASK(0) |
+		CAL_CSI2_VC_IRQ_CS_IRQ_MASK(1) |
+		CAL_CSI2_VC_IRQ_CS_IRQ_MASK(2) |
+		CAL_CSI2_VC_IRQ_CS_IRQ_MASK(3) |
+		CAL_CSI2_VC_IRQ_ECC_CORRECTION_IRQ_MASK(0) |
+		CAL_CSI2_VC_IRQ_ECC_CORRECTION_IRQ_MASK(1) |
+		CAL_CSI2_VC_IRQ_ECC_CORRECTION_IRQ_MASK(2) |
+		CAL_CSI2_VC_IRQ_ECC_CORRECTION_IRQ_MASK(3);
+
+	/* Enable CIO & VC error IRQs. */
 	cal_write(phy->cal, CAL_HL_IRQENABLE_SET(0),
-		  CAL_HL_IRQ_CIO_MASK(phy->instance));
+		  CAL_HL_IRQ_CIO_MASK(phy->instance) |
+		  CAL_HL_IRQ_VC_MASK(phy->instance));
 	cal_write(phy->cal, CAL_CSI2_COMPLEXIO_IRQENABLE(phy->instance),
 		  cio_err_mask);
+	cal_write(phy->cal, CAL_CSI2_VC_IRQENABLE(phy->instance),
+		  vc_err_mask);
 }
 
 static void cal_camerarx_disable_irqs(struct cal_camerarx *phy)
 {
 	/* Disable CIO error irqs */
 	cal_write(phy->cal, CAL_HL_IRQENABLE_CLR(0),
-		  CAL_HL_IRQ_CIO_MASK(phy->instance));
+		  CAL_HL_IRQ_CIO_MASK(phy->instance) |
+		  CAL_HL_IRQ_VC_MASK(phy->instance));
 	cal_write(phy->cal, CAL_CSI2_COMPLEXIO_IRQENABLE(phy->instance), 0);
+	cal_write(phy->cal, CAL_CSI2_VC_IRQENABLE(phy->instance), 0);
 }
 
 static void cal_camerarx_ppi_enable(struct cal_camerarx *phy)
 {
+	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
+			1, CAL_CSI2_PPI_CTRL_ECC_EN_MASK);
+
 	cal_write_field(phy->cal, CAL_CSI2_PPI_CTRL(phy->instance),
 			1, CAL_CSI2_PPI_CTRL_IF_EN_MASK);
 }
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 01363294b882..0568677674b4 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -577,6 +577,16 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 				cal_write(cal, CAL_CSI2_COMPLEXIO_IRQSTATUS(i),
 					  cio_stat);
 			}
+
+			if (status & CAL_HL_IRQ_VC_MASK(i)) {
+				u32 vc_stat = cal_read(cal, CAL_CSI2_VC_IRQSTATUS(i));
+
+				dev_err_ratelimited(cal->dev,
+						    "CIO%u VC error: %#08x\n",
+						    i, vc_stat);
+
+				cal_write(cal, CAL_CSI2_VC_IRQSTATUS(i), vc_stat);
+			}
 		}
 	}
 
-- 
2.25.1

[PATCH v3 15/38] media: ti-vpe: cal: remove wait when stopping camerarx

[Tomi Valkeinen]

Asserting ComplexIO reset seems to affect the HW (ie. asserting reset
will break an active capture), but the RESET_DONE bit never changes to
"reset is ongoing" state. Thus we always get a timeout.

Drop the wait, as it seems to achieve nothing.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index b36e55b63718..a8cca30f3f51 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -407,7 +407,6 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 
 static void cal_camerarx_stop(struct cal_camerarx *phy)
 {
-	unsigned int i;
 	int ret;
 
 	cal_camerarx_ppi_disable(phy);
@@ -421,19 +420,9 @@ static void cal_camerarx_stop(struct cal_camerarx *phy)
 			CAL_CSI2_COMPLEXIO_CFG_RESET_CTRL,
 			CAL_CSI2_COMPLEXIO_CFG_RESET_CTRL_MASK);
 
-	/* Wait for power down completion */
-	for (i = 0; i < 10; i++) {
-		if (cal_read_field(phy->cal,
-				   CAL_CSI2_COMPLEXIO_CFG(phy->instance),
-				   CAL_CSI2_COMPLEXIO_CFG_RESET_DONE_MASK) ==
-		    CAL_CSI2_COMPLEXIO_CFG_RESET_DONE_RESETONGOING)
-			break;
-		usleep_range(1000, 1100);
-	}
-	phy_dbg(3, phy, "CAL_CSI2_COMPLEXIO_CFG(%d) = 0x%08x Complex IO in Reset (%d) %s\n",
+	phy_dbg(3, phy, "CAL_CSI2_COMPLEXIO_CFG(%d) = 0x%08x Complex IO in Reset\n",
 		phy->instance,
-		cal_read(phy->cal, CAL_CSI2_COMPLEXIO_CFG(phy->instance)), i,
-		(i >= 10) ? "(timeout)" : "");
+		cal_read(phy->cal, CAL_CSI2_COMPLEXIO_CFG(phy->instance)));
 
 	/* Disable the phy */
 	cal_camerarx_disable(phy);
-- 
2.25.1

Re: [PATCH v3 15/38] media: ti-vpe: cal: remove wait when stopping camerarx

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:08:46PM +0300, Tomi Valkeinen wrote:
> Asserting ComplexIO reset seems to affect the HW (ie. asserting reset
> will break an active capture), but the RESET_DONE bit never changes to
> "reset is ongoing" state. Thus we always get a timeout.
> 
> Drop the wait, as it seems to achieve nothing.

I would be nice if this could get double-check by someone at TI with
more information than we have regarding the hardware implementation.

> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 15 ++-------------
>  1 file changed, 2 insertions(+), 13 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index b36e55b63718..a8cca30f3f51 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -407,7 +407,6 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
>  
>  static void cal_camerarx_stop(struct cal_camerarx *phy)
>  {
> -	unsigned int i;
>  	int ret;
>  
>  	cal_camerarx_ppi_disable(phy);
> @@ -421,19 +420,9 @@ static void cal_camerarx_stop(struct cal_camerarx *phy)
>  			CAL_CSI2_COMPLEXIO_CFG_RESET_CTRL,
>  			CAL_CSI2_COMPLEXIO_CFG_RESET_CTRL_MASK);
>  
> -	/* Wait for power down completion */
> -	for (i = 0; i < 10; i++) {
> -		if (cal_read_field(phy->cal,
> -				   CAL_CSI2_COMPLEXIO_CFG(phy->instance),
> -				   CAL_CSI2_COMPLEXIO_CFG_RESET_DONE_MASK) ==
> -		    CAL_CSI2_COMPLEXIO_CFG_RESET_DONE_RESETONGOING)
> -			break;
> -		usleep_range(1000, 1100);
> -	}
> -	phy_dbg(3, phy, "CAL_CSI2_COMPLEXIO_CFG(%d) = 0x%08x Complex IO in Reset (%d) %s\n",
> +	phy_dbg(3, phy, "CAL_CSI2_COMPLEXIO_CFG(%d) = 0x%08x Complex IO in Reset\n",
>  		phy->instance,
> -		cal_read(phy->cal, CAL_CSI2_COMPLEXIO_CFG(phy->instance)), i,
> -		(i >= 10) ? "(timeout)" : "");
> +		cal_read(phy->cal, CAL_CSI2_COMPLEXIO_CFG(phy->instance)));
>  
>  	/* Disable the phy */
>  	cal_camerarx_disable(phy);

-- 
Regards,

Laurent Pinchart

[PATCH v3 16/38] media: ti-vpe: cal: disable csi2 ctx and pix proc at ctx_stop

[Tomi Valkeinen]

Disable CSI2 context and pix proc in cal_ctx_stop() to ensure they are
not used if the same context is used later on a different PHY or without
pix proc.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 0568677674b4..f47d2fa31a39 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -484,6 +484,12 @@ void cal_ctx_stop(struct cal_ctx *ctx)
 		  CAL_HL_IRQ_WDMA_START_MASK(ctx->dma_ctx));
 
 	ctx->dma.state = CAL_DMA_STOPPED;
+
+	/* Disable CSI2 context */
+	cal_write(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx), 0);
+
+	/* Disable pix proc */
+	cal_write(ctx->cal, CAL_PIX_PROC(ctx->pix_proc), 0);
 }
 
 /* ------------------------------------------------------------------
-- 
2.25.1

[PATCH v3 17/38] media: ti-vpe: cal: allocate pix proc dynamically

[Tomi Valkeinen]

CAL has 4 pixel processing units but the units are not needed e.g. for
metadata. As we could be capturing 4 pixel streams and 4 metadata
streams, i.e. using 8 DMA contexts, we cannot assign a pixel processing
unit to every DMA context. Instead we need to reserve a pixel processing
unit only when needed.

Add functions to reserve and release a pix proc unit, and use them in
cal_ctx_prepare/unprepare. Note that for the time being we still always
reserve a pix proc unit.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 44 +++++++++++++++++++++++++++--
 drivers/media/platform/ti-vpe/cal.h |  2 ++
 2 files changed, 44 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index f47d2fa31a39..5ab87c9d3707 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -290,6 +290,37 @@ void cal_quickdump_regs(struct cal_dev *cal)
  * ------------------------------------------------------------------
  */
 
+#define CAL_MAX_PIX_PROC 4
+
+static int cal_reserve_pix_proc(struct cal_dev *cal)
+{
+	unsigned long ret;
+
+	spin_lock(&cal->v4l2_dev.lock);
+
+	ret = find_first_zero_bit(&cal->reserved_pix_proc_mask, CAL_MAX_PIX_PROC);
+
+	if (ret == CAL_MAX_PIX_PROC) {
+		spin_unlock(&cal->v4l2_dev.lock);
+		return -ENOSPC;
+	}
+
+	cal->reserved_pix_proc_mask |= BIT(ret);
+
+	spin_unlock(&cal->v4l2_dev.lock);
+
+	return ret;
+}
+
+static void cal_release_pix_proc(struct cal_dev *cal, unsigned int pix_proc_num)
+{
+	spin_lock(&cal->v4l2_dev.lock);
+
+	cal->reserved_pix_proc_mask &= ~BIT(pix_proc_num);
+
+	spin_unlock(&cal->v4l2_dev.lock);
+}
+
 static void cal_ctx_csi2_config(struct cal_ctx *ctx)
 {
 	u32 val;
@@ -433,12 +464,22 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
 
 int cal_ctx_prepare(struct cal_ctx *ctx)
 {
+	int ret;
+
+	ret = cal_reserve_pix_proc(ctx->cal);
+	if (ret < 0) {
+		ctx_err(ctx, "Failed to reserve pix proc: %d\n", ret);
+		return ret;
+	}
+
+	ctx->pix_proc = ret;
+
 	return 0;
 }
 
 void cal_ctx_unprepare(struct cal_ctx *ctx)
 {
-
+	cal_release_pix_proc(ctx->cal, ctx->pix_proc);
 }
 
 void cal_ctx_start(struct cal_ctx *ctx)
@@ -873,7 +914,6 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 	ctx->dma_ctx = inst;
 	ctx->csi2_ctx = inst;
 	ctx->cport = inst;
-	ctx->pix_proc = inst;
 
 	ret = cal_ctx_v4l2_init(ctx);
 	if (ret)
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index e4db2a905f68..5d8b3193be7d 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -188,6 +188,8 @@ struct cal_dev {
 	struct media_device	mdev;
 	struct v4l2_device	v4l2_dev;
 	struct v4l2_async_notifier notifier;
+
+	unsigned long		reserved_pix_proc_mask;
 };
 
 /*
-- 
2.25.1

[PATCH v3 18/38] media: ti-vpe: cal: add 'use_pix_proc' field

[Tomi Valkeinen]

We already have functions to reserve and release a pix proc unit, but we
always reserve such and the code expects the pix proc unit to be used.

Add a new field, 'use_pix_proc', to indicate if the pix prox unit has
been reserved and should be used. Use the flag to skip programming pix
proc unit when not needed.

Note that we still always set the use_pix_proc flag to true.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 10 +++++++---
 drivers/media/platform/ti-vpe/cal.h |  2 ++
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 5ab87c9d3707..c1a6e23ff713 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -473,13 +473,15 @@ int cal_ctx_prepare(struct cal_ctx *ctx)
 	}
 
 	ctx->pix_proc = ret;
+	ctx->use_pix_proc = true;
 
 	return 0;
 }
 
 void cal_ctx_unprepare(struct cal_ctx *ctx)
 {
-	cal_release_pix_proc(ctx->cal, ctx->pix_proc);
+	if (ctx->use_pix_proc)
+		cal_release_pix_proc(ctx->cal, ctx->pix_proc);
 }
 
 void cal_ctx_start(struct cal_ctx *ctx)
@@ -489,7 +491,8 @@ void cal_ctx_start(struct cal_ctx *ctx)
 
 	/* Configure the CSI-2, pixel processing and write DMA contexts. */
 	cal_ctx_csi2_config(ctx);
-	cal_ctx_pix_proc_config(ctx);
+	if (ctx->use_pix_proc)
+		cal_ctx_pix_proc_config(ctx);
 	cal_ctx_wr_dma_config(ctx);
 
 	/* Enable IRQ_WDMA_END and IRQ_WDMA_START. */
@@ -530,7 +533,8 @@ void cal_ctx_stop(struct cal_ctx *ctx)
 	cal_write(ctx->cal, CAL_CSI2_CTX(ctx->phy->instance, ctx->csi2_ctx), 0);
 
 	/* Disable pix proc */
-	cal_write(ctx->cal, CAL_PIX_PROC(ctx->pix_proc), 0);
+	if (ctx->use_pix_proc)
+		cal_write(ctx->cal, CAL_PIX_PROC(ctx->pix_proc), 0);
 }
 
 /* ------------------------------------------------------------------
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 5d8b3193be7d..d7cc399f47da 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -223,6 +223,8 @@ struct cal_ctx {
 	u8			cport;
 	u8			csi2_ctx;
 	u8			pix_proc;
+
+	bool			use_pix_proc;
 };
 
 extern unsigned int cal_debug;
-- 
2.25.1

[PATCH v3 19/38] media: ti-vpe: cal: add cal_ctx_wr_dma_enable and fix a race

[Tomi Valkeinen]

I have not noticed any errors due to this, but the DMA configuration
looks racy. Setting the DMA mode bitfield in CAL_WR_DMA_CTRL supposedly
enables the DMA. However, the driver currently a) continues configuring
the DMA after setting the mode, and b) enables the DMA interrupts only
after setting the mode.

This probably doesn't cause any issues as there should be no data coming
in to the DMA yet, but it's still better to fix this.

Add a new function, cal_ctx_wr_dma_enable(), to set the DMA mode field,
and call that function only after the DMA config and the irq enabling
has been done.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index c1a6e23ff713..778bef9bbdc4 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -409,8 +409,6 @@ static void cal_ctx_wr_dma_config(struct cal_ctx *ctx)
 		      CAL_WR_DMA_CTRL_YSIZE_MASK);
 	cal_set_field(&val, CAL_WR_DMA_CTRL_DTAG_PIX_DAT,
 		      CAL_WR_DMA_CTRL_DTAG_MASK);
-	cal_set_field(&val, CAL_WR_DMA_CTRL_MODE_CONST,
-		      CAL_WR_DMA_CTRL_MODE_MASK);
 	cal_set_field(&val, CAL_WR_DMA_CTRL_PATTERN_LINEAR,
 		      CAL_WR_DMA_CTRL_PATTERN_MASK);
 	cal_set_field(&val, 1, CAL_WR_DMA_CTRL_STALL_RD_MASK);
@@ -442,6 +440,15 @@ void cal_ctx_set_dma_addr(struct cal_ctx *ctx, dma_addr_t addr)
 	cal_write(ctx->cal, CAL_WR_DMA_ADDR(ctx->dma_ctx), addr);
 }
 
+static void cal_ctx_wr_dma_enable(struct cal_ctx *ctx)
+{
+	u32 val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx));
+
+	cal_set_field(&val, CAL_WR_DMA_CTRL_MODE_CONST,
+		      CAL_WR_DMA_CTRL_MODE_MASK);
+	cal_write(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx), val);
+}
+
 static void cal_ctx_wr_dma_disable(struct cal_ctx *ctx)
 {
 	u32 val = cal_read(ctx->cal, CAL_WR_DMA_CTRL(ctx->dma_ctx));
@@ -500,6 +507,8 @@ void cal_ctx_start(struct cal_ctx *ctx)
 		  CAL_HL_IRQ_WDMA_END_MASK(ctx->dma_ctx));
 	cal_write(ctx->cal, CAL_HL_IRQENABLE_SET(2),
 		  CAL_HL_IRQ_WDMA_START_MASK(ctx->dma_ctx));
+
+	cal_ctx_wr_dma_enable(ctx);
 }
 
 void cal_ctx_stop(struct cal_ctx *ctx)
-- 
2.25.1

[PATCH v3 20/38] media: ti-vpe: cal: add vc and datatype fields to cal_ctx

[Tomi Valkeinen]

In preparation for supporting multiple virtual channels and datatypes,
add vc and datatype fields to cal_ctx, initialize them to the currently
used values, and use those fields when writing to the register.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c      | 6 ++++--
 drivers/media/platform/ti-vpe/cal.h      | 2 ++
 drivers/media/platform/ti-vpe/cal_regs.h | 2 ++
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 778bef9bbdc4..ba8821a3b262 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -335,8 +335,8 @@ static void cal_ctx_csi2_config(struct cal_ctx *ctx)
 	 *  0x2A: RAW8   1 pixel  = 1 byte
 	 *  0x1E: YUV422 2 pixels = 4 bytes
 	 */
-	cal_set_field(&val, 0x1, CAL_CSI2_CTX_DT_MASK);
-	cal_set_field(&val, 0, CAL_CSI2_CTX_VC_MASK);
+	cal_set_field(&val, ctx->datatype, CAL_CSI2_CTX_DT_MASK);
+	cal_set_field(&val, ctx->vc, CAL_CSI2_CTX_VC_MASK);
 	cal_set_field(&val, ctx->v_fmt.fmt.pix.height, CAL_CSI2_CTX_LINES_MASK);
 	cal_set_field(&val, CAL_CSI2_CTX_ATT_PIX, CAL_CSI2_CTX_ATT_MASK);
 	cal_set_field(&val, CAL_CSI2_CTX_PACK_MODE_LINE,
@@ -927,6 +927,8 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 	ctx->dma_ctx = inst;
 	ctx->csi2_ctx = inst;
 	ctx->cport = inst;
+	ctx->vc = 0;
+	ctx->datatype = CAL_CSI2_CTX_DT_ANY;
 
 	ret = cal_ctx_v4l2_init(ctx);
 	if (ret)
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index d7cc399f47da..def0c9a3657d 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -223,6 +223,8 @@ struct cal_ctx {
 	u8			cport;
 	u8			csi2_ctx;
 	u8			pix_proc;
+	u8			vc;
+	u8			datatype;
 
 	bool			use_pix_proc;
 };
diff --git a/drivers/media/platform/ti-vpe/cal_regs.h b/drivers/media/platform/ti-vpe/cal_regs.h
index 7eeceeeb303e..40e4f972fcb7 100644
--- a/drivers/media/platform/ti-vpe/cal_regs.h
+++ b/drivers/media/platform/ti-vpe/cal_regs.h
@@ -414,6 +414,8 @@
 #define CAL_CSI2_VC_IRQ_ECC_CORRECTION_IRQ_MASK(n)	BIT(5 + ((n) * 8))
 
 #define CAL_CSI2_CTX_DT_MASK		GENMASK(5, 0)
+#define CAL_CSI2_CTX_DT_DISABLED	0
+#define CAL_CSI2_CTX_DT_ANY		1
 #define CAL_CSI2_CTX_VC_MASK		GENMASK(7, 6)
 #define CAL_CSI2_CTX_CPORT_MASK		GENMASK(12, 8)
 #define CAL_CSI2_CTX_ATT_MASK		BIT(13)
-- 
2.25.1

[PATCH v3 21/38] media: ti-vpe: cal: handle cal_ctx_v4l2_register error

[Tomi Valkeinen]

cal_async_notifier_complete() doesn't handle errors returned from
cal_ctx_v4l2_register(). Add the error handling.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index ba8821a3b262..9e051c2e84a9 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -743,8 +743,12 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
 	int ret = 0;
 
 	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
-		if (cal->ctx[i])
-			cal_ctx_v4l2_register(cal->ctx[i]);
+		if (!cal->ctx[i])
+			continue;
+
+		ret = cal_ctx_v4l2_register(cal->ctx[i]);
+		if (ret)
+			return ret;
 	}
 
 	if (cal_mc_api)
-- 
2.25.1

Re: [PATCH v3 21/38] media: ti-vpe: cal: handle cal_ctx_v4l2_register error

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:08:52PM +0300, Tomi Valkeinen wrote:
> cal_async_notifier_complete() doesn't handle errors returned from
> cal_ctx_v4l2_register(). Add the error handling.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index ba8821a3b262..9e051c2e84a9 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -743,8 +743,12 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>  	int ret = 0;
>  
>  	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> -		if (cal->ctx[i])
> -			cal_ctx_v4l2_register(cal->ctx[i]);
> +		if (!cal->ctx[i])
> +			continue;
> +
> +		ret = cal_ctx_v4l2_register(cal->ctx[i]);
> +		if (ret)
> +			return ret;

This part looks good, so

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Don't we need to call cal_ctx_v4l2_unregister() in the error path of
cal_async_notifier_register() though ?

>  	}
>  
>  	if (cal_mc_api)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 21/38] media: ti-vpe: cal: handle cal_ctx_v4l2_register error

[Tomi Valkeinen]

On 04/06/2021 16:47, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Mon, May 24, 2021 at 02:08:52PM +0300, Tomi Valkeinen wrote:
>> cal_async_notifier_complete() doesn't handle errors returned from
>> cal_ctx_v4l2_register(). Add the error handling.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/platform/ti-vpe/cal.c | 8 ++++++--
>>   1 file changed, 6 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>> index ba8821a3b262..9e051c2e84a9 100644
>> --- a/drivers/media/platform/ti-vpe/cal.c
>> +++ b/drivers/media/platform/ti-vpe/cal.c
>> @@ -743,8 +743,12 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>>   	int ret = 0;
>>   
>>   	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
>> -		if (cal->ctx[i])
>> -			cal_ctx_v4l2_register(cal->ctx[i]);
>> +		if (!cal->ctx[i])
>> +			continue;
>> +
>> +		ret = cal_ctx_v4l2_register(cal->ctx[i]);
>> +		if (ret)
>> +			return ret;
> 
> This part looks good, so
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Don't we need to call cal_ctx_v4l2_unregister() in the error path of
> cal_async_notifier_register() though ?

Hmm, can you elaborate? I don't understand where and why we need to call 
the unregister.

  Tomi

Re: [PATCH v3 21/38] media: ti-vpe: cal: handle cal_ctx_v4l2_register error

[Laurent Pinchart]

Hi Tomi,

On Mon, Jun 07, 2021 at 10:44:17AM +0300, Tomi Valkeinen wrote:
> On 04/06/2021 16:47, Laurent Pinchart wrote:
> > On Mon, May 24, 2021 at 02:08:52PM +0300, Tomi Valkeinen wrote:
> >> cal_async_notifier_complete() doesn't handle errors returned from
> >> cal_ctx_v4l2_register(). Add the error handling.
> >>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >>   drivers/media/platform/ti-vpe/cal.c | 8 ++++++--
> >>   1 file changed, 6 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> >> index ba8821a3b262..9e051c2e84a9 100644
> >> --- a/drivers/media/platform/ti-vpe/cal.c
> >> +++ b/drivers/media/platform/ti-vpe/cal.c
> >> @@ -743,8 +743,12 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
> >>   	int ret = 0;
> >>   
> >>   	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> >> -		if (cal->ctx[i])
> >> -			cal_ctx_v4l2_register(cal->ctx[i]);
> >> +		if (!cal->ctx[i])
> >> +			continue;
> >> +
> >> +		ret = cal_ctx_v4l2_register(cal->ctx[i]);
> >> +		if (ret)
> >> +			return ret;
> > 
> > This part looks good, so
> > 
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> > Don't we need to call cal_ctx_v4l2_unregister() in the error path of
> > cal_async_notifier_register() though ?
> 
> Hmm, can you elaborate? I don't understand where and why we need to call 
> the unregister.

cal_async_notifier_complete() can be called synchronously by
v4l2_async_notifier_register() if all async subdevs are present. If
cal_ctx_v4l2_register() fails for one contexts, the failure will be
propagated to cal_async_notifier_register(), then to
cal_media_register(), and cal_probe(). Unless I'm mistaken, the contexts
for which cal_ctx_v4l2_register() succeeded will not be cleaned
properly.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 21/38] media: ti-vpe: cal: handle cal_ctx_v4l2_register error

[Tomi Valkeinen]

On 07/06/2021 11:00, Laurent Pinchart wrote:
> Hi Tomi,
> 
> On Mon, Jun 07, 2021 at 10:44:17AM +0300, Tomi Valkeinen wrote:
>> On 04/06/2021 16:47, Laurent Pinchart wrote:
>>> On Mon, May 24, 2021 at 02:08:52PM +0300, Tomi Valkeinen wrote:
>>>> cal_async_notifier_complete() doesn't handle errors returned from
>>>> cal_ctx_v4l2_register(). Add the error handling.
>>>>
>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>> ---
>>>>    drivers/media/platform/ti-vpe/cal.c | 8 ++++++--
>>>>    1 file changed, 6 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>>>> index ba8821a3b262..9e051c2e84a9 100644
>>>> --- a/drivers/media/platform/ti-vpe/cal.c
>>>> +++ b/drivers/media/platform/ti-vpe/cal.c
>>>> @@ -743,8 +743,12 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>>>>    	int ret = 0;
>>>>    
>>>>    	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
>>>> -		if (cal->ctx[i])
>>>> -			cal_ctx_v4l2_register(cal->ctx[i]);
>>>> +		if (!cal->ctx[i])
>>>> +			continue;
>>>> +
>>>> +		ret = cal_ctx_v4l2_register(cal->ctx[i]);
>>>> +		if (ret)
>>>> +			return ret;
>>>
>>> This part looks good, so
>>>
>>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>
>>> Don't we need to call cal_ctx_v4l2_unregister() in the error path of
>>> cal_async_notifier_register() though ?
>>
>> Hmm, can you elaborate? I don't understand where and why we need to call
>> the unregister.
> 
> cal_async_notifier_complete() can be called synchronously by
> v4l2_async_notifier_register() if all async subdevs are present. If
> cal_ctx_v4l2_register() fails for one contexts, the failure will be
> propagated to cal_async_notifier_register(), then to
> cal_media_register(), and cal_probe(). Unless I'm mistaken, the contexts
> for which cal_ctx_v4l2_register() succeeded will not be cleaned
> properly.

Right. I think this can be solved easily by unrolling in the complete callback:

@@ -748,7 +748,16 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
  
  		ret = cal_ctx_v4l2_register(cal->ctx[i]);
  		if (ret)
-			return ret;
+			break;
+	}
+
+	if (ret) {
+		unsigned int j;
+
+		for (j = 0; j < i; ++j)
+			cal_ctx_v4l2_unregister(cal->ctx[j]);
+
+		return ret;
  	}
  
  	if (cal_mc_api)

I'll squash this diff to this patch.

  Tomi

Re: [PATCH v3 21/38] media: ti-vpe: cal: handle cal_ctx_v4l2_register error

[Laurent Pinchart]

Hi Tomi,

On Mon, Jun 07, 2021 at 11:53:54AM +0300, Tomi Valkeinen wrote:
> On 07/06/2021 11:00, Laurent Pinchart wrote:
> > On Mon, Jun 07, 2021 at 10:44:17AM +0300, Tomi Valkeinen wrote:
> >> On 04/06/2021 16:47, Laurent Pinchart wrote:
> >>> On Mon, May 24, 2021 at 02:08:52PM +0300, Tomi Valkeinen wrote:
> >>>> cal_async_notifier_complete() doesn't handle errors returned from
> >>>> cal_ctx_v4l2_register(). Add the error handling.
> >>>>
> >>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >>>> ---
> >>>>    drivers/media/platform/ti-vpe/cal.c | 8 ++++++--
> >>>>    1 file changed, 6 insertions(+), 2 deletions(-)
> >>>>
> >>>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> >>>> index ba8821a3b262..9e051c2e84a9 100644
> >>>> --- a/drivers/media/platform/ti-vpe/cal.c
> >>>> +++ b/drivers/media/platform/ti-vpe/cal.c
> >>>> @@ -743,8 +743,12 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
> >>>>    	int ret = 0;
> >>>>    
> >>>>    	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> >>>> -		if (cal->ctx[i])
> >>>> -			cal_ctx_v4l2_register(cal->ctx[i]);
> >>>> +		if (!cal->ctx[i])
> >>>> +			continue;
> >>>> +
> >>>> +		ret = cal_ctx_v4l2_register(cal->ctx[i]);
> >>>> +		if (ret)
> >>>> +			return ret;
> >>>
> >>> This part looks good, so
> >>>
> >>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>>
> >>> Don't we need to call cal_ctx_v4l2_unregister() in the error path of
> >>> cal_async_notifier_register() though ?
> >>
> >> Hmm, can you elaborate? I don't understand where and why we need to call
> >> the unregister.
> > 
> > cal_async_notifier_complete() can be called synchronously by
> > v4l2_async_notifier_register() if all async subdevs are present. If
> > cal_ctx_v4l2_register() fails for one contexts, the failure will be
> > propagated to cal_async_notifier_register(), then to
> > cal_media_register(), and cal_probe(). Unless I'm mistaken, the contexts
> > for which cal_ctx_v4l2_register() succeeded will not be cleaned
> > properly.
> 
> Right. I think this can be solved easily by unrolling in the complete callback:
> 
> @@ -748,7 +748,16 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>   
>   		ret = cal_ctx_v4l2_register(cal->ctx[i]);
>   		if (ret)
> -			return ret;
> +			break;
> +	}
> +
> +	if (ret) {
> +		unsigned int j;
> +
> +		for (j = 0; j < i; ++j)
> +			cal_ctx_v4l2_unregister(cal->ctx[j]);

This could also be written

		for ( ; i > 0; --i)
			cal_ctx_v4l2_unregister(cal->ctx[i-1]);

to avoid an additional variable, it's up to you.

> +
> +		return ret;
>   	}
>   
>   	if (cal_mc_api)

You also need to cal_ctx_v4l2_unregister() if the call in the next line
fails.

> I'll squash this diff to this patch.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 21/38] media: ti-vpe: cal: handle cal_ctx_v4l2_register error

[Tomi Valkeinen]

On 09/06/2021 15:36, Laurent Pinchart wrote:
> Hi Tomi,
> 
> On Mon, Jun 07, 2021 at 11:53:54AM +0300, Tomi Valkeinen wrote:
>> On 07/06/2021 11:00, Laurent Pinchart wrote:
>>> On Mon, Jun 07, 2021 at 10:44:17AM +0300, Tomi Valkeinen wrote:
>>>> On 04/06/2021 16:47, Laurent Pinchart wrote:
>>>>> On Mon, May 24, 2021 at 02:08:52PM +0300, Tomi Valkeinen wrote:
>>>>>> cal_async_notifier_complete() doesn't handle errors returned from
>>>>>> cal_ctx_v4l2_register(). Add the error handling.
>>>>>>
>>>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>>>> ---
>>>>>>     drivers/media/platform/ti-vpe/cal.c | 8 ++++++--
>>>>>>     1 file changed, 6 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>>>>>> index ba8821a3b262..9e051c2e84a9 100644
>>>>>> --- a/drivers/media/platform/ti-vpe/cal.c
>>>>>> +++ b/drivers/media/platform/ti-vpe/cal.c
>>>>>> @@ -743,8 +743,12 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>>>>>>     	int ret = 0;
>>>>>>     
>>>>>>     	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
>>>>>> -		if (cal->ctx[i])
>>>>>> -			cal_ctx_v4l2_register(cal->ctx[i]);
>>>>>> +		if (!cal->ctx[i])
>>>>>> +			continue;
>>>>>> +
>>>>>> +		ret = cal_ctx_v4l2_register(cal->ctx[i]);
>>>>>> +		if (ret)
>>>>>> +			return ret;
>>>>>
>>>>> This part looks good, so
>>>>>
>>>>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>>>
>>>>> Don't we need to call cal_ctx_v4l2_unregister() in the error path of
>>>>> cal_async_notifier_register() though ?
>>>>
>>>> Hmm, can you elaborate? I don't understand where and why we need to call
>>>> the unregister.
>>>
>>> cal_async_notifier_complete() can be called synchronously by
>>> v4l2_async_notifier_register() if all async subdevs are present. If
>>> cal_ctx_v4l2_register() fails for one contexts, the failure will be
>>> propagated to cal_async_notifier_register(), then to
>>> cal_media_register(), and cal_probe(). Unless I'm mistaken, the contexts
>>> for which cal_ctx_v4l2_register() succeeded will not be cleaned
>>> properly.
>>
>> Right. I think this can be solved easily by unrolling in the complete callback:
>>
>> @@ -748,7 +748,16 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
>>    
>>    		ret = cal_ctx_v4l2_register(cal->ctx[i]);
>>    		if (ret)
>> -			return ret;
>> +			break;
>> +	}
>> +
>> +	if (ret) {
>> +		unsigned int j;
>> +
>> +		for (j = 0; j < i; ++j)
>> +			cal_ctx_v4l2_unregister(cal->ctx[j]);
> 
> This could also be written
> 
> 		for ( ; i > 0; --i)
> 			cal_ctx_v4l2_unregister(cal->ctx[i-1]);
> 
> to avoid an additional variable, it's up to you.

Yes, that's a bit nicer.

>> +
>> +		return ret;
>>    	}
>>    
>>    	if (cal_mc_api)
> 
> You also need to cal_ctx_v4l2_unregister() if the call in the next line
> fails.

Ah, indeed. Thanks!

  Tomi

[PATCH v3 22/38] media: ti-vpe: cal: set field always to V4L2_FIELD_NONE

[Tomi Valkeinen]

cal_camerarx_sd_set_fmt() accepts any value for the format field, but
there should be no reason to have any other value accepted than
V4L2_FIELD_NONE. So set the field always to V4L2_FIELD_NONE.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index a8cca30f3f51..f2ea2bdb9ea3 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -705,10 +705,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 	if (!fmtinfo)
 		fmtinfo = &cal_formats[0];
 
-	/*
-	 * Clamp the size, update the code. The field and colorspace are
-	 * accepted as-is.
-	 */
+	/* Clamp the size, update the code. The colorspace is accepted as-is. */
 	bpp = ALIGN(fmtinfo->bpp, 8);
 
 	format->format.width = clamp_t(unsigned int, format->format.width,
@@ -718,6 +715,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 					CAL_MIN_HEIGHT_LINES,
 					CAL_MAX_HEIGHT_LINES);
 	format->format.code = fmtinfo->code;
+	format->format.field = V4L2_FIELD_NONE;
 
 	/* Store the format and propagate it to the source pad. */
 	fmt = cal_camerarx_get_pad_format(phy, sd_state,
-- 
2.25.1

Re: [PATCH v3 22/38] media: ti-vpe: cal: set field always to V4L2_FIELD_NONE

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:08:53PM +0300, Tomi Valkeinen wrote:
> cal_camerarx_sd_set_fmt() accepts any value for the format field, but
> there should be no reason to have any other value accepted than
> V4L2_FIELD_NONE. So set the field always to V4L2_FIELD_NONE.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index a8cca30f3f51..f2ea2bdb9ea3 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -705,10 +705,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  	if (!fmtinfo)
>  		fmtinfo = &cal_formats[0];
>  
> -	/*
> -	 * Clamp the size, update the code. The field and colorspace are
> -	 * accepted as-is.
> -	 */
> +	/* Clamp the size, update the code. The colorspace is accepted as-is. */
>  	bpp = ALIGN(fmtinfo->bpp, 8);
>  
>  	format->format.width = clamp_t(unsigned int, format->format.width,
> @@ -718,6 +715,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  					CAL_MIN_HEIGHT_LINES,
>  					CAL_MAX_HEIGHT_LINES);
>  	format->format.code = fmtinfo->code;
> +	format->format.field = V4L2_FIELD_NONE;
>  
>  	/* Store the format and propagate it to the source pad. */
>  	fmt = cal_camerarx_get_pad_format(phy, sd_state,

-- 
Regards,

Laurent Pinchart

[PATCH v3 23/38] media: ti-vpe: cal: fix typo in a comment

[Tomi Valkeinen]

Fix a typo in a comment in cal_camerarx_sd_set_fmt().

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index f2ea2bdb9ea3..7940e47964af 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -698,7 +698,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 		return cal_camerarx_sd_get_fmt(sd, sd_state, format);
 
 	/*
-	 * Default to the first format is the requested media bus code isn't
+	 * Default to the first format if the requested media bus code isn't
 	 * supported.
 	 */
 	fmtinfo = cal_format_by_code(format->format.code);
-- 
2.25.1

[PATCH v3 24/38] media: ti-vpe: cal: add mbus_code support to cal_mc_enum_fmt_vid_cap

[Tomi Valkeinen]

Commit e5b6b07a1b45 ("media: v4l2: Extend VIDIOC_ENUM_FMT to support
MC-centric devices") added support to enumerate formats based on
mbus-code.

Add this feature to cal driver.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 1311adfb35a7..24d14cd7dbb1 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -437,13 +437,28 @@ static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
 static int cal_mc_enum_fmt_vid_cap(struct file *file, void  *priv,
 				   struct v4l2_fmtdesc *f)
 {
+	unsigned int i;
+	unsigned int idx;
+
 	if (f->index >= cal_num_formats)
 		return -EINVAL;
 
-	f->pixelformat = cal_formats[f->index].fourcc;
-	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	idx = 0;
 
-	return 0;
+	for (i = 0; i < cal_num_formats; ++i) {
+		if (f->mbus_code && cal_formats[i].code != f->mbus_code)
+			continue;
+
+		if (idx == f->index) {
+			f->pixelformat = cal_formats[i].fourcc;
+			f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+			return 0;
+		}
+
+		idx++;
+	}
+
+	return -EINVAL;
 }
 
 static void cal_mc_try_fmt(struct cal_ctx *ctx, struct v4l2_format *f,
-- 
2.25.1

[PATCH v3 25/38] media: ti-vpe: cal: rename non-MC funcs to cal_legacy_*

[Tomi Valkeinen]

To make it more obvious if the function in question is dealing with
media-controller API or the legacy API, rename legacy API functions to
cal_legacy_*.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 58 +++++++++++------------
 1 file changed, 29 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 24d14cd7dbb1..7e97a43c6611 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -102,8 +102,8 @@ static const struct cal_format_info *find_format_by_code(struct cal_ctx *ctx,
 	return NULL;
 }
 
-static int cal_enum_fmt_vid_cap(struct file *file, void  *priv,
-				struct v4l2_fmtdesc *f)
+static int cal_legacy_enum_fmt_vid_cap(struct file *file, void *priv,
+				       struct v4l2_fmtdesc *f)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 	const struct cal_format_info *fmtinfo;
@@ -189,8 +189,8 @@ static void cal_calc_format_size(struct cal_ctx *ctx,
 		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
 }
 
-static int cal_try_fmt_vid_cap(struct file *file, void *priv,
-			       struct v4l2_format *f)
+static int cal_legacy_try_fmt_vid_cap(struct file *file, void *priv,
+				      struct v4l2_format *f)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 	const struct cal_format_info *fmtinfo;
@@ -249,8 +249,8 @@ static int cal_try_fmt_vid_cap(struct file *file, void *priv,
 	return 0;
 }
 
-static int cal_s_fmt_vid_cap(struct file *file, void *priv,
-			     struct v4l2_format *f)
+static int cal_legacy_s_fmt_vid_cap(struct file *file, void *priv,
+				    struct v4l2_format *f)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 	struct vb2_queue *q = &ctx->vb_vidq;
@@ -266,7 +266,7 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 		return -EBUSY;
 	}
 
-	ret = cal_try_fmt_vid_cap(file, priv, f);
+	ret = cal_legacy_try_fmt_vid_cap(file, priv, f);
 	if (ret < 0)
 		return ret;
 
@@ -300,8 +300,8 @@ static int cal_s_fmt_vid_cap(struct file *file, void *priv,
 	return 0;
 }
 
-static int cal_enum_framesizes(struct file *file, void *fh,
-			       struct v4l2_frmsizeenum *fsize)
+static int cal_legacy_enum_framesizes(struct file *file, void *fh,
+				      struct v4l2_frmsizeenum *fsize)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 	const struct cal_format_info *fmtinfo;
@@ -337,8 +337,8 @@ static int cal_enum_framesizes(struct file *file, void *fh,
 	return 0;
 }
 
-static int cal_enum_input(struct file *file, void *priv,
-			  struct v4l2_input *inp)
+static int cal_legacy_enum_input(struct file *file, void *priv,
+				 struct v4l2_input *inp)
 {
 	if (inp->index > 0)
 		return -EINVAL;
@@ -348,20 +348,20 @@ static int cal_enum_input(struct file *file, void *priv,
 	return 0;
 }
 
-static int cal_g_input(struct file *file, void *priv, unsigned int *i)
+static int cal_legacy_g_input(struct file *file, void *priv, unsigned int *i)
 {
 	*i = 0;
 	return 0;
 }
 
-static int cal_s_input(struct file *file, void *priv, unsigned int i)
+static int cal_legacy_s_input(struct file *file, void *priv, unsigned int i)
 {
 	return i > 0 ? -EINVAL : 0;
 }
 
 /* timeperframe is arbitrary and continuous */
-static int cal_enum_frameintervals(struct file *file, void *priv,
-				   struct v4l2_frmivalenum *fival)
+static int cal_legacy_enum_frameintervals(struct file *file, void *priv,
+					  struct v4l2_frmivalenum *fival)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 	const struct cal_format_info *fmtinfo;
@@ -388,27 +388,27 @@ static int cal_enum_frameintervals(struct file *file, void *priv,
 	return 0;
 }
 
-static int cal_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+static int cal_legacy_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 
 	return v4l2_g_parm_cap(video_devdata(file), ctx->phy->source, a);
 }
 
-static int cal_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+static int cal_legacy_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
 {
 	struct cal_ctx *ctx = video_drvdata(file);
 
 	return v4l2_s_parm_cap(video_devdata(file), ctx->phy->source, a);
 }
 
-static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
+static const struct v4l2_ioctl_ops cal_ioctl_legacy_ops = {
 	.vidioc_querycap      = cal_querycap,
-	.vidioc_enum_fmt_vid_cap  = cal_enum_fmt_vid_cap,
+	.vidioc_enum_fmt_vid_cap  = cal_legacy_enum_fmt_vid_cap,
 	.vidioc_g_fmt_vid_cap     = cal_g_fmt_vid_cap,
-	.vidioc_try_fmt_vid_cap   = cal_try_fmt_vid_cap,
-	.vidioc_s_fmt_vid_cap     = cal_s_fmt_vid_cap,
-	.vidioc_enum_framesizes   = cal_enum_framesizes,
+	.vidioc_try_fmt_vid_cap   = cal_legacy_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap     = cal_legacy_s_fmt_vid_cap,
+	.vidioc_enum_framesizes   = cal_legacy_enum_framesizes,
 	.vidioc_reqbufs       = vb2_ioctl_reqbufs,
 	.vidioc_create_bufs   = vb2_ioctl_create_bufs,
 	.vidioc_prepare_buf   = vb2_ioctl_prepare_buf,
@@ -416,17 +416,17 @@ static const struct v4l2_ioctl_ops cal_ioctl_video_ops = {
 	.vidioc_qbuf          = vb2_ioctl_qbuf,
 	.vidioc_dqbuf         = vb2_ioctl_dqbuf,
 	.vidioc_expbuf        = vb2_ioctl_expbuf,
-	.vidioc_enum_input    = cal_enum_input,
-	.vidioc_g_input       = cal_g_input,
-	.vidioc_s_input       = cal_s_input,
-	.vidioc_enum_frameintervals = cal_enum_frameintervals,
+	.vidioc_enum_input    = cal_legacy_enum_input,
+	.vidioc_g_input       = cal_legacy_g_input,
+	.vidioc_s_input       = cal_legacy_s_input,
+	.vidioc_enum_frameintervals = cal_legacy_enum_frameintervals,
 	.vidioc_streamon      = vb2_ioctl_streamon,
 	.vidioc_streamoff     = vb2_ioctl_streamoff,
 	.vidioc_log_status    = v4l2_ctrl_log_status,
 	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
 	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
-	.vidioc_g_parm		= cal_g_parm,
-	.vidioc_s_parm		= cal_s_parm,
+	.vidioc_g_parm		= cal_legacy_g_parm,
+	.vidioc_s_parm		= cal_legacy_s_parm,
 };
 
 /* ------------------------------------------------------------------
@@ -966,7 +966,7 @@ int cal_ctx_v4l2_init(struct cal_ctx *ctx)
 	vfd->queue = q;
 	snprintf(vfd->name, sizeof(vfd->name), "CAL output %u", ctx->dma_ctx);
 	vfd->release = video_device_release_empty;
-	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_video_ops;
+	vfd->ioctl_ops = cal_mc_api ? &cal_ioctl_mc_ops : &cal_ioctl_legacy_ops;
 	vfd->lock = &ctx->mutex;
 	video_set_drvdata(vfd, ctx);
 
-- 
2.25.1

[PATCH v3 26/38] media: ti-vpe: cal: init ctx->v_fmt correctly in MC mode

[Tomi Valkeinen]

CAL driver enumerates mbus codes in the connected subdev to create a
list of supported formats reported to userspace, and initializes
ctx->v_fmt and ctx->fmtinfo to one of those formats.

This works fine for legacy mode, but is not correct for MC mode, and the
list is not even used in MC mode.

Fix this by adding a new function, cal_ctx_v4l2_init_mc_format, which
only initializes ctx->v_fmt and ctx->fmtinfo to a default value.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 43 ++++++++++++++++++++---
 drivers/media/platform/ti-vpe/cal.h       |  2 +-
 2 files changed, 40 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 7e97a43c6611..d06488cb8c36 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -879,24 +879,59 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
 	return 0;
 }
 
+static int cal_ctx_v4l2_init_mc_format(struct cal_ctx *ctx)
+{
+	const struct cal_format_info *fmtinfo;
+	struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
+
+	fmtinfo = cal_format_by_code(MEDIA_BUS_FMT_UYVY8_2X8);
+	if (!fmtinfo)
+		return -EINVAL;
+
+	pix_fmt->width = 640;
+	pix_fmt->height = 480;
+	pix_fmt->field = V4L2_FIELD_NONE;
+	pix_fmt->colorspace = V4L2_COLORSPACE_SRGB;
+	pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
+	pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE;
+	pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB;
+	pix_fmt->pixelformat = fmtinfo->fourcc;
+
+	ctx->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+
+	/* Save current format */
+	cal_calc_format_size(ctx, fmtinfo, &ctx->v_fmt);
+	ctx->fmtinfo = fmtinfo;
+
+	return 0;
+}
+
 int cal_ctx_v4l2_register(struct cal_ctx *ctx)
 {
 	struct video_device *vfd = &ctx->vdev;
 	int ret;
 
-	ret = cal_ctx_v4l2_init_formats(ctx);
-	if (ret)
-		return ret;
-
 	if (!cal_mc_api) {
 		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
 
+		ret = cal_ctx_v4l2_init_formats(ctx);
+		if (ret) {
+			ctx_err(ctx, "Failed to init formats: %d\n", ret);
+			return ret;
+		}
+
 		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->source->ctrl_handler,
 					    NULL, true);
 		if (ret < 0) {
 			ctx_err(ctx, "Failed to add source ctrl handler\n");
 			return ret;
 		}
+	} else {
+		ret = cal_ctx_v4l2_init_mc_format(ctx);
+		if (ret) {
+			ctx_err(ctx, "Failed to init format: %d\n", ret);
+			return ret;
+		}
 	}
 
 	ret = video_register_device(vfd, VFL_TYPE_VIDEO, cal_video_nr);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index def0c9a3657d..ee42c9c48fa1 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -213,7 +213,7 @@ struct cal_ctx {
 	/* Used to store current pixel format */
 	struct v4l2_format	v_fmt;
 
-	/* Current subdev enumerated format */
+	/* Current subdev enumerated format (legacy) */
 	const struct cal_format_info	**active_fmt;
 	unsigned int		num_active_fmt;
 
-- 
2.25.1

Re: [PATCH v3 26/38] media: ti-vpe: cal: init ctx->v_fmt correctly in MC mode

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:08:57PM +0300, Tomi Valkeinen wrote:
> CAL driver enumerates mbus codes in the connected subdev to create a
> list of supported formats reported to userspace, and initializes
> ctx->v_fmt and ctx->fmtinfo to one of those formats.
> 
> This works fine for legacy mode, but is not correct for MC mode, and the
> list is not even used in MC mode.
> 
> Fix this by adding a new function, cal_ctx_v4l2_init_mc_format, which
> only initializes ctx->v_fmt and ctx->fmtinfo to a default value.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-video.c | 43 ++++++++++++++++++++---
>  drivers/media/platform/ti-vpe/cal.h       |  2 +-
>  2 files changed, 40 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> index 7e97a43c6611..d06488cb8c36 100644
> --- a/drivers/media/platform/ti-vpe/cal-video.c
> +++ b/drivers/media/platform/ti-vpe/cal-video.c
> @@ -879,24 +879,59 @@ static int cal_ctx_v4l2_init_formats(struct cal_ctx *ctx)
>  	return 0;
>  }
>  
> +static int cal_ctx_v4l2_init_mc_format(struct cal_ctx *ctx)
> +{
> +	const struct cal_format_info *fmtinfo;
> +	struct v4l2_pix_format *pix_fmt = &ctx->v_fmt.fmt.pix;
> +
> +	fmtinfo = cal_format_by_code(MEDIA_BUS_FMT_UYVY8_2X8);
> +	if (!fmtinfo)
> +		return -EINVAL;

I still think a WARN_ON() would be enough, or even dropping the check
completely, as this really can't happen :-) The function could then
become void. Either way,

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +
> +	pix_fmt->width = 640;
> +	pix_fmt->height = 480;
> +	pix_fmt->field = V4L2_FIELD_NONE;
> +	pix_fmt->colorspace = V4L2_COLORSPACE_SRGB;
> +	pix_fmt->ycbcr_enc = V4L2_YCBCR_ENC_601;
> +	pix_fmt->quantization = V4L2_QUANTIZATION_LIM_RANGE;
> +	pix_fmt->xfer_func = V4L2_XFER_FUNC_SRGB;
> +	pix_fmt->pixelformat = fmtinfo->fourcc;
> +
> +	ctx->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +
> +	/* Save current format */
> +	cal_calc_format_size(ctx, fmtinfo, &ctx->v_fmt);
> +	ctx->fmtinfo = fmtinfo;
> +
> +	return 0;
> +}
> +
>  int cal_ctx_v4l2_register(struct cal_ctx *ctx)
>  {
>  	struct video_device *vfd = &ctx->vdev;
>  	int ret;
>  
> -	ret = cal_ctx_v4l2_init_formats(ctx);
> -	if (ret)
> -		return ret;
> -
>  	if (!cal_mc_api) {
>  		struct v4l2_ctrl_handler *hdl = &ctx->ctrl_handler;
>  
> +		ret = cal_ctx_v4l2_init_formats(ctx);
> +		if (ret) {
> +			ctx_err(ctx, "Failed to init formats: %d\n", ret);
> +			return ret;
> +		}
> +
>  		ret = v4l2_ctrl_add_handler(hdl, ctx->phy->source->ctrl_handler,
>  					    NULL, true);
>  		if (ret < 0) {
>  			ctx_err(ctx, "Failed to add source ctrl handler\n");
>  			return ret;
>  		}
> +	} else {
> +		ret = cal_ctx_v4l2_init_mc_format(ctx);
> +		if (ret) {
> +			ctx_err(ctx, "Failed to init format: %d\n", ret);
> +			return ret;
> +		}
>  	}
>  
>  	ret = video_register_device(vfd, VFL_TYPE_VIDEO, cal_video_nr);
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index def0c9a3657d..ee42c9c48fa1 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -213,7 +213,7 @@ struct cal_ctx {
>  	/* Used to store current pixel format */
>  	struct v4l2_format	v_fmt;
>  
> -	/* Current subdev enumerated format */
> +	/* Current subdev enumerated format (legacy) */
>  	const struct cal_format_info	**active_fmt;
>  	unsigned int		num_active_fmt;
>  

-- 
Regards,

Laurent Pinchart

[PATCH v3 27/38] media: ti-vpe: cal: remove cal_camerarx->fmtinfo

[Tomi Valkeinen]

struct cal_camerarx has fmtinfo field which is used to point to the
current active input format. The only place where the field is used is
cal_camerarx_get_ext_link_freq().

With multiple streams the whole concept of single input format is not
valid anymore, so lets remove the field by looking up the format in
cal_camerarx_get_ext_link_freq(), making it easier to handle the
multistream-case in the following patches.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 12 ++++++++----
 drivers/media/platform/ti-vpe/cal.h          |  1 -
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 7940e47964af..82392499e663 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -49,9 +49,16 @@ static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
 {
 	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
 	u32 num_lanes = mipi_csi2->num_data_lanes;
-	u32 bpp = phy->fmtinfo->bpp;
+	const struct cal_format_info *fmtinfo;
+	u32 bpp;
 	s64 freq;
 
+	fmtinfo = cal_format_by_code(phy->formats[CAL_CAMERARX_PAD_SINK].code);
+	if (!fmtinfo)
+		return -EINVAL;
+
+	bpp = fmtinfo->bpp;
+
 	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
 	if (freq < 0) {
 		phy_err(phy, "failed to get link freq for subdev '%s'\n",
@@ -728,9 +735,6 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 					  format->which);
 	*fmt = format->format;
 
-	if (format->which == V4L2_SUBDEV_FORMAT_ACTIVE)
-		phy->fmtinfo = fmtinfo;
-
 	return 0;
 }
 
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index ee42c9c48fa1..7d9a2d697fc4 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -163,7 +163,6 @@ struct cal_camerarx {
 	struct v4l2_subdev	subdev;
 	struct media_pad	pads[2];
 	struct v4l2_mbus_framefmt	formats[2];
-	const struct cal_format_info	*fmtinfo;
 };
 
 struct cal_dev {
-- 
2.25.1

[PATCH v3 28/38] media: ti-vpe: cal: support 8 DMA contexts

[Tomi Valkeinen]

The current driver only ever needs 2 DMA contexts (one per PHY), but we
need to use more of the 8 contexts to add support for multiple streams.

Change the code so that we allocate DMA contexts as needed, which at
this time is 1 per PHY, but could be up to 8.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 31 ++++++++++-------------------
 drivers/media/platform/ti-vpe/cal.h |  5 +++--
 2 files changed, 14 insertions(+), 22 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 9e051c2e84a9..d43972c392fc 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -658,7 +658,7 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 		/* Clear Interrupt status */
 		cal_write(cal, CAL_HL_IRQSTATUS(1), status);
 
-		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
+		for (i = 0; i < cal->num_contexts; ++i) {
 			if (status & CAL_HL_IRQ_WDMA_END_MASK(i))
 				cal_irq_wdma_end(cal->ctx[i]);
 		}
@@ -672,7 +672,7 @@ static irqreturn_t cal_irq(int irq_cal, void *data)
 		/* Clear Interrupt status */
 		cal_write(cal, CAL_HL_IRQSTATUS(2), status);
 
-		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
+		for (i = 0; i < cal->num_contexts; ++i) {
 			if (status & CAL_HL_IRQ_WDMA_START_MASK(i))
 				cal_irq_wdma_start(cal->ctx[i]);
 		}
@@ -742,10 +742,7 @@ static int cal_async_notifier_complete(struct v4l2_async_notifier *notifier)
 	unsigned int i;
 	int ret = 0;
 
-	for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
-		if (!cal->ctx[i])
-			continue;
-
+	for (i = 0; i < cal->num_contexts; ++i) {
 		ret = cal_ctx_v4l2_register(cal->ctx[i]);
 		if (ret)
 			return ret;
@@ -851,10 +848,8 @@ static void cal_media_unregister(struct cal_dev *cal)
 	unsigned int i;
 
 	/* Unregister all the V4L2 video devices. */
-	for (i = 0; i < ARRAY_SIZE(cal->ctx); i++) {
-		if (cal->ctx[i])
-			cal_ctx_v4l2_unregister(cal->ctx[i]);
-	}
+	for (i = 0; i < cal->num_contexts; i++)
+		cal_ctx_v4l2_unregister(cal->ctx[i]);
 
 	cal_async_notifier_unregister(cal);
 	media_device_unregister(&cal->mdev);
@@ -901,10 +896,8 @@ static void cal_media_cleanup(struct cal_dev *cal)
 {
 	unsigned int i;
 
-	for (i = 0; i < ARRAY_SIZE(cal->ctx); i++) {
-		if (cal->ctx[i])
-			cal_ctx_v4l2_cleanup(cal->ctx[i]);
-	}
+	for (i = 0; i < cal->num_contexts; i++)
+		cal_ctx_v4l2_cleanup(cal->ctx[i]);
 
 	v4l2_device_unregister(&cal->v4l2_dev);
 	media_device_cleanup(&cal->mdev);
@@ -1053,7 +1046,6 @@ static int cal_init_camerarx_regmap(struct cal_dev *cal)
 static int cal_probe(struct platform_device *pdev)
 {
 	struct cal_dev *cal;
-	struct cal_ctx *ctx;
 	bool connected = false;
 	unsigned int i;
 	int ret;
@@ -1143,6 +1135,8 @@ static int cal_probe(struct platform_device *pdev)
 			ret = -ENODEV;
 			goto error_context;
 		}
+
+		cal->num_contexts++;
 	}
 
 	/* Register the media device. */
@@ -1153,11 +1147,8 @@ static int cal_probe(struct platform_device *pdev)
 	return 0;
 
 error_context:
-	for (i = 0; i < ARRAY_SIZE(cal->ctx); i++) {
-		ctx = cal->ctx[i];
-		if (ctx)
-			cal_ctx_v4l2_cleanup(ctx);
-	}
+	for (i = 0; i < cal->num_contexts; i++)
+		cal_ctx_v4l2_cleanup(cal->ctx[i]);
 
 error_camerarx:
 	for (i = 0; i < cal->data->num_csi2_phy; i++)
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 7d9a2d697fc4..400f95485d7c 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -29,7 +29,7 @@
 #include <media/videobuf2-v4l2.h>
 
 #define CAL_MODULE_NAME			"cal"
-#define CAL_NUM_CONTEXT			2
+#define CAL_MAX_NUM_CONTEXT		8
 #define CAL_NUM_CSI2_PORTS		2
 
 /*
@@ -182,7 +182,8 @@ struct cal_dev {
 	/* Camera Core Module handle */
 	struct cal_camerarx	*phy[CAL_NUM_CSI2_PORTS];
 
-	struct cal_ctx		*ctx[CAL_NUM_CONTEXT];
+	u32 num_contexts;
+	struct cal_ctx		*ctx[CAL_MAX_NUM_CONTEXT];
 
 	struct media_device	mdev;
 	struct v4l2_device	v4l2_dev;
-- 
2.25.1

[PATCH v3 29/38] media: ti-vpe: cal: cleanup phy iteration in cal_remove

[Tomi Valkeinen]

Most of the driver has moved from ARRAY_SIZE(cal->phy) to
cal->data->num_csi2_phy, but we have one place left in cal_remove. Also,
checking for cal->phy[i] != NULL is not needed as we always have all the
phys instantiated.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index d43972c392fc..bb99d0ce796f 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -1173,10 +1173,8 @@ static int cal_remove(struct platform_device *pdev)
 
 	cal_media_unregister(cal);
 
-	for (i = 0; i < ARRAY_SIZE(cal->phy); i++) {
-		if (cal->phy[i])
-			cal_camerarx_disable(cal->phy[i]);
-	}
+	for (i = 0; i < cal->data->num_csi2_phy; i++)
+		cal_camerarx_disable(cal->phy[i]);
 
 	cal_media_cleanup(cal);
 
-- 
2.25.1

Re: [PATCH v3 29/38] media: ti-vpe: cal: cleanup phy iteration in cal_remove

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:00PM +0300, Tomi Valkeinen wrote:
> Most of the driver has moved from ARRAY_SIZE(cal->phy) to
> cal->data->num_csi2_phy, but we have one place left in cal_remove. Also,
> checking for cal->phy[i] != NULL is not needed as we always have all the
> phys instantiated.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/ti-vpe/cal.c | 6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index d43972c392fc..bb99d0ce796f 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -1173,10 +1173,8 @@ static int cal_remove(struct platform_device *pdev)
>  
>  	cal_media_unregister(cal);
>  
> -	for (i = 0; i < ARRAY_SIZE(cal->phy); i++) {
> -		if (cal->phy[i])
> -			cal_camerarx_disable(cal->phy[i]);
> -	}
> +	for (i = 0; i < cal->data->num_csi2_phy; i++)
> +		cal_camerarx_disable(cal->phy[i]);
>  
>  	cal_media_cleanup(cal);
>  

-- 
Regards,

Laurent Pinchart

[PATCH v3 30/38] media: ti-vpe: cal: fix ctx uninitialization

[Tomi Valkeinen]

Somewhere along the way the context uninitialization has gone a bit odd:

* We have cal_ctx_create() but no matching destroy call, but we still
need to call cal_ctx_v4l2_cleanup() for each context.

* We have cal_media_cleanup() which calls cal_ctx_v4l2_cleanup() for all
contexts, but cal_media_init() is not where the contexts are created.

* The order of uninit steps in cal_remove() is different than the error
handling path in cal_probe().

* cal_probe()'s error handling calls cal_ctx_v4l2_cleanup() for each
context, but also calls cal_media_clean(), doing the same context
cleanup twice.

So fix these, by introducing cal_ctx_destroy(), and using that in
appropriate places instead of calling cal_ctx_v4l2_cleanup() in
cal_media_clean(). Also use normal kzalloc (and kfree) instead of devm
version as we anyway do manual cleanup for each context.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index bb99d0ce796f..888706187fd1 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -894,11 +894,6 @@ static int cal_media_init(struct cal_dev *cal)
  */
 static void cal_media_cleanup(struct cal_dev *cal)
 {
-	unsigned int i;
-
-	for (i = 0; i < cal->num_contexts; i++)
-		cal_ctx_v4l2_cleanup(cal->ctx[i]);
-
 	v4l2_device_unregister(&cal->v4l2_dev);
 	media_device_cleanup(&cal->mdev);
 
@@ -915,7 +910,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 	struct cal_ctx *ctx;
 	int ret;
 
-	ctx = devm_kzalloc(cal->dev, sizeof(*ctx), GFP_KERNEL);
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
 	if (!ctx)
 		return NULL;
 
@@ -934,6 +929,13 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 	return ctx;
 }
 
+static void cal_ctx_destroy(struct cal_ctx *ctx)
+{
+	cal_ctx_v4l2_cleanup(ctx);
+
+	kfree(ctx);
+}
+
 static const struct of_device_id cal_of_match[] = {
 	{
 		.compatible = "ti,dra72-cal",
@@ -1148,7 +1150,7 @@ static int cal_probe(struct platform_device *pdev)
 
 error_context:
 	for (i = 0; i < cal->num_contexts; i++)
-		cal_ctx_v4l2_cleanup(cal->ctx[i]);
+		cal_ctx_destroy(cal->ctx[i]);
 
 error_camerarx:
 	for (i = 0; i < cal->data->num_csi2_phy; i++)
@@ -1176,11 +1178,14 @@ static int cal_remove(struct platform_device *pdev)
 	for (i = 0; i < cal->data->num_csi2_phy; i++)
 		cal_camerarx_disable(cal->phy[i]);
 
-	cal_media_cleanup(cal);
+	for (i = 0; i < cal->num_contexts; i++)
+		cal_ctx_destroy(cal->ctx[i]);
 
 	for (i = 0; i < cal->data->num_csi2_phy; i++)
 		cal_camerarx_destroy(cal->phy[i]);
 
+	cal_media_cleanup(cal);
+
 	pm_runtime_put_sync(&pdev->dev);
 	pm_runtime_disable(&pdev->dev);
 
-- 
2.25.1

Re: [PATCH v3 30/38] media: ti-vpe: cal: fix ctx uninitialization

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:01PM +0300, Tomi Valkeinen wrote:
> Somewhere along the way the context uninitialization has gone a bit odd:
> 
> * We have cal_ctx_create() but no matching destroy call, but we still
> need to call cal_ctx_v4l2_cleanup() for each context.
> 
> * We have cal_media_cleanup() which calls cal_ctx_v4l2_cleanup() for all
> contexts, but cal_media_init() is not where the contexts are created.
> 
> * The order of uninit steps in cal_remove() is different than the error
> handling path in cal_probe().
> 
> * cal_probe()'s error handling calls cal_ctx_v4l2_cleanup() for each
> context, but also calls cal_media_clean(), doing the same context
> cleanup twice.
> 
> So fix these, by introducing cal_ctx_destroy(), and using that in
> appropriate places instead of calling cal_ctx_v4l2_cleanup() in
> cal_media_clean(). Also use normal kzalloc (and kfree) instead of devm
> version as we anyway do manual cleanup for each context.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/ti-vpe/cal.c | 21 +++++++++++++--------
>  1 file changed, 13 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index bb99d0ce796f..888706187fd1 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -894,11 +894,6 @@ static int cal_media_init(struct cal_dev *cal)
>   */
>  static void cal_media_cleanup(struct cal_dev *cal)
>  {
> -	unsigned int i;
> -
> -	for (i = 0; i < cal->num_contexts; i++)
> -		cal_ctx_v4l2_cleanup(cal->ctx[i]);
> -
>  	v4l2_device_unregister(&cal->v4l2_dev);
>  	media_device_cleanup(&cal->mdev);
>  
> @@ -915,7 +910,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
>  	struct cal_ctx *ctx;
>  	int ret;
>  
> -	ctx = devm_kzalloc(cal->dev, sizeof(*ctx), GFP_KERNEL);
> +	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
>  	if (!ctx)
>  		return NULL;
>  
> @@ -934,6 +929,13 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
>  	return ctx;
>  }
>  
> +static void cal_ctx_destroy(struct cal_ctx *ctx)
> +{
> +	cal_ctx_v4l2_cleanup(ctx);
> +
> +	kfree(ctx);
> +}
> +
>  static const struct of_device_id cal_of_match[] = {
>  	{
>  		.compatible = "ti,dra72-cal",
> @@ -1148,7 +1150,7 @@ static int cal_probe(struct platform_device *pdev)
>  
>  error_context:
>  	for (i = 0; i < cal->num_contexts; i++)
> -		cal_ctx_v4l2_cleanup(cal->ctx[i]);
> +		cal_ctx_destroy(cal->ctx[i]);
>  
>  error_camerarx:
>  	for (i = 0; i < cal->data->num_csi2_phy; i++)
> @@ -1176,11 +1178,14 @@ static int cal_remove(struct platform_device *pdev)
>  	for (i = 0; i < cal->data->num_csi2_phy; i++)
>  		cal_camerarx_disable(cal->phy[i]);
>  
> -	cal_media_cleanup(cal);
> +	for (i = 0; i < cal->num_contexts; i++)
> +		cal_ctx_destroy(cal->ctx[i]);
>  
>  	for (i = 0; i < cal->data->num_csi2_phy; i++)
>  		cal_camerarx_destroy(cal->phy[i]);
>  
> +	cal_media_cleanup(cal);
> +
>  	pm_runtime_put_sync(&pdev->dev);
>  	pm_runtime_disable(&pdev->dev);
>  

-- 
Regards,

Laurent Pinchart

[PATCH v3 31/38] media: ti-vpe: cal: fix queuing of the initial buffer

[Tomi Valkeinen]

When starting streaming the driver currently programs the buffer
address to the CAL base-address register and assigns the buffer pointer
to ctx->dma.pending. This is not correct, as the buffer is not
"pending", but active, and causes the first buffer to be needlessly
written twice.

Fix this by assigning the buffer pointer to ctx->dma.active.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-video.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index d06488cb8c36..efa08a9ccbd5 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -731,7 +731,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	spin_lock_irq(&ctx->dma.lock);
 	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
-	ctx->dma.pending = buf;
+	ctx->dma.active = buf;
 	list_del(&buf->list);
 	spin_unlock_irq(&ctx->dma.lock);
 
-- 
2.25.1

Re: [PATCH v3 31/38] media: ti-vpe: cal: fix queuing of the initial buffer

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:02PM +0300, Tomi Valkeinen wrote:
> When starting streaming the driver currently programs the buffer
> address to the CAL base-address register and assigns the buffer pointer
> to ctx->dma.pending. This is not correct, as the buffer is not
> "pending", but active, and causes the first buffer to be needlessly
> written twice.
> 
> Fix this by assigning the buffer pointer to ctx->dma.active.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/platform/ti-vpe/cal-video.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> index d06488cb8c36..efa08a9ccbd5 100644
> --- a/drivers/media/platform/ti-vpe/cal-video.c
> +++ b/drivers/media/platform/ti-vpe/cal-video.c
> @@ -731,7 +731,7 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>  
>  	spin_lock_irq(&ctx->dma.lock);
>  	buf = list_first_entry(&ctx->dma.queue, struct cal_buffer, list);
> -	ctx->dma.pending = buf;
> +	ctx->dma.active = buf;
>  	list_del(&buf->list);
>  	spin_unlock_irq(&ctx->dma.lock);
>  

-- 
Regards,

Laurent Pinchart

[PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Tomi Valkeinen]

The driver fills buf->vb.sequence with an increasing number which is
incremented by the driver. This feels a bit pointless, as the userspace
could as well track that kind of number itself. Instead, lets use the
frame number provided in the CSI-2 data from the sensor.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 7 +++++--
 drivers/media/platform/ti-vpe/cal.h | 1 -
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 888706187fd1..62c45add4efe 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
 
 void cal_ctx_start(struct cal_ctx *ctx)
 {
-	ctx->sequence = 0;
 	ctx->dma.state = CAL_DMA_RUNNING;
 
 	/* Configure the CSI-2, pixel processing and write DMA contexts. */
@@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
 static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
 {
 	struct cal_buffer *buf = NULL;
+	u32 frame_num;
+
+	frame_num = cal_read(ctx->cal, CAL_CSI2_STATUS(ctx->phy->instance,
+						       ctx->csi2_ctx)) & 0xffff;
 
 	spin_lock(&ctx->dma.lock);
 
@@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
 	if (buf) {
 		buf->vb.vb2_buf.timestamp = ktime_get_ns();
 		buf->vb.field = ctx->v_fmt.fmt.pix.field;
-		buf->vb.sequence = ctx->sequence++;
+		buf->vb.sequence = frame_num;
 		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
 	}
 }
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 400f95485d7c..ad08c189ad3b 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -217,7 +217,6 @@ struct cal_ctx {
 	const struct cal_format_info	**active_fmt;
 	unsigned int		num_active_fmt;
 
-	unsigned int		sequence;
 	struct vb2_queue	vb_vidq;
 	u8			dma_ctx;
 	u8			cport;
-- 
2.25.1

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
> The driver fills buf->vb.sequence with an increasing number which is
> incremented by the driver. This feels a bit pointless, as the userspace
> could as well track that kind of number itself. Instead, lets use the

s/lets/let's/

> frame number provided in the CSI-2 data from the sensor.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal.c | 7 +++++--
>  drivers/media/platform/ti-vpe/cal.h | 1 -
>  2 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index 888706187fd1..62c45add4efe 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>  
>  void cal_ctx_start(struct cal_ctx *ctx)
>  {
> -	ctx->sequence = 0;
>  	ctx->dma.state = CAL_DMA_RUNNING;
>  
>  	/* Configure the CSI-2, pixel processing and write DMA contexts. */
> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
>  static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>  {
>  	struct cal_buffer *buf = NULL;
> +	u32 frame_num;
> +
> +	frame_num = cal_read(ctx->cal, CAL_CSI2_STATUS(ctx->phy->instance,
> +						       ctx->csi2_ctx)) & 0xffff;
>  
>  	spin_lock(&ctx->dma.lock);
>  
> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>  	if (buf) {
>  		buf->vb.vb2_buf.timestamp = ktime_get_ns();
>  		buf->vb.field = ctx->v_fmt.fmt.pix.field;
> -		buf->vb.sequence = ctx->sequence++;
> +		buf->vb.sequence = frame_num;

We'll need something a bit more complicated. The CSI-2 frame number is
not mandatory, and when used, it is a 16-bit number starting at 1 and
counting to an unspecified value larger than one, resetting to 1 at the
end of the cycle. The V4L2 sequence number, on the other hand, is a
monotonic counter starting at 0 and wrapping only at 2^32-1. We should
thus keep a software sequence counter and

- increase it by 1 if the frame number is zero
- increase it by frame_num - last_frame_num (with wrap-around of
  frame_num handled) otherwise

>  		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
>  	}
>  }
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 400f95485d7c..ad08c189ad3b 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -217,7 +217,6 @@ struct cal_ctx {
>  	const struct cal_format_info	**active_fmt;
>  	unsigned int		num_active_fmt;
>  
> -	unsigned int		sequence;
>  	struct vb2_queue	vb_vidq;
>  	u8			dma_ctx;
>  	u8			cport;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Tomi Valkeinen]

On 04/06/2021 17:04, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
>> The driver fills buf->vb.sequence with an increasing number which is
>> incremented by the driver. This feels a bit pointless, as the userspace
>> could as well track that kind of number itself. Instead, lets use the
> 
> s/lets/let's/
> 
>> frame number provided in the CSI-2 data from the sensor.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/platform/ti-vpe/cal.c | 7 +++++--
>>   drivers/media/platform/ti-vpe/cal.h | 1 -
>>   2 files changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>> index 888706187fd1..62c45add4efe 100644
>> --- a/drivers/media/platform/ti-vpe/cal.c
>> +++ b/drivers/media/platform/ti-vpe/cal.c
>> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>>   
>>   void cal_ctx_start(struct cal_ctx *ctx)
>>   {
>> -	ctx->sequence = 0;
>>   	ctx->dma.state = CAL_DMA_RUNNING;
>>   
>>   	/* Configure the CSI-2, pixel processing and write DMA contexts. */
>> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
>>   static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>   {
>>   	struct cal_buffer *buf = NULL;
>> +	u32 frame_num;
>> +
>> +	frame_num = cal_read(ctx->cal, CAL_CSI2_STATUS(ctx->phy->instance,
>> +						       ctx->csi2_ctx)) & 0xffff;
>>   
>>   	spin_lock(&ctx->dma.lock);
>>   
>> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>   	if (buf) {
>>   		buf->vb.vb2_buf.timestamp = ktime_get_ns();
>>   		buf->vb.field = ctx->v_fmt.fmt.pix.field;
>> -		buf->vb.sequence = ctx->sequence++;
>> +		buf->vb.sequence = frame_num;
> 
> We'll need something a bit more complicated. The CSI-2 frame number is
> not mandatory, and when used, it is a 16-bit number starting at 1 and
> counting to an unspecified value larger than one, resetting to 1 at the
> end of the cycle. The V4L2 sequence number, on the other hand, is a
> monotonic counter starting at 0 and wrapping only at 2^32-1. We should
> thus keep a software sequence counter and
> 
> - increase it by 1 if the frame number is zero
> - increase it by frame_num - last_frame_num (with wrap-around of
>    frame_num handled) otherwise

Ok... I wonder if we need a new field for this, though. The problem I 
was solving when I changed this to use the CSI-2 frame-number was how to 
associate a pixel frame and a metadata frame.

Their CSI-2 frame-numbers match (as they are from the same original 
CSI-2 frame), so the userspace can use that to figure the matching 
frames. While the above method you suggest should give us identical 
sequence numbers for pixel and metadata, I think it's going a bit away 
from my intended purpose, and possibly risks ending up with different 
sequences for pixel and metadata.

So do we need the sequence number, as it is currently, and something new 
for this buffer matching purpose?

  Tomi

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Laurent Pinchart]

Hi Tomi,

On Mon, Jun 07, 2021 at 03:39:45PM +0300, Tomi Valkeinen wrote:
> On 04/06/2021 17:04, Laurent Pinchart wrote:
> > On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
> >> The driver fills buf->vb.sequence with an increasing number which is
> >> incremented by the driver. This feels a bit pointless, as the userspace
> >> could as well track that kind of number itself. Instead, lets use the
> > 
> > s/lets/let's/
> > 
> >> frame number provided in the CSI-2 data from the sensor.
> >>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >>   drivers/media/platform/ti-vpe/cal.c | 7 +++++--
> >>   drivers/media/platform/ti-vpe/cal.h | 1 -
> >>   2 files changed, 5 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> >> index 888706187fd1..62c45add4efe 100644
> >> --- a/drivers/media/platform/ti-vpe/cal.c
> >> +++ b/drivers/media/platform/ti-vpe/cal.c
> >> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
> >>   
> >>   void cal_ctx_start(struct cal_ctx *ctx)
> >>   {
> >> -	ctx->sequence = 0;
> >>   	ctx->dma.state = CAL_DMA_RUNNING;
> >>   
> >>   	/* Configure the CSI-2, pixel processing and write DMA contexts. */
> >> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
> >>   static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
> >>   {
> >>   	struct cal_buffer *buf = NULL;
> >> +	u32 frame_num;
> >> +
> >> +	frame_num = cal_read(ctx->cal, CAL_CSI2_STATUS(ctx->phy->instance,
> >> +						       ctx->csi2_ctx)) & 0xffff;
> >>   
> >>   	spin_lock(&ctx->dma.lock);
> >>   
> >> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
> >>   	if (buf) {
> >>   		buf->vb.vb2_buf.timestamp = ktime_get_ns();
> >>   		buf->vb.field = ctx->v_fmt.fmt.pix.field;
> >> -		buf->vb.sequence = ctx->sequence++;
> >> +		buf->vb.sequence = frame_num;
> > 
> > We'll need something a bit more complicated. The CSI-2 frame number is
> > not mandatory, and when used, it is a 16-bit number starting at 1 and
> > counting to an unspecified value larger than one, resetting to 1 at the
> > end of the cycle. The V4L2 sequence number, on the other hand, is a
> > monotonic counter starting at 0 and wrapping only at 2^32-1. We should
> > thus keep a software sequence counter and
> > 
> > - increase it by 1 if the frame number is zero
> > - increase it by frame_num - last_frame_num (with wrap-around of
> >    frame_num handled) otherwise
> 
> Ok... I wonder if we need a new field for this, though. The problem I 
> was solving when I changed this to use the CSI-2 frame-number was how to 
> associate a pixel frame and a metadata frame.
> 
> Their CSI-2 frame-numbers match (as they are from the same original 
> CSI-2 frame), so the userspace can use that to figure the matching 
> frames. While the above method you suggest should give us identical 
> sequence numbers for pixel and metadata, I think it's going a bit away 
> from my intended purpose, and possibly risks ending up with different 
> sequences for pixel and metadata.

Why do you think they could get out of sync (assuming the sensor
supports frame numbers of course, if it always returns 0, that's not
usable for the purpose of synchronization).

> So do we need the sequence number, as it is currently, and something new 
> for this buffer matching purpose?

When the sensor doesn't support frame numbers, the only thing that can
be used is the timestamp as far as I can see.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Tomi Valkeinen]

On 07/06/2021 16:42, Laurent Pinchart wrote:
> Hi Tomi,
> 
> On Mon, Jun 07, 2021 at 03:39:45PM +0300, Tomi Valkeinen wrote:
>> On 04/06/2021 17:04, Laurent Pinchart wrote:
>>> On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
>>>> The driver fills buf->vb.sequence with an increasing number which is
>>>> incremented by the driver. This feels a bit pointless, as the userspace
>>>> could as well track that kind of number itself. Instead, lets use the
>>>
>>> s/lets/let's/
>>>
>>>> frame number provided in the CSI-2 data from the sensor.
>>>>
>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>> ---
>>>>    drivers/media/platform/ti-vpe/cal.c | 7 +++++--
>>>>    drivers/media/platform/ti-vpe/cal.h | 1 -
>>>>    2 files changed, 5 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>>>> index 888706187fd1..62c45add4efe 100644
>>>> --- a/drivers/media/platform/ti-vpe/cal.c
>>>> +++ b/drivers/media/platform/ti-vpe/cal.c
>>>> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>>>>    
>>>>    void cal_ctx_start(struct cal_ctx *ctx)
>>>>    {
>>>> -	ctx->sequence = 0;
>>>>    	ctx->dma.state = CAL_DMA_RUNNING;
>>>>    
>>>>    	/* Configure the CSI-2, pixel processing and write DMA contexts. */
>>>> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
>>>>    static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>>>    {
>>>>    	struct cal_buffer *buf = NULL;
>>>> +	u32 frame_num;
>>>> +
>>>> +	frame_num = cal_read(ctx->cal, CAL_CSI2_STATUS(ctx->phy->instance,
>>>> +						       ctx->csi2_ctx)) & 0xffff;
>>>>    
>>>>    	spin_lock(&ctx->dma.lock);
>>>>    
>>>> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>>>    	if (buf) {
>>>>    		buf->vb.vb2_buf.timestamp = ktime_get_ns();
>>>>    		buf->vb.field = ctx->v_fmt.fmt.pix.field;
>>>> -		buf->vb.sequence = ctx->sequence++;
>>>> +		buf->vb.sequence = frame_num;
>>>
>>> We'll need something a bit more complicated. The CSI-2 frame number is
>>> not mandatory, and when used, it is a 16-bit number starting at 1 and
>>> counting to an unspecified value larger than one, resetting to 1 at the
>>> end of the cycle. The V4L2 sequence number, on the other hand, is a
>>> monotonic counter starting at 0 and wrapping only at 2^32-1. We should
>>> thus keep a software sequence counter and
>>>
>>> - increase it by 1 if the frame number is zero
>>> - increase it by frame_num - last_frame_num (with wrap-around of
>>>     frame_num handled) otherwise
>>
>> Ok... I wonder if we need a new field for this, though. The problem I
>> was solving when I changed this to use the CSI-2 frame-number was how to
>> associate a pixel frame and a metadata frame.
>>
>> Their CSI-2 frame-numbers match (as they are from the same original
>> CSI-2 frame), so the userspace can use that to figure the matching
>> frames. While the above method you suggest should give us identical
>> sequence numbers for pixel and metadata, I think it's going a bit away
>> from my intended purpose, and possibly risks ending up with different
>> sequences for pixel and metadata.
> 
> Why do you think they could get out of sync (assuming the sensor
> supports frame numbers of course, if it always returns 0, that's not
> usable for the purpose of synchronization).

If there's a requirement that the sequence starts from 0, it doesn't 
work, as the pixel and metadata video capture may be started at 
different times. When pixel capture starts, the frame number could be 10 
and pixel sequence would be 0, but when metadata capture starts, the 
frame number could be 12, and pixel seq would thus be 2 and meta seq 0.

But even if we allow the seq to start from the current frame number, 
this doesn't work if the frame number has wrapped between starting the 
pixel capture and starting the meta capture.

We have a FS irq for each virtual channel (the current sequence code is 
in the dma-frame-end irq code which is enabled per ctx only when 
capturing), but I don't right away see how tracking FS per VC would help 
here...

>> So do we need the sequence number, as it is currently, and something new
>> for this buffer matching purpose?
> 
> When the sensor doesn't support frame numbers, the only thing that can
> be used is the timestamp as far as I can see.

CAL doc says that if the sensor is not sending frame numbers, the frame 
number register is incremented by 1 on every FS. So we always have a 
frame number, but we don't know if it's from the sensor or not.

  Tomi

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Laurent Pinchart]

Hi Tomi,

On Mon, Jun 07, 2021 at 05:55:05PM +0300, Tomi Valkeinen wrote:
> On 07/06/2021 16:42, Laurent Pinchart wrote:
> > On Mon, Jun 07, 2021 at 03:39:45PM +0300, Tomi Valkeinen wrote:
> >> On 04/06/2021 17:04, Laurent Pinchart wrote:
> >>> On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
> >>>> The driver fills buf->vb.sequence with an increasing number which is
> >>>> incremented by the driver. This feels a bit pointless, as the userspace
> >>>> could as well track that kind of number itself. Instead, lets use the
> >>>
> >>> s/lets/let's/
> >>>
> >>>> frame number provided in the CSI-2 data from the sensor.
> >>>>
> >>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >>>> ---
> >>>>    drivers/media/platform/ti-vpe/cal.c | 7 +++++--
> >>>>    drivers/media/platform/ti-vpe/cal.h | 1 -
> >>>>    2 files changed, 5 insertions(+), 3 deletions(-)
> >>>>
> >>>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> >>>> index 888706187fd1..62c45add4efe 100644
> >>>> --- a/drivers/media/platform/ti-vpe/cal.c
> >>>> +++ b/drivers/media/platform/ti-vpe/cal.c
> >>>> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
> >>>>    
> >>>>    void cal_ctx_start(struct cal_ctx *ctx)
> >>>>    {
> >>>> -	ctx->sequence = 0;
> >>>>    	ctx->dma.state = CAL_DMA_RUNNING;
> >>>>    
> >>>>    	/* Configure the CSI-2, pixel processing and write DMA contexts. */
> >>>> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
> >>>>    static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
> >>>>    {
> >>>>    	struct cal_buffer *buf = NULL;
> >>>> +	u32 frame_num;
> >>>> +
> >>>> +	frame_num = cal_read(ctx->cal, CAL_CSI2_STATUS(ctx->phy->instance,
> >>>> +						       ctx->csi2_ctx)) & 0xffff;
> >>>>    
> >>>>    	spin_lock(&ctx->dma.lock);
> >>>>    
> >>>> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
> >>>>    	if (buf) {
> >>>>    		buf->vb.vb2_buf.timestamp = ktime_get_ns();
> >>>>    		buf->vb.field = ctx->v_fmt.fmt.pix.field;
> >>>> -		buf->vb.sequence = ctx->sequence++;
> >>>> +		buf->vb.sequence = frame_num;
> >>>
> >>> We'll need something a bit more complicated. The CSI-2 frame number is
> >>> not mandatory, and when used, it is a 16-bit number starting at 1 and
> >>> counting to an unspecified value larger than one, resetting to 1 at the
> >>> end of the cycle. The V4L2 sequence number, on the other hand, is a
> >>> monotonic counter starting at 0 and wrapping only at 2^32-1. We should
> >>> thus keep a software sequence counter and
> >>>
> >>> - increase it by 1 if the frame number is zero
> >>> - increase it by frame_num - last_frame_num (with wrap-around of
> >>>     frame_num handled) otherwise
> >>
> >> Ok... I wonder if we need a new field for this, though. The problem I
> >> was solving when I changed this to use the CSI-2 frame-number was how to
> >> associate a pixel frame and a metadata frame.
> >>
> >> Their CSI-2 frame-numbers match (as they are from the same original
> >> CSI-2 frame), so the userspace can use that to figure the matching
> >> frames. While the above method you suggest should give us identical
> >> sequence numbers for pixel and metadata, I think it's going a bit away
> >> from my intended purpose, and possibly risks ending up with different
> >> sequences for pixel and metadata.
> > 
> > Why do you think they could get out of sync (assuming the sensor
> > supports frame numbers of course, if it always returns 0, that's not
> > usable for the purpose of synchronization).
> 
> If there's a requirement that the sequence starts from 0, it doesn't 
> work, as the pixel and metadata video capture may be started at 
> different times. When pixel capture starts, the frame number could be 10 
> and pixel sequence would be 0, but when metadata capture starts, the 
> frame number could be 12, and pixel seq would thus be 2 and meta seq 0.
> 
> But even if we allow the seq to start from the current frame number, 

Good point. I think we can allow starting at a non-zero value to handle
this.

> this doesn't work if the frame number has wrapped between starting the 
> pixel capture and starting the meta capture.

The timestamp should be enough to handle this, the timestamp difference
between two wraparounds should be large enough to sync the two streams
without any risk.

> We have a FS irq for each virtual channel (the current sequence code is 
> in the dma-frame-end irq code which is enabled per ctx only when 
> capturing), but I don't right away see how tracking FS per VC would help 
> here...
> 
> >> So do we need the sequence number, as it is currently, and something new
> >> for this buffer matching purpose?
> > 
> > When the sensor doesn't support frame numbers, the only thing that can
> > be used is the timestamp as far as I can see.
> 
> CAL doc says that if the sensor is not sending frame numbers, the frame 
> number register is incremented by 1 on every FS. So we always have a 
> frame number, but we don't know if it's from the sensor or not.

Ah, that's a useful feature, it should simplify the implementation on
the driver side and avoid missing frame syncs.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Tomi Valkeinen]

On 07/06/2021 19:51, Laurent Pinchart wrote:
> Hi Tomi,
> 
> On Mon, Jun 07, 2021 at 05:55:05PM +0300, Tomi Valkeinen wrote:
>> On 07/06/2021 16:42, Laurent Pinchart wrote:
>>> On Mon, Jun 07, 2021 at 03:39:45PM +0300, Tomi Valkeinen wrote:
>>>> On 04/06/2021 17:04, Laurent Pinchart wrote:
>>>>> On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
>>>>>> The driver fills buf->vb.sequence with an increasing number which is
>>>>>> incremented by the driver. This feels a bit pointless, as the userspace
>>>>>> could as well track that kind of number itself. Instead, lets use the
>>>>>
>>>>> s/lets/let's/
>>>>>
>>>>>> frame number provided in the CSI-2 data from the sensor.
>>>>>>
>>>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>>>> ---
>>>>>>     drivers/media/platform/ti-vpe/cal.c | 7 +++++--
>>>>>>     drivers/media/platform/ti-vpe/cal.h | 1 -
>>>>>>     2 files changed, 5 insertions(+), 3 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>>>>>> index 888706187fd1..62c45add4efe 100644
>>>>>> --- a/drivers/media/platform/ti-vpe/cal.c
>>>>>> +++ b/drivers/media/platform/ti-vpe/cal.c
>>>>>> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>>>>>>     
>>>>>>     void cal_ctx_start(struct cal_ctx *ctx)
>>>>>>     {
>>>>>> -	ctx->sequence = 0;
>>>>>>     	ctx->dma.state = CAL_DMA_RUNNING;
>>>>>>     
>>>>>>     	/* Configure the CSI-2, pixel processing and write DMA contexts. */
>>>>>> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
>>>>>>     static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>>>>>     {
>>>>>>     	struct cal_buffer *buf = NULL;
>>>>>> +	u32 frame_num;
>>>>>> +
>>>>>> +	frame_num = cal_read(ctx->cal, CAL_CSI2_STATUS(ctx->phy->instance,
>>>>>> +						       ctx->csi2_ctx)) & 0xffff;
>>>>>>     
>>>>>>     	spin_lock(&ctx->dma.lock);
>>>>>>     
>>>>>> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>>>>>     	if (buf) {
>>>>>>     		buf->vb.vb2_buf.timestamp = ktime_get_ns();
>>>>>>     		buf->vb.field = ctx->v_fmt.fmt.pix.field;
>>>>>> -		buf->vb.sequence = ctx->sequence++;
>>>>>> +		buf->vb.sequence = frame_num;
>>>>>
>>>>> We'll need something a bit more complicated. The CSI-2 frame number is
>>>>> not mandatory, and when used, it is a 16-bit number starting at 1 and
>>>>> counting to an unspecified value larger than one, resetting to 1 at the
>>>>> end of the cycle. The V4L2 sequence number, on the other hand, is a
>>>>> monotonic counter starting at 0 and wrapping only at 2^32-1. We should
>>>>> thus keep a software sequence counter and
>>>>>
>>>>> - increase it by 1 if the frame number is zero
>>>>> - increase it by frame_num - last_frame_num (with wrap-around of
>>>>>      frame_num handled) otherwise
>>>>
>>>> Ok... I wonder if we need a new field for this, though. The problem I
>>>> was solving when I changed this to use the CSI-2 frame-number was how to
>>>> associate a pixel frame and a metadata frame.
>>>>
>>>> Their CSI-2 frame-numbers match (as they are from the same original
>>>> CSI-2 frame), so the userspace can use that to figure the matching
>>>> frames. While the above method you suggest should give us identical
>>>> sequence numbers for pixel and metadata, I think it's going a bit away
>>>> from my intended purpose, and possibly risks ending up with different
>>>> sequences for pixel and metadata.
>>>
>>> Why do you think they could get out of sync (assuming the sensor
>>> supports frame numbers of course, if it always returns 0, that's not
>>> usable for the purpose of synchronization).
>>
>> If there's a requirement that the sequence starts from 0, it doesn't
>> work, as the pixel and metadata video capture may be started at
>> different times. When pixel capture starts, the frame number could be 10
>> and pixel sequence would be 0, but when metadata capture starts, the
>> frame number could be 12, and pixel seq would thus be 2 and meta seq 0.
>>
>> But even if we allow the seq to start from the current frame number,
> 
> Good point. I think we can allow starting at a non-zero value to handle
> this.
> 
>> this doesn't work if the frame number has wrapped between starting the
>> pixel capture and starting the meta capture.
> 
> The timestamp should be enough to handle this, the timestamp difference
> between two wraparounds should be large enough to sync the two streams
> without any risk.

Well, this still won't work, as CAL doesn't know when the sensor's frame 
counter wraps. CAL can detect that the counter has wrapped, but it 
doesn't know if some frames were missed. This leads to the two streams 
getting out of sync.

I'll try to figure out if I can somehow handle the frame counter in a 
shared manner, so that multiple streams that originate from the same 
frame would always use the same sequence numbers for same frame numbers.

  Tomi

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Tomi Valkeinen]

On 08/06/2021 10:38, Tomi Valkeinen wrote:
> On 07/06/2021 19:51, Laurent Pinchart wrote:
>> Hi Tomi,
>>
>> On Mon, Jun 07, 2021 at 05:55:05PM +0300, Tomi Valkeinen wrote:
>>> On 07/06/2021 16:42, Laurent Pinchart wrote:
>>>> On Mon, Jun 07, 2021 at 03:39:45PM +0300, Tomi Valkeinen wrote:
>>>>> On 04/06/2021 17:04, Laurent Pinchart wrote:
>>>>>> On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
>>>>>>> The driver fills buf->vb.sequence with an increasing number which is
>>>>>>> incremented by the driver. This feels a bit pointless, as the 
>>>>>>> userspace
>>>>>>> could as well track that kind of number itself. Instead, lets use 
>>>>>>> the
>>>>>>
>>>>>> s/lets/let's/
>>>>>>
>>>>>>> frame number provided in the CSI-2 data from the sensor.
>>>>>>>
>>>>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>>>>> ---
>>>>>>>     drivers/media/platform/ti-vpe/cal.c | 7 +++++--
>>>>>>>     drivers/media/platform/ti-vpe/cal.h | 1 -
>>>>>>>     2 files changed, 5 insertions(+), 3 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/media/platform/ti-vpe/cal.c 
>>>>>>> b/drivers/media/platform/ti-vpe/cal.c
>>>>>>> index 888706187fd1..62c45add4efe 100644
>>>>>>> --- a/drivers/media/platform/ti-vpe/cal.c
>>>>>>> +++ b/drivers/media/platform/ti-vpe/cal.c
>>>>>>> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>>>>>>>     void cal_ctx_start(struct cal_ctx *ctx)
>>>>>>>     {
>>>>>>> -    ctx->sequence = 0;
>>>>>>>         ctx->dma.state = CAL_DMA_RUNNING;
>>>>>>>         /* Configure the CSI-2, pixel processing and write DMA 
>>>>>>> contexts. */
>>>>>>> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct 
>>>>>>> cal_ctx *ctx)
>>>>>>>     static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>>>>>>     {
>>>>>>>         struct cal_buffer *buf = NULL;
>>>>>>> +    u32 frame_num;
>>>>>>> +
>>>>>>> +    frame_num = cal_read(ctx->cal, 
>>>>>>> CAL_CSI2_STATUS(ctx->phy->instance,
>>>>>>> +                               ctx->csi2_ctx)) & 0xffff;
>>>>>>>         spin_lock(&ctx->dma.lock);
>>>>>>> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct 
>>>>>>> cal_ctx *ctx)
>>>>>>>         if (buf) {
>>>>>>>             buf->vb.vb2_buf.timestamp = ktime_get_ns();
>>>>>>>             buf->vb.field = ctx->v_fmt.fmt.pix.field;
>>>>>>> -        buf->vb.sequence = ctx->sequence++;
>>>>>>> +        buf->vb.sequence = frame_num;
>>>>>>
>>>>>> We'll need something a bit more complicated. The CSI-2 frame 
>>>>>> number is
>>>>>> not mandatory, and when used, it is a 16-bit number starting at 1 and
>>>>>> counting to an unspecified value larger than one, resetting to 1 
>>>>>> at the
>>>>>> end of the cycle. The V4L2 sequence number, on the other hand, is a
>>>>>> monotonic counter starting at 0 and wrapping only at 2^32-1. We 
>>>>>> should
>>>>>> thus keep a software sequence counter and
>>>>>>
>>>>>> - increase it by 1 if the frame number is zero
>>>>>> - increase it by frame_num - last_frame_num (with wrap-around of
>>>>>>      frame_num handled) otherwise
>>>>>
>>>>> Ok... I wonder if we need a new field for this, though. The problem I
>>>>> was solving when I changed this to use the CSI-2 frame-number was 
>>>>> how to
>>>>> associate a pixel frame and a metadata frame.
>>>>>
>>>>> Their CSI-2 frame-numbers match (as they are from the same original
>>>>> CSI-2 frame), so the userspace can use that to figure the matching
>>>>> frames. While the above method you suggest should give us identical
>>>>> sequence numbers for pixel and metadata, I think it's going a bit away
>>>>> from my intended purpose, and possibly risks ending up with different
>>>>> sequences for pixel and metadata.
>>>>
>>>> Why do you think they could get out of sync (assuming the sensor
>>>> supports frame numbers of course, if it always returns 0, that's not
>>>> usable for the purpose of synchronization).
>>>
>>> If there's a requirement that the sequence starts from 0, it doesn't
>>> work, as the pixel and metadata video capture may be started at
>>> different times. When pixel capture starts, the frame number could be 10
>>> and pixel sequence would be 0, but when metadata capture starts, the
>>> frame number could be 12, and pixel seq would thus be 2 and meta seq 0.
>>>
>>> But even if we allow the seq to start from the current frame number,
>>
>> Good point. I think we can allow starting at a non-zero value to handle
>> this.
>>
>>> this doesn't work if the frame number has wrapped between starting the
>>> pixel capture and starting the meta capture.
>>
>> The timestamp should be enough to handle this, the timestamp difference
>> between two wraparounds should be large enough to sync the two streams
>> without any risk.
> 
> Well, this still won't work, as CAL doesn't know when the sensor's frame 
> counter wraps. CAL can detect that the counter has wrapped, but it 
> doesn't know if some frames were missed. This leads to the two streams 
> getting out of sync.
> 
> I'll try to figure out if I can somehow handle the frame counter in a 
> shared manner, so that multiple streams that originate from the same 
> frame would always use the same sequence numbers for same frame numbers.

How about something like this:

commit 22fb554bd2b71fa6d245e2595424639bee0ed604
Author: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Date:   Tue Apr 13 11:20:41 2021 +0300

     media: ti-vpe: cal: use CSI-2 frame number
     
     The userspace needs a way to match received metadata buffers to pixel
     data buffers. The obvious way to do this is to use the CSI-2 frame
     number, as both the metadata and the pixel data have the same frame
     number as they come from the same frame.
     
     However, we don't have means to convey the frame number to userspace. We
     do have the 'sequence' field, which with a few tricks can be used for
     this purpose.
     
     To achieve this, track the frame number for each virtual channel and
     increase the sequence for each virtual channel by frame-number -
     previous-frame-number, also taking into account the eventual wrap of the
     CSI-2 frame number.
     
     This way we get a monotonically increasing sequence number which is
     common to all streams using the same virtual channel.
     
     Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 82392499e663..3ca629278fd4 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -801,6 +801,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
  	phy->cal = cal;
  	phy->instance = instance;
  
+	spin_lock_init(&phy->vc_lock);
+
  	phy->res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
  						(instance == 0) ?
  						"cal_rx_core0" :
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 7a92bb9429d4..0594e47a2c88 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -493,7 +493,22 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
  
  void cal_ctx_start(struct cal_ctx *ctx)
  {
-	ctx->sequence = 0;
+	struct cal_camerarx *phy = ctx->phy;
+
+	/*
+	 * Reset the frame number & sequence number, but only if the
+	 * virtual channel is not already in use.
+	 */
+
+	spin_lock(&phy->vc_lock);
+
+	if (phy->vc_enable_count[ctx->vc]++ == 0) {
+		phy->vc_frame_number[ctx->vc] = 0;
+		phy->vc_sequence[ctx->vc] = 0;
+	}
+
+	spin_unlock(&phy->vc_lock);
+
  	ctx->dma.state = CAL_DMA_RUNNING;
  
  	/* Configure the CSI-2, pixel processing and write DMA contexts. */
@@ -513,8 +528,15 @@ void cal_ctx_start(struct cal_ctx *ctx)
  
  void cal_ctx_stop(struct cal_ctx *ctx)
  {
+	struct cal_camerarx *phy = ctx->phy;
  	long timeout;
  
+	WARN_ON(phy->vc_enable_count[ctx->vc] == 0);
+
+	spin_lock(&phy->vc_lock);
+	phy->vc_enable_count[ctx->vc]--;
+	spin_unlock(&phy->vc_lock);
+
  	/*
  	 * Request DMA stop and wait until it completes. If completion times
  	 * out, forcefully disable the DMA.
@@ -586,7 +608,6 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
  static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
  {
  	struct cal_buffer *buf = NULL;
-
  	spin_lock(&ctx->dma.lock);
  
  	/* If the DMA context was stopping, it is now stopped. */
@@ -605,9 +626,28 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
  	spin_unlock(&ctx->dma.lock);
  
  	if (buf) {
+		struct cal_dev *cal = ctx->cal;
+		struct cal_camerarx *phy = ctx->phy;
+		u32 prev_frame_num, frame_num;
+		u8 vc = ctx->vc;
+
+		frame_num = cal_read(cal, CAL_CSI2_STATUS(ctx->phy->instance,
+							  ctx->csi2_ctx)) & 0xffff;
+
+		if (phy->vc_frame_number[vc] != frame_num) {
+			prev_frame_num = phy->vc_frame_number[vc];
+
+			if (prev_frame_num > frame_num)
+				prev_frame_num = 0;
+
+			phy->vc_sequence[vc] += frame_num - prev_frame_num;
+			phy->vc_frame_number[vc] = frame_num;
+		}
+
  		buf->vb.vb2_buf.timestamp = ktime_get_ns();
  		buf->vb.field = ctx->v_fmt.fmt.pix.field;
-		buf->vb.sequence = ctx->sequence++;
+		buf->vb.sequence = phy->vc_sequence[vc];
+
  		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
  	}
  }
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 400f95485d7c..309413e8a932 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -163,6 +163,12 @@ struct cal_camerarx {
  	struct v4l2_subdev	subdev;
  	struct media_pad	pads[2];
  	struct v4l2_mbus_framefmt	formats[2];
+
+	/* protects the vc_* fields below */
+	spinlock_t		vc_lock;
+	u8			vc_enable_count[4];
+	u8			vc_frame_number[4];
+	u32			vc_sequence[4];
  };
  
  struct cal_dev {
@@ -217,7 +223,6 @@ struct cal_ctx {
  	const struct cal_format_info	**active_fmt;
  	unsigned int		num_active_fmt;
  
-	unsigned int		sequence;
  	struct vb2_queue	vb_vidq;
  	u8			dma_ctx;
  	u8			cport;

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Laurent Pinchart]

Hi Tomi,

On Tue, Jun 08, 2021 at 03:46:21PM +0300, Tomi Valkeinen wrote:
> On 08/06/2021 10:38, Tomi Valkeinen wrote:
> > On 07/06/2021 19:51, Laurent Pinchart wrote:
> >> On Mon, Jun 07, 2021 at 05:55:05PM +0300, Tomi Valkeinen wrote:
> >>> On 07/06/2021 16:42, Laurent Pinchart wrote:
> >>>> On Mon, Jun 07, 2021 at 03:39:45PM +0300, Tomi Valkeinen wrote:
> >>>>> On 04/06/2021 17:04, Laurent Pinchart wrote:
> >>>>>> On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
> >>>>>>> The driver fills buf->vb.sequence with an increasing number which is
> >>>>>>> incremented by the driver. This feels a bit pointless, as the userspace
> >>>>>>> could as well track that kind of number itself. Instead, lets use the
> >>>>>>
> >>>>>> s/lets/let's/
> >>>>>>
> >>>>>>> frame number provided in the CSI-2 data from the sensor.
> >>>>>>>
> >>>>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >>>>>>> ---
> >>>>>>>     drivers/media/platform/ti-vpe/cal.c | 7 +++++--
> >>>>>>>     drivers/media/platform/ti-vpe/cal.h | 1 -
> >>>>>>>     2 files changed, 5 insertions(+), 3 deletions(-)
> >>>>>>>
> >>>>>>> diff --git a/drivers/media/platform/ti-vpe/cal.c 
> >>>>>>> b/drivers/media/platform/ti-vpe/cal.c
> >>>>>>> index 888706187fd1..62c45add4efe 100644
> >>>>>>> --- a/drivers/media/platform/ti-vpe/cal.c
> >>>>>>> +++ b/drivers/media/platform/ti-vpe/cal.c
> >>>>>>> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
> >>>>>>>     void cal_ctx_start(struct cal_ctx *ctx)
> >>>>>>>     {
> >>>>>>> -    ctx->sequence = 0;
> >>>>>>>         ctx->dma.state = CAL_DMA_RUNNING;
> >>>>>>>         /* Configure the CSI-2, pixel processing and write DMA 
> >>>>>>> contexts. */
> >>>>>>> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct 
> >>>>>>> cal_ctx *ctx)
> >>>>>>>     static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
> >>>>>>>     {
> >>>>>>>         struct cal_buffer *buf = NULL;
> >>>>>>> +    u32 frame_num;
> >>>>>>> +
> >>>>>>> +    frame_num = cal_read(ctx->cal, 
> >>>>>>> CAL_CSI2_STATUS(ctx->phy->instance,
> >>>>>>> +                               ctx->csi2_ctx)) & 0xffff;
> >>>>>>>         spin_lock(&ctx->dma.lock);
> >>>>>>> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct 
> >>>>>>> cal_ctx *ctx)
> >>>>>>>         if (buf) {
> >>>>>>>             buf->vb.vb2_buf.timestamp = ktime_get_ns();
> >>>>>>>             buf->vb.field = ctx->v_fmt.fmt.pix.field;
> >>>>>>> -        buf->vb.sequence = ctx->sequence++;
> >>>>>>> +        buf->vb.sequence = frame_num;
> >>>>>>
> >>>>>> We'll need something a bit more complicated. The CSI-2 frame 
> >>>>>> number is
> >>>>>> not mandatory, and when used, it is a 16-bit number starting at 1 and
> >>>>>> counting to an unspecified value larger than one, resetting to 1 
> >>>>>> at the
> >>>>>> end of the cycle. The V4L2 sequence number, on the other hand, is a
> >>>>>> monotonic counter starting at 0 and wrapping only at 2^32-1. We 
> >>>>>> should
> >>>>>> thus keep a software sequence counter and
> >>>>>>
> >>>>>> - increase it by 1 if the frame number is zero
> >>>>>> - increase it by frame_num - last_frame_num (with wrap-around of
> >>>>>>      frame_num handled) otherwise
> >>>>>
> >>>>> Ok... I wonder if we need a new field for this, though. The problem I
> >>>>> was solving when I changed this to use the CSI-2 frame-number was 
> >>>>> how to
> >>>>> associate a pixel frame and a metadata frame.
> >>>>>
> >>>>> Their CSI-2 frame-numbers match (as they are from the same original
> >>>>> CSI-2 frame), so the userspace can use that to figure the matching
> >>>>> frames. While the above method you suggest should give us identical
> >>>>> sequence numbers for pixel and metadata, I think it's going a bit away
> >>>>> from my intended purpose, and possibly risks ending up with different
> >>>>> sequences for pixel and metadata.
> >>>>
> >>>> Why do you think they could get out of sync (assuming the sensor
> >>>> supports frame numbers of course, if it always returns 0, that's not
> >>>> usable for the purpose of synchronization).
> >>>
> >>> If there's a requirement that the sequence starts from 0, it doesn't
> >>> work, as the pixel and metadata video capture may be started at
> >>> different times. When pixel capture starts, the frame number could be 10
> >>> and pixel sequence would be 0, but when metadata capture starts, the
> >>> frame number could be 12, and pixel seq would thus be 2 and meta seq 0.
> >>>
> >>> But even if we allow the seq to start from the current frame number,
> >>
> >> Good point. I think we can allow starting at a non-zero value to handle
> >> this.
> >>
> >>> this doesn't work if the frame number has wrapped between starting the
> >>> pixel capture and starting the meta capture.
> >>
> >> The timestamp should be enough to handle this, the timestamp difference
> >> between two wraparounds should be large enough to sync the two streams
> >> without any risk.
> > 
> > Well, this still won't work, as CAL doesn't know when the sensor's frame 
> > counter wraps. CAL can detect that the counter has wrapped, but it 
> > doesn't know if some frames were missed. This leads to the two streams 
> > getting out of sync.
> > 
> > I'll try to figure out if I can somehow handle the frame counter in a 
> > shared manner, so that multiple streams that originate from the same 
> > frame would always use the same sequence numbers for same frame numbers.
> 
> How about something like this:
> 
> commit 22fb554bd2b71fa6d245e2595424639bee0ed604
> Author: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> Date:   Tue Apr 13 11:20:41 2021 +0300
> 
>      media: ti-vpe: cal: use CSI-2 frame number
>      
>      The userspace needs a way to match received metadata buffers to pixel
>      data buffers. The obvious way to do this is to use the CSI-2 frame
>      number, as both the metadata and the pixel data have the same frame
>      number as they come from the same frame.
>      
>      However, we don't have means to convey the frame number to userspace. We
>      do have the 'sequence' field, which with a few tricks can be used for
>      this purpose.
>      
>      To achieve this, track the frame number for each virtual channel and
>      increase the sequence for each virtual channel by frame-number -
>      previous-frame-number, also taking into account the eventual wrap of the
>      CSI-2 frame number.
>      
>      This way we get a monotonically increasing sequence number which is
>      common to all streams using the same virtual channel.
>      
>      Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index 82392499e663..3ca629278fd4 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -801,6 +801,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>   	phy->cal = cal;
>   	phy->instance = instance;
>   
> +	spin_lock_init(&phy->vc_lock);
> +
>   	phy->res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
>   						(instance == 0) ?
>   						"cal_rx_core0" :
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index 7a92bb9429d4..0594e47a2c88 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -493,7 +493,22 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>   
>   void cal_ctx_start(struct cal_ctx *ctx)
>   {
> -	ctx->sequence = 0;
> +	struct cal_camerarx *phy = ctx->phy;
> +
> +	/*
> +	 * Reset the frame number & sequence number, but only if the
> +	 * virtual channel is not already in use.
> +	 */
> +
> +	spin_lock(&phy->vc_lock);
> +
> +	if (phy->vc_enable_count[ctx->vc]++ == 0) {
> +		phy->vc_frame_number[ctx->vc] = 0;
> +		phy->vc_sequence[ctx->vc] = 0;
> +	}
> +
> +	spin_unlock(&phy->vc_lock);
> +
>   	ctx->dma.state = CAL_DMA_RUNNING;
>   
>   	/* Configure the CSI-2, pixel processing and write DMA contexts. */
> @@ -513,8 +528,15 @@ void cal_ctx_start(struct cal_ctx *ctx)
>   
>   void cal_ctx_stop(struct cal_ctx *ctx)
>   {
> +	struct cal_camerarx *phy = ctx->phy;
>   	long timeout;
>   
> +	WARN_ON(phy->vc_enable_count[ctx->vc] == 0);
> +
> +	spin_lock(&phy->vc_lock);
> +	phy->vc_enable_count[ctx->vc]--;
> +	spin_unlock(&phy->vc_lock);

Can cal_ctx_start() and cal_ctx_stop() be called concurrently, aren't
they serialized at higher levels ? You could drop the spinlock in that
case.

> +
>   	/*
>   	 * Request DMA stop and wait until it completes. If completion times
>   	 * out, forcefully disable the DMA.
> @@ -586,7 +608,6 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
>   static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>   {
>   	struct cal_buffer *buf = NULL;
> -

Doesn't checkpatch warn about this ?

>   	spin_lock(&ctx->dma.lock);
>   
>   	/* If the DMA context was stopping, it is now stopped. */
> @@ -605,9 +626,28 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>   	spin_unlock(&ctx->dma.lock);
>   
>   	if (buf) {
> +		struct cal_dev *cal = ctx->cal;
> +		struct cal_camerarx *phy = ctx->phy;
> +		u32 prev_frame_num, frame_num;
> +		u8 vc = ctx->vc;
> +
> +		frame_num = cal_read(cal, CAL_CSI2_STATUS(ctx->phy->instance,
> +							  ctx->csi2_ctx)) & 0xffff;

I'm worried about race conditions here, as this is the DMA end IRQ
handler. When it the frame number updated in this register ?

> +
> +		if (phy->vc_frame_number[vc] != frame_num) {
> +			prev_frame_num = phy->vc_frame_number[vc];
> +
> +			if (prev_frame_num > frame_num)
> +				prev_frame_num = 0;

If we happen to miss a frame when wrapping around, I don't think this
will produce the right result. We don't know when the counter wraps
around, so there's not much we can do, except perhaps keeping track of
the maximum frame number to figure out when it wraps around. It may be
overkill, I'm not sure.

> +
> +			phy->vc_sequence[vc] += frame_num - prev_frame_num;
> +			phy->vc_frame_number[vc] = frame_num;
> +		}
> +
>   		buf->vb.vb2_buf.timestamp = ktime_get_ns();
>   		buf->vb.field = ctx->v_fmt.fmt.pix.field;
> -		buf->vb.sequence = ctx->sequence++;
> +		buf->vb.sequence = phy->vc_sequence[vc];
> +
>   		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
>   	}
>   }
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 400f95485d7c..309413e8a932 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -163,6 +163,12 @@ struct cal_camerarx {
>   	struct v4l2_subdev	subdev;
>   	struct media_pad	pads[2];
>   	struct v4l2_mbus_framefmt	formats[2];
> +
> +	/* protects the vc_* fields below */
> +	spinlock_t		vc_lock;
> +	u8			vc_enable_count[4];
> +	u8			vc_frame_number[4];
> +	u32			vc_sequence[4];
>   };
>   
>   struct cal_dev {
> @@ -217,7 +223,6 @@ struct cal_ctx {
>   	const struct cal_format_info	**active_fmt;
>   	unsigned int		num_active_fmt;
>   
> -	unsigned int		sequence;
>   	struct vb2_queue	vb_vidq;
>   	u8			dma_ctx;
>   	u8			cport;

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 32/38] media: ti-vpe: cal: use CSI-2 frame number

[Tomi Valkeinen]

On 09/06/2021 15:47, Laurent Pinchart wrote:
> Hi Tomi,
> 
> On Tue, Jun 08, 2021 at 03:46:21PM +0300, Tomi Valkeinen wrote:
>> On 08/06/2021 10:38, Tomi Valkeinen wrote:
>>> On 07/06/2021 19:51, Laurent Pinchart wrote:
>>>> On Mon, Jun 07, 2021 at 05:55:05PM +0300, Tomi Valkeinen wrote:
>>>>> On 07/06/2021 16:42, Laurent Pinchart wrote:
>>>>>> On Mon, Jun 07, 2021 at 03:39:45PM +0300, Tomi Valkeinen wrote:
>>>>>>> On 04/06/2021 17:04, Laurent Pinchart wrote:
>>>>>>>> On Mon, May 24, 2021 at 02:09:03PM +0300, Tomi Valkeinen wrote:
>>>>>>>>> The driver fills buf->vb.sequence with an increasing number which is
>>>>>>>>> incremented by the driver. This feels a bit pointless, as the userspace
>>>>>>>>> could as well track that kind of number itself. Instead, lets use the
>>>>>>>>
>>>>>>>> s/lets/let's/
>>>>>>>>
>>>>>>>>> frame number provided in the CSI-2 data from the sensor.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>>>>>>> ---
>>>>>>>>>      drivers/media/platform/ti-vpe/cal.c | 7 +++++--
>>>>>>>>>      drivers/media/platform/ti-vpe/cal.h | 1 -
>>>>>>>>>      2 files changed, 5 insertions(+), 3 deletions(-)
>>>>>>>>>
>>>>>>>>> diff --git a/drivers/media/platform/ti-vpe/cal.c
>>>>>>>>> b/drivers/media/platform/ti-vpe/cal.c
>>>>>>>>> index 888706187fd1..62c45add4efe 100644
>>>>>>>>> --- a/drivers/media/platform/ti-vpe/cal.c
>>>>>>>>> +++ b/drivers/media/platform/ti-vpe/cal.c
>>>>>>>>> @@ -493,7 +493,6 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>>>>>>>>>      void cal_ctx_start(struct cal_ctx *ctx)
>>>>>>>>>      {
>>>>>>>>> -    ctx->sequence = 0;
>>>>>>>>>          ctx->dma.state = CAL_DMA_RUNNING;
>>>>>>>>>          /* Configure the CSI-2, pixel processing and write DMA
>>>>>>>>> contexts. */
>>>>>>>>> @@ -586,6 +585,10 @@ static inline void cal_irq_wdma_start(struct
>>>>>>>>> cal_ctx *ctx)
>>>>>>>>>      static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>>>>>>>>      {
>>>>>>>>>          struct cal_buffer *buf = NULL;
>>>>>>>>> +    u32 frame_num;
>>>>>>>>> +
>>>>>>>>> +    frame_num = cal_read(ctx->cal,
>>>>>>>>> CAL_CSI2_STATUS(ctx->phy->instance,
>>>>>>>>> +                               ctx->csi2_ctx)) & 0xffff;
>>>>>>>>>          spin_lock(&ctx->dma.lock);
>>>>>>>>> @@ -607,7 +610,7 @@ static inline void cal_irq_wdma_end(struct
>>>>>>>>> cal_ctx *ctx)
>>>>>>>>>          if (buf) {
>>>>>>>>>              buf->vb.vb2_buf.timestamp = ktime_get_ns();
>>>>>>>>>              buf->vb.field = ctx->v_fmt.fmt.pix.field;
>>>>>>>>> -        buf->vb.sequence = ctx->sequence++;
>>>>>>>>> +        buf->vb.sequence = frame_num;
>>>>>>>>
>>>>>>>> We'll need something a bit more complicated. The CSI-2 frame
>>>>>>>> number is
>>>>>>>> not mandatory, and when used, it is a 16-bit number starting at 1 and
>>>>>>>> counting to an unspecified value larger than one, resetting to 1
>>>>>>>> at the
>>>>>>>> end of the cycle. The V4L2 sequence number, on the other hand, is a
>>>>>>>> monotonic counter starting at 0 and wrapping only at 2^32-1. We
>>>>>>>> should
>>>>>>>> thus keep a software sequence counter and
>>>>>>>>
>>>>>>>> - increase it by 1 if the frame number is zero
>>>>>>>> - increase it by frame_num - last_frame_num (with wrap-around of
>>>>>>>>       frame_num handled) otherwise
>>>>>>>
>>>>>>> Ok... I wonder if we need a new field for this, though. The problem I
>>>>>>> was solving when I changed this to use the CSI-2 frame-number was
>>>>>>> how to
>>>>>>> associate a pixel frame and a metadata frame.
>>>>>>>
>>>>>>> Their CSI-2 frame-numbers match (as they are from the same original
>>>>>>> CSI-2 frame), so the userspace can use that to figure the matching
>>>>>>> frames. While the above method you suggest should give us identical
>>>>>>> sequence numbers for pixel and metadata, I think it's going a bit away
>>>>>>> from my intended purpose, and possibly risks ending up with different
>>>>>>> sequences for pixel and metadata.
>>>>>>
>>>>>> Why do you think they could get out of sync (assuming the sensor
>>>>>> supports frame numbers of course, if it always returns 0, that's not
>>>>>> usable for the purpose of synchronization).
>>>>>
>>>>> If there's a requirement that the sequence starts from 0, it doesn't
>>>>> work, as the pixel and metadata video capture may be started at
>>>>> different times. When pixel capture starts, the frame number could be 10
>>>>> and pixel sequence would be 0, but when metadata capture starts, the
>>>>> frame number could be 12, and pixel seq would thus be 2 and meta seq 0.
>>>>>
>>>>> But even if we allow the seq to start from the current frame number,
>>>>
>>>> Good point. I think we can allow starting at a non-zero value to handle
>>>> this.
>>>>
>>>>> this doesn't work if the frame number has wrapped between starting the
>>>>> pixel capture and starting the meta capture.
>>>>
>>>> The timestamp should be enough to handle this, the timestamp difference
>>>> between two wraparounds should be large enough to sync the two streams
>>>> without any risk.
>>>
>>> Well, this still won't work, as CAL doesn't know when the sensor's frame
>>> counter wraps. CAL can detect that the counter has wrapped, but it
>>> doesn't know if some frames were missed. This leads to the two streams
>>> getting out of sync.
>>>
>>> I'll try to figure out if I can somehow handle the frame counter in a
>>> shared manner, so that multiple streams that originate from the same
>>> frame would always use the same sequence numbers for same frame numbers.
>>
>> How about something like this:
>>
>> commit 22fb554bd2b71fa6d245e2595424639bee0ed604
>> Author: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> Date:   Tue Apr 13 11:20:41 2021 +0300
>>
>>       media: ti-vpe: cal: use CSI-2 frame number
>>       
>>       The userspace needs a way to match received metadata buffers to pixel
>>       data buffers. The obvious way to do this is to use the CSI-2 frame
>>       number, as both the metadata and the pixel data have the same frame
>>       number as they come from the same frame.
>>       
>>       However, we don't have means to convey the frame number to userspace. We
>>       do have the 'sequence' field, which with a few tricks can be used for
>>       this purpose.
>>       
>>       To achieve this, track the frame number for each virtual channel and
>>       increase the sequence for each virtual channel by frame-number -
>>       previous-frame-number, also taking into account the eventual wrap of the
>>       CSI-2 frame number.
>>       
>>       This way we get a monotonically increasing sequence number which is
>>       common to all streams using the same virtual channel.
>>       
>>       Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>
>> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> index 82392499e663..3ca629278fd4 100644
>> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
>> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> @@ -801,6 +801,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>>    	phy->cal = cal;
>>    	phy->instance = instance;
>>    
>> +	spin_lock_init(&phy->vc_lock);
>> +
>>    	phy->res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
>>    						(instance == 0) ?
>>    						"cal_rx_core0" :
>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>> index 7a92bb9429d4..0594e47a2c88 100644
>> --- a/drivers/media/platform/ti-vpe/cal.c
>> +++ b/drivers/media/platform/ti-vpe/cal.c
>> @@ -493,7 +493,22 @@ void cal_ctx_unprepare(struct cal_ctx *ctx)
>>    
>>    void cal_ctx_start(struct cal_ctx *ctx)
>>    {
>> -	ctx->sequence = 0;
>> +	struct cal_camerarx *phy = ctx->phy;
>> +
>> +	/*
>> +	 * Reset the frame number & sequence number, but only if the
>> +	 * virtual channel is not already in use.
>> +	 */
>> +
>> +	spin_lock(&phy->vc_lock);
>> +
>> +	if (phy->vc_enable_count[ctx->vc]++ == 0) {
>> +		phy->vc_frame_number[ctx->vc] = 0;
>> +		phy->vc_sequence[ctx->vc] = 0;
>> +	}
>> +
>> +	spin_unlock(&phy->vc_lock);
>> +
>>    	ctx->dma.state = CAL_DMA_RUNNING;
>>    
>>    	/* Configure the CSI-2, pixel processing and write DMA contexts. */
>> @@ -513,8 +528,15 @@ void cal_ctx_start(struct cal_ctx *ctx)
>>    
>>    void cal_ctx_stop(struct cal_ctx *ctx)
>>    {
>> +	struct cal_camerarx *phy = ctx->phy;
>>    	long timeout;
>>    
>> +	WARN_ON(phy->vc_enable_count[ctx->vc] == 0);
>> +
>> +	spin_lock(&phy->vc_lock);
>> +	phy->vc_enable_count[ctx->vc]--;
>> +	spin_unlock(&phy->vc_lock);
> 
> Can cal_ctx_start() and cal_ctx_stop() be called concurrently, aren't
> they serialized at higher levels ? You could drop the spinlock in that
> case.

No, I don't there's anything serializing these calls.

>> +
>>    	/*
>>    	 * Request DMA stop and wait until it completes. If completion times
>>    	 * out, forcefully disable the DMA.
>> @@ -586,7 +608,6 @@ static inline void cal_irq_wdma_start(struct cal_ctx *ctx)
>>    static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>    {
>>    	struct cal_buffer *buf = NULL;
>> -
> 
> Doesn't checkpatch warn about this ?

Oops, I somehow missed that.

>>    	spin_lock(&ctx->dma.lock);
>>    
>>    	/* If the DMA context was stopping, it is now stopped. */
>> @@ -605,9 +626,28 @@ static inline void cal_irq_wdma_end(struct cal_ctx *ctx)
>>    	spin_unlock(&ctx->dma.lock);
>>    
>>    	if (buf) {
>> +		struct cal_dev *cal = ctx->cal;
>> +		struct cal_camerarx *phy = ctx->phy;
>> +		u32 prev_frame_num, frame_num;
>> +		u8 vc = ctx->vc;
>> +
>> +		frame_num = cal_read(cal, CAL_CSI2_STATUS(ctx->phy->instance,
>> +							  ctx->csi2_ctx)) & 0xffff;
> 
> I'm worried about race conditions here, as this is the DMA end IRQ
> handler. When it the frame number updated in this register ?

Afaik, it's updated when the CSI-2 RX gets the frame-start packet. So, 
while we'll always be racy, I guess this could use some improvement. If 
we record the frame_num at wdma start irq, we have more time to avoid 
the race.

Of course, the DMA itself is racy with CAL, and if we get multiple 
interrupts coalesced to one, I think we can't expect the frame number 
handling to work flawlessly either.

>> +
>> +		if (phy->vc_frame_number[vc] != frame_num) {
>> +			prev_frame_num = phy->vc_frame_number[vc];
>> +
>> +			if (prev_frame_num > frame_num)
>> +				prev_frame_num = 0;
> 
> If we happen to miss a frame when wrapping around, I don't think this
> will produce the right result. We don't know when the counter wraps
> around, so there's not much we can do, except perhaps keeping track of
> the maximum frame number to figure out when it wraps around. It may be
> overkill, I'm not sure.

What do you think happens? Afaics, we'll just skip some sequence numbers.

  Tomi

[PATCH v3 33/38] media: ti-vpe: cal: add camerarx locking

[Tomi Valkeinen]

We don't have any locking in camerarx for the subdev ops. We have
managed fine so far without locking, but in the future multiple video
capture devices can use the same camerarx, and locking is a must.

Add a mutex to protect the camerarx subdev ops. Some of the functions
were slightly restructured to make lock handling cleaner.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 81 ++++++++++++++------
 drivers/media/platform/ti-vpe/cal.h          |  3 +
 2 files changed, 61 insertions(+), 23 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 82392499e663..b87ffc52feb6 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -601,12 +601,18 @@ cal_camerarx_get_pad_format(struct cal_camerarx *phy,
 static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
 {
 	struct cal_camerarx *phy = to_cal_camerarx(sd);
+	int r = 0;
+
+	mutex_lock(&phy->mutex);
 
 	if (enable)
-		return cal_camerarx_start(phy);
+		r = cal_camerarx_start(phy);
+	else
+		cal_camerarx_stop(phy);
 
-	cal_camerarx_stop(phy);
-	return 0;
+	mutex_unlock(&phy->mutex);
+
+	return r;
 }
 
 static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
@@ -614,27 +620,36 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
 					  struct v4l2_subdev_mbus_code_enum *code)
 {
 	struct cal_camerarx *phy = to_cal_camerarx(sd);
+	int r = 0;
+
+	mutex_lock(&phy->mutex);
 
 	/* No transcoding, source and sink codes must match. */
 	if (code->pad == CAL_CAMERARX_PAD_SOURCE) {
 		struct v4l2_mbus_framefmt *fmt;
 
-		if (code->index > 0)
-			return -EINVAL;
+		if (code->index > 0) {
+			r = -EINVAL;
+			goto out;
+		}
 
 		fmt = cal_camerarx_get_pad_format(phy, sd_state,
 						  CAL_CAMERARX_PAD_SINK,
 						  code->which);
 		code->code = fmt->code;
-		return 0;
-	}
+	} else {
+		if (code->index >= cal_num_formats) {
+			r = -EINVAL;
+			goto out;
+		}
 
-	if (code->index >= cal_num_formats)
-		return -EINVAL;
+		code->code = cal_formats[code->index].code;
+	}
 
-	code->code = cal_formats[code->index].code;
+out:
+	mutex_unlock(&phy->mutex);
 
-	return 0;
+	return r;
 }
 
 static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
@@ -643,10 +658,13 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
 {
 	struct cal_camerarx *phy = to_cal_camerarx(sd);
 	const struct cal_format_info *fmtinfo;
+	int r = 0;
 
 	if (fse->index > 0)
 		return -EINVAL;
 
+	mutex_lock(&phy->mutex);
+
 	/* No transcoding, source and sink formats must match. */
 	if (fse->pad == CAL_CAMERARX_PAD_SOURCE) {
 		struct v4l2_mbus_framefmt *fmt;
@@ -654,27 +672,34 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
 		fmt = cal_camerarx_get_pad_format(phy, sd_state,
 						  CAL_CAMERARX_PAD_SINK,
 						  fse->which);
-		if (fse->code != fmt->code)
-			return -EINVAL;
+		if (fse->code != fmt->code) {
+			r = -EINVAL;
+			goto out;
+		}
 
 		fse->min_width = fmt->width;
 		fse->max_width = fmt->width;
 		fse->min_height = fmt->height;
 		fse->max_height = fmt->height;
+	} else {
+		fmtinfo = cal_format_by_code(fse->code);
+		if (!fmtinfo) {
+			r = -EINVAL;
+			goto out;
+		}
 
-		return 0;
+		fse->min_width =
+			CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
+		fse->max_width =
+			CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
+		fse->min_height = CAL_MIN_HEIGHT_LINES;
+		fse->max_height = CAL_MAX_HEIGHT_LINES;
 	}
 
-	fmtinfo = cal_format_by_code(fse->code);
-	if (!fmtinfo)
-		return -EINVAL;
-
-	fse->min_width = CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
-	fse->max_width = CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
-	fse->min_height = CAL_MIN_HEIGHT_LINES;
-	fse->max_height = CAL_MAX_HEIGHT_LINES;
+out:
+	mutex_unlock(&phy->mutex);
 
-	return 0;
+	return r;
 }
 
 static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
@@ -684,10 +709,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
 	struct cal_camerarx *phy = to_cal_camerarx(sd);
 	struct v4l2_mbus_framefmt *fmt;
 
+	mutex_lock(&phy->mutex);
+
 	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
 					  format->which);
 	format->format = *fmt;
 
+	mutex_unlock(&phy->mutex);
+
 	return 0;
 }
 
@@ -725,6 +754,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 	format->format.field = V4L2_FIELD_NONE;
 
 	/* Store the format and propagate it to the source pad. */
+	mutex_lock(&phy->mutex);
+
 	fmt = cal_camerarx_get_pad_format(phy, sd_state,
 					  CAL_CAMERARX_PAD_SINK,
 					  format->which);
@@ -735,6 +766,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 					  format->which);
 	*fmt = format->format;
 
+	mutex_unlock(&phy->mutex);
+
 	return 0;
 }
 
@@ -801,6 +834,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	phy->cal = cal;
 	phy->instance = instance;
 
+	mutex_init(&phy->mutex);
+
 	phy->res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
 						(instance == 0) ?
 						"cal_rx_core0" :
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index ad08c189ad3b..78bd2e041d9a 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -163,6 +163,9 @@ struct cal_camerarx {
 	struct v4l2_subdev	subdev;
 	struct media_pad	pads[2];
 	struct v4l2_mbus_framefmt	formats[2];
+
+	/* mutex for camerarx ops */
+	struct mutex		mutex;
 };
 
 struct cal_dev {
-- 
2.25.1

Re: [PATCH v3 33/38] media: ti-vpe: cal: add camerarx locking

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:04PM +0300, Tomi Valkeinen wrote:
> We don't have any locking in camerarx for the subdev ops. We have
> managed fine so far without locking, but in the future multiple video
> capture devices can use the same camerarx, and locking is a must.
> 
> Add a mutex to protect the camerarx subdev ops. Some of the functions
> were slightly restructured to make lock handling cleaner.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 81 ++++++++++++++------
>  drivers/media/platform/ti-vpe/cal.h          |  3 +
>  2 files changed, 61 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index 82392499e663..b87ffc52feb6 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -601,12 +601,18 @@ cal_camerarx_get_pad_format(struct cal_camerarx *phy,
>  static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
>  {
>  	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +	int r = 0;

The driver uses ret :-)

> +
> +	mutex_lock(&phy->mutex);
>  
>  	if (enable)
> -		return cal_camerarx_start(phy);
> +		r = cal_camerarx_start(phy);
> +	else
> +		cal_camerarx_stop(phy);
>  
> -	cal_camerarx_stop(phy);
> -	return 0;
> +	mutex_unlock(&phy->mutex);
> +
> +	return r;
>  }
>  
>  static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
> @@ -614,27 +620,36 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
>  					  struct v4l2_subdev_mbus_code_enum *code)
>  {
>  	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +	int r = 0;
> +
> +	mutex_lock(&phy->mutex);
>  
>  	/* No transcoding, source and sink codes must match. */
>  	if (code->pad == CAL_CAMERARX_PAD_SOURCE) {
>  		struct v4l2_mbus_framefmt *fmt;
>  
> -		if (code->index > 0)
> -			return -EINVAL;
> +		if (code->index > 0) {
> +			r = -EINVAL;
> +			goto out;
> +		}
>  
>  		fmt = cal_camerarx_get_pad_format(phy, sd_state,
>  						  CAL_CAMERARX_PAD_SINK,
>  						  code->which);
>  		code->code = fmt->code;
> -		return 0;
> -	}
> +	} else {
> +		if (code->index >= cal_num_formats) {
> +			r = -EINVAL;
> +			goto out;
> +		}
>  
> -	if (code->index >= cal_num_formats)
> -		return -EINVAL;
> +		code->code = cal_formats[code->index].code;
> +	}
>  
> -	code->code = cal_formats[code->index].code;
> +out:
> +	mutex_unlock(&phy->mutex);
>  
> -	return 0;
> +	return r;
>  }
>  
>  static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
> @@ -643,10 +658,13 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>  {
>  	struct cal_camerarx *phy = to_cal_camerarx(sd);
>  	const struct cal_format_info *fmtinfo;
> +	int r = 0;
>  
>  	if (fse->index > 0)
>  		return -EINVAL;
>  
> +	mutex_lock(&phy->mutex);
> +
>  	/* No transcoding, source and sink formats must match. */
>  	if (fse->pad == CAL_CAMERARX_PAD_SOURCE) {
>  		struct v4l2_mbus_framefmt *fmt;
> @@ -654,27 +672,34 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>  		fmt = cal_camerarx_get_pad_format(phy, sd_state,
>  						  CAL_CAMERARX_PAD_SINK,
>  						  fse->which);
> -		if (fse->code != fmt->code)
> -			return -EINVAL;
> +		if (fse->code != fmt->code) {
> +			r = -EINVAL;
> +			goto out;
> +		}
>  
>  		fse->min_width = fmt->width;
>  		fse->max_width = fmt->width;
>  		fse->min_height = fmt->height;
>  		fse->max_height = fmt->height;
> +	} else {
> +		fmtinfo = cal_format_by_code(fse->code);
> +		if (!fmtinfo) {
> +			r = -EINVAL;
> +			goto out;
> +		}
>  
> -		return 0;
> +		fse->min_width =
> +			CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> +		fse->max_width =
> +			CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);

This is a case where I'd write

		fse->min_width = CAL_MIN_WIDTH_BYTES * 8
			       / ALIGN(fmtinfo->bpp, 8);
		fse->max_width = CAL_MAX_WIDTH_BYTES * 8
			       / ALIGN(fmtinfo->bpp, 8);

or go slightly over 80 columns.

> +		fse->min_height = CAL_MIN_HEIGHT_LINES;
> +		fse->max_height = CAL_MAX_HEIGHT_LINES;
>  	}
>  
> -	fmtinfo = cal_format_by_code(fse->code);
> -	if (!fmtinfo)
> -		return -EINVAL;
> -
> -	fse->min_width = CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> -	fse->max_width = CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> -	fse->min_height = CAL_MIN_HEIGHT_LINES;
> -	fse->max_height = CAL_MAX_HEIGHT_LINES;
> +out:
> +	mutex_unlock(&phy->mutex);
>  
> -	return 0;
> +	return r;
>  }
>  
>  static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
> @@ -684,10 +709,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
>  	struct cal_camerarx *phy = to_cal_camerarx(sd);
>  	struct v4l2_mbus_framefmt *fmt;
>  
> +	mutex_lock(&phy->mutex);
> +
>  	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
>  					  format->which);
>  	format->format = *fmt;
>  
> +	mutex_unlock(&phy->mutex);
> +
>  	return 0;
>  }
>  
> @@ -725,6 +754,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  	format->format.field = V4L2_FIELD_NONE;
>  
>  	/* Store the format and propagate it to the source pad. */
> +	mutex_lock(&phy->mutex);
> +
>  	fmt = cal_camerarx_get_pad_format(phy, sd_state,
>  					  CAL_CAMERARX_PAD_SINK,
>  					  format->which);
> @@ -735,6 +766,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  					  format->which);
>  	*fmt = format->format;
>  
> +	mutex_unlock(&phy->mutex);
> +
>  	return 0;
>  }
>  
> @@ -801,6 +834,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	phy->cal = cal;
>  	phy->instance = instance;
>  
> +	mutex_init(&phy->mutex);

A mutex_destroy() somewhere would be nice.

> +
>  	phy->res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
>  						(instance == 0) ?
>  						"cal_rx_core0" :
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index ad08c189ad3b..78bd2e041d9a 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -163,6 +163,9 @@ struct cal_camerarx {
>  	struct v4l2_subdev	subdev;
>  	struct media_pad	pads[2];
>  	struct v4l2_mbus_framefmt	formats[2];
> +
> +	/* mutex for camerarx ops */
> +	struct mutex		mutex;

It's best when possible to list the fields protected by a lock, instead
of the functions. It seems to be cal_camerarx.formats (but would need to
be updated in subsequent patches).

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  };
>  
>  struct cal_dev {

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 33/38] media: ti-vpe: cal: add camerarx locking

[Tomi Valkeinen]

On 04/06/2021 17:14, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Mon, May 24, 2021 at 02:09:04PM +0300, Tomi Valkeinen wrote:
>> We don't have any locking in camerarx for the subdev ops. We have
>> managed fine so far without locking, but in the future multiple video
>> capture devices can use the same camerarx, and locking is a must.
>>
>> Add a mutex to protect the camerarx subdev ops. Some of the functions
>> were slightly restructured to make lock handling cleaner.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/platform/ti-vpe/cal-camerarx.c | 81 ++++++++++++++------
>>   drivers/media/platform/ti-vpe/cal.h          |  3 +
>>   2 files changed, 61 insertions(+), 23 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> index 82392499e663..b87ffc52feb6 100644
>> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
>> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> @@ -601,12 +601,18 @@ cal_camerarx_get_pad_format(struct cal_camerarx *phy,
>>   static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
>>   {
>>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
>> +	int r = 0;
> 
> The driver uses ret :-)

...fine... ;)

>> +
>> +	mutex_lock(&phy->mutex);
>>   
>>   	if (enable)
>> -		return cal_camerarx_start(phy);
>> +		r = cal_camerarx_start(phy);
>> +	else
>> +		cal_camerarx_stop(phy);
>>   
>> -	cal_camerarx_stop(phy);
>> -	return 0;
>> +	mutex_unlock(&phy->mutex);
>> +
>> +	return r;
>>   }
>>   
>>   static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
>> @@ -614,27 +620,36 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
>>   					  struct v4l2_subdev_mbus_code_enum *code)
>>   {
>>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
>> +	int r = 0;
>> +
>> +	mutex_lock(&phy->mutex);
>>   
>>   	/* No transcoding, source and sink codes must match. */
>>   	if (code->pad == CAL_CAMERARX_PAD_SOURCE) {
>>   		struct v4l2_mbus_framefmt *fmt;
>>   
>> -		if (code->index > 0)
>> -			return -EINVAL;
>> +		if (code->index > 0) {
>> +			r = -EINVAL;
>> +			goto out;
>> +		}
>>   
>>   		fmt = cal_camerarx_get_pad_format(phy, sd_state,
>>   						  CAL_CAMERARX_PAD_SINK,
>>   						  code->which);
>>   		code->code = fmt->code;
>> -		return 0;
>> -	}
>> +	} else {
>> +		if (code->index >= cal_num_formats) {
>> +			r = -EINVAL;
>> +			goto out;
>> +		}
>>   
>> -	if (code->index >= cal_num_formats)
>> -		return -EINVAL;
>> +		code->code = cal_formats[code->index].code;
>> +	}
>>   
>> -	code->code = cal_formats[code->index].code;
>> +out:
>> +	mutex_unlock(&phy->mutex);
>>   
>> -	return 0;
>> +	return r;
>>   }
>>   
>>   static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>> @@ -643,10 +658,13 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>>   {
>>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
>>   	const struct cal_format_info *fmtinfo;
>> +	int r = 0;
>>   
>>   	if (fse->index > 0)
>>   		return -EINVAL;
>>   
>> +	mutex_lock(&phy->mutex);
>> +
>>   	/* No transcoding, source and sink formats must match. */
>>   	if (fse->pad == CAL_CAMERARX_PAD_SOURCE) {
>>   		struct v4l2_mbus_framefmt *fmt;
>> @@ -654,27 +672,34 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>>   		fmt = cal_camerarx_get_pad_format(phy, sd_state,
>>   						  CAL_CAMERARX_PAD_SINK,
>>   						  fse->which);
>> -		if (fse->code != fmt->code)
>> -			return -EINVAL;
>> +		if (fse->code != fmt->code) {
>> +			r = -EINVAL;
>> +			goto out;
>> +		}
>>   
>>   		fse->min_width = fmt->width;
>>   		fse->max_width = fmt->width;
>>   		fse->min_height = fmt->height;
>>   		fse->max_height = fmt->height;
>> +	} else {
>> +		fmtinfo = cal_format_by_code(fse->code);
>> +		if (!fmtinfo) {
>> +			r = -EINVAL;
>> +			goto out;
>> +		}
>>   
>> -		return 0;
>> +		fse->min_width =
>> +			CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
>> +		fse->max_width =
>> +			CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> 
> This is a case where I'd write
> 
> 		fse->min_width = CAL_MIN_WIDTH_BYTES * 8
> 			       / ALIGN(fmtinfo->bpp, 8);
> 		fse->max_width = CAL_MAX_WIDTH_BYTES * 8
> 			       / ALIGN(fmtinfo->bpp, 8);
> 
> or go slightly over 80 columns.

Yes. clang-format insist on formatting like that, and I haven't figured 
out how to prevent it from moving everything to next line. It does that 
for some function calls too.

And I'm often relying on clang-format, as my editor doesn't like mixed 
spaces and tabs (googling shows that most of Internet hates mixed spaces 
and tabs...).

But wouldn't the operator usually be left in the earlier line, i.e.

fse->min_width = CAL_MIN_WIDTH_BYTES * 8 /
                  ALIGN(fmtinfo->bpp, 8);

? That's how I would split it.

>> +		fse->min_height = CAL_MIN_HEIGHT_LINES;
>> +		fse->max_height = CAL_MAX_HEIGHT_LINES;
>>   	}
>>   
>> -	fmtinfo = cal_format_by_code(fse->code);
>> -	if (!fmtinfo)
>> -		return -EINVAL;
>> -
>> -	fse->min_width = CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
>> -	fse->max_width = CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
>> -	fse->min_height = CAL_MIN_HEIGHT_LINES;
>> -	fse->max_height = CAL_MAX_HEIGHT_LINES;
>> +out:
>> +	mutex_unlock(&phy->mutex);
>>   
>> -	return 0;
>> +	return r;
>>   }
>>   
>>   static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
>> @@ -684,10 +709,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
>>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
>>   	struct v4l2_mbus_framefmt *fmt;
>>   
>> +	mutex_lock(&phy->mutex);
>> +
>>   	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
>>   					  format->which);
>>   	format->format = *fmt;
>>   
>> +	mutex_unlock(&phy->mutex);
>> +
>>   	return 0;
>>   }
>>   
>> @@ -725,6 +754,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>>   	format->format.field = V4L2_FIELD_NONE;
>>   
>>   	/* Store the format and propagate it to the source pad. */
>> +	mutex_lock(&phy->mutex);
>> +
>>   	fmt = cal_camerarx_get_pad_format(phy, sd_state,
>>   					  CAL_CAMERARX_PAD_SINK,
>>   					  format->which);
>> @@ -735,6 +766,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>>   					  format->which);
>>   	*fmt = format->format;
>>   
>> +	mutex_unlock(&phy->mutex);
>> +
>>   	return 0;
>>   }
>>   
>> @@ -801,6 +834,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>>   	phy->cal = cal;
>>   	phy->instance = instance;
>>   
>> +	mutex_init(&phy->mutex);
> 
> A mutex_destroy() somewhere would be nice.

Right.

>> +
>>   	phy->res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
>>   						(instance == 0) ?
>>   						"cal_rx_core0" :
>> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
>> index ad08c189ad3b..78bd2e041d9a 100644
>> --- a/drivers/media/platform/ti-vpe/cal.h
>> +++ b/drivers/media/platform/ti-vpe/cal.h
>> @@ -163,6 +163,9 @@ struct cal_camerarx {
>>   	struct v4l2_subdev	subdev;
>>   	struct media_pad	pads[2];
>>   	struct v4l2_mbus_framefmt	formats[2];
>> +
>> +	/* mutex for camerarx ops */
>> +	struct mutex		mutex;
> 
> It's best when possible to list the fields protected by a lock, instead
> of the functions. It seems to be cal_camerarx.formats (but would need to
> be updated in subsequent patches).

I've added this.

  Tomi

Re: [PATCH v3 33/38] media: ti-vpe: cal: add camerarx locking

[Laurent Pinchart]

Hi Tomi,

On Mon, Jun 07, 2021 at 02:55:04PM +0300, Tomi Valkeinen wrote:
> On 04/06/2021 17:14, Laurent Pinchart wrote:
> > On Mon, May 24, 2021 at 02:09:04PM +0300, Tomi Valkeinen wrote:
> >> We don't have any locking in camerarx for the subdev ops. We have
> >> managed fine so far without locking, but in the future multiple video
> >> capture devices can use the same camerarx, and locking is a must.
> >>
> >> Add a mutex to protect the camerarx subdev ops. Some of the functions
> >> were slightly restructured to make lock handling cleaner.
> >>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >>   drivers/media/platform/ti-vpe/cal-camerarx.c | 81 ++++++++++++++------
> >>   drivers/media/platform/ti-vpe/cal.h          |  3 +
> >>   2 files changed, 61 insertions(+), 23 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> >> index 82392499e663..b87ffc52feb6 100644
> >> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> >> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> >> @@ -601,12 +601,18 @@ cal_camerarx_get_pad_format(struct cal_camerarx *phy,
> >>   static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
> >>   {
> >>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
> >> +	int r = 0;
> > 
> > The driver uses ret :-)
> 
> ...fine... ;)
> 
> >> +
> >> +	mutex_lock(&phy->mutex);
> >>   
> >>   	if (enable)
> >> -		return cal_camerarx_start(phy);
> >> +		r = cal_camerarx_start(phy);
> >> +	else
> >> +		cal_camerarx_stop(phy);
> >>   
> >> -	cal_camerarx_stop(phy);
> >> -	return 0;
> >> +	mutex_unlock(&phy->mutex);
> >> +
> >> +	return r;
> >>   }
> >>   
> >>   static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
> >> @@ -614,27 +620,36 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
> >>   					  struct v4l2_subdev_mbus_code_enum *code)
> >>   {
> >>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
> >> +	int r = 0;
> >> +
> >> +	mutex_lock(&phy->mutex);
> >>   
> >>   	/* No transcoding, source and sink codes must match. */
> >>   	if (code->pad == CAL_CAMERARX_PAD_SOURCE) {
> >>   		struct v4l2_mbus_framefmt *fmt;
> >>   
> >> -		if (code->index > 0)
> >> -			return -EINVAL;
> >> +		if (code->index > 0) {
> >> +			r = -EINVAL;
> >> +			goto out;
> >> +		}
> >>   
> >>   		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> >>   						  CAL_CAMERARX_PAD_SINK,
> >>   						  code->which);
> >>   		code->code = fmt->code;
> >> -		return 0;
> >> -	}
> >> +	} else {
> >> +		if (code->index >= cal_num_formats) {
> >> +			r = -EINVAL;
> >> +			goto out;
> >> +		}
> >>   
> >> -	if (code->index >= cal_num_formats)
> >> -		return -EINVAL;
> >> +		code->code = cal_formats[code->index].code;
> >> +	}
> >>   
> >> -	code->code = cal_formats[code->index].code;
> >> +out:
> >> +	mutex_unlock(&phy->mutex);
> >>   
> >> -	return 0;
> >> +	return r;
> >>   }
> >>   
> >>   static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
> >> @@ -643,10 +658,13 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
> >>   {
> >>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
> >>   	const struct cal_format_info *fmtinfo;
> >> +	int r = 0;
> >>   
> >>   	if (fse->index > 0)
> >>   		return -EINVAL;
> >>   
> >> +	mutex_lock(&phy->mutex);
> >> +
> >>   	/* No transcoding, source and sink formats must match. */
> >>   	if (fse->pad == CAL_CAMERARX_PAD_SOURCE) {
> >>   		struct v4l2_mbus_framefmt *fmt;
> >> @@ -654,27 +672,34 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
> >>   		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> >>   						  CAL_CAMERARX_PAD_SINK,
> >>   						  fse->which);
> >> -		if (fse->code != fmt->code)
> >> -			return -EINVAL;
> >> +		if (fse->code != fmt->code) {
> >> +			r = -EINVAL;
> >> +			goto out;
> >> +		}
> >>   
> >>   		fse->min_width = fmt->width;
> >>   		fse->max_width = fmt->width;
> >>   		fse->min_height = fmt->height;
> >>   		fse->max_height = fmt->height;
> >> +	} else {
> >> +		fmtinfo = cal_format_by_code(fse->code);
> >> +		if (!fmtinfo) {
> >> +			r = -EINVAL;
> >> +			goto out;
> >> +		}
> >>   
> >> -		return 0;
> >> +		fse->min_width =
> >> +			CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> >> +		fse->max_width =
> >> +			CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> > 
> > This is a case where I'd write
> > 
> > 		fse->min_width = CAL_MIN_WIDTH_BYTES * 8
> > 			       / ALIGN(fmtinfo->bpp, 8);
> > 		fse->max_width = CAL_MAX_WIDTH_BYTES * 8
> > 			       / ALIGN(fmtinfo->bpp, 8);
> > 
> > or go slightly over 80 columns.
> 
> Yes. clang-format insist on formatting like that, and I haven't figured 
> out how to prevent it from moving everything to next line. It does that 
> for some function calls too.
> 
> And I'm often relying on clang-format, as my editor doesn't like mixed 
> spaces and tabs (googling shows that most of Internet hates mixed spaces 
> and tabs...).
> 
> But wouldn't the operator usually be left in the earlier line, i.e.
> 
> fse->min_width = CAL_MIN_WIDTH_BYTES * 8 /
>                   ALIGN(fmtinfo->bpp, 8);
> 
> ? That's how I would split it.

It's a matter of preference I think. I find operators at the beginning
of the next lien to be more readable, but that's just me.

> >> +		fse->min_height = CAL_MIN_HEIGHT_LINES;
> >> +		fse->max_height = CAL_MAX_HEIGHT_LINES;
> >>   	}
> >>   
> >> -	fmtinfo = cal_format_by_code(fse->code);
> >> -	if (!fmtinfo)
> >> -		return -EINVAL;
> >> -
> >> -	fse->min_width = CAL_MIN_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> >> -	fse->max_width = CAL_MAX_WIDTH_BYTES * 8 / ALIGN(fmtinfo->bpp, 8);
> >> -	fse->min_height = CAL_MIN_HEIGHT_LINES;
> >> -	fse->max_height = CAL_MAX_HEIGHT_LINES;
> >> +out:
> >> +	mutex_unlock(&phy->mutex);
> >>   
> >> -	return 0;
> >> +	return r;
> >>   }
> >>   
> >>   static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
> >> @@ -684,10 +709,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
> >>   	struct cal_camerarx *phy = to_cal_camerarx(sd);
> >>   	struct v4l2_mbus_framefmt *fmt;
> >>   
> >> +	mutex_lock(&phy->mutex);
> >> +
> >>   	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
> >>   					  format->which);
> >>   	format->format = *fmt;
> >>   
> >> +	mutex_unlock(&phy->mutex);
> >> +
> >>   	return 0;
> >>   }
> >>   
> >> @@ -725,6 +754,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
> >>   	format->format.field = V4L2_FIELD_NONE;
> >>   
> >>   	/* Store the format and propagate it to the source pad. */
> >> +	mutex_lock(&phy->mutex);
> >> +
> >>   	fmt = cal_camerarx_get_pad_format(phy, sd_state,
> >>   					  CAL_CAMERARX_PAD_SINK,
> >>   					  format->which);
> >> @@ -735,6 +766,8 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
> >>   					  format->which);
> >>   	*fmt = format->format;
> >>   
> >> +	mutex_unlock(&phy->mutex);
> >> +
> >>   	return 0;
> >>   }
> >>   
> >> @@ -801,6 +834,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
> >>   	phy->cal = cal;
> >>   	phy->instance = instance;
> >>   
> >> +	mutex_init(&phy->mutex);
> > 
> > A mutex_destroy() somewhere would be nice.
> 
> Right.
> 
> >> +
> >>   	phy->res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
> >>   						(instance == 0) ?
> >>   						"cal_rx_core0" :
> >> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> >> index ad08c189ad3b..78bd2e041d9a 100644
> >> --- a/drivers/media/platform/ti-vpe/cal.h
> >> +++ b/drivers/media/platform/ti-vpe/cal.h
> >> @@ -163,6 +163,9 @@ struct cal_camerarx {
> >>   	struct v4l2_subdev	subdev;
> >>   	struct media_pad	pads[2];
> >>   	struct v4l2_mbus_framefmt	formats[2];
> >> +
> >> +	/* mutex for camerarx ops */
> >> +	struct mutex		mutex;
> > 
> > It's best when possible to list the fields protected by a lock, instead
> > of the functions. It seems to be cal_camerarx.formats (but would need to
> > be updated in subsequent patches).
> 
> I've added this.

-- 
Regards,

Laurent Pinchart

[PATCH v3 34/38] media: ti-vpe: cal: add camerarx enable/disable refcounting

[Tomi Valkeinen]

The following patches add multistream support and we will have multiple
video devices using the same camerarx instances. Thus we need
enable/disable refcounting for the camerarx.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 10 ++++++++++
 drivers/media/platform/ti-vpe/cal.h          |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index b87ffc52feb6..803d53753e87 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -285,6 +285,11 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	u32 val;
 	int ret;
 
+	if (phy->enable_count > 0) {
+		phy->enable_count++;
+		return 0;
+	}
+
 	link_freq = cal_camerarx_get_ext_link_freq(phy);
 	if (link_freq < 0)
 		return link_freq;
@@ -409,6 +414,8 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
 	/* Finally, enable the PHY Protocol Interface (PPI). */
 	cal_camerarx_ppi_enable(phy);
 
+	phy->enable_count++;
+
 	return 0;
 }
 
@@ -416,6 +423,9 @@ static void cal_camerarx_stop(struct cal_camerarx *phy)
 {
 	int ret;
 
+	if (--phy->enable_count > 0)
+		return;
+
 	cal_camerarx_ppi_disable(phy);
 
 	cal_camerarx_disable_irqs(phy);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 78bd2e041d9a..8608a2c6c01a 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -166,6 +166,8 @@ struct cal_camerarx {
 
 	/* mutex for camerarx ops */
 	struct mutex		mutex;
+
+	unsigned int enable_count;
 };
 
 struct cal_dev {
-- 
2.25.1

Re: [PATCH v3 34/38] media: ti-vpe: cal: add camerarx enable/disable refcounting

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:05PM +0300, Tomi Valkeinen wrote:
> The following patches add multistream support and we will have multiple
> video devices using the same camerarx instances. Thus we need
> enable/disable refcounting for the camerarx.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 10 ++++++++++
>  drivers/media/platform/ti-vpe/cal.h          |  2 ++
>  2 files changed, 12 insertions(+)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index b87ffc52feb6..803d53753e87 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -285,6 +285,11 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
>  	u32 val;
>  	int ret;
>  
> +	if (phy->enable_count > 0) {
> +		phy->enable_count++;
> +		return 0;
> +	}
> +
>  	link_freq = cal_camerarx_get_ext_link_freq(phy);
>  	if (link_freq < 0)
>  		return link_freq;
> @@ -409,6 +414,8 @@ static int cal_camerarx_start(struct cal_camerarx *phy)
>  	/* Finally, enable the PHY Protocol Interface (PPI). */
>  	cal_camerarx_ppi_enable(phy);
>  
> +	phy->enable_count++;
> +
>  	return 0;
>  }
>  
> @@ -416,6 +423,9 @@ static void cal_camerarx_stop(struct cal_camerarx *phy)
>  {
>  	int ret;
>  
> +	if (--phy->enable_count > 0)
> +		return;
> +
>  	cal_camerarx_ppi_disable(phy);
>  
>  	cal_camerarx_disable_irqs(phy);
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 78bd2e041d9a..8608a2c6c01a 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -166,6 +166,8 @@ struct cal_camerarx {
>  
>  	/* mutex for camerarx ops */
>  	struct mutex		mutex;
> +
> +	unsigned int enable_count;

Can you align this as the other fields ?

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  };
>  
>  struct cal_dev {

-- 
Regards,

Laurent Pinchart

[PATCH v3 35/38] media: ti-vpe: cal: allow more than 1 source pads

[Tomi Valkeinen]

CAL RX has a single sink and a single source pad. To support multiple
streams, we will have multiple source pads (up to 8, one for each
CAL context).

Change the driver to allow creating more source pads and change the code
accordingly to handle multiple source pads. We still keep
CAL_CAMERARX_NUM_SOURCE_PADS as 1, and the behavior is unchanged.

Also rename CAL_CAMERARX_PAD_SOURCE to CAL_CAMERARX_PAD_FIRST_SOURCE to
highlight that it's the first source.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 13 ++++++------
 drivers/media/platform/ti-vpe/cal-video.c    |  9 +++++++--
 drivers/media/platform/ti-vpe/cal.h          | 21 +++++++++++++++++---
 3 files changed, 32 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index 803d53753e87..e3a4c20be1e6 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -635,7 +635,7 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
 	mutex_lock(&phy->mutex);
 
 	/* No transcoding, source and sink codes must match. */
-	if (code->pad == CAL_CAMERARX_PAD_SOURCE) {
+	if (cal_rx_pad_is_source(code->pad)) {
 		struct v4l2_mbus_framefmt *fmt;
 
 		if (code->index > 0) {
@@ -676,7 +676,7 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
 	mutex_lock(&phy->mutex);
 
 	/* No transcoding, source and sink formats must match. */
-	if (fse->pad == CAL_CAMERARX_PAD_SOURCE) {
+	if (cal_rx_pad_is_source(fse->pad)) {
 		struct v4l2_mbus_framefmt *fmt;
 
 		fmt = cal_camerarx_get_pad_format(phy, sd_state,
@@ -740,7 +740,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 	unsigned int bpp;
 
 	/* No transcoding, source and sink formats must match. */
-	if (format->pad == CAL_CAMERARX_PAD_SOURCE)
+	if (cal_rx_pad_is_source(format->pad))
 		return cal_camerarx_sd_get_fmt(sd, sd_state, format);
 
 	/*
@@ -771,8 +771,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 					  format->which);
 	*fmt = format->format;
 
-	fmt = cal_camerarx_get_pad_format(phy, sd_state,
-					  CAL_CAMERARX_PAD_SOURCE,
+	fmt = cal_camerarx_get_pad_format(phy, sd_state, CAL_CAMERARX_PAD_FIRST_SOURCE,
 					  format->which);
 	*fmt = format->format;
 
@@ -836,6 +835,7 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	struct cal_camerarx *phy;
 	struct v4l2_subdev *sd;
 	int ret;
+	unsigned int i;
 
 	phy = kzalloc(sizeof(*phy), GFP_KERNEL);
 	if (!phy)
@@ -877,7 +877,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	sd->dev = cal->dev;
 
 	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
-	phy->pads[CAL_CAMERARX_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
+	for (i = CAL_CAMERARX_PAD_FIRST_SOURCE; i < CAL_CAMERARX_NUM_PADS; ++i)
+		phy->pads[i].flags = MEDIA_PAD_FL_SOURCE;
 	sd->entity.ops = &cal_camerarx_media_ops;
 	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(phy->pads),
 				     phy->pads);
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index efa08a9ccbd5..8ecae7dc2774 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -687,8 +687,13 @@ static void cal_release_buffers(struct cal_ctx *ctx,
 static int cal_video_check_format(struct cal_ctx *ctx)
 {
 	const struct v4l2_mbus_framefmt *format;
+	struct media_pad *remote_pad;
 
-	format = &ctx->phy->formats[CAL_CAMERARX_PAD_SOURCE];
+	remote_pad = media_entity_remote_pad(&ctx->pad);
+	if (!remote_pad)
+		return -ENODEV;
+
+	format = &ctx->phy->formats[remote_pad->index];
 
 	if (ctx->fmtinfo->code != format->code ||
 	    ctx->v_fmt.fmt.pix.height != format->height ||
@@ -941,7 +946,7 @@ int cal_ctx_v4l2_register(struct cal_ctx *ctx)
 	}
 
 	ret = media_create_pad_link(&ctx->phy->subdev.entity,
-				    CAL_CAMERARX_PAD_SOURCE,
+				    CAL_CAMERARX_PAD_FIRST_SOURCE,
 				    &vfd->entity, 0,
 				    MEDIA_LNK_FL_IMMUTABLE |
 				    MEDIA_LNK_FL_ENABLED);
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 8608a2c6c01a..42a3f8004077 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -44,7 +44,22 @@
 #define CAL_MAX_HEIGHT_LINES		16383
 
 #define CAL_CAMERARX_PAD_SINK		0
-#define CAL_CAMERARX_PAD_SOURCE		1
+#define CAL_CAMERARX_PAD_FIRST_SOURCE	1
+#define CAL_CAMERARX_NUM_SOURCE_PADS	1
+#define CAL_CAMERARX_NUM_PADS		(1 + CAL_CAMERARX_NUM_SOURCE_PADS)
+
+static inline bool cal_rx_pad_is_sink(u32 pad)
+{
+	/* Camera RX has 1 sink pad, and N source pads */
+	return pad == 0;
+}
+
+static inline bool cal_rx_pad_is_source(u32 pad)
+{
+	/* Camera RX has 1 sink pad, and N source pads */
+	return pad >= CAL_CAMERARX_PAD_FIRST_SOURCE &&
+	       pad <= CAL_CAMERARX_NUM_SOURCE_PADS;
+}
 
 struct device;
 struct device_node;
@@ -161,8 +176,8 @@ struct cal_camerarx {
 	struct media_pipeline	pipe;
 
 	struct v4l2_subdev	subdev;
-	struct media_pad	pads[2];
-	struct v4l2_mbus_framefmt	formats[2];
+	struct media_pad	pads[CAL_CAMERARX_NUM_PADS];
+	struct v4l2_mbus_framefmt	formats[CAL_CAMERARX_NUM_PADS];
 
 	/* mutex for camerarx ops */
 	struct mutex		mutex;
-- 
2.25.1

Re: [PATCH v3 35/38] media: ti-vpe: cal: allow more than 1 source pads

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:06PM +0300, Tomi Valkeinen wrote:
> CAL RX has a single sink and a single source pad. To support multiple
> streams, we will have multiple source pads (up to 8, one for each
> CAL context).
> 
> Change the driver to allow creating more source pads and change the code
> accordingly to handle multiple source pads. We still keep
> CAL_CAMERARX_NUM_SOURCE_PADS as 1, and the behavior is unchanged.
> 
> Also rename CAL_CAMERARX_PAD_SOURCE to CAL_CAMERARX_PAD_FIRST_SOURCE to
> highlight that it's the first source.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 13 ++++++------
>  drivers/media/platform/ti-vpe/cal-video.c    |  9 +++++++--
>  drivers/media/platform/ti-vpe/cal.h          | 21 +++++++++++++++++---
>  3 files changed, 32 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index 803d53753e87..e3a4c20be1e6 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -635,7 +635,7 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
>  	mutex_lock(&phy->mutex);
>  
>  	/* No transcoding, source and sink codes must match. */
> -	if (code->pad == CAL_CAMERARX_PAD_SOURCE) {
> +	if (cal_rx_pad_is_source(code->pad)) {
>  		struct v4l2_mbus_framefmt *fmt;
>  
>  		if (code->index > 0) {
> @@ -676,7 +676,7 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>  	mutex_lock(&phy->mutex);
>  
>  	/* No transcoding, source and sink formats must match. */
> -	if (fse->pad == CAL_CAMERARX_PAD_SOURCE) {
> +	if (cal_rx_pad_is_source(fse->pad)) {
>  		struct v4l2_mbus_framefmt *fmt;
>  
>  		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> @@ -740,7 +740,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  	unsigned int bpp;
>  
>  	/* No transcoding, source and sink formats must match. */
> -	if (format->pad == CAL_CAMERARX_PAD_SOURCE)
> +	if (cal_rx_pad_is_source(format->pad))
>  		return cal_camerarx_sd_get_fmt(sd, sd_state, format);
>  
>  	/*
> @@ -771,8 +771,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  					  format->which);
>  	*fmt = format->format;
>  
> -	fmt = cal_camerarx_get_pad_format(phy, sd_state,
> -					  CAL_CAMERARX_PAD_SOURCE,
> +	fmt = cal_camerarx_get_pad_format(phy, sd_state, CAL_CAMERARX_PAD_FIRST_SOURCE,

I would have kept the line wrap.

>  					  format->which);
>  	*fmt = format->format;
>  
> @@ -836,6 +835,7 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	struct cal_camerarx *phy;
>  	struct v4l2_subdev *sd;
>  	int ret;
> +	unsigned int i;

Before ret ?

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  
>  	phy = kzalloc(sizeof(*phy), GFP_KERNEL);
>  	if (!phy)
> @@ -877,7 +877,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	sd->dev = cal->dev;
>  
>  	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> -	phy->pads[CAL_CAMERARX_PAD_SOURCE].flags = MEDIA_PAD_FL_SOURCE;
> +	for (i = CAL_CAMERARX_PAD_FIRST_SOURCE; i < CAL_CAMERARX_NUM_PADS; ++i)
> +		phy->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>  	sd->entity.ops = &cal_camerarx_media_ops;
>  	ret = media_entity_pads_init(&sd->entity, ARRAY_SIZE(phy->pads),
>  				     phy->pads);
> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> index efa08a9ccbd5..8ecae7dc2774 100644
> --- a/drivers/media/platform/ti-vpe/cal-video.c
> +++ b/drivers/media/platform/ti-vpe/cal-video.c
> @@ -687,8 +687,13 @@ static void cal_release_buffers(struct cal_ctx *ctx,
>  static int cal_video_check_format(struct cal_ctx *ctx)
>  {
>  	const struct v4l2_mbus_framefmt *format;
> +	struct media_pad *remote_pad;
>  
> -	format = &ctx->phy->formats[CAL_CAMERARX_PAD_SOURCE];
> +	remote_pad = media_entity_remote_pad(&ctx->pad);
> +	if (!remote_pad)
> +		return -ENODEV;
> +
> +	format = &ctx->phy->formats[remote_pad->index];
>  
>  	if (ctx->fmtinfo->code != format->code ||
>  	    ctx->v_fmt.fmt.pix.height != format->height ||
> @@ -941,7 +946,7 @@ int cal_ctx_v4l2_register(struct cal_ctx *ctx)
>  	}
>  
>  	ret = media_create_pad_link(&ctx->phy->subdev.entity,
> -				    CAL_CAMERARX_PAD_SOURCE,
> +				    CAL_CAMERARX_PAD_FIRST_SOURCE,
>  				    &vfd->entity, 0,
>  				    MEDIA_LNK_FL_IMMUTABLE |
>  				    MEDIA_LNK_FL_ENABLED);
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 8608a2c6c01a..42a3f8004077 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -44,7 +44,22 @@
>  #define CAL_MAX_HEIGHT_LINES		16383
>  
>  #define CAL_CAMERARX_PAD_SINK		0
> -#define CAL_CAMERARX_PAD_SOURCE		1
> +#define CAL_CAMERARX_PAD_FIRST_SOURCE	1
> +#define CAL_CAMERARX_NUM_SOURCE_PADS	1
> +#define CAL_CAMERARX_NUM_PADS		(1 + CAL_CAMERARX_NUM_SOURCE_PADS)
> +
> +static inline bool cal_rx_pad_is_sink(u32 pad)
> +{
> +	/* Camera RX has 1 sink pad, and N source pads */
> +	return pad == 0;
> +}
> +
> +static inline bool cal_rx_pad_is_source(u32 pad)
> +{
> +	/* Camera RX has 1 sink pad, and N source pads */
> +	return pad >= CAL_CAMERARX_PAD_FIRST_SOURCE &&
> +	       pad <= CAL_CAMERARX_NUM_SOURCE_PADS;
> +}
>  
>  struct device;
>  struct device_node;
> @@ -161,8 +176,8 @@ struct cal_camerarx {
>  	struct media_pipeline	pipe;
>  
>  	struct v4l2_subdev	subdev;
> -	struct media_pad	pads[2];
> -	struct v4l2_mbus_framefmt	formats[2];
> +	struct media_pad	pads[CAL_CAMERARX_NUM_PADS];
> +	struct v4l2_mbus_framefmt	formats[CAL_CAMERARX_NUM_PADS];
>  
>  	/* mutex for camerarx ops */
>  	struct mutex		mutex;

-- 
Regards,

Laurent Pinchart

[PATCH v3 36/38] media: ti-vpe: cal: add embedded data support

[Tomi Valkeinen]

Add support for capturing embedded data from the sensor. The only
difference with capturing pixel data and embedded data is that we need
to ensure the PIX PROC is disabled for embedded data so that CAL doesn't
repack the data.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal.c | 17 ++++++++++-------
 drivers/media/platform/ti-vpe/cal.h |  1 +
 2 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 62c45add4efe..fcc81024ae18 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -473,14 +473,17 @@ int cal_ctx_prepare(struct cal_ctx *ctx)
 {
 	int ret;
 
-	ret = cal_reserve_pix_proc(ctx->cal);
-	if (ret < 0) {
-		ctx_err(ctx, "Failed to reserve pix proc: %d\n", ret);
-		return ret;
-	}
+	ctx->use_pix_proc = !ctx->fmtinfo->meta;
+
+	if (ctx->use_pix_proc) {
+		ret = cal_reserve_pix_proc(ctx->cal);
+		if (ret < 0) {
+			ctx_err(ctx, "Failed to reserve pix proc: %d\n", ret);
+			return ret;
+		}
 
-	ctx->pix_proc = ret;
-	ctx->use_pix_proc = true;
+		ctx->pix_proc = ret;
+	}
 
 	return 0;
 }
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 42a3f8004077..29b865d1a238 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -88,6 +88,7 @@ struct cal_format_info {
 	u32	code;
 	/* Bits per pixel */
 	u8	bpp;
+	bool	meta;
 };
 
 /* buffer for one video frame */
-- 
2.25.1

[PATCH v3 37/38] media: ti-vpe: cal: use frame desc to get vc and dt

[Tomi Valkeinen]

Use get_frame_desc() to get the frame desc from the connected source,
and use the provided virtual channel and datatype instead of hardcoded
ones.

get_frame_desc() works per stream, but as we don't support multiple
streams yet, we will just always use stream 0.

If the source doesn't support get_frame_desc(), fall back to the
previous method of always capturing virtual channel 0 and any datatype.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 26 +++++++++++
 drivers/media/platform/ti-vpe/cal.c          | 49 +++++++++++++++++++-
 drivers/media/platform/ti-vpe/cal.h          |  3 ++
 3 files changed, 76 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index e3a4c20be1e6..cb6a37f47432 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -583,6 +583,32 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
 	return ret;
 }
 
+int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
+				       struct v4l2_mbus_frame_desc *fd)
+{
+	struct media_pad *pad;
+	int ret;
+
+	if (!phy->source)
+		return -ENODEV;
+
+	pad = media_entity_remote_pad(&phy->pads[CAL_CAMERARX_PAD_SINK]);
+	if (!pad)
+		return -ENODEV;
+
+	ret = v4l2_subdev_call(phy->source, pad, get_frame_desc, pad->index,
+			       fd);
+	if (ret)
+		return ret;
+
+	if (fd->type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
+		dev_err(phy->cal->dev, "Frame desc do not describe CSI-2 link");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 /* ------------------------------------------------------------------
  *	V4L2 Subdev Operations
  * ------------------------------------------------------------------
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index fcc81024ae18..7975bb449acd 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -469,10 +469,56 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
 	return stopped;
 }
 
+static int
+cal_get_remote_frame_desc_entry(struct cal_camerarx *phy, u32 stream,
+				struct v4l2_mbus_frame_desc_entry *entry)
+{
+	struct v4l2_mbus_frame_desc fd;
+	unsigned int i;
+	int ret;
+
+	ret = cal_camerarx_get_remote_frame_desc(phy, &fd);
+	if (ret) {
+		if (ret != -ENOIOCTLCMD)
+			dev_err(phy->cal->dev,
+				"Failed to get remote frame desc: %d\n", ret);
+		return ret;
+	}
+
+	for (i = 0; i < fd.num_entries; i++) {
+		if (stream == fd.entry[i].stream) {
+			*entry = fd.entry[i];
+			return 0;
+		}
+	}
+
+	return -ENODEV;
+}
+
 int cal_ctx_prepare(struct cal_ctx *ctx)
 {
+	struct v4l2_mbus_frame_desc_entry entry;
 	int ret;
 
+	ret = cal_get_remote_frame_desc_entry(ctx->phy, ctx->stream, &entry);
+
+	if (ret == -ENOIOCTLCMD) {
+		ctx->vc = 0;
+		ctx->datatype = CAL_CSI2_CTX_DT_ANY;
+	} else if (!ret) {
+		ctx_dbg(2, ctx, "Framedesc: stream %u, len %u, vc %u, dt %#x\n",
+		       entry.stream,
+		       entry.length,
+		       entry.bus.csi2.vc,
+		       entry.bus.csi2.dt);
+
+		ctx->vc = entry.bus.csi2.vc;
+		ctx->datatype = entry.bus.csi2.dt;
+	} else {
+		ctx_err(ctx, "Failed to get remote frame desc: %d\n", ret);
+		return ret;
+	}
+
 	ctx->use_pix_proc = !ctx->fmtinfo->meta;
 
 	if (ctx->use_pix_proc) {
@@ -925,8 +971,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 	ctx->dma_ctx = inst;
 	ctx->csi2_ctx = inst;
 	ctx->cport = inst;
-	ctx->vc = 0;
-	ctx->datatype = CAL_CSI2_CTX_DT_ANY;
+	ctx->stream = 0;
 
 	ret = cal_ctx_v4l2_init(ctx);
 	if (ret)
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 29b865d1a238..3aea444f8bf8 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -245,6 +245,7 @@ struct cal_ctx {
 	u8			pix_proc;
 	u8			vc;
 	u8			datatype;
+	u32			stream;
 
 	bool			use_pix_proc;
 };
@@ -318,6 +319,8 @@ const struct cal_format_info *cal_format_by_code(u32 code);
 
 void cal_quickdump_regs(struct cal_dev *cal);
 
+int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
+				       struct v4l2_mbus_frame_desc *fd);
 void cal_camerarx_disable(struct cal_camerarx *phy);
 void cal_camerarx_i913_errata(struct cal_camerarx *phy);
 struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
-- 
2.25.1

Re: [PATCH v3 37/38] media: ti-vpe: cal: use frame desc to get vc and dt

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:08PM +0300, Tomi Valkeinen wrote:
> Use get_frame_desc() to get the frame desc from the connected source,
> and use the provided virtual channel and datatype instead of hardcoded
> ones.
> 
> get_frame_desc() works per stream, but as we don't support multiple
> streams yet, we will just always use stream 0.
> 
> If the source doesn't support get_frame_desc(), fall back to the
> previous method of always capturing virtual channel 0 and any datatype.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 26 +++++++++++
>  drivers/media/platform/ti-vpe/cal.c          | 49 +++++++++++++++++++-
>  drivers/media/platform/ti-vpe/cal.h          |  3 ++
>  3 files changed, 76 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index e3a4c20be1e6..cb6a37f47432 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -583,6 +583,32 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
>  	return ret;
>  }
>  
> +int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
> +				       struct v4l2_mbus_frame_desc *fd)

Maybe s/fd/desc/ to avoid the confusion with file descriptor ?

> +{
> +	struct media_pad *pad;
> +	int ret;
> +
> +	if (!phy->source)
> +		return -ENODEV;

Would EPIPE (here and below) be a better error ? It will be returned to
userspace from VIDIOC_STREAMON(), which already uses EPIPE to indicate
that the links are not correctly set up.

> +
> +	pad = media_entity_remote_pad(&phy->pads[CAL_CAMERARX_PAD_SINK]);
> +	if (!pad)
> +		return -ENODEV;
> +
> +	ret = v4l2_subdev_call(phy->source, pad, get_frame_desc, pad->index,
> +			       fd);
> +	if (ret)
> +		return ret;
> +
> +	if (fd->type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
> +		dev_err(phy->cal->dev, "Frame desc do not describe CSI-2 link");

s/do not/does not/ and maybe s/desc/descriptor/

> +		return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
>  /* ------------------------------------------------------------------
>   *	V4L2 Subdev Operations
>   * ------------------------------------------------------------------
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index fcc81024ae18..7975bb449acd 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -469,10 +469,56 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
>  	return stopped;
>  }
>  
> +static int
> +cal_get_remote_frame_desc_entry(struct cal_camerarx *phy, u32 stream,
> +				struct v4l2_mbus_frame_desc_entry *entry)
> +{
> +	struct v4l2_mbus_frame_desc fd;
> +	unsigned int i;
> +	int ret;
> +
> +	ret = cal_camerarx_get_remote_frame_desc(phy, &fd);
> +	if (ret) {
> +		if (ret != -ENOIOCTLCMD)
> +			dev_err(phy->cal->dev,
> +				"Failed to get remote frame desc: %d\n", ret);
> +		return ret;
> +	}
> +
> +	for (i = 0; i < fd.num_entries; i++) {
> +		if (stream == fd.entry[i].stream) {
> +			*entry = fd.entry[i];
> +			return 0;
> +		}
> +	}
> +

A dev_dbg() would be good here, in which case you could drop the
ctx_err() in cal_ctx_prepare() as all error paths will print a message.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +	return -ENODEV;
> +}
> +
>  int cal_ctx_prepare(struct cal_ctx *ctx)
>  {
> +	struct v4l2_mbus_frame_desc_entry entry;
>  	int ret;
>  
> +	ret = cal_get_remote_frame_desc_entry(ctx->phy, ctx->stream, &entry);
> +
> +	if (ret == -ENOIOCTLCMD) {
> +		ctx->vc = 0;
> +		ctx->datatype = CAL_CSI2_CTX_DT_ANY;
> +	} else if (!ret) {
> +		ctx_dbg(2, ctx, "Framedesc: stream %u, len %u, vc %u, dt %#x\n",
> +		       entry.stream,
> +		       entry.length,
> +		       entry.bus.csi2.vc,
> +		       entry.bus.csi2.dt);

You can group multiple variables on the same line.

> +
> +		ctx->vc = entry.bus.csi2.vc;
> +		ctx->datatype = entry.bus.csi2.dt;
> +	} else {
> +		ctx_err(ctx, "Failed to get remote frame desc: %d\n", ret);
> +		return ret;
> +	}
> +
>  	ctx->use_pix_proc = !ctx->fmtinfo->meta;
>  
>  	if (ctx->use_pix_proc) {
> @@ -925,8 +971,7 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
>  	ctx->dma_ctx = inst;
>  	ctx->csi2_ctx = inst;
>  	ctx->cport = inst;
> -	ctx->vc = 0;
> -	ctx->datatype = CAL_CSI2_CTX_DT_ANY;
> +	ctx->stream = 0;
>  
>  	ret = cal_ctx_v4l2_init(ctx);
>  	if (ret)
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 29b865d1a238..3aea444f8bf8 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -245,6 +245,7 @@ struct cal_ctx {
>  	u8			pix_proc;
>  	u8			vc;
>  	u8			datatype;
> +	u32			stream;
>  
>  	bool			use_pix_proc;
>  };
> @@ -318,6 +319,8 @@ const struct cal_format_info *cal_format_by_code(u32 code);
>  
>  void cal_quickdump_regs(struct cal_dev *cal);
>  
> +int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
> +				       struct v4l2_mbus_frame_desc *fd);
>  void cal_camerarx_disable(struct cal_camerarx *phy);
>  void cal_camerarx_i913_errata(struct cal_camerarx *phy);
>  struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 37/38] media: ti-vpe: cal: use frame desc to get vc and dt

[Tomi Valkeinen]

On 04/06/2021 17:25, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Mon, May 24, 2021 at 02:09:08PM +0300, Tomi Valkeinen wrote:
>> Use get_frame_desc() to get the frame desc from the connected source,
>> and use the provided virtual channel and datatype instead of hardcoded
>> ones.
>>
>> get_frame_desc() works per stream, but as we don't support multiple
>> streams yet, we will just always use stream 0.
>>
>> If the source doesn't support get_frame_desc(), fall back to the
>> previous method of always capturing virtual channel 0 and any datatype.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/platform/ti-vpe/cal-camerarx.c | 26 +++++++++++
>>   drivers/media/platform/ti-vpe/cal.c          | 49 +++++++++++++++++++-
>>   drivers/media/platform/ti-vpe/cal.h          |  3 ++
>>   3 files changed, 76 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> index e3a4c20be1e6..cb6a37f47432 100644
>> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
>> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> @@ -583,6 +583,32 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
>>   	return ret;
>>   }
>>   
>> +int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
>> +				       struct v4l2_mbus_frame_desc *fd)
> 
> Maybe s/fd/desc/ to avoid the confusion with file descriptor ?

Good point.

>> +{
>> +	struct media_pad *pad;
>> +	int ret;
>> +
>> +	if (!phy->source)
>> +		return -ENODEV;
> 
> Would EPIPE (here and below) be a better error ? It will be returned to
> userspace from VIDIOC_STREAMON(), which already uses EPIPE to indicate
> that the links are not correctly set up.

Sound fine to me.

>> +
>> +	pad = media_entity_remote_pad(&phy->pads[CAL_CAMERARX_PAD_SINK]);
>> +	if (!pad)
>> +		return -ENODEV;
>> +
>> +	ret = v4l2_subdev_call(phy->source, pad, get_frame_desc, pad->index,
>> +			       fd);
>> +	if (ret)
>> +		return ret;
>> +
>> +	if (fd->type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
>> +		dev_err(phy->cal->dev, "Frame desc do not describe CSI-2 link");
> 
> s/do not/does not/ and maybe s/desc/descriptor/

Yep.

>> +		return -EINVAL;
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>>   /* ------------------------------------------------------------------
>>    *	V4L2 Subdev Operations
>>    * ------------------------------------------------------------------
>> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
>> index fcc81024ae18..7975bb449acd 100644
>> --- a/drivers/media/platform/ti-vpe/cal.c
>> +++ b/drivers/media/platform/ti-vpe/cal.c
>> @@ -469,10 +469,56 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
>>   	return stopped;
>>   }
>>   
>> +static int
>> +cal_get_remote_frame_desc_entry(struct cal_camerarx *phy, u32 stream,
>> +				struct v4l2_mbus_frame_desc_entry *entry)
>> +{
>> +	struct v4l2_mbus_frame_desc fd;
>> +	unsigned int i;
>> +	int ret;
>> +
>> +	ret = cal_camerarx_get_remote_frame_desc(phy, &fd);
>> +	if (ret) {
>> +		if (ret != -ENOIOCTLCMD)
>> +			dev_err(phy->cal->dev,
>> +				"Failed to get remote frame desc: %d\n", ret);
>> +		return ret;
>> +	}
>> +
>> +	for (i = 0; i < fd.num_entries; i++) {
>> +		if (stream == fd.entry[i].stream) {
>> +			*entry = fd.entry[i];
>> +			return 0;
>> +		}
>> +	}
>> +
> 
> A dev_dbg() would be good here, in which case you could drop the
> ctx_err() in cal_ctx_prepare() as all error paths will print a message.

Hmm, yes, or maybe the other way around: remove the dev_err above, and 
let the cal_ctx_prepare handle the printing (as it already does).

But perhaps it's better to add a dev_err (did you mean dev_err, when you 
said dev_dbg?) above before the return -ENODEV to make it clear if the 
error is coming from get_remote_frame_desc or from trying to find the 
stream.

> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
>> +	return -ENODEV;
>> +}
>> +
>>   int cal_ctx_prepare(struct cal_ctx *ctx)
>>   {
>> +	struct v4l2_mbus_frame_desc_entry entry;
>>   	int ret;
>>   
>> +	ret = cal_get_remote_frame_desc_entry(ctx->phy, ctx->stream, &entry);
>> +
>> +	if (ret == -ENOIOCTLCMD) {
>> +		ctx->vc = 0;
>> +		ctx->datatype = CAL_CSI2_CTX_DT_ANY;
>> +	} else if (!ret) {
>> +		ctx_dbg(2, ctx, "Framedesc: stream %u, len %u, vc %u, dt %#x\n",
>> +		       entry.stream,
>> +		       entry.length,
>> +		       entry.bus.csi2.vc,
>> +		       entry.bus.csi2.dt);
> 
> You can group multiple variables on the same line.

Yep.

Thanks!

  Tomi

Re: [PATCH v3 37/38] media: ti-vpe: cal: use frame desc to get vc and dt

[Laurent Pinchart]

Hi Tomi,

On Mon, Jun 07, 2021 at 03:07:42PM +0300, Tomi Valkeinen wrote:
> On 04/06/2021 17:25, Laurent Pinchart wrote:
> > On Mon, May 24, 2021 at 02:09:08PM +0300, Tomi Valkeinen wrote:
> >> Use get_frame_desc() to get the frame desc from the connected source,
> >> and use the provided virtual channel and datatype instead of hardcoded
> >> ones.
> >>
> >> get_frame_desc() works per stream, but as we don't support multiple
> >> streams yet, we will just always use stream 0.
> >>
> >> If the source doesn't support get_frame_desc(), fall back to the
> >> previous method of always capturing virtual channel 0 and any datatype.
> >>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >>   drivers/media/platform/ti-vpe/cal-camerarx.c | 26 +++++++++++
> >>   drivers/media/platform/ti-vpe/cal.c          | 49 +++++++++++++++++++-
> >>   drivers/media/platform/ti-vpe/cal.h          |  3 ++
> >>   3 files changed, 76 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> >> index e3a4c20be1e6..cb6a37f47432 100644
> >> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> >> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> >> @@ -583,6 +583,32 @@ static int cal_camerarx_parse_dt(struct cal_camerarx *phy)
> >>   	return ret;
> >>   }
> >>   
> >> +int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
> >> +				       struct v4l2_mbus_frame_desc *fd)
> > 
> > Maybe s/fd/desc/ to avoid the confusion with file descriptor ?
> 
> Good point.
> 
> >> +{
> >> +	struct media_pad *pad;
> >> +	int ret;
> >> +
> >> +	if (!phy->source)
> >> +		return -ENODEV;
> > 
> > Would EPIPE (here and below) be a better error ? It will be returned to
> > userspace from VIDIOC_STREAMON(), which already uses EPIPE to indicate
> > that the links are not correctly set up.
> 
> Sound fine to me.
> 
> >> +
> >> +	pad = media_entity_remote_pad(&phy->pads[CAL_CAMERARX_PAD_SINK]);
> >> +	if (!pad)
> >> +		return -ENODEV;
> >> +
> >> +	ret = v4l2_subdev_call(phy->source, pad, get_frame_desc, pad->index,
> >> +			       fd);
> >> +	if (ret)
> >> +		return ret;
> >> +
> >> +	if (fd->type != V4L2_MBUS_FRAME_DESC_TYPE_CSI2) {
> >> +		dev_err(phy->cal->dev, "Frame desc do not describe CSI-2 link");
> > 
> > s/do not/does not/ and maybe s/desc/descriptor/
> 
> Yep.
> 
> >> +		return -EINVAL;
> >> +	}
> >> +
> >> +	return 0;
> >> +}
> >> +
> >>   /* ------------------------------------------------------------------
> >>    *	V4L2 Subdev Operations
> >>    * ------------------------------------------------------------------
> >> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> >> index fcc81024ae18..7975bb449acd 100644
> >> --- a/drivers/media/platform/ti-vpe/cal.c
> >> +++ b/drivers/media/platform/ti-vpe/cal.c
> >> @@ -469,10 +469,56 @@ static bool cal_ctx_wr_dma_stopped(struct cal_ctx *ctx)
> >>   	return stopped;
> >>   }
> >>   
> >> +static int
> >> +cal_get_remote_frame_desc_entry(struct cal_camerarx *phy, u32 stream,
> >> +				struct v4l2_mbus_frame_desc_entry *entry)
> >> +{
> >> +	struct v4l2_mbus_frame_desc fd;
> >> +	unsigned int i;
> >> +	int ret;
> >> +
> >> +	ret = cal_camerarx_get_remote_frame_desc(phy, &fd);
> >> +	if (ret) {
> >> +		if (ret != -ENOIOCTLCMD)
> >> +			dev_err(phy->cal->dev,
> >> +				"Failed to get remote frame desc: %d\n", ret);
> >> +		return ret;
> >> +	}
> >> +
> >> +	for (i = 0; i < fd.num_entries; i++) {
> >> +		if (stream == fd.entry[i].stream) {
> >> +			*entry = fd.entry[i];
> >> +			return 0;
> >> +		}
> >> +	}
> >> +
> > 
> > A dev_dbg() would be good here, in which case you could drop the
> > ctx_err() in cal_ctx_prepare() as all error paths will print a message.
> 
> Hmm, yes, or maybe the other way around: remove the dev_err above, and 
> let the cal_ctx_prepare handle the printing (as it already does).
> 
> But perhaps it's better to add a dev_err (did you mean dev_err, when you 
> said dev_dbg?) above before the return -ENODEV to make it clear if the 
> error is coming from get_remote_frame_desc or from trying to find the 
> stream.

I usually prefer debug messages for anything that can be triggered by
userspace and isn't a kernel error. Not sure which one it is here :-)

> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> >> +	return -ENODEV;
> >> +}
> >> +
> >>   int cal_ctx_prepare(struct cal_ctx *ctx)
> >>   {
> >> +	struct v4l2_mbus_frame_desc_entry entry;
> >>   	int ret;
> >>   
> >> +	ret = cal_get_remote_frame_desc_entry(ctx->phy, ctx->stream, &entry);
> >> +
> >> +	if (ret == -ENOIOCTLCMD) {
> >> +		ctx->vc = 0;
> >> +		ctx->datatype = CAL_CSI2_CTX_DT_ANY;
> >> +	} else if (!ret) {
> >> +		ctx_dbg(2, ctx, "Framedesc: stream %u, len %u, vc %u, dt %#x\n",
> >> +		       entry.stream,
> >> +		       entry.length,
> >> +		       entry.bus.csi2.vc,
> >> +		       entry.bus.csi2.dt);
> > 
> > You can group multiple variables on the same line.
> 
> Yep.

-- 
Regards,

Laurent Pinchart

[PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Tomi Valkeinen]

Add routing and stream_config support to CAL driver.

Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
separate streams at the same time.

Add 8 video device nodes, each representing a single dma-engine, and set
the number of source pads on camerarx to 8. Each video node can be
connected to any of the source pads on either of the camerarx instances
using media links. Camerarx internal routing is used to route the
incoming CSI-2 streams to one of the 8 source pads.

CAL doesn't support transcoding, so the driver currently allows changes
only on the camerarx sink side, and then copies the sink pad config to
the source pad. This becomes slighly more complex with 8 source pads and
multiple streams on the sink pad. A helper,
cal_camerarx_get_opposite_stream_format(), is added, which uses the
routing table to get the format from the "opposite" side.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
 drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
 drivers/media/platform/ti-vpe/cal.c          |  34 ++-
 drivers/media/platform/ti-vpe/cal.h          |  12 +-
 4 files changed, 385 insertions(+), 67 deletions(-)

diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
index cb6a37f47432..d09b06780b15 100644
--- a/drivers/media/platform/ti-vpe/cal-camerarx.c
+++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
@@ -49,15 +49,33 @@ static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
 {
 	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
 	u32 num_lanes = mipi_csi2->num_data_lanes;
-	const struct cal_format_info *fmtinfo;
 	u32 bpp;
 	s64 freq;
 
-	fmtinfo = cal_format_by_code(phy->formats[CAL_CAMERARX_PAD_SINK].code);
-	if (!fmtinfo)
+	/*
+	 * With multistream input we don't have bpp, and cannot use
+	 * V4L2_CID_PIXEL_RATE. Passing 0 as bpp causes v4l2_get_link_freq()
+	 * to return an error if it falls back to V4L2_CID_PIXEL_RATE.
+	 */
+
+	if (phy->stream_configs.num_configs == 0)
 		return -EINVAL;
 
-	bpp = fmtinfo->bpp;
+	if (phy->stream_configs.num_configs > 2) {
+		bpp = 0;
+	} else {
+		const struct cal_format_info *fmtinfo;
+		struct v4l2_mbus_framefmt *fmt;
+
+		/* The first format is for the sink */
+		fmt = &phy->stream_configs.configs[0].fmt;
+
+		fmtinfo = cal_format_by_code(fmt->code);
+		if (!fmtinfo)
+			return -EINVAL;
+
+		bpp = fmtinfo->bpp;
+	}
 
 	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
 	if (freq < 0) {
@@ -619,19 +637,104 @@ static inline struct cal_camerarx *to_cal_camerarx(struct v4l2_subdev *sd)
 	return container_of(sd, struct cal_camerarx, subdev);
 }
 
-static struct v4l2_mbus_framefmt *
-cal_camerarx_get_pad_format(struct cal_camerarx *phy,
-			    struct v4l2_subdev_state *sd_state,
-			    unsigned int pad, u32 which)
-{
-	switch (which) {
-	case V4L2_SUBDEV_FORMAT_TRY:
-		return v4l2_subdev_get_try_format(&phy->subdev, sd_state, pad);
-	case V4L2_SUBDEV_FORMAT_ACTIVE:
-		return &phy->formats[pad];
-	default:
+struct cal_camerarx *
+cal_camerarx_get_phy_from_entity(struct media_entity *entity)
+{
+	struct v4l2_subdev *sd;
+
+	sd = media_entity_to_v4l2_subdev(entity);
+	if (!sd)
 		return NULL;
+
+	return to_cal_camerarx(sd);
+}
+
+static struct v4l2_subdev_krouting *
+cal_camerarx_get_routing_table(struct cal_camerarx *phy,
+			       struct v4l2_subdev_state *sd_state, u32 which)
+{
+	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
+		return &phy->routing;
+	else
+		return &sd_state->routing;
+}
+
+static struct v4l2_subdev_stream_configs *
+cal_camerarx_get_stream_configs(struct cal_camerarx *phy,
+				struct v4l2_subdev_state *sd_state, u32 which)
+{
+	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
+		return &phy->stream_configs;
+	else
+		return &sd_state->stream_configs;
+}
+
+struct v4l2_mbus_framefmt *
+cal_camerarx_get_stream_format(struct cal_camerarx *phy,
+			       struct v4l2_subdev_state *sd_state,
+			       unsigned int pad, u32 stream, u32 which)
+{
+	struct v4l2_subdev_stream_configs *stream_configs;
+	unsigned int i;
+
+	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
+
+	for (i = 0; i < stream_configs->num_configs; ++i) {
+		if (stream_configs->configs[i].pad == pad &&
+		    stream_configs->configs[i].stream == stream)
+			return &stream_configs->configs[i].fmt;
+	}
+
+	return NULL;
+}
+
+static int cal_camerarx_find_opposite_end(struct v4l2_subdev_krouting *routing,
+					  u32 pad, u32 stream, u32 *other_pad,
+					  u32 *other_stream)
+{
+	unsigned int i;
+
+	for (i = 0; i < routing->num_routes; ++i) {
+		struct v4l2_subdev_route *route = &routing->routes[i];
+
+		if (cal_rx_pad_is_source(pad)) {
+			if (route->source_pad == pad &&
+			    route->source_stream == stream) {
+				*other_pad = route->sink_pad;
+				*other_stream = route->sink_stream;
+				return 0;
+			}
+		} else {
+			if (route->sink_pad == pad &&
+			    route->sink_stream == stream) {
+				*other_pad = route->source_pad;
+				*other_stream = route->source_stream;
+				return 0;
+			}
+		}
 	}
+
+	return -EINVAL;
+}
+
+static struct v4l2_mbus_framefmt *
+cal_camerarx_get_opposite_stream_format(struct cal_camerarx *phy,
+					struct v4l2_subdev_state *sd_state,
+					u32 pad, u32 stream, u32 which)
+{
+	struct v4l2_subdev_krouting *routing;
+	u32 other_pad, other_stream;
+	int ret;
+
+	routing = cal_camerarx_get_routing_table(phy, sd_state, which);
+
+	ret = cal_camerarx_find_opposite_end(routing, pad, stream, &other_pad,
+					     &other_stream);
+	if (ret)
+		return NULL;
+
+	return cal_camerarx_get_stream_format(phy, sd_state, other_pad,
+					      other_stream, which);
 }
 
 static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
@@ -669,9 +772,15 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
 			goto out;
 		}
 
-		fmt = cal_camerarx_get_pad_format(phy, sd_state,
-						  CAL_CAMERARX_PAD_SINK,
-						  code->which);
+		fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state,
+					code->pad, code->stream,
+					code->which);
+
+		if (!fmt) {
+			r = -EINVAL;
+			goto out;
+		}
+
 		code->code = fmt->code;
 	} else {
 		if (code->index >= cal_num_formats) {
@@ -705,9 +814,14 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
 	if (cal_rx_pad_is_source(fse->pad)) {
 		struct v4l2_mbus_framefmt *fmt;
 
-		fmt = cal_camerarx_get_pad_format(phy, sd_state,
-						  CAL_CAMERARX_PAD_SINK,
-						  fse->which);
+		fmt = cal_camerarx_get_opposite_stream_format(
+			phy, sd_state, fse->pad, fse->stream, fse->which);
+
+		if (!fmt) {
+			r = -EINVAL;
+			goto out;
+		}
+
 		if (fse->code != fmt->code) {
 			r = -EINVAL;
 			goto out;
@@ -747,8 +861,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
 
 	mutex_lock(&phy->mutex);
 
-	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
-					  format->which);
+	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
+					     format->stream, format->which);
+
+	if (!fmt) {
+		mutex_unlock(&phy->mutex);
+		return -EINVAL;
+	}
+
 	format->format = *fmt;
 
 	mutex_unlock(&phy->mutex);
@@ -764,6 +884,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 	const struct cal_format_info *fmtinfo;
 	struct v4l2_mbus_framefmt *fmt;
 	unsigned int bpp;
+	int ret = 0;
 
 	/* No transcoding, source and sink formats must match. */
 	if (cal_rx_pad_is_source(format->pad))
@@ -792,40 +913,117 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
 	/* Store the format and propagate it to the source pad. */
 	mutex_lock(&phy->mutex);
 
-	fmt = cal_camerarx_get_pad_format(phy, sd_state,
-					  CAL_CAMERARX_PAD_SINK,
-					  format->which);
+	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
+					     format->stream, format->which);
+	if (!fmt) {
+		ret = -EINVAL;
+		goto out;
+	}
+
 	*fmt = format->format;
 
-	fmt = cal_camerarx_get_pad_format(phy, sd_state, CAL_CAMERARX_PAD_FIRST_SOURCE,
-					  format->which);
+	fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state, format->pad,
+						      format->stream,
+						      format->which);
+	if (!fmt) {
+		ret = -EINVAL;
+		goto out;
+	}
+
 	*fmt = format->format;
 
+out:
 	mutex_unlock(&phy->mutex);
 
+	return ret;
+}
+
+static int cal_camerarx_sd_get_routing(struct v4l2_subdev *sd,
+				       struct v4l2_subdev_state *sd_state,
+				       struct v4l2_subdev_krouting *routing)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+	struct v4l2_subdev_krouting *src;
+
+	src = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
+
+	return v4l2_subdev_cpy_routing(routing, src);
+}
+
+static void cal_camerarx_init_formats(struct v4l2_subdev *sd,
+				      struct v4l2_subdev_state *sd_state,
+				      u32 which)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+
+	static const struct v4l2_mbus_framefmt format = {
+		.width = 640,
+		.height = 480,
+		.code = MEDIA_BUS_FMT_UYVY8_2X8,
+		.field = V4L2_FIELD_NONE,
+		.colorspace = V4L2_COLORSPACE_SRGB,
+		.ycbcr_enc = V4L2_YCBCR_ENC_601,
+		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
+		.xfer_func = V4L2_XFER_FUNC_SRGB,
+	};
+
+	struct v4l2_subdev_stream_configs *stream_configs;
+	unsigned int i;
+
+	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
+
+	for (i = 0; i < stream_configs->num_configs; ++i)
+		stream_configs->configs[i].fmt = format;
+}
+
+static int cal_camerarx_sd_set_routing(struct v4l2_subdev *sd,
+				       struct v4l2_subdev_state *sd_state,
+				       struct v4l2_subdev_krouting *routing)
+{
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+	int ret;
+	struct v4l2_subdev_krouting *dst;
+	struct v4l2_subdev_stream_configs *stream_configs;
+
+	dst = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
+	stream_configs =
+		cal_camerarx_get_stream_configs(phy, sd_state, routing->which);
+
+	ret = v4l2_subdev_dup_routing(dst, routing);
+	if (ret)
+		return ret;
+
+	ret = v4l2_init_stream_configs(stream_configs, dst);
+	if (ret)
+		return ret;
+
+	/* Initialize stream formats */
+	cal_camerarx_init_formats(sd, sd_state, routing->which);
+
 	return 0;
 }
 
 static int cal_camerarx_sd_init_cfg(struct v4l2_subdev *sd,
 				    struct v4l2_subdev_state *sd_state)
 {
-	struct v4l2_subdev_format format = {
-		.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY
-		: V4L2_SUBDEV_FORMAT_ACTIVE,
-		.pad = CAL_CAMERARX_PAD_SINK,
-		.format = {
-			.width = 640,
-			.height = 480,
-			.code = MEDIA_BUS_FMT_UYVY8_2X8,
-			.field = V4L2_FIELD_NONE,
-			.colorspace = V4L2_COLORSPACE_SRGB,
-			.ycbcr_enc = V4L2_YCBCR_ENC_601,
-			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
-			.xfer_func = V4L2_XFER_FUNC_SRGB,
-		},
+	u32 which = sd_state ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE;
+
+	struct v4l2_subdev_route routes[] = { {
+		.sink_pad = 0,
+		.sink_stream = 0,
+		.source_pad = 1,
+		.source_stream = 0,
+		.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
+	} };
+
+	struct v4l2_subdev_krouting routing = {
+		.which = which,
+		.num_routes = 1,
+		.routes = routes,
 	};
 
-	return cal_camerarx_sd_set_fmt(sd, sd_state, &format);
+	/* Initialize routing to single route to the fist source pad */
+	return cal_camerarx_sd_set_routing(sd, sd_state, &routing);
 }
 
 static const struct v4l2_subdev_video_ops cal_camerarx_video_ops = {
@@ -838,6 +1036,8 @@ static const struct v4l2_subdev_pad_ops cal_camerarx_pad_ops = {
 	.enum_frame_size = cal_camerarx_sd_enum_frame_size,
 	.get_fmt = cal_camerarx_sd_get_fmt,
 	.set_fmt = cal_camerarx_sd_set_fmt,
+	.get_routing = cal_camerarx_sd_get_routing,
+	.set_routing = cal_camerarx_sd_set_routing,
 };
 
 static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
@@ -845,8 +1045,18 @@ static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
 	.pad = &cal_camerarx_pad_ops,
 };
 
+static bool cal_camerarx_has_route(struct media_entity *entity, unsigned int pad0,
+			  unsigned int pad1)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+	struct cal_camerarx *phy = to_cal_camerarx(sd);
+
+	return v4l2_subdev_has_route(&phy->routing, pad0, pad1);
+}
+
 static struct media_entity_operations cal_camerarx_media_ops = {
 	.link_validate = v4l2_subdev_link_validate,
+	.has_route = cal_camerarx_has_route,
 };
 
 /* ------------------------------------------------------------------
@@ -898,11 +1108,12 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	sd = &phy->subdev;
 	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
 	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
-	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_MULTIPLEXED;
 	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
 	sd->dev = cal->dev;
 
 	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
+
 	for (i = CAL_CAMERARX_PAD_FIRST_SOURCE; i < CAL_CAMERARX_NUM_PADS; ++i)
 		phy->pads[i].flags = MEDIA_PAD_FL_SOURCE;
 	sd->entity.ops = &cal_camerarx_media_ops;
@@ -922,6 +1133,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
 	return phy;
 
 error:
+	v4l2_subdev_free_routing(&phy->routing);
+	v4l2_uninit_stream_configs(&phy->stream_configs);
 	media_entity_cleanup(&phy->subdev.entity);
 	kfree(phy);
 	return ERR_PTR(ret);
@@ -933,6 +1146,8 @@ void cal_camerarx_destroy(struct cal_camerarx *phy)
 		return;
 
 	v4l2_device_unregister_subdev(&phy->subdev);
+	v4l2_subdev_free_routing(&phy->routing);
+	v4l2_uninit_stream_configs(&phy->stream_configs);
 	media_entity_cleanup(&phy->subdev.entity);
 	of_node_put(phy->source_ep_node);
 	of_node_put(phy->source_node);
diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
index 8ecae7dc2774..234af40a24fa 100644
--- a/drivers/media/platform/ti-vpe/cal-video.c
+++ b/drivers/media/platform/ti-vpe/cal-video.c
@@ -693,7 +693,11 @@ static int cal_video_check_format(struct cal_ctx *ctx)
 	if (!remote_pad)
 		return -ENODEV;
 
-	format = &ctx->phy->formats[remote_pad->index];
+	format = cal_camerarx_get_stream_format(ctx->phy, NULL,
+						remote_pad->index, 0,
+						V4L2_SUBDEV_FORMAT_ACTIVE);
+	if (!format)
+		return -EINVAL;
 
 	if (ctx->fmtinfo->code != format->code ||
 	    ctx->v_fmt.fmt.pix.height != format->height ||
@@ -711,6 +715,48 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
 	dma_addr_t addr;
 	int ret;
 
+	if (cal_mc_api) {
+		struct v4l2_subdev_route *route = NULL;
+		struct media_pad *remote_pad;
+		unsigned int i;
+
+		/* Find the PHY connected to this video device */
+
+		remote_pad = media_entity_remote_pad(&ctx->pad);
+		if (!remote_pad) {
+			ctx_err(ctx, "Context not connected\n");
+			ret = -ENODEV;
+			goto error_release_buffers;
+		}
+
+		ctx->phy = cal_camerarx_get_phy_from_entity(remote_pad->entity);
+
+		/* Find the stream */
+
+		for (i = 0; i < ctx->phy->routing.num_routes; ++i) {
+			struct v4l2_subdev_route *r =
+				&ctx->phy->routing.routes[i];
+
+			if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
+				continue;
+
+			if (r->source_pad != remote_pad->index)
+				continue;
+
+			route = r;
+
+			break;
+		}
+
+		if (!route) {
+			ctx_err(ctx, "Failed to find route\n");
+			ret = -ENODEV;
+			goto error_release_buffers;
+		}
+
+		ctx->stream = route->sink_stream;
+	}
+
 	ret = media_pipeline_start(ctx->vdev.entity.pads, &ctx->phy->pipe);
 	if (ret < 0) {
 		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
@@ -784,6 +830,9 @@ static void cal_stop_streaming(struct vb2_queue *vq)
 	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
 
 	media_pipeline_stop(ctx->vdev.entity.pads);
+
+	if (cal_mc_api)
+		ctx->phy = NULL;
 }
 
 static const struct vb2_ops cal_video_qops = {
@@ -945,16 +994,48 @@ int cal_ctx_v4l2_register(struct cal_ctx *ctx)
 		return ret;
 	}
 
-	ret = media_create_pad_link(&ctx->phy->subdev.entity,
-				    CAL_CAMERARX_PAD_FIRST_SOURCE,
-				    &vfd->entity, 0,
-				    MEDIA_LNK_FL_IMMUTABLE |
-				    MEDIA_LNK_FL_ENABLED);
-	if (ret) {
-		ctx_err(ctx, "Failed to create media link for context %u\n",
-			ctx->dma_ctx);
-		video_unregister_device(vfd);
-		return ret;
+	if (cal_mc_api) {
+		u16 phy_idx;
+		u16 pad_idx;
+
+		/* Create links from all video nodes to all PHYs */
+
+		for (phy_idx = 0; phy_idx < ctx->cal->data->num_csi2_phy; ++phy_idx) {
+			for (pad_idx = 1; pad_idx < CAL_CAMERARX_NUM_PADS; ++pad_idx) {
+				/*
+				 * Enable only links from video0 to PHY0 pad 1, and
+				 * video1 to PHY1 pad 1.
+				 */
+				bool enable = (ctx->dma_ctx == 0 &&
+					       phy_idx == 0 && pad_idx == 1) ||
+					      (ctx->dma_ctx == 1 &&
+					       phy_idx == 1 && pad_idx == 1);
+
+				ret = media_create_pad_link(
+					&ctx->cal->phy[phy_idx]->subdev.entity,
+					pad_idx, &vfd->entity, 0,
+					enable ? MEDIA_LNK_FL_ENABLED : 0);
+				if (ret) {
+					ctx_err(ctx,
+						"Failed to create media link for context %u\n",
+						ctx->dma_ctx);
+					video_unregister_device(vfd);
+					return ret;
+				}
+			}
+		}
+	} else {
+		ret = media_create_pad_link(
+			&ctx->phy->subdev.entity, CAL_CAMERARX_PAD_FIRST_SOURCE,
+			&vfd->entity, 0,
+			MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
+		if (ret) {
+			ctx_err(ctx,
+				"Failed to create media link for context %u\n",
+				ctx->dma_ctx);
+			video_unregister_device(vfd);
+			return ret;
+		}
 	}
 
 	ctx_info(ctx, "V4L2 device registered as %s\n",
diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
index 7975bb449acd..5fbb7a90c795 100644
--- a/drivers/media/platform/ti-vpe/cal.c
+++ b/drivers/media/platform/ti-vpe/cal.c
@@ -967,7 +967,6 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
 		return NULL;
 
 	ctx->cal = cal;
-	ctx->phy = cal->phy[inst];
 	ctx->dma_ctx = inst;
 	ctx->csi2_ctx = inst;
 	ctx->cport = inst;
@@ -1178,18 +1177,33 @@ static int cal_probe(struct platform_device *pdev)
 	}
 
 	/* Create contexts. */
-	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
-		if (!cal->phy[i]->source_node)
-			continue;
+	if (!cal_mc_api) {
+		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
+			if (!cal->phy[i]->source_node)
+				continue;
+
+			cal->ctx[i] = cal_ctx_create(cal, i);
+			if (!cal->ctx[i]) {
+				cal_err(cal, "Failed to create context %u\n", i);
+				ret = -ENODEV;
+				goto error_context;
+			}
 
-		cal->ctx[i] = cal_ctx_create(cal, i);
-		if (!cal->ctx[i]) {
-			cal_err(cal, "Failed to create context %u\n", i);
-			ret = -ENODEV;
-			goto error_context;
+			cal->ctx[i]->phy = cal->phy[i];
+
+			cal->num_contexts++;
 		}
+	} else {
+		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
+			cal->ctx[i] = cal_ctx_create(cal, i);
+			if (!cal->ctx[i]) {
+				cal_err(cal, "Failed to create context %u\n", i);
+				ret = -ENODEV;
+				goto error_context;
+			}
 
-		cal->num_contexts++;
+			cal->num_contexts++;
+		}
 	}
 
 	/* Register the media device. */
diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
index 3aea444f8bf8..6626c2a59fc2 100644
--- a/drivers/media/platform/ti-vpe/cal.h
+++ b/drivers/media/platform/ti-vpe/cal.h
@@ -45,7 +45,7 @@
 
 #define CAL_CAMERARX_PAD_SINK		0
 #define CAL_CAMERARX_PAD_FIRST_SOURCE	1
-#define CAL_CAMERARX_NUM_SOURCE_PADS	1
+#define CAL_CAMERARX_NUM_SOURCE_PADS	8
 #define CAL_CAMERARX_NUM_PADS		(1 + CAL_CAMERARX_NUM_SOURCE_PADS)
 
 static inline bool cal_rx_pad_is_sink(u32 pad)
@@ -178,12 +178,14 @@ struct cal_camerarx {
 
 	struct v4l2_subdev	subdev;
 	struct media_pad	pads[CAL_CAMERARX_NUM_PADS];
-	struct v4l2_mbus_framefmt	formats[CAL_CAMERARX_NUM_PADS];
 
 	/* mutex for camerarx ops */
 	struct mutex		mutex;
 
 	unsigned int enable_count;
+
+	struct v4l2_subdev_krouting routing;
+	struct v4l2_subdev_stream_configs stream_configs;
 };
 
 struct cal_dev {
@@ -321,6 +323,7 @@ void cal_quickdump_regs(struct cal_dev *cal);
 
 int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
 				       struct v4l2_mbus_frame_desc *fd);
+struct cal_camerarx *cal_camerarx_get_phy_from_entity(struct media_entity *entity);
 void cal_camerarx_disable(struct cal_camerarx *phy);
 void cal_camerarx_i913_errata(struct cal_camerarx *phy);
 struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
@@ -338,4 +341,9 @@ void cal_ctx_v4l2_unregister(struct cal_ctx *ctx);
 int cal_ctx_v4l2_init(struct cal_ctx *ctx);
 void cal_ctx_v4l2_cleanup(struct cal_ctx *ctx);
 
+struct v4l2_mbus_framefmt *
+cal_camerarx_get_stream_format(struct cal_camerarx *phy,
+			       struct v4l2_subdev_state *state,
+			       unsigned int pad, u32 stream, u32 which);
+
 #endif /* __TI_CAL_H__ */
-- 
2.25.1

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Pratyush Yadav]

Hi Tomi,

On 24/05/21 02:09PM, Tomi Valkeinen wrote:
> Add routing and stream_config support to CAL driver.
> 
> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
> separate streams at the same time.
> 
> Add 8 video device nodes, each representing a single dma-engine, and set
> the number of source pads on camerarx to 8. Each video node can be
> connected to any of the source pads on either of the camerarx instances
> using media links. Camerarx internal routing is used to route the
> incoming CSI-2 streams to one of the 8 source pads.
> 
> CAL doesn't support transcoding, so the driver currently allows changes
> only on the camerarx sink side, and then copies the sink pad config to
> the source pad. This becomes slighly more complex with 8 source pads and
> multiple streams on the sink pad. A helper,
> cal_camerarx_get_opposite_stream_format(), is added, which uses the
> routing table to get the format from the "opposite" side.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
>  drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
>  drivers/media/platform/ti-vpe/cal.c          |  34 ++-
>  drivers/media/platform/ti-vpe/cal.h          |  12 +-
>  4 files changed, 385 insertions(+), 67 deletions(-)
> 
[...]
> @@ -1178,18 +1177,33 @@ static int cal_probe(struct platform_device *pdev)
>  	}
>  
>  	/* Create contexts. */
> -	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> -		if (!cal->phy[i]->source_node)
> -			continue;
> +	if (!cal_mc_api) {
> +		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> +			if (!cal->phy[i]->source_node)
> +				continue;
> +
> +			cal->ctx[i] = cal_ctx_create(cal, i);
> +			if (!cal->ctx[i]) {
> +				cal_err(cal, "Failed to create context %u\n", i);
> +				ret = -ENODEV;
> +				goto error_context;
> +			}
>  
> -		cal->ctx[i] = cal_ctx_create(cal, i);
> -		if (!cal->ctx[i]) {
> -			cal_err(cal, "Failed to create context %u\n", i);
> -			ret = -ENODEV;
> -			goto error_context;
> +			cal->ctx[i]->phy = cal->phy[i];
> +
> +			cal->num_contexts++;
>  		}
> +	} else {
> +		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> +			cal->ctx[i] = cal_ctx_create(cal, i);
> +			if (!cal->ctx[i]) {
> +				cal_err(cal, "Failed to create context %u\n", i);
> +				ret = -ENODEV;
> +				goto error_context;
> +			}
>  
> -		cal->num_contexts++;
> +			cal->num_contexts++;

In cal_async_notifier_complete() I see:

  for (i = 0; i < cal->num_contexts; i++)
    ret = cal_ctx_v4l2_register();

This means that if the CAL device has 8 DMA contexts it will create 8 
/dev/videoX nodes, even if the hardware setup is only capable of 1 
stream.

Would it make more sense to populate /dev/videoX nodes based on the 
configured routing? So for example, if only one pad is being used to 
output, only create one node corresponding to that pad. If there are 3 
pads being populated then create 3 nodes and so on.

> +		}
>  	}
>  
>  	/* Register the media device. */

-- 
Regards,
Pratyush Yadav
Texas Instruments Inc.

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Tomi Valkeinen]

On 27/05/2021 19:06, Pratyush Yadav wrote:
> Hi Tomi,
> 
> On 24/05/21 02:09PM, Tomi Valkeinen wrote:
>> Add routing and stream_config support to CAL driver.
>>
>> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
>> separate streams at the same time.
>>
>> Add 8 video device nodes, each representing a single dma-engine, and set
>> the number of source pads on camerarx to 8. Each video node can be
>> connected to any of the source pads on either of the camerarx instances
>> using media links. Camerarx internal routing is used to route the
>> incoming CSI-2 streams to one of the 8 source pads.
>>
>> CAL doesn't support transcoding, so the driver currently allows changes
>> only on the camerarx sink side, and then copies the sink pad config to
>> the source pad. This becomes slighly more complex with 8 source pads and
>> multiple streams on the sink pad. A helper,
>> cal_camerarx_get_opposite_stream_format(), is added, which uses the
>> routing table to get the format from the "opposite" side.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
>>   drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
>>   drivers/media/platform/ti-vpe/cal.c          |  34 ++-
>>   drivers/media/platform/ti-vpe/cal.h          |  12 +-
>>   4 files changed, 385 insertions(+), 67 deletions(-)
>>
> [...]
>> @@ -1178,18 +1177,33 @@ static int cal_probe(struct platform_device *pdev)
>>   	}
>>   
>>   	/* Create contexts. */
>> -	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
>> -		if (!cal->phy[i]->source_node)
>> -			continue;
>> +	if (!cal_mc_api) {
>> +		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
>> +			if (!cal->phy[i]->source_node)
>> +				continue;
>> +
>> +			cal->ctx[i] = cal_ctx_create(cal, i);
>> +			if (!cal->ctx[i]) {
>> +				cal_err(cal, "Failed to create context %u\n", i);
>> +				ret = -ENODEV;
>> +				goto error_context;
>> +			}
>>   
>> -		cal->ctx[i] = cal_ctx_create(cal, i);
>> -		if (!cal->ctx[i]) {
>> -			cal_err(cal, "Failed to create context %u\n", i);
>> -			ret = -ENODEV;
>> -			goto error_context;
>> +			cal->ctx[i]->phy = cal->phy[i];
>> +
>> +			cal->num_contexts++;
>>   		}
>> +	} else {
>> +		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
>> +			cal->ctx[i] = cal_ctx_create(cal, i);
>> +			if (!cal->ctx[i]) {
>> +				cal_err(cal, "Failed to create context %u\n", i);
>> +				ret = -ENODEV;
>> +				goto error_context;
>> +			}
>>   
>> -		cal->num_contexts++;
>> +			cal->num_contexts++;
> 
> In cal_async_notifier_complete() I see:
> 
>    for (i = 0; i < cal->num_contexts; i++)
>      ret = cal_ctx_v4l2_register();
> 
> This means that if the CAL device has 8 DMA contexts it will create 8
> /dev/videoX nodes, even if the hardware setup is only capable of 1
> stream.
> 
> Would it make more sense to populate /dev/videoX nodes based on the
> configured routing? So for example, if only one pad is being used to
> output, only create one node corresponding to that pad. If there are 3
> pads being populated then create 3 nodes and so on.

Routing is a runtime configuration, so it could mean creating or 
removing video nodes every time the user changes the routing. I believe 
video nodes are supposed to be more permanent than that.

If we knew that the HW setup can only ever have N routes, we could limit 
the number of video nodes, but I don't think we have means to figure 
that out.

  Tomi

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Laurent Pinchart]

On Thu, May 27, 2021 at 07:10:42PM +0300, Tomi Valkeinen wrote:
> On 27/05/2021 19:06, Pratyush Yadav wrote:
> > On 24/05/21 02:09PM, Tomi Valkeinen wrote:
> >> Add routing and stream_config support to CAL driver.
> >>
> >> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
> >> separate streams at the same time.
> >>
> >> Add 8 video device nodes, each representing a single dma-engine, and set
> >> the number of source pads on camerarx to 8. Each video node can be
> >> connected to any of the source pads on either of the camerarx instances
> >> using media links. Camerarx internal routing is used to route the
> >> incoming CSI-2 streams to one of the 8 source pads.
> >>
> >> CAL doesn't support transcoding, so the driver currently allows changes
> >> only on the camerarx sink side, and then copies the sink pad config to
> >> the source pad. This becomes slighly more complex with 8 source pads and
> >> multiple streams on the sink pad. A helper,
> >> cal_camerarx_get_opposite_stream_format(), is added, which uses the
> >> routing table to get the format from the "opposite" side.
> >>
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >>   drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
> >>   drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
> >>   drivers/media/platform/ti-vpe/cal.c          |  34 ++-
> >>   drivers/media/platform/ti-vpe/cal.h          |  12 +-
> >>   4 files changed, 385 insertions(+), 67 deletions(-)
> >>
> > [...]
> >> @@ -1178,18 +1177,33 @@ static int cal_probe(struct platform_device *pdev)
> >>   	}
> >>   
> >>   	/* Create contexts. */
> >> -	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> >> -		if (!cal->phy[i]->source_node)
> >> -			continue;
> >> +	if (!cal_mc_api) {
> >> +		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> >> +			if (!cal->phy[i]->source_node)
> >> +				continue;
> >> +
> >> +			cal->ctx[i] = cal_ctx_create(cal, i);
> >> +			if (!cal->ctx[i]) {
> >> +				cal_err(cal, "Failed to create context %u\n", i);
> >> +				ret = -ENODEV;
> >> +				goto error_context;
> >> +			}
> >>   
> >> -		cal->ctx[i] = cal_ctx_create(cal, i);
> >> -		if (!cal->ctx[i]) {
> >> -			cal_err(cal, "Failed to create context %u\n", i);
> >> -			ret = -ENODEV;
> >> -			goto error_context;
> >> +			cal->ctx[i]->phy = cal->phy[i];
> >> +
> >> +			cal->num_contexts++;
> >>   		}
> >> +	} else {
> >> +		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> >> +			cal->ctx[i] = cal_ctx_create(cal, i);
> >> +			if (!cal->ctx[i]) {
> >> +				cal_err(cal, "Failed to create context %u\n", i);
> >> +				ret = -ENODEV;
> >> +				goto error_context;
> >> +			}
> >>   
> >> -		cal->num_contexts++;
> >> +			cal->num_contexts++;
> > 
> > In cal_async_notifier_complete() I see:
> > 
> >    for (i = 0; i < cal->num_contexts; i++)
> >      ret = cal_ctx_v4l2_register();
> > 
> > This means that if the CAL device has 8 DMA contexts it will create 8
> > /dev/videoX nodes, even if the hardware setup is only capable of 1
> > stream.
> > 
> > Would it make more sense to populate /dev/videoX nodes based on the
> > configured routing? So for example, if only one pad is being used to
> > output, only create one node corresponding to that pad. If there are 3
> > pads being populated then create 3 nodes and so on.
> 
> Routing is a runtime configuration, so it could mean creating or 
> removing video nodes every time the user changes the routing. I believe 
> video nodes are supposed to be more permanent than that.
> 
> If we knew that the HW setup can only ever have N routes, we could limit 
> the number of video nodes, but I don't think we have means to figure 
> that out.

And even if we did, I think that wouldn't help userspace. The media
graph is meant to model the hardware topology, it's best to minimize the
complexity on the kernel side and let userspace deal with routing
configuration.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Tomi Valkeinen]

On 27/05/2021 19:30, Laurent Pinchart wrote:
> On Thu, May 27, 2021 at 07:10:42PM +0300, Tomi Valkeinen wrote:
>> On 27/05/2021 19:06, Pratyush Yadav wrote:
>>> On 24/05/21 02:09PM, Tomi Valkeinen wrote:
>>>> Add routing and stream_config support to CAL driver.
>>>>
>>>> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
>>>> separate streams at the same time.
>>>>
>>>> Add 8 video device nodes, each representing a single dma-engine, and set
>>>> the number of source pads on camerarx to 8. Each video node can be
>>>> connected to any of the source pads on either of the camerarx instances
>>>> using media links. Camerarx internal routing is used to route the
>>>> incoming CSI-2 streams to one of the 8 source pads.
>>>>
>>>> CAL doesn't support transcoding, so the driver currently allows changes
>>>> only on the camerarx sink side, and then copies the sink pad config to
>>>> the source pad. This becomes slighly more complex with 8 source pads and
>>>> multiple streams on the sink pad. A helper,
>>>> cal_camerarx_get_opposite_stream_format(), is added, which uses the
>>>> routing table to get the format from the "opposite" side.
>>>>
>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>> ---
>>>>    drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
>>>>    drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
>>>>    drivers/media/platform/ti-vpe/cal.c          |  34 ++-
>>>>    drivers/media/platform/ti-vpe/cal.h          |  12 +-
>>>>    4 files changed, 385 insertions(+), 67 deletions(-)
>>>>
>>> [...]
>>>> @@ -1178,18 +1177,33 @@ static int cal_probe(struct platform_device *pdev)
>>>>    	}
>>>>    
>>>>    	/* Create contexts. */
>>>> -	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
>>>> -		if (!cal->phy[i]->source_node)
>>>> -			continue;
>>>> +	if (!cal_mc_api) {
>>>> +		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
>>>> +			if (!cal->phy[i]->source_node)
>>>> +				continue;
>>>> +
>>>> +			cal->ctx[i] = cal_ctx_create(cal, i);
>>>> +			if (!cal->ctx[i]) {
>>>> +				cal_err(cal, "Failed to create context %u\n", i);
>>>> +				ret = -ENODEV;
>>>> +				goto error_context;
>>>> +			}
>>>>    
>>>> -		cal->ctx[i] = cal_ctx_create(cal, i);
>>>> -		if (!cal->ctx[i]) {
>>>> -			cal_err(cal, "Failed to create context %u\n", i);
>>>> -			ret = -ENODEV;
>>>> -			goto error_context;
>>>> +			cal->ctx[i]->phy = cal->phy[i];
>>>> +
>>>> +			cal->num_contexts++;
>>>>    		}
>>>> +	} else {
>>>> +		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
>>>> +			cal->ctx[i] = cal_ctx_create(cal, i);
>>>> +			if (!cal->ctx[i]) {
>>>> +				cal_err(cal, "Failed to create context %u\n", i);
>>>> +				ret = -ENODEV;
>>>> +				goto error_context;
>>>> +			}
>>>>    
>>>> -		cal->num_contexts++;
>>>> +			cal->num_contexts++;
>>>
>>> In cal_async_notifier_complete() I see:
>>>
>>>     for (i = 0; i < cal->num_contexts; i++)
>>>       ret = cal_ctx_v4l2_register();
>>>
>>> This means that if the CAL device has 8 DMA contexts it will create 8
>>> /dev/videoX nodes, even if the hardware setup is only capable of 1
>>> stream.
>>>
>>> Would it make more sense to populate /dev/videoX nodes based on the
>>> configured routing? So for example, if only one pad is being used to
>>> output, only create one node corresponding to that pad. If there are 3
>>> pads being populated then create 3 nodes and so on.
>>
>> Routing is a runtime configuration, so it could mean creating or
>> removing video nodes every time the user changes the routing. I believe
>> video nodes are supposed to be more permanent than that.
>>
>> If we knew that the HW setup can only ever have N routes, we could limit
>> the number of video nodes, but I don't think we have means to figure
>> that out.
> 
> And even if we did, I think that wouldn't help userspace. The media
> graph is meant to model the hardware topology, it's best to minimize the
> complexity on the kernel side and let userspace deal with routing
> configuration.

I think it's a valid question. Maybe a CSI-2 RX uses system DMA, and can 
support, say, 128 contexts. We probably don't want 128 video nodes (of 
which perhaps 1-4 are ever used). But in CAL's case, I think always 
having all the 8 video nodes is acceptable.

  Tomi

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Laurent Pinchart]

Hi Tomi,

On Thu, May 27, 2021 at 07:33:57PM +0300, Tomi Valkeinen wrote:
> On 27/05/2021 19:30, Laurent Pinchart wrote:
> > On Thu, May 27, 2021 at 07:10:42PM +0300, Tomi Valkeinen wrote:
> >> On 27/05/2021 19:06, Pratyush Yadav wrote:
> >>> On 24/05/21 02:09PM, Tomi Valkeinen wrote:
> >>>> Add routing and stream_config support to CAL driver.
> >>>>
> >>>> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
> >>>> separate streams at the same time.
> >>>>
> >>>> Add 8 video device nodes, each representing a single dma-engine, and set
> >>>> the number of source pads on camerarx to 8. Each video node can be
> >>>> connected to any of the source pads on either of the camerarx instances
> >>>> using media links. Camerarx internal routing is used to route the
> >>>> incoming CSI-2 streams to one of the 8 source pads.
> >>>>
> >>>> CAL doesn't support transcoding, so the driver currently allows changes
> >>>> only on the camerarx sink side, and then copies the sink pad config to
> >>>> the source pad. This becomes slighly more complex with 8 source pads and
> >>>> multiple streams on the sink pad. A helper,
> >>>> cal_camerarx_get_opposite_stream_format(), is added, which uses the
> >>>> routing table to get the format from the "opposite" side.
> >>>>
> >>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >>>> ---
> >>>>    drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
> >>>>    drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
> >>>>    drivers/media/platform/ti-vpe/cal.c          |  34 ++-
> >>>>    drivers/media/platform/ti-vpe/cal.h          |  12 +-
> >>>>    4 files changed, 385 insertions(+), 67 deletions(-)
> >>>>
> >>> [...]
> >>>> @@ -1178,18 +1177,33 @@ static int cal_probe(struct platform_device *pdev)
> >>>>    	}
> >>>>    
> >>>>    	/* Create contexts. */
> >>>> -	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> >>>> -		if (!cal->phy[i]->source_node)
> >>>> -			continue;
> >>>> +	if (!cal_mc_api) {
> >>>> +		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> >>>> +			if (!cal->phy[i]->source_node)
> >>>> +				continue;
> >>>> +
> >>>> +			cal->ctx[i] = cal_ctx_create(cal, i);
> >>>> +			if (!cal->ctx[i]) {
> >>>> +				cal_err(cal, "Failed to create context %u\n", i);
> >>>> +				ret = -ENODEV;
> >>>> +				goto error_context;
> >>>> +			}
> >>>>    
> >>>> -		cal->ctx[i] = cal_ctx_create(cal, i);
> >>>> -		if (!cal->ctx[i]) {
> >>>> -			cal_err(cal, "Failed to create context %u\n", i);
> >>>> -			ret = -ENODEV;
> >>>> -			goto error_context;
> >>>> +			cal->ctx[i]->phy = cal->phy[i];
> >>>> +
> >>>> +			cal->num_contexts++;
> >>>>    		}
> >>>> +	} else {
> >>>> +		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> >>>> +			cal->ctx[i] = cal_ctx_create(cal, i);
> >>>> +			if (!cal->ctx[i]) {
> >>>> +				cal_err(cal, "Failed to create context %u\n", i);
> >>>> +				ret = -ENODEV;
> >>>> +				goto error_context;
> >>>> +			}
> >>>>    
> >>>> -		cal->num_contexts++;
> >>>> +			cal->num_contexts++;
> >>>
> >>> In cal_async_notifier_complete() I see:
> >>>
> >>>     for (i = 0; i < cal->num_contexts; i++)
> >>>       ret = cal_ctx_v4l2_register();
> >>>
> >>> This means that if the CAL device has 8 DMA contexts it will create 8
> >>> /dev/videoX nodes, even if the hardware setup is only capable of 1
> >>> stream.
> >>>
> >>> Would it make more sense to populate /dev/videoX nodes based on the
> >>> configured routing? So for example, if only one pad is being used to
> >>> output, only create one node corresponding to that pad. If there are 3
> >>> pads being populated then create 3 nodes and so on.
> >>
> >> Routing is a runtime configuration, so it could mean creating or
> >> removing video nodes every time the user changes the routing. I believe
> >> video nodes are supposed to be more permanent than that.
> >>
> >> If we knew that the HW setup can only ever have N routes, we could limit
> >> the number of video nodes, but I don't think we have means to figure
> >> that out.
> > 
> > And even if we did, I think that wouldn't help userspace. The media
> > graph is meant to model the hardware topology, it's best to minimize the
> > complexity on the kernel side and let userspace deal with routing
> > configuration.
> 
> I think it's a valid question. Maybe a CSI-2 RX uses system DMA, and can 
> support, say, 128 contexts. We probably don't want 128 video nodes (of 
> which perhaps 1-4 are ever used). But in CAL's case, I think always 
> having all the 8 video nodes is acceptable.

Agreed, I wouldn't want to see 128 video nodes. If we had to support a
large number of contexts (in which case those contexts would either not
map to dedicated hardware resources, or map to very cheap hardware
resources), then I'd vote for adding a context ID to the buffer and
streaming ioctls on the video node. VIDIOC_STREAMON and VIDIOC_STREAMOFF
would be problematic as there's no room for extension, but it could be a
good occasion to introduce a VIDIOC_S_STREAM.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Mon, May 24, 2021 at 02:09:09PM +0300, Tomi Valkeinen wrote:
> Add routing and stream_config support to CAL driver.
> 
> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
> separate streams at the same time.
> 
> Add 8 video device nodes, each representing a single dma-engine, and set
> the number of source pads on camerarx to 8. Each video node can be
> connected to any of the source pads on either of the camerarx instances
> using media links. Camerarx internal routing is used to route the
> incoming CSI-2 streams to one of the 8 source pads.
> 
> CAL doesn't support transcoding, so the driver currently allows changes
> only on the camerarx sink side, and then copies the sink pad config to
> the source pad. This becomes slighly more complex with 8 source pads and
> multiple streams on the sink pad. A helper,
> cal_camerarx_get_opposite_stream_format(), is added, which uses the
> routing table to get the format from the "opposite" side.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
>  drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
>  drivers/media/platform/ti-vpe/cal.c          |  34 ++-
>  drivers/media/platform/ti-vpe/cal.h          |  12 +-
>  4 files changed, 385 insertions(+), 67 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index cb6a37f47432..d09b06780b15 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -49,15 +49,33 @@ static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
>  {
>  	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
>  	u32 num_lanes = mipi_csi2->num_data_lanes;
> -	const struct cal_format_info *fmtinfo;
>  	u32 bpp;
>  	s64 freq;
>  
> -	fmtinfo = cal_format_by_code(phy->formats[CAL_CAMERARX_PAD_SINK].code);
> -	if (!fmtinfo)
> +	/*
> +	 * With multistream input we don't have bpp, and cannot use
> +	 * V4L2_CID_PIXEL_RATE. Passing 0 as bpp causes v4l2_get_link_freq()
> +	 * to return an error if it falls back to V4L2_CID_PIXEL_RATE.
> +	 */

I agree with the comment, but I think what should be explained here is
that we allow falling back to V4L2_CID_PIXEL_RATE when there's a single
stream, and require V4L2_CID_LINK_FREQ otherwise.

> +
> +	if (phy->stream_configs.num_configs == 0)
>  		return -EINVAL;
>  
> -	bpp = fmtinfo->bpp;
> +	if (phy->stream_configs.num_configs > 2) {

Should this be >= 2 ?

> +		bpp = 0;
> +	} else {
> +		const struct cal_format_info *fmtinfo;
> +		struct v4l2_mbus_framefmt *fmt;
> +
> +		/* The first format is for the sink */
> +		fmt = &phy->stream_configs.configs[0].fmt;
> +
> +		fmtinfo = cal_format_by_code(fmt->code);
> +		if (!fmtinfo)
> +			return -EINVAL;
> +
> +		bpp = fmtinfo->bpp;
> +	}
>  
>  	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
>  	if (freq < 0) {
> @@ -619,19 +637,104 @@ static inline struct cal_camerarx *to_cal_camerarx(struct v4l2_subdev *sd)
>  	return container_of(sd, struct cal_camerarx, subdev);
>  }
>  
> -static struct v4l2_mbus_framefmt *
> -cal_camerarx_get_pad_format(struct cal_camerarx *phy,
> -			    struct v4l2_subdev_state *sd_state,
> -			    unsigned int pad, u32 which)
> -{
> -	switch (which) {
> -	case V4L2_SUBDEV_FORMAT_TRY:
> -		return v4l2_subdev_get_try_format(&phy->subdev, sd_state, pad);
> -	case V4L2_SUBDEV_FORMAT_ACTIVE:
> -		return &phy->formats[pad];
> -	default:
> +struct cal_camerarx *
> +cal_camerarx_get_phy_from_entity(struct media_entity *entity)
> +{
> +	struct v4l2_subdev *sd;
> +
> +	sd = media_entity_to_v4l2_subdev(entity);
> +	if (!sd)
>  		return NULL;
> +
> +	return to_cal_camerarx(sd);
> +}
> +
> +static struct v4l2_subdev_krouting *
> +cal_camerarx_get_routing_table(struct cal_camerarx *phy,
> +			       struct v4l2_subdev_state *sd_state, u32 which)
> +{
> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
> +		return &phy->routing;
> +	else
> +		return &sd_state->routing;
> +}
> +
> +static struct v4l2_subdev_stream_configs *
> +cal_camerarx_get_stream_configs(struct cal_camerarx *phy,
> +				struct v4l2_subdev_state *sd_state, u32 which)
> +{
> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
> +		return &phy->stream_configs;
> +	else
> +		return &sd_state->stream_configs;
> +}
> +
> +struct v4l2_mbus_framefmt *
> +cal_camerarx_get_stream_format(struct cal_camerarx *phy,
> +			       struct v4l2_subdev_state *sd_state,
> +			       unsigned int pad, u32 stream, u32 which)
> +{
> +	struct v4l2_subdev_stream_configs *stream_configs;
> +	unsigned int i;
> +
> +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
> +
> +	for (i = 0; i < stream_configs->num_configs; ++i) {
> +		if (stream_configs->configs[i].pad == pad &&
> +		    stream_configs->configs[i].stream == stream)
> +			return &stream_configs->configs[i].fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +static int cal_camerarx_find_opposite_end(struct v4l2_subdev_krouting *routing,
> +					  u32 pad, u32 stream, u32 *other_pad,
> +					  u32 *other_stream)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < routing->num_routes; ++i) {
> +		struct v4l2_subdev_route *route = &routing->routes[i];
> +
> +		if (cal_rx_pad_is_source(pad)) {
> +			if (route->source_pad == pad &&
> +			    route->source_stream == stream) {
> +				*other_pad = route->sink_pad;
> +				*other_stream = route->sink_stream;
> +				return 0;
> +			}
> +		} else {
> +			if (route->sink_pad == pad &&
> +			    route->sink_stream == stream) {
> +				*other_pad = route->source_pad;
> +				*other_stream = route->source_stream;
> +				return 0;
> +			}
> +		}
>  	}
> +
> +	return -EINVAL;
> +}
> +
> +static struct v4l2_mbus_framefmt *
> +cal_camerarx_get_opposite_stream_format(struct cal_camerarx *phy,
> +					struct v4l2_subdev_state *sd_state,
> +					u32 pad, u32 stream, u32 which)
> +{
> +	struct v4l2_subdev_krouting *routing;
> +	u32 other_pad, other_stream;
> +	int ret;
> +
> +	routing = cal_camerarx_get_routing_table(phy, sd_state, which);
> +
> +	ret = cal_camerarx_find_opposite_end(routing, pad, stream, &other_pad,
> +					     &other_stream);
> +	if (ret)
> +		return NULL;
> +
> +	return cal_camerarx_get_stream_format(phy, sd_state, other_pad,
> +					      other_stream, which);
>  }
>  
>  static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
> @@ -669,9 +772,15 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
>  			goto out;
>  		}
>  
> -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> -						  CAL_CAMERARX_PAD_SINK,
> -						  code->which);
> +		fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state,
> +					code->pad, code->stream,
> +					code->which);
> +
> +		if (!fmt) {
> +			r = -EINVAL;
> +			goto out;
> +		}
> +
>  		code->code = fmt->code;
>  	} else {
>  		if (code->index >= cal_num_formats) {
> @@ -705,9 +814,14 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>  	if (cal_rx_pad_is_source(fse->pad)) {
>  		struct v4l2_mbus_framefmt *fmt;
>  
> -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> -						  CAL_CAMERARX_PAD_SINK,
> -						  fse->which);
> +		fmt = cal_camerarx_get_opposite_stream_format(
> +			phy, sd_state, fse->pad, fse->stream, fse->which);
> +
> +		if (!fmt) {
> +			r = -EINVAL;
> +			goto out;
> +		}
> +
>  		if (fse->code != fmt->code) {
>  			r = -EINVAL;
>  			goto out;
> @@ -747,8 +861,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
>  
>  	mutex_lock(&phy->mutex);
>  
> -	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
> -					  format->which);
> +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
> +					     format->stream, format->which);
> +
> +	if (!fmt) {
> +		mutex_unlock(&phy->mutex);
> +		return -EINVAL;
> +	}
> +
>  	format->format = *fmt;
>  
>  	mutex_unlock(&phy->mutex);
> @@ -764,6 +884,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  	const struct cal_format_info *fmtinfo;
>  	struct v4l2_mbus_framefmt *fmt;
>  	unsigned int bpp;
> +	int ret = 0;
>  
>  	/* No transcoding, source and sink formats must match. */
>  	if (cal_rx_pad_is_source(format->pad))
> @@ -792,40 +913,117 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  	/* Store the format and propagate it to the source pad. */
>  	mutex_lock(&phy->mutex);
>  
> -	fmt = cal_camerarx_get_pad_format(phy, sd_state,
> -					  CAL_CAMERARX_PAD_SINK,
> -					  format->which);
> +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
> +					     format->stream, format->which);
> +	if (!fmt) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
>  	*fmt = format->format;
>  
> -	fmt = cal_camerarx_get_pad_format(phy, sd_state, CAL_CAMERARX_PAD_FIRST_SOURCE,
> -					  format->which);
> +	fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state, format->pad,
> +						      format->stream,
> +						      format->which);
> +	if (!fmt) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
>  	*fmt = format->format;
>  
> +out:
>  	mutex_unlock(&phy->mutex);
>  
> +	return ret;
> +}
> +
> +static int cal_camerarx_sd_get_routing(struct v4l2_subdev *sd,
> +				       struct v4l2_subdev_state *sd_state,
> +				       struct v4l2_subdev_krouting *routing)
> +{
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +	struct v4l2_subdev_krouting *src;
> +
> +	src = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
> +
> +	return v4l2_subdev_cpy_routing(routing, src);
> +}
> +
> +static void cal_camerarx_init_formats(struct v4l2_subdev *sd,
> +				      struct v4l2_subdev_state *sd_state,
> +				      u32 which)
> +{
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +
> +	static const struct v4l2_mbus_framefmt format = {
> +		.width = 640,
> +		.height = 480,
> +		.code = MEDIA_BUS_FMT_UYVY8_2X8,
> +		.field = V4L2_FIELD_NONE,
> +		.colorspace = V4L2_COLORSPACE_SRGB,
> +		.ycbcr_enc = V4L2_YCBCR_ENC_601,
> +		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> +		.xfer_func = V4L2_XFER_FUNC_SRGB,
> +	};
> +
> +	struct v4l2_subdev_stream_configs *stream_configs;
> +	unsigned int i;
> +
> +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
> +
> +	for (i = 0; i < stream_configs->num_configs; ++i)
> +		stream_configs->configs[i].fmt = format;
> +}
> +
> +static int cal_camerarx_sd_set_routing(struct v4l2_subdev *sd,
> +				       struct v4l2_subdev_state *sd_state,
> +				       struct v4l2_subdev_krouting *routing)
> +{
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +	int ret;
> +	struct v4l2_subdev_krouting *dst;
> +	struct v4l2_subdev_stream_configs *stream_configs;
> +
> +	dst = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
> +	stream_configs =
> +		cal_camerarx_get_stream_configs(phy, sd_state, routing->which);
> +
> +	ret = v4l2_subdev_dup_routing(dst, routing);
> +	if (ret)
> +		return ret;
> +
> +	ret = v4l2_init_stream_configs(stream_configs, dst);
> +	if (ret)
> +		return ret;
> +
> +	/* Initialize stream formats */
> +	cal_camerarx_init_formats(sd, sd_state, routing->which);

Reinitializing all formats when routing is modified is harsh. We need to
discuss it as a formal spec, part of the API proposal, as commented in
separate e-mails. This will also influence the .set_fmt()
implementation.

> +
>  	return 0;
>  }
>  
>  static int cal_camerarx_sd_init_cfg(struct v4l2_subdev *sd,
>  				    struct v4l2_subdev_state *sd_state)
>  {
> -	struct v4l2_subdev_format format = {
> -		.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY
> -		: V4L2_SUBDEV_FORMAT_ACTIVE,
> -		.pad = CAL_CAMERARX_PAD_SINK,
> -		.format = {
> -			.width = 640,
> -			.height = 480,
> -			.code = MEDIA_BUS_FMT_UYVY8_2X8,
> -			.field = V4L2_FIELD_NONE,
> -			.colorspace = V4L2_COLORSPACE_SRGB,
> -			.ycbcr_enc = V4L2_YCBCR_ENC_601,
> -			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> -			.xfer_func = V4L2_XFER_FUNC_SRGB,
> -		},
> +	u32 which = sd_state ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE;
> +
> +	struct v4l2_subdev_route routes[] = { {
> +		.sink_pad = 0,
> +		.sink_stream = 0,
> +		.source_pad = 1,
> +		.source_stream = 0,
> +		.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> +	} };
> +
> +	struct v4l2_subdev_krouting routing = {
> +		.which = which,
> +		.num_routes = 1,
> +		.routes = routes,
>  	};
>  
> -	return cal_camerarx_sd_set_fmt(sd, sd_state, &format);
> +	/* Initialize routing to single route to the fist source pad */
> +	return cal_camerarx_sd_set_routing(sd, sd_state, &routing);
>  }
>  
>  static const struct v4l2_subdev_video_ops cal_camerarx_video_ops = {
> @@ -838,6 +1036,8 @@ static const struct v4l2_subdev_pad_ops cal_camerarx_pad_ops = {
>  	.enum_frame_size = cal_camerarx_sd_enum_frame_size,
>  	.get_fmt = cal_camerarx_sd_get_fmt,
>  	.set_fmt = cal_camerarx_sd_set_fmt,
> +	.get_routing = cal_camerarx_sd_get_routing,
> +	.set_routing = cal_camerarx_sd_set_routing,
>  };
>  
>  static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
> @@ -845,8 +1045,18 @@ static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
>  	.pad = &cal_camerarx_pad_ops,
>  };
>  
> +static bool cal_camerarx_has_route(struct media_entity *entity, unsigned int pad0,
> +			  unsigned int pad1)
> +{
> +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +
> +	return v4l2_subdev_has_route(&phy->routing, pad0, pad1);
> +}
> +
>  static struct media_entity_operations cal_camerarx_media_ops = {
>  	.link_validate = v4l2_subdev_link_validate,
> +	.has_route = cal_camerarx_has_route,

Once we store the active subdev state in the subdev, this can become a
standard v4l2_subdev_has_route() helper.

>  };
>  
>  /* ------------------------------------------------------------------
> @@ -898,11 +1108,12 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	sd = &phy->subdev;
>  	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
>  	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> -	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_MULTIPLEXED;
>  	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
>  	sd->dev = cal->dev;
>  
>  	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
>  	for (i = CAL_CAMERARX_PAD_FIRST_SOURCE; i < CAL_CAMERARX_NUM_PADS; ++i)
>  		phy->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>  	sd->entity.ops = &cal_camerarx_media_ops;
> @@ -922,6 +1133,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	return phy;
>  
>  error:
> +	v4l2_subdev_free_routing(&phy->routing);
> +	v4l2_uninit_stream_configs(&phy->stream_configs);
>  	media_entity_cleanup(&phy->subdev.entity);
>  	kfree(phy);
>  	return ERR_PTR(ret);
> @@ -933,6 +1146,8 @@ void cal_camerarx_destroy(struct cal_camerarx *phy)
>  		return;
>  
>  	v4l2_device_unregister_subdev(&phy->subdev);
> +	v4l2_subdev_free_routing(&phy->routing);
> +	v4l2_uninit_stream_configs(&phy->stream_configs);
>  	media_entity_cleanup(&phy->subdev.entity);
>  	of_node_put(phy->source_ep_node);
>  	of_node_put(phy->source_node);
> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> index 8ecae7dc2774..234af40a24fa 100644
> --- a/drivers/media/platform/ti-vpe/cal-video.c
> +++ b/drivers/media/platform/ti-vpe/cal-video.c
> @@ -693,7 +693,11 @@ static int cal_video_check_format(struct cal_ctx *ctx)
>  	if (!remote_pad)
>  		return -ENODEV;
>  
> -	format = &ctx->phy->formats[remote_pad->index];
> +	format = cal_camerarx_get_stream_format(ctx->phy, NULL,
> +						remote_pad->index, 0,
> +						V4L2_SUBDEV_FORMAT_ACTIVE);
> +	if (!format)
> +		return -EINVAL;
>  
>  	if (ctx->fmtinfo->code != format->code ||
>  	    ctx->v_fmt.fmt.pix.height != format->height ||
> @@ -711,6 +715,48 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>  	dma_addr_t addr;
>  	int ret;
>  
> +	if (cal_mc_api) {
> +		struct v4l2_subdev_route *route = NULL;
> +		struct media_pad *remote_pad;
> +		unsigned int i;
> +
> +		/* Find the PHY connected to this video device */
> +
> +		remote_pad = media_entity_remote_pad(&ctx->pad);
> +		if (!remote_pad) {
> +			ctx_err(ctx, "Context not connected\n");
> +			ret = -ENODEV;
> +			goto error_release_buffers;
> +		}
> +
> +		ctx->phy = cal_camerarx_get_phy_from_entity(remote_pad->entity);
> +
> +		/* Find the stream */
> +
> +		for (i = 0; i < ctx->phy->routing.num_routes; ++i) {
> +			struct v4l2_subdev_route *r =
> +				&ctx->phy->routing.routes[i];
> +
> +			if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> +				continue;
> +
> +			if (r->source_pad != remote_pad->index)
> +				continue;
> +
> +			route = r;
> +
> +			break;
> +		}
> +
> +		if (!route) {
> +			ctx_err(ctx, "Failed to find route\n");
> +			ret = -ENODEV;
> +			goto error_release_buffers;
> +		}
> +
> +		ctx->stream = route->sink_stream;
> +	}
> +
>  	ret = media_pipeline_start(ctx->vdev.entity.pads, &ctx->phy->pipe);
>  	if (ret < 0) {
>  		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
> @@ -784,6 +830,9 @@ static void cal_stop_streaming(struct vb2_queue *vq)
>  	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
>  
>  	media_pipeline_stop(ctx->vdev.entity.pads);
> +
> +	if (cal_mc_api)
> +		ctx->phy = NULL;
>  }
>  
>  static const struct vb2_ops cal_video_qops = {
> @@ -945,16 +994,48 @@ int cal_ctx_v4l2_register(struct cal_ctx *ctx)
>  		return ret;
>  	}
>  
> -	ret = media_create_pad_link(&ctx->phy->subdev.entity,
> -				    CAL_CAMERARX_PAD_FIRST_SOURCE,
> -				    &vfd->entity, 0,
> -				    MEDIA_LNK_FL_IMMUTABLE |
> -				    MEDIA_LNK_FL_ENABLED);
> -	if (ret) {
> -		ctx_err(ctx, "Failed to create media link for context %u\n",
> -			ctx->dma_ctx);
> -		video_unregister_device(vfd);
> -		return ret;
> +	if (cal_mc_api) {
> +		u16 phy_idx;
> +		u16 pad_idx;
> +
> +		/* Create links from all video nodes to all PHYs */
> +
> +		for (phy_idx = 0; phy_idx < ctx->cal->data->num_csi2_phy; ++phy_idx) {
> +			for (pad_idx = 1; pad_idx < CAL_CAMERARX_NUM_PADS; ++pad_idx) {
> +				/*
> +				 * Enable only links from video0 to PHY0 pad 1, and
> +				 * video1 to PHY1 pad 1.
> +				 */
> +				bool enable = (ctx->dma_ctx == 0 &&
> +					       phy_idx == 0 && pad_idx == 1) ||
> +					      (ctx->dma_ctx == 1 &&
> +					       phy_idx == 1 && pad_idx == 1);
> +
> +				ret = media_create_pad_link(
> +					&ctx->cal->phy[phy_idx]->subdev.entity,
> +					pad_idx, &vfd->entity, 0,
> +					enable ? MEDIA_LNK_FL_ENABLED : 0);
> +				if (ret) {
> +					ctx_err(ctx,
> +						"Failed to create media link for context %u\n",
> +						ctx->dma_ctx);
> +					video_unregister_device(vfd);
> +					return ret;
> +				}
> +			}
> +		}
> +	} else {
> +		ret = media_create_pad_link(
> +			&ctx->phy->subdev.entity, CAL_CAMERARX_PAD_FIRST_SOURCE,
> +			&vfd->entity, 0,
> +			MEDIA_LNK_FL_IMMUTABLE | MEDIA_LNK_FL_ENABLED);
> +		if (ret) {
> +			ctx_err(ctx,
> +				"Failed to create media link for context %u\n",
> +				ctx->dma_ctx);
> +			video_unregister_device(vfd);
> +			return ret;
> +		}
>  	}
>  
>  	ctx_info(ctx, "V4L2 device registered as %s\n",
> diff --git a/drivers/media/platform/ti-vpe/cal.c b/drivers/media/platform/ti-vpe/cal.c
> index 7975bb449acd..5fbb7a90c795 100644
> --- a/drivers/media/platform/ti-vpe/cal.c
> +++ b/drivers/media/platform/ti-vpe/cal.c
> @@ -967,7 +967,6 @@ static struct cal_ctx *cal_ctx_create(struct cal_dev *cal, int inst)
>  		return NULL;
>  
>  	ctx->cal = cal;
> -	ctx->phy = cal->phy[inst];
>  	ctx->dma_ctx = inst;
>  	ctx->csi2_ctx = inst;
>  	ctx->cport = inst;
> @@ -1178,18 +1177,33 @@ static int cal_probe(struct platform_device *pdev)
>  	}
>  
>  	/* Create contexts. */
> -	for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> -		if (!cal->phy[i]->source_node)
> -			continue;
> +	if (!cal_mc_api) {
> +		for (i = 0; i < cal->data->num_csi2_phy; ++i) {
> +			if (!cal->phy[i]->source_node)
> +				continue;
> +
> +			cal->ctx[i] = cal_ctx_create(cal, i);
> +			if (!cal->ctx[i]) {
> +				cal_err(cal, "Failed to create context %u\n", i);
> +				ret = -ENODEV;
> +				goto error_context;
> +			}
>  
> -		cal->ctx[i] = cal_ctx_create(cal, i);
> -		if (!cal->ctx[i]) {
> -			cal_err(cal, "Failed to create context %u\n", i);
> -			ret = -ENODEV;
> -			goto error_context;
> +			cal->ctx[i]->phy = cal->phy[i];
> +
> +			cal->num_contexts++;
>  		}
> +	} else {
> +		for (i = 0; i < ARRAY_SIZE(cal->ctx); ++i) {
> +			cal->ctx[i] = cal_ctx_create(cal, i);
> +			if (!cal->ctx[i]) {
> +				cal_err(cal, "Failed to create context %u\n", i);
> +				ret = -ENODEV;
> +				goto error_context;
> +			}
>  
> -		cal->num_contexts++;
> +			cal->num_contexts++;
> +		}
>  	}
>  
>  	/* Register the media device. */
> diff --git a/drivers/media/platform/ti-vpe/cal.h b/drivers/media/platform/ti-vpe/cal.h
> index 3aea444f8bf8..6626c2a59fc2 100644
> --- a/drivers/media/platform/ti-vpe/cal.h
> +++ b/drivers/media/platform/ti-vpe/cal.h
> @@ -45,7 +45,7 @@
>  
>  #define CAL_CAMERARX_PAD_SINK		0
>  #define CAL_CAMERARX_PAD_FIRST_SOURCE	1
> -#define CAL_CAMERARX_NUM_SOURCE_PADS	1
> +#define CAL_CAMERARX_NUM_SOURCE_PADS	8
>  #define CAL_CAMERARX_NUM_PADS		(1 + CAL_CAMERARX_NUM_SOURCE_PADS)
>  
>  static inline bool cal_rx_pad_is_sink(u32 pad)
> @@ -178,12 +178,14 @@ struct cal_camerarx {
>  
>  	struct v4l2_subdev	subdev;
>  	struct media_pad	pads[CAL_CAMERARX_NUM_PADS];
> -	struct v4l2_mbus_framefmt	formats[CAL_CAMERARX_NUM_PADS];
>  
>  	/* mutex for camerarx ops */
>  	struct mutex		mutex;
>  
>  	unsigned int enable_count;
> +
> +	struct v4l2_subdev_krouting routing;
> +	struct v4l2_subdev_stream_configs stream_configs;

Repeating a previous comment, let's store a v4l2_subdev_state in
v4l2_subdev :-) I'm not sure yet if it should be an embedded instance or
a pointer, a bit of experimentation is likely needed.

>  };
>  
>  struct cal_dev {
> @@ -321,6 +323,7 @@ void cal_quickdump_regs(struct cal_dev *cal);
>  
>  int cal_camerarx_get_remote_frame_desc(struct cal_camerarx *phy,
>  				       struct v4l2_mbus_frame_desc *fd);
> +struct cal_camerarx *cal_camerarx_get_phy_from_entity(struct media_entity *entity);
>  void cal_camerarx_disable(struct cal_camerarx *phy);
>  void cal_camerarx_i913_errata(struct cal_camerarx *phy);
>  struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
> @@ -338,4 +341,9 @@ void cal_ctx_v4l2_unregister(struct cal_ctx *ctx);
>  int cal_ctx_v4l2_init(struct cal_ctx *ctx);
>  void cal_ctx_v4l2_cleanup(struct cal_ctx *ctx);
>  
> +struct v4l2_mbus_framefmt *
> +cal_camerarx_get_stream_format(struct cal_camerarx *phy,
> +			       struct v4l2_subdev_state *state,
> +			       unsigned int pad, u32 stream, u32 which);
> +
>  #endif /* __TI_CAL_H__ */

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Tomi Valkeinen]

Hi,

On 06/06/2021 19:14, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Mon, May 24, 2021 at 02:09:09PM +0300, Tomi Valkeinen wrote:
>> Add routing and stream_config support to CAL driver.
>>
>> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
>> separate streams at the same time.
>>
>> Add 8 video device nodes, each representing a single dma-engine, and set
>> the number of source pads on camerarx to 8. Each video node can be
>> connected to any of the source pads on either of the camerarx instances
>> using media links. Camerarx internal routing is used to route the
>> incoming CSI-2 streams to one of the 8 source pads.
>>
>> CAL doesn't support transcoding, so the driver currently allows changes
>> only on the camerarx sink side, and then copies the sink pad config to
>> the source pad. This becomes slighly more complex with 8 source pads and
>> multiple streams on the sink pad. A helper,
>> cal_camerarx_get_opposite_stream_format(), is added, which uses the
>> routing table to get the format from the "opposite" side.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
>>   drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
>>   drivers/media/platform/ti-vpe/cal.c          |  34 ++-
>>   drivers/media/platform/ti-vpe/cal.h          |  12 +-
>>   4 files changed, 385 insertions(+), 67 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> index cb6a37f47432..d09b06780b15 100644
>> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
>> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> @@ -49,15 +49,33 @@ static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
>>   {
>>   	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
>>   	u32 num_lanes = mipi_csi2->num_data_lanes;
>> -	const struct cal_format_info *fmtinfo;
>>   	u32 bpp;
>>   	s64 freq;
>>   
>> -	fmtinfo = cal_format_by_code(phy->formats[CAL_CAMERARX_PAD_SINK].code);
>> -	if (!fmtinfo)
>> +	/*
>> +	 * With multistream input we don't have bpp, and cannot use
>> +	 * V4L2_CID_PIXEL_RATE. Passing 0 as bpp causes v4l2_get_link_freq()
>> +	 * to return an error if it falls back to V4L2_CID_PIXEL_RATE.
>> +	 */
> 
> I agree with the comment, but I think what should be explained here is
> that we allow falling back to V4L2_CID_PIXEL_RATE when there's a single
> stream, and require V4L2_CID_LINK_FREQ otherwise.
> 
>> +
>> +	if (phy->stream_configs.num_configs == 0)
>>   		return -EINVAL;
>>   
>> -	bpp = fmtinfo->bpp;
>> +	if (phy->stream_configs.num_configs > 2) {
> 
> Should this be >= 2 ?

If there is one stream, from a sink pad to a source pad, we have two 
configs, as each pad+stream tuple has its own config.

  Tomi

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Pratyush Yadav]

Hi Tomi,

Thanks for your work on this. I have used your patches to add 
multiplexed stream support on J721E. A few thoughts below that came to 
my mind when implementing it.

On 24/05/21 02:09PM, Tomi Valkeinen wrote:
> Add routing and stream_config support to CAL driver.
> 
> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
> separate streams at the same time.
> 
> Add 8 video device nodes, each representing a single dma-engine, and set
> the number of source pads on camerarx to 8. Each video node can be
> connected to any of the source pads on either of the camerarx instances
> using media links. Camerarx internal routing is used to route the
> incoming CSI-2 streams to one of the 8 source pads.
> 
> CAL doesn't support transcoding, so the driver currently allows changes
> only on the camerarx sink side, and then copies the sink pad config to
> the source pad. This becomes slighly more complex with 8 source pads and
> multiple streams on the sink pad. A helper,
> cal_camerarx_get_opposite_stream_format(), is added, which uses the
> routing table to get the format from the "opposite" side.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
>  drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
>  drivers/media/platform/ti-vpe/cal.c          |  34 ++-
>  drivers/media/platform/ti-vpe/cal.h          |  12 +-
>  4 files changed, 385 insertions(+), 67 deletions(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> index cb6a37f47432..d09b06780b15 100644
> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> @@ -49,15 +49,33 @@ static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
>  {
>  	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
>  	u32 num_lanes = mipi_csi2->num_data_lanes;
> -	const struct cal_format_info *fmtinfo;
>  	u32 bpp;
>  	s64 freq;
>  
> -	fmtinfo = cal_format_by_code(phy->formats[CAL_CAMERARX_PAD_SINK].code);
> -	if (!fmtinfo)
> +	/*
> +	 * With multistream input we don't have bpp, and cannot use
> +	 * V4L2_CID_PIXEL_RATE. Passing 0 as bpp causes v4l2_get_link_freq()
> +	 * to return an error if it falls back to V4L2_CID_PIXEL_RATE.
> +	 */
> +
> +	if (phy->stream_configs.num_configs == 0)
>  		return -EINVAL;
>  
> -	bpp = fmtinfo->bpp;
> +	if (phy->stream_configs.num_configs > 2) {
> +		bpp = 0;
> +	} else {
> +		const struct cal_format_info *fmtinfo;
> +		struct v4l2_mbus_framefmt *fmt;
> +
> +		/* The first format is for the sink */
> +		fmt = &phy->stream_configs.configs[0].fmt;
> +
> +		fmtinfo = cal_format_by_code(fmt->code);
> +		if (!fmtinfo)
> +			return -EINVAL;
> +
> +		bpp = fmtinfo->bpp;
> +	}
>  
>  	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
>  	if (freq < 0) {
> @@ -619,19 +637,104 @@ static inline struct cal_camerarx *to_cal_camerarx(struct v4l2_subdev *sd)
>  	return container_of(sd, struct cal_camerarx, subdev);
>  }
>  
> -static struct v4l2_mbus_framefmt *
> -cal_camerarx_get_pad_format(struct cal_camerarx *phy,
> -			    struct v4l2_subdev_state *sd_state,
> -			    unsigned int pad, u32 which)
> -{
> -	switch (which) {
> -	case V4L2_SUBDEV_FORMAT_TRY:
> -		return v4l2_subdev_get_try_format(&phy->subdev, sd_state, pad);
> -	case V4L2_SUBDEV_FORMAT_ACTIVE:
> -		return &phy->formats[pad];
> -	default:
> +struct cal_camerarx *
> +cal_camerarx_get_phy_from_entity(struct media_entity *entity)
> +{
> +	struct v4l2_subdev *sd;
> +
> +	sd = media_entity_to_v4l2_subdev(entity);
> +	if (!sd)
>  		return NULL;
> +
> +	return to_cal_camerarx(sd);
> +}
> +
> +static struct v4l2_subdev_krouting *
> +cal_camerarx_get_routing_table(struct cal_camerarx *phy,
> +			       struct v4l2_subdev_state *sd_state, u32 which)
> +{
> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
> +		return &phy->routing;
> +	else
> +		return &sd_state->routing;
> +}
> +
> +static struct v4l2_subdev_stream_configs *
> +cal_camerarx_get_stream_configs(struct cal_camerarx *phy,
> +				struct v4l2_subdev_state *sd_state, u32 which)
> +{
> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
> +		return &phy->stream_configs;
> +	else
> +		return &sd_state->stream_configs;
> +}
> +
> +struct v4l2_mbus_framefmt *
> +cal_camerarx_get_stream_format(struct cal_camerarx *phy,
> +			       struct v4l2_subdev_state *sd_state,
> +			       unsigned int pad, u32 stream, u32 which)
> +{
> +	struct v4l2_subdev_stream_configs *stream_configs;
> +	unsigned int i;
> +
> +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
> +
> +	for (i = 0; i < stream_configs->num_configs; ++i) {
> +		if (stream_configs->configs[i].pad == pad &&
> +		    stream_configs->configs[i].stream == stream)
> +			return &stream_configs->configs[i].fmt;
> +	}
> +
> +	return NULL;
> +}
> +
> +static int cal_camerarx_find_opposite_end(struct v4l2_subdev_krouting *routing,
> +					  u32 pad, u32 stream, u32 *other_pad,
> +					  u32 *other_stream)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < routing->num_routes; ++i) {
> +		struct v4l2_subdev_route *route = &routing->routes[i];
> +
> +		if (cal_rx_pad_is_source(pad)) {
> +			if (route->source_pad == pad &&
> +			    route->source_stream == stream) {
> +				*other_pad = route->sink_pad;
> +				*other_stream = route->sink_stream;
> +				return 0;
> +			}
> +		} else {
> +			if (route->sink_pad == pad &&
> +			    route->sink_stream == stream) {
> +				*other_pad = route->source_pad;
> +				*other_stream = route->source_stream;
> +				return 0;
> +			}
> +		}
>  	}
> +
> +	return -EINVAL;
> +}
> +
> +static struct v4l2_mbus_framefmt *
> +cal_camerarx_get_opposite_stream_format(struct cal_camerarx *phy,
> +					struct v4l2_subdev_state *sd_state,
> +					u32 pad, u32 stream, u32 which)
> +{
> +	struct v4l2_subdev_krouting *routing;
> +	u32 other_pad, other_stream;
> +	int ret;
> +
> +	routing = cal_camerarx_get_routing_table(phy, sd_state, which);
> +
> +	ret = cal_camerarx_find_opposite_end(routing, pad, stream, &other_pad,
> +					     &other_stream);
> +	if (ret)
> +		return NULL;
> +
> +	return cal_camerarx_get_stream_format(phy, sd_state, other_pad,
> +					      other_stream, which);
>  }

Would it make sense to move these functions to v4l2 core to reduce 
boilerplate in drivers? Are they generic enough?

>  
>  static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
> @@ -669,9 +772,15 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
>  			goto out;
>  		}
>  
> -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> -						  CAL_CAMERARX_PAD_SINK,
> -						  code->which);
> +		fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state,
> +					code->pad, code->stream,
> +					code->which);
> +
> +		if (!fmt) {
> +			r = -EINVAL;
> +			goto out;
> +		}
> +
>  		code->code = fmt->code;
>  	} else {
>  		if (code->index >= cal_num_formats) {
> @@ -705,9 +814,14 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>  	if (cal_rx_pad_is_source(fse->pad)) {
>  		struct v4l2_mbus_framefmt *fmt;
>  
> -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> -						  CAL_CAMERARX_PAD_SINK,
> -						  fse->which);
> +		fmt = cal_camerarx_get_opposite_stream_format(
> +			phy, sd_state, fse->pad, fse->stream, fse->which);
> +
> +		if (!fmt) {
> +			r = -EINVAL;
> +			goto out;
> +		}
> +
>  		if (fse->code != fmt->code) {
>  			r = -EINVAL;
>  			goto out;
> @@ -747,8 +861,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
>  
>  	mutex_lock(&phy->mutex);
>  
> -	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
> -					  format->which);
> +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
> +					     format->stream, format->which);
> +
> +	if (!fmt) {
> +		mutex_unlock(&phy->mutex);
> +		return -EINVAL;
> +	}
> +
>  	format->format = *fmt;
>  
>  	mutex_unlock(&phy->mutex);
> @@ -764,6 +884,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  	const struct cal_format_info *fmtinfo;
>  	struct v4l2_mbus_framefmt *fmt;
>  	unsigned int bpp;
> +	int ret = 0;
>  
>  	/* No transcoding, source and sink formats must match. */

If the bridge can't transcode, does it need {get,set}_fmt ops at all? 
What does it do with this information?

For example, on the cdns-csi2rx bridge driver used on J721E, I did not 
implement these ops at all. You can simply program the hardware to let 
everything through. The only time the bridge needs to know the format is 
when it needs to convert pixel rate to link frequency, but that can be 
done by asking the source.

>  	if (cal_rx_pad_is_source(format->pad))
> @@ -792,40 +913,117 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>  	/* Store the format and propagate it to the source pad. */
>  	mutex_lock(&phy->mutex);
>  
> -	fmt = cal_camerarx_get_pad_format(phy, sd_state,
> -					  CAL_CAMERARX_PAD_SINK,
> -					  format->which);
> +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
> +					     format->stream, format->which);
> +	if (!fmt) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
>  	*fmt = format->format;
>  
> -	fmt = cal_camerarx_get_pad_format(phy, sd_state, CAL_CAMERARX_PAD_FIRST_SOURCE,
> -					  format->which);
> +	fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state, format->pad,
> +						      format->stream,
> +						      format->which);
> +	if (!fmt) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
>  	*fmt = format->format;
>  
> +out:
>  	mutex_unlock(&phy->mutex);
>  
> +	return ret;
> +}
> +
> +static int cal_camerarx_sd_get_routing(struct v4l2_subdev *sd,
> +				       struct v4l2_subdev_state *sd_state,
> +				       struct v4l2_subdev_krouting *routing)
> +{
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +	struct v4l2_subdev_krouting *src;
> +
> +	src = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
> +
> +	return v4l2_subdev_cpy_routing(routing, src);
> +}
> +
> +static void cal_camerarx_init_formats(struct v4l2_subdev *sd,
> +				      struct v4l2_subdev_state *sd_state,
> +				      u32 which)
> +{
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +
> +	static const struct v4l2_mbus_framefmt format = {
> +		.width = 640,
> +		.height = 480,
> +		.code = MEDIA_BUS_FMT_UYVY8_2X8,
> +		.field = V4L2_FIELD_NONE,
> +		.colorspace = V4L2_COLORSPACE_SRGB,
> +		.ycbcr_enc = V4L2_YCBCR_ENC_601,
> +		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> +		.xfer_func = V4L2_XFER_FUNC_SRGB,
> +	};
> +
> +	struct v4l2_subdev_stream_configs *stream_configs;
> +	unsigned int i;
> +
> +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
> +
> +	for (i = 0; i < stream_configs->num_configs; ++i)
> +		stream_configs->configs[i].fmt = format;
> +}
> +
> +static int cal_camerarx_sd_set_routing(struct v4l2_subdev *sd,
> +				       struct v4l2_subdev_state *sd_state,
> +				       struct v4l2_subdev_krouting *routing)
> +{
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +	int ret;
> +	struct v4l2_subdev_krouting *dst;
> +	struct v4l2_subdev_stream_configs *stream_configs;
> +
> +	dst = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
> +	stream_configs =
> +		cal_camerarx_get_stream_configs(phy, sd_state, routing->which);
> +
> +	ret = v4l2_subdev_dup_routing(dst, routing);
> +	if (ret)
> +		return ret;
> +
> +	ret = v4l2_init_stream_configs(stream_configs, dst);
> +	if (ret)
> +		return ret;
> +
> +	/* Initialize stream formats */
> +	cal_camerarx_init_formats(sd, sd_state, routing->which);

What if an application calls this when one or more of the contexts are 
streaming? I don't see you prevent that. Can it lead to any 
undefined/unexpected behaviour?

> +
>  	return 0;
>  }
>  
>  static int cal_camerarx_sd_init_cfg(struct v4l2_subdev *sd,
>  				    struct v4l2_subdev_state *sd_state)
>  {
> -	struct v4l2_subdev_format format = {
> -		.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY
> -		: V4L2_SUBDEV_FORMAT_ACTIVE,
> -		.pad = CAL_CAMERARX_PAD_SINK,
> -		.format = {
> -			.width = 640,
> -			.height = 480,
> -			.code = MEDIA_BUS_FMT_UYVY8_2X8,
> -			.field = V4L2_FIELD_NONE,
> -			.colorspace = V4L2_COLORSPACE_SRGB,
> -			.ycbcr_enc = V4L2_YCBCR_ENC_601,
> -			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> -			.xfer_func = V4L2_XFER_FUNC_SRGB,
> -		},
> +	u32 which = sd_state ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE;
> +
> +	struct v4l2_subdev_route routes[] = { {
> +		.sink_pad = 0,
> +		.sink_stream = 0,
> +		.source_pad = 1,
> +		.source_stream = 0,
> +		.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> +	} };
> +
> +	struct v4l2_subdev_krouting routing = {
> +		.which = which,
> +		.num_routes = 1,
> +		.routes = routes,
>  	};
>  
> -	return cal_camerarx_sd_set_fmt(sd, sd_state, &format);
> +	/* Initialize routing to single route to the fist source pad */
> +	return cal_camerarx_sd_set_routing(sd, sd_state, &routing);
>  }
>  
>  static const struct v4l2_subdev_video_ops cal_camerarx_video_ops = {
> @@ -838,6 +1036,8 @@ static const struct v4l2_subdev_pad_ops cal_camerarx_pad_ops = {
>  	.enum_frame_size = cal_camerarx_sd_enum_frame_size,
>  	.get_fmt = cal_camerarx_sd_get_fmt,
>  	.set_fmt = cal_camerarx_sd_set_fmt,
> +	.get_routing = cal_camerarx_sd_get_routing,
> +	.set_routing = cal_camerarx_sd_set_routing,
>  };
>  
>  static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
> @@ -845,8 +1045,18 @@ static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
>  	.pad = &cal_camerarx_pad_ops,
>  };
>  
> +static bool cal_camerarx_has_route(struct media_entity *entity, unsigned int pad0,
> +			  unsigned int pad1)
> +{
> +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> +
> +	return v4l2_subdev_has_route(&phy->routing, pad0, pad1);
> +}
> +
>  static struct media_entity_operations cal_camerarx_media_ops = {
>  	.link_validate = v4l2_subdev_link_validate,
> +	.has_route = cal_camerarx_has_route,
>  };
>  
>  /* ------------------------------------------------------------------
> @@ -898,11 +1108,12 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	sd = &phy->subdev;
>  	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
>  	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> -	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_MULTIPLEXED;
>  	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
>  	sd->dev = cal->dev;
>  
>  	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> +
>  	for (i = CAL_CAMERARX_PAD_FIRST_SOURCE; i < CAL_CAMERARX_NUM_PADS; ++i)
>  		phy->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>  	sd->entity.ops = &cal_camerarx_media_ops;
> @@ -922,6 +1133,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>  	return phy;
>  
>  error:
> +	v4l2_subdev_free_routing(&phy->routing);
> +	v4l2_uninit_stream_configs(&phy->stream_configs);
>  	media_entity_cleanup(&phy->subdev.entity);
>  	kfree(phy);
>  	return ERR_PTR(ret);
> @@ -933,6 +1146,8 @@ void cal_camerarx_destroy(struct cal_camerarx *phy)
>  		return;
>  
>  	v4l2_device_unregister_subdev(&phy->subdev);
> +	v4l2_subdev_free_routing(&phy->routing);
> +	v4l2_uninit_stream_configs(&phy->stream_configs);
>  	media_entity_cleanup(&phy->subdev.entity);
>  	of_node_put(phy->source_ep_node);
>  	of_node_put(phy->source_node);
> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> index 8ecae7dc2774..234af40a24fa 100644
> --- a/drivers/media/platform/ti-vpe/cal-video.c
> +++ b/drivers/media/platform/ti-vpe/cal-video.c
> @@ -693,7 +693,11 @@ static int cal_video_check_format(struct cal_ctx *ctx)
>  	if (!remote_pad)
>  		return -ENODEV;
>  
> -	format = &ctx->phy->formats[remote_pad->index];
> +	format = cal_camerarx_get_stream_format(ctx->phy, NULL,
> +						remote_pad->index, 0,
> +						V4L2_SUBDEV_FORMAT_ACTIVE);
> +	if (!format)
> +		return -EINVAL;
>  
>  	if (ctx->fmtinfo->code != format->code ||
>  	    ctx->v_fmt.fmt.pix.height != format->height ||
> @@ -711,6 +715,48 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>  	dma_addr_t addr;
>  	int ret;
>  
> +	if (cal_mc_api) {
> +		struct v4l2_subdev_route *route = NULL;
> +		struct media_pad *remote_pad;
> +		unsigned int i;
> +
> +		/* Find the PHY connected to this video device */
> +
> +		remote_pad = media_entity_remote_pad(&ctx->pad);
> +		if (!remote_pad) {
> +			ctx_err(ctx, "Context not connected\n");
> +			ret = -ENODEV;
> +			goto error_release_buffers;
> +		}
> +
> +		ctx->phy = cal_camerarx_get_phy_from_entity(remote_pad->entity);
> +
> +		/* Find the stream */
> +
> +		for (i = 0; i < ctx->phy->routing.num_routes; ++i) {
> +			struct v4l2_subdev_route *r =
> +				&ctx->phy->routing.routes[i];
> +
> +			if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> +				continue;
> +
> +			if (r->source_pad != remote_pad->index)
> +				continue;
> +
> +			route = r;
> +
> +			break;
> +		}
> +
> +		if (!route) {
> +			ctx_err(ctx, "Failed to find route\n");
> +			ret = -ENODEV;
> +			goto error_release_buffers;
> +		}

Is it possible to generalize this boilerplate so every driver does not 
have to repeat it? Do you think it is generic enough?

> +
> +		ctx->stream = route->sink_stream;
> +	}
> +

Applications lose a bit of control over the cameras here. Say you only 
want to use 1 camera and don't care about the rest right now. With the 
current implementation, you propagate the s_stream(1) call to the 
FPD-Link/GMSL/whatever serializer subdev as soon any of the contexts 
start streaming, and don't send s_stream(0) until all contexts stop 
streaming.

We have an all or nothing policy here. Either all cameras are streaming, 
or none are. Would it make sense to add a way to control individual 
cameras? How much more complexity would it add?

On J721E, when you start all cameras but capture only one, the buffer 
for pixel data overflows very quickly and all streams are affected. The 
ability to select the exact cameras to stream could be useful, though I 
am not sure how often anyone would want to do that in a real use case.

>  	ret = media_pipeline_start(ctx->vdev.entity.pads, &ctx->phy->pipe);
>  	if (ret < 0) {
>  		ctx_err(ctx, "Failed to start media pipeline: %d\n", ret);
> @@ -784,6 +830,9 @@ static void cal_stop_streaming(struct vb2_queue *vq)
>  	cal_release_buffers(ctx, VB2_BUF_STATE_ERROR);
>  
>  	media_pipeline_stop(ctx->vdev.entity.pads);
> +
> +	if (cal_mc_api)
> +		ctx->phy = NULL;
>  }
>  
>  static const struct vb2_ops cal_video_qops = {
[...]

-- 
Regards,
Pratyush Yadav
Texas Instruments Inc.

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Tomi Valkeinen]

Hi,

On 03/08/2021 13:21, Pratyush Yadav wrote:
> Hi Tomi,
> 
> Thanks for your work on this. I have used your patches to add
> multiplexed stream support on J721E. A few thoughts below that came to
> my mind when implementing it.

That's great to hear!

> On 24/05/21 02:09PM, Tomi Valkeinen wrote:
>> Add routing and stream_config support to CAL driver.
>>
>> Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
>> separate streams at the same time.
>>
>> Add 8 video device nodes, each representing a single dma-engine, and set
>> the number of source pads on camerarx to 8. Each video node can be
>> connected to any of the source pads on either of the camerarx instances
>> using media links. Camerarx internal routing is used to route the
>> incoming CSI-2 streams to one of the 8 source pads.
>>
>> CAL doesn't support transcoding, so the driver currently allows changes
>> only on the camerarx sink side, and then copies the sink pad config to
>> the source pad. This becomes slighly more complex with 8 source pads and
>> multiple streams on the sink pad. A helper,
>> cal_camerarx_get_opposite_stream_format(), is added, which uses the
>> routing table to get the format from the "opposite" side.
>>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
>>   drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
>>   drivers/media/platform/ti-vpe/cal.c          |  34 ++-
>>   drivers/media/platform/ti-vpe/cal.h          |  12 +-
>>   4 files changed, 385 insertions(+), 67 deletions(-)
>>
>> diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> index cb6a37f47432..d09b06780b15 100644
>> --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
>> +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
>> @@ -49,15 +49,33 @@ static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
>>   {
>>   	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
>>   	u32 num_lanes = mipi_csi2->num_data_lanes;
>> -	const struct cal_format_info *fmtinfo;
>>   	u32 bpp;
>>   	s64 freq;
>>   
>> -	fmtinfo = cal_format_by_code(phy->formats[CAL_CAMERARX_PAD_SINK].code);
>> -	if (!fmtinfo)
>> +	/*
>> +	 * With multistream input we don't have bpp, and cannot use
>> +	 * V4L2_CID_PIXEL_RATE. Passing 0 as bpp causes v4l2_get_link_freq()
>> +	 * to return an error if it falls back to V4L2_CID_PIXEL_RATE.
>> +	 */
>> +
>> +	if (phy->stream_configs.num_configs == 0)
>>   		return -EINVAL;
>>   
>> -	bpp = fmtinfo->bpp;
>> +	if (phy->stream_configs.num_configs > 2) {
>> +		bpp = 0;
>> +	} else {
>> +		const struct cal_format_info *fmtinfo;
>> +		struct v4l2_mbus_framefmt *fmt;
>> +
>> +		/* The first format is for the sink */
>> +		fmt = &phy->stream_configs.configs[0].fmt;
>> +
>> +		fmtinfo = cal_format_by_code(fmt->code);
>> +		if (!fmtinfo)
>> +			return -EINVAL;
>> +
>> +		bpp = fmtinfo->bpp;
>> +	}
>>   
>>   	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
>>   	if (freq < 0) {
>> @@ -619,19 +637,104 @@ static inline struct cal_camerarx *to_cal_camerarx(struct v4l2_subdev *sd)
>>   	return container_of(sd, struct cal_camerarx, subdev);
>>   }
>>   
>> -static struct v4l2_mbus_framefmt *
>> -cal_camerarx_get_pad_format(struct cal_camerarx *phy,
>> -			    struct v4l2_subdev_state *sd_state,
>> -			    unsigned int pad, u32 which)
>> -{
>> -	switch (which) {
>> -	case V4L2_SUBDEV_FORMAT_TRY:
>> -		return v4l2_subdev_get_try_format(&phy->subdev, sd_state, pad);
>> -	case V4L2_SUBDEV_FORMAT_ACTIVE:
>> -		return &phy->formats[pad];
>> -	default:
>> +struct cal_camerarx *
>> +cal_camerarx_get_phy_from_entity(struct media_entity *entity)
>> +{
>> +	struct v4l2_subdev *sd;
>> +
>> +	sd = media_entity_to_v4l2_subdev(entity);
>> +	if (!sd)
>>   		return NULL;
>> +
>> +	return to_cal_camerarx(sd);
>> +}
>> +
>> +static struct v4l2_subdev_krouting *
>> +cal_camerarx_get_routing_table(struct cal_camerarx *phy,
>> +			       struct v4l2_subdev_state *sd_state, u32 which)
>> +{
>> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
>> +		return &phy->routing;
>> +	else
>> +		return &sd_state->routing;
>> +}
>> +
>> +static struct v4l2_subdev_stream_configs *
>> +cal_camerarx_get_stream_configs(struct cal_camerarx *phy,
>> +				struct v4l2_subdev_state *sd_state, u32 which)
>> +{
>> +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
>> +		return &phy->stream_configs;
>> +	else
>> +		return &sd_state->stream_configs;
>> +}
>> +
>> +struct v4l2_mbus_framefmt *
>> +cal_camerarx_get_stream_format(struct cal_camerarx *phy,
>> +			       struct v4l2_subdev_state *sd_state,
>> +			       unsigned int pad, u32 stream, u32 which)
>> +{
>> +	struct v4l2_subdev_stream_configs *stream_configs;
>> +	unsigned int i;
>> +
>> +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
>> +
>> +	for (i = 0; i < stream_configs->num_configs; ++i) {
>> +		if (stream_configs->configs[i].pad == pad &&
>> +		    stream_configs->configs[i].stream == stream)
>> +			return &stream_configs->configs[i].fmt;
>> +	}
>> +
>> +	return NULL;
>> +}
>> +
>> +static int cal_camerarx_find_opposite_end(struct v4l2_subdev_krouting *routing,
>> +					  u32 pad, u32 stream, u32 *other_pad,
>> +					  u32 *other_stream)
>> +{
>> +	unsigned int i;
>> +
>> +	for (i = 0; i < routing->num_routes; ++i) {
>> +		struct v4l2_subdev_route *route = &routing->routes[i];
>> +
>> +		if (cal_rx_pad_is_source(pad)) {
>> +			if (route->source_pad == pad &&
>> +			    route->source_stream == stream) {
>> +				*other_pad = route->sink_pad;
>> +				*other_stream = route->sink_stream;
>> +				return 0;
>> +			}
>> +		} else {
>> +			if (route->sink_pad == pad &&
>> +			    route->sink_stream == stream) {
>> +				*other_pad = route->source_pad;
>> +				*other_stream = route->source_stream;
>> +				return 0;
>> +			}
>> +		}
>>   	}
>> +
>> +	return -EINVAL;
>> +}
>> +
>> +static struct v4l2_mbus_framefmt *
>> +cal_camerarx_get_opposite_stream_format(struct cal_camerarx *phy,
>> +					struct v4l2_subdev_state *sd_state,
>> +					u32 pad, u32 stream, u32 which)
>> +{
>> +	struct v4l2_subdev_krouting *routing;
>> +	u32 other_pad, other_stream;
>> +	int ret;
>> +
>> +	routing = cal_camerarx_get_routing_table(phy, sd_state, which);
>> +
>> +	ret = cal_camerarx_find_opposite_end(routing, pad, stream, &other_pad,
>> +					     &other_stream);
>> +	if (ret)
>> +		return NULL;
>> +
>> +	return cal_camerarx_get_stream_format(phy, sd_state, other_pad,
>> +					      other_stream, which);
>>   }
> 
> Would it make sense to move these functions to v4l2 core to reduce
> boilerplate in drivers? Are they generic enough?

Yes, I have moved all these to the core in my WIP branch.

>>   static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
>> @@ -669,9 +772,15 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
>>   			goto out;
>>   		}
>>   
>> -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
>> -						  CAL_CAMERARX_PAD_SINK,
>> -						  code->which);
>> +		fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state,
>> +					code->pad, code->stream,
>> +					code->which);
>> +
>> +		if (!fmt) {
>> +			r = -EINVAL;
>> +			goto out;
>> +		}
>> +
>>   		code->code = fmt->code;
>>   	} else {
>>   		if (code->index >= cal_num_formats) {
>> @@ -705,9 +814,14 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
>>   	if (cal_rx_pad_is_source(fse->pad)) {
>>   		struct v4l2_mbus_framefmt *fmt;
>>   
>> -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
>> -						  CAL_CAMERARX_PAD_SINK,
>> -						  fse->which);
>> +		fmt = cal_camerarx_get_opposite_stream_format(
>> +			phy, sd_state, fse->pad, fse->stream, fse->which);
>> +
>> +		if (!fmt) {
>> +			r = -EINVAL;
>> +			goto out;
>> +		}
>> +
>>   		if (fse->code != fmt->code) {
>>   			r = -EINVAL;
>>   			goto out;
>> @@ -747,8 +861,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
>>   
>>   	mutex_lock(&phy->mutex);
>>   
>> -	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
>> -					  format->which);
>> +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
>> +					     format->stream, format->which);
>> +
>> +	if (!fmt) {
>> +		mutex_unlock(&phy->mutex);
>> +		return -EINVAL;
>> +	}
>> +
>>   	format->format = *fmt;
>>   
>>   	mutex_unlock(&phy->mutex);
>> @@ -764,6 +884,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>>   	const struct cal_format_info *fmtinfo;
>>   	struct v4l2_mbus_framefmt *fmt;
>>   	unsigned int bpp;
>> +	int ret = 0;
>>   
>>   	/* No transcoding, source and sink formats must match. */
> 
> If the bridge can't transcode, does it need {get,set}_fmt ops at all?

I don't know what is the official rule here (if any). But I think the 
code that verifies the media pipeline before the start won't be able to 
verify (just skips the verification) if the subdev doesn't support get_fmt.

> What does it do with this information?

Well, CAL driver has the ops because they were there for non-multiplexed 
case too. I can't say right away if CAL could do without those for 
multiplexed use.

> For example, on the cdns-csi2rx bridge driver used on J721E, I did not
> implement these ops at all. You can simply program the hardware to let
> everything through. The only time the bridge needs to know the format is
> when it needs to convert pixel rate to link frequency, but that can be
> done by asking the source.

What happens if the source is also pass-through, and doesn't implement 
these ops? You need a function to traverse the graph and find a subdev 
that has the format.

>>   	if (cal_rx_pad_is_source(format->pad))
>> @@ -792,40 +913,117 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
>>   	/* Store the format and propagate it to the source pad. */
>>   	mutex_lock(&phy->mutex);
>>   
>> -	fmt = cal_camerarx_get_pad_format(phy, sd_state,
>> -					  CAL_CAMERARX_PAD_SINK,
>> -					  format->which);
>> +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
>> +					     format->stream, format->which);
>> +	if (!fmt) {
>> +		ret = -EINVAL;
>> +		goto out;
>> +	}
>> +
>>   	*fmt = format->format;
>>   
>> -	fmt = cal_camerarx_get_pad_format(phy, sd_state, CAL_CAMERARX_PAD_FIRST_SOURCE,
>> -					  format->which);
>> +	fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state, format->pad,
>> +						      format->stream,
>> +						      format->which);
>> +	if (!fmt) {
>> +		ret = -EINVAL;
>> +		goto out;
>> +	}
>> +
>>   	*fmt = format->format;
>>   
>> +out:
>>   	mutex_unlock(&phy->mutex);
>>   
>> +	return ret;
>> +}
>> +
>> +static int cal_camerarx_sd_get_routing(struct v4l2_subdev *sd,
>> +				       struct v4l2_subdev_state *sd_state,
>> +				       struct v4l2_subdev_krouting *routing)
>> +{
>> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
>> +	struct v4l2_subdev_krouting *src;
>> +
>> +	src = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
>> +
>> +	return v4l2_subdev_cpy_routing(routing, src);
>> +}
>> +
>> +static void cal_camerarx_init_formats(struct v4l2_subdev *sd,
>> +				      struct v4l2_subdev_state *sd_state,
>> +				      u32 which)
>> +{
>> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
>> +
>> +	static const struct v4l2_mbus_framefmt format = {
>> +		.width = 640,
>> +		.height = 480,
>> +		.code = MEDIA_BUS_FMT_UYVY8_2X8,
>> +		.field = V4L2_FIELD_NONE,
>> +		.colorspace = V4L2_COLORSPACE_SRGB,
>> +		.ycbcr_enc = V4L2_YCBCR_ENC_601,
>> +		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
>> +		.xfer_func = V4L2_XFER_FUNC_SRGB,
>> +	};
>> +
>> +	struct v4l2_subdev_stream_configs *stream_configs;
>> +	unsigned int i;
>> +
>> +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
>> +
>> +	for (i = 0; i < stream_configs->num_configs; ++i)
>> +		stream_configs->configs[i].fmt = format;
>> +}
>> +
>> +static int cal_camerarx_sd_set_routing(struct v4l2_subdev *sd,
>> +				       struct v4l2_subdev_state *sd_state,
>> +				       struct v4l2_subdev_krouting *routing)
>> +{
>> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
>> +	int ret;
>> +	struct v4l2_subdev_krouting *dst;
>> +	struct v4l2_subdev_stream_configs *stream_configs;
>> +
>> +	dst = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
>> +	stream_configs =
>> +		cal_camerarx_get_stream_configs(phy, sd_state, routing->which);
>> +
>> +	ret = v4l2_subdev_dup_routing(dst, routing);
>> +	if (ret)
>> +		return ret;
>> +
>> +	ret = v4l2_init_stream_configs(stream_configs, dst);
>> +	if (ret)
>> +		return ret;
>> +
>> +	/* Initialize stream formats */
>> +	cal_camerarx_init_formats(sd, sd_state, routing->which);
> 
> What if an application calls this when one or more of the contexts are
> streaming? I don't see you prevent that. Can it lead to any
> undefined/unexpected behaviour?

Yes, I think changing routing needs to be prevented when streaming is 
enabled.

>> +
>>   	return 0;
>>   }
>>   
>>   static int cal_camerarx_sd_init_cfg(struct v4l2_subdev *sd,
>>   				    struct v4l2_subdev_state *sd_state)
>>   {
>> -	struct v4l2_subdev_format format = {
>> -		.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY
>> -		: V4L2_SUBDEV_FORMAT_ACTIVE,
>> -		.pad = CAL_CAMERARX_PAD_SINK,
>> -		.format = {
>> -			.width = 640,
>> -			.height = 480,
>> -			.code = MEDIA_BUS_FMT_UYVY8_2X8,
>> -			.field = V4L2_FIELD_NONE,
>> -			.colorspace = V4L2_COLORSPACE_SRGB,
>> -			.ycbcr_enc = V4L2_YCBCR_ENC_601,
>> -			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
>> -			.xfer_func = V4L2_XFER_FUNC_SRGB,
>> -		},
>> +	u32 which = sd_state ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE;
>> +
>> +	struct v4l2_subdev_route routes[] = { {
>> +		.sink_pad = 0,
>> +		.sink_stream = 0,
>> +		.source_pad = 1,
>> +		.source_stream = 0,
>> +		.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
>> +	} };
>> +
>> +	struct v4l2_subdev_krouting routing = {
>> +		.which = which,
>> +		.num_routes = 1,
>> +		.routes = routes,
>>   	};
>>   
>> -	return cal_camerarx_sd_set_fmt(sd, sd_state, &format);
>> +	/* Initialize routing to single route to the fist source pad */
>> +	return cal_camerarx_sd_set_routing(sd, sd_state, &routing);
>>   }
>>   
>>   static const struct v4l2_subdev_video_ops cal_camerarx_video_ops = {
>> @@ -838,6 +1036,8 @@ static const struct v4l2_subdev_pad_ops cal_camerarx_pad_ops = {
>>   	.enum_frame_size = cal_camerarx_sd_enum_frame_size,
>>   	.get_fmt = cal_camerarx_sd_get_fmt,
>>   	.set_fmt = cal_camerarx_sd_set_fmt,
>> +	.get_routing = cal_camerarx_sd_get_routing,
>> +	.set_routing = cal_camerarx_sd_set_routing,
>>   };
>>   
>>   static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
>> @@ -845,8 +1045,18 @@ static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
>>   	.pad = &cal_camerarx_pad_ops,
>>   };
>>   
>> +static bool cal_camerarx_has_route(struct media_entity *entity, unsigned int pad0,
>> +			  unsigned int pad1)
>> +{
>> +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
>> +	struct cal_camerarx *phy = to_cal_camerarx(sd);
>> +
>> +	return v4l2_subdev_has_route(&phy->routing, pad0, pad1);
>> +}
>> +
>>   static struct media_entity_operations cal_camerarx_media_ops = {
>>   	.link_validate = v4l2_subdev_link_validate,
>> +	.has_route = cal_camerarx_has_route,
>>   };
>>   
>>   /* ------------------------------------------------------------------
>> @@ -898,11 +1108,12 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>>   	sd = &phy->subdev;
>>   	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
>>   	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
>> -	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
>> +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_MULTIPLEXED;
>>   	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
>>   	sd->dev = cal->dev;
>>   
>>   	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
>> +
>>   	for (i = CAL_CAMERARX_PAD_FIRST_SOURCE; i < CAL_CAMERARX_NUM_PADS; ++i)
>>   		phy->pads[i].flags = MEDIA_PAD_FL_SOURCE;
>>   	sd->entity.ops = &cal_camerarx_media_ops;
>> @@ -922,6 +1133,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
>>   	return phy;
>>   
>>   error:
>> +	v4l2_subdev_free_routing(&phy->routing);
>> +	v4l2_uninit_stream_configs(&phy->stream_configs);
>>   	media_entity_cleanup(&phy->subdev.entity);
>>   	kfree(phy);
>>   	return ERR_PTR(ret);
>> @@ -933,6 +1146,8 @@ void cal_camerarx_destroy(struct cal_camerarx *phy)
>>   		return;
>>   
>>   	v4l2_device_unregister_subdev(&phy->subdev);
>> +	v4l2_subdev_free_routing(&phy->routing);
>> +	v4l2_uninit_stream_configs(&phy->stream_configs);
>>   	media_entity_cleanup(&phy->subdev.entity);
>>   	of_node_put(phy->source_ep_node);
>>   	of_node_put(phy->source_node);
>> diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
>> index 8ecae7dc2774..234af40a24fa 100644
>> --- a/drivers/media/platform/ti-vpe/cal-video.c
>> +++ b/drivers/media/platform/ti-vpe/cal-video.c
>> @@ -693,7 +693,11 @@ static int cal_video_check_format(struct cal_ctx *ctx)
>>   	if (!remote_pad)
>>   		return -ENODEV;
>>   
>> -	format = &ctx->phy->formats[remote_pad->index];
>> +	format = cal_camerarx_get_stream_format(ctx->phy, NULL,
>> +						remote_pad->index, 0,
>> +						V4L2_SUBDEV_FORMAT_ACTIVE);
>> +	if (!format)
>> +		return -EINVAL;
>>   
>>   	if (ctx->fmtinfo->code != format->code ||
>>   	    ctx->v_fmt.fmt.pix.height != format->height ||
>> @@ -711,6 +715,48 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
>>   	dma_addr_t addr;
>>   	int ret;
>>   
>> +	if (cal_mc_api) {
>> +		struct v4l2_subdev_route *route = NULL;
>> +		struct media_pad *remote_pad;
>> +		unsigned int i;
>> +
>> +		/* Find the PHY connected to this video device */
>> +
>> +		remote_pad = media_entity_remote_pad(&ctx->pad);
>> +		if (!remote_pad) {
>> +			ctx_err(ctx, "Context not connected\n");
>> +			ret = -ENODEV;
>> +			goto error_release_buffers;
>> +		}
>> +
>> +		ctx->phy = cal_camerarx_get_phy_from_entity(remote_pad->entity);
>> +
>> +		/* Find the stream */
>> +
>> +		for (i = 0; i < ctx->phy->routing.num_routes; ++i) {
>> +			struct v4l2_subdev_route *r =
>> +				&ctx->phy->routing.routes[i];
>> +
>> +			if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
>> +				continue;
>> +
>> +			if (r->source_pad != remote_pad->index)
>> +				continue;
>> +
>> +			route = r;
>> +
>> +			break;
>> +		}
>> +
>> +		if (!route) {
>> +			ctx_err(ctx, "Failed to find route\n");
>> +			ret = -ENODEV;
>> +			goto error_release_buffers;
>> +		}
> 
> Is it possible to generalize this boilerplate so every driver does not
> have to repeat it? Do you think it is generic enough?

Hmm, what's the boilerplate here? Isn't the above quite cal specific? 
How does the J7 code look like?

> 
>> +
>> +		ctx->stream = route->sink_stream;
>> +	}
>> +
> 
> Applications lose a bit of control over the cameras here. Say you only
> want to use 1 camera and don't care about the rest right now. With the
> current implementation, you propagate the s_stream(1) call to the
> FPD-Link/GMSL/whatever serializer subdev as soon any of the contexts
> start streaming, and don't send s_stream(0) until all contexts stop
> streaming.
> 
> We have an all or nothing policy here. Either all cameras are streaming,
> or none are. Would it make sense to add a way to control individual
> cameras? How much more complexity would it add?

It would make sense, but I haven't seriously considered it because it's 
just an optimization afaics. Probably adding stream based 
start/stop-streaming ops would do it, but if I have realized one thing 
with this work it is that nothing is simple here =).

> On J721E, when you start all cameras but capture only one, the buffer
> for pixel data overflows very quickly and all streams are affected. The
> ability to select the exact cameras to stream could be useful, though I
> am not sure how often anyone would want to do that in a real use case.

This sounds a bit odd. The CSI-2 RX hardware has rx buffers that will be 
filled no matter what the SW says?

We may also have a piece of HW that always sends multiple streams. What 
if you attach a sensor that always sends pixel data and metadata. Does 
that mean that the SW has to capture both, otherwise the buffers will 
overflow?

  Tomi

Re: [PATCH v3 38/38] media: ti-vpe: cal: add multiplexed streams support

[Pratyush Yadav]

On 03/08/21 05:51PM, Tomi Valkeinen wrote:
> Hi,
> 
> On 03/08/2021 13:21, Pratyush Yadav wrote:
> > Hi Tomi,
> > 
> > Thanks for your work on this. I have used your patches to add
> > multiplexed stream support on J721E. A few thoughts below that came to
> > my mind when implementing it.
> 
> That's great to hear!
> 
> > On 24/05/21 02:09PM, Tomi Valkeinen wrote:
> > > Add routing and stream_config support to CAL driver.
> > > 
> > > Add multiplexed streams support. CAL has 8 dma-engines and can capture 8
> > > separate streams at the same time.
> > > 
> > > Add 8 video device nodes, each representing a single dma-engine, and set
> > > the number of source pads on camerarx to 8. Each video node can be
> > > connected to any of the source pads on either of the camerarx instances
> > > using media links. Camerarx internal routing is used to route the
> > > incoming CSI-2 streams to one of the 8 source pads.
> > > 
> > > CAL doesn't support transcoding, so the driver currently allows changes
> > > only on the camerarx sink side, and then copies the sink pad config to
> > > the source pad. This becomes slighly more complex with 8 source pads and
> > > multiple streams on the sink pad. A helper,
> > > cal_camerarx_get_opposite_stream_format(), is added, which uses the
> > > routing table to get the format from the "opposite" side.
> > > 
> > > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> > > ---
> > >   drivers/media/platform/ti-vpe/cal-camerarx.c | 303 ++++++++++++++++---
> > >   drivers/media/platform/ti-vpe/cal-video.c    | 103 ++++++-
> > >   drivers/media/platform/ti-vpe/cal.c          |  34 ++-
> > >   drivers/media/platform/ti-vpe/cal.h          |  12 +-
> > >   4 files changed, 385 insertions(+), 67 deletions(-)
> > > 
> > > diff --git a/drivers/media/platform/ti-vpe/cal-camerarx.c b/drivers/media/platform/ti-vpe/cal-camerarx.c
> > > index cb6a37f47432..d09b06780b15 100644
> > > --- a/drivers/media/platform/ti-vpe/cal-camerarx.c
> > > +++ b/drivers/media/platform/ti-vpe/cal-camerarx.c
> > > @@ -49,15 +49,33 @@ static s64 cal_camerarx_get_ext_link_freq(struct cal_camerarx *phy)
> > >   {
> > >   	struct v4l2_fwnode_bus_mipi_csi2 *mipi_csi2 = &phy->endpoint.bus.mipi_csi2;
> > >   	u32 num_lanes = mipi_csi2->num_data_lanes;
> > > -	const struct cal_format_info *fmtinfo;
> > >   	u32 bpp;
> > >   	s64 freq;
> > > -	fmtinfo = cal_format_by_code(phy->formats[CAL_CAMERARX_PAD_SINK].code);
> > > -	if (!fmtinfo)
> > > +	/*
> > > +	 * With multistream input we don't have bpp, and cannot use
> > > +	 * V4L2_CID_PIXEL_RATE. Passing 0 as bpp causes v4l2_get_link_freq()
> > > +	 * to return an error if it falls back to V4L2_CID_PIXEL_RATE.
> > > +	 */
> > > +
> > > +	if (phy->stream_configs.num_configs == 0)
> > >   		return -EINVAL;
> > > -	bpp = fmtinfo->bpp;
> > > +	if (phy->stream_configs.num_configs > 2) {
> > > +		bpp = 0;
> > > +	} else {
> > > +		const struct cal_format_info *fmtinfo;
> > > +		struct v4l2_mbus_framefmt *fmt;
> > > +
> > > +		/* The first format is for the sink */
> > > +		fmt = &phy->stream_configs.configs[0].fmt;
> > > +
> > > +		fmtinfo = cal_format_by_code(fmt->code);
> > > +		if (!fmtinfo)
> > > +			return -EINVAL;
> > > +
> > > +		bpp = fmtinfo->bpp;
> > > +	}
> > >   	freq = v4l2_get_link_freq(phy->source->ctrl_handler, bpp, 2 * num_lanes);
> > >   	if (freq < 0) {
> > > @@ -619,19 +637,104 @@ static inline struct cal_camerarx *to_cal_camerarx(struct v4l2_subdev *sd)
> > >   	return container_of(sd, struct cal_camerarx, subdev);
> > >   }
> > > -static struct v4l2_mbus_framefmt *
> > > -cal_camerarx_get_pad_format(struct cal_camerarx *phy,
> > > -			    struct v4l2_subdev_state *sd_state,
> > > -			    unsigned int pad, u32 which)
> > > -{
> > > -	switch (which) {
> > > -	case V4L2_SUBDEV_FORMAT_TRY:
> > > -		return v4l2_subdev_get_try_format(&phy->subdev, sd_state, pad);
> > > -	case V4L2_SUBDEV_FORMAT_ACTIVE:
> > > -		return &phy->formats[pad];
> > > -	default:
> > > +struct cal_camerarx *
> > > +cal_camerarx_get_phy_from_entity(struct media_entity *entity)
> > > +{
> > > +	struct v4l2_subdev *sd;
> > > +
> > > +	sd = media_entity_to_v4l2_subdev(entity);
> > > +	if (!sd)
> > >   		return NULL;
> > > +
> > > +	return to_cal_camerarx(sd);
> > > +}
> > > +
> > > +static struct v4l2_subdev_krouting *
> > > +cal_camerarx_get_routing_table(struct cal_camerarx *phy,
> > > +			       struct v4l2_subdev_state *sd_state, u32 which)
> > > +{
> > > +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
> > > +		return &phy->routing;
> > > +	else
> > > +		return &sd_state->routing;
> > > +}
> > > +
> > > +static struct v4l2_subdev_stream_configs *
> > > +cal_camerarx_get_stream_configs(struct cal_camerarx *phy,
> > > +				struct v4l2_subdev_state *sd_state, u32 which)
> > > +{
> > > +	if (which == V4L2_SUBDEV_FORMAT_ACTIVE)
> > > +		return &phy->stream_configs;
> > > +	else
> > > +		return &sd_state->stream_configs;
> > > +}
> > > +
> > > +struct v4l2_mbus_framefmt *
> > > +cal_camerarx_get_stream_format(struct cal_camerarx *phy,
> > > +			       struct v4l2_subdev_state *sd_state,
> > > +			       unsigned int pad, u32 stream, u32 which)
> > > +{
> > > +	struct v4l2_subdev_stream_configs *stream_configs;
> > > +	unsigned int i;
> > > +
> > > +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
> > > +
> > > +	for (i = 0; i < stream_configs->num_configs; ++i) {
> > > +		if (stream_configs->configs[i].pad == pad &&
> > > +		    stream_configs->configs[i].stream == stream)
> > > +			return &stream_configs->configs[i].fmt;
> > > +	}
> > > +
> > > +	return NULL;
> > > +}
> > > +
> > > +static int cal_camerarx_find_opposite_end(struct v4l2_subdev_krouting *routing,
> > > +					  u32 pad, u32 stream, u32 *other_pad,
> > > +					  u32 *other_stream)
> > > +{
> > > +	unsigned int i;
> > > +
> > > +	for (i = 0; i < routing->num_routes; ++i) {
> > > +		struct v4l2_subdev_route *route = &routing->routes[i];
> > > +
> > > +		if (cal_rx_pad_is_source(pad)) {
> > > +			if (route->source_pad == pad &&
> > > +			    route->source_stream == stream) {
> > > +				*other_pad = route->sink_pad;
> > > +				*other_stream = route->sink_stream;
> > > +				return 0;
> > > +			}
> > > +		} else {
> > > +			if (route->sink_pad == pad &&
> > > +			    route->sink_stream == stream) {
> > > +				*other_pad = route->source_pad;
> > > +				*other_stream = route->source_stream;
> > > +				return 0;
> > > +			}
> > > +		}
> > >   	}
> > > +
> > > +	return -EINVAL;
> > > +}
> > > +
> > > +static struct v4l2_mbus_framefmt *
> > > +cal_camerarx_get_opposite_stream_format(struct cal_camerarx *phy,
> > > +					struct v4l2_subdev_state *sd_state,
> > > +					u32 pad, u32 stream, u32 which)
> > > +{
> > > +	struct v4l2_subdev_krouting *routing;
> > > +	u32 other_pad, other_stream;
> > > +	int ret;
> > > +
> > > +	routing = cal_camerarx_get_routing_table(phy, sd_state, which);
> > > +
> > > +	ret = cal_camerarx_find_opposite_end(routing, pad, stream, &other_pad,
> > > +					     &other_stream);
> > > +	if (ret)
> > > +		return NULL;
> > > +
> > > +	return cal_camerarx_get_stream_format(phy, sd_state, other_pad,
> > > +					      other_stream, which);
> > >   }
> > 
> > Would it make sense to move these functions to v4l2 core to reduce
> > boilerplate in drivers? Are they generic enough?
> 
> Yes, I have moved all these to the core in my WIP branch.

Great!

> 
> > >   static int cal_camerarx_sd_s_stream(struct v4l2_subdev *sd, int enable)
> > > @@ -669,9 +772,15 @@ static int cal_camerarx_sd_enum_mbus_code(struct v4l2_subdev *sd,
> > >   			goto out;
> > >   		}
> > > -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> > > -						  CAL_CAMERARX_PAD_SINK,
> > > -						  code->which);
> > > +		fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state,
> > > +					code->pad, code->stream,
> > > +					code->which);
> > > +
> > > +		if (!fmt) {
> > > +			r = -EINVAL;
> > > +			goto out;
> > > +		}
> > > +
> > >   		code->code = fmt->code;
> > >   	} else {
> > >   		if (code->index >= cal_num_formats) {
> > > @@ -705,9 +814,14 @@ static int cal_camerarx_sd_enum_frame_size(struct v4l2_subdev *sd,
> > >   	if (cal_rx_pad_is_source(fse->pad)) {
> > >   		struct v4l2_mbus_framefmt *fmt;
> > > -		fmt = cal_camerarx_get_pad_format(phy, sd_state,
> > > -						  CAL_CAMERARX_PAD_SINK,
> > > -						  fse->which);
> > > +		fmt = cal_camerarx_get_opposite_stream_format(
> > > +			phy, sd_state, fse->pad, fse->stream, fse->which);
> > > +
> > > +		if (!fmt) {
> > > +			r = -EINVAL;
> > > +			goto out;
> > > +		}
> > > +
> > >   		if (fse->code != fmt->code) {
> > >   			r = -EINVAL;
> > >   			goto out;
> > > @@ -747,8 +861,14 @@ static int cal_camerarx_sd_get_fmt(struct v4l2_subdev *sd,
> > >   	mutex_lock(&phy->mutex);
> > > -	fmt = cal_camerarx_get_pad_format(phy, sd_state, format->pad,
> > > -					  format->which);
> > > +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
> > > +					     format->stream, format->which);
> > > +
> > > +	if (!fmt) {
> > > +		mutex_unlock(&phy->mutex);
> > > +		return -EINVAL;
> > > +	}
> > > +
> > >   	format->format = *fmt;
> > >   	mutex_unlock(&phy->mutex);
> > > @@ -764,6 +884,7 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
> > >   	const struct cal_format_info *fmtinfo;
> > >   	struct v4l2_mbus_framefmt *fmt;
> > >   	unsigned int bpp;
> > > +	int ret = 0;
> > >   	/* No transcoding, source and sink formats must match. */
> > 
> > If the bridge can't transcode, does it need {get,set}_fmt ops at all?
> 
> I don't know what is the official rule here (if any). But I think the code
> that verifies the media pipeline before the start won't be able to verify
> (just skips the verification) if the subdev doesn't support get_fmt.
> 
> > What does it do with this information?
> 
> Well, CAL driver has the ops because they were there for non-multiplexed
> case too. I can't say right away if CAL could do without those for
> multiplexed use.
> 
> > For example, on the cdns-csi2rx bridge driver used on J721E, I did not
> > implement these ops at all. You can simply program the hardware to let
> > everything through. The only time the bridge needs to know the format is
> > when it needs to convert pixel rate to link frequency, but that can be
> > done by asking the source.
> 
> What happens if the source is also pass-through, and doesn't implement these
> ops? You need a function to traverse the graph and find a subdev that has
> the format.

Yes, that is what the code on J7 does. See ti_csi2rx_validate_pipeline() 
in [0].

> 
> > >   	if (cal_rx_pad_is_source(format->pad))
> > > @@ -792,40 +913,117 @@ static int cal_camerarx_sd_set_fmt(struct v4l2_subdev *sd,
> > >   	/* Store the format and propagate it to the source pad. */
> > >   	mutex_lock(&phy->mutex);
> > > -	fmt = cal_camerarx_get_pad_format(phy, sd_state,
> > > -					  CAL_CAMERARX_PAD_SINK,
> > > -					  format->which);
> > > +	fmt = cal_camerarx_get_stream_format(phy, sd_state, format->pad,
> > > +					     format->stream, format->which);
> > > +	if (!fmt) {
> > > +		ret = -EINVAL;
> > > +		goto out;
> > > +	}
> > > +
> > >   	*fmt = format->format;
> > > -	fmt = cal_camerarx_get_pad_format(phy, sd_state, CAL_CAMERARX_PAD_FIRST_SOURCE,
> > > -					  format->which);
> > > +	fmt = cal_camerarx_get_opposite_stream_format(phy, sd_state, format->pad,
> > > +						      format->stream,
> > > +						      format->which);
> > > +	if (!fmt) {
> > > +		ret = -EINVAL;
> > > +		goto out;
> > > +	}
> > > +
> > >   	*fmt = format->format;
> > > +out:
> > >   	mutex_unlock(&phy->mutex);
> > > +	return ret;
> > > +}
> > > +
> > > +static int cal_camerarx_sd_get_routing(struct v4l2_subdev *sd,
> > > +				       struct v4l2_subdev_state *sd_state,
> > > +				       struct v4l2_subdev_krouting *routing)
> > > +{
> > > +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> > > +	struct v4l2_subdev_krouting *src;
> > > +
> > > +	src = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
> > > +
> > > +	return v4l2_subdev_cpy_routing(routing, src);
> > > +}
> > > +
> > > +static void cal_camerarx_init_formats(struct v4l2_subdev *sd,
> > > +				      struct v4l2_subdev_state *sd_state,
> > > +				      u32 which)
> > > +{
> > > +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> > > +
> > > +	static const struct v4l2_mbus_framefmt format = {
> > > +		.width = 640,
> > > +		.height = 480,
> > > +		.code = MEDIA_BUS_FMT_UYVY8_2X8,
> > > +		.field = V4L2_FIELD_NONE,
> > > +		.colorspace = V4L2_COLORSPACE_SRGB,
> > > +		.ycbcr_enc = V4L2_YCBCR_ENC_601,
> > > +		.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> > > +		.xfer_func = V4L2_XFER_FUNC_SRGB,
> > > +	};
> > > +
> > > +	struct v4l2_subdev_stream_configs *stream_configs;
> > > +	unsigned int i;
> > > +
> > > +	stream_configs = cal_camerarx_get_stream_configs(phy, sd_state, which);
> > > +
> > > +	for (i = 0; i < stream_configs->num_configs; ++i)
> > > +		stream_configs->configs[i].fmt = format;
> > > +}
> > > +
> > > +static int cal_camerarx_sd_set_routing(struct v4l2_subdev *sd,
> > > +				       struct v4l2_subdev_state *sd_state,
> > > +				       struct v4l2_subdev_krouting *routing)
> > > +{
> > > +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> > > +	int ret;
> > > +	struct v4l2_subdev_krouting *dst;
> > > +	struct v4l2_subdev_stream_configs *stream_configs;
> > > +
> > > +	dst = cal_camerarx_get_routing_table(phy, sd_state, routing->which);
> > > +	stream_configs =
> > > +		cal_camerarx_get_stream_configs(phy, sd_state, routing->which);
> > > +
> > > +	ret = v4l2_subdev_dup_routing(dst, routing);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	ret = v4l2_init_stream_configs(stream_configs, dst);
> > > +	if (ret)
> > > +		return ret;
> > > +
> > > +	/* Initialize stream formats */
> > > +	cal_camerarx_init_formats(sd, sd_state, routing->which);
> > 
> > What if an application calls this when one or more of the contexts are
> > streaming? I don't see you prevent that. Can it lead to any
> > undefined/unexpected behaviour?
> 
> Yes, I think changing routing needs to be prevented when streaming is
> enabled.

Ok. I think checking for phy->enable_count should do it.

> 
> > > +
> > >   	return 0;
> > >   }
> > >   static int cal_camerarx_sd_init_cfg(struct v4l2_subdev *sd,
> > >   				    struct v4l2_subdev_state *sd_state)
> > >   {
> > > -	struct v4l2_subdev_format format = {
> > > -		.which = sd_state ? V4L2_SUBDEV_FORMAT_TRY
> > > -		: V4L2_SUBDEV_FORMAT_ACTIVE,
> > > -		.pad = CAL_CAMERARX_PAD_SINK,
> > > -		.format = {
> > > -			.width = 640,
> > > -			.height = 480,
> > > -			.code = MEDIA_BUS_FMT_UYVY8_2X8,
> > > -			.field = V4L2_FIELD_NONE,
> > > -			.colorspace = V4L2_COLORSPACE_SRGB,
> > > -			.ycbcr_enc = V4L2_YCBCR_ENC_601,
> > > -			.quantization = V4L2_QUANTIZATION_LIM_RANGE,
> > > -			.xfer_func = V4L2_XFER_FUNC_SRGB,
> > > -		},
> > > +	u32 which = sd_state ? V4L2_SUBDEV_FORMAT_TRY : V4L2_SUBDEV_FORMAT_ACTIVE;
> > > +
> > > +	struct v4l2_subdev_route routes[] = { {
> > > +		.sink_pad = 0,
> > > +		.sink_stream = 0,
> > > +		.source_pad = 1,
> > > +		.source_stream = 0,
> > > +		.flags = V4L2_SUBDEV_ROUTE_FL_ACTIVE,
> > > +	} };
> > > +
> > > +	struct v4l2_subdev_krouting routing = {
> > > +		.which = which,
> > > +		.num_routes = 1,
> > > +		.routes = routes,
> > >   	};
> > > -	return cal_camerarx_sd_set_fmt(sd, sd_state, &format);
> > > +	/* Initialize routing to single route to the fist source pad */
> > > +	return cal_camerarx_sd_set_routing(sd, sd_state, &routing);
> > >   }
> > >   static const struct v4l2_subdev_video_ops cal_camerarx_video_ops = {
> > > @@ -838,6 +1036,8 @@ static const struct v4l2_subdev_pad_ops cal_camerarx_pad_ops = {
> > >   	.enum_frame_size = cal_camerarx_sd_enum_frame_size,
> > >   	.get_fmt = cal_camerarx_sd_get_fmt,
> > >   	.set_fmt = cal_camerarx_sd_set_fmt,
> > > +	.get_routing = cal_camerarx_sd_get_routing,
> > > +	.set_routing = cal_camerarx_sd_set_routing,
> > >   };
> > >   static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
> > > @@ -845,8 +1045,18 @@ static const struct v4l2_subdev_ops cal_camerarx_subdev_ops = {
> > >   	.pad = &cal_camerarx_pad_ops,
> > >   };
> > > +static bool cal_camerarx_has_route(struct media_entity *entity, unsigned int pad0,
> > > +			  unsigned int pad1)
> > > +{
> > > +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> > > +	struct cal_camerarx *phy = to_cal_camerarx(sd);
> > > +
> > > +	return v4l2_subdev_has_route(&phy->routing, pad0, pad1);
> > > +}
> > > +
> > >   static struct media_entity_operations cal_camerarx_media_ops = {
> > >   	.link_validate = v4l2_subdev_link_validate,
> > > +	.has_route = cal_camerarx_has_route,
> > >   };
> > >   /* ------------------------------------------------------------------
> > > @@ -898,11 +1108,12 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
> > >   	sd = &phy->subdev;
> > >   	v4l2_subdev_init(sd, &cal_camerarx_subdev_ops);
> > >   	sd->entity.function = MEDIA_ENT_F_VID_IF_BRIDGE;
> > > -	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
> > > +	sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_MULTIPLEXED;
> > >   	snprintf(sd->name, sizeof(sd->name), "CAMERARX%u", instance);
> > >   	sd->dev = cal->dev;
> > >   	phy->pads[CAL_CAMERARX_PAD_SINK].flags = MEDIA_PAD_FL_SINK;
> > > +
> > >   	for (i = CAL_CAMERARX_PAD_FIRST_SOURCE; i < CAL_CAMERARX_NUM_PADS; ++i)
> > >   		phy->pads[i].flags = MEDIA_PAD_FL_SOURCE;
> > >   	sd->entity.ops = &cal_camerarx_media_ops;
> > > @@ -922,6 +1133,8 @@ struct cal_camerarx *cal_camerarx_create(struct cal_dev *cal,
> > >   	return phy;
> > >   error:
> > > +	v4l2_subdev_free_routing(&phy->routing);
> > > +	v4l2_uninit_stream_configs(&phy->stream_configs);
> > >   	media_entity_cleanup(&phy->subdev.entity);
> > >   	kfree(phy);
> > >   	return ERR_PTR(ret);
> > > @@ -933,6 +1146,8 @@ void cal_camerarx_destroy(struct cal_camerarx *phy)
> > >   		return;
> > >   	v4l2_device_unregister_subdev(&phy->subdev);
> > > +	v4l2_subdev_free_routing(&phy->routing);
> > > +	v4l2_uninit_stream_configs(&phy->stream_configs);
> > >   	media_entity_cleanup(&phy->subdev.entity);
> > >   	of_node_put(phy->source_ep_node);
> > >   	of_node_put(phy->source_node);
> > > diff --git a/drivers/media/platform/ti-vpe/cal-video.c b/drivers/media/platform/ti-vpe/cal-video.c
> > > index 8ecae7dc2774..234af40a24fa 100644
> > > --- a/drivers/media/platform/ti-vpe/cal-video.c
> > > +++ b/drivers/media/platform/ti-vpe/cal-video.c
> > > @@ -693,7 +693,11 @@ static int cal_video_check_format(struct cal_ctx *ctx)
> > >   	if (!remote_pad)
> > >   		return -ENODEV;
> > > -	format = &ctx->phy->formats[remote_pad->index];
> > > +	format = cal_camerarx_get_stream_format(ctx->phy, NULL,
> > > +						remote_pad->index, 0,
> > > +						V4L2_SUBDEV_FORMAT_ACTIVE);
> > > +	if (!format)
> > > +		return -EINVAL;
> > >   	if (ctx->fmtinfo->code != format->code ||
> > >   	    ctx->v_fmt.fmt.pix.height != format->height ||
> > > @@ -711,6 +715,48 @@ static int cal_start_streaming(struct vb2_queue *vq, unsigned int count)
> > >   	dma_addr_t addr;
> > >   	int ret;
> > > +	if (cal_mc_api) {
> > > +		struct v4l2_subdev_route *route = NULL;
> > > +		struct media_pad *remote_pad;
> > > +		unsigned int i;
> > > +
> > > +		/* Find the PHY connected to this video device */
> > > +
> > > +		remote_pad = media_entity_remote_pad(&ctx->pad);
> > > +		if (!remote_pad) {
> > > +			ctx_err(ctx, "Context not connected\n");
> > > +			ret = -ENODEV;
> > > +			goto error_release_buffers;
> > > +		}
> > > +
> > > +		ctx->phy = cal_camerarx_get_phy_from_entity(remote_pad->entity);
> > > +
> > > +		/* Find the stream */
> > > +
> > > +		for (i = 0; i < ctx->phy->routing.num_routes; ++i) {
> > > +			struct v4l2_subdev_route *r =
> > > +				&ctx->phy->routing.routes[i];
> > > +
> > > +			if (!(r->flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))
> > > +				continue;
> > > +
> > > +			if (r->source_pad != remote_pad->index)
> > > +				continue;
> > > +
> > > +			route = r;
> > > +
> > > +			break;
> > > +		}
> > > +
> > > +		if (!route) {
> > > +			ctx_err(ctx, "Failed to find route\n");
> > > +			ret = -ENODEV;
> > > +			goto error_release_buffers;
> > > +		}
> > 
> > Is it possible to generalize this boilerplate so every driver does not
> > have to repeat it? Do you think it is generic enough?
> 
> Hmm, what's the boilerplate here? Isn't the above quite cal specific? How
> does the J7 code look like?

The boilerplate is finding the route corresponding to the context. The 
code on J7 looks exactly the same. This is not a very big piece of code 
so I think it should be fine either way. Just wanted to bring this to 
attention.

> 
> > 
> > > +
> > > +		ctx->stream = route->sink_stream;
> > > +	}
> > > +
> > 
> > Applications lose a bit of control over the cameras here. Say you only
> > want to use 1 camera and don't care about the rest right now. With the
> > current implementation, you propagate the s_stream(1) call to the
> > FPD-Link/GMSL/whatever serializer subdev as soon any of the contexts
> > start streaming, and don't send s_stream(0) until all contexts stop
> > streaming.
> > 
> > We have an all or nothing policy here. Either all cameras are streaming,
> > or none are. Would it make sense to add a way to control individual
> > cameras? How much more complexity would it add?
> 
> It would make sense, but I haven't seriously considered it because it's just
> an optimization afaics. Probably adding stream based start/stop-streaming
> ops would do it, but if I have realized one thing with this work it is that
> nothing is simple here =).

Indeed! I fear writing drivers for multistream capable hardware would 
get a bit complicated, and hence more error prone. In fact, I found that 
writing a single stream capable driver isn't that simple either. V4L2 is 
a big subsystem with little documentation aimed at newbies.

> 
> > On J721E, when you start all cameras but capture only one, the buffer
> > for pixel data overflows very quickly and all streams are affected. The
> > ability to select the exact cameras to stream could be useful, though I
> > am not sure how often anyone would want to do that in a real use case.
> 
> This sounds a bit odd. The CSI-2 RX hardware has rx buffers that will be
> filled no matter what the SW says?

That's what it looks like to me. I think all the data goes into a common 
buffer/FIFO and then the different DMA contexts can extract the data 
they want based on virtual channel or data type.

This is my hypothesis seeing the results from my experiments. The TRM 
does not mention how exactly this works.

> 
> We may also have a piece of HW that always sends multiple streams. What if
> you attach a sensor that always sends pixel data and metadata. Does that
> mean that the SW has to capture both, otherwise the buffers will overflow?

I think it does need to capture both. Let me see if I can find some more 
details about this.

> 
>  Tomi

[0] https://patchwork.linuxtv.org/project/linux-media/patch/20210624192200.22559-10-p.yadav@ti.com/

-- 
Regards,
Pratyush Yadav
Texas Instruments Inc.