[PATCH v5 00/10] media: subdev: Improve stream enable/disable machinery

[PATCH v5 00/10] media: subdev: Improve stream enable/disable machinery

[Tomi Valkeinen]

This series works on the .s_stream, .enable_streams, .disable_streams
related code. The main feature introduced here, in the last two patchs,
is adding support for .enable/disable_streams() for subdevs that do not
implement full streams support.

Additionally we add support for the privacy led when
v4l2_subdev_enable/disable_streams() is used.

I have tested this on RPi5.

 Tomi

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
Changes in v5:
- Fix issues with non-routing case:
  - Always set 'found_streams' variable instead of leaving it
    uninitialized in "Support single-stream case in
    v4l2_subdev_enable/disable_streams()"
  - Fix the "implicit" stream from bit 1 to bit 0 in "Support
    non-routing subdevs in v4l2_subdev_s_stream_helper()"
- Link to v4: https://lore.kernel.org/r/20240416-enable-streams-impro-v4-0-1d370c9c2b6d@ideasonboard.com

Changes in v4:
- Added Rb tags
- Rename 'streaming_enabled' to 's_stream_enabled'
- Cosmetic changes (comments / patch descs)
- Added new patch "media: subdev: Support non-routing subdevs in  v4l2_subdev_s_stream_helper()".
- Link to v3: https://lore.kernel.org/r/20240410-enable-streams-impro-v3-0-e5e7a5da7420@ideasonboard.com

Changes in v3:
- Rebased on top of "[PATCH v2 1/1] media: v4l: Don't turn on privacy LED if streamon fails"
- Drop extra "!!" in "media: subdev: Fix use of sd->enabled_streams in  call_s_stream()"
- Enable privacy LED after a succesfull stream enable in  "media: subdev: Support privacy led in v4l2_subdev_enable/disable_streams()"
- Init 'cfg' variable when declaring in "media: subdev: Refactor v4l2_subdev_enable/disable_streams()"
- Link to v2: https://lore.kernel.org/r/20240405-enable-streams-impro-v2-0-22bca967665d@ideasonboard.com

Changes in v2:
- New patches for privacy led
- Use v4l2_subdev_has_op()
- Use BITS_PER_BYTE instead of 8
- Fix 80+ line issues
- Fix typos
- Check for pad < 64 also in the non-routing .enable/disable_streams code path
- Dropped "media: subdev: Support enable/disable_streams for non-streams
  subdevs", which is implemented in a different way in new patches in this series
- Link to v1: https://lore.kernel.org/r/20240404-enable-streams-impro-v1-0-1017a35bbe07@ideasonboard.com

---
Tomi Valkeinen (10):
      media: subdev: Add privacy led helpers
      media: subdev: Use v4l2_subdev_has_op() in v4l2_subdev_enable/disable_streams()
      media: subdev: Add checks for subdev features
      media: subdev: Fix use of sd->enabled_streams in call_s_stream()
      media: subdev: Improve v4l2_subdev_enable/disable_streams_fallback
      media: subdev: Add v4l2_subdev_is_streaming()
      media: subdev: Support privacy led in v4l2_subdev_enable/disable_streams()
      media: subdev: Refactor v4l2_subdev_enable/disable_streams()
      media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()
      media: subdev: Support non-routing subdevs in v4l2_subdev_s_stream_helper()

 drivers/media/v4l2-core/v4l2-subdev.c | 378 ++++++++++++++++++++--------------
 include/media/v4l2-subdev.h           |  25 ++-
 2 files changed, 245 insertions(+), 158 deletions(-)
---
base-commit: 6547a87b1ffc9b3c3a17f20f71016de98c169bbb
change-id: 20240404-enable-streams-impro-db8bcd898471

Best regards,
-- 
Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

[PATCH v5 01/10] media: subdev: Add privacy led helpers

[Tomi Valkeinen]

Add helper functions to enable and disable the privacy led. This moves
the #if from the call site to the privacy led functions, and makes
adding privacy led support to v4l2_subdev_enable/disable_streams()
cleaner.

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 30 +++++++++++++++++++++---------
 1 file changed, 21 insertions(+), 9 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 012b757eac9f..13957543d153 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -148,6 +148,23 @@ static int subdev_close(struct file *file)
 }
 #endif /* CONFIG_VIDEO_V4L2_SUBDEV_API */
 
+static void v4l2_subdev_enable_privacy_led(struct v4l2_subdev *sd)
+{
+#if IS_REACHABLE(CONFIG_LEDS_CLASS)
+	if (!IS_ERR_OR_NULL(sd->privacy_led))
+		led_set_brightness(sd->privacy_led,
+				   sd->privacy_led->max_brightness);
+#endif
+}
+
+static void v4l2_subdev_disable_privacy_led(struct v4l2_subdev *sd)
+{
+#if IS_REACHABLE(CONFIG_LEDS_CLASS)
+	if (!IS_ERR_OR_NULL(sd->privacy_led))
+		led_set_brightness(sd->privacy_led, 0);
+#endif
+}
+
 static inline int check_which(u32 which)
 {
 	if (which != V4L2_SUBDEV_FORMAT_TRY &&
@@ -422,15 +439,10 @@ static int call_s_stream(struct v4l2_subdev *sd, int enable)
 	if (!ret) {
 		sd->enabled_streams = enable ? BIT(0) : 0;
 
-#if IS_REACHABLE(CONFIG_LEDS_CLASS)
-		if (!IS_ERR_OR_NULL(sd->privacy_led)) {
-			if (enable)
-				led_set_brightness(sd->privacy_led,
-						   sd->privacy_led->max_brightness);
-			else
-				led_set_brightness(sd->privacy_led, 0);
-		}
-#endif
+		if (enable)
+			v4l2_subdev_enable_privacy_led(sd);
+		else
+			v4l2_subdev_disable_privacy_led(sd);
 	}
 
 	return ret;

-- 
2.34.1

[PATCH v5 02/10] media: subdev: Use v4l2_subdev_has_op() in v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

Use v4l2_subdev_has_op() in v4l2_subdev_enable/disable_streams() instead
of open coding the same.

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 13957543d153..4a531c2b16c4 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2133,7 +2133,7 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 		return 0;
 
 	/* Fallback on .s_stream() if .enable_streams() isn't available. */
-	if (!sd->ops->pad || !sd->ops->pad->enable_streams)
+	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
 		return v4l2_subdev_enable_streams_fallback(sd, pad,
 							   streams_mask);
 
@@ -2250,7 +2250,7 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 		return 0;
 
 	/* Fallback on .s_stream() if .disable_streams() isn't available. */
-	if (!sd->ops->pad || !sd->ops->pad->disable_streams)
+	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
 		return v4l2_subdev_disable_streams_fallback(sd, pad,
 							    streams_mask);
 

-- 
2.34.1

[PATCH v5 03/10] media: subdev: Add checks for subdev features

[Tomi Valkeinen]

Add some checks to verify that the subdev driver implements required
features.

A subdevice that supports streams (V4L2_SUBDEV_FL_STREAMS) must do one
of the following:

- Implement neither .enable/disable_streams() nor .s_stream(), if the
  subdev is part of a video driver that uses an internal method to
  enable the subdev.
- Implement only .enable/disable_streams(), if support for legacy
  sink-side subdevices is not needed.
- Implement .enable/disable_streams() and .s_stream(), if support for
  legacy sink-side subdevices is needed.

At the moment the framework doesn't check this requirement. Add the
check.

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 4a531c2b16c4..606a909cd778 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -1533,6 +1533,33 @@ int __v4l2_subdev_init_finalize(struct v4l2_subdev *sd, const char *name,
 				struct lock_class_key *key)
 {
 	struct v4l2_subdev_state *state;
+	struct device *dev = sd->dev;
+	bool has_disable_streams;
+	bool has_enable_streams;
+	bool has_s_stream;
+
+	/* Check that the subdevice implements the required features */
+
+	has_s_stream = v4l2_subdev_has_op(sd, video, s_stream);
+	has_enable_streams = v4l2_subdev_has_op(sd, pad, enable_streams);
+	has_disable_streams = v4l2_subdev_has_op(sd, pad, disable_streams);
+
+	if (has_enable_streams != has_disable_streams) {
+		dev_err(dev,
+			"subdev '%s' must implement both or neither of .enable_streams() and .disable_streams()\n",
+			sd->name);
+		return -EINVAL;
+	}
+
+	if (sd->flags & V4L2_SUBDEV_FL_STREAMS) {
+		if (has_s_stream && !has_enable_streams) {
+			dev_err(dev,
+				"subdev '%s' must implement .enable/disable_streams()\n",
+				sd->name);
+
+			return -EINVAL;
+		}
+	}
 
 	state = __v4l2_subdev_state_alloc(sd, name, key);
 	if (IS_ERR(state))

-- 
2.34.1

[PATCH v5 04/10] media: subdev: Fix use of sd->enabled_streams in call_s_stream()

[Tomi Valkeinen]

call_s_stream() uses sd->enabled_streams to track whether streaming has
already been enabled. However,
v4l2_subdev_enable/disable_streams_fallback(), which was the original
user of this field, already uses it, and
v4l2_subdev_enable/disable_streams_fallback() will call call_s_stream().

This leads to a conflict as both functions set the field. Afaics, both
functions set the field to the same value, so it won't cause a runtime
bug, but it's still wrong and if we, e.g., change how
v4l2_subdev_enable/disable_streams_fallback() operates we might easily
cause bugs.

Fix this by adding a new field, 's_stream_enabled', for
call_s_stream().

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 8 ++------
 include/media/v4l2-subdev.h           | 3 +++
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 606a909cd778..941cf7be22c3 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -421,12 +421,8 @@ static int call_s_stream(struct v4l2_subdev *sd, int enable)
 	 * The .s_stream() operation must never be called to start or stop an
 	 * already started or stopped subdev. Catch offenders but don't return
 	 * an error yet to avoid regressions.
-	 *
-	 * As .s_stream() is mutually exclusive with the .enable_streams() and
-	 * .disable_streams() operation, we can use the enabled_streams field
-	 * to store the subdev streaming state.
 	 */
-	if (WARN_ON(!!sd->enabled_streams == !!enable))
+	if (WARN_ON(sd->s_stream_enabled == !!enable))
 		return 0;
 
 	ret = sd->ops->video->s_stream(sd, enable);
@@ -437,7 +433,7 @@ static int call_s_stream(struct v4l2_subdev *sd, int enable)
 	}
 
 	if (!ret) {
-		sd->enabled_streams = enable ? BIT(0) : 0;
+		sd->s_stream_enabled = enable;
 
 		if (enable)
 			v4l2_subdev_enable_privacy_led(sd);
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index a9e6b8146279..b3c3777db464 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1043,6 +1043,8 @@ struct v4l2_subdev_platform_data {
  *		     v4l2_subdev_enable_streams() and
  *		     v4l2_subdev_disable_streams() helper functions for fallback
  *		     cases.
+ * @s_stream_enabled: Tracks whether streaming has been enabled with s_stream.
+ *                    This is only for call_s_stream() internal use.
  *
  * Each instance of a subdev driver should create this struct, either
  * stand-alone or embedded in a larger struct.
@@ -1091,6 +1093,7 @@ struct v4l2_subdev {
 	 */
 	struct v4l2_subdev_state *active_state;
 	u64 enabled_streams;
+	bool s_stream_enabled;
 };
 
 

-- 
2.34.1

[PATCH v5 05/10] media: subdev: Improve v4l2_subdev_enable/disable_streams_fallback

[Tomi Valkeinen]

v4l2_subdev_enable/disable_streams_fallback() supports falling back to
.s_stream() for subdevs with a single source pad. It also tracks the
enabled streams for that one pad in the sd->enabled_streams field.

Tracking the enabled streams with sd->enabled_streams does not make
sense, as with .s_stream() there can only be a single stream per pad.
Thus, as the v4l2_subdev_enable/disable_streams_fallback() only supports
a single source pad, all we really need is a boolean which tells whether
streaming has been enabled on this pad or not.

However, as we only need a true/false state for a pad (instead of
tracking which streams have been enabled for a pad), we can easily
extend the fallback mechanism to support multiple source pads as we only
need to keep track of which pads have been enabled.

Change the sd->enabled_streams field to sd->enabled_pads, which is a
64-bit bitmask tracking the enabled source pads. With this change we can
remove the restriction that
v4l2_subdev_enable/disable_streams_fallback() only supports a single
source pad.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 68 ++++++++++++++++++++---------------
 include/media/v4l2-subdev.h           |  9 +++--
 2 files changed, 44 insertions(+), 33 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 941cf7be22c3..3824159bbe79 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2104,37 +2104,43 @@ static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
 					       u64 streams_mask)
 {
 	struct device *dev = sd->entity.graph_obj.mdev->dev;
-	unsigned int i;
 	int ret;
 
 	/*
 	 * The subdev doesn't implement pad-based stream enable, fall back
-	 * on the .s_stream() operation. This can only be done for subdevs that
-	 * have a single source pad, as sd->enabled_streams is global to the
-	 * subdev.
+	 * to the .s_stream() operation.
 	 */
 	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
 		return -EOPNOTSUPP;
 
-	for (i = 0; i < sd->entity.num_pads; ++i) {
-		if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
-			return -EOPNOTSUPP;
-	}
+	/*
+	 * .s_stream() means there is no streams support, so the only allowed
+	 * stream is the implicit stream 0.
+	 */
+	if (streams_mask != BIT_ULL(0))
+		return -EOPNOTSUPP;
+
+	/*
+	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
+	 * with 64 pads or less can be supported.
+	 */
+	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
+		return -EOPNOTSUPP;
 
-	if (sd->enabled_streams & streams_mask) {
-		dev_dbg(dev, "set of streams %#llx already enabled on %s:%u\n",
-			streams_mask, sd->entity.name, pad);
+	if (sd->enabled_pads & BIT_ULL(pad)) {
+		dev_dbg(dev, "pad %u already enabled on %s\n",
+			pad, sd->entity.name);
 		return -EALREADY;
 	}
 
-	/* Start streaming when the first streams are enabled. */
-	if (!sd->enabled_streams) {
+	/* Start streaming when the first pad is enabled. */
+	if (!sd->enabled_pads) {
 		ret = v4l2_subdev_call(sd, video, s_stream, 1);
 		if (ret)
 			return ret;
 	}
 
-	sd->enabled_streams |= streams_mask;
+	sd->enabled_pads |= BIT_ULL(pad);
 
 	return 0;
 }
@@ -2221,37 +2227,43 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
 						u64 streams_mask)
 {
 	struct device *dev = sd->entity.graph_obj.mdev->dev;
-	unsigned int i;
 	int ret;
 
 	/*
-	 * If the subdev doesn't implement pad-based stream enable, fall  back
-	 * on the .s_stream() operation. This can only be done for subdevs that
-	 * have a single source pad, as sd->enabled_streams is global to the
-	 * subdev.
+	 * If the subdev doesn't implement pad-based stream enable, fall back
+	 * to the .s_stream() operation.
 	 */
 	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
 		return -EOPNOTSUPP;
 
-	for (i = 0; i < sd->entity.num_pads; ++i) {
-		if (i != pad && sd->entity.pads[i].flags & MEDIA_PAD_FL_SOURCE)
-			return -EOPNOTSUPP;
-	}
+	/*
+	 * .s_stream() means there is no streams support, so the only allowed
+	 * stream is the implicit stream 0.
+	 */
+	if (streams_mask != BIT_ULL(0))
+		return -EOPNOTSUPP;
+
+	/*
+	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
+	 * with 64 pads or less can be supported.
+	 */
+	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
+		return -EOPNOTSUPP;
 
-	if ((sd->enabled_streams & streams_mask) != streams_mask) {
-		dev_dbg(dev, "set of streams %#llx already disabled on %s:%u\n",
-			streams_mask, sd->entity.name, pad);
+	if (!(sd->enabled_pads & BIT_ULL(pad))) {
+		dev_dbg(dev, "pad %u already disabled on %s\n",
+			pad, sd->entity.name);
 		return -EALREADY;
 	}
 
 	/* Stop streaming when the last streams are disabled. */
-	if (!(sd->enabled_streams & ~streams_mask)) {
+	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
 		ret = v4l2_subdev_call(sd, video, s_stream, 0);
 		if (ret)
 			return ret;
 	}
 
-	sd->enabled_streams &= ~streams_mask;
+	sd->enabled_pads &= ~BIT_ULL(pad);
 
 	return 0;
 }
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index b3c3777db464..ddf22d7e5b9d 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1039,10 +1039,9 @@ struct v4l2_subdev_platform_data {
  * @active_state: Active state for the subdev (NULL for subdevs tracking the
  *		  state internally). Initialized by calling
  *		  v4l2_subdev_init_finalize().
- * @enabled_streams: Bitmask of enabled streams used by
- *		     v4l2_subdev_enable_streams() and
- *		     v4l2_subdev_disable_streams() helper functions for fallback
- *		     cases.
+ * @enabled_pads: Bitmask of enabled pads used by v4l2_subdev_enable_streams()
+ *		  and v4l2_subdev_disable_streams() helper functions for
+ *		  fallback cases.
  * @s_stream_enabled: Tracks whether streaming has been enabled with s_stream.
  *                    This is only for call_s_stream() internal use.
  *
@@ -1092,7 +1091,7 @@ struct v4l2_subdev {
 	 * doesn't support it.
 	 */
 	struct v4l2_subdev_state *active_state;
-	u64 enabled_streams;
+	u64 enabled_pads;
 	bool s_stream_enabled;
 };
 

-- 
2.34.1

[PATCH v5 06/10] media: subdev: Add v4l2_subdev_is_streaming()

[Tomi Valkeinen]

Add a helper function which returns whether the subdevice is streaming,
i.e. if .s_stream or .enable_streams has been called successfully.

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 25 +++++++++++++++++++++++++
 include/media/v4l2-subdev.h           | 13 +++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 3824159bbe79..06f87b15dadb 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2419,6 +2419,31 @@ void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_notify_event);
 
+bool v4l2_subdev_is_streaming(struct v4l2_subdev *sd)
+{
+	struct v4l2_subdev_state *state;
+
+	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
+		return sd->s_stream_enabled;
+
+	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS))
+		return !!sd->enabled_pads;
+
+	state = v4l2_subdev_get_locked_active_state(sd);
+
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
+		const struct v4l2_subdev_stream_config *cfg;
+
+		cfg = &state->stream_configs.configs[i];
+
+		if (cfg->enabled)
+			return true;
+	}
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_is_streaming);
+
 int v4l2_subdev_get_privacy_led(struct v4l2_subdev *sd)
 {
 #if IS_REACHABLE(CONFIG_LEDS_CLASS)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index ddf22d7e5b9d..dabe1b5dfe4a 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -1914,4 +1914,17 @@ extern const struct v4l2_subdev_ops v4l2_subdev_call_wrappers;
 void v4l2_subdev_notify_event(struct v4l2_subdev *sd,
 			      const struct v4l2_event *ev);
 
+/**
+ * v4l2_subdev_is_streaming() - Returns if the subdevice is streaming
+ * @sd: The subdevice
+ *
+ * v4l2_subdev_is_streaming() tells if the subdevice is currently streaming.
+ * "Streaming" here means whether .s_stream() or .enable_streams() has been
+ * successfully called, and the streaming has not yet been disabled.
+ *
+ * If the subdevice implements .enable_streams() this function must be called
+ * while holding the active state lock.
+ */
+bool v4l2_subdev_is_streaming(struct v4l2_subdev *sd);
+
 #endif /* _V4L2_SUBDEV_H */

-- 
2.34.1

[PATCH v5 07/10] media: subdev: Support privacy led in v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

We support camera privacy leds with the .s_stream() operation, in
call_s_stream(), but we don't have that support when the subdevice
implements .enable/disable_streams() operations.

Add the support by enabling the led when the first stream for a
subdevice is enabled, and disabling the led then the last stream is
disabled.

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 06f87b15dadb..38388b223564 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2150,6 +2150,7 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 {
 	struct device *dev = sd->entity.graph_obj.mdev->dev;
 	struct v4l2_subdev_state *state;
+	bool already_streaming;
 	u64 found_streams = 0;
 	unsigned int i;
 	int ret;
@@ -2198,6 +2199,8 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 
 	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
 
+	already_streaming = v4l2_subdev_is_streaming(sd);
+
 	/* Call the .enable_streams() operation. */
 	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
 			       streams_mask);
@@ -2216,6 +2219,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 			cfg->enabled = true;
 	}
 
+	if (!already_streaming)
+		v4l2_subdev_enable_privacy_led(sd);
+
 done:
 	v4l2_subdev_unlock_state(state);
 
@@ -2340,6 +2346,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 	}
 
 done:
+	if (!v4l2_subdev_is_streaming(sd))
+		v4l2_subdev_disable_privacy_led(sd);
+
 	v4l2_subdev_unlock_state(state);
 
 	return ret;

-- 
2.34.1

[PATCH v5 08/10] media: subdev: Refactor v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

Add two internal helper functions, v4l2_subdev_collect_streams() and
v4l2_subdev_set_streams_enabled(), which allows us to refactor
v4l2_subdev_enable/disable_streams() functions.

This (I think) makes the code a bit easier to read, and lets us more
easily add new functionality in the helper functions in the following
patch.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 109 +++++++++++++++++++---------------
 1 file changed, 60 insertions(+), 49 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 38388b223564..e45fd42da1e3 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2100,6 +2100,42 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_routing_validate);
 
+static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
+					struct v4l2_subdev_state *state,
+					u32 pad, u64 streams_mask,
+					u64 *found_streams,
+					u64 *enabled_streams)
+{
+	*found_streams = 0;
+	*enabled_streams = 0;
+
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
+		const struct v4l2_subdev_stream_config *cfg =
+			&state->stream_configs.configs[i];
+
+		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
+			continue;
+
+		*found_streams |= BIT_ULL(cfg->stream);
+		if (cfg->enabled)
+			*enabled_streams |= BIT_ULL(cfg->stream);
+	}
+}
+
+static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
+					    struct v4l2_subdev_state *state,
+					    u32 pad, u64 streams_mask,
+					    bool enabled)
+{
+	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
+		struct v4l2_subdev_stream_config *cfg =
+			&state->stream_configs.configs[i];
+
+		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
+			cfg->enabled = enabled;
+	}
+}
+
 static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
 					       u64 streams_mask)
 {
@@ -2151,8 +2187,8 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	struct device *dev = sd->entity.graph_obj.mdev->dev;
 	struct v4l2_subdev_state *state;
 	bool already_streaming;
-	u64 found_streams = 0;
-	unsigned int i;
+	u64 enabled_streams;
+	u64 found_streams;
 	int ret;
 
 	/* A few basic sanity checks first. */
@@ -2173,22 +2209,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	 * Verify that the requested streams exist and that they are not
 	 * already enabled.
 	 */
-	for (i = 0; i < state->stream_configs.num_configs; ++i) {
-		struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
 
-		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
-			continue;
-
-		found_streams |= BIT_ULL(cfg->stream);
-
-		if (cfg->enabled) {
-			dev_dbg(dev, "stream %u already enabled on %s:%u\n",
-				cfg->stream, sd->entity.name, pad);
-			ret = -EALREADY;
-			goto done;
-		}
-	}
+	v4l2_subdev_collect_streams(sd, state, pad, streams_mask,
+				    &found_streams, &enabled_streams);
 
 	if (found_streams != streams_mask) {
 		dev_dbg(dev, "streams 0x%llx not found on %s:%u\n",
@@ -2197,6 +2220,13 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
+	if (enabled_streams) {
+		dev_dbg(dev, "streams 0x%llx already enabled on %s:%u\n",
+			enabled_streams, sd->entity.name, pad);
+		ret = -EINVAL;
+		goto done;
+	}
+
 	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
 
 	already_streaming = v4l2_subdev_is_streaming(sd);
@@ -2211,13 +2241,7 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	}
 
 	/* Mark the streams as enabled. */
-	for (i = 0; i < state->stream_configs.num_configs; ++i) {
-		struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
-
-		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
-			cfg->enabled = true;
-	}
+	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
 
 	if (!already_streaming)
 		v4l2_subdev_enable_privacy_led(sd);
@@ -2279,8 +2303,8 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 {
 	struct device *dev = sd->entity.graph_obj.mdev->dev;
 	struct v4l2_subdev_state *state;
-	u64 found_streams = 0;
-	unsigned int i;
+	u64 enabled_streams;
+	u64 found_streams;
 	int ret;
 
 	/* A few basic sanity checks first. */
@@ -2301,22 +2325,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 	 * Verify that the requested streams exist and that they are not
 	 * already disabled.
 	 */
-	for (i = 0; i < state->stream_configs.num_configs; ++i) {
-		struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
-
-		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
-			continue;
 
-		found_streams |= BIT_ULL(cfg->stream);
-
-		if (!cfg->enabled) {
-			dev_dbg(dev, "stream %u already disabled on %s:%u\n",
-				cfg->stream, sd->entity.name, pad);
-			ret = -EALREADY;
-			goto done;
-		}
-	}
+	v4l2_subdev_collect_streams(sd, state, pad, streams_mask,
+				    &found_streams, &enabled_streams);
 
 	if (found_streams != streams_mask) {
 		dev_dbg(dev, "streams 0x%llx not found on %s:%u\n",
@@ -2325,6 +2336,13 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
+	if (enabled_streams != streams_mask) {
+		dev_dbg(dev, "streams 0x%llx already disabled on %s:%u\n",
+			streams_mask & ~enabled_streams, sd->entity.name, pad);
+		ret = -EINVAL;
+		goto done;
+	}
+
 	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
 
 	/* Call the .disable_streams() operation. */
@@ -2336,14 +2354,7 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 		goto done;
 	}
 
-	/* Mark the streams as disabled. */
-	for (i = 0; i < state->stream_configs.num_configs; ++i) {
-		struct v4l2_subdev_stream_config *cfg =
-			&state->stream_configs.configs[i];
-
-		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
-			cfg->enabled = false;
-	}
+	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
 
 done:
 	if (!v4l2_subdev_is_streaming(sd))

-- 
2.34.1

[PATCH v5 09/10] media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

At the moment the v4l2_subdev_enable/disable_streams() functions call
fallback helpers to handle the case where the subdev only implements
.s_stream(), and the main function handles the case where the subdev
implements streams (V4L2_SUBDEV_FL_STREAMS, which implies
.enable/disable_streams()).

What is missing is support for subdevs which do not implement streams
support, but do implement .enable/disable_streams(). Example cases of
these subdevices are single-stream cameras, where using
.enable/disable_streams() is not required but helps us remove the users
of the legacy .s_stream(), and subdevices with multiple source pads (but
single stream per pad), where .enable/disable_streams() allows the
subdevice to control the enable/disable state per pad.

The two single-streams cases (.s_stream() and .enable/disable_streams())
are very similar, and with small changes we can change the
v4l2_subdev_enable/disable_streams() functions to support all three
cases, without needing separate fallback functions.

A few potentially problematic details, though:

- For the single-streams cases we use sd->enabled_pads field, which
  limits the number of pads for the subdevice to 64. For simplicity I
  added the check for this limitation to the beginning of the function,
  and it also applies to the streams case.

- The fallback functions only allowed the target pad to be a source pad.
  It is not very clear to me why this check was needed, but it was not
  needed in the streams case. However, I doubt the
  v4l2_subdev_enable/disable_streams() code has ever been tested with
  sink pads, so to be on the safe side, I added the same check
  to the v4l2_subdev_enable/disable_streams() functions.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 187 ++++++++++++++--------------------
 1 file changed, 79 insertions(+), 108 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index e45fd42da1e3..10406acfb9d0 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2106,6 +2106,13 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
 					u64 *found_streams,
 					u64 *enabled_streams)
 {
+	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
+		*found_streams = BIT_ULL(0);
+		*enabled_streams =
+			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
+		return;
+	}
+
 	*found_streams = 0;
 	*enabled_streams = 0;
 
@@ -2127,6 +2134,14 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 					    u32 pad, u64 streams_mask,
 					    bool enabled)
 {
+	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
+		if (enabled)
+			sd->enabled_pads |= BIT_ULL(pad);
+		else
+			sd->enabled_pads &= ~BIT_ULL(pad);
+		return;
+	}
+
 	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
 		struct v4l2_subdev_stream_config *cfg =
 			&state->stream_configs.configs[i];
@@ -2136,51 +2151,6 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
 	}
 }
 
-static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
-					       u64 streams_mask)
-{
-	struct device *dev = sd->entity.graph_obj.mdev->dev;
-	int ret;
-
-	/*
-	 * The subdev doesn't implement pad-based stream enable, fall back
-	 * to the .s_stream() operation.
-	 */
-	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
-		return -EOPNOTSUPP;
-
-	/*
-	 * .s_stream() means there is no streams support, so the only allowed
-	 * stream is the implicit stream 0.
-	 */
-	if (streams_mask != BIT_ULL(0))
-		return -EOPNOTSUPP;
-
-	/*
-	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
-	 * with 64 pads or less can be supported.
-	 */
-	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
-		return -EOPNOTSUPP;
-
-	if (sd->enabled_pads & BIT_ULL(pad)) {
-		dev_dbg(dev, "pad %u already enabled on %s\n",
-			pad, sd->entity.name);
-		return -EALREADY;
-	}
-
-	/* Start streaming when the first pad is enabled. */
-	if (!sd->enabled_pads) {
-		ret = v4l2_subdev_call(sd, video, s_stream, 1);
-		if (ret)
-			return ret;
-	}
-
-	sd->enabled_pads |= BIT_ULL(pad);
-
-	return 0;
-}
-
 int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 			       u64 streams_mask)
 {
@@ -2189,21 +2159,33 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	bool already_streaming;
 	u64 enabled_streams;
 	u64 found_streams;
+	bool use_s_stream;
 	int ret;
 
 	/* A few basic sanity checks first. */
 	if (pad >= sd->entity.num_pads)
 		return -EINVAL;
 
+	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
+		return -EOPNOTSUPP;
+
+	/*
+	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
+	 * with 64 pads or less can be supported.
+	 */
+	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
+		return -EOPNOTSUPP;
+
 	if (!streams_mask)
 		return 0;
 
 	/* Fallback on .s_stream() if .enable_streams() isn't available. */
-	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
-		return v4l2_subdev_enable_streams_fallback(sd, pad,
-							   streams_mask);
+	use_s_stream = !v4l2_subdev_has_op(sd, pad, enable_streams);
 
-	state = v4l2_subdev_lock_and_get_active_state(sd);
+	if (!use_s_stream)
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+	else
+		state = NULL;
 
 	/*
 	 * Verify that the requested streams exist and that they are not
@@ -2231,9 +2213,18 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 
 	already_streaming = v4l2_subdev_is_streaming(sd);
 
-	/* Call the .enable_streams() operation. */
-	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
-			       streams_mask);
+	if (!use_s_stream) {
+		/* Call the .enable_streams() operation. */
+		ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
+				       streams_mask);
+	} else {
+		/* Start streaming when the first pad is enabled. */
+		if (!already_streaming)
+			ret = v4l2_subdev_call(sd, video, s_stream, 1);
+		else
+			ret = 0;
+	}
+
 	if (ret) {
 		dev_dbg(dev, "enable streams %u:%#llx failed: %d\n", pad,
 			streams_mask, ret);
@@ -2243,34 +2234,32 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
 	/* Mark the streams as enabled. */
 	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
 
-	if (!already_streaming)
+	if (!use_s_stream && !already_streaming)
 		v4l2_subdev_enable_privacy_led(sd);
 
 done:
-	v4l2_subdev_unlock_state(state);
+	if (!use_s_stream)
+		v4l2_subdev_unlock_state(state);
 
 	return ret;
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
 
-static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
-						u64 streams_mask)
+int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
+				u64 streams_mask)
 {
 	struct device *dev = sd->entity.graph_obj.mdev->dev;
+	struct v4l2_subdev_state *state;
+	u64 enabled_streams;
+	u64 found_streams;
+	bool use_s_stream;
 	int ret;
 
-	/*
-	 * If the subdev doesn't implement pad-based stream enable, fall back
-	 * to the .s_stream() operation.
-	 */
-	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
-		return -EOPNOTSUPP;
+	/* A few basic sanity checks first. */
+	if (pad >= sd->entity.num_pads)
+		return -EINVAL;
 
-	/*
-	 * .s_stream() means there is no streams support, so the only allowed
-	 * stream is the implicit stream 0.
-	 */
-	if (streams_mask != BIT_ULL(0))
+	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
 		return -EOPNOTSUPP;
 
 	/*
@@ -2280,46 +2269,16 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
 	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
 		return -EOPNOTSUPP;
 
-	if (!(sd->enabled_pads & BIT_ULL(pad))) {
-		dev_dbg(dev, "pad %u already disabled on %s\n",
-			pad, sd->entity.name);
-		return -EALREADY;
-	}
-
-	/* Stop streaming when the last streams are disabled. */
-	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
-		ret = v4l2_subdev_call(sd, video, s_stream, 0);
-		if (ret)
-			return ret;
-	}
-
-	sd->enabled_pads &= ~BIT_ULL(pad);
-
-	return 0;
-}
-
-int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
-				u64 streams_mask)
-{
-	struct device *dev = sd->entity.graph_obj.mdev->dev;
-	struct v4l2_subdev_state *state;
-	u64 enabled_streams;
-	u64 found_streams;
-	int ret;
-
-	/* A few basic sanity checks first. */
-	if (pad >= sd->entity.num_pads)
-		return -EINVAL;
-
 	if (!streams_mask)
 		return 0;
 
 	/* Fallback on .s_stream() if .disable_streams() isn't available. */
-	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
-		return v4l2_subdev_disable_streams_fallback(sd, pad,
-							    streams_mask);
+	use_s_stream = !v4l2_subdev_has_op(sd, pad, disable_streams);
 
-	state = v4l2_subdev_lock_and_get_active_state(sd);
+	if (!use_s_stream)
+		state = v4l2_subdev_lock_and_get_active_state(sd);
+	else
+		state = NULL;
 
 	/*
 	 * Verify that the requested streams exist and that they are not
@@ -2345,9 +2304,19 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 
 	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
 
-	/* Call the .disable_streams() operation. */
-	ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
-			       streams_mask);
+	if (!use_s_stream) {
+		/* Call the .disable_streams() operation. */
+		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
+				       streams_mask);
+	} else {
+		/* Stop streaming when the last streams are disabled. */
+
+		if (!(sd->enabled_pads & ~BIT_ULL(pad)))
+			ret = v4l2_subdev_call(sd, video, s_stream, 0);
+		else
+			ret = 0;
+	}
+
 	if (ret) {
 		dev_dbg(dev, "disable streams %u:%#llx failed: %d\n", pad,
 			streams_mask, ret);
@@ -2357,10 +2326,12 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
 	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
 
 done:
-	if (!v4l2_subdev_is_streaming(sd))
-		v4l2_subdev_disable_privacy_led(sd);
+	if (!use_s_stream) {
+		if (!v4l2_subdev_is_streaming(sd))
+			v4l2_subdev_disable_privacy_led(sd);
 
-	v4l2_subdev_unlock_state(state);
+		v4l2_subdev_unlock_state(state);
+	}
 
 	return ret;
 }

-- 
2.34.1

[PATCH v5 10/10] media: subdev: Support non-routing subdevs in v4l2_subdev_s_stream_helper()

[Tomi Valkeinen]

At the moment v4l2_subdev_s_stream_helper() only works for subdevices
that support routing. As enable/disable_streams now also works for
subdevices without routing, improve v4l2_subdev_s_stream_helper() to do
the same.

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 10406acfb9d0..64f52376ca7b 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -2360,15 +2360,24 @@ int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
 	if (WARN_ON(pad_index == -1))
 		return -EINVAL;
 
-	/*
-	 * As there's a single source pad, just collect all the source streams.
-	 */
-	state = v4l2_subdev_lock_and_get_active_state(sd);
+	if (sd->flags & V4L2_SUBDEV_FL_STREAMS) {
+		/*
+		 * As there's a single source pad, just collect all the source
+		 * streams.
+		 */
+		state = v4l2_subdev_lock_and_get_active_state(sd);
 
-	for_each_active_route(&state->routing, route)
-		source_mask |= BIT_ULL(route->source_stream);
+		for_each_active_route(&state->routing, route)
+			source_mask |= BIT_ULL(route->source_stream);
 
-	v4l2_subdev_unlock_state(state);
+		v4l2_subdev_unlock_state(state);
+	} else {
+		/*
+		 * For non-streams subdevices, there's a single implicit stream
+		 * per pad.
+		 */
+		source_mask = BIT_ULL(0);
+	}
 
 	if (enable)
 		return v4l2_subdev_enable_streams(sd, pad_index, source_mask);

-- 
2.34.1

Re: [PATCH v5 00/10] media: subdev: Improve stream enable/disable machinery

[Umang Jain]

Hi Tomi,

On 16/04/24 7:25 pm, Tomi Valkeinen wrote:
> This series works on the .s_stream, .enable_streams, .disable_streams
> related code. The main feature introduced here, in the last two patchs,
> is adding support for .enable/disable_streams() for subdevs that do not
> implement full streams support.
>
> Additionally we add support for the privacy led when
> v4l2_subdev_enable/disable_streams() is used.
>
> I have tested this on RPi5.

I have also tested the series on i.MX8MP with IMX283 hence,

Tested-by: Umang Jain <umang.jain@ideasonboard.com>
>
>   Tomi
>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
> Changes in v5:
> - Fix issues with non-routing case:
>    - Always set 'found_streams' variable instead of leaving it
>      uninitialized in "Support single-stream case in
>      v4l2_subdev_enable/disable_streams()"
>    - Fix the "implicit" stream from bit 1 to bit 0 in "Support
>      non-routing subdevs in v4l2_subdev_s_stream_helper()"
> - Link to v4: https://lore.kernel.org/r/20240416-enable-streams-impro-v4-0-1d370c9c2b6d@ideasonboard.com
>
> Changes in v4:
> - Added Rb tags
> - Rename 'streaming_enabled' to 's_stream_enabled'
> - Cosmetic changes (comments / patch descs)
> - Added new patch "media: subdev: Support non-routing subdevs in  v4l2_subdev_s_stream_helper()".
> - Link to v3: https://lore.kernel.org/r/20240410-enable-streams-impro-v3-0-e5e7a5da7420@ideasonboard.com
>
> Changes in v3:
> - Rebased on top of "[PATCH v2 1/1] media: v4l: Don't turn on privacy LED if streamon fails"
> - Drop extra "!!" in "media: subdev: Fix use of sd->enabled_streams in  call_s_stream()"
> - Enable privacy LED after a succesfull stream enable in  "media: subdev: Support privacy led in v4l2_subdev_enable/disable_streams()"
> - Init 'cfg' variable when declaring in "media: subdev: Refactor v4l2_subdev_enable/disable_streams()"
> - Link to v2: https://lore.kernel.org/r/20240405-enable-streams-impro-v2-0-22bca967665d@ideasonboard.com
>
> Changes in v2:
> - New patches for privacy led
> - Use v4l2_subdev_has_op()
> - Use BITS_PER_BYTE instead of 8
> - Fix 80+ line issues
> - Fix typos
> - Check for pad < 64 also in the non-routing .enable/disable_streams code path
> - Dropped "media: subdev: Support enable/disable_streams for non-streams
>    subdevs", which is implemented in a different way in new patches in this series
> - Link to v1: https://lore.kernel.org/r/20240404-enable-streams-impro-v1-0-1017a35bbe07@ideasonboard.com
>
> ---
> Tomi Valkeinen (10):
>        media: subdev: Add privacy led helpers
>        media: subdev: Use v4l2_subdev_has_op() in v4l2_subdev_enable/disable_streams()
>        media: subdev: Add checks for subdev features
>        media: subdev: Fix use of sd->enabled_streams in call_s_stream()
>        media: subdev: Improve v4l2_subdev_enable/disable_streams_fallback
>        media: subdev: Add v4l2_subdev_is_streaming()
>        media: subdev: Support privacy led in v4l2_subdev_enable/disable_streams()
>        media: subdev: Refactor v4l2_subdev_enable/disable_streams()
>        media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()
>        media: subdev: Support non-routing subdevs in v4l2_subdev_s_stream_helper()
>
>   drivers/media/v4l2-core/v4l2-subdev.c | 378 ++++++++++++++++++++--------------
>   include/media/v4l2-subdev.h           |  25 ++-
>   2 files changed, 245 insertions(+), 158 deletions(-)
> ---
> base-commit: 6547a87b1ffc9b3c3a17f20f71016de98c169bbb
> change-id: 20240404-enable-streams-impro-db8bcd898471
>
> Best regards,

Re: [PATCH v5 07/10] media: subdev: Support privacy led in v4l2_subdev_enable/disable_streams()

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Tue, Apr 16, 2024 at 04:55:10PM +0300, Tomi Valkeinen wrote:
> We support camera privacy leds with the .s_stream() operation, in
> call_s_stream(), but we don't have that support when the subdevice
> implements .enable/disable_streams() operations.
> 
> Add the support by enabling the led when the first stream for a
> subdevice is enabled, and disabling the led then the last stream is
> disabled.
> 
> Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 06f87b15dadb..38388b223564 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2150,6 +2150,7 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  {
>  	struct device *dev = sd->entity.graph_obj.mdev->dev;
>  	struct v4l2_subdev_state *state;
> +	bool already_streaming;
>  	u64 found_streams = 0;
>  	unsigned int i;
>  	int ret;
> @@ -2198,6 +2199,8 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  
>  	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
>  
> +	already_streaming = v4l2_subdev_is_streaming(sd);
> +
>  	/* Call the .enable_streams() operation. */
>  	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>  			       streams_mask);
> @@ -2216,6 +2219,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  			cfg->enabled = true;
>  	}
>  
> +	if (!already_streaming)
> +		v4l2_subdev_enable_privacy_led(sd);
> +
>  done:
>  	v4l2_subdev_unlock_state(state);
>  
> @@ -2340,6 +2346,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  	}
>  
>  done:
> +	if (!v4l2_subdev_is_streaming(sd))
> +		v4l2_subdev_disable_privacy_led(sd);
> +
>  	v4l2_subdev_unlock_state(state);
>  
>  	return ret;
> 

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 09/10] media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Tue, Apr 16, 2024 at 04:55:12PM +0300, Tomi Valkeinen wrote:
> At the moment the v4l2_subdev_enable/disable_streams() functions call
> fallback helpers to handle the case where the subdev only implements
> .s_stream(), and the main function handles the case where the subdev
> implements streams (V4L2_SUBDEV_FL_STREAMS, which implies
> .enable/disable_streams()).
> 
> What is missing is support for subdevs which do not implement streams
> support, but do implement .enable/disable_streams(). Example cases of
> these subdevices are single-stream cameras, where using
> .enable/disable_streams() is not required but helps us remove the users
> of the legacy .s_stream(), and subdevices with multiple source pads (but
> single stream per pad), where .enable/disable_streams() allows the
> subdevice to control the enable/disable state per pad.
> 
> The two single-streams cases (.s_stream() and .enable/disable_streams())
> are very similar, and with small changes we can change the
> v4l2_subdev_enable/disable_streams() functions to support all three
> cases, without needing separate fallback functions.
> 
> A few potentially problematic details, though:
> 
> - For the single-streams cases we use sd->enabled_pads field, which
>   limits the number of pads for the subdevice to 64. For simplicity I
>   added the check for this limitation to the beginning of the function,
>   and it also applies to the streams case.
> 
> - The fallback functions only allowed the target pad to be a source pad.
>   It is not very clear to me why this check was needed, but it was not
>   needed in the streams case. However, I doubt the
>   v4l2_subdev_enable/disable_streams() code has ever been tested with
>   sink pads, so to be on the safe side, I added the same check
>   to the v4l2_subdev_enable/disable_streams() functions.
> 

Now that v4l2_subdev_enable_streams() is usable by (almost) every
driver, should we update documentation to indicate that manual calls to
.s_stream() should be avoided ?

> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 187 ++++++++++++++--------------------
>  1 file changed, 79 insertions(+), 108 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index e45fd42da1e3..10406acfb9d0 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2106,6 +2106,13 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>  					u64 *found_streams,
>  					u64 *enabled_streams)
>  {
> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
> +		*found_streams = BIT_ULL(0);
> +		*enabled_streams =
> +			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
> +		return;
> +	}
> +
>  	*found_streams = 0;
>  	*enabled_streams = 0;
>  
> @@ -2127,6 +2134,14 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>  					    u32 pad, u64 streams_mask,
>  					    bool enabled)
>  {
> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
> +		if (enabled)
> +			sd->enabled_pads |= BIT_ULL(pad);
> +		else
> +			sd->enabled_pads &= ~BIT_ULL(pad);
> +		return;
> +	}
> +
>  	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>  		struct v4l2_subdev_stream_config *cfg =
>  			&state->stream_configs.configs[i];
> @@ -2136,51 +2151,6 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>  	}
>  }
>  
> -static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> -					       u64 streams_mask)
> -{
> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
> -	int ret;
> -
> -	/*
> -	 * The subdev doesn't implement pad-based stream enable, fall back
> -	 * to the .s_stream() operation.
> -	 */
> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> -		return -EOPNOTSUPP;
> -
> -	/*
> -	 * .s_stream() means there is no streams support, so the only allowed
> -	 * stream is the implicit stream 0.
> -	 */
> -	if (streams_mask != BIT_ULL(0))
> -		return -EOPNOTSUPP;
> -
> -	/*
> -	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
> -	 * with 64 pads or less can be supported.
> -	 */
> -	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> -		return -EOPNOTSUPP;
> -
> -	if (sd->enabled_pads & BIT_ULL(pad)) {
> -		dev_dbg(dev, "pad %u already enabled on %s\n",
> -			pad, sd->entity.name);
> -		return -EALREADY;
> -	}
> -
> -	/* Start streaming when the first pad is enabled. */
> -	if (!sd->enabled_pads) {
> -		ret = v4l2_subdev_call(sd, video, s_stream, 1);
> -		if (ret)
> -			return ret;
> -	}
> -
> -	sd->enabled_pads |= BIT_ULL(pad);
> -
> -	return 0;
> -}
> -
>  int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  			       u64 streams_mask)
>  {
> @@ -2189,21 +2159,33 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  	bool already_streaming;
>  	u64 enabled_streams;
>  	u64 found_streams;
> +	bool use_s_stream;
>  	int ret;
>  
>  	/* A few basic sanity checks first. */
>  	if (pad >= sd->entity.num_pads)
>  		return -EINVAL;
>  
> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> +		return -EOPNOTSUPP;
> +
> +	/*
> +	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
> +	 * with 64 pads or less can be supported.
> +	 */
> +	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> +		return -EOPNOTSUPP;
> +
>  	if (!streams_mask)
>  		return 0;
>  
>  	/* Fallback on .s_stream() if .enable_streams() isn't available. */
> -	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
> -		return v4l2_subdev_enable_streams_fallback(sd, pad,
> -							   streams_mask);
> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, enable_streams);
>  
> -	state = v4l2_subdev_lock_and_get_active_state(sd);
> +	if (!use_s_stream)
> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> +	else
> +		state = NULL;
>  
>  	/*
>  	 * Verify that the requested streams exist and that they are not
> @@ -2231,9 +2213,18 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  
>  	already_streaming = v4l2_subdev_is_streaming(sd);
>  
> -	/* Call the .enable_streams() operation. */
> -	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
> -			       streams_mask);
> +	if (!use_s_stream) {
> +		/* Call the .enable_streams() operation. */
> +		ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
> +				       streams_mask);
> +	} else {
> +		/* Start streaming when the first pad is enabled. */
> +		if (!already_streaming)
> +			ret = v4l2_subdev_call(sd, video, s_stream, 1);
> +		else
> +			ret = 0;
> +	}
> +
>  	if (ret) {
>  		dev_dbg(dev, "enable streams %u:%#llx failed: %d\n", pad,
>  			streams_mask, ret);
> @@ -2243,34 +2234,32 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  	/* Mark the streams as enabled. */
>  	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);

state gets set to a non-null value above if the subdev has an
enable_streams operation, while in v4l2_subdev_set_streams_enabled(),
you use it if the V4L2_SUBDEV_FL_STREAMS flag is set. The second
condition is a subset of the first one, so it should be safe now, but I
wonder if we're at risk of crashes in the future when reworking the
code. I'm probably worrying needlessly.

>  
> -	if (!already_streaming)
> +	if (!use_s_stream && !already_streaming)
>  		v4l2_subdev_enable_privacy_led(sd);

Once everybody switches to v4l2_subdev_enable_streams() (am I dreaming?)
we should move LED handling from the .s_stream() wrapper to here for the
fallback case too. How about a TODO comment ?

>  
>  done:
> -	v4l2_subdev_unlock_state(state);
> +	if (!use_s_stream)

	if (state)

would be better I think.

> +		v4l2_subdev_unlock_state(state);
>  
>  	return ret;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
>  
> -static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> -						u64 streams_mask)
> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> +				u64 streams_mask)
>  {
>  	struct device *dev = sd->entity.graph_obj.mdev->dev;
> +	struct v4l2_subdev_state *state;
> +	u64 enabled_streams;
> +	u64 found_streams;
> +	bool use_s_stream;
>  	int ret;
>  
> -	/*
> -	 * If the subdev doesn't implement pad-based stream enable, fall back
> -	 * to the .s_stream() operation.
> -	 */
> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> -		return -EOPNOTSUPP;
> +	/* A few basic sanity checks first. */
> +	if (pad >= sd->entity.num_pads)
> +		return -EINVAL;
>  
> -	/*
> -	 * .s_stream() means there is no streams support, so the only allowed
> -	 * stream is the implicit stream 0.
> -	 */
> -	if (streams_mask != BIT_ULL(0))
> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>  		return -EOPNOTSUPP;
>  
>  	/*
> @@ -2280,46 +2269,16 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>  	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>  		return -EOPNOTSUPP;
>  
> -	if (!(sd->enabled_pads & BIT_ULL(pad))) {
> -		dev_dbg(dev, "pad %u already disabled on %s\n",
> -			pad, sd->entity.name);
> -		return -EALREADY;
> -	}
> -
> -	/* Stop streaming when the last streams are disabled. */
> -	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
> -		ret = v4l2_subdev_call(sd, video, s_stream, 0);
> -		if (ret)
> -			return ret;
> -	}
> -
> -	sd->enabled_pads &= ~BIT_ULL(pad);
> -
> -	return 0;
> -}
> -
> -int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> -				u64 streams_mask)
> -{
> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
> -	struct v4l2_subdev_state *state;
> -	u64 enabled_streams;
> -	u64 found_streams;
> -	int ret;
> -
> -	/* A few basic sanity checks first. */
> -	if (pad >= sd->entity.num_pads)
> -		return -EINVAL;
> -
>  	if (!streams_mask)
>  		return 0;
>  
>  	/* Fallback on .s_stream() if .disable_streams() isn't available. */
> -	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
> -		return v4l2_subdev_disable_streams_fallback(sd, pad,
> -							    streams_mask);
> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, disable_streams);
>  
> -	state = v4l2_subdev_lock_and_get_active_state(sd);
> +	if (!use_s_stream)
> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> +	else
> +		state = NULL;
>  
>  	/*
>  	 * Verify that the requested streams exist and that they are not
> @@ -2345,9 +2304,19 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  
>  	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
>  
> -	/* Call the .disable_streams() operation. */
> -	ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
> -			       streams_mask);
> +	if (!use_s_stream) {
> +		/* Call the .disable_streams() operation. */
> +		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
> +				       streams_mask);
> +	} else {
> +		/* Stop streaming when the last streams are disabled. */
> +
> +		if (!(sd->enabled_pads & ~BIT_ULL(pad)))
> +			ret = v4l2_subdev_call(sd, video, s_stream, 0);
> +		else
> +			ret = 0;
> +	}
> +
>  	if (ret) {
>  		dev_dbg(dev, "disable streams %u:%#llx failed: %d\n", pad,
>  			streams_mask, ret);
> @@ -2357,10 +2326,12 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
>  
>  done:
> -	if (!v4l2_subdev_is_streaming(sd))
> -		v4l2_subdev_disable_privacy_led(sd);
> +	if (!use_s_stream) {
> +		if (!v4l2_subdev_is_streaming(sd))
> +			v4l2_subdev_disable_privacy_led(sd);

Do we want to disable the privacy LED on failure in all cases, in
particular when the .s_stream() or .disable_streams() operations are not
even called ?

And now that I wrote that, I realize the !v4l2_subdev_is_streaming()
condition will not be true if the operations failed, so it should be
fine.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  
> -	v4l2_subdev_unlock_state(state);
> +		v4l2_subdev_unlock_state(state);
> +	}
>  
>  	return ret;
>  }
> 

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 10/10] media: subdev: Support non-routing subdevs in v4l2_subdev_s_stream_helper()

[Laurent Pinchart]

Hi Tomi,

Thank you for the patch.

On Tue, Apr 16, 2024 at 04:55:13PM +0300, Tomi Valkeinen wrote:
> At the moment v4l2_subdev_s_stream_helper() only works for subdevices
> that support routing. As enable/disable_streams now also works for
> subdevices without routing, improve v4l2_subdev_s_stream_helper() to do
> the same.
> 
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 23 ++++++++++++++++-------
>  1 file changed, 16 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 10406acfb9d0..64f52376ca7b 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2360,15 +2360,24 @@ int v4l2_subdev_s_stream_helper(struct v4l2_subdev *sd, int enable)
>  	if (WARN_ON(pad_index == -1))
>  		return -EINVAL;
>  
> -	/*
> -	 * As there's a single source pad, just collect all the source streams.
> -	 */
> -	state = v4l2_subdev_lock_and_get_active_state(sd);
> +	if (sd->flags & V4L2_SUBDEV_FL_STREAMS) {
> +		/*
> +		 * As there's a single source pad, just collect all the source
> +		 * streams.
> +		 */
> +		state = v4l2_subdev_lock_and_get_active_state(sd);
>  
> -	for_each_active_route(&state->routing, route)
> -		source_mask |= BIT_ULL(route->source_stream);
> +		for_each_active_route(&state->routing, route)
> +			source_mask |= BIT_ULL(route->source_stream);
>  
> -	v4l2_subdev_unlock_state(state);
> +		v4l2_subdev_unlock_state(state);
> +	} else {
> +		/*
> +		 * For non-streams subdevices, there's a single implicit stream
> +		 * per pad.
> +		 */
> +		source_mask = BIT_ULL(0);
> +	}
>  
>  	if (enable)
>  		return v4l2_subdev_enable_streams(sd, pad_index, source_mask);

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 08/10] media: subdev: Refactor v4l2_subdev_enable/disable_streams()

[Hans Verkuil]

On 16/04/2024 15:55, Tomi Valkeinen wrote:
> Add two internal helper functions, v4l2_subdev_collect_streams() and
> v4l2_subdev_set_streams_enabled(), which allows us to refactor
> v4l2_subdev_enable/disable_streams() functions.
> 
> This (I think) makes the code a bit easier to read, and lets us more
> easily add new functionality in the helper functions in the following
> patch.
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 109 +++++++++++++++++++---------------
>  1 file changed, 60 insertions(+), 49 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 38388b223564..e45fd42da1e3 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -2100,6 +2100,42 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_routing_validate);
>  
> +static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
> +					struct v4l2_subdev_state *state,
> +					u32 pad, u64 streams_mask,
> +					u64 *found_streams,
> +					u64 *enabled_streams)
> +{
> +	*found_streams = 0;
> +	*enabled_streams = 0;
> +
> +	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> +		const struct v4l2_subdev_stream_config *cfg =
> +			&state->stream_configs.configs[i];
> +
> +		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
> +			continue;
> +
> +		*found_streams |= BIT_ULL(cfg->stream);
> +		if (cfg->enabled)
> +			*enabled_streams |= BIT_ULL(cfg->stream);
> +	}
> +}
> +
> +static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> +					    struct v4l2_subdev_state *state,
> +					    u32 pad, u64 streams_mask,
> +					    bool enabled)
> +{
> +	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> +		struct v4l2_subdev_stream_config *cfg =
> +			&state->stream_configs.configs[i];
> +
> +		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
> +			cfg->enabled = enabled;
> +	}
> +}
> +
>  static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>  					       u64 streams_mask)
>  {
> @@ -2151,8 +2187,8 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  	struct device *dev = sd->entity.graph_obj.mdev->dev;
>  	struct v4l2_subdev_state *state;
>  	bool already_streaming;
> -	u64 found_streams = 0;
> -	unsigned int i;
> +	u64 enabled_streams;
> +	u64 found_streams;
>  	int ret;
>  
>  	/* A few basic sanity checks first. */
> @@ -2173,22 +2209,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  	 * Verify that the requested streams exist and that they are not
>  	 * already enabled.
>  	 */
> -	for (i = 0; i < state->stream_configs.num_configs; ++i) {
> -		struct v4l2_subdev_stream_config *cfg =
> -			&state->stream_configs.configs[i];
>  
> -		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
> -			continue;
> -
> -		found_streams |= BIT_ULL(cfg->stream);
> -
> -		if (cfg->enabled) {
> -			dev_dbg(dev, "stream %u already enabled on %s:%u\n",
> -				cfg->stream, sd->entity.name, pad);
> -			ret = -EALREADY;
> -			goto done;
> -		}
> -	}
> +	v4l2_subdev_collect_streams(sd, state, pad, streams_mask,
> +				    &found_streams, &enabled_streams);
>  
>  	if (found_streams != streams_mask) {
>  		dev_dbg(dev, "streams 0x%llx not found on %s:%u\n",
> @@ -2197,6 +2220,13 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  		goto done;
>  	}
>  
> +	if (enabled_streams) {
> +		dev_dbg(dev, "streams 0x%llx already enabled on %s:%u\n",
> +			enabled_streams, sd->entity.name, pad);
> +		ret = -EINVAL;
> +		goto done;
> +	}
> +
>  	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
>  
>  	already_streaming = v4l2_subdev_is_streaming(sd);
> @@ -2211,13 +2241,7 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>  	}
>  
>  	/* Mark the streams as enabled. */
> -	for (i = 0; i < state->stream_configs.num_configs; ++i) {
> -		struct v4l2_subdev_stream_config *cfg =
> -			&state->stream_configs.configs[i];
> -
> -		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
> -			cfg->enabled = true;
> -	}
> +	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
>  
>  	if (!already_streaming)
>  		v4l2_subdev_enable_privacy_led(sd);
> @@ -2279,8 +2303,8 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  {
>  	struct device *dev = sd->entity.graph_obj.mdev->dev;
>  	struct v4l2_subdev_state *state;
> -	u64 found_streams = 0;
> -	unsigned int i;
> +	u64 enabled_streams;
> +	u64 found_streams;
>  	int ret;
>  
>  	/* A few basic sanity checks first. */
> @@ -2301,22 +2325,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  	 * Verify that the requested streams exist and that they are not
>  	 * already disabled.
>  	 */
> -	for (i = 0; i < state->stream_configs.num_configs; ++i) {
> -		struct v4l2_subdev_stream_config *cfg =
> -			&state->stream_configs.configs[i];
> -
> -		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
> -			continue;
>  
> -		found_streams |= BIT_ULL(cfg->stream);
> -
> -		if (!cfg->enabled) {
> -			dev_dbg(dev, "stream %u already disabled on %s:%u\n",
> -				cfg->stream, sd->entity.name, pad);
> -			ret = -EALREADY;

This used to return -EALREADY...

> -			goto done;
> -		}
> -	}
> +	v4l2_subdev_collect_streams(sd, state, pad, streams_mask,
> +				    &found_streams, &enabled_streams);
>  
>  	if (found_streams != streams_mask) {
>  		dev_dbg(dev, "streams 0x%llx not found on %s:%u\n",
> @@ -2325,6 +2336,13 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  		goto done;
>  	}
>  
> +	if (enabled_streams != streams_mask) {
> +		dev_dbg(dev, "streams 0x%llx already disabled on %s:%u\n",
> +			streams_mask & ~enabled_streams, sd->entity.name, pad);
> +		ret = -EINVAL;

...but now it returns -EINVAL.

Is that intentional?

I prefer EINVAL to be honest, but I was just wondering about this change.

It looks like the next patch removes the last of the EALREADY error returns
as well.

Regards,

	Hans

> +		goto done;
> +	}
> +
>  	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
>  
>  	/* Call the .disable_streams() operation. */
> @@ -2336,14 +2354,7 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>  		goto done;
>  	}
>  
> -	/* Mark the streams as disabled. */
> -	for (i = 0; i < state->stream_configs.num_configs; ++i) {
> -		struct v4l2_subdev_stream_config *cfg =
> -			&state->stream_configs.configs[i];
> -
> -		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
> -			cfg->enabled = false;
> -	}
> +	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
>  
>  done:
>  	if (!v4l2_subdev_is_streaming(sd))
> 

Re: [PATCH v5 09/10] media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

On 19/04/2024 18:36, Laurent Pinchart wrote:
> Hi Tomi,
> 
> Thank you for the patch.
> 
> On Tue, Apr 16, 2024 at 04:55:12PM +0300, Tomi Valkeinen wrote:
>> At the moment the v4l2_subdev_enable/disable_streams() functions call
>> fallback helpers to handle the case where the subdev only implements
>> .s_stream(), and the main function handles the case where the subdev
>> implements streams (V4L2_SUBDEV_FL_STREAMS, which implies
>> .enable/disable_streams()).
>>
>> What is missing is support for subdevs which do not implement streams
>> support, but do implement .enable/disable_streams(). Example cases of
>> these subdevices are single-stream cameras, where using
>> .enable/disable_streams() is not required but helps us remove the users
>> of the legacy .s_stream(), and subdevices with multiple source pads (but
>> single stream per pad), where .enable/disable_streams() allows the
>> subdevice to control the enable/disable state per pad.
>>
>> The two single-streams cases (.s_stream() and .enable/disable_streams())
>> are very similar, and with small changes we can change the
>> v4l2_subdev_enable/disable_streams() functions to support all three
>> cases, without needing separate fallback functions.
>>
>> A few potentially problematic details, though:
>>
>> - For the single-streams cases we use sd->enabled_pads field, which
>>    limits the number of pads for the subdevice to 64. For simplicity I
>>    added the check for this limitation to the beginning of the function,
>>    and it also applies to the streams case.
>>
>> - The fallback functions only allowed the target pad to be a source pad.
>>    It is not very clear to me why this check was needed, but it was not
>>    needed in the streams case. However, I doubt the
>>    v4l2_subdev_enable/disable_streams() code has ever been tested with
>>    sink pads, so to be on the safe side, I added the same check
>>    to the v4l2_subdev_enable/disable_streams() functions.
>>
> 
> Now that v4l2_subdev_enable_streams() is usable by (almost) every
> driver, should we update documentation to indicate that manual calls to
> .s_stream() should be avoided ?

How about:

diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index dabe1b5dfe4a..f52bb790773c 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -450,6 +450,11 @@ enum v4l2_subdev_pre_streamon_flags {
   *     already started or stopped subdev. Also see call_s_stream 
wrapper in
   *     v4l2-subdev.c.
   *
+ *     New drivers should instead implement 
&v4l2_subdev_pad_ops.enable_streams
+ *     and &v4l2_subdev_pad_ops.disable_streams operations, and use
+ *     v4l2_subdev_s_stream_helper for the &v4l2_subdev_video_ops.s_stream
+ *     operation to support legacy users.
+ *
   * @g_pixelaspect: callback to return the pixelaspect ratio.
   *
   * @s_dv_timings: Set custom dv timings in the sub device. This is used


Although now that I think about it, can we "ever" get rid of s_stream? 
enable_streams is only usable if the subdev has pads. If that's the 
case, the text above should probably clarify when one should continue 
using s_stream.

>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/v4l2-core/v4l2-subdev.c | 187 ++++++++++++++--------------------
>>   1 file changed, 79 insertions(+), 108 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index e45fd42da1e3..10406acfb9d0 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -2106,6 +2106,13 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>>   					u64 *found_streams,
>>   					u64 *enabled_streams)
>>   {
>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
>> +		*found_streams = BIT_ULL(0);
>> +		*enabled_streams =
>> +			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
>> +		return;
>> +	}
>> +
>>   	*found_streams = 0;
>>   	*enabled_streams = 0;
>>   
>> @@ -2127,6 +2134,14 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>>   					    u32 pad, u64 streams_mask,
>>   					    bool enabled)
>>   {
>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
>> +		if (enabled)
>> +			sd->enabled_pads |= BIT_ULL(pad);
>> +		else
>> +			sd->enabled_pads &= ~BIT_ULL(pad);
>> +		return;
>> +	}
>> +
>>   	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>>   		struct v4l2_subdev_stream_config *cfg =
>>   			&state->stream_configs.configs[i];
>> @@ -2136,51 +2151,6 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>>   	}
>>   }
>>   
>> -static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>> -					       u64 streams_mask)
>> -{
>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
>> -	int ret;
>> -
>> -	/*
>> -	 * The subdev doesn't implement pad-based stream enable, fall back
>> -	 * to the .s_stream() operation.
>> -	 */
>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>> -		return -EOPNOTSUPP;
>> -
>> -	/*
>> -	 * .s_stream() means there is no streams support, so the only allowed
>> -	 * stream is the implicit stream 0.
>> -	 */
>> -	if (streams_mask != BIT_ULL(0))
>> -		return -EOPNOTSUPP;
>> -
>> -	/*
>> -	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
>> -	 * with 64 pads or less can be supported.
>> -	 */
>> -	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>> -		return -EOPNOTSUPP;
>> -
>> -	if (sd->enabled_pads & BIT_ULL(pad)) {
>> -		dev_dbg(dev, "pad %u already enabled on %s\n",
>> -			pad, sd->entity.name);
>> -		return -EALREADY;
>> -	}
>> -
>> -	/* Start streaming when the first pad is enabled. */
>> -	if (!sd->enabled_pads) {
>> -		ret = v4l2_subdev_call(sd, video, s_stream, 1);
>> -		if (ret)
>> -			return ret;
>> -	}
>> -
>> -	sd->enabled_pads |= BIT_ULL(pad);
>> -
>> -	return 0;
>> -}
>> -
>>   int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   			       u64 streams_mask)
>>   {
>> @@ -2189,21 +2159,33 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   	bool already_streaming;
>>   	u64 enabled_streams;
>>   	u64 found_streams;
>> +	bool use_s_stream;
>>   	int ret;
>>   
>>   	/* A few basic sanity checks first. */
>>   	if (pad >= sd->entity.num_pads)
>>   		return -EINVAL;
>>   
>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>> +		return -EOPNOTSUPP;
>> +
>> +	/*
>> +	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
>> +	 * with 64 pads or less can be supported.
>> +	 */
>> +	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>> +		return -EOPNOTSUPP;
>> +
>>   	if (!streams_mask)
>>   		return 0;
>>   
>>   	/* Fallback on .s_stream() if .enable_streams() isn't available. */
>> -	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
>> -		return v4l2_subdev_enable_streams_fallback(sd, pad,
>> -							   streams_mask);
>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, enable_streams);
>>   
>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
>> +	if (!use_s_stream)
>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
>> +	else
>> +		state = NULL;
>>   
>>   	/*
>>   	 * Verify that the requested streams exist and that they are not
>> @@ -2231,9 +2213,18 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   
>>   	already_streaming = v4l2_subdev_is_streaming(sd);
>>   
>> -	/* Call the .enable_streams() operation. */
>> -	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>> -			       streams_mask);
>> +	if (!use_s_stream) {
>> +		/* Call the .enable_streams() operation. */
>> +		ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>> +				       streams_mask);
>> +	} else {
>> +		/* Start streaming when the first pad is enabled. */
>> +		if (!already_streaming)
>> +			ret = v4l2_subdev_call(sd, video, s_stream, 1);
>> +		else
>> +			ret = 0;
>> +	}
>> +
>>   	if (ret) {
>>   		dev_dbg(dev, "enable streams %u:%#llx failed: %d\n", pad,
>>   			streams_mask, ret);
>> @@ -2243,34 +2234,32 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   	/* Mark the streams as enabled. */
>>   	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
> 
> state gets set to a non-null value above if the subdev has an
> enable_streams operation, while in v4l2_subdev_set_streams_enabled(),

That's not quite true, v4l2_subdev_lock_and_get_active_state() returns 
NULL if there's no state. If the driver has .enable_streams but no 
subdev-state, we'll have NULL state. But is that a driver bug? Probably, 
although I'm not quite sure. Whereas V4L2_SUBDEV_FL_STREAMS without 
state is a driver bug.

> you use it if the V4L2_SUBDEV_FL_STREAMS flag is set. The second
> condition is a subset of the first one, so it should be safe now, but I
> wonder if we're at risk of crashes in the future when reworking the
> code. I'm probably worrying needlessly.

I agree that this is a bit messy. We have multiple different scenarios 
which we handle in the same functions, and it's somewhat unclear what 
op/state/flag combinations the drivers are allowed to have and what not.

I could add a !state check here, and then use sd->enabled_pads. But that 
doesn't feel correct, and I think it's just better to crash and then fix 
it if we end up there.

>>   
>> -	if (!already_streaming)
>> +	if (!use_s_stream && !already_streaming)
>>   		v4l2_subdev_enable_privacy_led(sd);
> 
> Once everybody switches to v4l2_subdev_enable_streams() (am I dreaming?)
> we should move LED handling from the .s_stream() wrapper to here for the
> fallback case too. How about a TODO comment ?

Let's get back to this after we figure out if s_stream can ever be removed.

>>   
>>   done:
>> -	v4l2_subdev_unlock_state(state);
>> +	if (!use_s_stream)
> 
> 	if (state)
> 
> would be better I think.

We get the state and state lock if (!use_s_stream), so I think it makes 
sense to use the same condition when unlocking to be symmetric. I'm sure 
I can be convinced to use if (state) too, though =).

  Tomi

>> +		v4l2_subdev_unlock_state(state);
>>   
>>   	return ret;
>>   }
>>   EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
>>   
>> -static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>> -						u64 streams_mask)
>> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>> +				u64 streams_mask)
>>   {
>>   	struct device *dev = sd->entity.graph_obj.mdev->dev;
>> +	struct v4l2_subdev_state *state;
>> +	u64 enabled_streams;
>> +	u64 found_streams;
>> +	bool use_s_stream;
>>   	int ret;
>>   
>> -	/*
>> -	 * If the subdev doesn't implement pad-based stream enable, fall back
>> -	 * to the .s_stream() operation.
>> -	 */
>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>> -		return -EOPNOTSUPP;
>> +	/* A few basic sanity checks first. */
>> +	if (pad >= sd->entity.num_pads)
>> +		return -EINVAL;
>>   
>> -	/*
>> -	 * .s_stream() means there is no streams support, so the only allowed
>> -	 * stream is the implicit stream 0.
>> -	 */
>> -	if (streams_mask != BIT_ULL(0))
>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>   		return -EOPNOTSUPP;
>>   
>>   	/*
>> @@ -2280,46 +2269,16 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>   	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>>   		return -EOPNOTSUPP;
>>   
>> -	if (!(sd->enabled_pads & BIT_ULL(pad))) {
>> -		dev_dbg(dev, "pad %u already disabled on %s\n",
>> -			pad, sd->entity.name);
>> -		return -EALREADY;
>> -	}
>> -
>> -	/* Stop streaming when the last streams are disabled. */
>> -	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
>> -		ret = v4l2_subdev_call(sd, video, s_stream, 0);
>> -		if (ret)
>> -			return ret;
>> -	}
>> -
>> -	sd->enabled_pads &= ~BIT_ULL(pad);
>> -
>> -	return 0;
>> -}
>> -
>> -int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>> -				u64 streams_mask)
>> -{
>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
>> -	struct v4l2_subdev_state *state;
>> -	u64 enabled_streams;
>> -	u64 found_streams;
>> -	int ret;
>> -
>> -	/* A few basic sanity checks first. */
>> -	if (pad >= sd->entity.num_pads)
>> -		return -EINVAL;
>> -
>>   	if (!streams_mask)
>>   		return 0;
>>   
>>   	/* Fallback on .s_stream() if .disable_streams() isn't available. */
>> -	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
>> -		return v4l2_subdev_disable_streams_fallback(sd, pad,
>> -							    streams_mask);
>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, disable_streams);
>>   
>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
>> +	if (!use_s_stream)
>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
>> +	else
>> +		state = NULL;
>>   
>>   	/*
>>   	 * Verify that the requested streams exist and that they are not
>> @@ -2345,9 +2304,19 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>   
>>   	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
>>   
>> -	/* Call the .disable_streams() operation. */
>> -	ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
>> -			       streams_mask);
>> +	if (!use_s_stream) {
>> +		/* Call the .disable_streams() operation. */
>> +		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
>> +				       streams_mask);
>> +	} else {
>> +		/* Stop streaming when the last streams are disabled. */
>> +
>> +		if (!(sd->enabled_pads & ~BIT_ULL(pad)))
>> +			ret = v4l2_subdev_call(sd, video, s_stream, 0);
>> +		else
>> +			ret = 0;
>> +	}
>> +
>>   	if (ret) {
>>   		dev_dbg(dev, "disable streams %u:%#llx failed: %d\n", pad,
>>   			streams_mask, ret);
>> @@ -2357,10 +2326,12 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>   	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
>>   
>>   done:
>> -	if (!v4l2_subdev_is_streaming(sd))
>> -		v4l2_subdev_disable_privacy_led(sd);
>> +	if (!use_s_stream) {
>> +		if (!v4l2_subdev_is_streaming(sd))
>> +			v4l2_subdev_disable_privacy_led(sd);
> 
> Do we want to disable the privacy LED on failure in all cases, in
> particular when the .s_stream() or .disable_streams() operations are not
> even called ?
> 
> And now that I wrote that, I realize the !v4l2_subdev_is_streaming()
> condition will not be true if the operations failed, so it should be
> fine.
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
>>   
>> -	v4l2_subdev_unlock_state(state);
>> +		v4l2_subdev_unlock_state(state);
>> +	}
>>   
>>   	return ret;
>>   }
>>
> 

Re: [PATCH v5 08/10] media: subdev: Refactor v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

On 24/04/2024 11:41, Hans Verkuil wrote:
> On 16/04/2024 15:55, Tomi Valkeinen wrote:
>> Add two internal helper functions, v4l2_subdev_collect_streams() and
>> v4l2_subdev_set_streams_enabled(), which allows us to refactor
>> v4l2_subdev_enable/disable_streams() functions.
>>
>> This (I think) makes the code a bit easier to read, and lets us more
>> easily add new functionality in the helper functions in the following
>> patch.
>>
>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>> ---
>>   drivers/media/v4l2-core/v4l2-subdev.c | 109 +++++++++++++++++++---------------
>>   1 file changed, 60 insertions(+), 49 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>> index 38388b223564..e45fd42da1e3 100644
>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>> @@ -2100,6 +2100,42 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd,
>>   }
>>   EXPORT_SYMBOL_GPL(v4l2_subdev_routing_validate);
>>   
>> +static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>> +					struct v4l2_subdev_state *state,
>> +					u32 pad, u64 streams_mask,
>> +					u64 *found_streams,
>> +					u64 *enabled_streams)
>> +{
>> +	*found_streams = 0;
>> +	*enabled_streams = 0;
>> +
>> +	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>> +		const struct v4l2_subdev_stream_config *cfg =
>> +			&state->stream_configs.configs[i];
>> +
>> +		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
>> +			continue;
>> +
>> +		*found_streams |= BIT_ULL(cfg->stream);
>> +		if (cfg->enabled)
>> +			*enabled_streams |= BIT_ULL(cfg->stream);
>> +	}
>> +}
>> +
>> +static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>> +					    struct v4l2_subdev_state *state,
>> +					    u32 pad, u64 streams_mask,
>> +					    bool enabled)
>> +{
>> +	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>> +		struct v4l2_subdev_stream_config *cfg =
>> +			&state->stream_configs.configs[i];
>> +
>> +		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
>> +			cfg->enabled = enabled;
>> +	}
>> +}
>> +
>>   static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>   					       u64 streams_mask)
>>   {
>> @@ -2151,8 +2187,8 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>   	struct v4l2_subdev_state *state;
>>   	bool already_streaming;
>> -	u64 found_streams = 0;
>> -	unsigned int i;
>> +	u64 enabled_streams;
>> +	u64 found_streams;
>>   	int ret;
>>   
>>   	/* A few basic sanity checks first. */
>> @@ -2173,22 +2209,9 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   	 * Verify that the requested streams exist and that they are not
>>   	 * already enabled.
>>   	 */
>> -	for (i = 0; i < state->stream_configs.num_configs; ++i) {
>> -		struct v4l2_subdev_stream_config *cfg =
>> -			&state->stream_configs.configs[i];
>>   
>> -		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
>> -			continue;
>> -
>> -		found_streams |= BIT_ULL(cfg->stream);
>> -
>> -		if (cfg->enabled) {
>> -			dev_dbg(dev, "stream %u already enabled on %s:%u\n",
>> -				cfg->stream, sd->entity.name, pad);
>> -			ret = -EALREADY;
>> -			goto done;
>> -		}
>> -	}
>> +	v4l2_subdev_collect_streams(sd, state, pad, streams_mask,
>> +				    &found_streams, &enabled_streams);
>>   
>>   	if (found_streams != streams_mask) {
>>   		dev_dbg(dev, "streams 0x%llx not found on %s:%u\n",
>> @@ -2197,6 +2220,13 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   		goto done;
>>   	}
>>   
>> +	if (enabled_streams) {
>> +		dev_dbg(dev, "streams 0x%llx already enabled on %s:%u\n",
>> +			enabled_streams, sd->entity.name, pad);
>> +		ret = -EINVAL;
>> +		goto done;
>> +	}
>> +
>>   	dev_dbg(dev, "enable streams %u:%#llx\n", pad, streams_mask);
>>   
>>   	already_streaming = v4l2_subdev_is_streaming(sd);
>> @@ -2211,13 +2241,7 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>   	}
>>   
>>   	/* Mark the streams as enabled. */
>> -	for (i = 0; i < state->stream_configs.num_configs; ++i) {
>> -		struct v4l2_subdev_stream_config *cfg =
>> -			&state->stream_configs.configs[i];
>> -
>> -		if (cfg->pad == pad && (streams_mask & BIT_ULL(cfg->stream)))
>> -			cfg->enabled = true;
>> -	}
>> +	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
>>   
>>   	if (!already_streaming)
>>   		v4l2_subdev_enable_privacy_led(sd);
>> @@ -2279,8 +2303,8 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>   {
>>   	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>   	struct v4l2_subdev_state *state;
>> -	u64 found_streams = 0;
>> -	unsigned int i;
>> +	u64 enabled_streams;
>> +	u64 found_streams;
>>   	int ret;
>>   
>>   	/* A few basic sanity checks first. */
>> @@ -2301,22 +2325,9 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>   	 * Verify that the requested streams exist and that they are not
>>   	 * already disabled.
>>   	 */
>> -	for (i = 0; i < state->stream_configs.num_configs; ++i) {
>> -		struct v4l2_subdev_stream_config *cfg =
>> -			&state->stream_configs.configs[i];
>> -
>> -		if (cfg->pad != pad || !(streams_mask & BIT_ULL(cfg->stream)))
>> -			continue;
>>   
>> -		found_streams |= BIT_ULL(cfg->stream);
>> -
>> -		if (!cfg->enabled) {
>> -			dev_dbg(dev, "stream %u already disabled on %s:%u\n",
>> -				cfg->stream, sd->entity.name, pad);
>> -			ret = -EALREADY;
> 
> This used to return -EALREADY...
> 
>> -			goto done;
>> -		}
>> -	}
>> +	v4l2_subdev_collect_streams(sd, state, pad, streams_mask,
>> +				    &found_streams, &enabled_streams);
>>   
>>   	if (found_streams != streams_mask) {
>>   		dev_dbg(dev, "streams 0x%llx not found on %s:%u\n",
>> @@ -2325,6 +2336,13 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>   		goto done;
>>   	}
>>   
>> +	if (enabled_streams != streams_mask) {
>> +		dev_dbg(dev, "streams 0x%llx already disabled on %s:%u\n",
>> +			streams_mask & ~enabled_streams, sd->entity.name, pad);
>> +		ret = -EINVAL;
> 
> ...but now it returns -EINVAL.
> 
> Is that intentional?

No. I think it was a copy-paste thing.

> I prefer EINVAL to be honest, but I was just wondering about this change.
> 
> It looks like the next patch removes the last of the EALREADY error returns
> as well.

Good catch. We document that the enable_streams() returns EALREADY if a 
stream is already enabled, so lets stick to that. I'll update the patches.

  Tomi

Re: [PATCH v5 09/10] media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()

[Laurent Pinchart]

On Wed, Apr 24, 2024 at 04:05:31PM +0300, Tomi Valkeinen wrote:
> On 19/04/2024 18:36, Laurent Pinchart wrote:
> > On Tue, Apr 16, 2024 at 04:55:12PM +0300, Tomi Valkeinen wrote:
> >> At the moment the v4l2_subdev_enable/disable_streams() functions call
> >> fallback helpers to handle the case where the subdev only implements
> >> .s_stream(), and the main function handles the case where the subdev
> >> implements streams (V4L2_SUBDEV_FL_STREAMS, which implies
> >> .enable/disable_streams()).
> >>
> >> What is missing is support for subdevs which do not implement streams
> >> support, but do implement .enable/disable_streams(). Example cases of
> >> these subdevices are single-stream cameras, where using
> >> .enable/disable_streams() is not required but helps us remove the users
> >> of the legacy .s_stream(), and subdevices with multiple source pads (but
> >> single stream per pad), where .enable/disable_streams() allows the
> >> subdevice to control the enable/disable state per pad.
> >>
> >> The two single-streams cases (.s_stream() and .enable/disable_streams())
> >> are very similar, and with small changes we can change the
> >> v4l2_subdev_enable/disable_streams() functions to support all three
> >> cases, without needing separate fallback functions.
> >>
> >> A few potentially problematic details, though:
> >>
> >> - For the single-streams cases we use sd->enabled_pads field, which
> >>    limits the number of pads for the subdevice to 64. For simplicity I
> >>    added the check for this limitation to the beginning of the function,
> >>    and it also applies to the streams case.
> >>
> >> - The fallback functions only allowed the target pad to be a source pad.
> >>    It is not very clear to me why this check was needed, but it was not
> >>    needed in the streams case. However, I doubt the
> >>    v4l2_subdev_enable/disable_streams() code has ever been tested with
> >>    sink pads, so to be on the safe side, I added the same check
> >>    to the v4l2_subdev_enable/disable_streams() functions.
> >>
> > 
> > Now that v4l2_subdev_enable_streams() is usable by (almost) every
> > driver, should we update documentation to indicate that manual calls to
> > .s_stream() should be avoided ?
> 
> How about:
> 
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index dabe1b5dfe4a..f52bb790773c 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -450,6 +450,11 @@ enum v4l2_subdev_pre_streamon_flags {
>   *     already started or stopped subdev. Also see call_s_stream wrapper in
>   *     v4l2-subdev.c.
>   *
> + *     New drivers should instead implement &v4l2_subdev_pad_ops.enable_streams
> + *     and &v4l2_subdev_pad_ops.disable_streams operations, and use
> + *     v4l2_subdev_s_stream_helper for the &v4l2_subdev_video_ops.s_stream
> + *     operation to support legacy users.A
> + *

Add

 *	Drivers should also not call the .s_stream() subdev operation directly,
 *	but use the v4l2_subdev_enable_streams() and
 *	v4l2_subdev_disable_streams() helpers.

>   * @g_pixelaspect: callback to return the pixelaspect ratio.
>   *
>   * @s_dv_timings: Set custom dv timings in the sub device. This is used
> 
> 
> Although now that I think about it, can we "ever" get rid of s_stream? 
> enable_streams is only usable if the subdev has pads.

Do we have pad-less subdevs that can stream ?

> If that's the 
> case, the text above should probably clarify when one should continue 
> using s_stream.
> 
> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >> ---
> >>   drivers/media/v4l2-core/v4l2-subdev.c | 187 ++++++++++++++--------------------
> >>   1 file changed, 79 insertions(+), 108 deletions(-)
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> >> index e45fd42da1e3..10406acfb9d0 100644
> >> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> >> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> >> @@ -2106,6 +2106,13 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
> >>   					u64 *found_streams,
> >>   					u64 *enabled_streams)
> >>   {
> >> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
> >> +		*found_streams = BIT_ULL(0);
> >> +		*enabled_streams =
> >> +			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
> >> +		return;
> >> +	}
> >> +
> >>   	*found_streams = 0;
> >>   	*enabled_streams = 0;
> >>   
> >> @@ -2127,6 +2134,14 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> >>   					    u32 pad, u64 streams_mask,
> >>   					    bool enabled)
> >>   {
> >> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
> >> +		if (enabled)
> >> +			sd->enabled_pads |= BIT_ULL(pad);
> >> +		else
> >> +			sd->enabled_pads &= ~BIT_ULL(pad);
> >> +		return;
> >> +	}
> >> +
> >>   	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> >>   		struct v4l2_subdev_stream_config *cfg =
> >>   			&state->stream_configs.configs[i];
> >> @@ -2136,51 +2151,6 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> >>   	}
> >>   }
> >>   
> >> -static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> >> -					       u64 streams_mask)
> >> -{
> >> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
> >> -	int ret;
> >> -
> >> -	/*
> >> -	 * The subdev doesn't implement pad-based stream enable, fall back
> >> -	 * to the .s_stream() operation.
> >> -	 */
> >> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >> -		return -EOPNOTSUPP;
> >> -
> >> -	/*
> >> -	 * .s_stream() means there is no streams support, so the only allowed
> >> -	 * stream is the implicit stream 0.
> >> -	 */
> >> -	if (streams_mask != BIT_ULL(0))
> >> -		return -EOPNOTSUPP;
> >> -
> >> -	/*
> >> -	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
> >> -	 * with 64 pads or less can be supported.
> >> -	 */
> >> -	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> >> -		return -EOPNOTSUPP;
> >> -
> >> -	if (sd->enabled_pads & BIT_ULL(pad)) {
> >> -		dev_dbg(dev, "pad %u already enabled on %s\n",
> >> -			pad, sd->entity.name);
> >> -		return -EALREADY;
> >> -	}
> >> -
> >> -	/* Start streaming when the first pad is enabled. */
> >> -	if (!sd->enabled_pads) {
> >> -		ret = v4l2_subdev_call(sd, video, s_stream, 1);
> >> -		if (ret)
> >> -			return ret;
> >> -	}
> >> -
> >> -	sd->enabled_pads |= BIT_ULL(pad);
> >> -
> >> -	return 0;
> >> -}
> >> -
> >>   int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>   			       u64 streams_mask)
> >>   {
> >> @@ -2189,21 +2159,33 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>   	bool already_streaming;
> >>   	u64 enabled_streams;
> >>   	u64 found_streams;
> >> +	bool use_s_stream;
> >>   	int ret;
> >>   
> >>   	/* A few basic sanity checks first. */
> >>   	if (pad >= sd->entity.num_pads)
> >>   		return -EINVAL;
> >>   
> >> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >> +		return -EOPNOTSUPP;
> >> +
> >> +	/*
> >> +	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
> >> +	 * with 64 pads or less can be supported.
> >> +	 */
> >> +	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> >> +		return -EOPNOTSUPP;
> >> +
> >>   	if (!streams_mask)
> >>   		return 0;
> >>   
> >>   	/* Fallback on .s_stream() if .enable_streams() isn't available. */
> >> -	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
> >> -		return v4l2_subdev_enable_streams_fallback(sd, pad,
> >> -							   streams_mask);
> >> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, enable_streams);
> >>   
> >> -	state = v4l2_subdev_lock_and_get_active_state(sd);
> >> +	if (!use_s_stream)
> >> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> >> +	else
> >> +		state = NULL;
> >>   
> >>   	/*
> >>   	 * Verify that the requested streams exist and that they are not
> >> @@ -2231,9 +2213,18 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>   
> >>   	already_streaming = v4l2_subdev_is_streaming(sd);
> >>   
> >> -	/* Call the .enable_streams() operation. */
> >> -	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
> >> -			       streams_mask);
> >> +	if (!use_s_stream) {
> >> +		/* Call the .enable_streams() operation. */
> >> +		ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
> >> +				       streams_mask);
> >> +	} else {
> >> +		/* Start streaming when the first pad is enabled. */
> >> +		if (!already_streaming)
> >> +			ret = v4l2_subdev_call(sd, video, s_stream, 1);
> >> +		else
> >> +			ret = 0;
> >> +	}
> >> +
> >>   	if (ret) {
> >>   		dev_dbg(dev, "enable streams %u:%#llx failed: %d\n", pad,
> >>   			streams_mask, ret);
> >> @@ -2243,34 +2234,32 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>   	/* Mark the streams as enabled. */
> >>   	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
> > 
> > state gets set to a non-null value above if the subdev has an
> > enable_streams operation, while in v4l2_subdev_set_streams_enabled(),
> 
> That's not quite true, v4l2_subdev_lock_and_get_active_state() returns 
> NULL if there's no state. If the driver has .enable_streams but no 
> subdev-state, we'll have NULL state. But is that a driver bug? Probably, 
> although I'm not quite sure. Whereas V4L2_SUBDEV_FL_STREAMS without 
> state is a driver bug.
> 
> > you use it if the V4L2_SUBDEV_FL_STREAMS flag is set. The second
> > condition is a subset of the first one, so it should be safe now, but I
> > wonder if we're at risk of crashes in the future when reworking the
> > code. I'm probably worrying needlessly.
> 
> I agree that this is a bit messy. We have multiple different scenarios 
> which we handle in the same functions, and it's somewhat unclear what 
> op/state/flag combinations the drivers are allowed to have and what not.
> 
> I could add a !state check here, and then use sd->enabled_pads. But that 
> doesn't feel correct, and I think it's just better to crash and then fix 
> it if we end up there.
> 
> >>   
> >> -	if (!already_streaming)
> >> +	if (!use_s_stream && !already_streaming)
> >>   		v4l2_subdev_enable_privacy_led(sd);
> > 
> > Once everybody switches to v4l2_subdev_enable_streams() (am I dreaming?)
> > we should move LED handling from the .s_stream() wrapper to here for the
> > fallback case too. How about a TODO comment ?
> 
> Let's get back to this after we figure out if s_stream can ever be removed.
> 
> >>   
> >>   done:
> >> -	v4l2_subdev_unlock_state(state);
> >> +	if (!use_s_stream)
> > 
> > 	if (state)
> > 
> > would be better I think.
> 
> We get the state and state lock if (!use_s_stream), so I think it makes 
> sense to use the same condition when unlocking to be symmetric. I'm sure 
> I can be convinced to use if (state) too, though =).
> 
>   Tomi
> 
> >> +		v4l2_subdev_unlock_state(state);
> >>   
> >>   	return ret;
> >>   }
> >>   EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
> >>   
> >> -static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> >> -						u64 streams_mask)
> >> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >> +				u64 streams_mask)
> >>   {
> >>   	struct device *dev = sd->entity.graph_obj.mdev->dev;
> >> +	struct v4l2_subdev_state *state;
> >> +	u64 enabled_streams;
> >> +	u64 found_streams;
> >> +	bool use_s_stream;
> >>   	int ret;
> >>   
> >> -	/*
> >> -	 * If the subdev doesn't implement pad-based stream enable, fall back
> >> -	 * to the .s_stream() operation.
> >> -	 */
> >> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >> -		return -EOPNOTSUPP;
> >> +	/* A few basic sanity checks first. */
> >> +	if (pad >= sd->entity.num_pads)
> >> +		return -EINVAL;
> >>   
> >> -	/*
> >> -	 * .s_stream() means there is no streams support, so the only allowed
> >> -	 * stream is the implicit stream 0.
> >> -	 */
> >> -	if (streams_mask != BIT_ULL(0))
> >> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >>   		return -EOPNOTSUPP;
> >>   
> >>   	/*
> >> @@ -2280,46 +2269,16 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> >>   	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> >>   		return -EOPNOTSUPP;
> >>   
> >> -	if (!(sd->enabled_pads & BIT_ULL(pad))) {
> >> -		dev_dbg(dev, "pad %u already disabled on %s\n",
> >> -			pad, sd->entity.name);
> >> -		return -EALREADY;
> >> -	}
> >> -
> >> -	/* Stop streaming when the last streams are disabled. */
> >> -	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
> >> -		ret = v4l2_subdev_call(sd, video, s_stream, 0);
> >> -		if (ret)
> >> -			return ret;
> >> -	}
> >> -
> >> -	sd->enabled_pads &= ~BIT_ULL(pad);
> >> -
> >> -	return 0;
> >> -}
> >> -
> >> -int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >> -				u64 streams_mask)
> >> -{
> >> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
> >> -	struct v4l2_subdev_state *state;
> >> -	u64 enabled_streams;
> >> -	u64 found_streams;
> >> -	int ret;
> >> -
> >> -	/* A few basic sanity checks first. */
> >> -	if (pad >= sd->entity.num_pads)
> >> -		return -EINVAL;
> >> -
> >>   	if (!streams_mask)
> >>   		return 0;
> >>   
> >>   	/* Fallback on .s_stream() if .disable_streams() isn't available. */
> >> -	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
> >> -		return v4l2_subdev_disable_streams_fallback(sd, pad,
> >> -							    streams_mask);
> >> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, disable_streams);
> >>   
> >> -	state = v4l2_subdev_lock_and_get_active_state(sd);
> >> +	if (!use_s_stream)
> >> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> >> +	else
> >> +		state = NULL;
> >>   
> >>   	/*
> >>   	 * Verify that the requested streams exist and that they are not
> >> @@ -2345,9 +2304,19 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >>   
> >>   	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
> >>   
> >> -	/* Call the .disable_streams() operation. */
> >> -	ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
> >> -			       streams_mask);
> >> +	if (!use_s_stream) {
> >> +		/* Call the .disable_streams() operation. */
> >> +		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
> >> +				       streams_mask);
> >> +	} else {
> >> +		/* Stop streaming when the last streams are disabled. */
> >> +
> >> +		if (!(sd->enabled_pads & ~BIT_ULL(pad)))
> >> +			ret = v4l2_subdev_call(sd, video, s_stream, 0);
> >> +		else
> >> +			ret = 0;
> >> +	}
> >> +
> >>   	if (ret) {
> >>   		dev_dbg(dev, "disable streams %u:%#llx failed: %d\n", pad,
> >>   			streams_mask, ret);
> >> @@ -2357,10 +2326,12 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >>   	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
> >>   
> >>   done:
> >> -	if (!v4l2_subdev_is_streaming(sd))
> >> -		v4l2_subdev_disable_privacy_led(sd);
> >> +	if (!use_s_stream) {
> >> +		if (!v4l2_subdev_is_streaming(sd))
> >> +			v4l2_subdev_disable_privacy_led(sd);
> > 
> > Do we want to disable the privacy LED on failure in all cases, in
> > particular when the .s_stream() or .disable_streams() operations are not
> > even called ?
> > 
> > And now that I wrote that, I realize the !v4l2_subdev_is_streaming()
> > condition will not be true if the operations failed, so it should be
> > fine.
> > 
> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> >>   
> >> -	v4l2_subdev_unlock_state(state);
> >> +		v4l2_subdev_unlock_state(state);
> >> +	}
> >>   
> >>   	return ret;
> >>   }
> >>
> > 
> 

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 09/10] media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

On 24/04/2024 16:09, Laurent Pinchart wrote:
> On Wed, Apr 24, 2024 at 04:05:31PM +0300, Tomi Valkeinen wrote:
>> On 19/04/2024 18:36, Laurent Pinchart wrote:
>>> On Tue, Apr 16, 2024 at 04:55:12PM +0300, Tomi Valkeinen wrote:
>>>> At the moment the v4l2_subdev_enable/disable_streams() functions call
>>>> fallback helpers to handle the case where the subdev only implements
>>>> .s_stream(), and the main function handles the case where the subdev
>>>> implements streams (V4L2_SUBDEV_FL_STREAMS, which implies
>>>> .enable/disable_streams()).
>>>>
>>>> What is missing is support for subdevs which do not implement streams
>>>> support, but do implement .enable/disable_streams(). Example cases of
>>>> these subdevices are single-stream cameras, where using
>>>> .enable/disable_streams() is not required but helps us remove the users
>>>> of the legacy .s_stream(), and subdevices with multiple source pads (but
>>>> single stream per pad), where .enable/disable_streams() allows the
>>>> subdevice to control the enable/disable state per pad.
>>>>
>>>> The two single-streams cases (.s_stream() and .enable/disable_streams())
>>>> are very similar, and with small changes we can change the
>>>> v4l2_subdev_enable/disable_streams() functions to support all three
>>>> cases, without needing separate fallback functions.
>>>>
>>>> A few potentially problematic details, though:
>>>>
>>>> - For the single-streams cases we use sd->enabled_pads field, which
>>>>     limits the number of pads for the subdevice to 64. For simplicity I
>>>>     added the check for this limitation to the beginning of the function,
>>>>     and it also applies to the streams case.
>>>>
>>>> - The fallback functions only allowed the target pad to be a source pad.
>>>>     It is not very clear to me why this check was needed, but it was not
>>>>     needed in the streams case. However, I doubt the
>>>>     v4l2_subdev_enable/disable_streams() code has ever been tested with
>>>>     sink pads, so to be on the safe side, I added the same check
>>>>     to the v4l2_subdev_enable/disable_streams() functions.
>>>>
>>>
>>> Now that v4l2_subdev_enable_streams() is usable by (almost) every
>>> driver, should we update documentation to indicate that manual calls to
>>> .s_stream() should be avoided ?
>>
>> How about:
>>
>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>> index dabe1b5dfe4a..f52bb790773c 100644
>> --- a/include/media/v4l2-subdev.h
>> +++ b/include/media/v4l2-subdev.h
>> @@ -450,6 +450,11 @@ enum v4l2_subdev_pre_streamon_flags {
>>    *     already started or stopped subdev. Also see call_s_stream wrapper in
>>    *     v4l2-subdev.c.
>>    *
>> + *     New drivers should instead implement &v4l2_subdev_pad_ops.enable_streams
>> + *     and &v4l2_subdev_pad_ops.disable_streams operations, and use
>> + *     v4l2_subdev_s_stream_helper for the &v4l2_subdev_video_ops.s_stream
>> + *     operation to support legacy users.A
>> + *
> 
> Add
> 
>   *	Drivers should also not call the .s_stream() subdev operation directly,
>   *	but use the v4l2_subdev_enable_streams() and
>   *	v4l2_subdev_disable_streams() helpers.
> 
>>    * @g_pixelaspect: callback to return the pixelaspect ratio.
>>    *
>>    * @s_dv_timings: Set custom dv timings in the sub device. This is used
>>
>>
>> Although now that I think about it, can we "ever" get rid of s_stream?
>> enable_streams is only usable if the subdev has pads.
> 
> Do we have pad-less subdevs that can stream ?

Probably not, but isn't s_stream used to enable any type of subdev? Or 
is there a different op to power on padless subdevs?

  Tomi

> 
>> If that's the
>> case, the text above should probably clarify when one should continue
>> using s_stream.
>>
>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>> ---
>>>>    drivers/media/v4l2-core/v4l2-subdev.c | 187 ++++++++++++++--------------------
>>>>    1 file changed, 79 insertions(+), 108 deletions(-)
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>>>> index e45fd42da1e3..10406acfb9d0 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>>>> @@ -2106,6 +2106,13 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>>>>    					u64 *found_streams,
>>>>    					u64 *enabled_streams)
>>>>    {
>>>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
>>>> +		*found_streams = BIT_ULL(0);
>>>> +		*enabled_streams =
>>>> +			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
>>>> +		return;
>>>> +	}
>>>> +
>>>>    	*found_streams = 0;
>>>>    	*enabled_streams = 0;
>>>>    
>>>> @@ -2127,6 +2134,14 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>>>>    					    u32 pad, u64 streams_mask,
>>>>    					    bool enabled)
>>>>    {
>>>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
>>>> +		if (enabled)
>>>> +			sd->enabled_pads |= BIT_ULL(pad);
>>>> +		else
>>>> +			sd->enabled_pads &= ~BIT_ULL(pad);
>>>> +		return;
>>>> +	}
>>>> +
>>>>    	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>>>>    		struct v4l2_subdev_stream_config *cfg =
>>>>    			&state->stream_configs.configs[i];
>>>> @@ -2136,51 +2151,6 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>>>>    	}
>>>>    }
>>>>    
>>>> -static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>>> -					       u64 streams_mask)
>>>> -{
>>>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>>> -	int ret;
>>>> -
>>>> -	/*
>>>> -	 * The subdev doesn't implement pad-based stream enable, fall back
>>>> -	 * to the .s_stream() operation.
>>>> -	 */
>>>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>> -		return -EOPNOTSUPP;
>>>> -
>>>> -	/*
>>>> -	 * .s_stream() means there is no streams support, so the only allowed
>>>> -	 * stream is the implicit stream 0.
>>>> -	 */
>>>> -	if (streams_mask != BIT_ULL(0))
>>>> -		return -EOPNOTSUPP;
>>>> -
>>>> -	/*
>>>> -	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
>>>> -	 * with 64 pads or less can be supported.
>>>> -	 */
>>>> -	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>>>> -		return -EOPNOTSUPP;
>>>> -
>>>> -	if (sd->enabled_pads & BIT_ULL(pad)) {
>>>> -		dev_dbg(dev, "pad %u already enabled on %s\n",
>>>> -			pad, sd->entity.name);
>>>> -		return -EALREADY;
>>>> -	}
>>>> -
>>>> -	/* Start streaming when the first pad is enabled. */
>>>> -	if (!sd->enabled_pads) {
>>>> -		ret = v4l2_subdev_call(sd, video, s_stream, 1);
>>>> -		if (ret)
>>>> -			return ret;
>>>> -	}
>>>> -
>>>> -	sd->enabled_pads |= BIT_ULL(pad);
>>>> -
>>>> -	return 0;
>>>> -}
>>>> -
>>>>    int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>    			       u64 streams_mask)
>>>>    {
>>>> @@ -2189,21 +2159,33 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>    	bool already_streaming;
>>>>    	u64 enabled_streams;
>>>>    	u64 found_streams;
>>>> +	bool use_s_stream;
>>>>    	int ret;
>>>>    
>>>>    	/* A few basic sanity checks first. */
>>>>    	if (pad >= sd->entity.num_pads)
>>>>    		return -EINVAL;
>>>>    
>>>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>> +		return -EOPNOTSUPP;
>>>> +
>>>> +	/*
>>>> +	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
>>>> +	 * with 64 pads or less can be supported.
>>>> +	 */
>>>> +	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>>>> +		return -EOPNOTSUPP;
>>>> +
>>>>    	if (!streams_mask)
>>>>    		return 0;
>>>>    
>>>>    	/* Fallback on .s_stream() if .enable_streams() isn't available. */
>>>> -	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
>>>> -		return v4l2_subdev_enable_streams_fallback(sd, pad,
>>>> -							   streams_mask);
>>>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, enable_streams);
>>>>    
>>>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
>>>> +	if (!use_s_stream)
>>>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
>>>> +	else
>>>> +		state = NULL;
>>>>    
>>>>    	/*
>>>>    	 * Verify that the requested streams exist and that they are not
>>>> @@ -2231,9 +2213,18 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>    
>>>>    	already_streaming = v4l2_subdev_is_streaming(sd);
>>>>    
>>>> -	/* Call the .enable_streams() operation. */
>>>> -	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>>>> -			       streams_mask);
>>>> +	if (!use_s_stream) {
>>>> +		/* Call the .enable_streams() operation. */
>>>> +		ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>>>> +				       streams_mask);
>>>> +	} else {
>>>> +		/* Start streaming when the first pad is enabled. */
>>>> +		if (!already_streaming)
>>>> +			ret = v4l2_subdev_call(sd, video, s_stream, 1);
>>>> +		else
>>>> +			ret = 0;
>>>> +	}
>>>> +
>>>>    	if (ret) {
>>>>    		dev_dbg(dev, "enable streams %u:%#llx failed: %d\n", pad,
>>>>    			streams_mask, ret);
>>>> @@ -2243,34 +2234,32 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>    	/* Mark the streams as enabled. */
>>>>    	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
>>>
>>> state gets set to a non-null value above if the subdev has an
>>> enable_streams operation, while in v4l2_subdev_set_streams_enabled(),
>>
>> That's not quite true, v4l2_subdev_lock_and_get_active_state() returns
>> NULL if there's no state. If the driver has .enable_streams but no
>> subdev-state, we'll have NULL state. But is that a driver bug? Probably,
>> although I'm not quite sure. Whereas V4L2_SUBDEV_FL_STREAMS without
>> state is a driver bug.
>>
>>> you use it if the V4L2_SUBDEV_FL_STREAMS flag is set. The second
>>> condition is a subset of the first one, so it should be safe now, but I
>>> wonder if we're at risk of crashes in the future when reworking the
>>> code. I'm probably worrying needlessly.
>>
>> I agree that this is a bit messy. We have multiple different scenarios
>> which we handle in the same functions, and it's somewhat unclear what
>> op/state/flag combinations the drivers are allowed to have and what not.
>>
>> I could add a !state check here, and then use sd->enabled_pads. But that
>> doesn't feel correct, and I think it's just better to crash and then fix
>> it if we end up there.
>>
>>>>    
>>>> -	if (!already_streaming)
>>>> +	if (!use_s_stream && !already_streaming)
>>>>    		v4l2_subdev_enable_privacy_led(sd);
>>>
>>> Once everybody switches to v4l2_subdev_enable_streams() (am I dreaming?)
>>> we should move LED handling from the .s_stream() wrapper to here for the
>>> fallback case too. How about a TODO comment ?
>>
>> Let's get back to this after we figure out if s_stream can ever be removed.
>>
>>>>    
>>>>    done:
>>>> -	v4l2_subdev_unlock_state(state);
>>>> +	if (!use_s_stream)
>>>
>>> 	if (state)
>>>
>>> would be better I think.
>>
>> We get the state and state lock if (!use_s_stream), so I think it makes
>> sense to use the same condition when unlocking to be symmetric. I'm sure
>> I can be convinced to use if (state) too, though =).
>>
>>    Tomi
>>
>>>> +		v4l2_subdev_unlock_state(state);
>>>>    
>>>>    	return ret;
>>>>    }
>>>>    EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
>>>>    
>>>> -static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>>> -						u64 streams_mask)
>>>> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>> +				u64 streams_mask)
>>>>    {
>>>>    	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>>> +	struct v4l2_subdev_state *state;
>>>> +	u64 enabled_streams;
>>>> +	u64 found_streams;
>>>> +	bool use_s_stream;
>>>>    	int ret;
>>>>    
>>>> -	/*
>>>> -	 * If the subdev doesn't implement pad-based stream enable, fall back
>>>> -	 * to the .s_stream() operation.
>>>> -	 */
>>>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>> -		return -EOPNOTSUPP;
>>>> +	/* A few basic sanity checks first. */
>>>> +	if (pad >= sd->entity.num_pads)
>>>> +		return -EINVAL;
>>>>    
>>>> -	/*
>>>> -	 * .s_stream() means there is no streams support, so the only allowed
>>>> -	 * stream is the implicit stream 0.
>>>> -	 */
>>>> -	if (streams_mask != BIT_ULL(0))
>>>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>>    		return -EOPNOTSUPP;
>>>>    
>>>>    	/*
>>>> @@ -2280,46 +2269,16 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>>>    	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>>>>    		return -EOPNOTSUPP;
>>>>    
>>>> -	if (!(sd->enabled_pads & BIT_ULL(pad))) {
>>>> -		dev_dbg(dev, "pad %u already disabled on %s\n",
>>>> -			pad, sd->entity.name);
>>>> -		return -EALREADY;
>>>> -	}
>>>> -
>>>> -	/* Stop streaming when the last streams are disabled. */
>>>> -	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
>>>> -		ret = v4l2_subdev_call(sd, video, s_stream, 0);
>>>> -		if (ret)
>>>> -			return ret;
>>>> -	}
>>>> -
>>>> -	sd->enabled_pads &= ~BIT_ULL(pad);
>>>> -
>>>> -	return 0;
>>>> -}
>>>> -
>>>> -int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>> -				u64 streams_mask)
>>>> -{
>>>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>>> -	struct v4l2_subdev_state *state;
>>>> -	u64 enabled_streams;
>>>> -	u64 found_streams;
>>>> -	int ret;
>>>> -
>>>> -	/* A few basic sanity checks first. */
>>>> -	if (pad >= sd->entity.num_pads)
>>>> -		return -EINVAL;
>>>> -
>>>>    	if (!streams_mask)
>>>>    		return 0;
>>>>    
>>>>    	/* Fallback on .s_stream() if .disable_streams() isn't available. */
>>>> -	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
>>>> -		return v4l2_subdev_disable_streams_fallback(sd, pad,
>>>> -							    streams_mask);
>>>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, disable_streams);
>>>>    
>>>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
>>>> +	if (!use_s_stream)
>>>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
>>>> +	else
>>>> +		state = NULL;
>>>>    
>>>>    	/*
>>>>    	 * Verify that the requested streams exist and that they are not
>>>> @@ -2345,9 +2304,19 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>    
>>>>    	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
>>>>    
>>>> -	/* Call the .disable_streams() operation. */
>>>> -	ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
>>>> -			       streams_mask);
>>>> +	if (!use_s_stream) {
>>>> +		/* Call the .disable_streams() operation. */
>>>> +		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
>>>> +				       streams_mask);
>>>> +	} else {
>>>> +		/* Stop streaming when the last streams are disabled. */
>>>> +
>>>> +		if (!(sd->enabled_pads & ~BIT_ULL(pad)))
>>>> +			ret = v4l2_subdev_call(sd, video, s_stream, 0);
>>>> +		else
>>>> +			ret = 0;
>>>> +	}
>>>> +
>>>>    	if (ret) {
>>>>    		dev_dbg(dev, "disable streams %u:%#llx failed: %d\n", pad,
>>>>    			streams_mask, ret);
>>>> @@ -2357,10 +2326,12 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>    	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
>>>>    
>>>>    done:
>>>> -	if (!v4l2_subdev_is_streaming(sd))
>>>> -		v4l2_subdev_disable_privacy_led(sd);
>>>> +	if (!use_s_stream) {
>>>> +		if (!v4l2_subdev_is_streaming(sd))
>>>> +			v4l2_subdev_disable_privacy_led(sd);
>>>
>>> Do we want to disable the privacy LED on failure in all cases, in
>>> particular when the .s_stream() or .disable_streams() operations are not
>>> even called ?
>>>
>>> And now that I wrote that, I realize the !v4l2_subdev_is_streaming()
>>> condition will not be true if the operations failed, so it should be
>>> fine.
>>>
>>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>
>>>>    
>>>> -	v4l2_subdev_unlock_state(state);
>>>> +		v4l2_subdev_unlock_state(state);
>>>> +	}
>>>>    
>>>>    	return ret;
>>>>    }
>>>>
>>>
>>
> 

Re: [PATCH v5 09/10] media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()

[Laurent Pinchart]

On Wed, Apr 24, 2024 at 04:11:36PM +0300, Tomi Valkeinen wrote:
> On 24/04/2024 16:09, Laurent Pinchart wrote:
> > On Wed, Apr 24, 2024 at 04:05:31PM +0300, Tomi Valkeinen wrote:
> >> On 19/04/2024 18:36, Laurent Pinchart wrote:
> >>> On Tue, Apr 16, 2024 at 04:55:12PM +0300, Tomi Valkeinen wrote:
> >>>> At the moment the v4l2_subdev_enable/disable_streams() functions call
> >>>> fallback helpers to handle the case where the subdev only implements
> >>>> .s_stream(), and the main function handles the case where the subdev
> >>>> implements streams (V4L2_SUBDEV_FL_STREAMS, which implies
> >>>> .enable/disable_streams()).
> >>>>
> >>>> What is missing is support for subdevs which do not implement streams
> >>>> support, but do implement .enable/disable_streams(). Example cases of
> >>>> these subdevices are single-stream cameras, where using
> >>>> .enable/disable_streams() is not required but helps us remove the users
> >>>> of the legacy .s_stream(), and subdevices with multiple source pads (but
> >>>> single stream per pad), where .enable/disable_streams() allows the
> >>>> subdevice to control the enable/disable state per pad.
> >>>>
> >>>> The two single-streams cases (.s_stream() and .enable/disable_streams())
> >>>> are very similar, and with small changes we can change the
> >>>> v4l2_subdev_enable/disable_streams() functions to support all three
> >>>> cases, without needing separate fallback functions.
> >>>>
> >>>> A few potentially problematic details, though:
> >>>>
> >>>> - For the single-streams cases we use sd->enabled_pads field, which
> >>>>     limits the number of pads for the subdevice to 64. For simplicity I
> >>>>     added the check for this limitation to the beginning of the function,
> >>>>     and it also applies to the streams case.
> >>>>
> >>>> - The fallback functions only allowed the target pad to be a source pad.
> >>>>     It is not very clear to me why this check was needed, but it was not
> >>>>     needed in the streams case. However, I doubt the
> >>>>     v4l2_subdev_enable/disable_streams() code has ever been tested with
> >>>>     sink pads, so to be on the safe side, I added the same check
> >>>>     to the v4l2_subdev_enable/disable_streams() functions.
> >>>>
> >>>
> >>> Now that v4l2_subdev_enable_streams() is usable by (almost) every
> >>> driver, should we update documentation to indicate that manual calls to
> >>> .s_stream() should be avoided ?
> >>
> >> How about:
> >>
> >> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> >> index dabe1b5dfe4a..f52bb790773c 100644
> >> --- a/include/media/v4l2-subdev.h
> >> +++ b/include/media/v4l2-subdev.h
> >> @@ -450,6 +450,11 @@ enum v4l2_subdev_pre_streamon_flags {
> >>    *     already started or stopped subdev. Also see call_s_stream wrapper in
> >>    *     v4l2-subdev.c.
> >>    *
> >> + *     New drivers should instead implement &v4l2_subdev_pad_ops.enable_streams
> >> + *     and &v4l2_subdev_pad_ops.disable_streams operations, and use
> >> + *     v4l2_subdev_s_stream_helper for the &v4l2_subdev_video_ops.s_stream
> >> + *     operation to support legacy users.A
> >> + *
> > 
> > Add
> > 
> >   *	Drivers should also not call the .s_stream() subdev operation directly,
> >   *	but use the v4l2_subdev_enable_streams() and
> >   *	v4l2_subdev_disable_streams() helpers.
> > 
> >>    * @g_pixelaspect: callback to return the pixelaspect ratio.
> >>    *
> >>    * @s_dv_timings: Set custom dv timings in the sub device. This is used
> >>
> >>
> >> Although now that I think about it, can we "ever" get rid of s_stream?
> >> enable_streams is only usable if the subdev has pads.
> > 
> > Do we have pad-less subdevs that can stream ?
> 
> Probably not, but isn't s_stream used to enable any type of subdev? Or 
> is there a different op to power on padless subdevs?

.s_stream() is supposed to start/stop streaming, and you can't really
have streams without pads, as they have to come from/go somewhere.

> >> If that's the
> >> case, the text above should probably clarify when one should continue
> >> using s_stream.
> >>
> >>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
> >>>> ---
> >>>>    drivers/media/v4l2-core/v4l2-subdev.c | 187 ++++++++++++++--------------------
> >>>>    1 file changed, 79 insertions(+), 108 deletions(-)
> >>>>
> >>>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> >>>> index e45fd42da1e3..10406acfb9d0 100644
> >>>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> >>>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> >>>> @@ -2106,6 +2106,13 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
> >>>>    					u64 *found_streams,
> >>>>    					u64 *enabled_streams)
> >>>>    {
> >>>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
> >>>> +		*found_streams = BIT_ULL(0);
> >>>> +		*enabled_streams =
> >>>> +			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
> >>>> +		return;
> >>>> +	}
> >>>> +
> >>>>    	*found_streams = 0;
> >>>>    	*enabled_streams = 0;
> >>>>    
> >>>> @@ -2127,6 +2134,14 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> >>>>    					    u32 pad, u64 streams_mask,
> >>>>    					    bool enabled)
> >>>>    {
> >>>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
> >>>> +		if (enabled)
> >>>> +			sd->enabled_pads |= BIT_ULL(pad);
> >>>> +		else
> >>>> +			sd->enabled_pads &= ~BIT_ULL(pad);
> >>>> +		return;
> >>>> +	}
> >>>> +
> >>>>    	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
> >>>>    		struct v4l2_subdev_stream_config *cfg =
> >>>>    			&state->stream_configs.configs[i];
> >>>> @@ -2136,51 +2151,6 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
> >>>>    	}
> >>>>    }
> >>>>    
> >>>> -static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> >>>> -					       u64 streams_mask)
> >>>> -{
> >>>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
> >>>> -	int ret;
> >>>> -
> >>>> -	/*
> >>>> -	 * The subdev doesn't implement pad-based stream enable, fall back
> >>>> -	 * to the .s_stream() operation.
> >>>> -	 */
> >>>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >>>> -		return -EOPNOTSUPP;
> >>>> -
> >>>> -	/*
> >>>> -	 * .s_stream() means there is no streams support, so the only allowed
> >>>> -	 * stream is the implicit stream 0.
> >>>> -	 */
> >>>> -	if (streams_mask != BIT_ULL(0))
> >>>> -		return -EOPNOTSUPP;
> >>>> -
> >>>> -	/*
> >>>> -	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
> >>>> -	 * with 64 pads or less can be supported.
> >>>> -	 */
> >>>> -	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> >>>> -		return -EOPNOTSUPP;
> >>>> -
> >>>> -	if (sd->enabled_pads & BIT_ULL(pad)) {
> >>>> -		dev_dbg(dev, "pad %u already enabled on %s\n",
> >>>> -			pad, sd->entity.name);
> >>>> -		return -EALREADY;
> >>>> -	}
> >>>> -
> >>>> -	/* Start streaming when the first pad is enabled. */
> >>>> -	if (!sd->enabled_pads) {
> >>>> -		ret = v4l2_subdev_call(sd, video, s_stream, 1);
> >>>> -		if (ret)
> >>>> -			return ret;
> >>>> -	}
> >>>> -
> >>>> -	sd->enabled_pads |= BIT_ULL(pad);
> >>>> -
> >>>> -	return 0;
> >>>> -}
> >>>> -
> >>>>    int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>>    			       u64 streams_mask)
> >>>>    {
> >>>> @@ -2189,21 +2159,33 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>>    	bool already_streaming;
> >>>>    	u64 enabled_streams;
> >>>>    	u64 found_streams;
> >>>> +	bool use_s_stream;
> >>>>    	int ret;
> >>>>    
> >>>>    	/* A few basic sanity checks first. */
> >>>>    	if (pad >= sd->entity.num_pads)
> >>>>    		return -EINVAL;
> >>>>    
> >>>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >>>> +		return -EOPNOTSUPP;
> >>>> +
> >>>> +	/*
> >>>> +	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
> >>>> +	 * with 64 pads or less can be supported.
> >>>> +	 */
> >>>> +	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> >>>> +		return -EOPNOTSUPP;
> >>>> +
> >>>>    	if (!streams_mask)
> >>>>    		return 0;
> >>>>    
> >>>>    	/* Fallback on .s_stream() if .enable_streams() isn't available. */
> >>>> -	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
> >>>> -		return v4l2_subdev_enable_streams_fallback(sd, pad,
> >>>> -							   streams_mask);
> >>>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, enable_streams);
> >>>>    
> >>>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
> >>>> +	if (!use_s_stream)
> >>>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> >>>> +	else
> >>>> +		state = NULL;
> >>>>    
> >>>>    	/*
> >>>>    	 * Verify that the requested streams exist and that they are not
> >>>> @@ -2231,9 +2213,18 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>>    
> >>>>    	already_streaming = v4l2_subdev_is_streaming(sd);
> >>>>    
> >>>> -	/* Call the .enable_streams() operation. */
> >>>> -	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
> >>>> -			       streams_mask);
> >>>> +	if (!use_s_stream) {
> >>>> +		/* Call the .enable_streams() operation. */
> >>>> +		ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
> >>>> +				       streams_mask);
> >>>> +	} else {
> >>>> +		/* Start streaming when the first pad is enabled. */
> >>>> +		if (!already_streaming)
> >>>> +			ret = v4l2_subdev_call(sd, video, s_stream, 1);
> >>>> +		else
> >>>> +			ret = 0;
> >>>> +	}
> >>>> +
> >>>>    	if (ret) {
> >>>>    		dev_dbg(dev, "enable streams %u:%#llx failed: %d\n", pad,
> >>>>    			streams_mask, ret);
> >>>> @@ -2243,34 +2234,32 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>>    	/* Mark the streams as enabled. */
> >>>>    	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
> >>>
> >>> state gets set to a non-null value above if the subdev has an
> >>> enable_streams operation, while in v4l2_subdev_set_streams_enabled(),
> >>
> >> That's not quite true, v4l2_subdev_lock_and_get_active_state() returns
> >> NULL if there's no state. If the driver has .enable_streams but no
> >> subdev-state, we'll have NULL state. But is that a driver bug? Probably,
> >> although I'm not quite sure. Whereas V4L2_SUBDEV_FL_STREAMS without
> >> state is a driver bug.
> >>
> >>> you use it if the V4L2_SUBDEV_FL_STREAMS flag is set. The second
> >>> condition is a subset of the first one, so it should be safe now, but I
> >>> wonder if we're at risk of crashes in the future when reworking the
> >>> code. I'm probably worrying needlessly.
> >>
> >> I agree that this is a bit messy. We have multiple different scenarios
> >> which we handle in the same functions, and it's somewhat unclear what
> >> op/state/flag combinations the drivers are allowed to have and what not.
> >>
> >> I could add a !state check here, and then use sd->enabled_pads. But that
> >> doesn't feel correct, and I think it's just better to crash and then fix
> >> it if we end up there.
> >>
> >>>>    
> >>>> -	if (!already_streaming)
> >>>> +	if (!use_s_stream && !already_streaming)
> >>>>    		v4l2_subdev_enable_privacy_led(sd);
> >>>
> >>> Once everybody switches to v4l2_subdev_enable_streams() (am I dreaming?)
> >>> we should move LED handling from the .s_stream() wrapper to here for the
> >>> fallback case too. How about a TODO comment ?
> >>
> >> Let's get back to this after we figure out if s_stream can ever be removed.
> >>
> >>>>    
> >>>>    done:
> >>>> -	v4l2_subdev_unlock_state(state);
> >>>> +	if (!use_s_stream)
> >>>
> >>> 	if (state)
> >>>
> >>> would be better I think.
> >>
> >> We get the state and state lock if (!use_s_stream), so I think it makes
> >> sense to use the same condition when unlocking to be symmetric. I'm sure
> >> I can be convinced to use if (state) too, though =).
> >>
> >>>> +		v4l2_subdev_unlock_state(state);
> >>>>    
> >>>>    	return ret;
> >>>>    }
> >>>>    EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
> >>>>    
> >>>> -static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> >>>> -						u64 streams_mask)
> >>>> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>> +				u64 streams_mask)
> >>>>    {
> >>>>    	struct device *dev = sd->entity.graph_obj.mdev->dev;
> >>>> +	struct v4l2_subdev_state *state;
> >>>> +	u64 enabled_streams;
> >>>> +	u64 found_streams;
> >>>> +	bool use_s_stream;
> >>>>    	int ret;
> >>>>    
> >>>> -	/*
> >>>> -	 * If the subdev doesn't implement pad-based stream enable, fall back
> >>>> -	 * to the .s_stream() operation.
> >>>> -	 */
> >>>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >>>> -		return -EOPNOTSUPP;
> >>>> +	/* A few basic sanity checks first. */
> >>>> +	if (pad >= sd->entity.num_pads)
> >>>> +		return -EINVAL;
> >>>>    
> >>>> -	/*
> >>>> -	 * .s_stream() means there is no streams support, so the only allowed
> >>>> -	 * stream is the implicit stream 0.
> >>>> -	 */
> >>>> -	if (streams_mask != BIT_ULL(0))
> >>>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
> >>>>    		return -EOPNOTSUPP;
> >>>>    
> >>>>    	/*
> >>>> @@ -2280,46 +2269,16 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
> >>>>    	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
> >>>>    		return -EOPNOTSUPP;
> >>>>    
> >>>> -	if (!(sd->enabled_pads & BIT_ULL(pad))) {
> >>>> -		dev_dbg(dev, "pad %u already disabled on %s\n",
> >>>> -			pad, sd->entity.name);
> >>>> -		return -EALREADY;
> >>>> -	}
> >>>> -
> >>>> -	/* Stop streaming when the last streams are disabled. */
> >>>> -	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
> >>>> -		ret = v4l2_subdev_call(sd, video, s_stream, 0);
> >>>> -		if (ret)
> >>>> -			return ret;
> >>>> -	}
> >>>> -
> >>>> -	sd->enabled_pads &= ~BIT_ULL(pad);
> >>>> -
> >>>> -	return 0;
> >>>> -}
> >>>> -
> >>>> -int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>> -				u64 streams_mask)
> >>>> -{
> >>>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
> >>>> -	struct v4l2_subdev_state *state;
> >>>> -	u64 enabled_streams;
> >>>> -	u64 found_streams;
> >>>> -	int ret;
> >>>> -
> >>>> -	/* A few basic sanity checks first. */
> >>>> -	if (pad >= sd->entity.num_pads)
> >>>> -		return -EINVAL;
> >>>> -
> >>>>    	if (!streams_mask)
> >>>>    		return 0;
> >>>>    
> >>>>    	/* Fallback on .s_stream() if .disable_streams() isn't available. */
> >>>> -	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
> >>>> -		return v4l2_subdev_disable_streams_fallback(sd, pad,
> >>>> -							    streams_mask);
> >>>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, disable_streams);
> >>>>    
> >>>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
> >>>> +	if (!use_s_stream)
> >>>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
> >>>> +	else
> >>>> +		state = NULL;
> >>>>    
> >>>>    	/*
> >>>>    	 * Verify that the requested streams exist and that they are not
> >>>> @@ -2345,9 +2304,19 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>>    
> >>>>    	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
> >>>>    
> >>>> -	/* Call the .disable_streams() operation. */
> >>>> -	ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
> >>>> -			       streams_mask);
> >>>> +	if (!use_s_stream) {
> >>>> +		/* Call the .disable_streams() operation. */
> >>>> +		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
> >>>> +				       streams_mask);
> >>>> +	} else {
> >>>> +		/* Stop streaming when the last streams are disabled. */
> >>>> +
> >>>> +		if (!(sd->enabled_pads & ~BIT_ULL(pad)))
> >>>> +			ret = v4l2_subdev_call(sd, video, s_stream, 0);
> >>>> +		else
> >>>> +			ret = 0;
> >>>> +	}
> >>>> +
> >>>>    	if (ret) {
> >>>>    		dev_dbg(dev, "disable streams %u:%#llx failed: %d\n", pad,
> >>>>    			streams_mask, ret);
> >>>> @@ -2357,10 +2326,12 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
> >>>>    	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
> >>>>    
> >>>>    done:
> >>>> -	if (!v4l2_subdev_is_streaming(sd))
> >>>> -		v4l2_subdev_disable_privacy_led(sd);
> >>>> +	if (!use_s_stream) {
> >>>> +		if (!v4l2_subdev_is_streaming(sd))
> >>>> +			v4l2_subdev_disable_privacy_led(sd);
> >>>
> >>> Do we want to disable the privacy LED on failure in all cases, in
> >>> particular when the .s_stream() or .disable_streams() operations are not
> >>> even called ?
> >>>
> >>> And now that I wrote that, I realize the !v4l2_subdev_is_streaming()
> >>> condition will not be true if the operations failed, so it should be
> >>> fine.
> >>>
> >>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>>
> >>>>    
> >>>> -	v4l2_subdev_unlock_state(state);
> >>>> +		v4l2_subdev_unlock_state(state);
> >>>> +	}
> >>>>    
> >>>>    	return ret;
> >>>>    }
> >>>>

-- 
Regards,

Laurent Pinchart

Re: [PATCH v5 09/10] media: subdev: Support single-stream case in v4l2_subdev_enable/disable_streams()

[Tomi Valkeinen]

On 24/04/2024 16:17, Laurent Pinchart wrote:
> On Wed, Apr 24, 2024 at 04:11:36PM +0300, Tomi Valkeinen wrote:
>> On 24/04/2024 16:09, Laurent Pinchart wrote:
>>> On Wed, Apr 24, 2024 at 04:05:31PM +0300, Tomi Valkeinen wrote:
>>>> On 19/04/2024 18:36, Laurent Pinchart wrote:
>>>>> On Tue, Apr 16, 2024 at 04:55:12PM +0300, Tomi Valkeinen wrote:
>>>>>> At the moment the v4l2_subdev_enable/disable_streams() functions call
>>>>>> fallback helpers to handle the case where the subdev only implements
>>>>>> .s_stream(), and the main function handles the case where the subdev
>>>>>> implements streams (V4L2_SUBDEV_FL_STREAMS, which implies
>>>>>> .enable/disable_streams()).
>>>>>>
>>>>>> What is missing is support for subdevs which do not implement streams
>>>>>> support, but do implement .enable/disable_streams(). Example cases of
>>>>>> these subdevices are single-stream cameras, where using
>>>>>> .enable/disable_streams() is not required but helps us remove the users
>>>>>> of the legacy .s_stream(), and subdevices with multiple source pads (but
>>>>>> single stream per pad), where .enable/disable_streams() allows the
>>>>>> subdevice to control the enable/disable state per pad.
>>>>>>
>>>>>> The two single-streams cases (.s_stream() and .enable/disable_streams())
>>>>>> are very similar, and with small changes we can change the
>>>>>> v4l2_subdev_enable/disable_streams() functions to support all three
>>>>>> cases, without needing separate fallback functions.
>>>>>>
>>>>>> A few potentially problematic details, though:
>>>>>>
>>>>>> - For the single-streams cases we use sd->enabled_pads field, which
>>>>>>      limits the number of pads for the subdevice to 64. For simplicity I
>>>>>>      added the check for this limitation to the beginning of the function,
>>>>>>      and it also applies to the streams case.
>>>>>>
>>>>>> - The fallback functions only allowed the target pad to be a source pad.
>>>>>>      It is not very clear to me why this check was needed, but it was not
>>>>>>      needed in the streams case. However, I doubt the
>>>>>>      v4l2_subdev_enable/disable_streams() code has ever been tested with
>>>>>>      sink pads, so to be on the safe side, I added the same check
>>>>>>      to the v4l2_subdev_enable/disable_streams() functions.
>>>>>>
>>>>>
>>>>> Now that v4l2_subdev_enable_streams() is usable by (almost) every
>>>>> driver, should we update documentation to indicate that manual calls to
>>>>> .s_stream() should be avoided ?
>>>>
>>>> How about:
>>>>
>>>> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
>>>> index dabe1b5dfe4a..f52bb790773c 100644
>>>> --- a/include/media/v4l2-subdev.h
>>>> +++ b/include/media/v4l2-subdev.h
>>>> @@ -450,6 +450,11 @@ enum v4l2_subdev_pre_streamon_flags {
>>>>     *     already started or stopped subdev. Also see call_s_stream wrapper in
>>>>     *     v4l2-subdev.c.
>>>>     *
>>>> + *     New drivers should instead implement &v4l2_subdev_pad_ops.enable_streams
>>>> + *     and &v4l2_subdev_pad_ops.disable_streams operations, and use
>>>> + *     v4l2_subdev_s_stream_helper for the &v4l2_subdev_video_ops.s_stream
>>>> + *     operation to support legacy users.A
>>>> + *
>>>
>>> Add
>>>
>>>    *	Drivers should also not call the .s_stream() subdev operation directly,
>>>    *	but use the v4l2_subdev_enable_streams() and
>>>    *	v4l2_subdev_disable_streams() helpers.
>>>
>>>>     * @g_pixelaspect: callback to return the pixelaspect ratio.
>>>>     *
>>>>     * @s_dv_timings: Set custom dv timings in the sub device. This is used
>>>>
>>>>
>>>> Although now that I think about it, can we "ever" get rid of s_stream?
>>>> enable_streams is only usable if the subdev has pads.
>>>
>>> Do we have pad-less subdevs that can stream ?
>>
>> Probably not, but isn't s_stream used to enable any type of subdev? Or
>> is there a different op to power on padless subdevs?
> 
> .s_stream() is supposed to start/stop streaming, and you can't really
> have streams without pads, as they have to come from/go somewhere.

Ok. I was under the impression that .s_stream() is also used to enable 
non-streaming subdevs when the pipeline's streaming is started. If 
that's not the case, I'll update the doc.

  Tomi

>>>> If that's the
>>>> case, the text above should probably clarify when one should continue
>>>> using s_stream.
>>>>
>>>>>> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
>>>>>> ---
>>>>>>     drivers/media/v4l2-core/v4l2-subdev.c | 187 ++++++++++++++--------------------
>>>>>>     1 file changed, 79 insertions(+), 108 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
>>>>>> index e45fd42da1e3..10406acfb9d0 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-subdev.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
>>>>>> @@ -2106,6 +2106,13 @@ static void v4l2_subdev_collect_streams(struct v4l2_subdev *sd,
>>>>>>     					u64 *found_streams,
>>>>>>     					u64 *enabled_streams)
>>>>>>     {
>>>>>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
>>>>>> +		*found_streams = BIT_ULL(0);
>>>>>> +		*enabled_streams =
>>>>>> +			(sd->enabled_pads & BIT_ULL(pad)) ? BIT_ULL(0) : 0;
>>>>>> +		return;
>>>>>> +	}
>>>>>> +
>>>>>>     	*found_streams = 0;
>>>>>>     	*enabled_streams = 0;
>>>>>>     
>>>>>> @@ -2127,6 +2134,14 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>>>>>>     					    u32 pad, u64 streams_mask,
>>>>>>     					    bool enabled)
>>>>>>     {
>>>>>> +	if (!(sd->flags & V4L2_SUBDEV_FL_STREAMS)) {
>>>>>> +		if (enabled)
>>>>>> +			sd->enabled_pads |= BIT_ULL(pad);
>>>>>> +		else
>>>>>> +			sd->enabled_pads &= ~BIT_ULL(pad);
>>>>>> +		return;
>>>>>> +	}
>>>>>> +
>>>>>>     	for (unsigned int i = 0; i < state->stream_configs.num_configs; ++i) {
>>>>>>     		struct v4l2_subdev_stream_config *cfg =
>>>>>>     			&state->stream_configs.configs[i];
>>>>>> @@ -2136,51 +2151,6 @@ static void v4l2_subdev_set_streams_enabled(struct v4l2_subdev *sd,
>>>>>>     	}
>>>>>>     }
>>>>>>     
>>>>>> -static int v4l2_subdev_enable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>>>>> -					       u64 streams_mask)
>>>>>> -{
>>>>>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>>>>> -	int ret;
>>>>>> -
>>>>>> -	/*
>>>>>> -	 * The subdev doesn't implement pad-based stream enable, fall back
>>>>>> -	 * to the .s_stream() operation.
>>>>>> -	 */
>>>>>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>>>> -		return -EOPNOTSUPP;
>>>>>> -
>>>>>> -	/*
>>>>>> -	 * .s_stream() means there is no streams support, so the only allowed
>>>>>> -	 * stream is the implicit stream 0.
>>>>>> -	 */
>>>>>> -	if (streams_mask != BIT_ULL(0))
>>>>>> -		return -EOPNOTSUPP;
>>>>>> -
>>>>>> -	/*
>>>>>> -	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
>>>>>> -	 * with 64 pads or less can be supported.
>>>>>> -	 */
>>>>>> -	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>>>>>> -		return -EOPNOTSUPP;
>>>>>> -
>>>>>> -	if (sd->enabled_pads & BIT_ULL(pad)) {
>>>>>> -		dev_dbg(dev, "pad %u already enabled on %s\n",
>>>>>> -			pad, sd->entity.name);
>>>>>> -		return -EALREADY;
>>>>>> -	}
>>>>>> -
>>>>>> -	/* Start streaming when the first pad is enabled. */
>>>>>> -	if (!sd->enabled_pads) {
>>>>>> -		ret = v4l2_subdev_call(sd, video, s_stream, 1);
>>>>>> -		if (ret)
>>>>>> -			return ret;
>>>>>> -	}
>>>>>> -
>>>>>> -	sd->enabled_pads |= BIT_ULL(pad);
>>>>>> -
>>>>>> -	return 0;
>>>>>> -}
>>>>>> -
>>>>>>     int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>>     			       u64 streams_mask)
>>>>>>     {
>>>>>> @@ -2189,21 +2159,33 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>>     	bool already_streaming;
>>>>>>     	u64 enabled_streams;
>>>>>>     	u64 found_streams;
>>>>>> +	bool use_s_stream;
>>>>>>     	int ret;
>>>>>>     
>>>>>>     	/* A few basic sanity checks first. */
>>>>>>     	if (pad >= sd->entity.num_pads)
>>>>>>     		return -EINVAL;
>>>>>>     
>>>>>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>>>> +		return -EOPNOTSUPP;
>>>>>> +
>>>>>> +	/*
>>>>>> +	 * We use a 64-bit bitmask for tracking enabled pads, so only subdevices
>>>>>> +	 * with 64 pads or less can be supported.
>>>>>> +	 */
>>>>>> +	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>>>>>> +		return -EOPNOTSUPP;
>>>>>> +
>>>>>>     	if (!streams_mask)
>>>>>>     		return 0;
>>>>>>     
>>>>>>     	/* Fallback on .s_stream() if .enable_streams() isn't available. */
>>>>>> -	if (!v4l2_subdev_has_op(sd, pad, enable_streams))
>>>>>> -		return v4l2_subdev_enable_streams_fallback(sd, pad,
>>>>>> -							   streams_mask);
>>>>>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, enable_streams);
>>>>>>     
>>>>>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
>>>>>> +	if (!use_s_stream)
>>>>>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
>>>>>> +	else
>>>>>> +		state = NULL;
>>>>>>     
>>>>>>     	/*
>>>>>>     	 * Verify that the requested streams exist and that they are not
>>>>>> @@ -2231,9 +2213,18 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>>     
>>>>>>     	already_streaming = v4l2_subdev_is_streaming(sd);
>>>>>>     
>>>>>> -	/* Call the .enable_streams() operation. */
>>>>>> -	ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>>>>>> -			       streams_mask);
>>>>>> +	if (!use_s_stream) {
>>>>>> +		/* Call the .enable_streams() operation. */
>>>>>> +		ret = v4l2_subdev_call(sd, pad, enable_streams, state, pad,
>>>>>> +				       streams_mask);
>>>>>> +	} else {
>>>>>> +		/* Start streaming when the first pad is enabled. */
>>>>>> +		if (!already_streaming)
>>>>>> +			ret = v4l2_subdev_call(sd, video, s_stream, 1);
>>>>>> +		else
>>>>>> +			ret = 0;
>>>>>> +	}
>>>>>> +
>>>>>>     	if (ret) {
>>>>>>     		dev_dbg(dev, "enable streams %u:%#llx failed: %d\n", pad,
>>>>>>     			streams_mask, ret);
>>>>>> @@ -2243,34 +2234,32 @@ int v4l2_subdev_enable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>>     	/* Mark the streams as enabled. */
>>>>>>     	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, true);
>>>>>
>>>>> state gets set to a non-null value above if the subdev has an
>>>>> enable_streams operation, while in v4l2_subdev_set_streams_enabled(),
>>>>
>>>> That's not quite true, v4l2_subdev_lock_and_get_active_state() returns
>>>> NULL if there's no state. If the driver has .enable_streams but no
>>>> subdev-state, we'll have NULL state. But is that a driver bug? Probably,
>>>> although I'm not quite sure. Whereas V4L2_SUBDEV_FL_STREAMS without
>>>> state is a driver bug.
>>>>
>>>>> you use it if the V4L2_SUBDEV_FL_STREAMS flag is set. The second
>>>>> condition is a subset of the first one, so it should be safe now, but I
>>>>> wonder if we're at risk of crashes in the future when reworking the
>>>>> code. I'm probably worrying needlessly.
>>>>
>>>> I agree that this is a bit messy. We have multiple different scenarios
>>>> which we handle in the same functions, and it's somewhat unclear what
>>>> op/state/flag combinations the drivers are allowed to have and what not.
>>>>
>>>> I could add a !state check here, and then use sd->enabled_pads. But that
>>>> doesn't feel correct, and I think it's just better to crash and then fix
>>>> it if we end up there.
>>>>
>>>>>>     
>>>>>> -	if (!already_streaming)
>>>>>> +	if (!use_s_stream && !already_streaming)
>>>>>>     		v4l2_subdev_enable_privacy_led(sd);
>>>>>
>>>>> Once everybody switches to v4l2_subdev_enable_streams() (am I dreaming?)
>>>>> we should move LED handling from the .s_stream() wrapper to here for the
>>>>> fallback case too. How about a TODO comment ?
>>>>
>>>> Let's get back to this after we figure out if s_stream can ever be removed.
>>>>
>>>>>>     
>>>>>>     done:
>>>>>> -	v4l2_subdev_unlock_state(state);
>>>>>> +	if (!use_s_stream)
>>>>>
>>>>> 	if (state)
>>>>>
>>>>> would be better I think.
>>>>
>>>> We get the state and state lock if (!use_s_stream), so I think it makes
>>>> sense to use the same condition when unlocking to be symmetric. I'm sure
>>>> I can be convinced to use if (state) too, though =).
>>>>
>>>>>> +		v4l2_subdev_unlock_state(state);
>>>>>>     
>>>>>>     	return ret;
>>>>>>     }
>>>>>>     EXPORT_SYMBOL_GPL(v4l2_subdev_enable_streams);
>>>>>>     
>>>>>> -static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>>>>> -						u64 streams_mask)
>>>>>> +int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>> +				u64 streams_mask)
>>>>>>     {
>>>>>>     	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>>>>> +	struct v4l2_subdev_state *state;
>>>>>> +	u64 enabled_streams;
>>>>>> +	u64 found_streams;
>>>>>> +	bool use_s_stream;
>>>>>>     	int ret;
>>>>>>     
>>>>>> -	/*
>>>>>> -	 * If the subdev doesn't implement pad-based stream enable, fall back
>>>>>> -	 * to the .s_stream() operation.
>>>>>> -	 */
>>>>>> -	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>>>> -		return -EOPNOTSUPP;
>>>>>> +	/* A few basic sanity checks first. */
>>>>>> +	if (pad >= sd->entity.num_pads)
>>>>>> +		return -EINVAL;
>>>>>>     
>>>>>> -	/*
>>>>>> -	 * .s_stream() means there is no streams support, so the only allowed
>>>>>> -	 * stream is the implicit stream 0.
>>>>>> -	 */
>>>>>> -	if (streams_mask != BIT_ULL(0))
>>>>>> +	if (!(sd->entity.pads[pad].flags & MEDIA_PAD_FL_SOURCE))
>>>>>>     		return -EOPNOTSUPP;
>>>>>>     
>>>>>>     	/*
>>>>>> @@ -2280,46 +2269,16 @@ static int v4l2_subdev_disable_streams_fallback(struct v4l2_subdev *sd, u32 pad,
>>>>>>     	if (pad >= sizeof(sd->enabled_pads) * BITS_PER_BYTE)
>>>>>>     		return -EOPNOTSUPP;
>>>>>>     
>>>>>> -	if (!(sd->enabled_pads & BIT_ULL(pad))) {
>>>>>> -		dev_dbg(dev, "pad %u already disabled on %s\n",
>>>>>> -			pad, sd->entity.name);
>>>>>> -		return -EALREADY;
>>>>>> -	}
>>>>>> -
>>>>>> -	/* Stop streaming when the last streams are disabled. */
>>>>>> -	if (!(sd->enabled_pads & ~BIT_ULL(pad))) {
>>>>>> -		ret = v4l2_subdev_call(sd, video, s_stream, 0);
>>>>>> -		if (ret)
>>>>>> -			return ret;
>>>>>> -	}
>>>>>> -
>>>>>> -	sd->enabled_pads &= ~BIT_ULL(pad);
>>>>>> -
>>>>>> -	return 0;
>>>>>> -}
>>>>>> -
>>>>>> -int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>> -				u64 streams_mask)
>>>>>> -{
>>>>>> -	struct device *dev = sd->entity.graph_obj.mdev->dev;
>>>>>> -	struct v4l2_subdev_state *state;
>>>>>> -	u64 enabled_streams;
>>>>>> -	u64 found_streams;
>>>>>> -	int ret;
>>>>>> -
>>>>>> -	/* A few basic sanity checks first. */
>>>>>> -	if (pad >= sd->entity.num_pads)
>>>>>> -		return -EINVAL;
>>>>>> -
>>>>>>     	if (!streams_mask)
>>>>>>     		return 0;
>>>>>>     
>>>>>>     	/* Fallback on .s_stream() if .disable_streams() isn't available. */
>>>>>> -	if (!v4l2_subdev_has_op(sd, pad, disable_streams))
>>>>>> -		return v4l2_subdev_disable_streams_fallback(sd, pad,
>>>>>> -							    streams_mask);
>>>>>> +	use_s_stream = !v4l2_subdev_has_op(sd, pad, disable_streams);
>>>>>>     
>>>>>> -	state = v4l2_subdev_lock_and_get_active_state(sd);
>>>>>> +	if (!use_s_stream)
>>>>>> +		state = v4l2_subdev_lock_and_get_active_state(sd);
>>>>>> +	else
>>>>>> +		state = NULL;
>>>>>>     
>>>>>>     	/*
>>>>>>     	 * Verify that the requested streams exist and that they are not
>>>>>> @@ -2345,9 +2304,19 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>>     
>>>>>>     	dev_dbg(dev, "disable streams %u:%#llx\n", pad, streams_mask);
>>>>>>     
>>>>>> -	/* Call the .disable_streams() operation. */
>>>>>> -	ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
>>>>>> -			       streams_mask);
>>>>>> +	if (!use_s_stream) {
>>>>>> +		/* Call the .disable_streams() operation. */
>>>>>> +		ret = v4l2_subdev_call(sd, pad, disable_streams, state, pad,
>>>>>> +				       streams_mask);
>>>>>> +	} else {
>>>>>> +		/* Stop streaming when the last streams are disabled. */
>>>>>> +
>>>>>> +		if (!(sd->enabled_pads & ~BIT_ULL(pad)))
>>>>>> +			ret = v4l2_subdev_call(sd, video, s_stream, 0);
>>>>>> +		else
>>>>>> +			ret = 0;
>>>>>> +	}
>>>>>> +
>>>>>>     	if (ret) {
>>>>>>     		dev_dbg(dev, "disable streams %u:%#llx failed: %d\n", pad,
>>>>>>     			streams_mask, ret);
>>>>>> @@ -2357,10 +2326,12 @@ int v4l2_subdev_disable_streams(struct v4l2_subdev *sd, u32 pad,
>>>>>>     	v4l2_subdev_set_streams_enabled(sd, state, pad, streams_mask, false);
>>>>>>     
>>>>>>     done:
>>>>>> -	if (!v4l2_subdev_is_streaming(sd))
>>>>>> -		v4l2_subdev_disable_privacy_led(sd);
>>>>>> +	if (!use_s_stream) {
>>>>>> +		if (!v4l2_subdev_is_streaming(sd))
>>>>>> +			v4l2_subdev_disable_privacy_led(sd);
>>>>>
>>>>> Do we want to disable the privacy LED on failure in all cases, in
>>>>> particular when the .s_stream() or .disable_streams() operations are not
>>>>> even called ?
>>>>>
>>>>> And now that I wrote that, I realize the !v4l2_subdev_is_streaming()
>>>>> condition will not be true if the operations failed, so it should be
>>>>> fine.
>>>>>
>>>>> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>>>
>>>>>>     
>>>>>> -	v4l2_subdev_unlock_state(state);
>>>>>> +		v4l2_subdev_unlock_state(state);
>>>>>> +	}
>>>>>>     
>>>>>>     	return ret;
>>>>>>     }
>>>>>>
>