[RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Kieran Bingham]

Provide a streaming attribute to allow userspace to interogate if a device
is actively streaming or not.

This will allow desktop notifications to report if a camera or device
is active on the system, rather than just 'open' which can occur when
configuring the device.

Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
---

This is a quick POC to see if such a facility makes sense.
I'm weary that not all video devices may have the queues registered on
the struct video_device, but this seems like an effective way to be able
to determine if a device is actively streaming on a system.


 Documentation/ABI/stable/sysfs-class-video |  9 +++++++++
 MAINTAINERS                                |  1 +
 drivers/media/v4l2-core/v4l2-dev.c         | 13 +++++++++++++
 3 files changed, 23 insertions(+)
 create mode 100644 Documentation/ABI/stable/sysfs-class-video

diff --git a/Documentation/ABI/stable/sysfs-class-video b/Documentation/ABI/stable/sysfs-class-video
new file mode 100644
index 000000000000..99dd27475a92
--- /dev/null
+++ b/Documentation/ABI/stable/sysfs-class-video
@@ -0,0 +1,9 @@
+What:		/sys/class/video4linux/video<n>/streaming
+Date:		January 2023
+KernelVersion:	6.3
+Contact:	Kieran Bingham <kieran.bingham@ideasonboard.com>
+Description:
+		Indicates if the video device has an actively streaming queue.
+		This may indicate that the device is capturing or outputing
+		video data.
+Users:
diff --git a/MAINTAINERS b/MAINTAINERS
index 11987154eeee..8449f5a6c0da 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12867,6 +12867,7 @@ S:	Maintained
 W:	https://linuxtv.org
 Q:	http://patchwork.kernel.org/project/linux-media/list/
 T:	git git://linuxtv.org/media_tree.git
+F:	Documentation/ABI/stable/sysfs-class-video
 F:	Documentation/admin-guide/media/
 F:	Documentation/devicetree/bindings/media/
 F:	Documentation/driver-api/media/
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index 397d553177fa..7d800309d076 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -30,6 +30,7 @@
 #include <media/v4l2-device.h>
 #include <media/v4l2-ioctl.h>
 #include <media/v4l2-event.h>
+#include <media/videobuf2-core.h>
 
 #define VIDEO_NUM_DEVICES	256
 #define VIDEO_NAME              "video4linux"
@@ -85,7 +86,19 @@ static ssize_t name_show(struct device *cd,
 }
 static DEVICE_ATTR_RO(name);
 
+static ssize_t streaming_show(struct device *cd,
+			      struct device_attribute *attr, char *buf)
+{
+	struct video_device *vdev = to_video_device(cd);
+
+	int active = vdev->queue ? vb2_is_streaming(vdev->queue) : false;
+
+	return sprintf(buf, "%s\n", active ? "active" : "inactive");
+}
+static DEVICE_ATTR_RO(streaming);
+
 static struct attribute *video_device_attrs[] = {
+	&dev_attr_streaming.attr,
 	&dev_attr_name.attr,
 	&dev_attr_dev_debug.attr,
 	&dev_attr_index.attr,
-- 
2.34.1

Re: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Laurent Pinchart]

Hi Kieran,

Thank you for the patch.

On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote:
> Provide a streaming attribute to allow userspace to interogate if a device
> is actively streaming or not.
> 
> This will allow desktop notifications to report if a camera or device
> is active on the system, rather than just 'open' which can occur when
> configuring the device.
> 
> Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> ---
> 
> This is a quick POC to see if such a facility makes sense.
> I'm weary that not all video devices may have the queues registered on
> the struct video_device, but this seems like an effective way to be able
> to determine if a device is actively streaming on a system.

I can imagine multiple problems, from race conditions to permissions and
privacy. In order to comment on the fitness of this solution to address
the problem you're trying to solve, could you describe the actual
problem ?

>  Documentation/ABI/stable/sysfs-class-video |  9 +++++++++
>  MAINTAINERS                                |  1 +
>  drivers/media/v4l2-core/v4l2-dev.c         | 13 +++++++++++++
>  3 files changed, 23 insertions(+)
>  create mode 100644 Documentation/ABI/stable/sysfs-class-video
> 
> diff --git a/Documentation/ABI/stable/sysfs-class-video b/Documentation/ABI/stable/sysfs-class-video
> new file mode 100644
> index 000000000000..99dd27475a92
> --- /dev/null
> +++ b/Documentation/ABI/stable/sysfs-class-video
> @@ -0,0 +1,9 @@
> +What:		/sys/class/video4linux/video<n>/streaming
> +Date:		January 2023
> +KernelVersion:	6.3
> +Contact:	Kieran Bingham <kieran.bingham@ideasonboard.com>
> +Description:
> +		Indicates if the video device has an actively streaming queue.
> +		This may indicate that the device is capturing or outputing
> +		video data.
> +Users:
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 11987154eeee..8449f5a6c0da 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -12867,6 +12867,7 @@ S:	Maintained
>  W:	https://linuxtv.org
>  Q:	http://patchwork.kernel.org/project/linux-media/list/
>  T:	git git://linuxtv.org/media_tree.git
> +F:	Documentation/ABI/stable/sysfs-class-video
>  F:	Documentation/admin-guide/media/
>  F:	Documentation/devicetree/bindings/media/
>  F:	Documentation/driver-api/media/
> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
> index 397d553177fa..7d800309d076 100644
> --- a/drivers/media/v4l2-core/v4l2-dev.c
> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> @@ -30,6 +30,7 @@
>  #include <media/v4l2-device.h>
>  #include <media/v4l2-ioctl.h>
>  #include <media/v4l2-event.h>
> +#include <media/videobuf2-core.h>
>  
>  #define VIDEO_NUM_DEVICES	256
>  #define VIDEO_NAME              "video4linux"
> @@ -85,7 +86,19 @@ static ssize_t name_show(struct device *cd,
>  }
>  static DEVICE_ATTR_RO(name);
>  
> +static ssize_t streaming_show(struct device *cd,
> +			      struct device_attribute *attr, char *buf)
> +{
> +	struct video_device *vdev = to_video_device(cd);
> +
> +	int active = vdev->queue ? vb2_is_streaming(vdev->queue) : false;
> +
> +	return sprintf(buf, "%s\n", active ? "active" : "inactive");
> +}
> +static DEVICE_ATTR_RO(streaming);
> +
>  static struct attribute *video_device_attrs[] = {
> +	&dev_attr_streaming.attr,
>  	&dev_attr_name.attr,
>  	&dev_attr_dev_debug.attr,
>  	&dev_attr_index.attr,

-- 
Regards,

Laurent Pinchart

Re: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Barnabás Pőcze]

Hi


On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote:

> Hi Kieran,
> 
> Thank you for the patch.
> 
> On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote:
> 
> > Provide a streaming attribute to allow userspace to interogate if a device
> > is actively streaming or not.
> > 
> > This will allow desktop notifications to report if a camera or device
> > is active on the system, rather than just 'open' which can occur when
> > configuring the device.
> > 
> > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
> > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com
> > ---
> > 
> > This is a quick POC to see if such a facility makes sense.
> > I'm weary that not all video devices may have the queues registered on
> > the struct video_device, but this seems like an effective way to be able
> > to determine if a device is actively streaming on a system.
> 
> 
> I can imagine multiple problems, from race conditions to permissions and
> privacy. In order to comment on the fitness of this solution to address
> the problem you're trying to solve, could you describe the actual
> problem ?


The issue is explained in the following thread:
https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388

In short, the user wants to show a "camera-in-use" indicator when the laptop camera
is used. The script that the user previously had only checked if /dev/video0
was open in any process, if it was, the indicator was shown. However, libcamera
- at least at the moment - keeps the file descriptor open as long as the Camera
object exists, which pipewire keeps alive for the entire lifetime of the device,
therefore the "camera-in-use" indicator is always shown.

> [...]


Regards,
Barnabás Pőcze

Re: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Sakari Ailus]

Hi Barnabás, others,

On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote:
> Hi
> 
> 
> On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote:
> 
> > Hi Kieran,
> > 
> > Thank you for the patch.
> > 
> > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote:
> > 
> > > Provide a streaming attribute to allow userspace to interogate if a device
> > > is actively streaming or not.
> > > 
> > > This will allow desktop notifications to report if a camera or device
> > > is active on the system, rather than just 'open' which can occur when
> > > configuring the device.
> > > 
> > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
> > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com
> > > ---
> > > 
> > > This is a quick POC to see if such a facility makes sense.
> > > I'm weary that not all video devices may have the queues registered on
> > > the struct video_device, but this seems like an effective way to be able
> > > to determine if a device is actively streaming on a system.
> > 
> > 
> > I can imagine multiple problems, from race conditions to permissions and
> > privacy. In order to comment on the fitness of this solution to address
> > the problem you're trying to solve, could you describe the actual
> > problem ?
> 
> 
> The issue is explained in the following thread:
> https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388
> 
> In short, the user wants to show a "camera-in-use" indicator when the laptop camera
> is used. The script that the user previously had only checked if /dev/video0
> was open in any process, if it was, the indicator was shown. However, libcamera
> - at least at the moment - keeps the file descriptor open as long as the Camera
> object exists, which pipewire keeps alive for the entire lifetime of the device,
> therefore the "camera-in-use" indicator is always shown.

A sysfs attribute is not a great way to address this.

libcamera certainly has information on whether streaming is ongoing. The
information should come from there. Or Pipewire. Dbus perhaps?

Alternatively libcamera could close the video devices while not streaming
but that would involve e.g. releasing possible video buffer allocations as
well, increasing streaming start latency.

-- 
Kind regards,

Sakari Ailus

Re: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Laurent Pinchart]

Hello,

On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote:
> On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote:
> > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote:
> > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote:
> > > 
> > > > Provide a streaming attribute to allow userspace to interogate if a device
> > > > is actively streaming or not.
> > > > 
> > > > This will allow desktop notifications to report if a camera or device
> > > > is active on the system, rather than just 'open' which can occur when
> > > > configuring the device.
> > > > 
> > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
> > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com
> > > > ---
> > > > 
> > > > This is a quick POC to see if such a facility makes sense.
> > > > I'm weary that not all video devices may have the queues registered on
> > > > the struct video_device, but this seems like an effective way to be able
> > > > to determine if a device is actively streaming on a system.
> > > 
> > > I can imagine multiple problems, from race conditions to permissions and
> > > privacy. In order to comment on the fitness of this solution to address
> > > the problem you're trying to solve, could you describe the actual
> > > problem ?
> > 
> > The issue is explained in the following thread:
> > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388
> > 
> > In short, the user wants to show a "camera-in-use" indicator when the laptop camera
> > is used. The script that the user previously had only checked if /dev/video0
> > was open in any process, if it was, the indicator was shown. However, libcamera
> > - at least at the moment - keeps the file descriptor open as long as the Camera
> > object exists, which pipewire keeps alive for the entire lifetime of the device,
> > therefore the "camera-in-use" indicator is always shown.
> 
> A sysfs attribute is not a great way to address this.
> 
> libcamera certainly has information on whether streaming is ongoing. The
> information should come from there. Or Pipewire. Dbus perhaps?

I tend to agree, I think this is best solved in userspace where PipeWire
can have a centralized view of all cameras in the system, and of their
users.

> Alternatively libcamera could close the video devices while not streaming
> but that would involve e.g. releasing possible video buffer allocations as
> well, increasing streaming start latency.

Closing video (and subdev) nodes when the camera is not in use would be
good I think. It doesn't mean we have to open them when starting
capture, explicit open/close operation (or similar, maybe introducing a
capture session object in the libcamera API would also make sense, it
should be considered as part of the same issue) could help with this.

-- 
Regards,

Laurent Pinchart

Re: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Kieran Bingham]

Quoting Laurent Pinchart (2023-01-02 13:35:33)
> Hello,
> 
> On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote:
> > On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote:
> > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote:
> > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote:
> > > > 
> > > > > Provide a streaming attribute to allow userspace to interogate if a device
> > > > > is actively streaming or not.
> > > > > 
> > > > > This will allow desktop notifications to report if a camera or device
> > > > > is active on the system, rather than just 'open' which can occur when
> > > > > configuring the device.
> > > > > 
> > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
> > > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com
> > > > > ---
> > > > > 
> > > > > This is a quick POC to see if such a facility makes sense.
> > > > > I'm weary that not all video devices may have the queues registered on
> > > > > the struct video_device, but this seems like an effective way to be able
> > > > > to determine if a device is actively streaming on a system.
> > > > 
> > > > I can imagine multiple problems, from race conditions to permissions and
> > > > privacy. In order to comment on the fitness of this solution to address
> > > > the problem you're trying to solve, could you describe the actual
> > > > problem ?
> > > 
> > > The issue is explained in the following thread:
> > > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388
> > > 
> > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera
> > > is used. The script that the user previously had only checked if /dev/video0
> > > was open in any process, if it was, the indicator was shown. However, libcamera
> > > - at least at the moment - keeps the file descriptor open as long as the Camera
> > > object exists, which pipewire keeps alive for the entire lifetime of the device,
> > > therefore the "camera-in-use" indicator is always shown.
> > 
> > A sysfs attribute is not a great way to address this.
> > 
> > libcamera certainly has information on whether streaming is ongoing. The
> > information should come from there. Or Pipewire. Dbus perhaps?
> 
> I tend to agree, I think this is best solved in userspace where PipeWire
> can have a centralized view of all cameras in the system, and of their
> users.

I fear that misses the entire point I was trying to make.

Lets say pipewire 'is' available and in use and can be used to capture
video streams for video calls, that's fine. But what happens if a user
runs a gstreamer pipeline without using the pipewire source, or a
suspcious process runs "yavta" and captures an image or stream
discreetly...

Only the kernel has a true centralised view of what devices are in use.

> > Alternatively libcamera could close the video devices while not streaming
> > but that would involve e.g. releasing possible video buffer allocations as
> > well, increasing streaming start latency.

Or is it just that in that case 'lsof' should be sufficient?

The problem I have with that is - just like with the issue when the
Privacy LED comes on during power up/probe - then any time a device is
opened to identify the device and not necessarily use it - the 'camera
in use' notification would get flashed...

> Closing video (and subdev) nodes when the camera is not in use would be
> good I think. It doesn't mean we have to open them when starting
> capture, explicit open/close operation (or similar, maybe introducing a
> capture session object in the libcamera API would also make sense, it
> should be considered as part of the same issue) could help with this.

I'm not talking about libcamera in this thread. It's how does a user
correctly identify when a camera is in use globally in a system.
--
Kieran

Re: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Laurent Pinchart]

Hi Kieran,

On Tue, Jan 03, 2023 at 09:33:22AM +0000, Kieran Bingham wrote:
> Quoting Laurent Pinchart (2023-01-02 13:35:33)
> > On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote:
> > > On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote:
> > > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote:
> > > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote:
> > > > > 
> > > > > > Provide a streaming attribute to allow userspace to interogate if a device
> > > > > > is actively streaming or not.
> > > > > > 
> > > > > > This will allow desktop notifications to report if a camera or device
> > > > > > is active on the system, rather than just 'open' which can occur when
> > > > > > configuring the device.
> > > > > > 
> > > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
> > > > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com
> > > > > > ---
> > > > > > 
> > > > > > This is a quick POC to see if such a facility makes sense.
> > > > > > I'm weary that not all video devices may have the queues registered on
> > > > > > the struct video_device, but this seems like an effective way to be able
> > > > > > to determine if a device is actively streaming on a system.
> > > > > 
> > > > > I can imagine multiple problems, from race conditions to permissions and
> > > > > privacy. In order to comment on the fitness of this solution to address
> > > > > the problem you're trying to solve, could you describe the actual
> > > > > problem ?
> > > > 
> > > > The issue is explained in the following thread:
> > > > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388
> > > > 
> > > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera
> > > > is used. The script that the user previously had only checked if /dev/video0
> > > > was open in any process, if it was, the indicator was shown. However, libcamera
> > > > - at least at the moment - keeps the file descriptor open as long as the Camera
> > > > object exists, which pipewire keeps alive for the entire lifetime of the device,
> > > > therefore the "camera-in-use" indicator is always shown.
> > > 
> > > A sysfs attribute is not a great way to address this.
> > > 
> > > libcamera certainly has information on whether streaming is ongoing. The
> > > information should come from there. Or Pipewire. Dbus perhaps?
> > 
> > I tend to agree, I think this is best solved in userspace where PipeWire
> > can have a centralized view of all cameras in the system, and of their
> > users.
> 
> I fear that misses the entire point I was trying to make.
> 
> Lets say pipewire 'is' available and in use and can be used to capture
> video streams for video calls, that's fine. But what happens if a user
> runs a gstreamer pipeline without using the pipewire source, or a
> suspcious process runs "yavta" and captures an image or stream
> discreetly...
> 
> Only the kernel has a true centralised view of what devices are in use.

That's right, but at the same time, the kernel as little view of what a
"camera" is.

At the beginning of V4L a video capture node was a TV capture card (soon
with a few exceptions), then it also modelled a camera, for the past ten
years at least it's "just" a DMA engine in many cases, and relatively
recently even evolved to simply model a data flow endpoint with the
addition of metadata video nodes. This doesn't even mention usage of
video capture nodes in codecs or other memory to memory devices. Video
devices are now in many cases just one of the many components in a
camera pipeline.

In most cases drivers can reasonably decide which video devices most
likely represent a "camera", but that an approximation in any case, and
not a general guarantee. In userspace the situation is worse, the link
between a video device and a camera has been long lost.  We started
recovering it with libcamera, which is, today, the only open-source
component available in Linux systems that has knowledge of cameras, not
just video device nodes.

> > > Alternatively libcamera could close the video devices while not streaming
> > > but that would involve e.g. releasing possible video buffer allocations as
> > > well, increasing streaming start latency.
> 
> Or is it just that in that case 'lsof' should be sufficient?
> 
> The problem I have with that is - just like with the issue when the
> Privacy LED comes on during power up/probe - then any time a device is
> opened to identify the device and not necessarily use it - the 'camera
> in use' notification would get flashed...

Regardless of whether an open device node indication or a streaming
status is used, you don't want to indicate a camera is used because the
user is watching a movie and the V4L2-based codec is in use. You thus
need to at least filter out unrelated video devices in userspace, and if
you want to do so for privacy reasons, hardcoding in PipeWire (or
anywhere else) a heuristic will be prone to false positives or false
negatives. That isn't a good idea in my opinion, I believe this problem
can only be solved by handling the concept of "camera" in userspace.

> > Closing video (and subdev) nodes when the camera is not in use would be
> > good I think. It doesn't mean we have to open them when starting
> > capture, explicit open/close operation (or similar, maybe introducing a
> > capture session object in the libcamera API would also make sense, it
> > should be considered as part of the same issue) could help with this.
> 
> I'm not talking about libcamera in this thread. It's how does a user
> correctly identify when a camera is in use globally in a system.

-- 
Regards,

Laurent Pinchart

Re: [RFC PATCH] media: v4l2-dev: sysfs: Support streaming attribute

[Kieran Bingham]

Quoting Laurent Pinchart (2023-01-03 13:31:28)
> Hi Kieran,
> 
> On Tue, Jan 03, 2023 at 09:33:22AM +0000, Kieran Bingham wrote:
> > Quoting Laurent Pinchart (2023-01-02 13:35:33)
> > > On Mon, Jan 02, 2023 at 01:14:15PM +0000, Sakari Ailus wrote:
> > > > On Wed, Dec 28, 2022 at 01:44:38AM +0000, Barnabás Pőcze wrote:
> > > > > On 2022. december 26., hétfő 10:52, Laurent Pinchart wrote:
> > > > > > On Fri, Dec 23, 2022 at 11:17:35PM +0000, Kieran Bingham wrote:
> > > > > > 
> > > > > > > Provide a streaming attribute to allow userspace to interogate if a device
> > > > > > > is actively streaming or not.
> > > > > > > 
> > > > > > > This will allow desktop notifications to report if a camera or device
> > > > > > > is active on the system, rather than just 'open' which can occur when
> > > > > > > configuring the device.
> > > > > > > 
> > > > > > > Bug: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669
> > > > > > > Signed-off-by: Kieran Bingham kieran.bingham@ideasonboard.com
> > > > > > > ---
> > > > > > > 
> > > > > > > This is a quick POC to see if such a facility makes sense.
> > > > > > > I'm weary that not all video devices may have the queues registered on
> > > > > > > the struct video_device, but this seems like an effective way to be able
> > > > > > > to determine if a device is actively streaming on a system.
> > > > > > 
> > > > > > I can imagine multiple problems, from race conditions to permissions and
> > > > > > privacy. In order to comment on the fitness of this solution to address
> > > > > > the problem you're trying to solve, could you describe the actual
> > > > > > problem ?
> > > > > 
> > > > > The issue is explained in the following thread:
> > > > > https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2669#note_1697388
> > > > > 
> > > > > In short, the user wants to show a "camera-in-use" indicator when the laptop camera
> > > > > is used. The script that the user previously had only checked if /dev/video0
> > > > > was open in any process, if it was, the indicator was shown. However, libcamera
> > > > > - at least at the moment - keeps the file descriptor open as long as the Camera
> > > > > object exists, which pipewire keeps alive for the entire lifetime of the device,
> > > > > therefore the "camera-in-use" indicator is always shown.
> > > > 
> > > > A sysfs attribute is not a great way to address this.
> > > > 
> > > > libcamera certainly has information on whether streaming is ongoing. The
> > > > information should come from there. Or Pipewire. Dbus perhaps?
> > > 
> > > I tend to agree, I think this is best solved in userspace where PipeWire
> > > can have a centralized view of all cameras in the system, and of their
> > > users.
> > 
> > I fear that misses the entire point I was trying to make.
> > 
> > Lets say pipewire 'is' available and in use and can be used to capture
> > video streams for video calls, that's fine. But what happens if a user
> > runs a gstreamer pipeline without using the pipewire source, or a
> > suspcious process runs "yavta" and captures an image or stream
> > discreetly...
> > 
> > Only the kernel has a true centralised view of what devices are in use.
> 
> That's right, but at the same time, the kernel as little view of what a
> "camera" is.
> 
> At the beginning of V4L a video capture node was a TV capture card (soon
> with a few exceptions), then it also modelled a camera, for the past ten
> years at least it's "just" a DMA engine in many cases, and relatively
> recently even evolved to simply model a data flow endpoint with the
> addition of metadata video nodes. This doesn't even mention usage of
> video capture nodes in codecs or other memory to memory devices. Video
> devices are now in many cases just one of the many components in a
> camera pipeline.
> 
> In most cases drivers can reasonably decide which video devices most
> likely represent a "camera", but that an approximation in any case, and
> not a general guarantee. In userspace the situation is worse, the link
> between a video device and a camera has been long lost.  We started
> recovering it with libcamera, which is, today, the only open-source
> component available in Linux systems that has knowledge of cameras, not
> just video device nodes.
> 
> > > > Alternatively libcamera could close the video devices while not streaming
> > > > but that would involve e.g. releasing possible video buffer allocations as
> > > > well, increasing streaming start latency.
> > 
> > Or is it just that in that case 'lsof' should be sufficient?
> > 
> > The problem I have with that is - just like with the issue when the
> > Privacy LED comes on during power up/probe - then any time a device is
> > opened to identify the device and not necessarily use it - the 'camera
> > in use' notification would get flashed...
> 
> Regardless of whether an open device node indication or a streaming
> status is used, you don't want to indicate a camera is used because the
> user is watching a movie and the V4L2-based codec is in use. You thus
> need to at least filter out unrelated video devices in userspace, and if
> you want to do so for privacy reasons, hardcoding in PipeWire (or
> anywhere else) a heuristic will be prone to false positives or false
> negatives. That isn't a good idea in my opinion, I believe this problem
> can only be solved by handling the concept of "camera" in userspace.

Indeed, with v4l2 based codecs, this gets far more complex.

I think it's a shame we can't get this state globally, in a way that
can't be 'hidden' or 'subverted' but I'll just consider this patch
dropped.

--
Kieran


> 
> > > Closing video (and subdev) nodes when the camera is not in use would be
> > > good I think. It doesn't mean we have to open them when starting
> > > capture, explicit open/close operation (or similar, maybe introducing a
> > > capture session object in the libcamera API would also make sense, it
> > > should be considered as part of the same issue) could help with this.
> > 
> > I'm not talking about libcamera in this thread. It's how does a user
> > correctly identify when a camera is in use globally in a system.
> 
> -- 
> Regards,
> 
> Laurent Pinchart