[PATCH 0/3 v7] uvcvideo: metadata nodes

[PATCH 0/3 v7] uvcvideo: metadata nodes

[Guennadi Liakhovetski]

From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>

Comments by Laurent and Hans addressed, thanks.

Regards
Guennadi

Guennadi Liakhovetski (3):
  V4L: Add a UVC Metadata format
  uvcvideo: add extensible device information
  uvcvideo: add a metadata device node

 Documentation/media/uapi/v4l/meta-formats.rst    |   1 +
 Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst |  50 ++++++
 drivers/media/usb/uvc/Makefile                   |   2 +-
 drivers/media/usb/uvc/uvc_driver.c               | 142 ++++++++++------
 drivers/media/usb/uvc/uvc_isight.c               |   2 +-
 drivers/media/usb/uvc/uvc_metadata.c             | 204 +++++++++++++++++++++++
 drivers/media/usb/uvc/uvc_queue.c                |  41 ++++-
 drivers/media/usb/uvc/uvc_video.c                | 127 ++++++++++++--
 drivers/media/usb/uvc/uvcvideo.h                 |  19 ++-
 drivers/media/v4l2-core/v4l2-ioctl.c             |   1 +
 include/uapi/linux/uvcvideo.h                    |  26 +++
 include/uapi/linux/videodev2.h                   |   1 +
 12 files changed, 546 insertions(+), 70 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
 create mode 100644 drivers/media/usb/uvc/uvc_metadata.c

-- 
1.9.3

[PATCH 1/3 v7] V4L: Add a UVC Metadata format

[Guennadi Liakhovetski]

From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>

Add a pixel format, used by the UVC driver to stream metadata.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
---

v7: alphabetic order, update documentation.

 Documentation/media/uapi/v4l/meta-formats.rst    |  1 +
 Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst | 50 ++++++++++++++++++++++++
 include/uapi/linux/videodev2.h                   |  1 +
 3 files changed, 52 insertions(+)
 create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst

diff --git a/Documentation/media/uapi/v4l/meta-formats.rst b/Documentation/media/uapi/v4l/meta-formats.rst
index 01e24e3..0c4e1ec 100644
--- a/Documentation/media/uapi/v4l/meta-formats.rst
+++ b/Documentation/media/uapi/v4l/meta-formats.rst
@@ -12,5 +12,6 @@ These formats are used for the :ref:`metadata` interface only.
 .. toctree::
     :maxdepth: 1
 
+    pixfmt-meta-uvc
     pixfmt-meta-vsp1-hgo
     pixfmt-meta-vsp1-hgt
diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst b/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
new file mode 100644
index 0000000..06f603c
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
@@ -0,0 +1,50 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-meta-fmt-uvc:
+
+*******************************
+V4L2_META_FMT_UVC ('UVCH')
+*******************************
+
+UVC Payload Header Data
+
+
+Description
+===========
+
+This format describes standard UVC metadata, extracted from UVC packet headers
+and provided by the UVC driver through metadata video nodes. That data includes
+exact copies of the standard part of UVC Payload Header contents and auxiliary
+timing information, required for precise interpretation of timestamps, contained
+in those headers. See section "2.4.3.3 Video and Still Image Payload Headers" of
+the "UVC 1.5 Class specification" for details.
+
+Each UVC payload header can be between 2 and 12 bytes large. Buffers can contain
+multiple headers, if multiple such headers have been transmitted by the camera
+for the respective frame. However, headers, containing no useful information,
+e.g. those without the SCR field or with that field identical to the previous
+header, will be dropped by the driver.
+
+Each individual block contains the following fields:
+
+.. flat-table:: UVC Metadata Block
+    :widths: 1 4
+    :header-rows:  1
+    :stub-columns: 0
+
+    * - Field
+      - Description
+    * - __u64 ts;
+      - system timestamp in host byte order, measured by the driver upon
+        reception of the payload
+    * - __u16 sof;
+      - USB Frame Number in host byte order, also obtained by the driver as
+        close as possible to the above timestamp to enable correlation between
+        them
+    * - :cspan:`1` *The rest is an exact copy of the UVC payload header:*
+    * - __u8 length;
+      - length of the rest of the block, including this field
+    * - __u8 flags;
+      - Flags, indicating presence of other standard UVC fields
+    * - __u8 buf[];
+      - The rest of the header, possibly including UVC PTS and SCR fields
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 185d6a0..0d07b2d 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -687,6 +687,7 @@ struct v4l2_pix_format {
 /* Meta-data formats */
 #define V4L2_META_FMT_VSP1_HGO    v4l2_fourcc('V', 'S', 'P', 'H') /* R-Car VSP1 1-D Histogram */
 #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
+#define V4L2_META_FMT_UVC         v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
 
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
-- 
1.9.3

Re: [PATCH 1/3 v7] V4L: Add a UVC Metadata format

[Laurent Pinchart]

Hi Guennadi,

Thank you for the patch.

Overall this looks good to me. Please see below for one small comment.

On Wednesday, 8 November 2017 18:00:12 EET Guennadi Liakhovetski wrote:
> From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> 
> Add a pixel format, used by the UVC driver to stream metadata.
> 
> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> ---
> 
> v7: alphabetic order, update documentation.
> 
>  Documentation/media/uapi/v4l/meta-formats.rst    |  1 +
>  Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst | 50 +++++++++++++++++++++
>  include/uapi/linux/videodev2.h                   |  1 +
>  3 files changed, 52 insertions(+)
>  create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
> 
> diff --git a/Documentation/media/uapi/v4l/meta-formats.rst
> b/Documentation/media/uapi/v4l/meta-formats.rst index 01e24e3..0c4e1ec
> 100644
> --- a/Documentation/media/uapi/v4l/meta-formats.rst
> +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> @@ -12,5 +12,6 @@ These formats are used for the :ref:`metadata` interface
> only.
>  .. toctree::
>      :maxdepth: 1
> 
> +    pixfmt-meta-uvc
>      pixfmt-meta-vsp1-hgo
>      pixfmt-meta-vsp1-hgt
> diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
> b/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst new file mode 100644
> index 0000000..06f603c
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
> @@ -0,0 +1,50 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _v4l2-meta-fmt-uvc:
> +
> +*******************************
> +V4L2_META_FMT_UVC ('UVCH')
> +*******************************
> +
> +UVC Payload Header Data
> +
> +
> +Description
> +===========
> +
> +This format describes standard UVC metadata, extracted from UVC packet
> headers +and provided by the UVC driver through metadata video nodes. That
> data includes +exact copies of the standard part of UVC Payload Header
> contents and auxiliary +timing information, required for precise
> interpretation of timestamps, contained +in those headers. See section
> "2.4.3.3 Video and Still Image Payload Headers" of +the "UVC 1.5 Class
> specification" for details.
> +
> +Each UVC payload header can be between 2 and 12 bytes large. Buffers can
> contain +multiple headers, if multiple such headers have been transmitted
> by the camera +for the respective frame. However, headers, containing no
> useful information, +e.g. those without the SCR field or with that field
> identical to the previous +header, will be dropped by the driver.

If the driver receives too many headers with different SCR (more than the 
buffer can hold for instance) it will have to drop some of them. The simplest 
implementation would be to start dropping them when the buffer is full, but 
I'd like to leave room for the driver to be a bit more clever and drop headers 
that have a SCR too close to the previous one for instance. I propose wording 
the above paragraph as follows.

"Each UVC payload header can be between 2 and 12 bytes large. Buffers can
contain multiple headers, if multiple such headers have been transmitted
by the camera for the respective frame. However, the driver may drop headers 
when the buffer is full, when they contain no useful information (e.g. those 
without the SCR field or with that field identical to the previous header), or 
generally to perform rate limiting when the device sends a large number of 
headers".

If you're fine with this there's no need to resent, I can update the 
documentation when applying, and

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +Each individual block contains the following fields:
> +
> +.. flat-table:: UVC Metadata Block
> +    :widths: 1 4
> +    :header-rows:  1
> +    :stub-columns: 0
> +
> +    * - Field
> +      - Description
> +    * - __u64 ts;
> +      - system timestamp in host byte order, measured by the driver upon
> +        reception of the payload
> +    * - __u16 sof;
> +      - USB Frame Number in host byte order, also obtained by the driver as
> +        close as possible to the above timestamp to enable correlation
> between +        them
> +    * - :cspan:`1` *The rest is an exact copy of the UVC payload header:*
> +    * - __u8 length;
> +      - length of the rest of the block, including this field
> +    * - __u8 flags;
> +      - Flags, indicating presence of other standard UVC fields
> +    * - __u8 buf[];
> +      - The rest of the header, possibly including UVC PTS and SCR fields
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 185d6a0..0d07b2d 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -687,6 +687,7 @@ struct v4l2_pix_format {
>  /* Meta-data formats */
>  #define V4L2_META_FMT_VSP1_HGO    v4l2_fourcc('V', 'S', 'P', 'H') /* R-Car
> VSP1 1-D Histogram */ #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V',
> 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */ +#define V4L2_META_FMT_UVC   
>      v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> 
>  /* priv field value to indicates that subsequent fields are valid. */
>  #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe


-- 
Regards,

Laurent Pinchart

Re: [PATCH 1/3 v7] V4L: Add a UVC Metadata format

[Guennadi Liakhovetski]

Hi Laurent,

Thanks for the review.

On Tue, 28 Nov 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> Thank you for the patch.
> 
> Overall this looks good to me. Please see below for one small comment.

Yes, looks good to me, feel free to use your wording.

Thanks
Guennadi

> On Wednesday, 8 November 2017 18:00:12 EET Guennadi Liakhovetski wrote:
> > From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > 
> > Add a pixel format, used by the UVC driver to stream metadata.
> > 
> > Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > ---
> > 
> > v7: alphabetic order, update documentation.
> > 
> >  Documentation/media/uapi/v4l/meta-formats.rst    |  1 +
> >  Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst | 50 +++++++++++++++++++++
> >  include/uapi/linux/videodev2.h                   |  1 +
> >  3 files changed, 52 insertions(+)
> >  create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
> > 
> > diff --git a/Documentation/media/uapi/v4l/meta-formats.rst
> > b/Documentation/media/uapi/v4l/meta-formats.rst index 01e24e3..0c4e1ec
> > 100644
> > --- a/Documentation/media/uapi/v4l/meta-formats.rst
> > +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> > @@ -12,5 +12,6 @@ These formats are used for the :ref:`metadata` interface
> > only.
> >  .. toctree::
> >      :maxdepth: 1
> > 
> > +    pixfmt-meta-uvc
> >      pixfmt-meta-vsp1-hgo
> >      pixfmt-meta-vsp1-hgt
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
> > b/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst new file mode 100644
> > index 0000000..06f603c
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-meta-uvc.rst
> > @@ -0,0 +1,50 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _v4l2-meta-fmt-uvc:
> > +
> > +*******************************
> > +V4L2_META_FMT_UVC ('UVCH')
> > +*******************************
> > +
> > +UVC Payload Header Data
> > +
> > +
> > +Description
> > +===========
> > +
> > +This format describes standard UVC metadata, extracted from UVC packet
> > headers +and provided by the UVC driver through metadata video nodes. That
> > data includes +exact copies of the standard part of UVC Payload Header
> > contents and auxiliary +timing information, required for precise
> > interpretation of timestamps, contained +in those headers. See section
> > "2.4.3.3 Video and Still Image Payload Headers" of +the "UVC 1.5 Class
> > specification" for details.
> > +
> > +Each UVC payload header can be between 2 and 12 bytes large. Buffers can
> > contain +multiple headers, if multiple such headers have been transmitted
> > by the camera +for the respective frame. However, headers, containing no
> > useful information, +e.g. those without the SCR field or with that field
> > identical to the previous +header, will be dropped by the driver.
> 
> If the driver receives too many headers with different SCR (more than the 
> buffer can hold for instance) it will have to drop some of them. The simplest 
> implementation would be to start dropping them when the buffer is full, but 
> I'd like to leave room for the driver to be a bit more clever and drop headers 
> that have a SCR too close to the previous one for instance. I propose wording 
> the above paragraph as follows.
> 
> "Each UVC payload header can be between 2 and 12 bytes large. Buffers can
> contain multiple headers, if multiple such headers have been transmitted
> by the camera for the respective frame. However, the driver may drop headers 
> when the buffer is full, when they contain no useful information (e.g. those 
> without the SCR field or with that field identical to the previous header), or 
> generally to perform rate limiting when the device sends a large number of 
> headers".
> 
> If you're fine with this there's no need to resent, I can update the 
> documentation when applying, and
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> > +Each individual block contains the following fields:
> > +
> > +.. flat-table:: UVC Metadata Block
> > +    :widths: 1 4
> > +    :header-rows:  1
> > +    :stub-columns: 0
> > +
> > +    * - Field
> > +      - Description
> > +    * - __u64 ts;
> > +      - system timestamp in host byte order, measured by the driver upon
> > +        reception of the payload
> > +    * - __u16 sof;
> > +      - USB Frame Number in host byte order, also obtained by the driver as
> > +        close as possible to the above timestamp to enable correlation
> > between +        them
> > +    * - :cspan:`1` *The rest is an exact copy of the UVC payload header:*
> > +    * - __u8 length;
> > +      - length of the rest of the block, including this field
> > +    * - __u8 flags;
> > +      - Flags, indicating presence of other standard UVC fields
> > +    * - __u8 buf[];
> > +      - The rest of the header, possibly including UVC PTS and SCR fields
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 185d6a0..0d07b2d 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -687,6 +687,7 @@ struct v4l2_pix_format {
> >  /* Meta-data formats */
> >  #define V4L2_META_FMT_VSP1_HGO    v4l2_fourcc('V', 'S', 'P', 'H') /* R-Car
> > VSP1 1-D Histogram */ #define V4L2_META_FMT_VSP1_HGT    v4l2_fourcc('V',
> > 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */ +#define V4L2_META_FMT_UVC   
> >      v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> > 
> >  /* priv field value to indicates that subsequent fields are valid. */
> >  #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
> 
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

[PATCH 2/3 v7] uvcvideo: add extensible device information

[Guennadi Liakhovetski]

From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>

Currently the UVC driver assigns a quirk bitmask to the .driver_info
field of struct usb_device_id. This patch instroduces a struct to store
quirks and possibly other per-device parameters in the future.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
---

v7: this is a new patch in the series. Needed to enable specifying
private camera metadata formats

 drivers/media/usb/uvc/uvc_driver.c | 127 +++++++++++++++++++++++--------------
 1 file changed, 78 insertions(+), 49 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 6d22b22..cbf79b9 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -2001,11 +2001,18 @@ static int uvc_register_chains(struct uvc_device *dev)
  * USB probe, disconnect, suspend and resume
  */
 
+struct uvc_device_info {
+	u32	quirks;
+};
+
 static int uvc_probe(struct usb_interface *intf,
 		     const struct usb_device_id *id)
 {
 	struct usb_device *udev = interface_to_usbdev(intf);
 	struct uvc_device *dev;
+	const struct uvc_device_info *info =
+		(const struct uvc_device_info *)id->driver_info;
+	u32 quirks = info ? info->quirks : 0;
 	int function;
 	int ret;
 
@@ -2032,7 +2039,7 @@ static int uvc_probe(struct usb_interface *intf,
 	dev->intf = usb_get_intf(intf);
 	dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
 	dev->quirks = (uvc_quirks_param == -1)
-		    ? id->driver_info : uvc_quirks_param;
+		    ? quirks : uvc_quirks_param;
 
 	if (udev->product != NULL)
 		strlcpy(dev->name, udev->product, sizeof dev->name);
@@ -2073,7 +2080,7 @@ static int uvc_probe(struct usb_interface *intf,
 		le16_to_cpu(udev->descriptor.idVendor),
 		le16_to_cpu(udev->descriptor.idProduct));
 
-	if (dev->quirks != id->driver_info) {
+	if (dev->quirks != quirks) {
 		uvc_printk(KERN_INFO, "Forcing device quirks to 0x%x by module "
 			"parameter for testing purpose.\n", dev->quirks);
 		uvc_printk(KERN_INFO, "Please report required quirks to the "
@@ -2271,6 +2278,28 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
  * Driver initialization and cleanup
  */
 
+static struct uvc_device_info uvc_quirk_probe_minmax = {
+	.quirks = UVC_QUIRK_PROBE_MINMAX,
+};
+
+static struct uvc_device_info uvc_quirk_fix_bandwidth = {
+	.quirks = UVC_QUIRK_FIX_BANDWIDTH,
+};
+
+static struct uvc_device_info uvc_quirk_probe_def = {
+	.quirks = UVC_QUIRK_PROBE_DEF,
+};
+
+static struct uvc_device_info uvc_quirk_stream_no_fid = {
+	.quirks = UVC_QUIRK_STREAM_NO_FID,
+};
+
+static struct uvc_device_info uvc_quirk_force_y8 = {
+	.quirks = UVC_QUIRK_FORCE_Y8,
+};
+
+#define UVC_QUIRK_INFO(q) (kernel_ulong_t)&(struct uvc_device_info){.quirks = q}
+
 /*
  * The Logitech cameras listed below have their interface class set to
  * VENDOR_SPEC because they don't announce themselves as UVC devices, even
@@ -2285,7 +2314,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* Genius eFace 2025 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2294,7 +2323,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* Microsoft Lifecam NX-6000 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2303,7 +2332,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* Microsoft Lifecam NX-3000 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2312,7 +2341,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_DEF },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
 	/* Microsoft Lifecam VX-7000 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2321,7 +2350,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* Logitech Quickcam Fusion */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2378,7 +2407,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_RESTORE_CTRLS_ON_INIT },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_RESTORE_CTRLS_ON_INIT) },
 	/* Chicony CNF7129 (Asus EEE 100HE) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2387,7 +2416,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_RESTRICT_FRAME_RATE },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_RESTRICT_FRAME_RATE) },
 	/* Alcor Micro AU3820 (Future Boy PC USB Webcam) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2396,7 +2425,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* Dell XPS m1530 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2405,7 +2434,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info 		= UVC_QUIRK_PROBE_DEF },
+	  .driver_info 		= (kernel_ulong_t)&uvc_quirk_probe_def },
 	/* Dell SP2008WFP Monitor */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2414,7 +2443,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info 		= UVC_QUIRK_PROBE_DEF },
+	  .driver_info 		= (kernel_ulong_t)&uvc_quirk_probe_def },
 	/* Dell Alienware X51 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2423,7 +2452,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info	= UVC_QUIRK_PROBE_DEF },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_DEF) },
 	/* Dell Studio Hybrid 140g (OmniVision webcam) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2432,7 +2461,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_DEF },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
 	/* Dell XPS M1330 (OmniVision OV7670 webcam) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2441,7 +2470,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_DEF },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
 	/* Apple Built-In iSight */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2450,8 +2479,8 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info 		= UVC_QUIRK_PROBE_MINMAX
-				| UVC_QUIRK_BUILTIN_ISIGHT },
+	  .driver_info 		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
+					| UVC_QUIRK_BUILTIN_ISIGHT) },
 	/* Apple Built-In iSight via iBridge */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2460,7 +2489,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_DEF },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
 	/* Foxlink ("HP Webcam" on HP Mini 5103) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2469,7 +2498,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
 	/* Genesys Logic USB 2.0 PC Camera */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2478,7 +2507,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Hercules Classic Silver */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2487,7 +2516,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
 	/* ViMicro Vega */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2496,7 +2525,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
 	/* ViMicro - Minoru3D */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2505,7 +2534,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
 	/* ViMicro Venus - Minoru3D */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2514,7 +2543,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
 	/* Ophir Optronics - SPCAM 620U */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2523,7 +2552,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* MT6227 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2532,8 +2561,8 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
-				| UVC_QUIRK_PROBE_DEF },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
+					| UVC_QUIRK_PROBE_DEF) },
 	/* IMC Networks (Medion Akoya) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2542,7 +2571,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* JMicron USB2.0 XGA WebCam */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2551,7 +2580,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* Syntek (HP Spartan) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2560,7 +2589,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Syntek (Samsung Q310) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2569,7 +2598,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Syntek (Packard Bell EasyNote MX52 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2578,7 +2607,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Syntek (Asus F9SG) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2587,7 +2616,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Syntek (Asus U3S) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2596,7 +2625,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Syntek (JAOtech Smart Terminal) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2605,7 +2634,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Miricle 307K */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2614,7 +2643,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Lenovo Thinkpad SL400/SL500 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2623,7 +2652,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
 	/* Aveo Technology USB 2.0 Camera */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2632,8 +2661,8 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
-				| UVC_QUIRK_PROBE_EXTRAFIELDS },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
+					| UVC_QUIRK_PROBE_EXTRAFIELDS) },
 	/* Aveo Technology USB 2.0 Camera (Tasco USB Microscope) */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2650,7 +2679,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_EXTRAFIELDS },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_EXTRAFIELDS) },
 	/* Manta MM-353 Plako */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2659,7 +2688,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* FSC WebCam V30S */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2668,7 +2697,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* Arkmicro unbranded */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2677,7 +2706,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_DEF },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
 	/* The Imaging Source USB CCD cameras */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2696,7 +2725,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_STATUS_INTERVAL },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_STATUS_INTERVAL) },
 	/* MSI StarCam 370i */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2705,7 +2734,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
 	/* SiGma Micro USB Web Camera */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2714,8 +2743,8 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
-				| UVC_QUIRK_IGNORE_SELECTOR_UNIT },
+	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
+					| UVC_QUIRK_IGNORE_SELECTOR_UNIT) },
 	/* Oculus VR Positional Tracker DK2 */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2724,7 +2753,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VIDEO,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_FORCE_Y8 },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_force_y8 },
 	/* Oculus VR Rift Sensor */
 	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
 				| USB_DEVICE_ID_MATCH_INT_INFO,
@@ -2733,7 +2762,7 @@ static int uvc_clock_param_set(const char *val, struct kernel_param *kp)
 	  .bInterfaceClass	= USB_CLASS_VENDOR_SPEC,
 	  .bInterfaceSubClass	= 1,
 	  .bInterfaceProtocol	= 0,
-	  .driver_info		= UVC_QUIRK_FORCE_Y8 },
+	  .driver_info		= (kernel_ulong_t)&uvc_quirk_force_y8 },
 	/* Generic USB Video Class */
 	{ USB_INTERFACE_INFO(USB_CLASS_VIDEO, 1, UVC_PC_PROTOCOL_UNDEFINED) },
 	{ USB_INTERFACE_INFO(USB_CLASS_VIDEO, 1, UVC_PC_PROTOCOL_15) },
-- 
1.9.3

Re: [PATCH 2/3 v7] uvcvideo: add extensible device information

[Laurent Pinchart]

Hi Guennadi,

Thank you for the patch.

On Wednesday, 8 November 2017 18:00:13 EET Guennadi Liakhovetski wrote:
> From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> 
> Currently the UVC driver assigns a quirk bitmask to the .driver_info
> field of struct usb_device_id. This patch instroduces a struct to store
> quirks and possibly other per-device parameters in the future.

I'd drop the "possibly" as I'm fairly certain we will have other parameters in 
the future. Otherwise the patch would be a bit pointless :-)

> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> ---
> 
> v7: this is a new patch in the series. Needed to enable specifying
> private camera metadata formats
> 
>  drivers/media/usb/uvc/uvc_driver.c | 127 ++++++++++++++++++++--------------
>  1 file changed, 78 insertions(+), 49 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index 6d22b22..cbf79b9 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -2001,11 +2001,18 @@ static int uvc_register_chains(struct uvc_device
> *dev) * USB probe, disconnect, suspend and resume
>   */
> 
> +struct uvc_device_info {
> +	u32	quirks;
> +};
> +
>  static int uvc_probe(struct usb_interface *intf,
>  		     const struct usb_device_id *id)
>  {
>  	struct usb_device *udev = interface_to_usbdev(intf);
>  	struct uvc_device *dev;
> +	const struct uvc_device_info *info =
> +		(const struct uvc_device_info *)id->driver_info;
> +	u32 quirks = info ? info->quirks : 0;
>  	int function;
>  	int ret;
> 
> @@ -2032,7 +2039,7 @@ static int uvc_probe(struct usb_interface *intf,
>  	dev->intf = usb_get_intf(intf);
>  	dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
>  	dev->quirks = (uvc_quirks_param == -1)
> -		    ? id->driver_info : uvc_quirks_param;
> +		    ? quirks : uvc_quirks_param;
> 
>  	if (udev->product != NULL)
>  		strlcpy(dev->name, udev->product, sizeof dev->name);
> @@ -2073,7 +2080,7 @@ static int uvc_probe(struct usb_interface *intf,
>  		le16_to_cpu(udev->descriptor.idVendor),
>  		le16_to_cpu(udev->descriptor.idProduct));
> 
> -	if (dev->quirks != id->driver_info) {
> +	if (dev->quirks != quirks) {
>  		uvc_printk(KERN_INFO, "Forcing device quirks to 0x%x by module "
>  			"parameter for testing purpose.\n", dev->quirks);
>  		uvc_printk(KERN_INFO, "Please report required quirks to the "
> @@ -2271,6 +2278,28 @@ static int uvc_clock_param_set(const char *val,
> struct kernel_param *kp) * Driver initialization and cleanup
>   */
> 
> +static struct uvc_device_info uvc_quirk_probe_minmax = {
> +	.quirks = UVC_QUIRK_PROBE_MINMAX,
> +};
> +
> +static struct uvc_device_info uvc_quirk_fix_bandwidth = {
> +	.quirks = UVC_QUIRK_FIX_BANDWIDTH,
> +};
> +
> +static struct uvc_device_info uvc_quirk_probe_def = {
> +	.quirks = UVC_QUIRK_PROBE_DEF,
> +};
> +
> +static struct uvc_device_info uvc_quirk_stream_no_fid = {
> +	.quirks = UVC_QUIRK_STREAM_NO_FID,
> +};
> +
> +static struct uvc_device_info uvc_quirk_force_y8 = {
> +	.quirks = UVC_QUIRK_FORCE_Y8,
> +};

You can make all of these static const.

> +#define UVC_QUIRK_INFO(q) (kernel_ulong_t)&(struct uvc_device_info){.quirks
> = q}
> +
>  /*
>   * The Logitech cameras listed below have their interface class set to
>   * VENDOR_SPEC because they don't announce themselves as UVC devices, even
> @@ -2285,7 +2314,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* Genius eFace 2025 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2294,7 +2323,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* Microsoft Lifecam NX-6000 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2303,7 +2332,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* Microsoft Lifecam NX-3000 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2312,7 +2341,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
>  	/* Microsoft Lifecam VX-7000 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2321,7 +2350,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* Logitech Quickcam Fusion */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2378,7 +2407,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_RESTORE_CTRLS_ON_INIT },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_RESTORE_CTRLS_ON_INIT) },
>  	/* Chicony CNF7129 (Asus EEE 100HE) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2387,7 +2416,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_RESTRICT_FRAME_RATE },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_RESTRICT_FRAME_RATE) },
>  	/* Alcor Micro AU3820 (Future Boy PC USB Webcam) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2396,7 +2425,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* Dell XPS m1530 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2405,7 +2434,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info 		= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info 		= (kernel_ulong_t)&uvc_quirk_probe_def },
>  	/* Dell SP2008WFP Monitor */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2414,7 +2443,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info 		= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info 		= (kernel_ulong_t)&uvc_quirk_probe_def },
>  	/* Dell Alienware X51 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2423,7 +2452,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info	= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_DEF) },

This one can be (kernel_ulong_t)&uvc_quirk_probe_def.

If you're fine with those changes there's no need to resubmit, I'll update the 
patch when applying, and

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  	/* Dell Studio Hybrid 140g (OmniVision webcam) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2432,7 +2461,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
>  	/* Dell XPS M1330 (OmniVision OV7670 webcam) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2441,7 +2470,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
>  	/* Apple Built-In iSight */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2450,8 +2479,8 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info 		= UVC_QUIRK_PROBE_MINMAX
> -				| UVC_QUIRK_BUILTIN_ISIGHT },
> +	  .driver_info 		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> +					| UVC_QUIRK_BUILTIN_ISIGHT) },
>  	/* Apple Built-In iSight via iBridge */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2460,7 +2489,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
>  	/* Foxlink ("HP Webcam" on HP Mini 5103) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2469,7 +2498,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
>  	/* Genesys Logic USB 2.0 PC Camera */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2478,7 +2507,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Hercules Classic Silver */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2487,7 +2516,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
>  	/* ViMicro Vega */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2496,7 +2525,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
>  	/* ViMicro - Minoru3D */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2505,7 +2534,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
>  	/* ViMicro Venus - Minoru3D */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2514,7 +2543,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
>  	/* Ophir Optronics - SPCAM 620U */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2523,7 +2552,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* MT6227 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2532,8 +2561,8 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
> -				| UVC_QUIRK_PROBE_DEF },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> +					| UVC_QUIRK_PROBE_DEF) },
>  	/* IMC Networks (Medion Akoya) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2542,7 +2571,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* JMicron USB2.0 XGA WebCam */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2551,7 +2580,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* Syntek (HP Spartan) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2560,7 +2589,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Syntek (Samsung Q310) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2569,7 +2598,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Syntek (Packard Bell EasyNote MX52 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2578,7 +2607,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Syntek (Asus F9SG) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2587,7 +2616,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Syntek (Asus U3S) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2596,7 +2625,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Syntek (JAOtech Smart Terminal) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2605,7 +2634,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Miricle 307K */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2614,7 +2643,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Lenovo Thinkpad SL400/SL500 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2623,7 +2652,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
>  	/* Aveo Technology USB 2.0 Camera */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2632,8 +2661,8 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
> -				| UVC_QUIRK_PROBE_EXTRAFIELDS },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> +					| UVC_QUIRK_PROBE_EXTRAFIELDS) },
>  	/* Aveo Technology USB 2.0 Camera (Tasco USB Microscope) */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2650,7 +2679,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_EXTRAFIELDS },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_EXTRAFIELDS) },
>  	/* Manta MM-353 Plako */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2659,7 +2688,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* FSC WebCam V30S */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2668,7 +2697,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* Arkmicro unbranded */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2677,7 +2706,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
>  	/* The Imaging Source USB CCD cameras */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2696,7 +2725,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_STATUS_INTERVAL },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_STATUS_INTERVAL) },
>  	/* MSI StarCam 370i */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2705,7 +2734,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
>  	/* SiGma Micro USB Web Camera */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2714,8 +2743,8 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
> -				| UVC_QUIRK_IGNORE_SELECTOR_UNIT },
> +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> +					| UVC_QUIRK_IGNORE_SELECTOR_UNIT) },
>  	/* Oculus VR Positional Tracker DK2 */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2724,7 +2753,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_FORCE_Y8 },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_force_y8 },
>  	/* Oculus VR Rift Sensor */
>  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> 
>  				| USB_DEVICE_ID_MATCH_INT_INFO,
> 
> @@ -2733,7 +2762,7 @@ static int uvc_clock_param_set(const char *val, struct
> kernel_param *kp) .bInterfaceClass	= USB_CLASS_VENDOR_SPEC,
>  	  .bInterfaceSubClass	= 1,
>  	  .bInterfaceProtocol	= 0,
> -	  .driver_info		= UVC_QUIRK_FORCE_Y8 },
> +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_force_y8 },
>  	/* Generic USB Video Class */
>  	{ USB_INTERFACE_INFO(USB_CLASS_VIDEO, 1, UVC_PC_PROTOCOL_UNDEFINED) },
>  	{ USB_INTERFACE_INFO(USB_CLASS_VIDEO, 1, UVC_PC_PROTOCOL_15) },


-- 
Regards,

Laurent Pinchart

Re: [PATCH 2/3 v7] uvcvideo: add extensible device information

[Guennadi Liakhovetski]

Hi Laurent,

Thanks for reviewing.

On Tue, 28 Nov 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> Thank you for the patch.
> 
> On Wednesday, 8 November 2017 18:00:13 EET Guennadi Liakhovetski wrote:
> > From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > 
> > Currently the UVC driver assigns a quirk bitmask to the .driver_info
> > field of struct usb_device_id. This patch instroduces a struct to store
> > quirks and possibly other per-device parameters in the future.
> 
> I'd drop the "possibly" as I'm fairly certain we will have other parameters in 
> the future. Otherwise the patch would be a bit pointless :-)
> 
> > Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > ---
> > 
> > v7: this is a new patch in the series. Needed to enable specifying
> > private camera metadata formats
> > 
> >  drivers/media/usb/uvc/uvc_driver.c | 127 ++++++++++++++++++++--------------
> >  1 file changed, 78 insertions(+), 49 deletions(-)
> > 
> > diff --git a/drivers/media/usb/uvc/uvc_driver.c
> > b/drivers/media/usb/uvc/uvc_driver.c index 6d22b22..cbf79b9 100644
> > --- a/drivers/media/usb/uvc/uvc_driver.c
> > +++ b/drivers/media/usb/uvc/uvc_driver.c
> > @@ -2001,11 +2001,18 @@ static int uvc_register_chains(struct uvc_device
> > *dev) * USB probe, disconnect, suspend and resume
> >   */
> > 
> > +struct uvc_device_info {
> > +	u32	quirks;
> > +};
> > +
> >  static int uvc_probe(struct usb_interface *intf,
> >  		     const struct usb_device_id *id)
> >  {
> >  	struct usb_device *udev = interface_to_usbdev(intf);
> >  	struct uvc_device *dev;
> > +	const struct uvc_device_info *info =
> > +		(const struct uvc_device_info *)id->driver_info;
> > +	u32 quirks = info ? info->quirks : 0;
> >  	int function;
> >  	int ret;
> > 
> > @@ -2032,7 +2039,7 @@ static int uvc_probe(struct usb_interface *intf,
> >  	dev->intf = usb_get_intf(intf);
> >  	dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
> >  	dev->quirks = (uvc_quirks_param == -1)
> > -		    ? id->driver_info : uvc_quirks_param;
> > +		    ? quirks : uvc_quirks_param;
> > 
> >  	if (udev->product != NULL)
> >  		strlcpy(dev->name, udev->product, sizeof dev->name);
> > @@ -2073,7 +2080,7 @@ static int uvc_probe(struct usb_interface *intf,
> >  		le16_to_cpu(udev->descriptor.idVendor),
> >  		le16_to_cpu(udev->descriptor.idProduct));
> > 
> > -	if (dev->quirks != id->driver_info) {
> > +	if (dev->quirks != quirks) {
> >  		uvc_printk(KERN_INFO, "Forcing device quirks to 0x%x by module "
> >  			"parameter for testing purpose.\n", dev->quirks);
> >  		uvc_printk(KERN_INFO, "Please report required quirks to the "
> > @@ -2271,6 +2278,28 @@ static int uvc_clock_param_set(const char *val,
> > struct kernel_param *kp) * Driver initialization and cleanup
> >   */
> > 
> > +static struct uvc_device_info uvc_quirk_probe_minmax = {
> > +	.quirks = UVC_QUIRK_PROBE_MINMAX,
> > +};
> > +
> > +static struct uvc_device_info uvc_quirk_fix_bandwidth = {
> > +	.quirks = UVC_QUIRK_FIX_BANDWIDTH,
> > +};
> > +
> > +static struct uvc_device_info uvc_quirk_probe_def = {
> > +	.quirks = UVC_QUIRK_PROBE_DEF,
> > +};
> > +
> > +static struct uvc_device_info uvc_quirk_stream_no_fid = {
> > +	.quirks = UVC_QUIRK_STREAM_NO_FID,
> > +};
> > +
> > +static struct uvc_device_info uvc_quirk_force_y8 = {
> > +	.quirks = UVC_QUIRK_FORCE_Y8,
> > +};
> 
> You can make all of these static const.
> 
> > +#define UVC_QUIRK_INFO(q) (kernel_ulong_t)&(struct uvc_device_info){.quirks
> > = q}
> > +
> >  /*
> >   * The Logitech cameras listed below have their interface class set to
> >   * VENDOR_SPEC because they don't announce themselves as UVC devices, even
> > @@ -2285,7 +2314,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* Genius eFace 2025 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2294,7 +2323,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* Microsoft Lifecam NX-6000 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2303,7 +2332,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* Microsoft Lifecam NX-3000 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2312,7 +2341,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
> >  	/* Microsoft Lifecam VX-7000 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2321,7 +2350,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* Logitech Quickcam Fusion */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2378,7 +2407,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_RESTORE_CTRLS_ON_INIT },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_RESTORE_CTRLS_ON_INIT) },
> >  	/* Chicony CNF7129 (Asus EEE 100HE) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2387,7 +2416,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_RESTRICT_FRAME_RATE },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_RESTRICT_FRAME_RATE) },
> >  	/* Alcor Micro AU3820 (Future Boy PC USB Webcam) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2396,7 +2425,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* Dell XPS m1530 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2405,7 +2434,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info 		= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info 		= (kernel_ulong_t)&uvc_quirk_probe_def },
> >  	/* Dell SP2008WFP Monitor */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2414,7 +2443,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info 		= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info 		= (kernel_ulong_t)&uvc_quirk_probe_def },
> >  	/* Dell Alienware X51 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2423,7 +2452,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info	= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_DEF) },
> 
> This one can be (kernel_ulong_t)&uvc_quirk_probe_def.
> 
> If you're fine with those changes there's no need to resubmit, I'll update the 
> patch when applying, and
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Yes, please, do.

Thanks
Guennadi

> 
> >  	/* Dell Studio Hybrid 140g (OmniVision webcam) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2432,7 +2461,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
> >  	/* Dell XPS M1330 (OmniVision OV7670 webcam) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2441,7 +2470,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
> >  	/* Apple Built-In iSight */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2450,8 +2479,8 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info 		= UVC_QUIRK_PROBE_MINMAX
> > -				| UVC_QUIRK_BUILTIN_ISIGHT },
> > +	  .driver_info 		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > +					| UVC_QUIRK_BUILTIN_ISIGHT) },
> >  	/* Apple Built-In iSight via iBridge */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2460,7 +2489,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
> >  	/* Foxlink ("HP Webcam" on HP Mini 5103) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2469,7 +2498,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
> >  	/* Genesys Logic USB 2.0 PC Camera */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2478,7 +2507,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Hercules Classic Silver */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2487,7 +2516,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
> >  	/* ViMicro Vega */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2496,7 +2525,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
> >  	/* ViMicro - Minoru3D */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2505,7 +2534,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
> >  	/* ViMicro Venus - Minoru3D */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2514,7 +2543,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_FIX_BANDWIDTH },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_fix_bandwidth },
> >  	/* Ophir Optronics - SPCAM 620U */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2523,7 +2552,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* MT6227 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2532,8 +2561,8 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
> > -				| UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > +					| UVC_QUIRK_PROBE_DEF) },
> >  	/* IMC Networks (Medion Akoya) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2542,7 +2571,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* JMicron USB2.0 XGA WebCam */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2551,7 +2580,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* Syntek (HP Spartan) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2560,7 +2589,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Syntek (Samsung Q310) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2569,7 +2598,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Syntek (Packard Bell EasyNote MX52 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2578,7 +2607,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Syntek (Asus F9SG) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2587,7 +2616,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Syntek (Asus U3S) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2596,7 +2625,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Syntek (JAOtech Smart Terminal) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2605,7 +2634,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Miricle 307K */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2614,7 +2643,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Lenovo Thinkpad SL400/SL500 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2623,7 +2652,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STREAM_NO_FID },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_stream_no_fid },
> >  	/* Aveo Technology USB 2.0 Camera */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2632,8 +2661,8 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
> > -				| UVC_QUIRK_PROBE_EXTRAFIELDS },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > +					| UVC_QUIRK_PROBE_EXTRAFIELDS) },
> >  	/* Aveo Technology USB 2.0 Camera (Tasco USB Microscope) */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2650,7 +2679,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_EXTRAFIELDS },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_EXTRAFIELDS) },
> >  	/* Manta MM-353 Plako */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2659,7 +2688,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* FSC WebCam V30S */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2668,7 +2697,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* Arkmicro unbranded */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2677,7 +2706,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_DEF },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_def },
> >  	/* The Imaging Source USB CCD cameras */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2696,7 +2725,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_STATUS_INTERVAL },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_STATUS_INTERVAL) },
> >  	/* MSI StarCam 370i */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2705,7 +2734,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_probe_minmax },
> >  	/* SiGma Micro USB Web Camera */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2714,8 +2743,8 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_PROBE_MINMAX
> > -				| UVC_QUIRK_IGNORE_SELECTOR_UNIT },
> > +	  .driver_info		= UVC_QUIRK_INFO(UVC_QUIRK_PROBE_MINMAX
> > +					| UVC_QUIRK_IGNORE_SELECTOR_UNIT) },
> >  	/* Oculus VR Positional Tracker DK2 */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2724,7 +2753,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VIDEO,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_FORCE_Y8 },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_force_y8 },
> >  	/* Oculus VR Rift Sensor */
> >  	{ .match_flags		= USB_DEVICE_ID_MATCH_DEVICE
> > 
> >  				| USB_DEVICE_ID_MATCH_INT_INFO,
> > 
> > @@ -2733,7 +2762,7 @@ static int uvc_clock_param_set(const char *val, struct
> > kernel_param *kp) .bInterfaceClass	= USB_CLASS_VENDOR_SPEC,
> >  	  .bInterfaceSubClass	= 1,
> >  	  .bInterfaceProtocol	= 0,
> > -	  .driver_info		= UVC_QUIRK_FORCE_Y8 },
> > +	  .driver_info		= (kernel_ulong_t)&uvc_quirk_force_y8 },
> >  	/* Generic USB Video Class */
> >  	{ USB_INTERFACE_INFO(USB_CLASS_VIDEO, 1, UVC_PC_PROTOCOL_UNDEFINED) },
> >  	{ USB_INTERFACE_INFO(USB_CLASS_VIDEO, 1, UVC_PC_PROTOCOL_15) },
> 
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

[PATCH 3/3 v7] uvcvideo: add a metadata device node

[Guennadi Liakhovetski]

Some UVC video cameras contain metadata in their payload headers. This
patch extracts that data, adding more clock synchronisation information,
on both bulk and isochronous endpoints and makes it available to the
user space on a separate video node, using the V4L2_CAP_META_CAPTURE
capability and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. Even
though different cameras will have different metadata formats, we use
the same V4L2_META_FMT_UVC pixel format for all of them. Users have to
parse data, based on the specific camera model information. This
version of the patch only creates such metadata nodes for cameras,
specifying a UVC_QUIRK_METADATA_NODE quirk flag.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
---

v7: support up to two metadata formats per camera - the standard one and
an optional private one, if specified in device information

 drivers/media/usb/uvc/Makefile       |   2 +-
 drivers/media/usb/uvc/uvc_driver.c   |  15 +++
 drivers/media/usb/uvc/uvc_isight.c   |   2 +-
 drivers/media/usb/uvc/uvc_metadata.c | 204 +++++++++++++++++++++++++++++++++++
 drivers/media/usb/uvc/uvc_queue.c    |  41 +++++--
 drivers/media/usb/uvc/uvc_video.c    | 127 ++++++++++++++++++++--
 drivers/media/usb/uvc/uvcvideo.h     |  19 +++-
 drivers/media/v4l2-core/v4l2-ioctl.c |   1 +
 include/uapi/linux/uvcvideo.h        |  26 +++++
 9 files changed, 416 insertions(+), 21 deletions(-)
 create mode 100644 drivers/media/usb/uvc/uvc_metadata.c

diff --git a/drivers/media/usb/uvc/Makefile b/drivers/media/usb/uvc/Makefile
index c26d12f..06c7cd3 100644
--- a/drivers/media/usb/uvc/Makefile
+++ b/drivers/media/usb/uvc/Makefile
@@ -1,5 +1,5 @@
 uvcvideo-objs  := uvc_driver.o uvc_queue.o uvc_v4l2.o uvc_video.o uvc_ctrl.o \
-		  uvc_status.o uvc_isight.o uvc_debugfs.o
+		  uvc_status.o uvc_isight.o uvc_debugfs.o uvc_metadata.o
 ifeq ($(CONFIG_MEDIA_CONTROLLER),y)
 uvcvideo-objs  += uvc_entity.o
 endif
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index cbf79b9..5f7ce97 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -1877,6 +1877,7 @@ static void uvc_unregister_video(struct uvc_device *dev)
 			continue;
 
 		video_unregister_device(&stream->vdev);
+		video_unregister_device(&stream->meta.vdev);
 
 		uvc_debugfs_cleanup_stream(stream);
 	}
@@ -1934,6 +1935,11 @@ static int uvc_register_video(struct uvc_device *dev,
 		return ret;
 	}
 
+	/* Register a metadata node, but ignore a possible failure, complete
+	 * registration of video nodes anyway.
+	 */
+	uvc_meta_register(stream);
+
 	if (stream->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
 		stream->chain->caps |= V4L2_CAP_VIDEO_CAPTURE;
 	else
@@ -2003,6 +2009,7 @@ static int uvc_register_chains(struct uvc_device *dev)
 
 struct uvc_device_info {
 	u32	quirks;
+	u32	meta_format;
 };
 
 static int uvc_probe(struct usb_interface *intf,
@@ -2038,8 +2045,16 @@ static int uvc_probe(struct usb_interface *intf,
 	dev->udev = usb_get_dev(udev);
 	dev->intf = usb_get_intf(intf);
 	dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
+	if (uvc_quirks_param != -1 &&
+	    uvc_quirks_param & UVC_DEV_FLAG_METADATA_NODE) {
+		uvc_quirks_param &= ~UVC_DEV_FLAG_METADATA_NODE;
+		if (uvc_quirks_param == 0)
+			uvc_quirks_param = -1;
+	}
 	dev->quirks = (uvc_quirks_param == -1)
 		    ? quirks : uvc_quirks_param;
+	if (info)
+		dev->meta_format = info->meta_format;
 
 	if (udev->product != NULL)
 		strlcpy(dev->name, udev->product, sizeof dev->name);
diff --git a/drivers/media/usb/uvc/uvc_isight.c b/drivers/media/usb/uvc/uvc_isight.c
index 8510e725..fb940cf 100644
--- a/drivers/media/usb/uvc/uvc_isight.c
+++ b/drivers/media/usb/uvc/uvc_isight.c
@@ -100,7 +100,7 @@ static int isight_decode(struct uvc_video_queue *queue, struct uvc_buffer *buf,
 }
 
 void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
-		struct uvc_buffer *buf)
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	int ret, i;
 
diff --git a/drivers/media/usb/uvc/uvc_metadata.c b/drivers/media/usb/uvc/uvc_metadata.c
new file mode 100644
index 0000000..21eeee9
--- /dev/null
+++ b/drivers/media/usb/uvc/uvc_metadata.c
@@ -0,0 +1,204 @@
+/*
+ *      uvc_metadata.c  --  USB Video Class driver - Metadata handling
+ *
+ *      Copyright (C) 2016
+ *          Guennadi Liakhovetski (guennadi.liakhovetski@intel.com)
+ *
+ *      This program is free software; you can redistribute it and/or modify
+ *      it under the terms of the GNU General Public License as published by
+ *      the Free Software Foundation; either version 2 of the License, or
+ *      (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/usb.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-v4l2.h>
+#include <media/videobuf2-vmalloc.h>
+
+#include "uvcvideo.h"
+
+/* -----------------------------------------------------------------------------
+ * videobuf2 Queue Operations
+ */
+
+/* -----------------------------------------------------------------------------
+ * V4L2 ioctls
+ */
+
+static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
+				  struct v4l2_capability *cap)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+
+	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
+	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
+	usb_make_path(stream->dev->udev, cap->bus_info, sizeof(cap->bus_info));
+
+	return 0;
+}
+
+static int uvc_meta_v4l2_get_format(struct file *file, void *fh,
+				    struct v4l2_format *format)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct v4l2_meta_format *fmt = &format->fmt.meta;
+
+	if (format->type != vfh->vdev->queue->type)
+		return -EINVAL;
+
+	memset(fmt, 0, sizeof(*fmt));
+
+	fmt->dataformat = stream->cur_meta_format;
+	fmt->buffersize = UVC_METATADA_BUF_SIZE;
+
+	return 0;
+}
+
+static int uvc_meta_v4l2_try_format(struct file *file, void *fh,
+				    struct v4l2_format *format)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct uvc_device *dev = stream->dev;
+	struct v4l2_meta_format *fmt = &format->fmt.meta;
+	u32 fmeta = fmt->dataformat;
+
+	if (format->type != vfh->vdev->queue->type)
+		return -EINVAL;
+
+	memset(fmt, 0, sizeof(*fmt));
+
+	fmt->dataformat = fmeta == dev->meta_format ? fmeta : V4L2_META_FMT_UVC;
+	fmt->buffersize = UVC_METATADA_BUF_SIZE;
+
+	return 0;
+}
+
+static int uvc_meta_v4l2_set_format(struct file *file, void *fh,
+				    struct v4l2_format *format)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct v4l2_meta_format *fmt = &format->fmt.meta;
+	int ret = uvc_meta_v4l2_try_format(file, fh, format);
+	if (ret < 0)
+		return ret;
+
+	/*
+	 * We could in principle switch at any time, also during streaming.
+	 * Metadata buffers would still be perfectly parseable, but it's more
+	 * consistent and cleaner to disallow that
+	 */
+	mutex_lock(&stream->mutex);
+
+	if (uvc_queue_allocated(&stream->queue))
+		ret = -EBUSY;
+	else
+		stream->cur_meta_format = fmt->dataformat;
+
+	mutex_unlock(&stream->mutex);
+
+	return ret;
+}
+
+static int uvc_meta_v4l2_enum_formats(struct file *file, void *fh,
+				      struct v4l2_fmtdesc *fdesc)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct uvc_device *dev = stream->dev;
+	u32 index = fdesc->index;
+
+	if (fdesc->type != vfh->vdev->queue->type ||
+	    index > 1U || (index && !dev->meta_format))
+		return -EINVAL;
+
+	memset(fdesc, 0, sizeof(*fdesc));
+
+	fdesc->type = vfh->vdev->queue->type;
+	fdesc->index = index;
+	fdesc->pixelformat = index ? dev->meta_format : V4L2_META_FMT_UVC;
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops uvc_meta_ioctl_ops = {
+	.vidioc_querycap		= uvc_meta_v4l2_querycap,
+	.vidioc_g_fmt_meta_cap		= uvc_meta_v4l2_get_format,
+	.vidioc_s_fmt_meta_cap		= uvc_meta_v4l2_set_format,
+	.vidioc_try_fmt_meta_cap	= uvc_meta_v4l2_try_format,
+	.vidioc_enum_fmt_meta_cap	= uvc_meta_v4l2_enum_formats,
+	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
+	.vidioc_querybuf		= vb2_ioctl_querybuf,
+	.vidioc_qbuf			= vb2_ioctl_qbuf,
+	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
+	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
+	.vidioc_streamon		= vb2_ioctl_streamon,
+	.vidioc_streamoff		= vb2_ioctl_streamoff,
+};
+
+/* -----------------------------------------------------------------------------
+ * V4L2 File Operations
+ */
+
+static struct v4l2_file_operations uvc_meta_fops = {
+	.owner = THIS_MODULE,
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+};
+
+int uvc_meta_register(struct uvc_streaming *stream)
+{
+	struct uvc_device *dev = stream->dev;
+	struct uvc_meta_device *meta = &stream->meta;
+	struct video_device *vdev = &meta->vdev;
+	struct uvc_video_queue *quvc = &meta->queue;
+	int ret;
+
+	/*
+	 * Register metadata device nodes only if the METADATA_NODE quirk is
+	 * set.
+	 */
+	if (!(dev->quirks & UVC_DEV_FLAG_METADATA_NODE))
+		return 0;
+
+	/* COMMENT: consider re-using the code from uvc_driver.c::uvc_register_video() */
+	vdev->v4l2_dev = &dev->vdev;
+	vdev->fops = &uvc_meta_fops;
+	vdev->ioctl_ops = &uvc_meta_ioctl_ops;
+	vdev->release = stream->vdev.release;
+	vdev->prio = &stream->chain->prio;
+	vdev->vfl_dir = VFL_DIR_RX;
+	vdev->queue = &quvc->queue;
+	vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
+	strlcpy(vdev->name, dev->name, sizeof(vdev->name));
+
+	video_set_drvdata(vdev, stream);
+
+	/* Initialize the video buffer queue. */
+	ret = uvc_queue_init(quvc, V4L2_BUF_TYPE_META_CAPTURE, 0);
+	if (ret < 0)
+		return ret;
+
+	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+	if (ret < 0)
+		uvc_printk(KERN_ERR,
+			   "Failed to register metadata device (%d).\n", ret);
+	else
+		kref_get(&dev->ref);
+
+	stream->cur_meta_format = V4L2_META_FMT_UVC;
+
+	return ret;
+}
diff --git a/drivers/media/usb/uvc/uvc_queue.c b/drivers/media/usb/uvc/uvc_queue.c
index c8d78b2..b2998f5 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -79,8 +79,19 @@ static int uvc_queue_setup(struct vb2_queue *vq,
 			   unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
-	struct uvc_streaming *stream = uvc_queue_to_stream(queue);
-	unsigned size = stream->ctrl.dwMaxVideoFrameSize;
+	struct uvc_streaming *stream;
+	unsigned int size;
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_META_CAPTURE:
+		size = UVC_METATADA_BUF_SIZE;
+		break;
+
+	default:
+		stream = uvc_queue_to_stream(queue);
+		size = stream->ctrl.dwMaxVideoFrameSize;
+		break;
+	}
 
 	/*
 	 * When called with plane sizes, validate them. The driver supports
@@ -114,7 +125,7 @@ static int uvc_buffer_prepare(struct vb2_buffer *vb)
 	buf->error = 0;
 	buf->mem = vb2_plane_vaddr(vb, 0);
 	buf->length = vb2_plane_size(vb, 0);
-	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (vb->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
 		buf->bytesused = 0;
 	else
 		buf->bytesused = vb2_get_plane_payload(vb, 0);
@@ -177,10 +188,10 @@ static int uvc_start_streaming(struct vb2_queue *vq, unsigned int count)
 static void uvc_stop_streaming(struct vb2_queue *vq)
 {
 	struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
-	struct uvc_streaming *stream = uvc_queue_to_stream(queue);
 	unsigned long flags;
 
-	uvc_video_enable(stream, 0);
+	if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
+		uvc_video_enable(uvc_queue_to_stream(queue), 0);
 
 	spin_lock_irqsave(&queue->irqlock, flags);
 	uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
@@ -198,20 +209,36 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
 	.stop_streaming = uvc_stop_streaming,
 };
 
+static struct vb2_ops uvc_meta_queue_qops = {
+	.queue_setup = uvc_queue_setup,
+	.buf_prepare = uvc_buffer_prepare,
+	.buf_queue = uvc_buffer_queue,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.stop_streaming = uvc_stop_streaming,
+};
+
 int uvc_queue_init(struct uvc_video_queue *queue, enum v4l2_buf_type type,
 		    int drop_corrupted)
 {
 	int ret;
 
 	queue->queue.type = type;
-	queue->queue.io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+	queue->queue.io_modes = VB2_MMAP | VB2_USERPTR;
 	queue->queue.drv_priv = queue;
 	queue->queue.buf_struct_size = sizeof(struct uvc_buffer);
-	queue->queue.ops = &uvc_queue_qops;
 	queue->queue.mem_ops = &vb2_vmalloc_memops;
 	queue->queue.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC
 		| V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
 	queue->queue.lock = &queue->mutex;
+	switch (type) {
+	case V4L2_BUF_TYPE_META_CAPTURE:
+		queue->queue.ops = &uvc_meta_queue_qops;
+		break;
+	default:
+		queue->queue.io_modes |= VB2_DMABUF;
+		queue->queue.ops = &uvc_queue_qops;
+	}
 	ret = vb2_queue_init(&queue->queue);
 	if (ret)
 		return ret;
diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index fb86d6a..034b79b 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1135,6 +1135,76 @@ static int uvc_video_encode_data(struct uvc_streaming *stream,
 }
 
 /* ------------------------------------------------------------------------
+ * Metadata
+ */
+
+/*
+ * Ideally we want to capture all payload headers for each frame. However their
+ * number is unknown and unlimited. Additionally to the payload headers we want
+ * to provide the user with USB Frame Numbers and system time values. We also
+ * choose to drop headers, containing only a standard header, which either
+ * contains no SCR value or a value, equal to the previous payload. The
+ * resulting buffer is composed of blocks, containing a 64-bit timestamp in
+ * nanoseconds, a 16-bit USB Frame Number, and a copy of the payload header.
+ */
+static void uvc_video_decode_meta(struct uvc_streaming *stream,
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf,
+			u8 *mem, unsigned int length)
+{
+	struct uvc_meta_buf *meta;
+	size_t len_std = 2;
+	bool has_pts, has_scr;
+	unsigned long flags;
+	struct timespec ts;
+	u8 *scr;
+
+	if (!meta_buf || length == 2 ||
+	    meta_buf->length - meta_buf->bytesused <
+	    length + sizeof(meta->ns) + sizeof(meta->sof))
+		return;
+
+	has_pts = mem[1] & UVC_STREAM_PTS;
+	has_scr = mem[1] & UVC_STREAM_SCR;
+
+	if (has_pts) {
+		len_std += 4;
+		scr = mem + 6;
+	} else {
+		scr = mem + 2;
+	}
+
+	if (has_scr)
+		len_std += 6;
+
+	if (stream->cur_meta_format == V4L2_META_FMT_UVC)
+		length = len_std;
+
+	if (length == len_std && (!has_scr ||
+				  !memcmp(scr, stream->clock.last_scr, 6)))
+		return;
+
+	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
+	local_irq_save(flags);
+	uvc_video_get_ts(&ts);
+	meta->sof = usb_get_current_frame_number(stream->dev->udev);
+	local_irq_restore(flags);
+	meta->ns = timespec_to_ns(&ts);
+
+	if (has_scr)
+		memcpy(stream->clock.last_scr, scr, 6);
+
+	memcpy(&meta->length, mem, length);
+	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
+
+	uvc_trace(UVC_TRACE_FRAME,
+		  "%s(): t-sys %lu.%09lus, SOF %u, len %u, flags 0x%x, PTS %u, STC %u frame SOF %u\n",
+		  __func__, ts.tv_sec, ts.tv_nsec, meta->sof,
+		  meta->length, meta->flags, has_pts ? *(u32 *)meta->buf : 0,
+		  has_scr ? *(u32 *)scr : 0,
+		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
+}
+
+/* ------------------------------------------------------------------------
  * URB handling
  */
 
@@ -1152,8 +1222,28 @@ static void uvc_video_validate_buffer(const struct uvc_streaming *stream,
 /*
  * Completion handler for video URBs.
  */
+
+static struct uvc_buffer *uvc_video_step_buffers(struct uvc_streaming *stream,
+		struct uvc_buffer *video_buf, struct uvc_buffer **meta_buf)
+{
+	if (*meta_buf) {
+		struct vb2_v4l2_buffer *vb2_meta = &(*meta_buf)->buf;
+		const struct vb2_v4l2_buffer *vb2_video = &video_buf->buf;
+
+		vb2_meta->sequence = vb2_video->sequence;
+		vb2_meta->field = vb2_video->field;
+		vb2_meta->vb2_buf.timestamp = vb2_video->vb2_buf.timestamp;
+
+		(*meta_buf)->state = UVC_BUF_STATE_READY;
+		(*meta_buf)->error = video_buf->error;
+		*meta_buf = uvc_queue_next_buffer(&stream->meta.queue,
+						  *meta_buf);
+	}
+	return uvc_queue_next_buffer(&stream->queue, video_buf);
+}
+
 static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	u8 *mem;
 	int ret, i;
@@ -1175,14 +1265,16 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
 				urb->iso_frame_desc[i].actual_length);
 			if (ret == -EAGAIN) {
 				uvc_video_validate_buffer(stream, buf);
-				buf = uvc_queue_next_buffer(&stream->queue,
-							    buf);
+				buf = uvc_video_step_buffers(stream, buf,
+							     &meta_buf);
 			}
 		} while (ret == -EAGAIN);
 
 		if (ret < 0)
 			continue;
 
+		uvc_video_decode_meta(stream, buf, meta_buf, mem, ret);
+
 		/* Decode the payload data. */
 		uvc_video_decode_data(stream, buf, mem + ret,
 			urb->iso_frame_desc[i].actual_length - ret);
@@ -1193,13 +1285,13 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
 
 		if (buf->state == UVC_BUF_STATE_READY) {
 			uvc_video_validate_buffer(stream, buf);
-			buf = uvc_queue_next_buffer(&stream->queue, buf);
+			buf = uvc_video_step_buffers(stream, buf, &meta_buf);
 		}
 	}
 }
 
 static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	u8 *mem;
 	int len, ret;
@@ -1222,8 +1314,8 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 		do {
 			ret = uvc_video_decode_start(stream, buf, mem, len);
 			if (ret == -EAGAIN)
-				buf = uvc_queue_next_buffer(&stream->queue,
-							    buf);
+				buf = uvc_video_step_buffers(stream, buf,
+							     &meta_buf);
 		} while (ret == -EAGAIN);
 
 		/* If an error occurred skip the rest of the payload. */
@@ -1233,6 +1325,8 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 			memcpy(stream->bulk.header, mem, ret);
 			stream->bulk.header_size = ret;
 
+			uvc_video_decode_meta(stream, buf, meta_buf, mem, ret);
+
 			mem += ret;
 			len -= ret;
 		}
@@ -1256,7 +1350,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 			uvc_video_decode_end(stream, buf, stream->bulk.header,
 				stream->bulk.payload_size);
 			if (buf->state == UVC_BUF_STATE_READY)
-				uvc_queue_next_buffer(&stream->queue, buf);
+				uvc_video_step_buffers(stream, buf, &meta_buf);
 		}
 
 		stream->bulk.header_size = 0;
@@ -1266,7 +1360,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 }
 
 static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+	struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	u8 *mem = urb->transfer_buffer;
 	int len = stream->urb_size, ret;
@@ -1312,7 +1406,10 @@ static void uvc_video_complete(struct urb *urb)
 {
 	struct uvc_streaming *stream = urb->context;
 	struct uvc_video_queue *queue = &stream->queue;
+	struct uvc_video_queue *qmeta = &stream->meta.queue;
+	struct vb2_queue *vb2_qmeta = stream->meta.vdev.queue;
 	struct uvc_buffer *buf = NULL;
+	struct uvc_buffer *buf_meta = NULL;
 	unsigned long flags;
 	int ret;
 
@@ -1331,6 +1428,8 @@ static void uvc_video_complete(struct urb *urb)
 	case -ECONNRESET:	/* usb_unlink_urb() called. */
 	case -ESHUTDOWN:	/* The endpoint is being disabled. */
 		uvc_queue_cancel(queue, urb->status == -ESHUTDOWN);
+		if (vb2_qmeta)
+			uvc_queue_cancel(qmeta, urb->status == -ESHUTDOWN);
 		return;
 	}
 
@@ -1340,7 +1439,15 @@ static void uvc_video_complete(struct urb *urb)
 				       queue);
 	spin_unlock_irqrestore(&queue->irqlock, flags);
 
-	stream->decode(urb, stream, buf);
+	if (vb2_qmeta) {
+		spin_lock_irqsave(&qmeta->irqlock, flags);
+		if (!list_empty(&qmeta->irqqueue))
+			buf_meta = list_first_entry(&qmeta->irqqueue,
+						    struct uvc_buffer, queue);
+		spin_unlock_irqrestore(&qmeta->irqlock, flags);
+	}
+
+	stream->decode(urb, stream, buf, buf_meta);
 
 	if ((ret = usb_submit_urb(urb, GFP_ATOMIC)) < 0) {
 		uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 34c7ee6..a657bc1 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -185,6 +185,7 @@
 #define UVC_QUIRK_RESTRICT_FRAME_RATE	0x00000200
 #define UVC_QUIRK_RESTORE_CTRLS_ON_INIT	0x00000400
 #define UVC_QUIRK_FORCE_Y8		0x00000800
+#define UVC_DEV_FLAG_METADATA_NODE	0x80000000
 
 /* Format flags */
 #define UVC_FMT_FLAG_COMPRESSED		0x00000001
@@ -473,6 +474,13 @@ struct uvc_stats_stream {
 	unsigned int max_sof;		/* Maximum STC.SOF value */
 };
 
+struct uvc_meta_device {
+	struct video_device vdev;
+	struct uvc_video_queue queue;
+};
+
+#define UVC_METATADA_BUF_SIZE 1024
+
 struct uvc_streaming {
 	struct list_head list;
 	struct uvc_device *dev;
@@ -504,7 +512,10 @@ struct uvc_streaming {
 	unsigned int frozen : 1;
 	struct uvc_video_queue queue;
 	void (*decode) (struct urb *urb, struct uvc_streaming *video,
-			struct uvc_buffer *buf);
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf);
+
+	struct uvc_meta_device meta;
+	__u32 cur_meta_format;
 
 	/* Context data used by the bulk completion handler. */
 	struct {
@@ -546,6 +557,8 @@ struct uvc_streaming {
 		u16 last_sof;
 		u16 sof_offset;
 
+		u8 last_scr[6];
+
 		spinlock_t lock;
 	} clock;
 };
@@ -555,6 +568,7 @@ struct uvc_device {
 	struct usb_interface *intf;
 	unsigned long warnings;
 	__u32 quirks;
+	__u32 meta_format;
 	int intfnum;
 	char name[32];
 
@@ -709,6 +723,7 @@ extern int uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
 void uvc_video_clock_update(struct uvc_streaming *stream,
 			    struct vb2_v4l2_buffer *vbuf,
 			    struct uvc_buffer *buf);
+int uvc_meta_register(struct uvc_streaming *stream);
 
 /* Status */
 extern int uvc_status_init(struct uvc_device *dev);
@@ -763,7 +778,7 @@ extern struct usb_host_endpoint *uvc_find_endpoint(
 
 /* Quirks support */
 void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
-		struct uvc_buffer *buf);
+		struct uvc_buffer *buf, struct uvc_buffer *meta_buf);
 
 /* debugfs and statistics */
 void uvc_debugfs_init(void);
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 7961499..fba8021 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1246,6 +1246,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_TCH_FMT_TU08:		descr = "8-bit unsigned touch data"; break;
 	case V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break;
 	case V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break;
+	case V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/uvcvideo.h b/include/uapi/linux/uvcvideo.h
index 3b08186..8381ca7c 100644
--- a/include/uapi/linux/uvcvideo.h
+++ b/include/uapi/linux/uvcvideo.h
@@ -67,4 +67,30 @@ struct uvc_xu_control_query {
 #define UVCIOC_CTRL_MAP		_IOWR('u', 0x20, struct uvc_xu_control_mapping)
 #define UVCIOC_CTRL_QUERY	_IOWR('u', 0x21, struct uvc_xu_control_query)
 
+/*
+ * Metadata node
+ */
+
+/**
+ * struct uvc_meta_buf - metadata buffer building block
+ * @ns		- system timestamp of the payload in nanoseconds
+ * @sof		- USB Frame Number
+ * @length	- length of the payload header
+ * @flags	- payload header flags
+ * @buf		- optional device-specific header data
+ *
+ * UVC metadata nodes fill buffers with possibly multiple instances of this
+ * struct. The first two fields are added by the driver, they can be used for
+ * clock synchronisation. The rest is an exact copy of a UVC payload header.
+ * Only complete objects with complete buffers are included. Therefore it's
+ * always sizeof(meta->ts) + sizeof(meta->sof) + meta->length bytes large.
+ */
+struct uvc_meta_buf {
+	__u64 ns;
+	__u16 sof;
+	__u8 length;
+	__u8 flags;
+	__u8 buf[];
+} __packed;
+
 #endif
-- 
1.9.3

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

Thank you for the patch. We're getting very close, I only have small comments, 
please see below.

On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> Some UVC video cameras contain metadata in their payload headers. This
> patch extracts that data, adding more clock synchronisation information,
> on both bulk and isochronous endpoints and makes it available to the
> user space on a separate video node, using the V4L2_CAP_META_CAPTURE
> capability and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. Even
> though different cameras will have different metadata formats, we use
> the same V4L2_META_FMT_UVC pixel format for all of them. Users have to
> parse data, based on the specific camera model information. This
> version of the patch only creates such metadata nodes for cameras,
> specifying a UVC_QUIRK_METADATA_NODE quirk flag.

I don't think this is correct anymore, as we'll use different 4CCs for 
different vendor metadata. How would you like to rephrase the commit message ?

> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> ---
> 
> v7: support up to two metadata formats per camera - the standard one and
> an optional private one, if specified in device information
> 
>  drivers/media/usb/uvc/Makefile       |   2 +-
>  drivers/media/usb/uvc/uvc_driver.c   |  15 +++
>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
>  drivers/media/usb/uvc/uvc_metadata.c | 204 ++++++++++++++++++++++++++++++++
>  drivers/media/usb/uvc/uvc_queue.c    |  41 +++++--
>  drivers/media/usb/uvc/uvc_video.c    | 127 ++++++++++++++++++++--
>  drivers/media/usb/uvc/uvcvideo.h     |  19 +++-
>  drivers/media/v4l2-core/v4l2-ioctl.c |   1 +
>  include/uapi/linux/uvcvideo.h        |  26 +++++
>  9 files changed, 416 insertions(+), 21 deletions(-)
>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c

[snip]

> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index cbf79b9..5f7ce97 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -1877,6 +1877,7 @@ static void uvc_unregister_video(struct uvc_device
> *dev) continue;
> 
>  		video_unregister_device(&stream->vdev);
> +		video_unregister_device(&stream->meta.vdev);
> 
>  		uvc_debugfs_cleanup_stream(stream);
>  	}
> @@ -1934,6 +1935,11 @@ static int uvc_register_video(struct uvc_device *dev,
> return ret;
>  	}
> 
> +	/* Register a metadata node, but ignore a possible failure, complete
> +	 * registration of video nodes anyway.
> +	 */
> +	uvc_meta_register(stream);
> +
>  	if (stream->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
>  		stream->chain->caps |= V4L2_CAP_VIDEO_CAPTURE;
>  	else
> @@ -2003,6 +2009,7 @@ static int uvc_register_chains(struct uvc_device *dev)
> 
>  struct uvc_device_info {
>  	u32	quirks;
> +	u32	meta_format;
>  };
> 
>  static int uvc_probe(struct usb_interface *intf,
> @@ -2038,8 +2045,16 @@ static int uvc_probe(struct usb_interface *intf,
>  	dev->udev = usb_get_dev(udev);
>  	dev->intf = usb_get_intf(intf);
>  	dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
> +	if (uvc_quirks_param != -1 &&
> +	    uvc_quirks_param & UVC_DEV_FLAG_METADATA_NODE) {
> +		uvc_quirks_param &= ~UVC_DEV_FLAG_METADATA_NODE;
> +		if (uvc_quirks_param == 0)
> +			uvc_quirks_param = -1;
> +	}

I think we can remove the UVC_DEV_FLAG_METADATA_NODE flag. We can use the 
metadata format specified in the device information structure if present, and 
default to the standard metadata format otherwise. A metadata video node will 
always be registered, which I think is the right thing to do as exposing 
timestamp information to userspace is useful.

Do you see any adverse effect of registering a metadata video node 
unconditionally ?

>  	dev->quirks = (uvc_quirks_param == -1)
>  		    ? quirks : uvc_quirks_param;
> +	if (info)
> +		dev->meta_format = info->meta_format;
> 
>  	if (udev->product != NULL)
>  		strlcpy(dev->name, udev->product, sizeof dev->name);

[snip]

> diff --git a/drivers/media/usb/uvc/uvc_metadata.c
> b/drivers/media/usb/uvc/uvc_metadata.c new file mode 100644
> index 0000000..21eeee9
> --- /dev/null
> +++ b/drivers/media/usb/uvc/uvc_metadata.c

[snip]

> +/* ------------------------------------------------------------------------
> + * videobuf2 Queue Operations
> + */
> +

You can remove this now that the section is empty :-)

> +/* ------------------------------------------------------------------------
> + * V4L2 ioctls
> + */
> +
> +static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
> +				  struct v4l2_capability *cap)
> +{
> +	struct v4l2_fh *vfh = file->private_data;
> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> +
> +	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
> +	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
> +	usb_make_path(stream->dev->udev, cap->bus_info, sizeof(cap->bus_info));
> +
> +	return 0;
> +}

Do you think we could reuse uvc_ioctl_querycap() as-is ?

> +
> +static int uvc_meta_v4l2_get_format(struct file *file, void *fh,
> +				    struct v4l2_format *format)
> +{
> +	struct v4l2_fh *vfh = file->private_data;
> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> +	struct v4l2_meta_format *fmt = &format->fmt.meta;
> +
> +	if (format->type != vfh->vdev->queue->type)
> +		return -EINVAL;
> +
> +	memset(fmt, 0, sizeof(*fmt));
> +
> +	fmt->dataformat = stream->cur_meta_format;
> +	fmt->buffersize = UVC_METATADA_BUF_SIZE;

You need to take the stream->mutex lock here to protect against races with the 
set format ioctl.

> +	return 0;
> +}
> +
> +static int uvc_meta_v4l2_try_format(struct file *file, void *fh,
> +				    struct v4l2_format *format)
> +{
> +	struct v4l2_fh *vfh = file->private_data;
> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> +	struct uvc_device *dev = stream->dev;
> +	struct v4l2_meta_format *fmt = &format->fmt.meta;
> +	u32 fmeta = fmt->dataformat;
> +
> +	if (format->type != vfh->vdev->queue->type)
> +		return -EINVAL;
> +
> +	memset(fmt, 0, sizeof(*fmt));
> +
> +	fmt->dataformat = fmeta == dev->meta_format ? fmeta : V4L2_META_FMT_UVC;
> +	fmt->buffersize = UVC_METATADA_BUF_SIZE;
> +
> +	return 0;
> +}
> +
> +static int uvc_meta_v4l2_set_format(struct file *file, void *fh,
> +				    struct v4l2_format *format)
> +{
> +	struct v4l2_fh *vfh = file->private_data;
> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> +	struct v4l2_meta_format *fmt = &format->fmt.meta;
> +	int ret = uvc_meta_v4l2_try_format(file, fh, format);
> +	if (ret < 0)
> +		return ret;

I expect checkpatch to complain here that a blank line is missing between 
variable declaration and code. I would thus write it

	int ret;

	ret = uvc_meta_v4l2_try_format(file, fh, format);
	if (ret < 0)
		return ret;

> +
> +	/*
> +	 * We could in principle switch at any time, also during streaming.
> +	 * Metadata buffers would still be perfectly parseable, but it's more
> +	 * consistent and cleaner to disallow that

Nitpicking, could you add a period at the end of sentences in comments ?

> +	 */
> +	mutex_lock(&stream->mutex);
> +
> +	if (uvc_queue_allocated(&stream->queue))
> +		ret = -EBUSY;
> +	else
> +		stream->cur_meta_format = fmt->dataformat;
> +
> +	mutex_unlock(&stream->mutex);
> +
> +	return ret;
> +}
> +
> +static int uvc_meta_v4l2_enum_formats(struct file *file, void *fh,
> +				      struct v4l2_fmtdesc *fdesc)
> +{
> +	struct v4l2_fh *vfh = file->private_data;
> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> +	struct uvc_device *dev = stream->dev;
> +	u32 index = fdesc->index;
> +
> +	if (fdesc->type != vfh->vdev->queue->type ||
> +	    index > 1U || (index && !dev->meta_format))
> +		return -EINVAL;
> +
> +	memset(fdesc, 0, sizeof(*fdesc));
> +
> +	fdesc->type = vfh->vdev->queue->type;
> +	fdesc->index = index;
> +	fdesc->pixelformat = index ? dev->meta_format : V4L2_META_FMT_UVC;
> +
> +	return 0;
> +}
> +
> +static const struct v4l2_ioctl_ops uvc_meta_ioctl_ops = {
> +	.vidioc_querycap		= uvc_meta_v4l2_querycap,
> +	.vidioc_g_fmt_meta_cap		= uvc_meta_v4l2_get_format,
> +	.vidioc_s_fmt_meta_cap		= uvc_meta_v4l2_set_format,
> +	.vidioc_try_fmt_meta_cap	= uvc_meta_v4l2_try_format,
> +	.vidioc_enum_fmt_meta_cap	= uvc_meta_v4l2_enum_formats,
> +	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
> +	.vidioc_querybuf		= vb2_ioctl_querybuf,
> +	.vidioc_qbuf			= vb2_ioctl_qbuf,
> +	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
> +	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
> +	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
> +	.vidioc_streamon		= vb2_ioctl_streamon,
> +	.vidioc_streamoff		= vb2_ioctl_streamoff,
> +};
> +
> +/* ------------------------------------------------------------------------
> + * V4L2 File Operations
> + */
> +
> +static struct v4l2_file_operations uvc_meta_fops = {

This should be static const.

> +	.owner = THIS_MODULE,
> +	.unlocked_ioctl = video_ioctl2,
> +	.open = v4l2_fh_open,
> +	.release = vb2_fop_release,
> +	.poll = vb2_fop_poll,
> +	.mmap = vb2_fop_mmap,
> +};
> +
> +int uvc_meta_register(struct uvc_streaming *stream)
> +{
> +	struct uvc_device *dev = stream->dev;
> +	struct uvc_meta_device *meta = &stream->meta;
> +	struct video_device *vdev = &meta->vdev;
> +	struct uvc_video_queue *quvc = &meta->queue;
> +	int ret;
> +
> +	/*
> +	 * Register metadata device nodes only if the METADATA_NODE quirk is
> +	 * set.
> +	 */
> +	if (!(dev->quirks & UVC_DEV_FLAG_METADATA_NODE))
> +		return 0;
> +
> +	/* COMMENT: consider re-using the code from
> uvc_driver.c::uvc_register_video() */

Who is that comment for ? :-)

Could https://www.mail-archive.com/linux-media@vger.kernel.org/msg122737.html 
help ?

> +	vdev->v4l2_dev = &dev->vdev;
> +	vdev->fops = &uvc_meta_fops;
> +	vdev->ioctl_ops = &uvc_meta_ioctl_ops;
> +	vdev->release = stream->vdev.release;
> +	vdev->prio = &stream->chain->prio;
> +	vdev->vfl_dir = VFL_DIR_RX;
> +	vdev->queue = &quvc->queue;

uvc_register_video() doesn't set vdev->queue, because the driver handles 
locking manually. While thus could possibly be reworked, careful consideration 
is needed, so I would probably keep queue unset in uvc_register_video_device() 
and set it here.

> +	vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
> +	strlcpy(vdev->name, dev->name, sizeof(vdev->name));
> +
> +	video_set_drvdata(vdev, stream);
> +
> +	/* Initialize the video buffer queue. */
> +	ret = uvc_queue_init(quvc, V4L2_BUF_TYPE_META_CAPTURE, 0);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> +	if (ret < 0)
> +		uvc_printk(KERN_ERR,
> +			   "Failed to register metadata device (%d).\n", ret);
> +	else
> +		kref_get(&dev->ref);
> +
> +	stream->cur_meta_format = V4L2_META_FMT_UVC;

You should set this before registering the video device, otherwise you risk a 
race with userspace.

> +
> +	return ret;
> +}

With the uvc_register_video_device() helper this function would become

int uvc_meta_register(struct uvc_streaming *stream)
{
	struct uvc_device *dev = stream->dev;
	struct video_device *vdev = &stream->meta.vdev;
	struct uvc_video_queue *queue = &stream->meta.queue;

	/*
	 * Register metadata device nodes only if the METADATA_NODE quirk is
	 * set.
	 */
	if (!(dev->quirks & UVC_DEV_FLAG_METADATA_NODE))
		return 0;

	stream->cur_meta_format = V4L2_META_FMT_UVC;

	/*
	 * The video interface queue uses manual locking and thus does not set
	 * the queue pointer. Set it manually here.
	 */
	vdev->queue = &queue->queue;

	return uvc_register_video_device(dev, stream, vdev, queue,
					 V4L2_BUF_TYPE_META_CAPTURE,
					 &uvc_meta_fops, &uvc_meta_ioctl_ops);
}

What do you think, is it worth it ?

> diff --git a/drivers/media/usb/uvc/uvc_queue.c
> b/drivers/media/usb/uvc/uvc_queue.c index c8d78b2..b2998f5 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c

[snip]

> @@ -198,20 +209,36 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
>  	.stop_streaming = uvc_stop_streaming,
>  };
> 
> +static struct vb2_ops uvc_meta_queue_qops = {

This should be static const.

> +	.queue_setup = uvc_queue_setup,
> +	.buf_prepare = uvc_buffer_prepare,
> +	.buf_queue = uvc_buffer_queue,
> +	.wait_prepare = vb2_ops_wait_prepare,
> +	.wait_finish = vb2_ops_wait_finish,
> +	.stop_streaming = uvc_stop_streaming,
> +};
> +
>  int uvc_queue_init(struct uvc_video_queue *queue, enum v4l2_buf_type type,
>  		    int drop_corrupted)
>  {
>  	int ret;
> 
>  	queue->queue.type = type;
> -	queue->queue.io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
> +	queue->queue.io_modes = VB2_MMAP | VB2_USERPTR;
>  	queue->queue.drv_priv = queue;
>  	queue->queue.buf_struct_size = sizeof(struct uvc_buffer);
> -	queue->queue.ops = &uvc_queue_qops;
>  	queue->queue.mem_ops = &vb2_vmalloc_memops;
>  	queue->queue.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC
>  		| V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
>  	queue->queue.lock = &queue->mutex;
> +	switch (type) {
> +	case V4L2_BUF_TYPE_META_CAPTURE:
> +		queue->queue.ops = &uvc_meta_queue_qops;
> +		break;
> +	default:
> +		queue->queue.io_modes |= VB2_DMABUF;
> +		queue->queue.ops = &uvc_queue_qops;

Could you please add a break here ?

> +	}

And I think a blank line here and before the switch would improve readability.

>  	ret = vb2_queue_init(&queue->queue);
>  	if (ret)
>  		return ret;
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index fb86d6a..034b79b 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1135,6 +1135,76 @@ static int uvc_video_encode_data(struct uvc_streaming
> *stream, }
> 
>  /* ------------------------------------------------------------------------
> + * Metadata
> + */
> +
> +/*
> + * Ideally we want to capture all payload headers for each frame. However
> their
> + * number is unknown and unlimited. Additionally to the payload headers we
> want
> + * to provide the user with USB Frame Numbers and system time values. We
> also
> + * choose to drop headers, containing only a standard header, which either
> + * contains no SCR value or a value, equal to the previous payload. The
> + * resulting buffer is composed of blocks, containing a 64-bit timestamp in
> + * nanoseconds, a 16-bit USB Frame Number, and a copy of the payload
> header.

To underline the rationale behind the decision to drop some headers, I would 
phrase this as

/*
 * Additionally to the payload headers we also want to provide the user with
 * USB Frame Numbers and system time values. The resulting buffer is thus
 * composed of blocks, containing a 64-bit timestamp in  nanoseconds, a 16-bit
 * USB Frame Number, and a copy of the payload header.
 *
 * Ideally we want to capture all payload headers for each frame. However,
 * their number is unknown and unbound. We thus drop headers that contain no
 * vendor data and that either contain no SCR value or an SCR value identical
 * to the previous header.
 */

> + */
> +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> +			struct uvc_buffer *buf, struct uvc_buffer *meta_buf,
> +			u8 *mem, unsigned int length)

The buf parameter is unused, you can remove it. mem isn't modified, I would 
make it const.

> +{
> +	struct uvc_meta_buf *meta;
> +	size_t len_std = 2;
> +	bool has_pts, has_scr;
> +	unsigned long flags;
> +	struct timespec ts;
> +	u8 *scr;

And scr should be const too.

> +
> +	if (!meta_buf || length == 2 ||
> +	    meta_buf->length - meta_buf->bytesused <
> +	    length + sizeof(meta->ns) + sizeof(meta->sof))
> +		return;

If the metadata buffer overflows should we also set the error bit like we do 
for video buffers ? I have mixed feelings about this, I'd appreciate your 
input.

> +	has_pts = mem[1] & UVC_STREAM_PTS;
> +	has_scr = mem[1] & UVC_STREAM_SCR;
> +
> +	if (has_pts) {
> +		len_std += 4;
> +		scr = mem + 6;
> +	} else {
> +		scr = mem + 2;
> +	}
> +
> +	if (has_scr)
> +		len_std += 6;
> +
> +	if (stream->cur_meta_format == V4L2_META_FMT_UVC)
> +		length = len_std;
> +
> +	if (length == len_std && (!has_scr ||
> +				  !memcmp(scr, stream->clock.last_scr, 6)))
> +		return;
> +
> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
> +	local_irq_save(flags);
> +	uvc_video_get_ts(&ts);

FYI, Arnd has posted https://patchwork.kernel.org/patch/10076887/. If the 
patch gets merged first I can help with the rebasing.

> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> +	local_irq_restore(flags);
> +	meta->ns = timespec_to_ns(&ts);

The meta pointer can be unaligned as the structure is packed and its size 
isn't a multiple of the size of the largest field (and it can contain an 
unspecified amount of vendor data anyway). You thus can't access it directly 
on all architectures, you will need to use the put_unaligned macro. As I 
haven't checked whether all architectures can handle unaligned accesses 
without generating a trap, I would store the USB frame number in a local 
variable and use the put_unaligned macro output of the IRQ disabled section 
(feel free to show me that I'm unnecessarily cautious :-)).

> +	if (has_scr)
> +		memcpy(stream->clock.last_scr, scr, 6);
> +
> +	memcpy(&meta->length, mem, length);
> +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> +
> +	uvc_trace(UVC_TRACE_FRAME,
> +		  "%s(): t-sys %lu.%09lus, SOF %u, len %u, flags 0x%x, PTS %u, STC %u
> frame SOF %u\n",
> +		  __func__, ts.tv_sec, ts.tv_nsec, meta->sof,
> +		  meta->length, meta->flags, has_pts ? *(u32 *)meta->buf : 0,
> +		  has_scr ? *(u32 *)scr : 0,
> +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> +}
> +
> +/* ------------------------------------------------------------------------
>   * URB handling
>   */
> 
> @@ -1152,8 +1222,28 @@ static void uvc_video_validate_buffer(const struct
> uvc_streaming *stream,
>  /*
>   * Completion handler for video URBs.
>   */
> +
> +static struct uvc_buffer *uvc_video_step_buffers(struct uvc_streaming
> *stream,
> +		struct uvc_buffer *video_buf, struct uvc_buffer **meta_buf)

I find the name uvc_video_step_buffers() a bit confusing and would have gone 
for uvc_video_next_buffers(), but that might just be because I'm not used to 
the new name :-)

I still dislike the fact that the video and metadata buffers are handled 
differently :-/ Do you really think it would be a bad idea to pass both as 
struct uvc_buffer ** ?

> +{
> +	if (*meta_buf) {
> +		struct vb2_v4l2_buffer *vb2_meta = &(*meta_buf)->buf;
> +		const struct vb2_v4l2_buffer *vb2_video = &video_buf->buf;
> +
> +		vb2_meta->sequence = vb2_video->sequence;
> +		vb2_meta->field = vb2_video->field;
> +		vb2_meta->vb2_buf.timestamp = vb2_video->vb2_buf.timestamp;
> +
> +		(*meta_buf)->state = UVC_BUF_STATE_READY;
> +		(*meta_buf)->error = video_buf->error;

If we set the error bit when the metadata buffer overflows (see above) then we 
shouldn't overwrite it unconditionally here.

> +		*meta_buf = uvc_queue_next_buffer(&stream->meta.queue,
> +						  *meta_buf);
> +	}
> +	return uvc_queue_next_buffer(&stream->queue, video_buf);
> +}

[snip]

> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index 34c7ee6..a657bc1 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -185,6 +185,7 @@
>  #define UVC_QUIRK_RESTRICT_FRAME_RATE	0x00000200
>  #define UVC_QUIRK_RESTORE_CTRLS_ON_INIT	0x00000400
>  #define UVC_QUIRK_FORCE_Y8		0x00000800
> +#define UVC_DEV_FLAG_METADATA_NODE	0x80000000
> 
>  /* Format flags */
>  #define UVC_FMT_FLAG_COMPRESSED		0x00000001
> @@ -473,6 +474,13 @@ struct uvc_stats_stream {
>  	unsigned int max_sof;		/* Maximum STC.SOF value */
>  };
> 
> +struct uvc_meta_device {
> +	struct video_device vdev;
> +	struct uvc_video_queue queue;
> +};
> +
> +#define UVC_METATADA_BUF_SIZE 1024
> +
>  struct uvc_streaming {
>  	struct list_head list;
>  	struct uvc_device *dev;
> @@ -504,7 +512,10 @@ struct uvc_streaming {
>  	unsigned int frozen : 1;
>  	struct uvc_video_queue queue;
>  	void (*decode) (struct urb *urb, struct uvc_streaming *video,
> -			struct uvc_buffer *buf);
> +			struct uvc_buffer *buf, struct uvc_buffer *meta_buf);
> +
> +	struct uvc_meta_device meta;
> +	__u32 cur_meta_format;

Nitpicking, as the uvc_meta_device structure isn't really needed as a separate 
structure, I'd write this

	struct {
		struct video_device vdev;
		struct uvc_video_queue queue;
		__u32 format;
	} meta;

> 
>  	/* Context data used by the bulk completion handler. */
>  	struct {

[snip]

> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c index 7961499..fba8021 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1246,6 +1246,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_TCH_FMT_TU08:		descr = "8-bit unsigned touch data"; break; 
case
> V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break; case
> V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break; +	case
> V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
> 
>  	default:
>  		/* Compressed formats */

I think this belongs to patch 1/3. I'll move it there.

[snip]

For your convenience I've rebased your patch series on top of the two patches 
I mentioned and added another patch on top that contains fixes for all the 
small issues mentioned above. The result is available at

	git://linuxtv.org/pinchartl/media.git uvc/metadata

There are just a handful of issues or questions I haven't addressed, if we 
handle them I think we'll be good to go.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Tuesday, 5 December 2017 02:24:30 EET Laurent Pinchart wrote:
> On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:

[snip]

> > +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> > +			struct uvc_buffer *buf, struct uvc_buffer *meta_buf,
> > +			u8 *mem, unsigned int length)
> 
> The buf parameter is unused, you can remove it. mem isn't modified, I would
> make it const.
> 
> > +{
> > +	struct uvc_meta_buf *meta;
> > +	size_t len_std = 2;
> > +	bool has_pts, has_scr;
> > +	unsigned long flags;
> > +	struct timespec ts;
> > +	u8 *scr;
> 
> And scr should be const too.
> 
> > +
> > +	if (!meta_buf || length == 2 ||
> > +	    meta_buf->length - meta_buf->bytesused <
> > +	    length + sizeof(meta->ns) + sizeof(meta->sof))
> > +		return;
> 
> If the metadata buffer overflows should we also set the error bit like we do
> for video buffers ? I have mixed feelings about this, I'd appreciate your
> input.
> 
> > +	has_pts = mem[1] & UVC_STREAM_PTS;
> > +	has_scr = mem[1] & UVC_STREAM_SCR;
> > +
> > +	if (has_pts) {
> > +		len_std += 4;
> > +		scr = mem + 6;
> > +	} else {
> > +		scr = mem + 2;
> > +	}
> > +
> > +	if (has_scr)
> > +		len_std += 6;
> > +
> > +	if (stream->cur_meta_format == V4L2_META_FMT_UVC)
> > +		length = len_std;
> > +
> > +	if (length == len_std && (!has_scr ||
> > +				  !memcmp(scr, stream->clock.last_scr, 6)))
> > +		return;
> > +
> > +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> > meta_buf->bytesused); +	local_irq_save(flags);
> > +	uvc_video_get_ts(&ts);
> 
> FYI, Arnd has posted https://patchwork.kernel.org/patch/10076887/. If the
> patch gets merged first I can help with the rebasing.

I've reviewed and merged Arnd patches in my tree, and...

> > +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > +	local_irq_restore(flags);
> > +	meta->ns = timespec_to_ns(&ts);
> 
> The meta pointer can be unaligned as the structure is packed and its size
> isn't a multiple of the size of the largest field (and it can contain an
> unspecified amount of vendor data anyway). You thus can't access it directly
> on all architectures, you will need to use the put_unaligned macro. As I
> haven't checked whether all architectures can handle unaligned accesses
> without generating a trap, I would store the USB frame number in a local
> variable and use the put_unaligned macro output of the IRQ disabled section
> (feel free to show me that I'm unnecessarily cautious :-)).
> 
> > +	if (has_scr)
> > +		memcpy(stream->clock.last_scr, scr, 6);
> > +
> > +	memcpy(&meta->length, mem, length);
> > +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> > +
> > +	uvc_trace(UVC_TRACE_FRAME,
> > +		  "%s(): t-sys %lu.%09lus, SOF %u, len %u, flags 0x%x, PTS %u, STC 
%u
> > frame SOF %u\n",
> > +		  __func__, ts.tv_sec, ts.tv_nsec, meta->sof,
> > +		  meta->length, meta->flags, has_pts ? *(u32 *)meta->buf : 0,
> > +		  has_scr ? *(u32 *)scr : 0,
> > +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> > +}

[snip]

> For your convenience I've rebased your patch series on top of the two
> patches I mentioned and added another patch on top that contains fixes for
> all the small issues mentioned above. The result is available at
> 
> 	git://linuxtv.org/pinchartl/media.git uvc/metadata
> 
> There are just a handful of issues or questions I haven't addressed, if we
> handle them I think we'll be good to go.

... updated the above branch with a rebased version of the series.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Guennadi Liakhovetski]

Hi Laurent,

On Tue, 5 Dec 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> On Tuesday, 5 December 2017 02:24:30 EET Laurent Pinchart wrote:
> > On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> 
> [snip]
> 
> > > +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> > > +			struct uvc_buffer *buf, struct uvc_buffer *meta_buf,
> > > +			u8 *mem, unsigned int length)
> > 
> > The buf parameter is unused, you can remove it. mem isn't modified, I would
> > make it const.
> > 
> > > +{
> > > +	struct uvc_meta_buf *meta;
> > > +	size_t len_std = 2;
> > > +	bool has_pts, has_scr;
> > > +	unsigned long flags;
> > > +	struct timespec ts;
> > > +	u8 *scr;
> > 
> > And scr should be const too.
> > 
> > > +
> > > +	if (!meta_buf || length == 2 ||
> > > +	    meta_buf->length - meta_buf->bytesused <
> > > +	    length + sizeof(meta->ns) + sizeof(meta->sof))
> > > +		return;
> > 
> > If the metadata buffer overflows should we also set the error bit like we do
> > for video buffers ? I have mixed feelings about this, I'd appreciate your
> > input.
> > 
> > > +	has_pts = mem[1] & UVC_STREAM_PTS;
> > > +	has_scr = mem[1] & UVC_STREAM_SCR;
> > > +
> > > +	if (has_pts) {
> > > +		len_std += 4;
> > > +		scr = mem + 6;
> > > +	} else {
> > > +		scr = mem + 2;
> > > +	}
> > > +
> > > +	if (has_scr)
> > > +		len_std += 6;
> > > +
> > > +	if (stream->cur_meta_format == V4L2_META_FMT_UVC)
> > > +		length = len_std;
> > > +
> > > +	if (length == len_std && (!has_scr ||
> > > +				  !memcmp(scr, stream->clock.last_scr, 6)))
> > > +		return;
> > > +
> > > +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> > > meta_buf->bytesused); +	local_irq_save(flags);
> > > +	uvc_video_get_ts(&ts);
> > 
> > FYI, Arnd has posted https://patchwork.kernel.org/patch/10076887/. If the
> > patch gets merged first I can help with the rebasing.
> 
> I've reviewed and merged Arnd patches in my tree, and...
> 
> > > +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > > +	local_irq_restore(flags);
> > > +	meta->ns = timespec_to_ns(&ts);
> > 
> > The meta pointer can be unaligned as the structure is packed and its size
> > isn't a multiple of the size of the largest field (and it can contain an
> > unspecified amount of vendor data anyway). You thus can't access it directly
> > on all architectures, you will need to use the put_unaligned macro. As I
> > haven't checked whether all architectures can handle unaligned accesses
> > without generating a trap, I would store the USB frame number in a local
> > variable and use the put_unaligned macro output of the IRQ disabled section
> > (feel free to show me that I'm unnecessarily cautious :-)).
> > 
> > > +	if (has_scr)
> > > +		memcpy(stream->clock.last_scr, scr, 6);
> > > +
> > > +	memcpy(&meta->length, mem, length);
> > > +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> > > +
> > > +	uvc_trace(UVC_TRACE_FRAME,
> > > +		  "%s(): t-sys %lu.%09lus, SOF %u, len %u, flags 0x%x, PTS %u, STC 
> %u
> > > frame SOF %u\n",
> > > +		  __func__, ts.tv_sec, ts.tv_nsec, meta->sof,
> > > +		  meta->length, meta->flags, has_pts ? *(u32 *)meta->buf : 0,
> > > +		  has_scr ? *(u32 *)scr : 0,
> > > +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> > > +}
> 
> [snip]
> 
> > For your convenience I've rebased your patch series on top of the two
> > patches I mentioned and added another patch on top that contains fixes for
> > all the small issues mentioned above. The result is available at

In your rebased version you've also dropped the hunk for 
drivers/media/v4l2-core/v4l2-ioctl.c adding a description for the new 
V4L2_META_FMT_UVC format - is that on purpose?

Thanks
Guennadi

> > 
> > 	git://linuxtv.org/pinchartl/media.git uvc/metadata
> > 
> > There are just a handful of issues or questions I haven't addressed, if we
> > handle them I think we'll be good to go.
> 
> ... updated the above branch with a rebased version of the series.
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Tuesday, 5 December 2017 10:06:02 EET Guennadi Liakhovetski wrote:
> On Tue, 5 Dec 2017, Laurent Pinchart wrote:
> > On Tuesday, 5 December 2017 02:24:30 EET Laurent Pinchart wrote:
> >> On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> > [snip]
> > 
> >>> +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> >>> +			struct uvc_buffer *buf, struct uvc_buffer *meta_buf,
> >>> +			u8 *mem, unsigned int length)
> >> 
> >> The buf parameter is unused, you can remove it. mem isn't modified, I
> >> would make it const.
> >> 
> >>> +{
> >>> +	struct uvc_meta_buf *meta;
> >>> +	size_t len_std = 2;
> >>> +	bool has_pts, has_scr;
> >>> +	unsigned long flags;
> >>> +	struct timespec ts;
> >>> +	u8 *scr;
> >> 
> >> And scr should be const too.
> >> 
> >>> +
> >>> +	if (!meta_buf || length == 2 ||
> >>> +	    meta_buf->length - meta_buf->bytesused <
> >>> +	    length + sizeof(meta->ns) + sizeof(meta->sof))
> >>> +		return;
> >> 
> >> If the metadata buffer overflows should we also set the error bit like
> >> we do for video buffers ? I have mixed feelings about this, I'd
> >> appreciate your input.
> >> 
> >>> +	has_pts = mem[1] & UVC_STREAM_PTS;
> >>> +	has_scr = mem[1] & UVC_STREAM_SCR;
> >>> +
> >>> +	if (has_pts) {
> >>> +		len_std += 4;
> >>> +		scr = mem + 6;
> >>> +	} else {
> >>> +		scr = mem + 2;
> >>> +	}
> >>> +
> >>> +	if (has_scr)
> >>> +		len_std += 6;
> >>> +
> >>> +	if (stream->cur_meta_format == V4L2_META_FMT_UVC)
> >>> +		length = len_std;
> >>> +
> >>> +	if (length == len_std && (!has_scr ||
> >>> +				  !memcmp(scr, stream->clock.last_scr, 6)))
> >>> +		return;
> >>> +
> >>> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> >>> meta_buf->bytesused);
> >>> +	local_irq_save(flags);
> >>> +	uvc_video_get_ts(&ts);
> >> 
> >> FYI, Arnd has posted https://patchwork.kernel.org/patch/10076887/. If
> >> the patch gets merged first I can help with the rebasing.
> > 
> > I've reviewed and merged Arnd patches in my tree, and...
> > 
> >>> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> >>> +	local_irq_restore(flags);
> >>> +	meta->ns = timespec_to_ns(&ts);
> >> 
> >> The meta pointer can be unaligned as the structure is packed and its
> >> size
> >> isn't a multiple of the size of the largest field (and it can contain an
> >> unspecified amount of vendor data anyway). You thus can't access it
> >> directly on all architectures, you will need to use the put_unaligned
> >> macro. As I haven't checked whether all architectures can handle
> >> unaligned accesses without generating a trap, I would store the USB
> >> frame number in a local variable and use the put_unaligned macro output
> >> of the IRQ disabled section (feel free to show me that I'm
> >> unnecessarily cautious :-)).
> >> 
> >>> +	if (has_scr)
> >>> +		memcpy(stream->clock.last_scr, scr, 6);
> >>> +
> >>> +	memcpy(&meta->length, mem, length);
> >>> +	meta_buf->bytesused += length + sizeof(meta->ns) +
> >>> sizeof(meta->sof);
> >>> +
> >>> +	uvc_trace(UVC_TRACE_FRAME,
> >>> +		  "%s(): t-sys %lu.%09lus, SOF %u, len %u, flags 0x%x, PTS %u,
> >>> STC %u frame SOF %u\n",
> >>> +		  __func__, ts.tv_sec, ts.tv_nsec, meta->sof,
> >>> +		  meta->length, meta->flags, has_pts ? *(u32 *)meta->buf : 0,
> >>> +		  has_scr ? *(u32 *)scr : 0,
> >>> +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> >>> +}
> > 
> > [snip]
> > 
> >> For your convenience I've rebased your patch series on top of the two
> >> patches I mentioned and added another patch on top that contains fixes
> >> for all the small issues mentioned above. The result is available at
> 
> In your rebased version you've also dropped the hunk for
> drivers/media/v4l2-core/v4l2-ioctl.c adding a description for the new
> V4L2_META_FMT_UVC format - is that on purpose?

Unless I'm mistaken I've moved it to patch 1/3.

> >> 	git://linuxtv.org/pinchartl/media.git uvc/metadata
> >> 
> >> There are just a handful of issues or questions I haven't addressed, if
> >> we handle them I think we'll be good to go.
> > 
> > ... updated the above branch with a rebased version of the series.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Guennadi Liakhovetski]

Hi Laurent,

Thanks for a review.

On Tue, 5 Dec 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> Thank you for the patch. We're getting very close, I only have small comments, 
> please see below.
> 
> On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> > Some UVC video cameras contain metadata in their payload headers. This
> > patch extracts that data, adding more clock synchronisation information,
> > on both bulk and isochronous endpoints and makes it available to the
> > user space on a separate video node, using the V4L2_CAP_META_CAPTURE
> > capability and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. Even
> > though different cameras will have different metadata formats, we use
> > the same V4L2_META_FMT_UVC pixel format for all of them. Users have to
> > parse data, based on the specific camera model information. This
> > version of the patch only creates such metadata nodes for cameras,
> > specifying a UVC_QUIRK_METADATA_NODE quirk flag.
> 
> I don't think this is correct anymore, as we'll use different 4CCs for 
> different vendor metadata. How would you like to rephrase the commit message ?

Something like

"
Some UVC video cameras contain metadata in their payload headers. This 
patch extracts that data, adding more clock synchronisation information, 
on both bulk and isochronous endpoints and makes it available to the user 
space on a separate video node, using the V4L2_CAP_META_CAPTURE capability 
and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By default, only the 
V4L2_META_FMT_UVC pixel format is available from those nodes. However, 
cameras can be added to the device ID table to additionally specify their 
own metadata format, in which case that format will also become available 
from the metadata node.
"

> > Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > ---
> > 
> > v7: support up to two metadata formats per camera - the standard one and
> > an optional private one, if specified in device information
> > 
> >  drivers/media/usb/uvc/Makefile       |   2 +-
> >  drivers/media/usb/uvc/uvc_driver.c   |  15 +++
> >  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> >  drivers/media/usb/uvc/uvc_metadata.c | 204 ++++++++++++++++++++++++++++++++
> >  drivers/media/usb/uvc/uvc_queue.c    |  41 +++++--
> >  drivers/media/usb/uvc/uvc_video.c    | 127 ++++++++++++++++++++--
> >  drivers/media/usb/uvc/uvcvideo.h     |  19 +++-
> >  drivers/media/v4l2-core/v4l2-ioctl.c |   1 +
> >  include/uapi/linux/uvcvideo.h        |  26 +++++
> >  9 files changed, 416 insertions(+), 21 deletions(-)
> >  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> 
> [snip]
> 
> > diff --git a/drivers/media/usb/uvc/uvc_driver.c
> > b/drivers/media/usb/uvc/uvc_driver.c index cbf79b9..5f7ce97 100644
> > --- a/drivers/media/usb/uvc/uvc_driver.c
> > +++ b/drivers/media/usb/uvc/uvc_driver.c
> > @@ -1877,6 +1877,7 @@ static void uvc_unregister_video(struct uvc_device
> > *dev) continue;
> > 
> >  		video_unregister_device(&stream->vdev);
> > +		video_unregister_device(&stream->meta.vdev);
> > 
> >  		uvc_debugfs_cleanup_stream(stream);
> >  	}
> > @@ -1934,6 +1935,11 @@ static int uvc_register_video(struct uvc_device *dev,
> > return ret;
> >  	}
> > 
> > +	/* Register a metadata node, but ignore a possible failure, complete
> > +	 * registration of video nodes anyway.
> > +	 */
> > +	uvc_meta_register(stream);
> > +
> >  	if (stream->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> >  		stream->chain->caps |= V4L2_CAP_VIDEO_CAPTURE;
> >  	else
> > @@ -2003,6 +2009,7 @@ static int uvc_register_chains(struct uvc_device *dev)
> > 
> >  struct uvc_device_info {
> >  	u32	quirks;
> > +	u32	meta_format;
> >  };
> > 
> >  static int uvc_probe(struct usb_interface *intf,
> > @@ -2038,8 +2045,16 @@ static int uvc_probe(struct usb_interface *intf,
> >  	dev->udev = usb_get_dev(udev);
> >  	dev->intf = usb_get_intf(intf);
> >  	dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
> > +	if (uvc_quirks_param != -1 &&
> > +	    uvc_quirks_param & UVC_DEV_FLAG_METADATA_NODE) {
> > +		uvc_quirks_param &= ~UVC_DEV_FLAG_METADATA_NODE;
> > +		if (uvc_quirks_param == 0)
> > +			uvc_quirks_param = -1;
> > +	}
> 
> I think we can remove the UVC_DEV_FLAG_METADATA_NODE flag. We can use the 
> metadata format specified in the device information structure if present, and 
> default to the standard metadata format otherwise. A metadata video node will 
> always be registered, which I think is the right thing to do as exposing 
> timestamp information to userspace is useful.
> 
> Do you see any adverse effect of registering a metadata video node 
> unconditionally ?

No, apart from confusion on behalf of applications and users.

> >  	dev->quirks = (uvc_quirks_param == -1)
> >  		    ? quirks : uvc_quirks_param;
> > +	if (info)
> > +		dev->meta_format = info->meta_format;
> > 
> >  	if (udev->product != NULL)
> >  		strlcpy(dev->name, udev->product, sizeof dev->name);
> 
> [snip]
> 
> > diff --git a/drivers/media/usb/uvc/uvc_metadata.c
> > b/drivers/media/usb/uvc/uvc_metadata.c new file mode 100644
> > index 0000000..21eeee9
> > --- /dev/null
> > +++ b/drivers/media/usb/uvc/uvc_metadata.c
> 
> [snip]
> 
> > +/* ------------------------------------------------------------------------
> > + * videobuf2 Queue Operations
> > + */
> > +
> 
> You can remove this now that the section is empty :-)
> 
> > +/* ------------------------------------------------------------------------
> > + * V4L2 ioctls
> > + */
> > +
> > +static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
> > +				  struct v4l2_capability *cap)
> > +{
> > +	struct v4l2_fh *vfh = file->private_data;
> > +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> > +
> > +	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
> > +	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
> > +	usb_make_path(stream->dev->udev, cap->bus_info, sizeof(cap->bus_info));
> > +
> > +	return 0;
> > +}
> 
> Do you think we could reuse uvc_ioctl_querycap() as-is ?

AFAICS it still has

	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
			  | chain->caps;

in it, which doesn't suit the metadata node?

> > +
> > +static int uvc_meta_v4l2_get_format(struct file *file, void *fh,
> > +				    struct v4l2_format *format)
> > +{
> > +	struct v4l2_fh *vfh = file->private_data;
> > +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> > +	struct v4l2_meta_format *fmt = &format->fmt.meta;
> > +
> > +	if (format->type != vfh->vdev->queue->type)
> > +		return -EINVAL;
> > +
> > +	memset(fmt, 0, sizeof(*fmt));
> > +
> > +	fmt->dataformat = stream->cur_meta_format;
> > +	fmt->buffersize = UVC_METATADA_BUF_SIZE;
> 
> You need to take the stream->mutex lock here to protect against races with the 
> set format ioctl.

Well, strictly speaking you don't? The buffersize is constant and getting 
the current metadata format is an atomic read.

[snip]

> With the uvc_register_video_device() helper this function would become
> 
> int uvc_meta_register(struct uvc_streaming *stream)
> {
> 	struct uvc_device *dev = stream->dev;
> 	struct video_device *vdev = &stream->meta.vdev;
> 	struct uvc_video_queue *queue = &stream->meta.queue;
> 
> 	/*
> 	 * Register metadata device nodes only if the METADATA_NODE quirk is
> 	 * set.
> 	 */
> 	if (!(dev->quirks & UVC_DEV_FLAG_METADATA_NODE))
> 		return 0;
> 
> 	stream->cur_meta_format = V4L2_META_FMT_UVC;
> 
> 	/*
> 	 * The video interface queue uses manual locking and thus does not set
> 	 * the queue pointer. Set it manually here.
> 	 */
> 	vdev->queue = &queue->queue;
> 
> 	return uvc_register_video_device(dev, stream, vdev, queue,
> 					 V4L2_BUF_TYPE_META_CAPTURE,
> 					 &uvc_meta_fops, &uvc_meta_ioctl_ops);
> }
> 
> What do you think, is it worth it ?

Sure, looks good to me, thanks.

> > diff --git a/drivers/media/usb/uvc/uvc_queue.c
> > b/drivers/media/usb/uvc/uvc_queue.c index c8d78b2..b2998f5 100644
> > --- a/drivers/media/usb/uvc/uvc_queue.c
> > +++ b/drivers/media/usb/uvc/uvc_queue.c

[snip]

> > +	if (!meta_buf || length == 2 ||
> > +	    meta_buf->length - meta_buf->bytesused <
> > +	    length + sizeof(meta->ns) + sizeof(meta->sof))
> > +		return;
> 
> If the metadata buffer overflows should we also set the error bit like we do 
> for video buffers ? I have mixed feelings about this, I'd appreciate your 
> input.

I think it would be good to know if we ever run into such overruns. 
Setting an error flag is more likely to be noticed and reported than a 
printk()?

> > +	has_pts = mem[1] & UVC_STREAM_PTS;
> > +	has_scr = mem[1] & UVC_STREAM_SCR;
> > +
> > +	if (has_pts) {
> > +		len_std += 4;
> > +		scr = mem + 6;
> > +	} else {
> > +		scr = mem + 2;
> > +	}
> > +
> > +	if (has_scr)
> > +		len_std += 6;
> > +
> > +	if (stream->cur_meta_format == V4L2_META_FMT_UVC)
> > +		length = len_std;
> > +
> > +	if (length == len_std && (!has_scr ||
> > +				  !memcmp(scr, stream->clock.last_scr, 6)))
> > +		return;
> > +
> > +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
> > +	local_irq_save(flags);
> > +	uvc_video_get_ts(&ts);
> 
> FYI, Arnd has posted https://patchwork.kernel.org/patch/10076887/. If the 
> patch gets merged first I can help with the rebasing.
> 
> > +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > +	local_irq_restore(flags);
> > +	meta->ns = timespec_to_ns(&ts);
> 
> The meta pointer can be unaligned as the structure is packed and its size 
> isn't a multiple of the size of the largest field (and it can contain an 
> unspecified amount of vendor data anyway). You thus can't access it directly 
> on all architectures, you will need to use the put_unaligned macro. As I 
> haven't checked whether all architectures can handle unaligned accesses 
> without generating a trap, I would store the USB frame number in a local 
> variable and use the put_unaligned macro output of the IRQ disabled section 
> (feel free to show me that I'm unnecessarily cautious :-)).

I think you're right. But __put_unaligned_cpu64() is an inline function, 
not a macro, so, no local variable is needed?

> > +	if (has_scr)
> > +		memcpy(stream->clock.last_scr, scr, 6);
> > +
> > +	memcpy(&meta->length, mem, length);
> > +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> > +
> > +	uvc_trace(UVC_TRACE_FRAME,
> > +		  "%s(): t-sys %lu.%09lus, SOF %u, len %u, flags 0x%x, PTS %u, STC %u
> > frame SOF %u\n",
> > +		  __func__, ts.tv_sec, ts.tv_nsec, meta->sof,
> > +		  meta->length, meta->flags, has_pts ? *(u32 *)meta->buf : 0,
> > +		  has_scr ? *(u32 *)scr : 0,
> > +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> > +}
> > +
> > +/* ------------------------------------------------------------------------
> >   * URB handling
> >   */
> > 
> > @@ -1152,8 +1222,28 @@ static void uvc_video_validate_buffer(const struct
> > uvc_streaming *stream,
> >  /*
> >   * Completion handler for video URBs.
> >   */
> > +
> > +static struct uvc_buffer *uvc_video_step_buffers(struct uvc_streaming
> > *stream,
> > +		struct uvc_buffer *video_buf, struct uvc_buffer **meta_buf)
> 
> I find the name uvc_video_step_buffers() a bit confusing and would have gone 
> for uvc_video_next_buffers(), but that might just be because I'm not used to 
> the new name :-)

I don't like step_buffers either, but next_buffers doesn't sound a lot 
better to me. Anyway, if you prefer that, let's rename.

> I still dislike the fact that the video and metadata buffers are handled 
> differently :-/ Do you really think it would be a bad idea to pass both as 
> struct uvc_buffer ** ?

This will result in one superfluous assignment per bulk decode call :-) 
But ok, let's do that, will look better indeed. We couls then also change 
the uvc_queue_next_buffer() function similarly to have an input-output 
parameter instead of returning the next buffer, but that isn't related, 
would be a separate patch.

> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c index 7961499..fba8021 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1246,6 +1246,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > case V4L2_TCH_FMT_TU08:		descr = "8-bit unsigned touch data"; break; 
> case
> > V4L2_META_FMT_VSP1_HGO:	descr = "R-Car VSP1 1-D Histogram"; break; case
> > V4L2_META_FMT_VSP1_HGT:	descr = "R-Car VSP1 2-D Histogram"; break; +	case
> > V4L2_META_FMT_UVC:		descr = "UVC payload header metadata"; break;
> > 
> >  	default:
> >  		/* Compressed formats */
> 
> I think this belongs to patch 1/3. I'll move it there.
> 
> [snip]
> 
> For your convenience I've rebased your patch series on top of the two patches 
> I mentioned and added another patch on top that contains fixes for all the 
> small issues mentioned above. The result is available at
> 
> 	git://linuxtv.org/pinchartl/media.git uvc/metadata
> 
> There are just a handful of issues or questions I haven't addressed, if we 
> handle them I think we'll be good to go.

Thanks for rebasing. I've addressed the remaining issues, and once I get 
your answers to remaining questions above, I'll send an updated version.

Thanks
Guennadi

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Tuesday, 5 December 2017 12:56:53 EET Guennadi Liakhovetski wrote:
> On Tue, 5 Dec 2017, Laurent Pinchart wrote:
> > On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> >> Some UVC video cameras contain metadata in their payload headers. This
> >> patch extracts that data, adding more clock synchronisation information,
> >> on both bulk and isochronous endpoints and makes it available to the
> >> user space on a separate video node, using the V4L2_CAP_META_CAPTURE
> >> capability and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. Even
> >> though different cameras will have different metadata formats, we use
> >> the same V4L2_META_FMT_UVC pixel format for all of them. Users have to
> >> parse data, based on the specific camera model information. This
> >> version of the patch only creates such metadata nodes for cameras,
> >> specifying a UVC_QUIRK_METADATA_NODE quirk flag.
> > 
> > I don't think this is correct anymore, as we'll use different 4CCs for
> > different vendor metadata. How would you like to rephrase the commit
> > message ?
> 
> Something like
> 
> "
> Some UVC video cameras contain metadata in their payload headers. This
> patch extracts that data, adding more clock synchronisation information,
> on both bulk and isochronous endpoints and makes it available to the user
> space on a separate video node, using the V4L2_CAP_META_CAPTURE capability
> and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By default, only the
> V4L2_META_FMT_UVC pixel format is available from those nodes. However,
> cameras can be added to the device ID table to additionally specify their
> own metadata format, in which case that format will also become available
> from the metadata node.
> "

Sounds good to me.

> >> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> >> ---
> >> 
> >> v7: support up to two metadata formats per camera - the standard one and
> >> an optional private one, if specified in device information
> >> 
> >>  drivers/media/usb/uvc/Makefile       |   2 +-
> >>  drivers/media/usb/uvc/uvc_driver.c   |  15 +++
> >>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> >>  drivers/media/usb/uvc/uvc_metadata.c | 204 +++++++++++++++++++++++++++++
> >>  drivers/media/usb/uvc/uvc_queue.c    |  41 +++++--
> >>  drivers/media/usb/uvc/uvc_video.c    | 127 ++++++++++++++++++++--
> >>  drivers/media/usb/uvc/uvcvideo.h     |  19 +++-
> >>  drivers/media/v4l2-core/v4l2-ioctl.c |   1 +
> >>  include/uapi/linux/uvcvideo.h        |  26 +++++
> >>  9 files changed, 416 insertions(+), 21 deletions(-)
> >>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
[snip]

> >> diff --git a/drivers/media/usb/uvc/uvc_metadata.c
> >> b/drivers/media/usb/uvc/uvc_metadata.c new file mode 100644
> >> index 0000000..21eeee9
> >> --- /dev/null
> >> +++ b/drivers/media/usb/uvc/uvc_metadata.c

[snip]

> >> +static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
> >> +				  struct v4l2_capability *cap)
> >> +{
> >> +	struct v4l2_fh *vfh = file->private_data;
> >> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> >> +
> >> +	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
> >> +	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
> >> +	usb_make_path(stream->dev->udev, cap->bus_info,
> >> sizeof(cap->bus_info));
> >> +
> >> +	return 0;
> >> +}
> > 
> > Do you think we could reuse uvc_ioctl_querycap() as-is ?
> 
> AFAICS it still has
> 
> 	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
> 			  | chain->caps;
> 
> in it, which doesn't suit the metadata node?

I'd say this is debatable, isn't the capabilities field supposed to include 
all capabilities from all video nodes for the device ? chain->caps would need 
to include metadata capability in that case.

Code reuse is not a big deal as the function is small, but getting the 
capabilities value right is important regardless.

> >> +
> >> +static int uvc_meta_v4l2_get_format(struct file *file, void *fh,
> >> +				    struct v4l2_format *format)
> >> +{
> >> +	struct v4l2_fh *vfh = file->private_data;
> >> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> >> +	struct v4l2_meta_format *fmt = &format->fmt.meta;
> >> +
> >> +	if (format->type != vfh->vdev->queue->type)
> >> +		return -EINVAL;
> >> +
> >> +	memset(fmt, 0, sizeof(*fmt));
> >> +
> >> +	fmt->dataformat = stream->cur_meta_format;
> >> +	fmt->buffersize = UVC_METATADA_BUF_SIZE;
> > 
> > You need to take the stream->mutex lock here to protect against races with
> > the set format ioctl.
> 
> Well, strictly speaking you don't? The buffersize is constant and getting
> the current metadata format is an atomic read.

I was concerned by the race condition due to lack of memory barriers, but if 
userspace issues concurrent G_FMT and S_FMT calls the order isn't guaranteed 
anyway, so you're right about that.

[snip]

> > > diff --git a/drivers/media/usb/uvc/uvc_queue.c
> > > b/drivers/media/usb/uvc/uvc_queue.c index c8d78b2..b2998f5 100644
> > > --- a/drivers/media/usb/uvc/uvc_queue.c
> > > +++ b/drivers/media/usb/uvc/uvc_queue.c
> 
> [snip]
> 
> >> +	if (!meta_buf || length == 2 ||
> >> +	    meta_buf->length - meta_buf->bytesused <
> >> +	    length + sizeof(meta->ns) + sizeof(meta->sof))
> >> +		return;
> > 
> > If the metadata buffer overflows should we also set the error bit like we
> > do for video buffers ? I have mixed feelings about this, I'd appreciate
> > your input.
> 
> I think it would be good to know if we ever run into such overruns.
> Setting an error flag is more likely to be noticed and reported than a
> printk()?

I believe so, yes.

> >> +	has_pts = mem[1] & UVC_STREAM_PTS;
> >> +	has_scr = mem[1] & UVC_STREAM_SCR;
> >> +
> >> +	if (has_pts) {
> >> +		len_std += 4;
> >> +		scr = mem + 6;
> >> +	} else {
> >> +		scr = mem + 2;
> >> +	}
> >> +
> >> +	if (has_scr)
> >> +		len_std += 6;
> >> +
> >> +	if (stream->cur_meta_format == V4L2_META_FMT_UVC)
> >> +		length = len_std;
> >> +
> >> +	if (length == len_std && (!has_scr ||
> >> +				  !memcmp(scr, stream->clock.last_scr, 6)))
> >> +		return;
> >> +
> >> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> >> meta_buf->bytesused);
> >> +	local_irq_save(flags);
> >> +	uvc_video_get_ts(&ts);
> > 
> > FYI, Arnd has posted https://patchwork.kernel.org/patch/10076887/. If the
> > patch gets merged first I can help with the rebasing.
> > 
> >> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> >> +	local_irq_restore(flags);
> >> +	meta->ns = timespec_to_ns(&ts);
> > 
> > The meta pointer can be unaligned as the structure is packed and its size
> > isn't a multiple of the size of the largest field (and it can contain an
> > unspecified amount of vendor data anyway). You thus can't access it
> > directly on all architectures, you will need to use the put_unaligned
> > macro. As I haven't checked whether all architectures can handle
> > unaligned accesses without generating a trap, I would store the USB frame
> > number in a local variable and use the put_unaligned macro output of the
> > IRQ disabled section (feel free to show me that I'm unnecessarily
> > cautious :-)).
> 
> I think you're right. But __put_unaligned_cpu64() is an inline function,
> not a macro, so, no local variable is needed?

Even if it's a function I'm not 100% confident that it can work well with 
interrupts disabled on all architectures, but I'm probably overcautious.

> >> +	if (has_scr)
> >> +		memcpy(stream->clock.last_scr, scr, 6);
> >> +
> >> +	memcpy(&meta->length, mem, length);
> >> +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> >> +
> >> +	uvc_trace(UVC_TRACE_FRAME,
> >> +		  "%s(): t-sys %lu.%09lus, SOF %u, len %u, flags 0x%x, PTS %u, STC
> >> %u frame SOF %u\n",
> >> +		  __func__, ts.tv_sec, ts.tv_nsec, meta->sof,
> >> +		  meta->length, meta->flags, has_pts ? *(u32 *)meta->buf : 0,
> >> +		  has_scr ? *(u32 *)scr : 0,
> >> +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> >> +}

[snip]

> > For your convenience I've rebased your patch series on top of the two
> > patches I mentioned and added another patch on top that contains fixes
> > for all the small issues mentioned above. The result is available at
> > 
> > 	git://linuxtv.org/pinchartl/media.git uvc/metadata
> > 
> > There are just a handful of issues or questions I haven't addressed, if we
> > handle them I think we'll be good to go.
> 
> Thanks for rebasing. I've addressed the remaining issues, and once I get
> your answers to remaining questions above, I'll send an updated version.

Done :-)

Thank you.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Guennadi Liakhovetski]

On Tue, 5 Dec 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> On Tuesday, 5 December 2017 12:56:53 EET Guennadi Liakhovetski wrote:
> > On Tue, 5 Dec 2017, Laurent Pinchart wrote:
> > > On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> > >> Some UVC video cameras contain metadata in their payload headers. This
> > >> patch extracts that data, adding more clock synchronisation information,
> > >> on both bulk and isochronous endpoints and makes it available to the
> > >> user space on a separate video node, using the V4L2_CAP_META_CAPTURE
> > >> capability and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. Even
> > >> though different cameras will have different metadata formats, we use
> > >> the same V4L2_META_FMT_UVC pixel format for all of them. Users have to
> > >> parse data, based on the specific camera model information. This
> > >> version of the patch only creates such metadata nodes for cameras,
> > >> specifying a UVC_QUIRK_METADATA_NODE quirk flag.
> > > 
> > > I don't think this is correct anymore, as we'll use different 4CCs for
> > > different vendor metadata. How would you like to rephrase the commit
> > > message ?
> > 
> > Something like
> > 
> > "
> > Some UVC video cameras contain metadata in their payload headers. This
> > patch extracts that data, adding more clock synchronisation information,
> > on both bulk and isochronous endpoints and makes it available to the user
> > space on a separate video node, using the V4L2_CAP_META_CAPTURE capability
> > and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By default, only the
> > V4L2_META_FMT_UVC pixel format is available from those nodes. However,
> > cameras can be added to the device ID table to additionally specify their
> > own metadata format, in which case that format will also become available
> > from the metadata node.
> > "
> 
> Sounds good to me.
> 
> > >> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > >> ---
> > >> 
> > >> v7: support up to two metadata formats per camera - the standard one and
> > >> an optional private one, if specified in device information
> > >> 
> > >>  drivers/media/usb/uvc/Makefile       |   2 +-
> > >>  drivers/media/usb/uvc/uvc_driver.c   |  15 +++
> > >>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> > >>  drivers/media/usb/uvc/uvc_metadata.c | 204 +++++++++++++++++++++++++++++
> > >>  drivers/media/usb/uvc/uvc_queue.c    |  41 +++++--
> > >>  drivers/media/usb/uvc/uvc_video.c    | 127 ++++++++++++++++++++--
> > >>  drivers/media/usb/uvc/uvcvideo.h     |  19 +++-
> > >>  drivers/media/v4l2-core/v4l2-ioctl.c |   1 +
> > >>  include/uapi/linux/uvcvideo.h        |  26 +++++
> > >>  9 files changed, 416 insertions(+), 21 deletions(-)
> > >>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> [snip]
> 
> > >> diff --git a/drivers/media/usb/uvc/uvc_metadata.c
> > >> b/drivers/media/usb/uvc/uvc_metadata.c new file mode 100644
> > >> index 0000000..21eeee9
> > >> --- /dev/null
> > >> +++ b/drivers/media/usb/uvc/uvc_metadata.c
> 
> [snip]
> 
> > >> +static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
> > >> +				  struct v4l2_capability *cap)
> > >> +{
> > >> +	struct v4l2_fh *vfh = file->private_data;
> > >> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> > >> +
> > >> +	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
> > >> +	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
> > >> +	usb_make_path(stream->dev->udev, cap->bus_info,
> > >> sizeof(cap->bus_info));
> > >> +
> > >> +	return 0;
> > >> +}
> > > 
> > > Do you think we could reuse uvc_ioctl_querycap() as-is ?
> > 
> > AFAICS it still has
> > 
> > 	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
> > 			  | chain->caps;
> > 
> > in it, which doesn't suit the metadata node?
> 
> I'd say this is debatable, isn't the capabilities field supposed to include 
> all capabilities from all video nodes for the device ? chain->caps would need 
> to include metadata capability in that case.
> 
> Code reuse is not a big deal as the function is small, but getting the 
> capabilities value right is important regardless.

Hm, but that's how applications would naturally work - open a node, query 
its capabilities and handle it accordingly. What good would be having 
equal capabilities on all nodes? Why do you think that should be the case?

> > >> +
> > >> +static int uvc_meta_v4l2_get_format(struct file *file, void *fh,
> > >> +				    struct v4l2_format *format)
> > >> +{
> > >> +	struct v4l2_fh *vfh = file->private_data;
> > >> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> > >> +	struct v4l2_meta_format *fmt = &format->fmt.meta;
> > >> +
> > >> +	if (format->type != vfh->vdev->queue->type)
> > >> +		return -EINVAL;
> > >> +
> > >> +	memset(fmt, 0, sizeof(*fmt));
> > >> +
> > >> +	fmt->dataformat = stream->cur_meta_format;
> > >> +	fmt->buffersize = UVC_METATADA_BUF_SIZE;
> > > 
> > > You need to take the stream->mutex lock here to protect against races with
> > > the set format ioctl.
> > 
> > Well, strictly speaking you don't? The buffersize is constant and getting
> > the current metadata format is an atomic read.
> 
> I was concerned by the race condition due to lack of memory barriers, but if 
> userspace issues concurrent G_FMT and S_FMT calls the order isn't guaranteed 
> anyway, so you're right about that.
> 
> [snip]
> 
> > > > diff --git a/drivers/media/usb/uvc/uvc_queue.c
> > > > b/drivers/media/usb/uvc/uvc_queue.c index c8d78b2..b2998f5 100644
> > > > --- a/drivers/media/usb/uvc/uvc_queue.c
> > > > +++ b/drivers/media/usb/uvc/uvc_queue.c
> > 
> > [snip]
> > 
> > >> +	if (!meta_buf || length == 2 ||
> > >> +	    meta_buf->length - meta_buf->bytesused <
> > >> +	    length + sizeof(meta->ns) + sizeof(meta->sof))
> > >> +		return;
> > > 
> > > If the metadata buffer overflows should we also set the error bit like we
> > > do for video buffers ? I have mixed feelings about this, I'd appreciate
> > > your input.
> > 
> > I think it would be good to know if we ever run into such overruns.
> > Setting an error flag is more likely to be noticed and reported than a
> > printk()?
> 
> I believe so, yes.

OK, so we set an error.

Thanks
Guennadi

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Tuesday, 5 December 2017 15:44:34 EET Guennadi Liakhovetski wrote:
> On Tue, 5 Dec 2017, Laurent Pinchart wrote:
> > On Tuesday, 5 December 2017 12:56:53 EET Guennadi Liakhovetski wrote:
> >> On Tue, 5 Dec 2017, Laurent Pinchart wrote:
> >>> On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> >>>> Some UVC video cameras contain metadata in their payload headers.
> >>>> This patch extracts that data, adding more clock synchronisation
> >>>> information, on both bulk and isochronous endpoints and makes it
> >>>> available to the user space on a separate video node, using the
> >>>> V4L2_CAP_META_CAPTURE capability and the V4L2_BUF_TYPE_META_CAPTURE
> >>>> buffer queue type. Even though different cameras will have different
> >>>> metadata formats, we use the same V4L2_META_FMT_UVC pixel format for
> >>>> all of them. Users have to parse data, based on the specific camera
> >>>> model information. This version of the patch only creates such metadata
> >>>> nodes for cameras, specifying a UVC_QUIRK_METADATA_NODE quirk flag.
> >>> 
> >>> I don't think this is correct anymore, as we'll use different 4CCs for
> >>> different vendor metadata. How would you like to rephrase the commit
> >>> message ?
> >> 
> >> Something like
> >> 
> >> "
> >> Some UVC video cameras contain metadata in their payload headers. This
> >> patch extracts that data, adding more clock synchronisation information,
> >> on both bulk and isochronous endpoints and makes it available to the
> >> user space on a separate video node, using the V4L2_CAP_META_CAPTURE
> >> capability and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By
> >> default, only the V4L2_META_FMT_UVC pixel format is available from those
> >> nodes. However, cameras can be added to the device ID table to
> >> additionally specify their own metadata format, in which case that
> >> format will also become available from the metadata node.
> >> "
> > 
> > Sounds good to me.
> > 
> >>>> Signed-off-by: Guennadi Liakhovetski
> >>>> <guennadi.liakhovetski@intel.com>
> >>>> ---
> >>>> 
> >>>> v7: support up to two metadata formats per camera - the standard one
> >>>> and an optional private one, if specified in device information
> >>>> 
> >>>>  drivers/media/usb/uvc/Makefile       |   2 +-
> >>>>  drivers/media/usb/uvc/uvc_driver.c   |  15 +++
> >>>>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> >>>>  drivers/media/usb/uvc/uvc_metadata.c | 204 +++++++++++++++++++++++++++
> >>>>  drivers/media/usb/uvc/uvc_queue.c    |  41 +++++--
> >>>>  drivers/media/usb/uvc/uvc_video.c    | 127 ++++++++++++++++++++--
> >>>>  drivers/media/usb/uvc/uvcvideo.h     |  19 +++-
> >>>>  drivers/media/v4l2-core/v4l2-ioctl.c |   1 +
> >>>>  include/uapi/linux/uvcvideo.h        |  26 +++++
> >>>>  9 files changed, 416 insertions(+), 21 deletions(-)
> >>>>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> > 
> > [snip]
> > 
> >>>> diff --git a/drivers/media/usb/uvc/uvc_metadata.c
> >>>> b/drivers/media/usb/uvc/uvc_metadata.c new file mode 100644
> >>>> index 0000000..21eeee9
> >>>> --- /dev/null
> >>>> +++ b/drivers/media/usb/uvc/uvc_metadata.c
> > 
> > [snip]
> > 
> >>>> +static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
> >>>> +				  struct v4l2_capability *cap)
> >>>> +{
> >>>> +	struct v4l2_fh *vfh = file->private_data;
> >>>> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> >>>> +
> >>>> +	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
> >>>> +	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
> >>>> +	usb_make_path(stream->dev->udev, cap->bus_info,
> >>>> sizeof(cap->bus_info));
> >>>> +
> >>>> +	return 0;
> >>>> +}
> >>> 
> >>> Do you think we could reuse uvc_ioctl_querycap() as-is ?
> >> 
> >> AFAICS it still has
> >> 
> >> 	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
> > 			  | chain->caps;
> >> 
> >> in it, which doesn't suit the metadata node?
> > 
> > I'd say this is debatable, isn't the capabilities field supposed to
> > include all capabilities from all video nodes for the device ? chain->caps
> > would need to include metadata capability in that case.
> > 
> > Code reuse is not a big deal as the function is small, but getting the
> > capabilities value right is important regardless.
> 
> Hm, but that's how applications would naturally work - open a node, query
> its capabilities and handle it accordingly. What good would be having
> equal capabilities on all nodes? Why do you think that should be the case?

Quoting https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/vidioc-querycap.html,

__u32 	capabilities

Available capabilities of the physical device as a whole, see Device 
Capabilities Flags. The same physical device can export multiple devices in /
dev (e.g. /dev/videoX, /dev/vbiY and /dev/radioZ). The capabilities field 
should contain a union of all capabilities available around the several V4L2 
devices exported to userspace. For all those devices the capabilities field 
returns the same set of capabilities. This allows applications to open just 
one of the devices (typically the video device) and discover whether video, 
vbi and/or radio are also supported.

__u32 	device_caps

Device capabilities of the opened device, see Device Capabilities Flags. 
Should contain the available capabilities of that specific device node. So, 
for example, device_caps of a radio device will only contain radio related 
capabilities and no video or vbi capabilities. This field is only set if the 
capabilities field contains the V4L2_CAP_DEVICE_CAPS capability. Only the 
capabilities field can have the V4L2_CAP_DEVICE_CAPS capability, device_caps 
will never set V4L2_CAP_DEVICE_CAPS.


By the way, on a side note, how have you tested this patch series ? I'd like 
to perform some tests on my side too, and avoid reinventing the wheel if 
possible.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Guennadi Liakhovetski]

On Tue, 5 Dec 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> On Tuesday, 5 December 2017 15:44:34 EET Guennadi Liakhovetski wrote:
> > On Tue, 5 Dec 2017, Laurent Pinchart wrote:
> > > On Tuesday, 5 December 2017 12:56:53 EET Guennadi Liakhovetski wrote:
> > >> On Tue, 5 Dec 2017, Laurent Pinchart wrote:
> > >>> On Wednesday, 8 November 2017 18:00:14 EET Guennadi Liakhovetski wrote:
> > >>>> Some UVC video cameras contain metadata in their payload headers.
> > >>>> This patch extracts that data, adding more clock synchronisation
> > >>>> information, on both bulk and isochronous endpoints and makes it
> > >>>> available to the user space on a separate video node, using the
> > >>>> V4L2_CAP_META_CAPTURE capability and the V4L2_BUF_TYPE_META_CAPTURE
> > >>>> buffer queue type. Even though different cameras will have different
> > >>>> metadata formats, we use the same V4L2_META_FMT_UVC pixel format for
> > >>>> all of them. Users have to parse data, based on the specific camera
> > >>>> model information. This version of the patch only creates such metadata
> > >>>> nodes for cameras, specifying a UVC_QUIRK_METADATA_NODE quirk flag.
> > >>> 
> > >>> I don't think this is correct anymore, as we'll use different 4CCs for
> > >>> different vendor metadata. How would you like to rephrase the commit
> > >>> message ?
> > >> 
> > >> Something like
> > >> 
> > >> "
> > >> Some UVC video cameras contain metadata in their payload headers. This
> > >> patch extracts that data, adding more clock synchronisation information,
> > >> on both bulk and isochronous endpoints and makes it available to the
> > >> user space on a separate video node, using the V4L2_CAP_META_CAPTURE
> > >> capability and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By
> > >> default, only the V4L2_META_FMT_UVC pixel format is available from those
> > >> nodes. However, cameras can be added to the device ID table to
> > >> additionally specify their own metadata format, in which case that
> > >> format will also become available from the metadata node.
> > >> "
> > > 
> > > Sounds good to me.
> > > 
> > >>>> Signed-off-by: Guennadi Liakhovetski
> > >>>> <guennadi.liakhovetski@intel.com>
> > >>>> ---
> > >>>> 
> > >>>> v7: support up to two metadata formats per camera - the standard one
> > >>>> and an optional private one, if specified in device information
> > >>>> 
> > >>>>  drivers/media/usb/uvc/Makefile       |   2 +-
> > >>>>  drivers/media/usb/uvc/uvc_driver.c   |  15 +++
> > >>>>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> > >>>>  drivers/media/usb/uvc/uvc_metadata.c | 204 +++++++++++++++++++++++++++
> > >>>>  drivers/media/usb/uvc/uvc_queue.c    |  41 +++++--
> > >>>>  drivers/media/usb/uvc/uvc_video.c    | 127 ++++++++++++++++++++--
> > >>>>  drivers/media/usb/uvc/uvcvideo.h     |  19 +++-
> > >>>>  drivers/media/v4l2-core/v4l2-ioctl.c |   1 +
> > >>>>  include/uapi/linux/uvcvideo.h        |  26 +++++
> > >>>>  9 files changed, 416 insertions(+), 21 deletions(-)
> > >>>>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> > > 
> > > [snip]
> > > 
> > >>>> diff --git a/drivers/media/usb/uvc/uvc_metadata.c
> > >>>> b/drivers/media/usb/uvc/uvc_metadata.c new file mode 100644
> > >>>> index 0000000..21eeee9
> > >>>> --- /dev/null
> > >>>> +++ b/drivers/media/usb/uvc/uvc_metadata.c
> > > 
> > > [snip]
> > > 
> > >>>> +static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
> > >>>> +				  struct v4l2_capability *cap)
> > >>>> +{
> > >>>> +	struct v4l2_fh *vfh = file->private_data;
> > >>>> +	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
> > >>>> +
> > >>>> +	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
> > >>>> +	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
> > >>>> +	usb_make_path(stream->dev->udev, cap->bus_info,
> > >>>> sizeof(cap->bus_info));
> > >>>> +
> > >>>> +	return 0;
> > >>>> +}
> > >>> 
> > >>> Do you think we could reuse uvc_ioctl_querycap() as-is ?
> > >> 
> > >> AFAICS it still has
> > >> 
> > >> 	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
> > > 			  | chain->caps;
> > >> 
> > >> in it, which doesn't suit the metadata node?
> > > 
> > > I'd say this is debatable, isn't the capabilities field supposed to
> > > include all capabilities from all video nodes for the device ? chain->caps
> > > would need to include metadata capability in that case.
> > > 
> > > Code reuse is not a big deal as the function is small, but getting the
> > > capabilities value right is important regardless.
> > 
> > Hm, but that's how applications would naturally work - open a node, query
> > its capabilities and handle it accordingly. What good would be having
> > equal capabilities on all nodes? Why do you think that should be the case?
> 
> Quoting https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/vidioc-querycap.html,
> 
> __u32 	capabilities
> 
> Available capabilities of the physical device as a whole, see Device 
> Capabilities Flags. The same physical device can export multiple devices in /
> dev (e.g. /dev/videoX, /dev/vbiY and /dev/radioZ). The capabilities field 
> should contain a union of all capabilities available around the several V4L2 
> devices exported to userspace. For all those devices the capabilities field 
> returns the same set of capabilities. This allows applications to open just 
> one of the devices (typically the video device) and discover whether video, 
> vbi and/or radio are also supported.
> 
> __u32 	device_caps
> 
> Device capabilities of the opened device, see Device Capabilities Flags. 
> Should contain the available capabilities of that specific device node. So, 
> for example, device_caps of a radio device will only contain radio related 
> capabilities and no video or vbi capabilities. This field is only set if the 
> capabilities field contains the V4L2_CAP_DEVICE_CAPS capability. Only the 
> capabilities field can have the V4L2_CAP_DEVICE_CAPS capability, device_caps 
> will never set V4L2_CAP_DEVICE_CAPS.

Ah, ok, thanks, will try to do that then.

> By the way, on a side note, how have you tested this patch series ? I'd like 
> to perform some tests on my side too, and avoid reinventing the wheel if 
> possible.

I modified the capture-example tool to dequeue metadata buffers, I've also 
added some primitive buffer parsing. If you want I can send you that 
modified version off-list.

Thanks
Guennadi

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Guennadi Liakhovetski]

Hi Laurent,

While testing the new patch version, we did introduce a couple of 
differences:

1. We cannot (easily) reuse .vidioc_querycap() - the metadata node uses 
v4l2_fh_open() directly, so, it has a different struct file::private_data 
pointer.

2. After your video device unification, the order has swapped: now 
/dev/video0 is a metadata node and /dev/video1 is a video node. Is that 
how you wanted to have this or you don't mind or shall I swap them back? 
For now I've swapped them back, I think that would be more appropriate.

Thanks
Guennadi

Re: [PATCH 3/3 v7] uvcvideo: add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Wednesday, 6 December 2017 17:08:14 EET Guennadi Liakhovetski wrote:
> Hi Laurent,
> 
> While testing the new patch version, we did introduce a couple of
> differences:
> 
> 1. We cannot (easily) reuse .vidioc_querycap() - the metadata node uses
> v4l2_fh_open() directly, so, it has a different struct file::private_data
> pointer.

OK, no problem, it's not a big deal.µ

> 2. After your video device unification, the order has swapped: now
> /dev/video0 is a metadata node and /dev/video1 is a video node. Is that
> how you wanted to have this or you don't mind or shall I swap them back?
> For now I've swapped them back, I think that would be more appropriate.

I agree, that's better, thank you.

-- 
Regards,

Laurent Pinchart

[PATCH v8] uvcvideo: Add a metadata device node

[Guennadi Liakhovetski]

From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>

Some UVC video cameras contain metadata in their payload headers. This
patch extracts that data, adding more clock synchronisation information,
on both bulk and isochronous endpoints and makes it available to the user
space on a separate video node, using the V4L2_CAP_META_CAPTURE capability
and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By default, only the
V4L2_META_FMT_UVC pixel format is available from those nodes. However,
cameras can be added to the device ID table to additionally specify their
own metadata format, in which case that format will also become available
from the metadata node.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
---

v8: addressed comments and integrated changes from Laurent, thanks again, 
e.g.:

- multiple stylistic changes
- remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now 
  created unconditionally
- reuse uvc_ioctl_querycap()
- reuse code in uvc_register_video()
- set an error flag when the metadata buffer overflows

 drivers/media/usb/uvc/Makefile       |   2 +-
 drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
 drivers/media/usb/uvc/uvc_isight.c   |   2 +-
 drivers/media/usb/uvc/uvc_metadata.c | 179 +++++++++++++++++++++++++++++++++++
 drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
 drivers/media/usb/uvc/uvc_video.c    | 132 ++++++++++++++++++++++++--
 drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
 include/uapi/linux/uvcvideo.h        |  26 +++++
 8 files changed, 394 insertions(+), 22 deletions(-)
 create mode 100644 drivers/media/usb/uvc/uvc_metadata.c

diff --git a/drivers/media/usb/uvc/Makefile b/drivers/media/usb/uvc/Makefile
index c26d12f..06c7cd3 100644
--- a/drivers/media/usb/uvc/Makefile
+++ b/drivers/media/usb/uvc/Makefile
@@ -1,5 +1,5 @@
 uvcvideo-objs  := uvc_driver.o uvc_queue.o uvc_v4l2.o uvc_video.o uvc_ctrl.o \
-		  uvc_status.o uvc_isight.o uvc_debugfs.o
+		  uvc_status.o uvc_isight.o uvc_debugfs.o uvc_metadata.o
 ifeq ($(CONFIG_MEDIA_CONTROLLER),y)
 uvcvideo-objs  += uvc_entity.o
 endif
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 88032c2..36061f3 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -1883,6 +1883,7 @@ static void uvc_unregister_video(struct uvc_device *dev)
 			continue;
 
 		video_unregister_device(&stream->vdev);
+		video_unregister_device(&stream->meta.vdev);
 
 		uvc_debugfs_cleanup_stream(stream);
 	}
@@ -1930,6 +1931,9 @@ int uvc_register_video_device(struct uvc_device *dev,
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT:
 		vdev->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
 		break;
+	case V4L2_BUF_TYPE_META_CAPTURE:
+		vdev->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
+		break;
 	}
 
 	strlcpy(vdev->name, dev->name, sizeof vdev->name);
@@ -1965,7 +1969,8 @@ static int uvc_register_video(struct uvc_device *dev,
 	}
 
 	if (stream->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
-		stream->chain->caps |= V4L2_CAP_VIDEO_CAPTURE;
+		stream->chain->caps |= V4L2_CAP_VIDEO_CAPTURE
+			| V4L2_CAP_META_CAPTURE;
 	else
 		stream->chain->caps |= V4L2_CAP_VIDEO_OUTPUT;
 
@@ -2003,6 +2008,11 @@ static int uvc_register_terms(struct uvc_device *dev,
 		if (ret < 0)
 			return ret;
 
+		/* Register a metadata node, but ignore a possible failure,
+		 * complete registration of video nodes anyway.
+		 */
+		uvc_meta_register(stream);
+
 		term->vdev = &stream->vdev;
 	}
 
@@ -2037,6 +2047,7 @@ static int uvc_register_chains(struct uvc_device *dev)
 
 struct uvc_device_info {
 	u32	quirks;
+	u32	meta_format;
 };
 
 static int uvc_probe(struct usb_interface *intf,
@@ -2074,6 +2085,8 @@ static int uvc_probe(struct usb_interface *intf,
 	dev->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
 	dev->quirks = (uvc_quirks_param == -1)
 		    ? quirks : uvc_quirks_param;
+	if (info)
+		dev->meta_format = info->meta_format;
 
 	if (udev->product != NULL)
 		strlcpy(dev->name, udev->product, sizeof dev->name);
diff --git a/drivers/media/usb/uvc/uvc_isight.c b/drivers/media/usb/uvc/uvc_isight.c
index 8510e725..fb940cf 100644
--- a/drivers/media/usb/uvc/uvc_isight.c
+++ b/drivers/media/usb/uvc/uvc_isight.c
@@ -100,7 +100,7 @@ static int isight_decode(struct uvc_video_queue *queue, struct uvc_buffer *buf,
 }
 
 void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
-		struct uvc_buffer *buf)
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	int ret, i;
 
diff --git a/drivers/media/usb/uvc/uvc_metadata.c b/drivers/media/usb/uvc/uvc_metadata.c
new file mode 100644
index 0000000..cd1aec1
--- /dev/null
+++ b/drivers/media/usb/uvc/uvc_metadata.c
@@ -0,0 +1,179 @@
+/*
+ *      uvc_metadata.c  --  USB Video Class driver - Metadata handling
+ *
+ *      Copyright (C) 2016
+ *          Guennadi Liakhovetski (guennadi.liakhovetski@intel.com)
+ *
+ *      This program is free software; you can redistribute it and/or modify
+ *      it under the terms of the GNU General Public License as published by
+ *      the Free Software Foundation; either version 2 of the License, or
+ *      (at your option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/usb.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-v4l2.h>
+#include <media/videobuf2-vmalloc.h>
+
+#include "uvcvideo.h"
+
+/* -----------------------------------------------------------------------------
+ * V4L2 ioctls
+ */
+
+static int uvc_meta_v4l2_querycap(struct file *file, void *fh,
+				  struct v4l2_capability *cap)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct uvc_video_chain *chain = stream->chain;
+
+	strlcpy(cap->driver, "uvcvideo", sizeof(cap->driver));
+	strlcpy(cap->card, vfh->vdev->name, sizeof(cap->card));
+	usb_make_path(stream->dev->udev, cap->bus_info, sizeof(cap->bus_info));
+	cap->capabilities = V4L2_CAP_DEVICE_CAPS | V4L2_CAP_STREAMING
+			  | chain->caps;
+
+	return 0;
+}
+
+static int uvc_meta_v4l2_get_format(struct file *file, void *fh,
+				    struct v4l2_format *format)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct v4l2_meta_format *fmt = &format->fmt.meta;
+
+	if (format->type != vfh->vdev->queue->type)
+		return -EINVAL;
+
+	memset(fmt, 0, sizeof(*fmt));
+
+	fmt->dataformat = stream->meta.format;
+	fmt->buffersize = UVC_METATADA_BUF_SIZE;
+
+	return 0;
+}
+
+static int uvc_meta_v4l2_try_format(struct file *file, void *fh,
+				    struct v4l2_format *format)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct uvc_device *dev = stream->dev;
+	struct v4l2_meta_format *fmt = &format->fmt.meta;
+	u32 fmeta = fmt->dataformat;
+
+	if (format->type != vfh->vdev->queue->type)
+		return -EINVAL;
+
+	memset(fmt, 0, sizeof(*fmt));
+
+	fmt->dataformat = fmeta == dev->meta_format ? fmeta : V4L2_META_FMT_UVC;
+	fmt->buffersize = UVC_METATADA_BUF_SIZE;
+
+	return 0;
+}
+
+static int uvc_meta_v4l2_set_format(struct file *file, void *fh,
+				    struct v4l2_format *format)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct v4l2_meta_format *fmt = &format->fmt.meta;
+	int ret;
+
+	ret = uvc_meta_v4l2_try_format(file, fh, format);
+	if (ret < 0)
+		return ret;
+
+	/*
+	 * We could in principle switch at any time, also during streaming.
+	 * Metadata buffers would still be perfectly parseable, but it's more
+	 * consistent and cleaner to disallow that.
+	 */
+	mutex_lock(&stream->mutex);
+
+	if (uvc_queue_allocated(&stream->queue))
+		ret = -EBUSY;
+	else
+		stream->meta.format = fmt->dataformat;
+
+	mutex_unlock(&stream->mutex);
+
+	return ret;
+}
+
+static int uvc_meta_v4l2_enum_formats(struct file *file, void *fh,
+				      struct v4l2_fmtdesc *fdesc)
+{
+	struct v4l2_fh *vfh = file->private_data;
+	struct uvc_streaming *stream = video_get_drvdata(vfh->vdev);
+	struct uvc_device *dev = stream->dev;
+	u32 index = fdesc->index;
+
+	if (fdesc->type != vfh->vdev->queue->type ||
+	    index > 1U || (index && !dev->meta_format))
+		return -EINVAL;
+
+	memset(fdesc, 0, sizeof(*fdesc));
+
+	fdesc->type = vfh->vdev->queue->type;
+	fdesc->index = index;
+	fdesc->pixelformat = index ? dev->meta_format : V4L2_META_FMT_UVC;
+
+	return 0;
+}
+
+static const struct v4l2_ioctl_ops uvc_meta_ioctl_ops = {
+	.vidioc_querycap		= uvc_meta_v4l2_querycap,
+	.vidioc_g_fmt_meta_cap		= uvc_meta_v4l2_get_format,
+	.vidioc_s_fmt_meta_cap		= uvc_meta_v4l2_set_format,
+	.vidioc_try_fmt_meta_cap	= uvc_meta_v4l2_try_format,
+	.vidioc_enum_fmt_meta_cap	= uvc_meta_v4l2_enum_formats,
+	.vidioc_reqbufs			= vb2_ioctl_reqbufs,
+	.vidioc_querybuf		= vb2_ioctl_querybuf,
+	.vidioc_qbuf			= vb2_ioctl_qbuf,
+	.vidioc_dqbuf			= vb2_ioctl_dqbuf,
+	.vidioc_create_bufs		= vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf		= vb2_ioctl_prepare_buf,
+	.vidioc_streamon		= vb2_ioctl_streamon,
+	.vidioc_streamoff		= vb2_ioctl_streamoff,
+};
+
+/* -----------------------------------------------------------------------------
+ * V4L2 File Operations
+ */
+
+static const struct v4l2_file_operations uvc_meta_fops = {
+	.owner = THIS_MODULE,
+	.unlocked_ioctl = video_ioctl2,
+	.open = v4l2_fh_open,
+	.release = vb2_fop_release,
+	.poll = vb2_fop_poll,
+	.mmap = vb2_fop_mmap,
+};
+
+int uvc_meta_register(struct uvc_streaming *stream)
+{
+	struct uvc_device *dev = stream->dev;
+	struct video_device *vdev = &stream->meta.vdev;
+	struct uvc_video_queue *queue = &stream->meta.queue;
+
+	stream->meta.format = V4L2_META_FMT_UVC;
+
+	/*
+	 * The video interface queue uses manual locking and thus does not set
+	 * the queue pointer. Set it manually here.
+	 */
+	vdev->queue = &queue->queue;
+
+	return uvc_register_video_device(dev, stream, vdev, queue,
+					 V4L2_BUF_TYPE_META_CAPTURE,
+					 &uvc_meta_fops, &uvc_meta_ioctl_ops);
+}
diff --git a/drivers/media/usb/uvc/uvc_queue.c b/drivers/media/usb/uvc/uvc_queue.c
index c8d78b2..cd2ea5a 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -79,8 +79,19 @@ static int uvc_queue_setup(struct vb2_queue *vq,
 			   unsigned int sizes[], struct device *alloc_devs[])
 {
 	struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
-	struct uvc_streaming *stream = uvc_queue_to_stream(queue);
-	unsigned size = stream->ctrl.dwMaxVideoFrameSize;
+	struct uvc_streaming *stream;
+	unsigned int size;
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_META_CAPTURE:
+		size = UVC_METATADA_BUF_SIZE;
+		break;
+
+	default:
+		stream = uvc_queue_to_stream(queue);
+		size = stream->ctrl.dwMaxVideoFrameSize;
+		break;
+	}
 
 	/*
 	 * When called with plane sizes, validate them. The driver supports
@@ -114,7 +125,7 @@ static int uvc_buffer_prepare(struct vb2_buffer *vb)
 	buf->error = 0;
 	buf->mem = vb2_plane_vaddr(vb, 0);
 	buf->length = vb2_plane_size(vb, 0);
-	if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
+	if (vb->type != V4L2_BUF_TYPE_VIDEO_OUTPUT)
 		buf->bytesused = 0;
 	else
 		buf->bytesused = vb2_get_plane_payload(vb, 0);
@@ -177,10 +188,10 @@ static int uvc_start_streaming(struct vb2_queue *vq, unsigned int count)
 static void uvc_stop_streaming(struct vb2_queue *vq)
 {
 	struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
-	struct uvc_streaming *stream = uvc_queue_to_stream(queue);
 	unsigned long flags;
 
-	uvc_video_enable(stream, 0);
+	if (vq->type != V4L2_BUF_TYPE_META_CAPTURE)
+		uvc_video_enable(uvc_queue_to_stream(queue), 0);
 
 	spin_lock_irqsave(&queue->irqlock, flags);
 	uvc_queue_return_buffers(queue, UVC_BUF_STATE_ERROR);
@@ -198,20 +209,39 @@ static void uvc_stop_streaming(struct vb2_queue *vq)
 	.stop_streaming = uvc_stop_streaming,
 };
 
+static const struct vb2_ops uvc_meta_queue_qops = {
+	.queue_setup = uvc_queue_setup,
+	.buf_prepare = uvc_buffer_prepare,
+	.buf_queue = uvc_buffer_queue,
+	.wait_prepare = vb2_ops_wait_prepare,
+	.wait_finish = vb2_ops_wait_finish,
+	.stop_streaming = uvc_stop_streaming,
+};
+
 int uvc_queue_init(struct uvc_video_queue *queue, enum v4l2_buf_type type,
 		    int drop_corrupted)
 {
 	int ret;
 
 	queue->queue.type = type;
-	queue->queue.io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF;
+	queue->queue.io_modes = VB2_MMAP | VB2_USERPTR;
 	queue->queue.drv_priv = queue;
 	queue->queue.buf_struct_size = sizeof(struct uvc_buffer);
-	queue->queue.ops = &uvc_queue_qops;
 	queue->queue.mem_ops = &vb2_vmalloc_memops;
 	queue->queue.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC
 		| V4L2_BUF_FLAG_TSTAMP_SRC_SOE;
 	queue->queue.lock = &queue->mutex;
+
+	switch (type) {
+	case V4L2_BUF_TYPE_META_CAPTURE:
+		queue->queue.ops = &uvc_meta_queue_qops;
+		break;
+	default:
+		queue->queue.io_modes |= VB2_DMABUF;
+		queue->queue.ops = &uvc_queue_qops;
+		break;
+	}
+
 	ret = vb2_queue_init(&queue->queue);
 	if (ret)
 		return ret;
diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 13f459e..2fc0bf2 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1120,6 +1120,82 @@ static int uvc_video_encode_data(struct uvc_streaming *stream,
 }
 
 /* ------------------------------------------------------------------------
+ * Metadata
+ */
+
+/*
+ * Additionally to the payload headers we also want to provide the user with USB
+ * Frame Numbers and system time values. The resulting buffer is thus composed
+ * of blocks, containing a 64-bit timestamp in  nanoseconds, a 16-bit USB Frame
+ * Number, and a copy of the payload header.
+ *
+ * Ideally we want to capture all payload headers for each frame. However, their
+ * number is unknown and unbound. We thus drop headers that contain no vendor
+ * data and that either contain no SCR value or an SCR value identical to the
+ * previous header.
+ */
+static void uvc_video_decode_meta(struct uvc_streaming *stream,
+				  struct uvc_buffer *meta_buf,
+				  const u8 *mem, unsigned int length)
+{
+	struct uvc_meta_buf *meta;
+	size_t len_std = 2;
+	bool has_pts, has_scr;
+	unsigned long flags;
+	ktime_t time;
+	const u8 *scr;
+
+	if (!meta_buf || length == 2)
+		return;
+
+	if (meta_buf->length - meta_buf->bytesused <
+	    length + sizeof(meta->ns) + sizeof(meta->sof)) {
+		meta_buf->error = 1;
+		return;
+	}
+
+	has_pts = mem[1] & UVC_STREAM_PTS;
+	has_scr = mem[1] & UVC_STREAM_SCR;
+
+	if (has_pts) {
+		len_std += 4;
+		scr = mem + 6;
+	} else {
+		scr = mem + 2;
+	}
+
+	if (has_scr)
+		len_std += 6;
+
+	if (stream->meta.format == V4L2_META_FMT_UVC)
+		length = len_std;
+
+	if (length == len_std && (!has_scr ||
+				  !memcmp(scr, stream->clock.last_scr, 6)))
+		return;
+
+	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
+	local_irq_save(flags);
+	time = uvc_video_get_time();
+	meta->sof = usb_get_current_frame_number(stream->dev->udev);
+	local_irq_restore(flags);
+	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
+
+	if (has_scr)
+		memcpy(stream->clock.last_scr, scr, 6);
+
+	memcpy(&meta->length, mem, length);
+	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
+
+	uvc_trace(UVC_TRACE_FRAME,
+		  "%s(): t-sys %lluns, SOF %u, len %u, flags 0x%x, PTS %u, STC %u frame SOF %u\n",
+		  __func__, time, meta->sof, meta->length, meta->flags,
+		  has_pts ? *(u32 *)meta->buf : 0,
+		  has_scr ? *(u32 *)scr : 0,
+		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
+}
+
+/* ------------------------------------------------------------------------
  * URB handling
  */
 
@@ -1137,8 +1213,29 @@ static void uvc_video_validate_buffer(const struct uvc_streaming *stream,
 /*
  * Completion handler for video URBs.
  */
+
+static void uvc_video_next_buffers(struct uvc_streaming *stream,
+		struct uvc_buffer **video_buf, struct uvc_buffer **meta_buf)
+{
+	if (*meta_buf) {
+		struct vb2_v4l2_buffer *vb2_meta = &(*meta_buf)->buf;
+		const struct vb2_v4l2_buffer *vb2_video = &(*video_buf)->buf;
+
+		vb2_meta->sequence = vb2_video->sequence;
+		vb2_meta->field = vb2_video->field;
+		vb2_meta->vb2_buf.timestamp = vb2_video->vb2_buf.timestamp;
+
+		(*meta_buf)->state = UVC_BUF_STATE_READY;
+		if (!(*meta_buf)->error)
+			(*meta_buf)->error = (*video_buf)->error;
+		*meta_buf = uvc_queue_next_buffer(&stream->meta.queue,
+						  *meta_buf);
+	}
+	*video_buf = uvc_queue_next_buffer(&stream->queue, *video_buf);
+}
+
 static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	u8 *mem;
 	int ret, i;
@@ -1160,14 +1257,15 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
 				urb->iso_frame_desc[i].actual_length);
 			if (ret == -EAGAIN) {
 				uvc_video_validate_buffer(stream, buf);
-				buf = uvc_queue_next_buffer(&stream->queue,
-							    buf);
+				uvc_video_next_buffers(stream, &buf, &meta_buf);
 			}
 		} while (ret == -EAGAIN);
 
 		if (ret < 0)
 			continue;
 
+		uvc_video_decode_meta(stream, meta_buf, mem, ret);
+
 		/* Decode the payload data. */
 		uvc_video_decode_data(stream, buf, mem + ret,
 			urb->iso_frame_desc[i].actual_length - ret);
@@ -1178,13 +1276,13 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
 
 		if (buf->state == UVC_BUF_STATE_READY) {
 			uvc_video_validate_buffer(stream, buf);
-			buf = uvc_queue_next_buffer(&stream->queue, buf);
+			uvc_video_next_buffers(stream, &buf, &meta_buf);
 		}
 	}
 }
 
 static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	u8 *mem;
 	int len, ret;
@@ -1207,8 +1305,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 		do {
 			ret = uvc_video_decode_start(stream, buf, mem, len);
 			if (ret == -EAGAIN)
-				buf = uvc_queue_next_buffer(&stream->queue,
-							    buf);
+				uvc_video_next_buffers(stream, &buf, &meta_buf);
 		} while (ret == -EAGAIN);
 
 		/* If an error occurred skip the rest of the payload. */
@@ -1218,6 +1315,8 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 			memcpy(stream->bulk.header, mem, ret);
 			stream->bulk.header_size = ret;
 
+			uvc_video_decode_meta(stream, meta_buf, mem, ret);
+
 			mem += ret;
 			len -= ret;
 		}
@@ -1241,7 +1340,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 			uvc_video_decode_end(stream, buf, stream->bulk.header,
 				stream->bulk.payload_size);
 			if (buf->state == UVC_BUF_STATE_READY)
-				uvc_queue_next_buffer(&stream->queue, buf);
+				uvc_video_next_buffers(stream, &buf, &meta_buf);
 		}
 
 		stream->bulk.header_size = 0;
@@ -1251,7 +1350,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 }
 
 static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+	struct uvc_buffer *buf, struct uvc_buffer *meta_buf)
 {
 	u8 *mem = urb->transfer_buffer;
 	int len = stream->urb_size, ret;
@@ -1297,7 +1396,10 @@ static void uvc_video_complete(struct urb *urb)
 {
 	struct uvc_streaming *stream = urb->context;
 	struct uvc_video_queue *queue = &stream->queue;
+	struct uvc_video_queue *qmeta = &stream->meta.queue;
+	struct vb2_queue *vb2_qmeta = stream->meta.vdev.queue;
 	struct uvc_buffer *buf = NULL;
+	struct uvc_buffer *buf_meta = NULL;
 	unsigned long flags;
 	int ret;
 
@@ -1316,6 +1418,8 @@ static void uvc_video_complete(struct urb *urb)
 	case -ECONNRESET:	/* usb_unlink_urb() called. */
 	case -ESHUTDOWN:	/* The endpoint is being disabled. */
 		uvc_queue_cancel(queue, urb->status == -ESHUTDOWN);
+		if (vb2_qmeta)
+			uvc_queue_cancel(qmeta, urb->status == -ESHUTDOWN);
 		return;
 	}
 
@@ -1325,7 +1429,15 @@ static void uvc_video_complete(struct urb *urb)
 				       queue);
 	spin_unlock_irqrestore(&queue->irqlock, flags);
 
-	stream->decode(urb, stream, buf);
+	if (vb2_qmeta) {
+		spin_lock_irqsave(&qmeta->irqlock, flags);
+		if (!list_empty(&qmeta->irqqueue))
+			buf_meta = list_first_entry(&qmeta->irqqueue,
+						    struct uvc_buffer, queue);
+		spin_unlock_irqrestore(&qmeta->irqlock, flags);
+	}
+
+	stream->decode(urb, stream, buf, buf_meta);
 
 	if ((ret = usb_submit_urb(urb, GFP_ATOMIC)) < 0) {
 		uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index e0266bf..b65e86b 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -478,6 +478,8 @@ struct uvc_stats_stream {
 	unsigned int max_sof;		/* Maximum STC.SOF value */
 };
 
+#define UVC_METATADA_BUF_SIZE 1024
+
 struct uvc_streaming {
 	struct list_head list;
 	struct uvc_device *dev;
@@ -509,7 +511,13 @@ struct uvc_streaming {
 	unsigned int frozen : 1;
 	struct uvc_video_queue queue;
 	void (*decode) (struct urb *urb, struct uvc_streaming *video,
-			struct uvc_buffer *buf);
+			struct uvc_buffer *buf, struct uvc_buffer *meta_buf);
+
+	struct {
+		struct video_device vdev;
+		struct uvc_video_queue queue;
+		__u32 format;
+	} meta;
 
 	/* Context data used by the bulk completion handler. */
 	struct {
@@ -551,6 +559,8 @@ struct uvc_streaming {
 		u16 last_sof;
 		u16 sof_offset;
 
+		u8 last_scr[6];
+
 		spinlock_t lock;
 	} clock;
 };
@@ -560,6 +570,7 @@ struct uvc_device {
 	struct usb_interface *intf;
 	unsigned long warnings;
 	__u32 quirks;
+	__u32 meta_format;
 	int intfnum;
 	char name[32];
 
@@ -714,6 +725,7 @@ extern int uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
 void uvc_video_clock_update(struct uvc_streaming *stream,
 			    struct vb2_v4l2_buffer *vbuf,
 			    struct uvc_buffer *buf);
+int uvc_meta_register(struct uvc_streaming *stream);
 
 int uvc_register_video_device(struct uvc_device *dev,
 			      struct uvc_streaming *stream,
@@ -776,7 +788,7 @@ extern struct usb_host_endpoint *uvc_find_endpoint(
 
 /* Quirks support */
 void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
-		struct uvc_buffer *buf);
+		struct uvc_buffer *buf, struct uvc_buffer *meta_buf);
 
 /* debugfs and statistics */
 void uvc_debugfs_init(void);
diff --git a/include/uapi/linux/uvcvideo.h b/include/uapi/linux/uvcvideo.h
index 3b08186..8381ca7c 100644
--- a/include/uapi/linux/uvcvideo.h
+++ b/include/uapi/linux/uvcvideo.h
@@ -67,4 +67,30 @@ struct uvc_xu_control_query {
 #define UVCIOC_CTRL_MAP		_IOWR('u', 0x20, struct uvc_xu_control_mapping)
 #define UVCIOC_CTRL_QUERY	_IOWR('u', 0x21, struct uvc_xu_control_query)
 
+/*
+ * Metadata node
+ */
+
+/**
+ * struct uvc_meta_buf - metadata buffer building block
+ * @ns		- system timestamp of the payload in nanoseconds
+ * @sof		- USB Frame Number
+ * @length	- length of the payload header
+ * @flags	- payload header flags
+ * @buf		- optional device-specific header data
+ *
+ * UVC metadata nodes fill buffers with possibly multiple instances of this
+ * struct. The first two fields are added by the driver, they can be used for
+ * clock synchronisation. The rest is an exact copy of a UVC payload header.
+ * Only complete objects with complete buffers are included. Therefore it's
+ * always sizeof(meta->ts) + sizeof(meta->sof) + meta->length bytes large.
+ */
+struct uvc_meta_buf {
+	__u64 ns;
+	__u16 sof;
+	__u8 length;
+	__u8 flags;
+	__u8 buf[];
+} __packed;
+
 #endif
-- 
1.9.3

Re: [PATCH v8] uvcvideo: Add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

Thank you for the patch.

On Wednesday, 6 December 2017 17:15:40 EET Guennadi Liakhovetski wrote:
> From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> 
> Some UVC video cameras contain metadata in their payload headers. This
> patch extracts that data, adding more clock synchronisation information,
> on both bulk and isochronous endpoints and makes it available to the user
> space on a separate video node, using the V4L2_CAP_META_CAPTURE capability
> and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By default, only the
> V4L2_META_FMT_UVC pixel format is available from those nodes. However,
> cameras can be added to the device ID table to additionally specify their
> own metadata format, in which case that format will also become available
> from the metadata node.
> 
> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> ---
> 
> v8: addressed comments and integrated changes from Laurent, thanks again,
> e.g.:
> 
> - multiple stylistic changes
> - remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now
>   created unconditionally
> - reuse uvc_ioctl_querycap()
> - reuse code in uvc_register_video()
> - set an error flag when the metadata buffer overflows
> 
>  drivers/media/usb/uvc/Makefile       |   2 +-
>  drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
>  drivers/media/usb/uvc/uvc_metadata.c | 179 ++++++++++++++++++++++++++++++++
>  drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
>  drivers/media/usb/uvc/uvc_video.c    | 132 ++++++++++++++++++++++++--
>  drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
>  include/uapi/linux/uvcvideo.h        |  26 +++++
>  8 files changed, 394 insertions(+), 22 deletions(-)
>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c

[snip]

> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 13f459e..2fc0bf2 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c

[snip]

> +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> +				  struct uvc_buffer *meta_buf,
> +				  const u8 *mem, unsigned int length)
> +{
> +	struct uvc_meta_buf *meta;
> +	size_t len_std = 2;
> +	bool has_pts, has_scr;
> +	unsigned long flags;
> +	ktime_t time;
> +	const u8 *scr;
> +
> +	if (!meta_buf || length == 2)
> +		return;
> +
> +	if (meta_buf->length - meta_buf->bytesused <
> +	    length + sizeof(meta->ns) + sizeof(meta->sof)) {
> +		meta_buf->error = 1;
> +		return;
> +	}
> +
> +	has_pts = mem[1] & UVC_STREAM_PTS;
> +	has_scr = mem[1] & UVC_STREAM_SCR;
> +
> +	if (has_pts) {
> +		len_std += 4;
> +		scr = mem + 6;
> +	} else {
> +		scr = mem + 2;
> +	}
> +
> +	if (has_scr)
> +		len_std += 6;
> +
> +	if (stream->meta.format == V4L2_META_FMT_UVC)
> +		length = len_std;
> +
> +	if (length == len_std && (!has_scr ||
> +				  !memcmp(scr, stream->clock.last_scr, 6)))
> +		return;
> +
> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
> +	local_irq_save(flags);
> +	time = uvc_video_get_time();
> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);

You need a put_unaligned here too. If you're fine with the patch below there's 
no need to resubmit, and

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Would you mind sending me your test tool patch ?

diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/
uvc_video.c
index 2fc0bf2221db..02e4997a32bb 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1142,6 +1142,7 @@ static void uvc_video_decode_meta(struct uvc_streaming 
*stream,
 	size_t len_std = 2;
 	bool has_pts, has_scr;
 	unsigned long flags;
+	unsigned int sof;
 	ktime_t time;
 	const u8 *scr;
 
@@ -1177,9 +1178,10 @@ static void uvc_video_decode_meta(struct uvc_streaming 
*stream,
 	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
 	local_irq_save(flags);
 	time = uvc_video_get_time();
-	meta->sof = usb_get_current_frame_number(stream->dev->udev);
+	sof = usb_get_current_frame_number(stream->dev->udev);
 	local_irq_restore(flags);
 	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
+	__put_unaligned_cpu16(sof, &meta->sof);
 
 	if (has_scr)
 		memcpy(stream->clock.last_scr, scr, 6);

> +	local_irq_restore(flags);
> +	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> +
> +	if (has_scr)
> +		memcpy(stream->clock.last_scr, scr, 6);
> +
> +	memcpy(&meta->length, mem, length);
> +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> +
> +	uvc_trace(UVC_TRACE_FRAME,
> +		  "%s(): t-sys %lluns, SOF %u, len %u, flags 0x%x, PTS %u, STC %u frame
> SOF %u\n", +		  __func__, time, meta->sof, meta->length, meta->flags,
> +		  has_pts ? *(u32 *)meta->buf : 0,
> +		  has_scr ? *(u32 *)scr : 0,
> +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> +}

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v8] uvcvideo: Add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Monday, 11 December 2017 22:16:23 EET Laurent Pinchart wrote:
> On Wednesday, 6 December 2017 17:15:40 EET Guennadi Liakhovetski wrote:
> > From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > 
> > Some UVC video cameras contain metadata in their payload headers. This
> > patch extracts that data, adding more clock synchronisation information,
> > on both bulk and isochronous endpoints and makes it available to the user
> > space on a separate video node, using the V4L2_CAP_META_CAPTURE capability
> > and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By default, only the
> > V4L2_META_FMT_UVC pixel format is available from those nodes. However,
> > cameras can be added to the device ID table to additionally specify their
> > own metadata format, in which case that format will also become available
> > from the metadata node.
> > 
> > Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > ---
> > 
> > v8: addressed comments and integrated changes from Laurent, thanks again,
> > e.g.:
> > 
> > - multiple stylistic changes
> > - remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now
> >   created unconditionally
> > - reuse uvc_ioctl_querycap()
> > - reuse code in uvc_register_video()
> > - set an error flag when the metadata buffer overflows
> > 
> >  drivers/media/usb/uvc/Makefile       |   2 +-
> >  drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
> >  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> >  drivers/media/usb/uvc/uvc_metadata.c | 179 ++++++++++++++++++++++++++++++
> >  drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
> >  drivers/media/usb/uvc/uvc_video.c    | 132 ++++++++++++++++++++++++--
> >  drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
> >  include/uapi/linux/uvcvideo.h        |  26 +++++
> >  8 files changed, 394 insertions(+), 22 deletions(-)
> >  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> 
> [snip]
> 
> > diff --git a/drivers/media/usb/uvc/uvc_video.c
> > b/drivers/media/usb/uvc/uvc_video.c index 13f459e..2fc0bf2 100644
> > --- a/drivers/media/usb/uvc/uvc_video.c
> > +++ b/drivers/media/usb/uvc/uvc_video.c
> 
> [snip]
> 
> > +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> > +				  struct uvc_buffer *meta_buf,
> > +				  const u8 *mem, unsigned int length)
> > +{
> > +	struct uvc_meta_buf *meta;
> > +	size_t len_std = 2;
> > +	bool has_pts, has_scr;
> > +	unsigned long flags;
> > +	ktime_t time;
> > +	const u8 *scr;
> > +
> > +	if (!meta_buf || length == 2)
> > +		return;
> > +
> > +	if (meta_buf->length - meta_buf->bytesused <
> > +	    length + sizeof(meta->ns) + sizeof(meta->sof)) {
> > +		meta_buf->error = 1;
> > +		return;
> > +	}
> > +
> > +	has_pts = mem[1] & UVC_STREAM_PTS;
> > +	has_scr = mem[1] & UVC_STREAM_SCR;
> > +
> > +	if (has_pts) {
> > +		len_std += 4;
> > +		scr = mem + 6;
> > +	} else {
> > +		scr = mem + 2;
> > +	}
> > +
> > +	if (has_scr)
> > +		len_std += 6;
> > +
> > +	if (stream->meta.format == V4L2_META_FMT_UVC)
> > +		length = len_std;
> > +
> > +	if (length == len_std && (!has_scr ||
> > +				  !memcmp(scr, stream->clock.last_scr, 6)))
> > +		return;
> > +
> > +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> > meta_buf->bytesused); +	local_irq_save(flags);
> > +	time = uvc_video_get_time();
> > +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> 
> You need a put_unaligned here too. If you're fine with the patch below
> there's no need to resubmit, and

One more thing, __put_unaligned_cpu16() and __put_unaligned_cpu64() don't 
compile on x86_64 with v4.12 (using media_build.git). I propose replacing them 
with put_unaligned() which compiles and should do the right thing.

> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
> Would you mind sending me your test tool patch ?
> 
> diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/
> uvc_video.c
> index 2fc0bf2221db..02e4997a32bb 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1142,6 +1142,7 @@ static void uvc_video_decode_meta(struct uvc_streaming
> *stream,
>  	size_t len_std = 2;
>  	bool has_pts, has_scr;
>  	unsigned long flags;
> +	unsigned int sof;
>  	ktime_t time;
>  	const u8 *scr;
> 
> @@ -1177,9 +1178,10 @@ static void uvc_video_decode_meta(struct
> uvc_streaming *stream,
>  	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
> local_irq_save(flags);
>  	time = uvc_video_get_time();
> -	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> +	sof = usb_get_current_frame_number(stream->dev->udev);
>  	local_irq_restore(flags);
>  	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> +	__put_unaligned_cpu16(sof, &meta->sof);
> 
>  	if (has_scr)
>  		memcpy(stream->clock.last_scr, scr, 6);
> 
> > +	local_irq_restore(flags);
> > +	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> > +
> > +	if (has_scr)
> > +		memcpy(stream->clock.last_scr, scr, 6);
> > +
> > +	memcpy(&meta->length, mem, length);
> > +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> > +
> > +	uvc_trace(UVC_TRACE_FRAME,
> > +		  "%s(): t-sys %lluns, SOF %u, len %u, flags 0x%x, PTS %u, STC %u 
frame
> > SOF %u\n", +		  __func__, time, meta->sof, meta->length, meta->flags,
> > +		  has_pts ? *(u32 *)meta->buf : 0,
> > +		  has_scr ? *(u32 *)scr : 0,
> > +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> > +}
> 
> [snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v8] uvcvideo: Add a metadata device node

[Guennadi Liakhovetski]

Hi Laurent,

On Mon, 11 Dec 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> On Monday, 11 December 2017 22:16:23 EET Laurent Pinchart wrote:
> > On Wednesday, 6 December 2017 17:15:40 EET Guennadi Liakhovetski wrote:
> > > From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > > 
> > > Some UVC video cameras contain metadata in their payload headers. This
> > > patch extracts that data, adding more clock synchronisation information,
> > > on both bulk and isochronous endpoints and makes it available to the user
> > > space on a separate video node, using the V4L2_CAP_META_CAPTURE capability
> > > and the V4L2_BUF_TYPE_META_CAPTURE buffer queue type. By default, only the
> > > V4L2_META_FMT_UVC pixel format is available from those nodes. However,
> > > cameras can be added to the device ID table to additionally specify their
> > > own metadata format, in which case that format will also become available
> > > from the metadata node.
> > > 
> > > Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > > ---
> > > 
> > > v8: addressed comments and integrated changes from Laurent, thanks again,
> > > e.g.:
> > > 
> > > - multiple stylistic changes
> > > - remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now
> > >   created unconditionally
> > > - reuse uvc_ioctl_querycap()
> > > - reuse code in uvc_register_video()
> > > - set an error flag when the metadata buffer overflows
> > > 
> > >  drivers/media/usb/uvc/Makefile       |   2 +-
> > >  drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
> > >  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> > >  drivers/media/usb/uvc/uvc_metadata.c | 179 ++++++++++++++++++++++++++++++
> > >  drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
> > >  drivers/media/usb/uvc/uvc_video.c    | 132 ++++++++++++++++++++++++--
> > >  drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
> > >  include/uapi/linux/uvcvideo.h        |  26 +++++
> > >  8 files changed, 394 insertions(+), 22 deletions(-)
> > >  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> > 
> > [snip]
> > 
> > > diff --git a/drivers/media/usb/uvc/uvc_video.c
> > > b/drivers/media/usb/uvc/uvc_video.c index 13f459e..2fc0bf2 100644
> > > --- a/drivers/media/usb/uvc/uvc_video.c
> > > +++ b/drivers/media/usb/uvc/uvc_video.c
> > 
> > [snip]
> > 
> > > +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> > > +				  struct uvc_buffer *meta_buf,
> > > +				  const u8 *mem, unsigned int length)
> > > +{
> > > +	struct uvc_meta_buf *meta;
> > > +	size_t len_std = 2;
> > > +	bool has_pts, has_scr;
> > > +	unsigned long flags;
> > > +	ktime_t time;
> > > +	const u8 *scr;
> > > +
> > > +	if (!meta_buf || length == 2)
> > > +		return;
> > > +
> > > +	if (meta_buf->length - meta_buf->bytesused <
> > > +	    length + sizeof(meta->ns) + sizeof(meta->sof)) {
> > > +		meta_buf->error = 1;
> > > +		return;
> > > +	}
> > > +
> > > +	has_pts = mem[1] & UVC_STREAM_PTS;
> > > +	has_scr = mem[1] & UVC_STREAM_SCR;
> > > +
> > > +	if (has_pts) {
> > > +		len_std += 4;
> > > +		scr = mem + 6;
> > > +	} else {
> > > +		scr = mem + 2;
> > > +	}
> > > +
> > > +	if (has_scr)
> > > +		len_std += 6;
> > > +
> > > +	if (stream->meta.format == V4L2_META_FMT_UVC)
> > > +		length = len_std;
> > > +
> > > +	if (length == len_std && (!has_scr ||
> > > +				  !memcmp(scr, stream->clock.last_scr, 6)))
> > > +		return;
> > > +
> > > +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> > > meta_buf->bytesused); +	local_irq_save(flags);
> > > +	time = uvc_video_get_time();
> > > +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > 
> > You need a put_unaligned here too. If you're fine with the patch below
> > there's no need to resubmit, and
> 
> One more thing, __put_unaligned_cpu16() and __put_unaligned_cpu64() don't 
> compile on x86_64 with v4.12 (using media_build.git). I propose replacing them 
> with put_unaligned() which compiles and should do the right thing.

Sure, thanks for catching! Shall I fix and resubmit?

> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > 
> > Would you mind sending me your test tool patch ?

Will send it offline.

Thanks
Guennadi

> > 
> > diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/
> > uvc_video.c
> > index 2fc0bf2221db..02e4997a32bb 100644
> > --- a/drivers/media/usb/uvc/uvc_video.c
> > +++ b/drivers/media/usb/uvc/uvc_video.c
> > @@ -1142,6 +1142,7 @@ static void uvc_video_decode_meta(struct uvc_streaming
> > *stream,
> >  	size_t len_std = 2;
> >  	bool has_pts, has_scr;
> >  	unsigned long flags;
> > +	unsigned int sof;
> >  	ktime_t time;
> >  	const u8 *scr;
> > 
> > @@ -1177,9 +1178,10 @@ static void uvc_video_decode_meta(struct
> > uvc_streaming *stream,
> >  	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem + meta_buf->bytesused);
> > local_irq_save(flags);
> >  	time = uvc_video_get_time();
> > -	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > +	sof = usb_get_current_frame_number(stream->dev->udev);
> >  	local_irq_restore(flags);
> >  	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> > +	__put_unaligned_cpu16(sof, &meta->sof);
> > 
> >  	if (has_scr)
> >  		memcpy(stream->clock.last_scr, scr, 6);
> > 
> > > +	local_irq_restore(flags);
> > > +	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> > > +
> > > +	if (has_scr)
> > > +		memcpy(stream->clock.last_scr, scr, 6);
> > > +
> > > +	memcpy(&meta->length, mem, length);
> > > +	meta_buf->bytesused += length + sizeof(meta->ns) + sizeof(meta->sof);
> > > +
> > > +	uvc_trace(UVC_TRACE_FRAME,
> > > +		  "%s(): t-sys %lluns, SOF %u, len %u, flags 0x%x, PTS %u, STC %u 
> frame
> > > SOF %u\n", +		  __func__, time, meta->sof, meta->length, meta->flags,
> > > +		  has_pts ? *(u32 *)meta->buf : 0,
> > > +		  has_scr ? *(u32 *)scr : 0,
> > > +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> > > +}
> > 
> > [snip]
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

Re: [PATCH v8] uvcvideo: Add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Monday, 11 December 2017 23:44:09 EET Guennadi Liakhovetski wrote:
> On Mon, 11 Dec 2017, Laurent Pinchart wrote:
> > On Monday, 11 December 2017 22:16:23 EET Laurent Pinchart wrote:
> >> On Wednesday, 6 December 2017 17:15:40 EET Guennadi Liakhovetski wrote:
> >>> From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> >>> 
> >>> Some UVC video cameras contain metadata in their payload headers. This
> >>> patch extracts that data, adding more clock synchronisation
> >>> information, on both bulk and isochronous endpoints and makes it
> >>> available to the user space on a separate video node, using the
> >>> V4L2_CAP_META_CAPTURE capability and the V4L2_BUF_TYPE_META_CAPTURE
> >>> buffer queue type. By default, only the V4L2_META_FMT_UVC pixel format
> >>> is available from those nodes. However, cameras can be added to the
> >>> device ID table to additionally specify their own metadata format, in
> >>> which case that format will also become available from the metadata
> >>> node.
> >>> 
> >>> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> >>> ---
> >>> 
> >>> v8: addressed comments and integrated changes from Laurent, thanks
> >>> again, e.g.:
> >>> 
> >>> - multiple stylistic changes
> >>> - remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now
> >>>   created unconditionally
> >>> - reuse uvc_ioctl_querycap()
> >>> - reuse code in uvc_register_video()
> >>> - set an error flag when the metadata buffer overflows
> >>> 
> >>>  drivers/media/usb/uvc/Makefile       |   2 +-
> >>>  drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
> >>>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> >>>  drivers/media/usb/uvc/uvc_metadata.c | 179 ++++++++++++++++++++++++++++
> >>>  drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
> >>>  drivers/media/usb/uvc/uvc_video.c    | 132 ++++++++++++++++++++++++--
> >>>  drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
> >>>  include/uapi/linux/uvcvideo.h        |  26 +++++
> >>>  8 files changed, 394 insertions(+), 22 deletions(-)
> >>>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> >> 
> >> [snip]
> >> 
> >> > diff --git a/drivers/media/usb/uvc/uvc_video.c
> >> > b/drivers/media/usb/uvc/uvc_video.c index 13f459e..2fc0bf2 100644
> >> > --- a/drivers/media/usb/uvc/uvc_video.c
> >> > +++ b/drivers/media/usb/uvc/uvc_video.c
> >> 
> >> [snip]
> >> 
> >>> +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> >>> +				  struct uvc_buffer *meta_buf,
> >>> +				  const u8 *mem, unsigned int length)
> >>> +{
> >>> +	struct uvc_meta_buf *meta;
> >>> +	size_t len_std = 2;
> >>> +	bool has_pts, has_scr;
> >>> +	unsigned long flags;
> >>> +	ktime_t time;
> >>> +	const u8 *scr;
> >>> +
> >>> +	if (!meta_buf || length == 2)
> >>> +		return;
> >>> +
> >>> +	if (meta_buf->length - meta_buf->bytesused <
> >>> +	    length + sizeof(meta->ns) + sizeof(meta->sof)) {
> >>> +		meta_buf->error = 1;
> >>> +		return;
> >>> +	}
> >>> +
> >>> +	has_pts = mem[1] & UVC_STREAM_PTS;
> >>> +	has_scr = mem[1] & UVC_STREAM_SCR;
> >>> +
> >>> +	if (has_pts) {
> >>> +		len_std += 4;
> >>> +		scr = mem + 6;
> >>> +	} else {
> >>> +		scr = mem + 2;
> >>> +	}
> >>> +
> >>> +	if (has_scr)
> >>> +		len_std += 6;
> >>> +
> >>> +	if (stream->meta.format == V4L2_META_FMT_UVC)
> >>> +		length = len_std;
> >>> +
> >>> +	if (length == len_std && (!has_scr ||
> >>> +				  !memcmp(scr, stream->clock.last_scr, 6)))
> >>> +		return;
> >>> +
> >>> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> >>> meta_buf->bytesused); +	local_irq_save(flags);
> >>> +	time = uvc_video_get_time();
> >>> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> >> 
> >> You need a put_unaligned here too. If you're fine with the patch below
> >> there's no need to resubmit, and
> > 
> > One more thing, __put_unaligned_cpu16() and __put_unaligned_cpu64() don't
> > compile on x86_64 with v4.12 (using media_build.git). I propose replacing
> > them with put_unaligned() which compiles and should do the right thing.
> Sure, thanks for catching! Shall I fix and resubmit?

If you're fine with

	git://linuxtv.org/pinchartl/media.git uvc/next

there's no need to resubmit.

By the way, could you please review "uvcvideo: Factor out video device 
registration to a function" and "uvcvideo: Report V4L2 device caps through the 
video_device structure" ?

I will send a pull request after testing the code and getting those two 
patches reviewed.

> >> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >> 
> >> Would you mind sending me your test tool patch ?
> 
> Will send it offline.
> 
> Thanks
> Guennadi
> 
> >> diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/
> >> uvc_video.c
> >> index 2fc0bf2221db..02e4997a32bb 100644
> >> --- a/drivers/media/usb/uvc/uvc_video.c
> >> +++ b/drivers/media/usb/uvc/uvc_video.c
> >> @@ -1142,6 +1142,7 @@ static void uvc_video_decode_meta(struct
> >> uvc_streaming *stream,
> >>  	size_t len_std = 2;
> >>  	bool has_pts, has_scr;
> >>  	unsigned long flags;
> >> +	unsigned int sof;
> >>  	ktime_t time;
> >>  	const u8 *scr;
> >> 
> >> @@ -1177,9 +1178,10 @@ static void uvc_video_decode_meta(struct
> >> uvc_streaming *stream,
> >>  	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> >>  	meta_buf->bytesused);
> >> 
> >>  local_irq_save(flags);
> >>  	time = uvc_video_get_time();
> >> -	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> >> +	sof = usb_get_current_frame_number(stream->dev->udev);
> >>  	local_irq_restore(flags);
> >>  	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> >> +	__put_unaligned_cpu16(sof, &meta->sof);
> >> 
> >>  	if (has_scr)
> >>  		memcpy(stream->clock.last_scr, scr, 6);
> >>
> >>> +	local_irq_restore(flags);
> >>> +	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> >>> +
> >>> +	if (has_scr)
> >>> +		memcpy(stream->clock.last_scr, scr, 6);
> >>> +
> >>> +	memcpy(&meta->length, mem, length);
> >>> +	meta_buf->bytesused += length + sizeof(meta->ns) +
> >>> sizeof(meta->sof);
> >>> +
> >>> +	uvc_trace(UVC_TRACE_FRAME,
> >>> +		  "%s(): t-sys %lluns, SOF %u, len %u, flags 0x%x, PTS %u, STC %u
> >>> frame
> >>> SOF %u\n", +		  __func__, time, meta->sof, meta->length, meta-
>flags,
> >>> +		  has_pts ? *(u32 *)meta->buf : 0,
> >>> +		  has_scr ? *(u32 *)scr : 0,
> >>> +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> >>> +}
> >> 
> >> [snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v8] uvcvideo: Add a metadata device node

[Guennadi Liakhovetski]

Hi Laurent,

On Mon, 11 Dec 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> On Monday, 11 December 2017 23:44:09 EET Guennadi Liakhovetski wrote:
> > On Mon, 11 Dec 2017, Laurent Pinchart wrote:
> > > On Monday, 11 December 2017 22:16:23 EET Laurent Pinchart wrote:
> > >> On Wednesday, 6 December 2017 17:15:40 EET Guennadi Liakhovetski wrote:
> > >>> From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > >>> 
> > >>> Some UVC video cameras contain metadata in their payload headers. This
> > >>> patch extracts that data, adding more clock synchronisation
> > >>> information, on both bulk and isochronous endpoints and makes it
> > >>> available to the user space on a separate video node, using the
> > >>> V4L2_CAP_META_CAPTURE capability and the V4L2_BUF_TYPE_META_CAPTURE
> > >>> buffer queue type. By default, only the V4L2_META_FMT_UVC pixel format
> > >>> is available from those nodes. However, cameras can be added to the
> > >>> device ID table to additionally specify their own metadata format, in
> > >>> which case that format will also become available from the metadata
> > >>> node.
> > >>> 
> > >>> Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > >>> ---
> > >>> 
> > >>> v8: addressed comments and integrated changes from Laurent, thanks
> > >>> again, e.g.:
> > >>> 
> > >>> - multiple stylistic changes
> > >>> - remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now
> > >>>   created unconditionally
> > >>> - reuse uvc_ioctl_querycap()
> > >>> - reuse code in uvc_register_video()
> > >>> - set an error flag when the metadata buffer overflows
> > >>> 
> > >>>  drivers/media/usb/uvc/Makefile       |   2 +-
> > >>>  drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
> > >>>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> > >>>  drivers/media/usb/uvc/uvc_metadata.c | 179 ++++++++++++++++++++++++++++
> > >>>  drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
> > >>>  drivers/media/usb/uvc/uvc_video.c    | 132 ++++++++++++++++++++++++--
> > >>>  drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
> > >>>  include/uapi/linux/uvcvideo.h        |  26 +++++
> > >>>  8 files changed, 394 insertions(+), 22 deletions(-)
> > >>>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> > >> 
> > >> [snip]
> > >> 
> > >> > diff --git a/drivers/media/usb/uvc/uvc_video.c
> > >> > b/drivers/media/usb/uvc/uvc_video.c index 13f459e..2fc0bf2 100644
> > >> > --- a/drivers/media/usb/uvc/uvc_video.c
> > >> > +++ b/drivers/media/usb/uvc/uvc_video.c
> > >> 
> > >> [snip]
> > >> 
> > >>> +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> > >>> +				  struct uvc_buffer *meta_buf,
> > >>> +				  const u8 *mem, unsigned int length)
> > >>> +{
> > >>> +	struct uvc_meta_buf *meta;
> > >>> +	size_t len_std = 2;
> > >>> +	bool has_pts, has_scr;
> > >>> +	unsigned long flags;
> > >>> +	ktime_t time;
> > >>> +	const u8 *scr;
> > >>> +
> > >>> +	if (!meta_buf || length == 2)
> > >>> +		return;
> > >>> +
> > >>> +	if (meta_buf->length - meta_buf->bytesused <
> > >>> +	    length + sizeof(meta->ns) + sizeof(meta->sof)) {
> > >>> +		meta_buf->error = 1;
> > >>> +		return;
> > >>> +	}
> > >>> +
> > >>> +	has_pts = mem[1] & UVC_STREAM_PTS;
> > >>> +	has_scr = mem[1] & UVC_STREAM_SCR;
> > >>> +
> > >>> +	if (has_pts) {
> > >>> +		len_std += 4;
> > >>> +		scr = mem + 6;
> > >>> +	} else {
> > >>> +		scr = mem + 2;
> > >>> +	}
> > >>> +
> > >>> +	if (has_scr)
> > >>> +		len_std += 6;
> > >>> +
> > >>> +	if (stream->meta.format == V4L2_META_FMT_UVC)
> > >>> +		length = len_std;
> > >>> +
> > >>> +	if (length == len_std && (!has_scr ||
> > >>> +				  !memcmp(scr, stream->clock.last_scr, 6)))
> > >>> +		return;
> > >>> +
> > >>> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> > >>> meta_buf->bytesused); +	local_irq_save(flags);
> > >>> +	time = uvc_video_get_time();
> > >>> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > >> 
> > >> You need a put_unaligned here too. If you're fine with the patch below
> > >> there's no need to resubmit, and
> > > 
> > > One more thing, __put_unaligned_cpu16() and __put_unaligned_cpu64() don't
> > > compile on x86_64 with v4.12 (using media_build.git). I propose replacing
> > > them with put_unaligned() which compiles and should do the right thing.
> > Sure, thanks for catching! Shall I fix and resubmit?
> 
> If you're fine with
> 
> 	git://linuxtv.org/pinchartl/media.git uvc/next

I was a bit concerned about just using "int" for unaligned writing of a 
16-bit value, but looking at definitions, after a couple of macros 
put_unaligned() currently resolves to one inline functions, which should 
make that safe. However, at least theoretically, an arch could decide to 
implement put_unaligned() as a macro, which might turn out to be unsafe 
for this... Not sure how concerned should we be about such a possibility 
:-) If you think, that's fine, then I'm ok with using the version from 
that your branch.

Thanks
Guennadi

> 
> there's no need to resubmit.
> 
> By the way, could you please review "uvcvideo: Factor out video device 
> registration to a function" and "uvcvideo: Report V4L2 device caps through the 
> video_device structure" ?
> 
> I will send a pull request after testing the code and getting those two 
> patches reviewed.
> 
> > >> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > >> 
> > >> Would you mind sending me your test tool patch ?
> > 
> > Will send it offline.
> > 
> > Thanks
> > Guennadi
> > 
> > >> diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/
> > >> uvc_video.c
> > >> index 2fc0bf2221db..02e4997a32bb 100644
> > >> --- a/drivers/media/usb/uvc/uvc_video.c
> > >> +++ b/drivers/media/usb/uvc/uvc_video.c
> > >> @@ -1142,6 +1142,7 @@ static void uvc_video_decode_meta(struct
> > >> uvc_streaming *stream,
> > >>  	size_t len_std = 2;
> > >>  	bool has_pts, has_scr;
> > >>  	unsigned long flags;
> > >> +	unsigned int sof;
> > >>  	ktime_t time;
> > >>  	const u8 *scr;
> > >> 
> > >> @@ -1177,9 +1178,10 @@ static void uvc_video_decode_meta(struct
> > >> uvc_streaming *stream,
> > >>  	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> > >>  	meta_buf->bytesused);
> > >> 
> > >>  local_irq_save(flags);
> > >>  	time = uvc_video_get_time();
> > >> -	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > >> +	sof = usb_get_current_frame_number(stream->dev->udev);
> > >>  	local_irq_restore(flags);
> > >>  	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> > >> +	__put_unaligned_cpu16(sof, &meta->sof);
> > >> 
> > >>  	if (has_scr)
> > >>  		memcpy(stream->clock.last_scr, scr, 6);
> > >>
> > >>> +	local_irq_restore(flags);
> > >>> +	__put_unaligned_cpu64(ktime_to_ns(time), &meta->ns);
> > >>> +
> > >>> +	if (has_scr)
> > >>> +		memcpy(stream->clock.last_scr, scr, 6);
> > >>> +
> > >>> +	memcpy(&meta->length, mem, length);
> > >>> +	meta_buf->bytesused += length + sizeof(meta->ns) +
> > >>> sizeof(meta->sof);
> > >>> +
> > >>> +	uvc_trace(UVC_TRACE_FRAME,
> > >>> +		  "%s(): t-sys %lluns, SOF %u, len %u, flags 0x%x, PTS %u, STC %u
> > >>> frame
> > >>> SOF %u\n", +		  __func__, time, meta->sof, meta->length, meta-
> >flags,
> > >>> +		  has_pts ? *(u32 *)meta->buf : 0,
> > >>> +		  has_scr ? *(u32 *)scr : 0,
> > >>> +		  has_scr ? *(u32 *)(scr + 4) & 0x7ff : 0);
> > >>> +}
> > >> 
> > >> [snip]
> 
> -- 
> Regards,
> 
> Laurent Pinchart
> 

Re: [PATCH v8] uvcvideo: Add a metadata device node

[Laurent Pinchart]

Hi Guennadi,

On Tuesday, 12 December 2017 10:30:39 EET Guennadi Liakhovetski wrote:
> On Mon, 11 Dec 2017, Laurent Pinchart wrote:
> > On Monday, 11 December 2017 23:44:09 EET Guennadi Liakhovetski wrote:
> >> On Mon, 11 Dec 2017, Laurent Pinchart wrote:
> >>> On Monday, 11 December 2017 22:16:23 EET Laurent Pinchart wrote:
> >>>> On Wednesday, 6 December 2017 17:15:40 EET Guennadi Liakhovetski wrote:
> >>>>> From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> >>>>> 
> >>>>> Some UVC video cameras contain metadata in their payload headers.
> >>>>> This patch extracts that data, adding more clock synchronisation
> >>>>> information, on both bulk and isochronous endpoints and makes it
> >>>>> available to the user space on a separate video node, using the
> >>>>> V4L2_CAP_META_CAPTURE capability and the V4L2_BUF_TYPE_META_CAPTURE
> >>>>> buffer queue type. By default, only the V4L2_META_FMT_UVC pixel
> >>>>> format is available from those nodes. However, cameras can be added to
> >>>>> the device ID table to additionally specify their own metadata format,
> >>>>> in which case that format will also become available from the metadata
> >>>>> node.
> >>>>> 
> >>>>> Signed-off-by: Guennadi Liakhovetski
> >>>>> <guennadi.liakhovetski@intel.com>
> >>>>> ---
> >>>>> 
> >>>>> v8: addressed comments and integrated changes from Laurent, thanks
> >>>>> again, e.g.:
> >>>>> 
> >>>>> - multiple stylistic changes
> >>>>> - remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now
> >>>>>   created unconditionally
> >>>>> - reuse uvc_ioctl_querycap()
> >>>>> - reuse code in uvc_register_video()
> >>>>> - set an error flag when the metadata buffer overflows
> >>>>> 
> >>>>>  drivers/media/usb/uvc/Makefile       |   2 +-
> >>>>>  drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
> >>>>>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> >>>>>  drivers/media/usb/uvc/uvc_metadata.c | 179+++++++++++++++++++++++++++
> >>>>>  drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
> >>>>>  drivers/media/usb/uvc/uvc_video.c    | 132 +++++++++++++++++++++++--
> >>>>>  drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
> >>>>>  include/uapi/linux/uvcvideo.h        |  26 +++++
> >>>>>  8 files changed, 394 insertions(+), 22 deletions(-)
> >>>>>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> >>>> 
> >>>> [snip]
> >>>> 
> >>>>> diff --git a/drivers/media/usb/uvc/uvc_video.c
> >>>>> b/drivers/media/usb/uvc/uvc_video.c index 13f459e..2fc0bf2 100644
> >>>>> --- a/drivers/media/usb/uvc/uvc_video.c
> >>>>> +++ b/drivers/media/usb/uvc/uvc_video.c
> >>>> 
> >>>> [snip]
> >>>> 
> >>>>> +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> >>>>> +				  struct uvc_buffer *meta_buf,
> >>>>> +				  const u8 *mem, unsigned int length)
> >>>>> +{
> >>>>> +	struct uvc_meta_buf *meta;
> >>>>> +	size_t len_std = 2;
> >>>>> +	bool has_pts, has_scr;
> >>>>> +	unsigned long flags;
> >>>>> +	ktime_t time;
> >>>>> +	const u8 *scr;

[snip]

> >>>>> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> >>>>> meta_buf->bytesused);
> >>>>> +	local_irq_save(flags);
> >>>>> +	time = uvc_video_get_time();
> >>>>> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> >>>> 
> >>>> You need a put_unaligned here too. If you're fine with the patch
> >>>> below there's no need to resubmit, and
> >>> 
> >>> One more thing, __put_unaligned_cpu16() and __put_unaligned_cpu64()
> >>> don't compile on x86_64 with v4.12 (using media_build.git). I propose
> >>> replacing them with put_unaligned() which compiles and should do the
> >>> right thing.
> >> 
> >> Sure, thanks for catching! Shall I fix and resubmit?
> > 
> > If you're fine with
> > 
> > 	git://linuxtv.org/pinchartl/media.git uvc/next
> 
> I was a bit concerned about just using "int" for unaligned writing of a
> 16-bit value, but looking at definitions, after a couple of macros
> put_unaligned() currently resolves to one inline functions, which should
> make that safe. However, at least theoretically, an arch could decide to
> implement put_unaligned() as a macro, which might turn out to be unsafe
> for this... Not sure how concerned should we be about such a possibility
> :-) If you think, that's fine, then I'm ok with using the version from
> that your branch.

Why do you think that would be unsafe ? The return type of 
usb_get_current_frame_number() is int, so introducing an intermediate int 
variable should at least not make things worse. If put_unaligned() is 
implemented solely using macros I would still expect them to operate on the 
type of the destination operand, and cast the source value appropriately.

[snip]

-- 
Regards,

Laurent Pinchart

Re: [PATCH v8] uvcvideo: Add a metadata device node

[Guennadi Liakhovetski]

Hi Laurent,

On Wed, 13 Dec 2017, Laurent Pinchart wrote:

> Hi Guennadi,
> 
> On Tuesday, 12 December 2017 10:30:39 EET Guennadi Liakhovetski wrote:
> > On Mon, 11 Dec 2017, Laurent Pinchart wrote:
> > > On Monday, 11 December 2017 23:44:09 EET Guennadi Liakhovetski wrote:
> > >> On Mon, 11 Dec 2017, Laurent Pinchart wrote:
> > >>> On Monday, 11 December 2017 22:16:23 EET Laurent Pinchart wrote:
> > >>>> On Wednesday, 6 December 2017 17:15:40 EET Guennadi Liakhovetski wrote:
> > >>>>> From: Guennadi Liakhovetski <guennadi.liakhovetski@intel.com>
> > >>>>> 
> > >>>>> Some UVC video cameras contain metadata in their payload headers.
> > >>>>> This patch extracts that data, adding more clock synchronisation
> > >>>>> information, on both bulk and isochronous endpoints and makes it
> > >>>>> available to the user space on a separate video node, using the
> > >>>>> V4L2_CAP_META_CAPTURE capability and the V4L2_BUF_TYPE_META_CAPTURE
> > >>>>> buffer queue type. By default, only the V4L2_META_FMT_UVC pixel
> > >>>>> format is available from those nodes. However, cameras can be added to
> > >>>>> the device ID table to additionally specify their own metadata format,
> > >>>>> in which case that format will also become available from the metadata
> > >>>>> node.
> > >>>>> 
> > >>>>> Signed-off-by: Guennadi Liakhovetski
> > >>>>> <guennadi.liakhovetski@intel.com>
> > >>>>> ---
> > >>>>> 
> > >>>>> v8: addressed comments and integrated changes from Laurent, thanks
> > >>>>> again, e.g.:
> > >>>>> 
> > >>>>> - multiple stylistic changes
> > >>>>> - remove the UVC_DEV_FLAG_METADATA_NODE flag / quirk: nodes are now
> > >>>>>   created unconditionally
> > >>>>> - reuse uvc_ioctl_querycap()
> > >>>>> - reuse code in uvc_register_video()
> > >>>>> - set an error flag when the metadata buffer overflows
> > >>>>> 
> > >>>>>  drivers/media/usb/uvc/Makefile       |   2 +-
> > >>>>>  drivers/media/usb/uvc/uvc_driver.c   |  15 ++-
> > >>>>>  drivers/media/usb/uvc/uvc_isight.c   |   2 +-
> > >>>>>  drivers/media/usb/uvc/uvc_metadata.c | 179+++++++++++++++++++++++++++
> > >>>>>  drivers/media/usb/uvc/uvc_queue.c    |  44 +++++++--
> > >>>>>  drivers/media/usb/uvc/uvc_video.c    | 132 +++++++++++++++++++++++--
> > >>>>>  drivers/media/usb/uvc/uvcvideo.h     |  16 +++-
> > >>>>>  include/uapi/linux/uvcvideo.h        |  26 +++++
> > >>>>>  8 files changed, 394 insertions(+), 22 deletions(-)
> > >>>>>  create mode 100644 drivers/media/usb/uvc/uvc_metadata.c
> > >>>> 
> > >>>> [snip]
> > >>>> 
> > >>>>> diff --git a/drivers/media/usb/uvc/uvc_video.c
> > >>>>> b/drivers/media/usb/uvc/uvc_video.c index 13f459e..2fc0bf2 100644
> > >>>>> --- a/drivers/media/usb/uvc/uvc_video.c
> > >>>>> +++ b/drivers/media/usb/uvc/uvc_video.c
> > >>>> 
> > >>>> [snip]
> > >>>> 
> > >>>>> +static void uvc_video_decode_meta(struct uvc_streaming *stream,
> > >>>>> +				  struct uvc_buffer *meta_buf,
> > >>>>> +				  const u8 *mem, unsigned int length)
> > >>>>> +{
> > >>>>> +	struct uvc_meta_buf *meta;
> > >>>>> +	size_t len_std = 2;
> > >>>>> +	bool has_pts, has_scr;
> > >>>>> +	unsigned long flags;
> > >>>>> +	ktime_t time;
> > >>>>> +	const u8 *scr;
> 
> [snip]
> 
> > >>>>> +	meta = (struct uvc_meta_buf *)((u8 *)meta_buf->mem +
> > >>>>> meta_buf->bytesused);
> > >>>>> +	local_irq_save(flags);
> > >>>>> +	time = uvc_video_get_time();
> > >>>>> +	meta->sof = usb_get_current_frame_number(stream->dev->udev);
> > >>>> 
> > >>>> You need a put_unaligned here too. If you're fine with the patch
> > >>>> below there's no need to resubmit, and
> > >>> 
> > >>> One more thing, __put_unaligned_cpu16() and __put_unaligned_cpu64()
> > >>> don't compile on x86_64 with v4.12 (using media_build.git). I propose
> > >>> replacing them with put_unaligned() which compiles and should do the
> > >>> right thing.
> > >> 
> > >> Sure, thanks for catching! Shall I fix and resubmit?
> > > 
> > > If you're fine with
> > > 
> > > 	git://linuxtv.org/pinchartl/media.git uvc/next
> > 
> > I was a bit concerned about just using "int" for unaligned writing of a
> > 16-bit value, but looking at definitions, after a couple of macros
> > put_unaligned() currently resolves to one inline functions, which should
> > make that safe. However, at least theoretically, an arch could decide to
> > implement put_unaligned() as a macro, which might turn out to be unsafe
> > for this... Not sure how concerned should we be about such a possibility
> > :-) If you think, that's fine, then I'm ok with using the version from
> > that your branch.
> 
> Why do you think that would be unsafe ? The return type of 
> usb_get_current_frame_number() is int, so introducing an intermediate int 
> variable should at least not make things worse. If put_unaligned() is 
> implemented solely using macros I would still expect them to operate on the 
> type of the destination operand, and cast the source value appropriately.

Well, you *can* implement that badly - check the destination type, which 
in this case is 16 bit, and use the address of the first argument to get 
16 bits from there... But yes, I agree, that would be a bug, so, maybe 
it's ok to rely on those implementations being bug-free.

Thanks
Guennadi