[PATCH v1 00/19] UVC 1.5 VP8 support for uvcvideo

[PATCH v1 00/19] UVC 1.5 VP8 support for uvcvideo

[Pawel Osciak]

Hello everyone,

This series adds support for UVC 1.5 and VP8 encoding cameras to the uvcvideo
driver. The official specification for the new standard can be found here:
http://www.usb.org/developers/devclass_docs.

The main change in 1.5 is support for encoding cameras. Those cameras contain
additional UVC entities, called Encoding Units, with their own set of controls
governing encode parameters. Typical encoding cameras (see examples in class
spec) expose two USB Video Streaming Interfaces (VSIs): one for raw stream
formats and one for encoded streams. Typically, both get their source stream
from a single sensor, producing raw and encoded versions of the video feed
simultaneously.
Encoding Units may also support the so-called "simulcast" formats, which allow
additional sub-streams, or layers, used to achieve temporal scalability.
The spec allows up to 4 simulcast layers. Those layers are encoded in the same
format, but encoding parameters, such as resolution, bitrate, etc., may,
depending on the camera capabilities, be changed independently for each layer,
and their streaming state may also be controlled independently as well. The
layers are streamed from the same USB VSI, and the information which layer
a frame belongs to is contained in its payload header.

In V4L2 API, a separate video node is created for each VSI: one for raw formats
VSI and another for the encoded formats VSI. Both can operate completely
independently from each other. In addition, if the Encoding Unit supports
simulcast, one V4L2 node is created for each stream layer instead, and each
can be controlled independently, including streamon/streamoff state, setting
resolution and controls. Once a simulcast format is successfully set for one
of the simulcast video nodes however, it cannot be changed, unless all connected
nodes are idle, i.e. they are not streaming and their buffers are freed.

The userspace can discover if a set of nodes belongs to one encoding unit
by traversing media controller topology of the camera.


I will be gradually posting documentation changes for new features after initial
rounds of reviews. This is a relatively major change to the UVC driver and
although I tried to keep the existing logic for UVC <1.5 cameras intact as much
as possible, I would very much appreciate it if these patches could get some
testing from you as well, on your own devices/systems.

Thanks,
Pawel Osciak


Pawel Osciak (19):
      uvcvideo: Add UVC query tracing.
      uvcvideo: Return 0 when setting probe control succeeds.
      uvcvideo: Add support for multiple chains with common roots.
      uvcvideo: Create separate debugfs entries for each streaming interface.
      uvcvideo: Add support for UVC1.5 P&C control.
      uvcvideo: Recognize UVC 1.5 encoding units.
      uvcvideo: Unify error reporting during format descriptor parsing.
      uvcvideo: Add UVC1.5 VP8 format support.
      uvcvideo: Reorganize uvc_{get,set}_le_value.
      uvcvideo: Support UVC 1.5 runtime control property.
      uvcvideo: Support V4L2_CTRL_TYPE_BITMASK controls.
      uvcvideo: Reorganize next buffer handling.
      uvcvideo: Unify UVC payload header parsing.
      v4l: Add v4l2_buffer flags for VP8-specific special frames.
      uvcvideo: Add support for VP8 special frame flags.
      v4l: Add encoding camera controls.
      uvcvideo: Add UVC 1.5 Encoding Unit controls.
      v4l: Add V4L2_PIX_FMT_VP8_SIMULCAST format.
      uvcvideo: Add support for UVC 1.5 VP8 simulcast.

 drivers/media/usb/uvc/uvc_ctrl.c     | 960 ++++++++++++++++++++++++++++++++---
 drivers/media/usb/uvc/uvc_debugfs.c  |   3 +-
 drivers/media/usb/uvc/uvc_driver.c   | 604 ++++++++++++++--------
 drivers/media/usb/uvc/uvc_entity.c   | 129 ++++-
 drivers/media/usb/uvc/uvc_isight.c   |  12 +-
 drivers/media/usb/uvc/uvc_queue.c    |  25 +-
 drivers/media/usb/uvc/uvc_v4l2.c     | 284 +++++++++--
 drivers/media/usb/uvc/uvc_video.c    | 704 ++++++++++++++++---------
 drivers/media/usb/uvc/uvcvideo.h     | 214 +++++++-
 drivers/media/v4l2-core/v4l2-ctrls.c |  29 ++
 include/uapi/linux/usb/video.h       |  45 ++
 include/uapi/linux/v4l2-controls.h   |  31 ++
 include/uapi/linux/videodev2.h       |   8 +
 13 files changed, 2421 insertions(+), 627 deletions(-)

[PATCH v1 01/19] uvcvideo: Add UVC query tracing.

[Pawel Osciak]

Add a new trace argument enabling UVC query details and contents logging.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_video.c | 45 +++++++++++++++++++++++++--------------
 drivers/media/usb/uvc/uvcvideo.h  |  9 ++++++++
 2 files changed, 38 insertions(+), 16 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 3394c34..695f6d9 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -29,22 +29,6 @@
 /* ------------------------------------------------------------------------
  * UVC Controls
  */
-
-static int __uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
-			__u8 intfnum, __u8 cs, void *data, __u16 size,
-			int timeout)
-{
-	__u8 type = USB_TYPE_CLASS | USB_RECIP_INTERFACE;
-	unsigned int pipe;
-
-	pipe = (query & 0x80) ? usb_rcvctrlpipe(dev->udev, 0)
-			      : usb_sndctrlpipe(dev->udev, 0);
-	type |= (query & 0x80) ? USB_DIR_IN : USB_DIR_OUT;
-
-	return usb_control_msg(dev->udev, pipe, query, type, cs << 8,
-			unit << 8 | intfnum, data, size, timeout);
-}
-
 static const char *uvc_query_name(__u8 query)
 {
 	switch (query) {
@@ -69,6 +53,35 @@ static const char *uvc_query_name(__u8 query)
 	}
 }
 
+static int __uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
+			__u8 intfnum, __u8 cs, void *data, __u16 size,
+			int timeout)
+{
+	__u8 type = USB_TYPE_CLASS | USB_RECIP_INTERFACE;
+	unsigned int pipe;
+	int ret;
+
+	pipe = (query & 0x80) ? usb_rcvctrlpipe(dev->udev, 0)
+			      : usb_sndctrlpipe(dev->udev, 0);
+	type |= (query & 0x80) ? USB_DIR_IN : USB_DIR_OUT;
+
+	uvc_trace(UVC_TRACE_QUERY,
+			"%s (%d): size=%d, unit=%d, cs=%d, intf=%d\n",
+			uvc_query_name(query), query, size, unit, cs, intfnum);
+	uvc_trace(UVC_TRACE_QUERY, "Sent:\n");
+	uvc_print_hex_dump(UVC_TRACE_QUERY, data, size);
+
+	ret = usb_control_msg(dev->udev, pipe, query, type, cs << 8,
+			unit << 8 | intfnum, data, size, timeout);
+	if (ret == -EPIPE)
+		uvc_trace(UVC_TRACE_QUERY, "Got device STALL on query!\n");
+
+	uvc_trace(UVC_TRACE_QUERY, "Received:\n");
+	uvc_print_hex_dump(UVC_TRACE_QUERY, data, size);
+
+	return ret;
+}
+
 int uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
 			__u8 intfnum, __u8 cs, void *data, __u16 size)
 {
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 9e35982..75e0153 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -574,6 +574,7 @@ struct uvc_driver {
 #define UVC_TRACE_VIDEO		(1 << 10)
 #define UVC_TRACE_STATS		(1 << 11)
 #define UVC_TRACE_CLOCK		(1 << 12)
+#define UVC_TRACE_QUERY		(1 << 13)
 
 #define UVC_WARN_MINMAX		0
 #define UVC_WARN_PROBE_DEF	1
@@ -599,6 +600,14 @@ extern unsigned int uvc_timeout_param;
 #define uvc_printk(level, msg...) \
 	printk(level "uvcvideo: " msg)
 
+#define uvc_print_hex_dump(flag, buf, len) \
+	do { \
+		if (uvc_trace_param & flag) { \
+			print_hex_dump(KERN_DEBUG, "uvcvideo: ", \
+				DUMP_PREFIX_NONE, 16, 1, buf, len, false); \
+		} \
+	} while (0)
+
 /* --------------------------------------------------------------------------
  * Internal functions.
  */
-- 
1.8.4

[PATCH v1 02/19] uvcvideo: Return 0 when setting probe control succeeds.

[Pawel Osciak]

Return 0 instead of returning size of the probe control on successful set.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_video.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 695f6d9..1198989 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -296,6 +296,8 @@ static int uvc_set_video_ctrl(struct uvc_streaming *stream,
 			"%d (exp. %u).\n", probe ? "probe" : "commit",
 			ret, size);
 		ret = -EIO;
+	} else {
+		ret = 0;
 	}
 
 	kfree(data);
-- 
1.8.4

[PATCH v1 03/19] uvcvideo: Add support for multiple chains with common roots.

[Pawel Osciak]

This adds support for pipelines that fork into branches consisting of more
than one entity, creating a chain for each fork and putting common entities
on all chains that share them.

This requires us to share the ctrl_mutex across forked chains. Whenever we
discover a shared part of a chain, we assign the pointer to an existing
mutex to the sharing chain instead of creating a new one.

The "forward scan" is not needed anymore, as after scanning back from OTs,
we go over all entities which are not on a path from an OT and accept
single-XU branches, adding them to the existing chains.

Also extract control locking into __uvc_ctrl_{lock,unlock} functions.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_ctrl.c   |  70 ++++++++-----
 drivers/media/usb/uvc/uvc_driver.c | 210 +++++++++++++++++++++----------------
 drivers/media/usb/uvc/uvc_entity.c |  15 ++-
 drivers/media/usb/uvc/uvc_v4l2.c   |   9 +-
 drivers/media/usb/uvc/uvcvideo.h   |  20 +++-
 5 files changed, 199 insertions(+), 125 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index a2f4501..ba159a4 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -841,6 +841,7 @@ static struct uvc_control *uvc_find_control(struct uvc_video_chain *chain,
 {
 	struct uvc_control *ctrl = NULL;
 	struct uvc_entity *entity;
+	struct uvc_chain_entry *entry;
 	int next = v4l2_id & V4L2_CTRL_FLAG_NEXT_CTRL;
 
 	*mapping = NULL;
@@ -849,7 +850,8 @@ static struct uvc_control *uvc_find_control(struct uvc_video_chain *chain,
 	v4l2_id &= V4L2_CTRL_ID_MASK;
 
 	/* Find the control. */
-	list_for_each_entry(entity, &chain->entities, chain) {
+	list_for_each_entry(entry, &chain->entities, chain_entry) {
+		entity = entry->entity;
 		__uvc_find_control(entity, v4l2_id, mapping, &ctrl, next);
 		if (ctrl && !next)
 			return ctrl;
@@ -1048,6 +1050,16 @@ static int __uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
 	return 0;
 }
 
+int __uvc_ctrl_lock(struct uvc_video_chain *chain)
+{
+	return mutex_lock_interruptible(&chain->pipeline->ctrl_mutex) ?
+					-ERESTARTSYS : 0;
+}
+void __uvc_ctrl_unlock(struct uvc_video_chain *chain)
+{
+	mutex_unlock(&chain->pipeline->ctrl_mutex);
+}
+
 int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
 	struct v4l2_queryctrl *v4l2_ctrl)
 {
@@ -1055,9 +1067,9 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
 	struct uvc_control_mapping *mapping;
 	int ret;
 
-	ret = mutex_lock_interruptible(&chain->ctrl_mutex);
+	ret = __uvc_ctrl_lock(chain);
 	if (ret < 0)
-		return -ERESTARTSYS;
+		return ret;
 
 	ctrl = uvc_find_control(chain, v4l2_ctrl->id, &mapping);
 	if (ctrl == NULL) {
@@ -1067,7 +1079,7 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
 
 	ret = __uvc_query_v4l2_ctrl(chain, ctrl, mapping, v4l2_ctrl);
 done:
-	mutex_unlock(&chain->ctrl_mutex);
+	__uvc_ctrl_unlock(chain);
 	return ret;
 }
 
@@ -1094,9 +1106,9 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
 	query_menu->id = id;
 	query_menu->index = index;
 
-	ret = mutex_lock_interruptible(&chain->ctrl_mutex);
+	ret = __uvc_ctrl_lock(chain);
 	if (ret < 0)
-		return -ERESTARTSYS;
+		return ret;
 
 	ctrl = uvc_find_control(chain, query_menu->id, &mapping);
 	if (ctrl == NULL || mapping->v4l2_type != V4L2_CTRL_TYPE_MENU) {
@@ -1132,7 +1144,7 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
 	strlcpy(query_menu->name, menu_info->name, sizeof query_menu->name);
 
 done:
-	mutex_unlock(&chain->ctrl_mutex);
+	__uvc_ctrl_unlock(chain);
 	return ret;
 }
 
@@ -1257,9 +1269,9 @@ static int uvc_ctrl_add_event(struct v4l2_subscribed_event *sev, unsigned elems)
 	struct uvc_control *ctrl;
 	int ret;
 
-	ret = mutex_lock_interruptible(&handle->chain->ctrl_mutex);
+	ret = __uvc_ctrl_lock(handle->chain);
 	if (ret < 0)
-		return -ERESTARTSYS;
+		return ret;
 
 	ctrl = uvc_find_control(handle->chain, sev->id, &mapping);
 	if (ctrl == NULL) {
@@ -1285,7 +1297,7 @@ static int uvc_ctrl_add_event(struct v4l2_subscribed_event *sev, unsigned elems)
 	}
 
 done:
-	mutex_unlock(&handle->chain->ctrl_mutex);
+	__uvc_ctrl_unlock(handle->chain);
 	return ret;
 }
 
@@ -1293,9 +1305,9 @@ static void uvc_ctrl_del_event(struct v4l2_subscribed_event *sev)
 {
 	struct uvc_fh *handle = container_of(sev->fh, struct uvc_fh, vfh);
 
-	mutex_lock(&handle->chain->ctrl_mutex);
+	__uvc_ctrl_lock(handle->chain);
 	list_del(&sev->node);
-	mutex_unlock(&handle->chain->ctrl_mutex);
+	__uvc_ctrl_unlock(handle->chain);
 }
 
 const struct v4l2_subscribed_event_ops uvc_ctrl_sub_ev_ops = {
@@ -1331,7 +1343,7 @@ const struct v4l2_subscribed_event_ops uvc_ctrl_sub_ev_ops = {
  */
 int uvc_ctrl_begin(struct uvc_video_chain *chain)
 {
-	return mutex_lock_interruptible(&chain->ctrl_mutex) ? -ERESTARTSYS : 0;
+	return __uvc_ctrl_lock(chain);
 }
 
 static int uvc_ctrl_commit_entity(struct uvc_device *dev,
@@ -1390,10 +1402,12 @@ int __uvc_ctrl_commit(struct uvc_fh *handle, int rollback,
 {
 	struct uvc_video_chain *chain = handle->chain;
 	struct uvc_entity *entity;
+	struct uvc_chain_entry *entry;
 	int ret = 0;
 
 	/* Find the control. */
-	list_for_each_entry(entity, &chain->entities, chain) {
+	list_for_each_entry(entry, &chain->entities, chain_entry) {
+		entity = entry->entity;
 		ret = uvc_ctrl_commit_entity(chain->dev, entity, rollback);
 		if (ret < 0)
 			goto done;
@@ -1402,7 +1416,7 @@ int __uvc_ctrl_commit(struct uvc_fh *handle, int rollback,
 	if (!rollback)
 		uvc_ctrl_send_events(handle, xctrls, xctrls_count);
 done:
-	mutex_unlock(&chain->ctrl_mutex);
+	__uvc_ctrl_unlock(chain);
 	return ret;
 }
 
@@ -1667,7 +1681,8 @@ static int uvc_ctrl_init_xu_ctrl(struct uvc_device *dev,
 int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
 	struct uvc_xu_control_query *xqry)
 {
-	struct uvc_entity *entity;
+	struct uvc_entity *entity = NULL;
+	struct uvc_chain_entry *entry;
 	struct uvc_control *ctrl;
 	unsigned int i, found = 0;
 	__u32 reqflags;
@@ -1676,13 +1691,14 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
 	int ret;
 
 	/* Find the extension unit. */
-	list_for_each_entry(entity, &chain->entities, chain) {
+	list_for_each_entry(entry, &chain->entities, chain_entry) {
+		entity = entry->entity;
 		if (UVC_ENTITY_TYPE(entity) == UVC_VC_EXTENSION_UNIT &&
 		    entity->id == xqry->unit)
 			break;
 	}
 
-	if (entity->id != xqry->unit) {
+	if (!entity || entity->id != xqry->unit) {
 		uvc_trace(UVC_TRACE_CONTROL, "Extension unit %u not found.\n",
 			xqry->unit);
 		return -ENOENT;
@@ -1703,8 +1719,9 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
 		return -ENOENT;
 	}
 
-	if (mutex_lock_interruptible(&chain->ctrl_mutex))
-		return -ERESTARTSYS;
+	ret = __uvc_ctrl_lock(chain);
+	if (ret < 0)
+		return ret;
 
 	ret = uvc_ctrl_init_xu_ctrl(chain->dev, ctrl);
 	if (ret < 0) {
@@ -1778,7 +1795,7 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
 		ret = -EFAULT;
 done:
 	kfree(data);
-	mutex_unlock(&chain->ctrl_mutex);
+	__uvc_ctrl_unlock(chain);
 	return ret;
 }
 
@@ -1904,6 +1921,7 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain *chain,
 	const struct uvc_control_mapping *mapping)
 {
 	struct uvc_device *dev = chain->dev;
+	struct uvc_chain_entry *entry;
 	struct uvc_control_mapping *map;
 	struct uvc_entity *entity;
 	struct uvc_control *ctrl;
@@ -1918,8 +1936,9 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain *chain,
 	}
 
 	/* Search for the matching (GUID/CS) control on the current chain */
-	list_for_each_entry(entity, &chain->entities, chain) {
+	list_for_each_entry(entry, &chain->entities, chain_entry) {
 		unsigned int i;
+		entity = entry->entity;
 
 		if (UVC_ENTITY_TYPE(entity) != UVC_VC_EXTENSION_UNIT ||
 		    !uvc_entity_match_guid(entity, mapping->entity))
@@ -1939,8 +1958,9 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain *chain,
 	if (!found)
 		return -ENOENT;
 
-	if (mutex_lock_interruptible(&chain->ctrl_mutex))
-		return -ERESTARTSYS;
+	ret = __uvc_ctrl_lock(chain);
+	if (ret < 0)
+		return ret;
 
 	/* Perform delayed initialization of XU controls */
 	ret = uvc_ctrl_init_xu_ctrl(dev, ctrl);
@@ -1974,7 +1994,7 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain *chain,
 		atomic_dec(&dev->nmappings);
 
 done:
-	mutex_unlock(&chain->ctrl_mutex);
+	__uvc_ctrl_unlock(chain);
 	return ret;
 }
 
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 81695d4..d7ff707 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -1215,6 +1215,36 @@ next_descriptor:
  * UVC device scan
  */
 
+static int uvc_add_chain_entry(struct uvc_video_chain *chain,
+				struct uvc_entity *entity)
+{
+	struct uvc_chain_entry *chain_entry;
+
+	chain_entry = kzalloc(sizeof(struct uvc_chain_entry), GFP_KERNEL);
+	if (!chain_entry)
+		return -ENOMEM;
+
+	chain_entry->entity = entity;
+	list_add_tail(&chain_entry->chain_entry, &chain->entities);
+	if (!entity->chain)
+		entity->chain = chain;
+
+	return 0;
+}
+
+static void uvc_delete_chain(struct uvc_video_chain *chain)
+{
+	struct list_head *p, *n;
+	struct uvc_chain_entry *entry;
+
+	list_for_each_safe(p, n, &chain->entities) {
+		entry = list_entry(p, struct uvc_chain_entry, chain_entry);
+		kfree(entry);
+	}
+
+	kfree(chain);
+}
+
 /*
  * Scan the UVC descriptors to locate a chain starting at an Output Terminal
  * and containing the following units:
@@ -1320,72 +1350,7 @@ static int uvc_scan_chain_entity(struct uvc_video_chain *chain,
 		return -1;
 	}
 
-	list_add_tail(&entity->chain, &chain->entities);
-	return 0;
-}
-
-static int uvc_scan_chain_forward(struct uvc_video_chain *chain,
-	struct uvc_entity *entity, struct uvc_entity *prev)
-{
-	struct uvc_entity *forward;
-	int found;
-
-	/* Forward scan */
-	forward = NULL;
-	found = 0;
-
-	while (1) {
-		forward = uvc_entity_by_reference(chain->dev, entity->id,
-			forward);
-		if (forward == NULL)
-			break;
-		if (forward == prev)
-			continue;
-
-		switch (UVC_ENTITY_TYPE(forward)) {
-		case UVC_VC_EXTENSION_UNIT:
-			if (forward->bNrInPins != 1) {
-				uvc_trace(UVC_TRACE_DESCR, "Extension unit %d "
-					  "has more than 1 input pin.\n",
-					  entity->id);
-				return -EINVAL;
-			}
-
-			list_add_tail(&forward->chain, &chain->entities);
-			if (uvc_trace_param & UVC_TRACE_PROBE) {
-				if (!found)
-					printk(" (->");
-
-				printk(" XU %d", forward->id);
-				found = 1;
-			}
-			break;
-
-		case UVC_OTT_VENDOR_SPECIFIC:
-		case UVC_OTT_DISPLAY:
-		case UVC_OTT_MEDIA_TRANSPORT_OUTPUT:
-		case UVC_TT_STREAMING:
-			if (UVC_ENTITY_IS_ITERM(forward)) {
-				uvc_trace(UVC_TRACE_DESCR, "Unsupported input "
-					"terminal %u.\n", forward->id);
-				return -EINVAL;
-			}
-
-			list_add_tail(&forward->chain, &chain->entities);
-			if (uvc_trace_param & UVC_TRACE_PROBE) {
-				if (!found)
-					printk(" (->");
-
-				printk(" OT %d", forward->id);
-				found = 1;
-			}
-			break;
-		}
-	}
-	if (found)
-		printk(")");
-
-	return 0;
+	return uvc_add_chain_entry(chain, entity);
 }
 
 static int uvc_scan_chain_backward(struct uvc_video_chain *chain,
@@ -1394,6 +1359,7 @@ static int uvc_scan_chain_backward(struct uvc_video_chain *chain,
 	struct uvc_entity *entity = *_entity;
 	struct uvc_entity *term;
 	int id = -EINVAL, i;
+	int ret;
 
 	switch (UVC_ENTITY_TYPE(entity)) {
 	case UVC_VC_EXTENSION_UNIT:
@@ -1425,8 +1391,9 @@ static int uvc_scan_chain_backward(struct uvc_video_chain *chain,
 			if (uvc_trace_param & UVC_TRACE_PROBE)
 				printk(" %d", term->id);
 
-			list_add_tail(&term->chain, &chain->entities);
-			uvc_scan_chain_forward(chain, term, entity);
+			ret = uvc_add_chain_entry(chain, term);
+			if (ret)
+				return ret;
 		}
 
 		if (uvc_trace_param & UVC_TRACE_PROBE)
@@ -1473,23 +1440,23 @@ static int uvc_scan_chain(struct uvc_video_chain *chain,
 	prev = NULL;
 
 	while (entity != NULL) {
-		/* Entity must not be part of an existing chain */
-		if (entity->chain.next || entity->chain.prev) {
-			uvc_trace(UVC_TRACE_DESCR, "Found reference to "
-				"entity %d already in chain.\n", entity->id);
+		if (entity->chain == chain) {
+			uvc_trace(UVC_TRACE_DESCR, "Found a cycle in the "
+					"chain");
 			return -EINVAL;
 		}
 
+		/* If this entity is a part of an existing chain, the
+		 * current chain belongs to the same pipeline.
+		 */
+		if (entity->chain)
+			chain->pipeline = entity->chain->pipeline;
+
 		/* Process entity */
 		if (uvc_scan_chain_entity(chain, entity) < 0)
 			return -EINVAL;
 
-		/* Forward scan */
-		if (uvc_scan_chain_forward(chain, entity, prev) < 0)
-			return -EINVAL;
-
 		/* Backward scan */
-		prev = entity;
 		if (uvc_scan_chain_backward(chain, &entity) < 0)
 			return -EINVAL;
 	}
@@ -1501,10 +1468,12 @@ static unsigned int uvc_print_terms(struct list_head *terms, u16 dir,
 		char *buffer)
 {
 	struct uvc_entity *term;
+	struct uvc_chain_entry *entry;
 	unsigned int nterms = 0;
 	char *p = buffer;
 
-	list_for_each_entry(term, terms, chain) {
+	list_for_each_entry(entry, terms, chain_entry) {
+		term = entry->entity;
 		if (!UVC_ENTITY_IS_TERM(term) ||
 		    UVC_TERM_DIRECTION(term) != dir)
 			continue;
@@ -1541,39 +1510,58 @@ static const char *uvc_print_chain(struct uvc_video_chain *chain)
 static int uvc_scan_device(struct uvc_device *dev)
 {
 	struct uvc_video_chain *chain;
-	struct uvc_entity *term;
+	struct uvc_entity *entity, *source;
+	int ret;
 
-	list_for_each_entry(term, &dev->entities, list) {
-		if (!UVC_ENTITY_IS_OTERM(term))
+	list_for_each_entry(entity, &dev->entities, list) {
+		if (!UVC_ENTITY_IS_OTERM(entity))
 			continue;
 
-		/* If the terminal is already included in a chain, skip it.
-		 * This can happen for chains that have multiple output
-		 * terminals, where all output terminals beside the first one
-		 * will be inserted in the chain in forward scans.
+		/* Allow single-unit branches of Output Terminals to reside
+		 * on the existing chains.
 		 */
-		if (term->chain.next || term->chain.prev)
-			continue;
+		source = uvc_entity_by_id(dev, entity->baSourceID[0]);
+		if (entity == NULL) {
+			uvc_trace(UVC_TRACE_DESCR, "Found reference to "
+				"unknown entity %d.\n", entity->baSourceID[0]);
+			return -EINVAL;
+		}
 
 		chain = kzalloc(sizeof(*chain), GFP_KERNEL);
 		if (chain == NULL)
 			return -ENOMEM;
 
 		INIT_LIST_HEAD(&chain->entities);
-		mutex_init(&chain->ctrl_mutex);
 		chain->dev = dev;
 		v4l2_prio_init(&chain->prio);
 
-		term->flags |= UVC_ENTITY_FLAG_DEFAULT;
+		entity->flags |= UVC_ENTITY_FLAG_DEFAULT;
 
-		if (uvc_scan_chain(chain, term) < 0) {
-			kfree(chain);
+		if (uvc_scan_chain(chain, entity) < 0) {
+			uvc_delete_chain(chain);
 			continue;
 		}
 
 		uvc_trace(UVC_TRACE_PROBE, "Found a valid video chain (%s).\n",
 			  uvc_print_chain(chain));
 
+		/*
+		 * If none of the entities are shared, allocate a new pipeline.
+		 * Otherwise, the shared pipeline is already set up.
+		 */
+		if (!chain->pipeline) {
+			chain->pipeline = kzalloc(sizeof(*chain->pipeline),
+						  GFP_KERNEL);
+			if (!chain->pipeline) {
+				uvc_delete_chain(chain);
+				return -ENOMEM;
+			}
+			mutex_init(&chain->pipeline->ctrl_mutex);
+			atomic_set(&chain->pipeline->num_chains, 1);
+		} else {
+			atomic_inc(&chain->pipeline->num_chains);
+		}
+
 		list_add_tail(&chain->list, &dev->chains);
 	}
 
@@ -1582,6 +1570,38 @@ static int uvc_scan_device(struct uvc_device *dev)
 		return -1;
 	}
 
+	/* Find branches with no OTERMs (if any) by looking for entities not
+	 * on any chain. Accept only branches with a single Extension Unit.
+	 */
+	list_for_each_entry(entity, &dev->entities, list) {
+		if (entity->chain)
+			continue;
+
+		if (UVC_ENTITY_TYPE(entity) != UVC_VC_EXTENSION_UNIT
+			|| entity->bNrInPins != 1
+			|| uvc_entity_by_reference(dev, entity->id, NULL)) {
+			uvc_printk(KERN_INFO, "Found an invalid branch "
+				"starting at entity id %d.\n", entity->id);
+			return -1;
+		}
+
+		/* Single-unit XU branch. */
+		source = uvc_entity_by_id(dev, entity->baSourceID[0]);
+		if (source == NULL) {
+			uvc_trace(UVC_TRACE_DESCR, "Found reference to "
+				"unknown entity %d.\n", entity->baSourceID[0]);
+			return -EINVAL;
+		}
+		if (!source->chain)
+			continue;
+
+		ret = uvc_add_chain_entry(source->chain, entity);
+		if (ret)
+			return ret;
+		uvc_trace(UVC_TRACE_DESCR, "XU %d <- (%d)\n",
+				entity->id, source->id);
+	}
+
 	return 0;
 }
 
@@ -1619,7 +1639,9 @@ static void uvc_delete(struct uvc_device *dev)
 	list_for_each_safe(p, n, &dev->chains) {
 		struct uvc_video_chain *chain;
 		chain = list_entry(p, struct uvc_video_chain, list);
-		kfree(chain);
+		if (atomic_dec_and_test(&chain->pipeline->num_chains))
+			kfree(chain->pipeline);
+		uvc_delete_chain(chain);
 	}
 
 	list_for_each_safe(p, n, &dev->entities) {
@@ -1763,9 +1785,11 @@ static int uvc_register_terms(struct uvc_device *dev,
 {
 	struct uvc_streaming *stream;
 	struct uvc_entity *term;
+	struct uvc_chain_entry *entry;
 	int ret;
 
-	list_for_each_entry(term, &chain->entities, chain) {
+	list_for_each_entry(entry, &chain->entities, chain_entry) {
+		term = entry->entity;
 		if (UVC_ENTITY_TYPE(term) != UVC_TT_STREAMING)
 			continue;
 
diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
index dc56a59..657f49a 100644
--- a/drivers/media/usb/uvc/uvc_entity.c
+++ b/drivers/media/usb/uvc/uvc_entity.c
@@ -104,9 +104,14 @@ static int uvc_mc_init_entity(struct uvc_entity *entity)
 int uvc_mc_register_entities(struct uvc_video_chain *chain)
 {
 	struct uvc_entity *entity;
+	struct uvc_chain_entry *entry;
 	int ret;
 
-	list_for_each_entry(entity, &chain->entities, chain) {
+	list_for_each_entry(entry, &chain->entities, chain_entry) {
+		entity = entry->entity;
+		if (entity->registered)
+			continue;
+
 		ret = uvc_mc_init_entity(entity);
 		if (ret < 0) {
 			uvc_printk(KERN_INFO, "Failed to initialize entity for "
@@ -115,13 +120,19 @@ int uvc_mc_register_entities(struct uvc_video_chain *chain)
 		}
 	}
 
-	list_for_each_entry(entity, &chain->entities, chain) {
+	list_for_each_entry(entry, &chain->entities, chain_entry) {
+		entity = entry->entity;
+		if (entity->registered)
+			continue;
+
 		ret = uvc_mc_register_entity(chain, entity);
 		if (ret < 0) {
 			uvc_printk(KERN_INFO, "Failed to register entity for "
 				   "entity %u\n", entity->id);
 			return ret;
 		}
+
+		entity->registered = true;
 	}
 
 	return 0;
diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
index 3afff92..a899159 100644
--- a/drivers/media/usb/uvc/uvc_v4l2.c
+++ b/drivers/media/usb/uvc/uvc_v4l2.c
@@ -713,6 +713,7 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		const struct uvc_entity *selector = chain->selector;
 		struct v4l2_input *input = arg;
 		struct uvc_entity *iterm = NULL;
+		struct uvc_chain_entry *entry;
 		u32 index = input->index;
 		int pin = 0;
 
@@ -720,14 +721,18 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		    (chain->dev->quirks & UVC_QUIRK_IGNORE_SELECTOR_UNIT)) {
 			if (index != 0)
 				return -EINVAL;
-			list_for_each_entry(iterm, &chain->entities, chain) {
+			list_for_each_entry(entry, &chain->entities,
+						chain_entry) {
+				iterm = entry->entity;
 				if (UVC_ENTITY_IS_ITERM(iterm))
 					break;
 			}
 			pin = iterm->id;
 		} else if (index < selector->bNrInPins) {
 			pin = selector->baSourceID[index];
-			list_for_each_entry(iterm, &chain->entities, chain) {
+			list_for_each_entry(entry, &chain->entities,
+						chain_entry) {
+				iterm = entry->entity;
 				if (!UVC_ENTITY_IS_ITERM(iterm))
 					continue;
 				if (iterm->id == pin)
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 75e0153..731b378 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -229,8 +229,8 @@ struct uvc_format_desc {
 
 struct uvc_entity {
 	struct list_head list;		/* Entity as part of a UVC device. */
-	struct list_head chain;		/* Entity as part of a video device
-					 * chain. */
+	struct uvc_video_chain *chain;  /* Entity as a part of a video device
+					   chain. */
 	unsigned int flags;
 
 	__u8 id;
@@ -243,6 +243,7 @@ struct uvc_entity {
 	unsigned int num_pads;
 	unsigned int num_links;
 	struct media_pad *pads;
+	bool registered;		/* True if already registered with MC */
 
 	union {
 		struct {
@@ -289,6 +290,12 @@ struct uvc_entity {
 	struct uvc_control *controls;
 };
 
+struct uvc_chain_entry {
+	struct list_head chain_entry;
+	struct uvc_entity *entity;
+	struct uvc_video_chain *chain;
+};
+
 struct uvc_frame {
 	__u8  bFrameIndex;
 	__u8  bmCapabilities;
@@ -366,6 +373,12 @@ struct uvc_video_queue {
 	struct list_head irqqueue;
 };
 
+struct uvc_video_pipeline {
+	struct mutex ctrl_mutex;                /* Protects controls in all
+						   chains of this pipeline */
+	atomic_t num_chains;
+};
+
 struct uvc_video_chain {
 	struct uvc_device *dev;
 	struct list_head list;
@@ -374,7 +387,8 @@ struct uvc_video_chain {
 	struct uvc_entity *processing;		/* Processing unit */
 	struct uvc_entity *selector;		/* Selector unit */
 
-	struct mutex ctrl_mutex;		/* Protects ctrl.info */
+	struct uvc_video_pipeline *pipeline;    /* Pipeline this chain
+						   belongs to */
 
 	struct v4l2_prio_state prio;		/* V4L2 priority state */
 	u32 caps;				/* V4L2 chain-wide caps */
-- 
1.8.4

[PATCH v1 04/19] uvcvideo: Create separate debugfs entries for each streaming interface.

[Pawel Osciak]

Add interface number to debugfs entry name to be able to create separate
entries for each streaming interface for devices exposing more than one,
instead of failing to create more than one.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_debugfs.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/usb/uvc/uvc_debugfs.c b/drivers/media/usb/uvc/uvc_debugfs.c
index 14561a5..0663fbd 100644
--- a/drivers/media/usb/uvc/uvc_debugfs.c
+++ b/drivers/media/usb/uvc/uvc_debugfs.c
@@ -84,7 +84,8 @@ int uvc_debugfs_init_stream(struct uvc_streaming *stream)
 	if (uvc_debugfs_root_dir == NULL)
 		return -ENODEV;
 
-	sprintf(dir_name, "%u-%u", udev->bus->busnum, udev->devnum);
+	sprintf(dir_name, "%u-%u-%u", udev->bus->busnum, udev->devnum,
+			stream->intfnum);
 
 	dent = debugfs_create_dir(dir_name, uvc_debugfs_root_dir);
 	if (IS_ERR_OR_NULL(dent)) {
-- 
1.8.4

[PATCH v1 05/19] uvcvideo: Add support for UVC1.5 P&C control.

[Pawel Osciak]

Add support for UVC 1.5 Probe & Commit control.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_video.c | 52 ++++++++++++++++++++++++++++++++++++---
 include/uapi/linux/usb/video.h    |  7 ++++++
 2 files changed, 55 insertions(+), 4 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 1198989..b4ebccd 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -168,14 +168,25 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming *stream,
 	}
 }
 
+int uvc_get_probe_ctrl_size(struct uvc_streaming *stream)
+{
+	if (stream->dev->uvc_version < 0x0110)
+		return 26;
+	else if (stream->dev->uvc_version < 0x0150)
+		return 34;
+	else
+		return 48;
+}
+
 static int uvc_get_video_ctrl(struct uvc_streaming *stream,
 	struct uvc_streaming_control *ctrl, int probe, __u8 query)
 {
 	__u8 *data;
 	__u16 size;
 	int ret;
+	int i;
 
-	size = stream->dev->uvc_version >= 0x0110 ? 34 : 26;
+	size = uvc_get_probe_ctrl_size(stream);
 	if ((stream->dev->quirks & UVC_QUIRK_PROBE_DEF) &&
 			query == UVC_GET_DEF)
 		return -EIO;
@@ -230,7 +241,7 @@ static int uvc_get_video_ctrl(struct uvc_streaming *stream,
 	ctrl->dwMaxVideoFrameSize = get_unaligned_le32(&data[18]);
 	ctrl->dwMaxPayloadTransferSize = get_unaligned_le32(&data[22]);
 
-	if (size == 34) {
+	if (size >= 34) {
 		ctrl->dwClockFrequency = get_unaligned_le32(&data[26]);
 		ctrl->bmFramingInfo = data[30];
 		ctrl->bPreferedVersion = data[31];
@@ -244,6 +255,26 @@ static int uvc_get_video_ctrl(struct uvc_streaming *stream,
 		ctrl->bMaxVersion = 0;
 	}
 
+	if (size >= 48) {
+		ctrl->bUsage = data[34];
+		ctrl->bBitDepthLuma = data[35];
+		ctrl->bmSetting = data[36];
+		ctrl->bMaxNumberOfRefFramesPlus1 = data[37];
+		ctrl->bmRateControlModes = get_unaligned_le16(&data[38]);
+		for (i = 0; i < ARRAY_SIZE(ctrl->bmLayoutPerStream); ++i) {
+			ctrl->bmLayoutPerStream[i] =
+				get_unaligned_le16(&data[40 + i * 2]);
+		}
+	} else {
+		ctrl->bUsage = 0;
+		ctrl->bBitDepthLuma = 0;
+		ctrl->bmSetting = 0;
+		ctrl->bMaxNumberOfRefFramesPlus1 = 0;
+		ctrl->bmRateControlModes = 0;
+		for (i = 0; i < ARRAY_SIZE(ctrl->bmLayoutPerStream); ++i)
+			ctrl->bmLayoutPerStream[i] = 0;
+	}
+
 	/* Some broken devices return null or wrong dwMaxVideoFrameSize and
 	 * dwMaxPayloadTransferSize fields. Try to get the value from the
 	 * format and frame descriptors.
@@ -262,8 +293,9 @@ static int uvc_set_video_ctrl(struct uvc_streaming *stream,
 	__u8 *data;
 	__u16 size;
 	int ret;
+	int i;
 
-	size = stream->dev->uvc_version >= 0x0110 ? 34 : 26;
+	size = uvc_get_probe_ctrl_size(stream);
 	data = kzalloc(size, GFP_KERNEL);
 	if (data == NULL)
 		return -ENOMEM;
@@ -280,7 +312,7 @@ static int uvc_set_video_ctrl(struct uvc_streaming *stream,
 	put_unaligned_le32(ctrl->dwMaxVideoFrameSize, &data[18]);
 	put_unaligned_le32(ctrl->dwMaxPayloadTransferSize, &data[22]);
 
-	if (size == 34) {
+	if (size >= 34) {
 		put_unaligned_le32(ctrl->dwClockFrequency, &data[26]);
 		data[30] = ctrl->bmFramingInfo;
 		data[31] = ctrl->bPreferedVersion;
@@ -288,6 +320,18 @@ static int uvc_set_video_ctrl(struct uvc_streaming *stream,
 		data[33] = ctrl->bMaxVersion;
 	}
 
+	if (size >= 48) {
+		data[34] = ctrl->bUsage;
+		data[35] = ctrl->bBitDepthLuma;
+		data[36] = ctrl->bmSetting;
+		data[37] = ctrl->bMaxNumberOfRefFramesPlus1;
+		*(__le16 *)&data[38] = cpu_to_le16(ctrl->bmRateControlModes);
+		for (i = 0; i < ARRAY_SIZE(ctrl->bmLayoutPerStream); ++i) {
+			*(__le16 *)&data[40 + i * 2] =
+				cpu_to_le16(ctrl->bmLayoutPerStream[i]);
+		}
+	}
+
 	ret = __uvc_query_ctrl(stream->dev, UVC_SET_CUR, 0, stream->intfnum,
 		probe ? UVC_VS_PROBE_CONTROL : UVC_VS_COMMIT_CONTROL, data,
 		size, uvc_timeout_param);
diff --git a/include/uapi/linux/usb/video.h b/include/uapi/linux/usb/video.h
index 3b3b95e..331c071 100644
--- a/include/uapi/linux/usb/video.h
+++ b/include/uapi/linux/usb/video.h
@@ -432,6 +432,7 @@ struct uvc_color_matching_descriptor {
 #define UVC_DT_COLOR_MATCHING_SIZE			6
 
 /* 4.3.1.1. Video Probe and Commit Controls */
+#define UVC_NUM_SIMULCAST_STREAMS			4
 struct uvc_streaming_control {
 	__u16 bmHint;
 	__u8  bFormatIndex;
@@ -449,6 +450,12 @@ struct uvc_streaming_control {
 	__u8  bPreferedVersion;
 	__u8  bMinVersion;
 	__u8  bMaxVersion;
+	__u8  bUsage;
+	__u8  bBitDepthLuma;
+	__u8  bmSetting;
+	__u8  bMaxNumberOfRefFramesPlus1;
+	__u16 bmRateControlModes;
+	__u16 bmLayoutPerStream[UVC_NUM_SIMULCAST_STREAMS];
 } __attribute__((__packed__));
 
 /* Uncompressed Payload - 3.1.1. Uncompressed Video Format Descriptor */
-- 
1.8.4

[PATCH v1 06/19] uvcvideo: Recognize UVC 1.5 encoding units.

[Pawel Osciak]

Add encoding unit definitions and descriptor parsing code and allow them to
be added to chains.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_ctrl.c   | 37 ++++++++++++++++++---
 drivers/media/usb/uvc/uvc_driver.c | 67 +++++++++++++++++++++++++++++++++-----
 drivers/media/usb/uvc/uvcvideo.h   | 14 +++++++-
 include/uapi/linux/usb/video.h     |  1 +
 4 files changed, 105 insertions(+), 14 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index ba159a4..72d6724 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -777,6 +777,7 @@ static const __u8 uvc_processing_guid[16] = UVC_GUID_UVC_PROCESSING;
 static const __u8 uvc_camera_guid[16] = UVC_GUID_UVC_CAMERA;
 static const __u8 uvc_media_transport_input_guid[16] =
 	UVC_GUID_UVC_MEDIA_TRANSPORT_INPUT;
+static const __u8 uvc_encoding_guid[16] = UVC_GUID_UVC_ENCODING;
 
 static int uvc_entity_match_guid(const struct uvc_entity *entity,
 	const __u8 guid[16])
@@ -795,6 +796,9 @@ static int uvc_entity_match_guid(const struct uvc_entity *entity,
 		return memcmp(entity->extension.guidExtensionCode,
 			      guid, 16) == 0;
 
+	case UVC_VC_ENCODING_UNIT:
+		return memcmp(uvc_encoding_guid, guid, 16) == 0;
+
 	default:
 		return 0;
 	}
@@ -2105,12 +2109,13 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
 {
 	struct uvc_entity *entity;
 	unsigned int i;
+	int num_found;
 
 	/* Walk the entities list and instantiate controls */
 	list_for_each_entry(entity, &dev->entities, list) {
 		struct uvc_control *ctrl;
-		unsigned int bControlSize = 0, ncontrols;
-		__u8 *bmControls = NULL;
+		unsigned int bControlSize = 0, ncontrols = 0;
+		__u8 *bmControls = NULL, *bmControlsRuntime = NULL;
 
 		if (UVC_ENTITY_TYPE(entity) == UVC_VC_EXTENSION_UNIT) {
 			bmControls = entity->extension.bmControls;
@@ -2121,13 +2126,25 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
 		} else if (UVC_ENTITY_TYPE(entity) == UVC_ITT_CAMERA) {
 			bmControls = entity->camera.bmControls;
 			bControlSize = entity->camera.bControlSize;
+		} else if (UVC_ENTITY_TYPE(entity) == UVC_VC_ENCODING_UNIT) {
+			bmControls = entity->encoding.bmControls;
+			bmControlsRuntime = entity->encoding.bmControlsRuntime;
+			bControlSize = entity->encoding.bControlSize;
 		}
 
 		/* Remove bogus/blacklisted controls */
 		uvc_ctrl_prune_entity(dev, entity);
 
 		/* Count supported controls and allocate the controls array */
-		ncontrols = memweight(bmControls, bControlSize);
+		for (i = 0; i < bControlSize; ++i) {
+			if (bmControlsRuntime) {
+				ncontrols += hweight8(bmControls[i]
+						      | bmControlsRuntime[i]);
+			} else {
+				ncontrols += hweight8(bmControls[i]);
+			}
+		}
+
 		if (ncontrols == 0)
 			continue;
 
@@ -2139,8 +2156,17 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
 
 		/* Initialize all supported controls */
 		ctrl = entity->controls;
-		for (i = 0; i < bControlSize * 8; ++i) {
-			if (uvc_test_bit(bmControls, i) == 0)
+		for (i = 0, num_found = 0;
+			i < bControlSize * 8 && num_found < ncontrols; ++i) {
+			if (uvc_test_bit(bmControls, i) == 1)
+				ctrl->on_init = 1;
+			if (bmControlsRuntime &&
+				uvc_test_bit(bmControlsRuntime, i) == 1)
+				ctrl->in_runtime = 1;
+			else if (!bmControlsRuntime)
+				ctrl->in_runtime = ctrl->on_init;
+
+			if (ctrl->on_init == 0 && ctrl->in_runtime == 0)
 				continue;
 
 			ctrl->entity = entity;
@@ -2148,6 +2174,7 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
 
 			uvc_ctrl_init_ctrl(dev, ctrl);
 			ctrl++;
+			num_found++;
 		}
 	}
 
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index d7ff707..d950b40 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -1155,6 +1155,37 @@ static int uvc_parse_standard_control(struct uvc_device *dev,
 		list_add_tail(&unit->list, &dev->entities);
 		break;
 
+	case UVC_VC_ENCODING_UNIT:
+		n = buflen >= 7 ? buffer[6] : 0;
+
+		if (buflen < 7 + 2 * n) {
+			uvc_trace(UVC_TRACE_DESCR, "device %d videocontrol "
+				"interface %d ENCODING_UNIT error\n",
+				udev->devnum, alts->desc.bInterfaceNumber);
+			return -EINVAL;
+		}
+
+		unit = uvc_alloc_entity(buffer[2], buffer[3], 2, 2 * n);
+		if (unit == NULL)
+			return -ENOMEM;
+
+		memcpy(unit->baSourceID, &buffer[4], 1);
+		unit->encoding.bControlSize = buffer[6];
+		unit->encoding.bmControls = (__u8 *)unit + sizeof(*unit);
+		memcpy(unit->encoding.bmControls, &buffer[7], n);
+		unit->encoding.bmControlsRuntime = unit->encoding.bmControls
+						 + n;
+		memcpy(unit->encoding.bmControlsRuntime, &buffer[7 + n], n);
+
+		if (buffer[5] != 0)
+			usb_string(udev, buffer[5], unit->name,
+				   sizeof(unit->name));
+		else
+			sprintf(unit->name, "encoding %u", buffer[3]);
+
+		list_add_tail(&unit->list, &dev->entities);
+		break;
+
 	default:
 		uvc_trace(UVC_TRACE_DESCR, "Found an unknown CS_INTERFACE "
 			"descriptor (%u)\n", buffer[2]);
@@ -1251,25 +1282,31 @@ static void uvc_delete_chain(struct uvc_video_chain *chain)
  *
  * - one or more Output Terminals (USB Streaming or Display)
  * - zero or one Processing Unit
+ * - zero or one Encoding Unit
  * - zero, one or more single-input Selector Units
  * - zero or one multiple-input Selector Units, provided all inputs are
  *   connected to input terminals
- * - zero, one or mode single-input Extension Units
+ * - zero, one or more single-input Extension Units
  * - one or more Input Terminals (Camera, External or USB Streaming)
  *
- * The terminal and units must match on of the following structures:
+ * The terminal and units must match one of the following structures:
  *
- * ITT_*(0) -> +---------+    +---------+    +---------+ -> TT_STREAMING(0)
- * ...         | SU{0,1} | -> | PU{0,1} | -> | XU{0,n} |    ...
- * ITT_*(n) -> +---------+    +---------+    +---------+ -> TT_STREAMING(n)
+ * ITT_*(0) -> +---------+                        -> TT_STREAMING(0)
+ * ...         | SU{0,1} | ->        (...)           ...
+ * ITT_*(n) -> +---------+                        -> TT_STREAMING(n)
+ *
+ *    Where (...), in any order:
+ *             +---------+    +---------+    +---------+
+ *             | PU{0,1} | -> | XU{0,n} | -> | EU{0,1} |
+ *             +---------+    +---------+    +---------+
  *
  *                 +---------+    +---------+ -> OTT_*(0)
  * TT_STREAMING -> | PU{0,1} | -> | XU{0,n} |    ...
  *                 +---------+    +---------+ -> OTT_*(n)
  *
- * The Processing Unit and Extension Units can be in any order. Additional
- * Extension Units connected to the main chain as single-unit branches are
- * also supported. Single-input Selector Units are ignored.
+ * The Processing Unit, the Encoding Unit and Extension Units can be in any
+ * order. Additional Extension Units connected to the main chain as single-unit
+ * branches are also supported. Single-input Selector Units are ignored.
  */
 static int uvc_scan_chain_entity(struct uvc_video_chain *chain,
 	struct uvc_entity *entity)
@@ -1317,6 +1354,19 @@ static int uvc_scan_chain_entity(struct uvc_video_chain *chain,
 		chain->selector = entity;
 		break;
 
+	case UVC_VC_ENCODING_UNIT:
+		if (uvc_trace_param & UVC_TRACE_PROBE)
+			printk(" <- EU %d", entity->id);
+
+		if (chain->encoding != NULL) {
+			uvc_trace(UVC_TRACE_DESCR, "Found multiple "
+				"Encoding Units in chain.\n");
+			return -1;
+		}
+
+		chain->encoding = entity;
+		break;
+
 	case UVC_ITT_VENDOR_SPECIFIC:
 	case UVC_ITT_CAMERA:
 	case UVC_ITT_MEDIA_TRANSPORT_INPUT:
@@ -1364,6 +1414,7 @@ static int uvc_scan_chain_backward(struct uvc_video_chain *chain,
 	switch (UVC_ENTITY_TYPE(entity)) {
 	case UVC_VC_EXTENSION_UNIT:
 	case UVC_VC_PROCESSING_UNIT:
+	case UVC_VC_ENCODING_UNIT:
 		id = entity->baSourceID[0];
 		break;
 
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 731b378..109c0a2 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -54,6 +54,9 @@
 #define UVC_GUID_UVC_SELECTOR \
 	{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
 	 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x02}
+#define UVC_GUID_UVC_ENCODING \
+	{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, \
+	 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x03}
 
 #define UVC_GUID_FORMAT_MJPEG \
 	{ 'M',  'J',  'P',  'G', 0x00, 0x00, 0x10, 0x00, \
@@ -199,7 +202,9 @@ struct uvc_control {
 	     loaded:1,
 	     modified:1,
 	     cached:1,
-	     initialized:1;
+	     initialized:1,
+	     on_init:1, /* supported during initialization */
+	     in_runtime:1; /* supported in runtime */
 
 	__u8 *uvc_data;
 };
@@ -281,6 +286,12 @@ struct uvc_entity {
 			__u8  *bmControls;
 			__u8  *bmControlsType;
 		} extension;
+
+		struct {
+			__u8  bControlSize;
+			__u8  *bmControls;
+			__u8  *bmControlsRuntime;
+		} encoding;
 	};
 
 	__u8 bNrInPins;
@@ -386,6 +397,7 @@ struct uvc_video_chain {
 	struct list_head entities;		/* All entities */
 	struct uvc_entity *processing;		/* Processing unit */
 	struct uvc_entity *selector;		/* Selector unit */
+	struct uvc_entity *encoding;		/* Encoding unit */
 
 	struct uvc_video_pipeline *pipeline;    /* Pipeline this chain
 						   belongs to */
diff --git a/include/uapi/linux/usb/video.h b/include/uapi/linux/usb/video.h
index 331c071..eb48ba8 100644
--- a/include/uapi/linux/usb/video.h
+++ b/include/uapi/linux/usb/video.h
@@ -37,6 +37,7 @@
 #define UVC_VC_SELECTOR_UNIT				0x04
 #define UVC_VC_PROCESSING_UNIT				0x05
 #define UVC_VC_EXTENSION_UNIT				0x06
+#define UVC_VC_ENCODING_UNIT				0x07
 
 /* A.6. Video Class-Specific VS Interface Descriptor Subtypes */
 #define UVC_VS_UNDEFINED				0x00
-- 
1.8.4

[PATCH v1 07/19] uvcvideo: Unify error reporting during format descriptor parsing.

[Pawel Osciak]

Add common error handling paths for format parsing failures.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_driver.c | 35 ++++++++++++++---------------------
 1 file changed, 14 insertions(+), 21 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index d950b40..936ddc7 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -322,13 +322,8 @@ static int uvc_parse_format(struct uvc_device *dev,
 	case UVC_VS_FORMAT_UNCOMPRESSED:
 	case UVC_VS_FORMAT_FRAME_BASED:
 		n = buffer[2] == UVC_VS_FORMAT_UNCOMPRESSED ? 27 : 28;
-		if (buflen < n) {
-			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
-			       "interface %d FORMAT error\n",
-			       dev->udev->devnum,
-			       alts->desc.bInterfaceNumber);
-			return -EINVAL;
-		}
+		if (buflen < n)
+			goto format_error;
 
 		/* Find the format descriptor from its GUID. */
 		fmtdesc = uvc_format_by_guid(&buffer[5]);
@@ -356,13 +351,8 @@ static int uvc_parse_format(struct uvc_device *dev,
 		break;
 
 	case UVC_VS_FORMAT_MJPEG:
-		if (buflen < 11) {
-			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
-			       "interface %d FORMAT error\n",
-			       dev->udev->devnum,
-			       alts->desc.bInterfaceNumber);
-			return -EINVAL;
-		}
+		if (buflen < 11)
+			goto format_error;
 
 		strlcpy(format->name, "MJPEG", sizeof format->name);
 		format->fcc = V4L2_PIX_FMT_MJPEG;
@@ -372,13 +362,8 @@ static int uvc_parse_format(struct uvc_device *dev,
 		break;
 
 	case UVC_VS_FORMAT_DV:
-		if (buflen < 9) {
-			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
-			       "interface %d FORMAT error\n",
-			       dev->udev->devnum,
-			       alts->desc.bInterfaceNumber);
-			return -EINVAL;
-		}
+		if (buflen < 9)
+			goto format_error;
 
 		switch (buffer[8] & 0x7f) {
 		case 0:
@@ -542,6 +527,14 @@ static int uvc_parse_format(struct uvc_device *dev,
 	}
 
 	return buffer - start;
+
+format_error:
+	uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
+			"interface %d FORMAT error\n",
+			dev->udev->devnum,
+			alts->desc.bInterfaceNumber);
+	return -EINVAL;
+
 }
 
 static int uvc_parse_streaming(struct uvc_device *dev,
-- 
1.8.4

[PATCH v1 08/19] uvcvideo: Add UVC1.5 VP8 format support.

[Pawel Osciak]

Add detection and parsing of VP8 format and frame descriptors and
reorganize format parsing code.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_driver.c | 120 ++++++++++++++++++++++++++++---------
 drivers/media/usb/uvc/uvcvideo.h   |   4 +-
 include/uapi/linux/usb/video.h     |   8 +++
 3 files changed, 104 insertions(+), 28 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 936ddc7..27a7a11 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -312,7 +312,7 @@ static int uvc_parse_format(struct uvc_device *dev,
 	struct uvc_frame *frame;
 	const unsigned char *start = buffer;
 	unsigned int interval;
-	unsigned int i, n;
+	unsigned int i, n, intervals_off;
 	__u8 ftype;
 
 	format->type = buffer[2];
@@ -401,6 +401,18 @@ static int uvc_parse_format(struct uvc_device *dev,
 		format->nframes = 1;
 		break;
 
+	case UVC_VS_FORMAT_VP8:
+		if (buflen < 13)
+			goto format_error;
+
+		format->bpp = 0;
+		format->flags = UVC_FMT_FLAG_COMPRESSED;
+		ftype = UVC_VS_FRAME_VP8;
+		strlcpy(format->name, "VP8", sizeof(format->name));
+		format->fcc = V4L2_PIX_FMT_VP8;
+
+		break;
+
 	case UVC_VS_FORMAT_MPEG2TS:
 	case UVC_VS_FORMAT_STREAM_BASED:
 		/* Not supported yet. */
@@ -417,44 +429,83 @@ static int uvc_parse_format(struct uvc_device *dev,
 	buflen -= buffer[0];
 	buffer += buffer[0];
 
-	/* Parse the frame descriptors. Only uncompressed, MJPEG and frame
-	 * based formats have frame descriptors.
+	/* Parse the frame descriptors. Only uncompressed, MJPEG, temporally
+	 * encoded and frame based formats have frame descriptors.
 	 */
 	while (buflen > 2 && buffer[1] == USB_DT_CS_INTERFACE &&
 	       buffer[2] == ftype) {
 		frame = &format->frame[format->nframes];
-		if (ftype != UVC_VS_FRAME_FRAME_BASED)
-			n = buflen > 25 ? buffer[25] : 0;
-		else
-			n = buflen > 21 ? buffer[21] : 0;
-
-		n = n ? n : 3;
 
-		if (buflen < 26 + 4*n) {
-			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
-			       "interface %d FRAME error\n", dev->udev->devnum,
-			       alts->desc.bInterfaceNumber);
-			return -EINVAL;
-		}
-
-		frame->bFrameIndex = buffer[3];
-		frame->bmCapabilities = buffer[4];
-		frame->wWidth = get_unaligned_le16(&buffer[5]);
-		frame->wHeight = get_unaligned_le16(&buffer[7]);
-		frame->dwMinBitRate = get_unaligned_le32(&buffer[9]);
-		frame->dwMaxBitRate = get_unaligned_le32(&buffer[13]);
-		if (ftype != UVC_VS_FRAME_FRAME_BASED) {
+		switch (ftype) {
+		case UVC_VS_FRAME_UNCOMPRESSED:
+		case UVC_VS_FRAME_MJPEG:
+			intervals_off = 26;
+			if (buflen < intervals_off)
+				goto frame_error;
+
+			frame->bFrameIndex = buffer[3];
+			frame->bmCapabilities = buffer[4];
+			frame->wWidth = get_unaligned_le16(&buffer[5]);
+			frame->wHeight = get_unaligned_le16(&buffer[7]);
+			frame->dwMinBitRate = get_unaligned_le32(&buffer[9]);
+			frame->dwMaxBitRate = get_unaligned_le32(&buffer[13]);
 			frame->dwMaxVideoFrameBufferSize =
 				get_unaligned_le32(&buffer[17]);
 			frame->dwDefaultFrameInterval =
 				get_unaligned_le32(&buffer[21]);
-			frame->bFrameIntervalType = buffer[25];
-		} else {
+			frame->bFrameIntervalType = n = buffer[25];
+			break;
+
+		case UVC_VS_FRAME_FRAME_BASED:
+			intervals_off = 26;
+			if (buflen < intervals_off)
+				goto frame_error;
+
+			frame->bFrameIndex = buffer[3];
+			frame->bmCapabilities = buffer[4];
+			frame->wWidth = get_unaligned_le16(&buffer[5]);
+			frame->wHeight = get_unaligned_le16(&buffer[7]);
+			frame->dwMinBitRate = get_unaligned_le32(&buffer[9]);
+			frame->dwMaxBitRate = get_unaligned_le32(&buffer[13]);
 			frame->dwMaxVideoFrameBufferSize = 0;
 			frame->dwDefaultFrameInterval =
 				get_unaligned_le32(&buffer[17]);
-			frame->bFrameIntervalType = buffer[21];
+			frame->bFrameIntervalType = n = buffer[21];
+			break;
+
+		case UVC_VS_FRAME_VP8:
+			intervals_off = 31;
+			if (buflen < intervals_off)
+				goto frame_error;
+
+			frame->bFrameIndex = buffer[3];
+			frame->bmSupportedUsages =
+				get_unaligned_le32(&buffer[8]);
+			frame->bmCapabilities =
+				get_unaligned_le16(&buffer[12]);
+			frame->bmScalabilityCapabilities =
+				get_unaligned_le32(&buffer[14]);
+			frame->wWidth = get_unaligned_le16(&buffer[4]);
+			frame->wHeight = get_unaligned_le16(&buffer[6]);
+			frame->dwMinBitRate = get_unaligned_le32(&buffer[18]);
+			frame->dwMaxBitRate = get_unaligned_le32(&buffer[22]);
+			frame->dwMaxVideoFrameBufferSize = 0;
+			frame->dwDefaultFrameInterval =
+				get_unaligned_le32(&buffer[26]);
+			frame->bFrameIntervalType = n = buffer[30];
+			break;
+
+		default:
+			uvc_trace(UVC_TRACE_CONTROL,
+				"Unsupported frame descriptor %d\n", ftype);
+			return -EINVAL;
 		}
+
+		/* For n=0 - continuous intervals given, always 3 values. */
+		n = n ? n : 3;
+		if (buflen < intervals_off + 4 * n)
+			goto frame_error;
+
 		frame->dwFrameInterval = *intervals;
 
 		/* Several UVC chipsets screw up dwMaxVideoFrameBufferSize
@@ -475,12 +526,14 @@ static int uvc_parse_format(struct uvc_device *dev,
 		 * some other divisions by zero that could happen.
 		 */
 		for (i = 0; i < n; ++i) {
-			interval = get_unaligned_le32(&buffer[26+4*i]);
+			interval = get_unaligned_le32(
+					&buffer[intervals_off + 4 * i]);
 			*(*intervals)++ = interval ? interval : 1;
 		}
 
 		/* Make sure that the default frame interval stays between
 		 * the boundaries.
+		 * For type = 0, the last value is interval step, so skip it.
 		 */
 		n -= frame->bFrameIntervalType ? 1 : 2;
 		frame->dwDefaultFrameInterval =
@@ -535,6 +588,11 @@ format_error:
 			alts->desc.bInterfaceNumber);
 	return -EINVAL;
 
+frame_error:
+	uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
+			"interface %d FRAME error\n", dev->udev->devnum,
+			alts->desc.bInterfaceNumber);
+	return -EINVAL;
 }
 
 static int uvc_parse_streaming(struct uvc_device *dev,
@@ -672,6 +730,7 @@ static int uvc_parse_streaming(struct uvc_device *dev,
 		case UVC_VS_FORMAT_UNCOMPRESSED:
 		case UVC_VS_FORMAT_MJPEG:
 		case UVC_VS_FORMAT_FRAME_BASED:
+		case UVC_VS_FORMAT_VP8:
 			nformats++;
 			break;
 
@@ -704,6 +763,12 @@ static int uvc_parse_streaming(struct uvc_device *dev,
 			if (_buflen > 21)
 				nintervals += _buffer[21] ? _buffer[21] : 3;
 			break;
+
+		case UVC_VS_FRAME_VP8:
+			nframes++;
+			if (_buflen > 30)
+				nintervals += _buffer[30] ? _buffer[30] : 0;
+			break;
 		}
 
 		_buflen -= _buffer[0];
@@ -738,6 +803,7 @@ static int uvc_parse_streaming(struct uvc_device *dev,
 		case UVC_VS_FORMAT_MJPEG:
 		case UVC_VS_FORMAT_DV:
 		case UVC_VS_FORMAT_FRAME_BASED:
+		case UVC_VS_FORMAT_VP8:
 			format->frame = frame;
 			ret = uvc_parse_format(dev, streaming, format,
 				&interval, buffer, buflen);
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 109c0a2..88f5e38 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -309,7 +309,7 @@ struct uvc_chain_entry {
 
 struct uvc_frame {
 	__u8  bFrameIndex;
-	__u8  bmCapabilities;
+	__u16 bmCapabilities;
 	__u16 wWidth;
 	__u16 wHeight;
 	__u32 dwMinBitRate;
@@ -318,6 +318,8 @@ struct uvc_frame {
 	__u8  bFrameIntervalType;
 	__u32 dwDefaultFrameInterval;
 	__u32 *dwFrameInterval;
+	__u32 bmSupportedUsages;
+	__u32 bmScalabilityCapabilities;
 };
 
 struct uvc_format {
diff --git a/include/uapi/linux/usb/video.h b/include/uapi/linux/usb/video.h
index eb48ba8..e09c50b 100644
--- a/include/uapi/linux/usb/video.h
+++ b/include/uapi/linux/usb/video.h
@@ -54,6 +54,8 @@
 #define UVC_VS_FORMAT_FRAME_BASED			0x10
 #define UVC_VS_FRAME_FRAME_BASED			0x11
 #define UVC_VS_FORMAT_STREAM_BASED			0x12
+#define UVC_VS_FORMAT_VP8				0x16
+#define UVC_VS_FRAME_VP8				0x17
 
 /* A.7. Video Class-Specific Endpoint Descriptor Subtypes */
 #define UVC_EP_UNDEFINED				0x00
@@ -163,6 +165,8 @@
 
 /* 2.4.3.3. Payload Header Information */
 #define UVC_STREAM_EOH					(1 << 7)
+/* SLI bit replaces EOH for VP8 formats */
+#define UVC_STREAM_SLI					(1 << 7)
 #define UVC_STREAM_ERR					(1 << 6)
 #define UVC_STREAM_STI					(1 << 5)
 #define UVC_STREAM_RES					(1 << 4)
@@ -171,6 +175,10 @@
 #define UVC_STREAM_EOF					(1 << 1)
 #define UVC_STREAM_FID					(1 << 0)
 
+#define UVC_STREAM_VP8_GRF				(1 << 2)
+#define UVC_STREAM_VP8_ARF				(1 << 1)
+#define UVC_STREAM_VP8_PRF				(1 << 0)
+
 /* 4.1.2. Control Capabilities */
 #define UVC_CONTROL_CAP_GET				(1 << 0)
 #define UVC_CONTROL_CAP_SET				(1 << 1)
-- 
1.8.4

[PATCH v1 09/19] uvcvideo: Reorganize uvc_{get,set}_le_value.

[Pawel Osciak]

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_ctrl.c | 62 ++++++++++++++++++++++++----------------
 1 file changed, 37 insertions(+), 25 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index 72d6724..d735c88 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -707,18 +707,12 @@ static inline void uvc_clear_bit(__u8 *data, int bit)
 	data[bit >> 3] &= ~(1 << (bit & 7));
 }
 
-/* Extract the bit string specified by mapping->offset and mapping->size
- * from the little-endian data stored at 'data' and return the result as
- * a signed 32bit integer. Sign extension will be performed if the mapping
- * references a signed data type.
- */
-static __s32 uvc_get_le_value(struct uvc_control_mapping *mapping,
-	__u8 query, const __u8 *data)
+static int __uvc_get_le_value(int bits, int offset, const __u8 *data,
+				__u32 data_type)
 {
-	int bits = mapping->size;
-	int offset = mapping->offset;
 	__s32 value = 0;
 	__u8 mask;
+	int size = bits;
 
 	data += offset / 8;
 	offset &= 7;
@@ -733,22 +727,49 @@ static __s32 uvc_get_le_value(struct uvc_control_mapping *mapping,
 	}
 
 	/* Sign-extend the value if needed. */
-	if (mapping->data_type == UVC_CTRL_DATA_TYPE_SIGNED)
-		value |= -(value & (1 << (mapping->size - 1)));
+	if (data_type == UVC_CTRL_DATA_TYPE_SIGNED)
+		value |= -(value & (1 << (size - 1)));
 
 	return value;
 }
 
+/* Extract the bit string specified by mapping->offset and mapping->size
+ * from the little-endian data stored at 'data' and return the result as
+ * a signed 32bit integer. Sign extension will be performed if the mapping
+ * references a signed data type.
+ */
+static __s32 uvc_get_le_value(struct uvc_control_mapping *mapping,
+	__u8 query, const __u8 *data)
+{
+	return __uvc_get_le_value(mapping->size, mapping->offset, data,
+					mapping->data_type);
+}
+
+static void __uvc_set_le_value(int bits, int offset, __s32 value, __u8 *data,
+				bool keep_existing)
+{
+	__u8 mask;
+
+	data += offset / 8;
+	offset &= 7;
+
+	for (; bits > 0; data++) {
+		mask = ((1LL << bits) - 1) << offset;
+		if (!keep_existing)
+			*data = (*data & ~mask);
+		*data |= ((value << offset) & mask);
+		value >>= (8 - offset);
+		bits -= 8 - offset;
+		offset = 0;
+	}
+}
+
 /* Set the bit string specified by mapping->offset and mapping->size
  * in the little-endian data stored at 'data' to the value 'value'.
  */
 static void uvc_set_le_value(struct uvc_control_mapping *mapping,
 	__s32 value, __u8 *data)
 {
-	int bits = mapping->size;
-	int offset = mapping->offset;
-	__u8 mask;
-
 	/* According to the v4l2 spec, writing any value to a button control
 	 * should result in the action belonging to the button control being
 	 * triggered. UVC devices however want to see a 1 written -> override
@@ -757,16 +778,7 @@ static void uvc_set_le_value(struct uvc_control_mapping *mapping,
 	if (mapping->v4l2_type == V4L2_CTRL_TYPE_BUTTON)
 		value = -1;
 
-	data += offset / 8;
-	offset &= 7;
-
-	for (; bits > 0; data++) {
-		mask = ((1LL << bits) - 1) << offset;
-		*data = (*data & ~mask) | ((value << offset) & mask);
-		value >>= offset ? offset : 8;
-		bits -= 8 - offset;
-		offset = 0;
-	}
+	__uvc_set_le_value(mapping->size, mapping->offset, value, data, false);
 }
 
 /* ------------------------------------------------------------------------
-- 
1.8.4

[PATCH v1 10/19] uvcvideo: Support UVC 1.5 runtime control property.

[Pawel Osciak]

UVC 1.5 introduces the concept of runtime controls, which can be set during
streaming. Non-runtime controls can only be changed while device is idle.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_ctrl.c | 45 +++++++++++++++++++++++++++++++++-------
 drivers/media/usb/uvc/uvc_v4l2.c | 18 ++++++++++------
 drivers/media/usb/uvc/uvcvideo.h | 12 +++++++----
 3 files changed, 57 insertions(+), 18 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index d735c88..b0a19b9 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -1076,8 +1076,19 @@ void __uvc_ctrl_unlock(struct uvc_video_chain *chain)
 	mutex_unlock(&chain->pipeline->ctrl_mutex);
 }
 
+static int uvc_check_ctrl_runtime(struct uvc_control *ctrl, bool streaming)
+{
+	if (streaming && !ctrl->in_runtime) {
+		uvc_trace(UVC_TRACE_CONTROL,
+				"Control disabled while streaming\n");
+		return -EBUSY;
+	}
+
+	return 0;
+}
+
 int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
-	struct v4l2_queryctrl *v4l2_ctrl)
+			struct v4l2_queryctrl *v4l2_ctrl, bool streaming)
 {
 	struct uvc_control *ctrl;
 	struct uvc_control_mapping *mapping;
@@ -1093,6 +1104,10 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
 		goto done;
 	}
 
+	ret = uvc_check_ctrl_runtime(ctrl, streaming);
+	if (ret < 0)
+		goto done;
+
 	ret = __uvc_query_v4l2_ctrl(chain, ctrl, mapping, v4l2_ctrl);
 done:
 	__uvc_ctrl_unlock(chain);
@@ -1109,7 +1124,7 @@ done:
  * manually.
  */
 int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
-	struct v4l2_querymenu *query_menu)
+	struct v4l2_querymenu *query_menu, bool streaming)
 {
 	struct uvc_menu_info *menu_info;
 	struct uvc_control_mapping *mapping;
@@ -1132,6 +1147,10 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
 		goto done;
 	}
 
+	ret = uvc_check_ctrl_runtime(ctrl, streaming);
+	if (ret < 0)
+		goto done;
+
 	if (query_menu->index >= mapping->menu_count) {
 		ret = -EINVAL;
 		goto done;
@@ -1436,21 +1455,26 @@ done:
 	return ret;
 }
 
-int uvc_ctrl_get(struct uvc_video_chain *chain,
-	struct v4l2_ext_control *xctrl)
+int uvc_ctrl_get(struct uvc_video_chain *chain, struct v4l2_ext_control *xctrl,
+		 bool streaming)
 {
 	struct uvc_control *ctrl;
 	struct uvc_control_mapping *mapping;
+	int ret;
 
 	ctrl = uvc_find_control(chain, xctrl->id, &mapping);
 	if (ctrl == NULL)
 		return -EINVAL;
 
+	ret = uvc_check_ctrl_runtime(ctrl, streaming);
+	if (ret < 0)
+		return ret;
+
 	return __uvc_ctrl_get(chain, ctrl, mapping, &xctrl->value);
 }
 
-int uvc_ctrl_set(struct uvc_video_chain *chain,
-	struct v4l2_ext_control *xctrl)
+int uvc_ctrl_set(struct uvc_video_chain *chain, struct v4l2_ext_control *xctrl,
+		 bool streaming)
 {
 	struct uvc_control *ctrl;
 	struct uvc_control_mapping *mapping;
@@ -1466,6 +1490,10 @@ int uvc_ctrl_set(struct uvc_video_chain *chain,
 	if (!(ctrl->info.flags & UVC_CTRL_FLAG_SET_CUR))
 		return -EACCES;
 
+	ret = uvc_check_ctrl_runtime(ctrl, streaming);
+	if (ret < 0)
+		return ret;
+
 	/* Clamp out of range values. */
 	switch (mapping->v4l2_type) {
 	case V4L2_CTRL_TYPE_INTEGER:
@@ -1885,8 +1913,9 @@ static int uvc_ctrl_add_info(struct uvc_device *dev, struct uvc_control *ctrl,
 	ctrl->initialized = 1;
 
 	uvc_trace(UVC_TRACE_CONTROL, "Added control %pUl/%u to device %s "
-		"entity %u\n", ctrl->info.entity, ctrl->info.selector,
-		dev->udev->devpath, ctrl->entity->id);
+		"entity %u, init/runtime %d/%d\n", ctrl->info.entity,
+		ctrl->info.selector, dev->udev->devpath, ctrl->entity->id,
+		ctrl->on_init, ctrl->in_runtime);
 
 done:
 	if (ret < 0)
diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
index a899159..decd65f 100644
--- a/drivers/media/usb/uvc/uvc_v4l2.c
+++ b/drivers/media/usb/uvc/uvc_v4l2.c
@@ -597,7 +597,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 
 	/* Get, Set & Query control */
 	case VIDIOC_QUERYCTRL:
-		return uvc_query_v4l2_ctrl(chain, arg);
+		return uvc_query_v4l2_ctrl(chain, arg,
+					uvc_is_stream_streaming(stream));
 
 	case VIDIOC_G_CTRL:
 	{
@@ -611,7 +612,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (ret < 0)
 			return ret;
 
-		ret = uvc_ctrl_get(chain, &xctrl);
+		ret = uvc_ctrl_get(chain, &xctrl,
+					uvc_is_stream_streaming(stream));
 		uvc_ctrl_rollback(handle);
 		if (ret >= 0)
 			ctrl->value = xctrl.value;
@@ -635,7 +637,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (ret < 0)
 			return ret;
 
-		ret = uvc_ctrl_set(chain, &xctrl);
+		ret = uvc_ctrl_set(chain, &xctrl,
+					uvc_is_stream_streaming(stream));
 		if (ret < 0) {
 			uvc_ctrl_rollback(handle);
 			return ret;
@@ -647,7 +650,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 	}
 
 	case VIDIOC_QUERYMENU:
-		return uvc_query_v4l2_menu(chain, arg);
+		return uvc_query_v4l2_menu(chain, arg,
+					uvc_is_stream_streaming(stream));
 
 	case VIDIOC_G_EXT_CTRLS:
 	{
@@ -660,7 +664,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 			return ret;
 
 		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
-			ret = uvc_ctrl_get(chain, ctrl);
+			ret = uvc_ctrl_get(chain, ctrl,
+					uvc_is_stream_streaming(stream));
 			if (ret < 0) {
 				uvc_ctrl_rollback(handle);
 				ctrls->error_idx = i;
@@ -688,7 +693,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 			return ret;
 
 		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
-			ret = uvc_ctrl_set(chain, ctrl);
+			ret = uvc_ctrl_set(chain, ctrl,
+					uvc_is_stream_streaming(stream));
 			if (ret < 0) {
 				uvc_ctrl_rollback(handle);
 				ctrls->error_idx = cmd == VIDIOC_S_EXT_CTRLS
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 88f5e38..46ffd92 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -694,6 +694,10 @@ extern int uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
 void uvc_video_clock_update(struct uvc_streaming *stream,
 			    struct v4l2_buffer *v4l2_buf,
 			    struct uvc_buffer *buf);
+static inline bool uvc_is_stream_streaming(struct uvc_streaming *stream)
+{
+	return vb2_is_streaming(&stream->queue.queue);
+}
 
 /* Status */
 extern int uvc_status_init(struct uvc_device *dev);
@@ -705,9 +709,9 @@ extern void uvc_status_stop(struct uvc_device *dev);
 extern const struct v4l2_subscribed_event_ops uvc_ctrl_sub_ev_ops;
 
 extern int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
-		struct v4l2_queryctrl *v4l2_ctrl);
+		struct v4l2_queryctrl *v4l2_ctrl, bool streaming);
 extern int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
-		struct v4l2_querymenu *query_menu);
+		struct v4l2_querymenu *query_menu, bool streaming);
 
 extern int uvc_ctrl_add_mapping(struct uvc_video_chain *chain,
 		const struct uvc_control_mapping *mapping);
@@ -731,9 +735,9 @@ static inline int uvc_ctrl_rollback(struct uvc_fh *handle)
 }
 
 extern int uvc_ctrl_get(struct uvc_video_chain *chain,
-		struct v4l2_ext_control *xctrl);
+		struct v4l2_ext_control *xctrl, bool streaming);
 extern int uvc_ctrl_set(struct uvc_video_chain *chain,
-		struct v4l2_ext_control *xctrl);
+		struct v4l2_ext_control *xctrl, bool streaming);
 
 extern int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
 		struct uvc_xu_control_query *xqry);
-- 
1.8.4

[PATCH v1 11/19] uvcvideo: Support V4L2_CTRL_TYPE_BITMASK controls.

[Pawel Osciak]

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_ctrl.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index b0a19b9..a0493d6 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -1550,6 +1550,24 @@ int uvc_ctrl_set(struct uvc_video_chain *chain, struct v4l2_ext_control *xctrl,
 
 		break;
 
+	case V4L2_CTRL_TYPE_BITMASK:
+		value = xctrl->value;
+
+		/* If GET_RES is supported, it will return a bitmask of bits
+		 * that can be set. If it isn't, allow any value.
+		 */
+		if (ctrl->info.flags & UVC_CTRL_FLAG_GET_RES) {
+			if (!ctrl->cached) {
+				ret = uvc_ctrl_populate_cache(chain, ctrl);
+				if (ret < 0)
+					return ret;
+			}
+			step = mapping->get(mapping, UVC_GET_RES,
+					uvc_ctrl_data(ctrl, UVC_CTRL_DATA_RES));
+			if (value & ~step)
+				return -ERANGE;
+		}
+
 	default:
 		value = xctrl->value;
 		break;
-- 
1.8.4

[PATCH v1 12/19] uvcvideo: Reorganize next buffer handling.

[Pawel Osciak]

Move getting the first buffer from the current queue to a uvc_queue function
and out of the USB completion handler.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_isight.c |  6 ++++--
 drivers/media/usb/uvc/uvc_queue.c  | 14 ++++++++++++++
 drivers/media/usb/uvc/uvc_video.c  | 29 ++++++++++++-----------------
 drivers/media/usb/uvc/uvcvideo.h   |  7 +++----
 4 files changed, 33 insertions(+), 23 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_isight.c b/drivers/media/usb/uvc/uvc_isight.c
index 8510e72..ab01286 100644
--- a/drivers/media/usb/uvc/uvc_isight.c
+++ b/drivers/media/usb/uvc/uvc_isight.c
@@ -99,10 +99,12 @@ static int isight_decode(struct uvc_video_queue *queue, struct uvc_buffer *buf,
 	return 0;
 }
 
-void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
-		struct uvc_buffer *buf)
+void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream)
 {
 	int ret, i;
+	struct uvc_buffer *buf;
+
+	buf = uvc_queue_get_first_buf(&stream->queue);
 
 	for (i = 0; i < urb->number_of_packets; ++i) {
 		if (urb->iso_frame_desc[i].status < 0) {
diff --git a/drivers/media/usb/uvc/uvc_queue.c b/drivers/media/usb/uvc/uvc_queue.c
index cd962be..55d2670 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -352,6 +352,20 @@ void uvc_queue_cancel(struct uvc_video_queue *queue, int disconnect)
 	spin_unlock_irqrestore(&queue->irqlock, flags);
 }
 
+struct uvc_buffer *uvc_queue_get_first_buf(struct uvc_video_queue *queue)
+{
+	struct uvc_buffer *buf = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&queue->irqlock, flags);
+	if (!list_empty(&queue->irqqueue))
+		buf = list_first_entry(&queue->irqqueue, struct uvc_buffer,
+					queue);
+	spin_unlock_irqrestore(&queue->irqlock, flags);
+
+	return buf;
+}
+
 struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
 		struct uvc_buffer *buf)
 {
diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index b4ebccd..2f9a5fa 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1193,11 +1193,11 @@ static int uvc_video_encode_data(struct uvc_streaming *stream,
 /*
  * Completion handler for video URBs.
  */
-static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 {
 	u8 *mem;
 	int ret, i;
+	struct uvc_buffer *buf = NULL;
 
 	for (i = 0; i < urb->number_of_packets; ++i) {
 		if (urb->iso_frame_desc[i].status < 0) {
@@ -1211,6 +1211,7 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
 
 		/* Decode the payload header. */
 		mem = urb->transfer_buffer + urb->iso_frame_desc[i].offset;
+		buf = uvc_queue_get_first_buf(&stream->queue);
 		do {
 			ret = uvc_video_decode_start(stream, buf, mem,
 				urb->iso_frame_desc[i].actual_length);
@@ -1241,11 +1242,11 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream,
 	}
 }
 
-static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 {
 	u8 *mem;
 	int len, ret;
+	struct uvc_buffer *buf;
 
 	/*
 	 * Ignore ZLPs if they're not part of a frame, otherwise process them
@@ -1258,6 +1259,8 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 	len = urb->actual_length;
 	stream->bulk.payload_size += len;
 
+	buf = uvc_queue_get_first_buf(&stream->queue);
+
 	/* If the URB is the first of its payload, decode and save the
 	 * header.
 	 */
@@ -1309,12 +1312,13 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream,
 	}
 }
 
-static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming *stream,
-	struct uvc_buffer *buf)
+static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming *stream)
 {
 	u8 *mem = urb->transfer_buffer;
 	int len = stream->urb_size, ret;
+	struct uvc_buffer *buf;
 
+	buf = uvc_queue_get_first_buf(&stream->queue);
 	if (buf == NULL) {
 		urb->transfer_buffer_length = 0;
 		return;
@@ -1355,9 +1359,6 @@ static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming *stream,
 static void uvc_video_complete(struct urb *urb)
 {
 	struct uvc_streaming *stream = urb->context;
-	struct uvc_video_queue *queue = &stream->queue;
-	struct uvc_buffer *buf = NULL;
-	unsigned long flags;
 	int ret;
 
 	switch (urb->status) {
@@ -1374,17 +1375,11 @@ static void uvc_video_complete(struct urb *urb)
 
 	case -ECONNRESET:	/* usb_unlink_urb() called. */
 	case -ESHUTDOWN:	/* The endpoint is being disabled. */
-		uvc_queue_cancel(queue, urb->status == -ESHUTDOWN);
+		uvc_queue_cancel(&stream->queue, urb->status == -ESHUTDOWN);
 		return;
 	}
 
-	spin_lock_irqsave(&queue->irqlock, flags);
-	if (!list_empty(&queue->irqqueue))
-		buf = list_first_entry(&queue->irqqueue, struct uvc_buffer,
-				       queue);
-	spin_unlock_irqrestore(&queue->irqlock, flags);
-
-	stream->decode(urb, stream, buf);
+	stream->decode(urb, stream);
 
 	if ((ret = usb_submit_urb(urb, GFP_ATOMIC)) < 0) {
 		uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index 46ffd92..bca8715 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -482,8 +482,7 @@ struct uvc_streaming {
 	/* Buffers queue. */
 	unsigned int frozen : 1;
 	struct uvc_video_queue queue;
-	void (*decode) (struct urb *urb, struct uvc_streaming *video,
-			struct uvc_buffer *buf);
+	void (*decode) (struct urb *urb, struct uvc_streaming *video);
 
 	/* Context data used by the bulk completion handler. */
 	struct {
@@ -659,6 +658,7 @@ extern int uvc_dequeue_buffer(struct uvc_video_queue *queue,
 		struct v4l2_buffer *v4l2_buf, int nonblocking);
 extern int uvc_queue_enable(struct uvc_video_queue *queue, int enable);
 extern void uvc_queue_cancel(struct uvc_video_queue *queue, int disconnect);
+struct uvc_buffer *uvc_queue_get_first_buf(struct uvc_video_queue *queue);
 extern struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
 		struct uvc_buffer *buf);
 extern int uvc_queue_mmap(struct uvc_video_queue *queue,
@@ -751,8 +751,7 @@ extern struct usb_host_endpoint *uvc_find_endpoint(
 		struct usb_host_interface *alts, __u8 epaddr);
 
 /* Quirks support */
-void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
-		struct uvc_buffer *buf);
+void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream);
 
 /* debugfs and statistics */
 int uvc_debugfs_init(void);
-- 
1.8.4

[PATCH v1 13/19] uvcvideo: Unify UVC payload header parsing.

[Pawel Osciak]

Create a separate function for parsing UVC payload headers and extract code
from other functions into it. Store the parsed values in a header struct.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_video.c | 270 +++++++++++++++++++-------------------
 drivers/media/usb/uvc/uvcvideo.h  |  21 +++
 2 files changed, 157 insertions(+), 134 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 2f9a5fa..59f57a2 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -422,40 +422,14 @@ static int uvc_commit_video(struct uvc_streaming *stream,
 
 static void
 uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf,
-		       const __u8 *data, int len)
+			struct uvc_payload_header *header)
 {
 	struct uvc_clock_sample *sample;
-	unsigned int header_size;
-	bool has_pts = false;
-	bool has_scr = false;
 	unsigned long flags;
 	struct timespec ts;
 	u16 host_sof;
 	u16 dev_sof;
 
-	switch (data[1] & (UVC_STREAM_PTS | UVC_STREAM_SCR)) {
-	case UVC_STREAM_PTS | UVC_STREAM_SCR:
-		header_size = 12;
-		has_pts = true;
-		has_scr = true;
-		break;
-	case UVC_STREAM_PTS:
-		header_size = 6;
-		has_pts = true;
-		break;
-	case UVC_STREAM_SCR:
-		header_size = 8;
-		has_scr = true;
-		break;
-	default:
-		header_size = 2;
-		break;
-	}
-
-	/* Check for invalid headers. */
-	if (len < header_size)
-		return;
-
 	/* Extract the timestamps:
 	 *
 	 * - store the frame PTS in the buffer structure
@@ -463,17 +437,17 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf,
 	 *   kernel timestamps and store them with the SCR STC and SOF fields
 	 *   in the ring buffer
 	 */
-	if (has_pts && buf != NULL)
-		buf->pts = get_unaligned_le32(&data[2]);
+	if (header->has_pts && buf != NULL)
+		buf->pts = header->pts;
 
-	if (!has_scr)
+	if (!header->has_scr)
 		return;
 
 	/* To limit the amount of data, drop SCRs with an SOF identical to the
 	 * previous one.
 	 */
-	dev_sof = get_unaligned_le16(&data[header_size - 2]);
-	if (dev_sof == stream->clock.last_sof)
+	dev_sof = header->sof;
+	if (dev_sof <= stream->clock.last_sof)
 		return;
 
 	stream->clock.last_sof = dev_sof;
@@ -513,7 +487,7 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf,
 	spin_lock_irqsave(&stream->clock.lock, flags);
 
 	sample = &stream->clock.samples[stream->clock.head];
-	sample->dev_stc = get_unaligned_le32(&data[header_size - 6]);
+	sample->dev_stc = header->stc;
 	sample->dev_sof = dev_sof;
 	sample->host_sof = host_sof;
 	sample->host_ts = ts;
@@ -756,114 +730,74 @@ done:
  */
 
 static void uvc_video_stats_decode(struct uvc_streaming *stream,
-		const __u8 *data, int len)
+				    struct uvc_payload_header *header)
 {
-	unsigned int header_size;
-	bool has_pts = false;
-	bool has_scr = false;
-	u16 uninitialized_var(scr_sof);
-	u32 uninitialized_var(scr_stc);
-	u32 uninitialized_var(pts);
-
 	if (stream->stats.stream.nb_frames == 0 &&
 	    stream->stats.frame.nb_packets == 0)
 		ktime_get_ts(&stream->stats.stream.start_ts);
 
-	switch (data[1] & (UVC_STREAM_PTS | UVC_STREAM_SCR)) {
-	case UVC_STREAM_PTS | UVC_STREAM_SCR:
-		header_size = 12;
-		has_pts = true;
-		has_scr = true;
-		break;
-	case UVC_STREAM_PTS:
-		header_size = 6;
-		has_pts = true;
-		break;
-	case UVC_STREAM_SCR:
-		header_size = 8;
-		has_scr = true;
-		break;
-	default:
-		header_size = 2;
-		break;
-	}
-
-	/* Check for invalid headers. */
-	if (len < header_size || data[0] < header_size) {
-		stream->stats.frame.nb_invalid++;
-		return;
-	}
-
-	/* Extract the timestamps. */
-	if (has_pts)
-		pts = get_unaligned_le32(&data[2]);
-
-	if (has_scr) {
-		scr_stc = get_unaligned_le32(&data[header_size - 6]);
-		scr_sof = get_unaligned_le16(&data[header_size - 2]);
-	}
-
 	/* Is PTS constant through the whole frame ? */
-	if (has_pts && stream->stats.frame.nb_pts) {
-		if (stream->stats.frame.pts != pts) {
+	if (header->has_pts && stream->stats.frame.nb_pts) {
+		if (stream->stats.frame.pts != header->pts) {
 			stream->stats.frame.nb_pts_diffs++;
 			stream->stats.frame.last_pts_diff =
 				stream->stats.frame.nb_packets;
 		}
 	}
 
-	if (has_pts) {
+	if (header->has_pts) {
 		stream->stats.frame.nb_pts++;
-		stream->stats.frame.pts = pts;
+		stream->stats.frame.pts = header->pts;
 	}
 
 	/* Do all frames have a PTS in their first non-empty packet, or before
 	 * their first empty packet ?
 	 */
 	if (stream->stats.frame.size == 0) {
-		if (len > header_size)
-			stream->stats.frame.has_initial_pts = has_pts;
-		if (len == header_size && has_pts)
+		if (header->payload_size > 0)
+			stream->stats.frame.has_initial_pts = header->has_pts;
+		if (header->payload_size == 0 && header->has_pts)
 			stream->stats.frame.has_early_pts = true;
 	}
 
 	/* Do the SCR.STC and SCR.SOF fields vary through the frame ? */
-	if (has_scr && stream->stats.frame.nb_scr) {
-		if (stream->stats.frame.scr_stc != scr_stc)
+	if (header->has_scr && stream->stats.frame.nb_scr) {
+		if (stream->stats.frame.scr_stc != header->stc)
 			stream->stats.frame.nb_scr_diffs++;
 	}
 
-	if (has_scr) {
+	if (header->has_scr) {
 		/* Expand the SOF counter to 32 bits and store its value. */
 		if (stream->stats.stream.nb_frames > 0 ||
 		    stream->stats.frame.nb_scr > 0)
 			stream->stats.stream.scr_sof_count +=
-				(scr_sof - stream->stats.stream.scr_sof) % 2048;
-		stream->stats.stream.scr_sof = scr_sof;
+				(header->sof - stream->stats.stream.scr_sof)
+				% 2048;
+		stream->stats.stream.scr_sof = header->sof;
 
 		stream->stats.frame.nb_scr++;
-		stream->stats.frame.scr_stc = scr_stc;
-		stream->stats.frame.scr_sof = scr_sof;
+		stream->stats.frame.scr_stc = header->stc;
+		stream->stats.frame.scr_sof = header->sof;
 
-		if (scr_sof < stream->stats.stream.min_sof)
-			stream->stats.stream.min_sof = scr_sof;
-		if (scr_sof > stream->stats.stream.max_sof)
-			stream->stats.stream.max_sof = scr_sof;
+		if (header->sof < stream->stats.stream.min_sof)
+			stream->stats.stream.min_sof = header->sof;
+		if (header->sof > stream->stats.stream.max_sof)
+			stream->stats.stream.max_sof = header->sof;
 	}
 
 	/* Record the first non-empty packet number. */
-	if (stream->stats.frame.size == 0 && len > header_size)
+	if (stream->stats.frame.size == 0 && header->payload_size > 0)
 		stream->stats.frame.first_data = stream->stats.frame.nb_packets;
 
 	/* Update the frame size. */
-	stream->stats.frame.size += len - header_size;
+	stream->stats.frame.size += header->payload_size;
 
 	/* Update the packets counters. */
 	stream->stats.frame.nb_packets++;
-	if (len > header_size)
+	if (header->payload_size == 0)
 		stream->stats.frame.nb_empty++;
 
-	if (data[1] & UVC_STREAM_ERR)
+	if (header->has_err)
 		stream->stats.frame.nb_errors++;
 }
 
@@ -1006,21 +940,9 @@ static void uvc_video_stats_stop(struct uvc_streaming *stream)
  * uvc_video_decode_end will never be called with a NULL buffer.
  */
 static int uvc_video_decode_start(struct uvc_streaming *stream,
-		struct uvc_buffer *buf, const __u8 *data, int len)
+		struct uvc_buffer *buf, struct uvc_payload_header *header)
 {
-	__u8 fid;
-
-	/* Sanity checks:
-	 * - packet must be at least 2 bytes long
-	 * - bHeaderLength value must be at least 2 bytes (see above)
-	 * - bHeaderLength value can't be larger than the packet size.
-	 */
-	if (len < 2 || data[0] < 2 || data[0] > len) {
-		stream->stats.frame.nb_invalid++;
-		return -EINVAL;
-	}
-
-	fid = data[1] & UVC_STREAM_FID;
+	u8 fid = header->fid;
 
 	/* Increase the sequence number regardless of any buffer states, so
 	 * that discontinuous sequence numbers always indicate lost frames.
@@ -1031,8 +953,8 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 			uvc_video_stats_update(stream);
 	}
 
-	uvc_video_clock_decode(stream, buf, data, len);
-	uvc_video_stats_decode(stream, data, len);
+	uvc_video_clock_decode(stream, buf, header);
+	uvc_video_stats_decode(stream, header);
 
 	/* Store the payload FID bit and return immediately when the buffer is
 	 * NULL.
@@ -1043,7 +965,7 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 	}
 
 	/* Mark the buffer as bad if the error bit is set. */
-	if (data[1] & UVC_STREAM_ERR) {
+	if (header->has_err) {
 		uvc_trace(UVC_TRACE_FRAME, "Marking buffer as bad (error bit "
 			  "set).\n");
 		buf->error = 1;
@@ -1064,7 +986,7 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 			uvc_trace(UVC_TRACE_FRAME, "Dropping payload (out of "
 				"sync).\n");
 			if ((stream->dev->quirks & UVC_QUIRK_STREAM_NO_FID) &&
-			    (data[1] & UVC_STREAM_EOF))
+			    (header->has_eof))
 				stream->last_fid ^= UVC_STREAM_FID;
 			return -ENODATA;
 		}
@@ -1107,7 +1029,7 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 
 	stream->last_fid = fid;
 
-	return data[0];
+	return 0;
 }
 
 static void uvc_video_decode_data(struct uvc_streaming *stream,
@@ -1128,18 +1050,20 @@ static void uvc_video_decode_data(struct uvc_streaming *stream,
 
 	/* Complete the current frame if the buffer size was exceeded. */
 	if (len > maxlen) {
-		uvc_trace(UVC_TRACE_FRAME, "Frame complete (overflow).\n");
+		uvc_trace(UVC_TRACE_FRAME, "Frame complete (overflow) "
+				"len=%d, buffer size=%d used=%d\n",
+				len, buf->length, buf->bytesused);
 		buf->state = UVC_BUF_STATE_READY;
 	}
 }
 
 static void uvc_video_decode_end(struct uvc_streaming *stream,
-		struct uvc_buffer *buf, const __u8 *data, int len)
+		struct uvc_buffer *buf, struct uvc_payload_header *header)
 {
 	/* Mark the buffer as done if the EOF marker is set. */
-	if (data[1] & UVC_STREAM_EOF && buf->bytesused != 0) {
+	if (header->has_eof && buf->bytesused != 0) {
 		uvc_trace(UVC_TRACE_FRAME, "Frame complete (EOF found).\n");
-		if (data[0] == len)
+		if (header->payload_size == 0)
 			uvc_trace(UVC_TRACE_FRAME, "EOF in empty payload.\n");
 		buf->state = UVC_BUF_STATE_READY;
 		if (stream->dev->quirks & UVC_QUIRK_STREAM_NO_FID)
@@ -1186,6 +1110,75 @@ static int uvc_video_encode_data(struct uvc_streaming *stream,
 	return nbytes;
 }
 
+static int uvc_video_parse_header(struct uvc_streaming *stream,
+		const __u8 *data, int len, struct uvc_payload_header *header)
+{
+	int off = 2;
+
+	/* Sanity checks:
+	 * - packet must be at least 2 bytes long
+	 * - bHeaderLength value must be at least 2 bytes (see above)
+	 */
+	if (len < 2 || data[0] < 2)
+		goto error;
+
+	header->length = 2; /* 1 byte of header length + 1 byte of BFH. */
+
+	header->has_sli = false;
+	header->has_eof = data[1] & UVC_STREAM_EOF;
+	header->has_pts = data[1] & UVC_STREAM_PTS;
+	header->has_scr = data[1] & UVC_STREAM_SCR;
+	header->has_err = data[1] & UVC_STREAM_ERR;
+
+	if (header->has_pts)
+		header->length += 4;
+
+	if (header->has_scr)
+		header->length += 6;
+
+	if (stream->cur_format->fcc == V4L2_PIX_FMT_VP8) {
+		/* VP8 payload has 2 additional bytes of BFH. */
+		header->length += 2;
+		off += 2;
+
+		/* SLI always present for VP8 simulcast (at the end of header),
+		 * allowed for VP8 non-simulcast.
+		 */
+		header->has_sli = data[1] & UVC_STREAM_SLI;
+		if (header->has_sli)
+			header->length += 2;
+	}
+
+	/* - bHeaderLength value can't be larger than the packet size. */
+	if (len < data[0] || data[0] != header->length)
+		goto error;
+
+	/* PTS 4 bytes, STC 4 bytes, SOF 2 bytes. */
+	if (header->has_pts) {
+		header->pts = get_unaligned_le32(&data[off]);
+		off += 4;
+	}
+
+	if (header->has_scr) {
+		header->stc = get_unaligned_le32(&data[off]);
+		off += 4;
+		header->sof = get_unaligned_le16(&data[off]);
+		off += 2;
+	}
+
+	if (header->has_sli)
+		header->sli = get_unaligned_le16(&data[off]);
+
+	header->payload_size = len - header->length;
+	header->fid = data[1] & UVC_STREAM_FID;
+
+	return 0;
+
+error:
+	stream->stats.frame.nb_invalid++;
+	return -EINVAL;
+}
+
 /* ------------------------------------------------------------------------
  * URB handling
  */
@@ -1195,9 +1188,11 @@ static int uvc_video_encode_data(struct uvc_streaming *stream,
  */
 static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 {
+	unsigned int len;
 	u8 *mem;
 	int ret, i;
 	struct uvc_buffer *buf = NULL;
+	struct uvc_payload_header header;
 
 	for (i = 0; i < urb->number_of_packets; ++i) {
 		if (urb->iso_frame_desc[i].status < 0) {
@@ -1209,12 +1204,16 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 			continue;
 		}
 
-		/* Decode the payload header. */
 		mem = urb->transfer_buffer + urb->iso_frame_desc[i].offset;
+		len = urb->iso_frame_desc[i].actual_length;
+
+		ret = uvc_video_parse_header(stream, mem, len, &header);
+		if (ret < 0)
+			continue;
+
 		buf = uvc_queue_get_first_buf(&stream->queue);
 		do {
-			ret = uvc_video_decode_start(stream, buf, mem,
-				urb->iso_frame_desc[i].actual_length);
+			ret = uvc_video_decode_start(stream, buf, &header);
 			if (ret == -EAGAIN)
 				buf = uvc_queue_next_buffer(&stream->queue,
 							    buf);
@@ -1224,12 +1223,11 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 			continue;
 
 		/* Decode the payload data. */
-		uvc_video_decode_data(stream, buf, mem + ret,
-			urb->iso_frame_desc[i].actual_length - ret);
+		uvc_video_decode_data(stream, buf, mem + header.length,
+			urb->iso_frame_desc[i].actual_length - header.length);
 
 		/* Process the header again. */
-		uvc_video_decode_end(stream, buf, mem,
-			urb->iso_frame_desc[i].actual_length);
+		uvc_video_decode_end(stream, buf, &header);
 
 		if (buf->state == UVC_BUF_STATE_READY) {
 			if (buf->length != buf->bytesused &&
@@ -1246,6 +1244,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 {
 	u8 *mem;
 	int len, ret;
+	struct uvc_payload_header header;
 	struct uvc_buffer *buf;
 
 	/*
@@ -1259,6 +1258,10 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 	len = urb->actual_length;
 	stream->bulk.payload_size += len;
 
+	ret = uvc_video_parse_header(stream, mem, len, &header);
+	if (ret < 0)
+		return;
+
 	buf = uvc_queue_get_first_buf(&stream->queue);
 
 	/* If the URB is the first of its payload, decode and save the
@@ -1266,7 +1269,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 	 */
 	if (stream->bulk.header_size == 0 && !stream->bulk.skip_payload) {
 		do {
-			ret = uvc_video_decode_start(stream, buf, mem, len);
+			ret = uvc_video_decode_start(stream, buf, &header);
 			if (ret == -EAGAIN)
 				buf = uvc_queue_next_buffer(&stream->queue,
 							    buf);
@@ -1276,11 +1279,11 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 		if (ret < 0 || buf == NULL) {
 			stream->bulk.skip_payload = 1;
 		} else {
-			memcpy(stream->bulk.header, mem, ret);
-			stream->bulk.header_size = ret;
+			memcpy(stream->bulk.header, mem, header.length);
+			stream->bulk.header_size = header.length;
 
-			mem += ret;
-			len -= ret;
+			mem += header.length;
+			len -= header.length;
 		}
 	}
 
@@ -1299,8 +1302,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 	if (urb->actual_length < urb->transfer_buffer_length ||
 	    stream->bulk.payload_size >= stream->bulk.max_payload_size) {
 		if (!stream->bulk.skip_payload && buf != NULL) {
-			uvc_video_decode_end(stream, buf, stream->bulk.header,
-				stream->bulk.payload_size);
+			uvc_video_decode_end(stream, buf, &header);
 			if (buf->state == UVC_BUF_STATE_READY)
 				buf = uvc_queue_next_buffer(&stream->queue,
 							    buf);
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index bca8715..b355b2c 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -453,6 +453,27 @@ struct uvc_stats_stream {
 	unsigned int max_sof;		/* Maximum STC.SOF value */
 };
 
+struct uvc_payload_header {
+	bool has_eof;
+
+	bool has_pts;
+	u32 pts;
+
+	bool has_scr;
+	u16 sof;
+	u32 stc;
+
+	bool has_sli;
+	u16 sli;
+
+	u8 fid;
+
+	bool has_err;
+
+	int length;
+	int payload_size;
+};
+
 struct uvc_streaming {
 	struct list_head list;
 	struct uvc_device *dev;
-- 
1.8.4

[PATCH v1 14/19] v4l: Add v4l2_buffer flags for VP8-specific special frames.

[Pawel Osciak]

Add bits for previous, golden and altref frame types.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 include/uapi/linux/videodev2.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 437f1b0..c011ee0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -687,6 +687,10 @@ struct v4l2_buffer {
 #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x0000
 #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x2000
 #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x4000
+/* VP8 special frames */
+#define V4L2_BUF_FLAG_PREV_FRAME		0x10000  /* VP8 prev frame */
+#define V4L2_BUF_FLAG_GOLDEN_FRAME		0x20000  /* VP8 golden frame */
+#define V4L2_BUF_FLAG_ALTREF_FRAME		0x40000  /* VP8 altref frame */
 
 /**
  * struct v4l2_exportbuffer - export of video buffer as DMABUF file descriptor
-- 
1.8.4

[PATCH v1 15/19] uvcvideo: Add support for VP8 special frame flags.

[Pawel Osciak]

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_video.c | 18 +++++++++++++++++-
 drivers/media/usb/uvc/uvcvideo.h  | 10 ++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 59f57a2..0291817 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -1136,6 +1136,8 @@ static int uvc_video_parse_header(struct uvc_streaming *stream,
 	if (header->has_scr)
 		header->length += 6;
 
+	header->buf_flags = 0;
+
 	if (stream->cur_format->fcc == V4L2_PIX_FMT_VP8) {
 		/* VP8 payload has 2 additional bytes of BFH. */
 		header->length += 2;
@@ -1147,6 +1149,16 @@ static int uvc_video_parse_header(struct uvc_streaming *stream,
 		header->has_sli = data[1] & UVC_STREAM_SLI;
 		if (header->has_sli)
 			header->length += 2;
+
+		/* Codec-specific flags for v4l2_buffer. */
+		header->buf_flags |= (data[1] & UVC_STREAM_STI) ?
+					V4L2_BUF_FLAG_KEYFRAME : 0;
+		header->buf_flags |= (data[2] & UVC_STREAM_VP8_PRF) ?
+					V4L2_BUF_FLAG_PREV_FRAME : 0;
+		header->buf_flags |= (data[2] & UVC_STREAM_VP8_ARF) ?
+					V4L2_BUF_FLAG_ALTREF_FRAME : 0;
+		header->buf_flags |= (data[2] & UVC_STREAM_VP8_GRF) ?
+					V4L2_BUF_FLAG_GOLDEN_FRAME : 0;
 	}
 
 	/* - bHeaderLength value can't be larger than the packet size. */
@@ -1222,6 +1234,8 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 		if (ret < 0)
 			continue;
 
+		buf->buf.v4l2_buf.flags |= header.buf_flags;
+
 		/* Decode the payload data. */
 		uvc_video_decode_data(stream, buf, mem + header.length,
 			urb->iso_frame_desc[i].actual_length - header.length);
@@ -1293,8 +1307,10 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 	 */
 
 	/* Process video data. */
-	if (!stream->bulk.skip_payload && buf != NULL)
+	if (!stream->bulk.skip_payload && buf != NULL) {
 		uvc_video_decode_data(stream, buf, mem, len);
+		buf->buf.v4l2_buf.flags |= header.buf_flags;
+	}
 
 	/* Detect the payload end by a URB smaller than the maximum size (or
 	 * a payload size equal to the maximum) and process the header again.
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index b355b2c..fb21459 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -145,6 +145,14 @@
 #define UVC_FMT_FLAG_COMPRESSED		0x00000001
 #define UVC_FMT_FLAG_STREAM		0x00000002
 
+/* v4l2_buffer codec flags */
+#define UVC_V4L2_BUFFER_CODEC_FLAGS	(V4L2_BUF_FLAG_KEYFRAME | \
+					 V4L2_BUF_FLAG_PFRAME | \
+					 V4L2_BUF_FLAG_BFRAME | \
+					 V4L2_BUF_FLAG_PREV_FRAME | \
+					 V4L2_BUF_FLAG_GOLDEN_FRAME | \
+					 V4L2_BUF_FLAG_ALTREF_FRAME)
+
 /* ------------------------------------------------------------------------
  * Structures.
  */
@@ -472,6 +480,8 @@ struct uvc_payload_header {
 
 	int length;
 	int payload_size;
+
+	__u32 buf_flags; /* v4l2_buffer flags */
 };
 
 struct uvc_streaming {
-- 
1.8.4

[PATCH v1 16/19] v4l: Add encoding camera controls.

[Pawel Osciak]

Add defines for controls found in UVC 1.5 encoding cameras.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 29 +++++++++++++++++++++++++++++
 include/uapi/linux/v4l2-controls.h   | 31 +++++++++++++++++++++++++++++++
 2 files changed, 60 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index c3f0803..0b3a632 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -781,6 +781,35 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_AUTO_FOCUS_STATUS:	return "Auto Focus, Status";
 	case V4L2_CID_AUTO_FOCUS_RANGE:		return "Auto Focus, Range";
 
+	case V4L2_CID_ENCODER_MIN_FRAME_INTERVAL: return "Encoder, min. frame interval";
+	case V4L2_CID_ENCODER_RATE_CONTROL_MODE: return "Encoder, rate control mode";
+	case V4L2_CID_ENCODER_AVERAGE_BITRATE:	return "Encoder, average bitrate";
+	case V4L2_CID_ENCODER_CPB_SIZE:		return "Encoder, CPB size";
+	case V4L2_CID_ENCODER_PEAK_BIT_RATE:	return "Encoder, peak bit rate";
+	case V4L2_CID_ENCODER_QP_PARAM_I:	return "Encoder, QP param for I frames";
+	case V4L2_CID_ENCODER_QP_PARAM_P:	return "Encoder, QP param for P frames";
+	case V4L2_CID_ENCODER_QP_PARAM_BG:	return "Encoder, QP param for B/G frames";
+	case V4L2_CID_ENCODER_NUM_GDR_FRAMES:	return "Encoder, number of GDR frames";
+	case V4L2_CID_ENCODER_LTR_BUFFER_CONTROL: return "Encoder, LTR buffer control";
+	case V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE: return "Encoder, LTR buffer trust mode";
+	case V4L2_CID_ENCODER_LTR_PICTURE_POSITION: return "Encoder, LTR picture position";
+	case V4L2_CID_ENCODER_LTR_PICTURE_MODE:	return "Encoder, LTR picture mode";
+	case V4L2_CID_ENCODER_LTR_VALIDATION:	return "Encoder, LTR validation";
+	case V4L2_CID_ENCODER_MIN_QP:		return "Encoder, minimum QP param";
+	case V4L2_CID_ENCODER_MAX_QP:		return "Encoder, maximum QP param";
+	case V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL: return "Encoder, sync frame interval";
+	case V4L2_CID_ENCODER_ERROR_RESILIENCY:	return "Encoder, error resiliency";
+	case V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE: return "Encoder, temporal layer enable";
+
+	case V4L2_CID_ENCODER_VP8_SLICE_MODE:	return "Encoder, VP8 slice mode";
+	case V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE: return "Encoder, VP8 sync frame type";
+	case V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME: return "Encoder, VP8, DCT partitions per frame";
+
+	case V4L2_CID_ENCODER_H264_PROFILE_TOOLSET: return "Encoder, H.264 profile and toolset";
+	case V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT: return "Encoder, H.264 level IDC limit";
+	case V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE: return "Encoder, H.264 SEI payload type";
+	case V4L2_CID_ENCODER_H264_LAYER_PRIORITY: return "Encoder, H.264 layer priority";
+
 	/* FM Radio Modulator control */
 	/* Keep the order of the 'case's the same as in videodev2.h! */
 	case V4L2_CID_FM_TX_CLASS:		return "FM Radio Modulator Controls";
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 083bb5a..ef3a30d 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -729,6 +729,37 @@ enum v4l2_auto_focus_range {
 	V4L2_AUTO_FOCUS_RANGE_INFINITY		= 3,
 };
 
+/* Controls found in UVC 1.5 encoding cameras */
+#define V4L2_CID_ENCODER_MIN_FRAME_INTERVAL	(V4L2_CID_CAMERA_CLASS_BASE+32)
+#define V4L2_CID_ENCODER_RATE_CONTROL_MODE	(V4L2_CID_CAMERA_CLASS_BASE+33)
+#define V4L2_CID_ENCODER_AVERAGE_BITRATE	(V4L2_CID_CAMERA_CLASS_BASE+34)
+#define V4L2_CID_ENCODER_CPB_SIZE		(V4L2_CID_CAMERA_CLASS_BASE+35)
+#define V4L2_CID_ENCODER_PEAK_BIT_RATE		(V4L2_CID_CAMERA_CLASS_BASE+36)
+#define V4L2_CID_ENCODER_QP_PARAM_I		(V4L2_CID_CAMERA_CLASS_BASE+37)
+#define V4L2_CID_ENCODER_QP_PARAM_P		(V4L2_CID_CAMERA_CLASS_BASE+38)
+#define V4L2_CID_ENCODER_QP_PARAM_BG		(V4L2_CID_CAMERA_CLASS_BASE+39)
+#define V4L2_CID_ENCODER_NUM_GDR_FRAMES		(V4L2_CID_CAMERA_CLASS_BASE+40)
+#define V4L2_CID_ENCODER_LTR_BUFFER_CONTROL	(V4L2_CID_CAMERA_CLASS_BASE+41)
+#define V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE	(V4L2_CID_CAMERA_CLASS_BASE+42)
+#define V4L2_CID_ENCODER_LTR_PICTURE_POSITION	(V4L2_CID_CAMERA_CLASS_BASE+43)
+#define V4L2_CID_ENCODER_LTR_PICTURE_MODE	(V4L2_CID_CAMERA_CLASS_BASE+44)
+#define V4L2_CID_ENCODER_LTR_VALIDATION		(V4L2_CID_CAMERA_CLASS_BASE+45)
+#define V4L2_CID_ENCODER_MIN_QP			(V4L2_CID_CAMERA_CLASS_BASE+46)
+#define V4L2_CID_ENCODER_MAX_QP			(V4L2_CID_CAMERA_CLASS_BASE+47)
+#define V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL	(V4L2_CID_CAMERA_CLASS_BASE+48)
+#define V4L2_CID_ENCODER_ERROR_RESILIENCY	(V4L2_CID_CAMERA_CLASS_BASE+49)
+#define V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE	(V4L2_CID_CAMERA_CLASS_BASE+50)
+
+/* VP8-specific controls */
+#define V4L2_CID_ENCODER_VP8_SLICE_MODE		(V4L2_CID_CAMERA_CLASS_BASE+51)
+#define V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME (V4L2_CID_CAMERA_CLASS_BASE+52)
+#define V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE	(V4L2_CID_CAMERA_CLASS_BASE+53)
+
+/* H.264-specific controls */
+#define V4L2_CID_ENCODER_H264_PROFILE_TOOLSET	(V4L2_CID_CAMERA_CLASS_BASE+54)
+#define V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT	(V4L2_CID_CAMERA_CLASS_BASE+55)
+#define V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE	(V4L2_CID_CAMERA_CLASS_BASE+56)
+#define V4L2_CID_ENCODER_H264_LAYER_PRIORITY	(V4L2_CID_CAMERA_CLASS_BASE+57)
 
 /* FM Modulator class control IDs */
 
-- 
1.8.4

[PATCH v1 17/19] uvcvideo: Add UVC 1.5 Encoding Unit controls.

[Pawel Osciak]

These controls allow modifying encoding parameters.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_ctrl.c | 445 +++++++++++++++++++++++++++++++++++++++
 include/uapi/linux/usb/video.h   |  23 ++
 2 files changed, 468 insertions(+)

diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index a0493d6..cd02c99 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -351,6 +351,167 @@ static struct uvc_control_info uvc_ctrls[] = {
 				| UVC_CTRL_FLAG_RESTORE
 				| UVC_CTRL_FLAG_AUTO_UPDATE,
 	},
+	/*
+	 * All EU controls are marked as AUTO_UPDATE, because many are, and also
+	 * we can't cache all of them as they are stream/layer dependent, which
+	 * would be too slow/too much to cache.
+	 */
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_PROFILE_TOOLSET_CONTROL,
+		.index		= 1,
+		.size		= 6,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_MIN_FRAME_INTERVAL_CONTROL,
+		.index		= 3,
+		.size		= 4,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SLICE_MODE_CONTROL,
+		.index		= 4,
+		.size		= 4,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_RATE_CONTROL_MODE_CONTROL,
+		.index		= 5,
+		.size		= 1,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_AVERAGE_BITRATE_CONTROL,
+		.index		= 6,
+		.size		= 4,
+		.flags		= UVC_CTRL_FLAG_SET_CUR
+				| UVC_CTRL_FLAG_GET_RANGE
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_CPB_SIZE_CONTROL,
+		.index		= 7,
+		.size		= 4,
+		.flags		= UVC_CTRL_FLAG_SET_CUR
+				| UVC_CTRL_FLAG_GET_RANGE
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_PEAK_BIT_RATE_CONTROL,
+		.index		= 8,
+		.size		= 4,
+		.flags		= UVC_CTRL_FLAG_SET_CUR
+				| UVC_CTRL_FLAG_GET_RANGE
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
+		.index		= 9,
+		.size		= 6,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
+		.index		= 10,
+		.size		= 4,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_BUFFER_CONTROL,
+		.index		= 11,
+		.size		= 2,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MAX | UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_PICTURE_CONTROL,
+		.index		= 12,
+		.size		= 2,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_VALIDATION_CONTROL,
+		.index		= 13,
+		.size		= 2,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LEVEL_IDC_LIMIT_CONTROL,
+		.index		= 14,
+		.size		= 1,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SEI_PAYLOADTYPE_CONTROL,
+		.index		= 15,
+		.size		= 8,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MAX | UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_QP_RANGE_CONTROL,
+		.index		= 16,
+		.size		= 2,
+		.flags		= UVC_CTRL_FLAG_SET_CUR
+				| UVC_CTRL_FLAG_GET_RANGE
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_PRIORITY_CONTROL,
+		.index		= 17,
+		.size		= 1,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_ERROR_RESILIENCY_CONTROL,
+		.index		= 19,
+		.size		= 2,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_DEF | UVC_CTRL_FLAG_GET_RES
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
 };
 
 static struct uvc_menu_info power_line_frequency_controls[] = {
@@ -366,6 +527,32 @@ static struct uvc_menu_info exposure_auto_controls[] = {
 	{ 8, "Aperture Priority Mode" },
 };
 
+static struct uvc_menu_info rate_control_mode_controls[] = {
+	{ 1, "VBR" },
+	{ 2, "CBR" },
+	{ 3, "Constant QP" },
+	{ 4, "GVBR" },
+	{ 5, "VBRN" },
+	{ 6, "GVBRN" },
+};
+
+static struct uvc_menu_info encoder_vp8_sync_frame_type_controls[] = {
+	{ 0, "Reset" },
+	{ 1, "Generate Intra Frame" },
+	{ 2, "GDR" },
+	{ 3, "GDR and Update Golden" },
+	{ 4, "GDR and Update Alt" },
+	{ 5, "GDR and Update Golden + Alt" },
+	{ 6, "Update Golden" },
+	{ 7, "Update Alt" },
+	{ 8, "Update Golden + Alt" },
+};
+
+static struct uvc_menu_info encoder_vp8_slice_mode_controls[] = {
+	{ 0, "No Partitioning" },
+	{ 1, "DCT Partitions Per Frame" },
+};
+
 static __s32 uvc_ctrl_get_zoom(struct uvc_control_mapping *mapping,
 	__u8 query, const __u8 *data)
 {
@@ -686,6 +873,264 @@ static struct uvc_control_mapping uvc_ctrl_mappings[] = {
 		.v4l2_type	= V4L2_CTRL_TYPE_BOOLEAN,
 		.data_type	= UVC_CTRL_DATA_TYPE_BOOLEAN,
 	},
+	/* Encoder controls. */
+	{
+		.id		= V4L2_CID_ENCODER_H264_PROFILE_TOOLSET,
+		.name		= "Encoder, H.264 profile/toolset",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_PROFILE_TOOLSET_CONTROL,
+		.size		= 40,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_MIN_FRAME_INTERVAL,
+		.name		= "Encoder, minimum frame interval",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_MIN_FRAME_INTERVAL_CONTROL,
+		.size		= 32,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_VP8_SLICE_MODE,
+		.name		= "Encoder, VP8 slice mode",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SLICE_MODE_CONTROL,
+		.size		= 16,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_MENU,
+		.data_type	= UVC_CTRL_DATA_TYPE_ENUM,
+		.menu_info	= encoder_vp8_slice_mode_controls,
+		.menu_count	= ARRAY_SIZE(encoder_vp8_slice_mode_controls),
+	},
+	{
+		.id		= V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME,
+		.name		= "Encoder, VP8 DCT partns/frame",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SLICE_MODE_CONTROL,
+		.size		= 16,
+		.offset		= 16,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_RATE_CONTROL_MODE,
+		.name		= "Encoder, rate control mode",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_RATE_CONTROL_MODE_CONTROL,
+		.size		= 4,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_MENU,
+		.data_type	= UVC_CTRL_DATA_TYPE_ENUM,
+		.menu_info	= rate_control_mode_controls,
+		.menu_count	= ARRAY_SIZE(rate_control_mode_controls),
+	},
+	{
+		.id		= V4L2_CID_ENCODER_AVERAGE_BITRATE,
+		.name		= "Encoder, average bitrate",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_AVERAGE_BITRATE_CONTROL,
+		.size		= 32,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_CPB_SIZE,
+		.name		= "Encoder, CPB size",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_CPB_SIZE_CONTROL,
+		.size		= 32,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_PEAK_BIT_RATE,
+		.name		= "Encoder, peak bit rate",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_PEAK_BIT_RATE_CONTROL,
+		.size		= 32,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_QP_PARAM_I,
+		.name		= "Encoder, QP param, I frames",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
+		.size		= 16,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_QP_PARAM_P,
+		.name		= "Encoder, QP param, P frames",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
+		.size		= 16,
+		.offset		= 16,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_QP_PARAM_BG,
+		.name		= "Encoder, QP param, B/G frames",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
+		.size		= 16,
+		.offset		= 32,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE,
+		.name		= "Encoder, VP8 sync frame type",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
+		.size		= 8,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_MENU,
+		.data_type	= UVC_CTRL_DATA_TYPE_ENUM,
+		.menu_info	= encoder_vp8_sync_frame_type_controls,
+		.menu_count	=
+			ARRAY_SIZE(encoder_vp8_sync_frame_type_controls),
+	},
+	{
+		.id		= V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL,
+		.name		= "Encoder, sync frame interval",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
+		.size		= 16,
+		.offset		= 8,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_NUM_GDR_FRAMES,
+		.name		= "Encoder, number of GDR frames",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
+		.size		= 8,
+		.offset		= 24,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_LTR_BUFFER_CONTROL,
+		.name		= "Encoder, LTR buffer control",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_BUFFER_CONTROL,
+		.size		= 8,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE,
+		.name		= "Encoder, LTR buffer trust mode",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_BUFFER_CONTROL,
+		.size		= 8,
+		.offset		= 8,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_LTR_PICTURE_POSITION,
+		.name		= "Encoder, LTR picture position",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_PICTURE_CONTROL,
+		.size		= 8,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_LTR_PICTURE_MODE,
+		.name		= "Encoder, LTR picture mode",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_PICTURE_CONTROL,
+		.size		= 8,
+		.offset		= 8,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_LTR_VALIDATION,
+		.name		= "Encoder, LTR validation",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LTR_VALIDATION_CONTROL,
+		.size		= 16,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_BITMASK,
+		.data_type	= UVC_CTRL_DATA_TYPE_BITMASK,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT,
+		.name		= "Encoder, H.264 level IDC limit",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_LEVEL_IDC_LIMIT_CONTROL,
+		.size		= 8,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE,
+		.name		= "Encoder, H.264 SEI payload type",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SEI_PAYLOADTYPE_CONTROL,
+		.size		= 64,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_BITMASK,
+		.data_type	= UVC_CTRL_DATA_TYPE_BITMASK,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_MIN_QP,
+		.name		= "Encoder, minimum QP param",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_QP_RANGE_CONTROL,
+		.size		= 8,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_SIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_MAX_QP,
+		.name		= "Encoder, maximum QP param",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_QP_RANGE_CONTROL,
+		.size		= 8,
+		.offset		= 8,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_SIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_H264_LAYER_PRIORITY,
+		.name		= "Encoder, H.264 layer priority",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_PRIORITY_CONTROL,
+		.size		= 8,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
+		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
+	},
+	{
+		.id		= V4L2_CID_ENCODER_ERROR_RESILIENCY,
+		.name		= "Encoder, error resiliency",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_ERROR_RESILIENCY_CONTROL,
+		.size		= 16,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_BITMASK,
+		.data_type	= UVC_CTRL_DATA_TYPE_BITMASK,
+	},
 };
 
 /* ------------------------------------------------------------------------
diff --git a/include/uapi/linux/usb/video.h b/include/uapi/linux/usb/video.h
index e09c50b..fd1d4b0 100644
--- a/include/uapi/linux/usb/video.h
+++ b/include/uapi/linux/usb/video.h
@@ -127,6 +127,29 @@
 #define UVC_PU_ANALOG_VIDEO_STANDARD_CONTROL		0x11
 #define UVC_PU_ANALOG_LOCK_STATUS_CONTROL		0x12
 
+/* Encoding Unit Control Selectors */
+#define UVC_EU_CONTROL_UNDEFINED			0x00
+#define UVC_EU_SELECT_LAYER_CONTROL			0x01
+#define UVC_EU_PROFILE_TOOLSET_CONTROL			0x02
+#define UVC_EU_VIDEO_RESOLUTION_CONTROL			0x03
+#define UVC_EU_MIN_FRAME_INTERVAL_CONTROL		0x04
+#define UVC_EU_SLICE_MODE_CONTROL			0x05
+#define UVC_EU_RATE_CONTROL_MODE_CONTROL		0x06
+#define UVC_EU_AVERAGE_BITRATE_CONTROL			0x07
+#define UVC_EU_CPB_SIZE_CONTROL				0x08
+#define UVC_EU_PEAK_BIT_RATE_CONTROL			0x09
+#define UVC_EU_QUANTIZATION_PARAMS_CONTROL		0x0a
+#define UVC_EU_SYNC_REF_FRAME_CONTROL			0x0b
+#define UVC_EU_LTR_BUFFER_CONTROL			0x0c
+#define UVC_EU_LTR_PICTURE_CONTROL			0x0d
+#define UVC_EU_LTR_VALIDATION_CONTROL			0x0e
+#define UVC_EU_LEVEL_IDC_LIMIT_CONTROL			0x0f
+#define UVC_EU_SEI_PAYLOADTYPE_CONTROL			0x10
+#define UVC_EU_QP_RANGE_CONTROL				0x11
+#define UVC_EU_PRIORITY_CONTROL				0x12
+#define UVC_EU_START_OR_STOP_LAYER_CONTROL		0x13
+#define UVC_EU_ERROR_RESILIENCY_CONTROL			0x14
+
 /* A.9.7. VideoStreaming Interface Control Selectors */
 #define UVC_VS_CONTROL_UNDEFINED			0x00
 #define UVC_VS_PROBE_CONTROL				0x01
-- 
1.8.4

[PATCH v1 18/19] v4l: Add V4L2_PIX_FMT_VP8_SIMULCAST format.

[Pawel Osciak]

This format is used by UVC 1.5 VP8-encoding cameras. When it is used, the camera
may encode captured frames into one or more streams, each of which may
be configured differently. This allows simultaneous capture of streams
with different resolutions, bitrates, and other settings, depending on the
camera capabilities.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 include/uapi/linux/videodev2.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index c011ee0..8b0d4ad 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -402,6 +402,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_VC1_ANNEX_G v4l2_fourcc('V', 'C', '1', 'G') /* SMPTE 421M Annex G compliant stream */
 #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
 #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
+#define V4L2_PIX_FMT_VP8_SIMULCAST v4l2_fourcc('V', 'P', '8', 'S') /* VP8 simulcast */
 
 /*  Vendor-specific formats   */
 #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
@@ -691,6 +692,9 @@ struct v4l2_buffer {
 #define V4L2_BUF_FLAG_PREV_FRAME		0x10000  /* VP8 prev frame */
 #define V4L2_BUF_FLAG_GOLDEN_FRAME		0x20000  /* VP8 golden frame */
 #define V4L2_BUF_FLAG_ALTREF_FRAME		0x40000  /* VP8 altref frame */
+/* Simulcast layer structure. */
+#define V4L2_BUF_FLAG_LAYER_STRUCTURE_SHIFT	19  /* Bits 19-20 for layer */
+#define V4L2_BUF_FLAG_LAYER_STRUCTURE_MASK	0x3 /* structure information. */
 
 /**
  * struct v4l2_exportbuffer - export of video buffer as DMABUF file descriptor
-- 
1.8.4

[PATCH v1 19/19] uvcvideo: Add support for UVC 1.5 VP8 simulcast.

[Pawel Osciak]

Simulcast allows streaming up to 4 sub-streams (layers) from one video
streaming interface. Those stream are captured by the same sensor, but
encoding parameters can be set for each stream stream separately. Those
parameters may include properties such as resolution, bitrate, etc. Each
layer may be controlled separately and can be paused or resumed as needed,
without influencing other layers.

In V4L2, we create a separate video node for each stream and allow opening
and controlling them separately. Setting resolution, controls and
streamon/streamoff can be done independently on each node. Due to the
limitations inherent to the nature of simulcast though, those nodes do not
behave in a completely independent way with regards to format settings and
buffer allocation. When simulcast format is selected for one of the nodes,
it is set for all simulcast nodes and cannot be changed until all nodes are
idle. Once buffers are allocated for any of the nodes, the format cannot
be changed back to non-simulcast until buffers for all nodes have been
freed.

Internally, we assign and manage a separate video buffer queue for each
simulcast layer.

Signed-off-by: Pawel Osciak <posciak@chromium.org>
---
 drivers/media/usb/uvc/uvc_ctrl.c   | 283 +++++++++++++++++++++++++++++++++++
 drivers/media/usb/uvc/uvc_driver.c | 176 +++++++++++++++-------
 drivers/media/usb/uvc/uvc_entity.c | 114 ++++++++++----
 drivers/media/usb/uvc/uvc_isight.c |   8 +-
 drivers/media/usb/uvc/uvc_queue.c  |  11 +-
 drivers/media/usb/uvc/uvc_v4l2.c   | 257 ++++++++++++++++++++++++++------
 drivers/media/usb/uvc/uvc_video.c  | 298 +++++++++++++++++++++++++++----------
 drivers/media/usb/uvc/uvcvideo.h   | 119 +++++++++++++--
 include/uapi/linux/usb/video.h     |   6 +
 9 files changed, 1046 insertions(+), 226 deletions(-)

diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
index cd02c99..6a39020c 100644
--- a/drivers/media/usb/uvc/uvc_ctrl.c
+++ b/drivers/media/usb/uvc/uvc_ctrl.c
@@ -358,6 +358,14 @@ static struct uvc_control_info uvc_ctrls[] = {
 	 */
 	{
 		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_SELECT_LAYER_CONTROL,
+		.index		= 0,
+		.size		= 2,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
 		.selector	= UVC_EU_PROFILE_TOOLSET_CONTROL,
 		.index		= 1,
 		.size		= 6,
@@ -367,6 +375,16 @@ static struct uvc_control_info uvc_ctrls[] = {
 	},
 	{
 		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_VIDEO_RESOLUTION_CONTROL,
+		.index		= 2,
+		.size		= 4,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
+				| UVC_CTRL_FLAG_GET_DEF
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
 		.selector	= UVC_EU_MIN_FRAME_INTERVAL_CONTROL,
 		.index		= 3,
 		.size		= 4,
@@ -505,6 +523,14 @@ static struct uvc_control_info uvc_ctrls[] = {
 	},
 	{
 		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_START_OR_STOP_LAYER_CONTROL,
+		.index		= 18,
+		.size		= 1,
+		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
+				| UVC_CTRL_FLAG_AUTO_UPDATE,
+	},
+	{
+		.entity		= UVC_GUID_UVC_ENCODING,
 		.selector	= UVC_EU_ERROR_RESILIENCY_CONTROL,
 		.index		= 19,
 		.size		= 2,
@@ -1131,6 +1157,16 @@ static struct uvc_control_mapping uvc_ctrl_mappings[] = {
 		.v4l2_type	= V4L2_CTRL_TYPE_BITMASK,
 		.data_type	= UVC_CTRL_DATA_TYPE_BITMASK,
 	},
+	{
+		.id		= V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE,
+		.name		= "Encoder, temporal layer enable",
+		.entity		= UVC_GUID_UVC_ENCODING,
+		.selector	= UVC_EU_START_OR_STOP_LAYER_CONTROL,
+		.size		= 1,
+		.offset		= 0,
+		.v4l2_type	= V4L2_CTRL_TYPE_BOOLEAN,
+		.data_type	= UVC_CTRL_DATA_TYPE_BOOLEAN,
+	},
 };
 
 /* ------------------------------------------------------------------------
@@ -1918,6 +1954,20 @@ int uvc_ctrl_get(struct uvc_video_chain *chain, struct v4l2_ext_control *xctrl,
 	return __uvc_ctrl_get(chain, ctrl, mapping, &xctrl->value);
 }
 
+int uvc_ctrl_set_raw_val(struct uvc_control *ctrl,
+				struct uvc_control_mapping *mapping, u32 value)
+{
+	struct uvc_device *dev = ctrl->entity->chain->dev;
+
+	mapping->set(mapping, value,
+			uvc_ctrl_data(ctrl, UVC_CTRL_DATA_CURRENT));
+
+	return uvc_query_ctrl(dev, UVC_SET_CUR, ctrl->entity->id,
+				dev->intfnum, ctrl->info.selector,
+				uvc_ctrl_data(ctrl, UVC_CTRL_DATA_CURRENT),
+				ctrl->info.size);
+}
+
 int uvc_ctrl_set(struct uvc_video_chain *chain, struct v4l2_ext_control *xctrl,
 		 bool streaming)
 {
@@ -2307,6 +2357,239 @@ done:
 }
 
 /* --------------------------------------------------------------------------
+ * Non-exported encoder controls
+ */
+/* Must be called under chain lock */
+int __uvc_video_layer_select(struct uvc_stream_layer *layer, int temporal_id)
+{
+	struct uvc_streaming *stream = layer->stream;
+	struct uvc_video_chain *chain = stream->chain;
+	struct uvc_control *ctrl;
+	__u8 data[2];
+
+	if (!chain->encoding || !chain->layer_ctrl)
+		return 0;
+
+	uvc_trace(UVC_TRACE_CONTROL, "Selecting layer: %d temporal_id: %d\n",
+			layer->layer_id, temporal_id);
+
+	if (uvc_cur_fmt_has_tmprl_layers(stream)) {
+		if (temporal_id > stream->cur_frame->bmScalabilityCapabilities
+			&& temporal_id != UVC_VP8_TEMPORAL_LAYOUT_WILDCARD) {
+			uvc_trace(UVC_TRACE_CONTROL,
+				  "Invalid temporal id %d\n", temporal_id);
+			return -EINVAL;
+		}
+	} else {
+		temporal_id = 0;
+	}
+
+	ctrl = chain->layer_ctrl;
+
+	memset(data, 0, 2);
+	__uvc_set_le_value(3, 7, temporal_id, data, false);
+	__uvc_set_le_value(3, 10, layer->layer_id, data, true);
+
+	return uvc_query_ctrl(chain->dev, UVC_SET_CUR, ctrl->entity->id,
+			      chain->dev->intfnum, ctrl->info.selector,
+			      data, ctrl->info.size);
+}
+
+bool uvc_is_layer_negotiated(struct uvc_streaming *stream, int layer_id)
+{
+	if (layer_id > stream->num_layers)
+		return false;
+
+	if (stream->ctrl.bmLayoutPerStream[layer_id] & 0x1)
+		return true;
+	else
+		return false;
+}
+
+/* Must be called under stream->mutex. */
+int uvc_ctrl_layer_enable(struct uvc_stream_layer *layer, int enable)
+{
+	struct uvc_video_chain *chain = layer->stream->chain;
+	struct uvc_control *ctrl;
+	__u8 data[4];
+	int ret;
+
+	if (!chain->encoding || !chain->startstop_ctrl)
+		return -EINVAL;
+
+	if (layer->stream->streaming_layers) {
+		/* We are streaming. If we can't enable/disable during
+		 * streaming, check if stream has already been enabled on P&C,
+		 * if so, it couldn't have been changed during streaming, so
+		 * just return success. If it hasn't been enabled on P&C,
+		 * we can't do it now so return EBUSY.
+		 */
+		if (!(layer->stream->simulcast_caps
+				 & UVC_SIMUL_CAP_STARTSTOP_LAYER_STREAMTIME)) {
+			if (uvc_is_layer_negotiated(layer->stream,
+							layer->layer_id))
+				return 0;
+			else
+				return -EBUSY;
+		} /* else we can enable/disable and will do so below. */
+	} else if (!(layer->stream->simulcast_caps
+				& UVC_SIMUL_CAP_STARTSTOP_LAYER)) {
+		/* Not streaming, but change not allowed even now.
+		 * If the layer has been negotiated on P&C, then it's already
+		 * enabled, return success, else we can't enable it at all.
+		 */
+		if (uvc_is_layer_negotiated(layer->stream, layer->layer_id))
+			return 0;
+		else
+			return -EBUSY;
+	} /* else not streaming and can enable/disable, do so below. */
+
+	ret = __uvc_ctrl_lock(chain);
+	if (ret < 0)
+		return ret;
+
+	/* TODO: might need to cache current temporal id configuration
+	 * for suspend/resume.
+	 */
+	ret = __uvc_video_layer_select(layer, 0x7);
+	if (ret)
+		goto unlock;
+
+	ctrl = chain->startstop_ctrl;
+	memset(data, 0, 4);
+	__uvc_set_le_value(1, 0, !!enable, data, false);
+	ret = uvc_query_ctrl(chain->dev, UVC_SET_CUR, ctrl->entity->id,
+			     chain->dev->intfnum, ctrl->info.selector,
+			     data, ctrl->info.size);
+unlock:
+	__uvc_ctrl_unlock(chain);
+	return ret;
+}
+
+int __uvc_video_layer_query_resolution(struct uvc_stream_layer *layer,
+					int width, int height,
+					__u8 query)
+{
+	struct uvc_video_chain *chain = layer->stream->chain;
+	struct uvc_control *ctrl;
+	__u8 data[4];
+	int ret;
+
+	ret = __uvc_ctrl_lock(chain);
+	if (ret < 0)
+		return ret;
+
+	ret = __uvc_video_layer_select(layer, 0);
+	if (ret)
+		goto unlock;
+
+	ctrl = chain->res_ctrl;
+	memset(data, 0, 4);
+	__uvc_set_le_value(16, 0, width, data, false);
+	__uvc_set_le_value(16, 16, height, data, true);
+	ret = uvc_query_ctrl(chain->dev, query, ctrl->entity->id,
+			     chain->dev->intfnum, ctrl->info.selector,
+			     data, ctrl->info.size);
+
+	if (ret == 0 && (query == UVC_SET_CUR || query == UVC_GET_CUR)) {
+		layer->width = __uvc_get_le_value(16, 0, data,
+						UVC_CTRL_DATA_TYPE_UNSIGNED);
+		layer->height = __uvc_get_le_value(16, 16, data,
+						UVC_CTRL_DATA_TYPE_UNSIGNED);
+	}
+
+unlock:
+	__uvc_ctrl_unlock(chain);
+	return ret;
+}
+
+int uvc_video_layer_set_resolution(struct uvc_stream_layer *layer,
+					int width, int height)
+{
+	struct uvc_video_chain *chain = layer->stream->chain;
+	int ret;
+
+	/* TODO: will fail if not exactly supported resolution. Not a huge
+	 * problem, but would be nice to use bResolutionScaling in format
+	 * descriptors to adjust to the nearest supported resolution instead.
+	 */
+	if (!chain->encoding || !chain->res_ctrl)
+		return -EINVAL;
+
+	if (layer->stream->streaming_layers) {
+		if (!(layer->stream->simulcast_caps
+			 & UVC_SIMUL_CAP_RES_STREAMTIME))
+			return -EBUSY;
+	} else if (!(layer->stream->simulcast_caps & UVC_SIMUL_CAP_RES)) {
+		/* This function is only called if we are simulcasting.
+		 * Resolution control has to be usable in that case at least
+		 * when not streaming, otherwise there would be no other
+		 * way to set/get stream resolution. So if we get here,
+		 * the device is not really usable.
+		 */
+		return -EINVAL;
+	}
+
+	ret = __uvc_video_layer_query_resolution(layer, width, height,
+						 UVC_SET_CUR);
+	if (ret == 0) {
+		uvc_trace(UVC_TRACE_CONTROL,
+			"Layer %d resolution set to %dx%d\n",
+			layer->layer_id, width, height);
+	}
+
+	return ret;
+}
+
+int uvc_video_layer_get_resolution(struct uvc_stream_layer *layer)
+{
+	struct uvc_video_chain *chain = layer->stream->chain;
+
+	if (!chain->encoding || !uvc_is_simulcasting(layer->stream)
+			|| !chain->res_ctrl) {
+		layer->width = layer->stream->cur_frame->wWidth;
+		layer->height = layer->stream->cur_frame->wHeight;
+		return 0;
+	}
+
+	return __uvc_video_layer_query_resolution(layer, 0, 0, UVC_GET_CUR);
+}
+
+void uvc_ctrl_simulcast_init(struct uvc_streaming *stream)
+{
+	struct uvc_control *ctrl;
+
+	ctrl = stream->chain->startstop_ctrl;
+	if (ctrl) {
+		if (ctrl->on_init)
+			stream->simulcast_caps |= UVC_SIMUL_CAP_STARTSTOP_LAYER;
+		if (ctrl->in_runtime)
+			stream->simulcast_caps |=
+				UVC_SIMUL_CAP_STARTSTOP_LAYER_STREAMTIME;
+	}
+
+	ctrl = stream->chain->res_ctrl;
+	if (ctrl) {
+		if (ctrl->on_init)
+			stream->simulcast_caps |= UVC_SIMUL_CAP_RES;
+		if (ctrl->in_runtime)
+			stream->simulcast_caps |=
+				UVC_SIMUL_CAP_RES_STREAMTIME;
+	}
+
+	uvc_trace(UVC_TRACE_CONTROL, "Available simulcast capabilities: "
+			"start/stop layer: %c (while streaming: %c), "
+			"resolution change: %c (while streaming: %c).\n",
+		stream->simulcast_caps
+			& UVC_SIMUL_CAP_STARTSTOP_LAYER ? 'Y' : 'N',
+		stream->simulcast_caps
+			& UVC_SIMUL_CAP_STARTSTOP_LAYER_STREAMTIME ? 'Y' : 'N',
+		stream->simulcast_caps & UVC_SIMUL_CAP_RES ? 'Y' : 'N',
+		stream->simulcast_caps
+			& UVC_SIMUL_CAP_RES_STREAMTIME ? 'Y' : 'N');
+}
+
+/* --------------------------------------------------------------------------
  * Suspend/resume
  */
 
diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c
index 27a7a11..ea5b63a 100644
--- a/drivers/media/usb/uvc/uvc_driver.c
+++ b/drivers/media/usb/uvc/uvc_driver.c
@@ -402,14 +402,23 @@ static int uvc_parse_format(struct uvc_device *dev,
 		break;
 
 	case UVC_VS_FORMAT_VP8:
+	case UVC_VS_FORMAT_VP8_SIMULCAST:
 		if (buflen < 13)
 			goto format_error;
 
 		format->bpp = 0;
 		format->flags = UVC_FMT_FLAG_COMPRESSED;
 		ftype = UVC_VS_FRAME_VP8;
-		strlcpy(format->name, "VP8", sizeof(format->name));
-		format->fcc = V4L2_PIX_FMT_VP8;
+		if (buffer[2] == UVC_VS_FORMAT_VP8) {
+			strlcpy(format->name, "VP8", sizeof(format->name));
+			format->fcc = V4L2_PIX_FMT_VP8;
+		} else {
+			strlcpy(format->name, "VP8 Simulcast",
+					sizeof(format->name));
+			format->fcc = V4L2_PIX_FMT_VP8_SIMULCAST;
+			streaming->num_layers = UVC_SIMULCAST_MAX_LAYERS;
+			format->flags |= UVC_FMT_FLAG_SIMULCAST;
+		}
 
 		break;
 
@@ -635,6 +644,12 @@ static int uvc_parse_streaming(struct uvc_device *dev,
 	streaming->dev = dev;
 	streaming->intf = usb_get_intf(intf);
 	streaming->intfnum = intf->cur_altsetting->desc.bInterfaceNumber;
+	streaming->num_layers = 1;
+	atomic_set(&dev->num_vdevs, 0);
+	streaming->streaming_layers = 0;
+	streaming->allocated_layers = 0;
+	for (i = 0; i < UVC_SIMULCAST_MAX_LAYERS; ++i)
+		streaming->layers[i].layer_id = i;
 
 	/* The Pico iMage webcam has its class-specific interface descriptors
 	 * after the endpoint descriptors.
@@ -731,6 +746,7 @@ static int uvc_parse_streaming(struct uvc_device *dev,
 		case UVC_VS_FORMAT_MJPEG:
 		case UVC_VS_FORMAT_FRAME_BASED:
 		case UVC_VS_FORMAT_VP8:
+		case UVC_VS_FORMAT_VP8_SIMULCAST:
 			nformats++;
 			break;
 
@@ -804,6 +820,7 @@ static int uvc_parse_streaming(struct uvc_device *dev,
 		case UVC_VS_FORMAT_DV:
 		case UVC_VS_FORMAT_FRAME_BASED:
 		case UVC_VS_FORMAT_VP8:
+		case UVC_VS_FORMAT_VP8_SIMULCAST:
 			format->frame = frame;
 			ret = uvc_parse_format(dev, streaming, format,
 				&interval, buffer, buflen);
@@ -1732,6 +1749,7 @@ static int uvc_scan_device(struct uvc_device *dev)
 static void uvc_delete(struct uvc_device *dev)
 {
 	struct list_head *p, *n;
+	int i;
 
 	usb_put_intf(dev->intf);
 	usb_put_dev(dev->udev);
@@ -1760,16 +1778,17 @@ static void uvc_delete(struct uvc_device *dev)
 #ifdef CONFIG_MEDIA_CONTROLLER
 		uvc_mc_cleanup_entity(entity);
 #endif
-		if (entity->vdev) {
-			video_device_release(entity->vdev);
-			entity->vdev = NULL;
-		}
 		kfree(entity);
 	}
 
 	list_for_each_safe(p, n, &dev->streams) {
 		struct uvc_streaming *streaming;
 		streaming = list_entry(p, struct uvc_streaming, list);
+		for (i = 0; i < streaming->num_layers; ++i) {
+			if (streaming->vdev[i])
+				video_device_release(streaming->vdev[i]);
+			streaming->vdev[i] = NULL;
+		}
 		usb_driver_release_interface(&uvc_driver.driver,
 			streaming->intf);
 		usb_put_intf(streaming->intf);
@@ -1783,13 +1802,13 @@ static void uvc_delete(struct uvc_device *dev)
 
 static void uvc_release(struct video_device *vdev)
 {
-	struct uvc_streaming *stream = video_get_drvdata(vdev);
-	struct uvc_device *dev = stream->dev;
+	struct uvc_stream_layer *layer = video_get_drvdata(vdev);
+	struct uvc_device *dev = layer->stream->dev;
 
-	/* Decrement the registered streams count and delete the device when it
+	/* Decrement the registered vdevs count and delete the device when it
 	 * reaches zero.
 	 */
-	if (atomic_dec_and_test(&dev->nstreams))
+	if (atomic_dec_and_test(&dev->num_vdevs))
 		uvc_delete(dev);
 }
 
@@ -1799,20 +1818,23 @@ static void uvc_release(struct video_device *vdev)
 static void uvc_unregister_video(struct uvc_device *dev)
 {
 	struct uvc_streaming *stream;
+	int i;
 
 	/* Unregistering all video devices might result in uvc_delete() being
 	 * called from inside the loop if there's no open file handle. To avoid
-	 * that, increment the stream count before iterating over the streams
+	 * that, increment the vdevs count before iterating over the vdevs
 	 * and decrement it when done.
 	 */
-	atomic_inc(&dev->nstreams);
+	atomic_inc(&dev->num_vdevs);
 
 	list_for_each_entry(stream, &dev->streams, list) {
-		if (stream->vdev == NULL)
-			continue;
+		for (i = 0; i < stream->num_layers; ++i) {
+			if (stream->vdev[i] == NULL)
+				continue;
 
-		video_unregister_device(stream->vdev);
-		stream->vdev = NULL;
+			video_unregister_device(stream->vdev[i]);
+			stream->vdev[i] = NULL;
+		}
 
 		uvc_debugfs_cleanup_stream(stream);
 	}
@@ -1820,7 +1842,7 @@ static void uvc_unregister_video(struct uvc_device *dev)
 	/* Decrement the stream count and call uvc_delete explicitly if there
 	 * are no stream left.
 	 */
-	if (atomic_dec_and_test(&dev->nstreams))
+	if (atomic_dec_and_test(&dev->num_vdevs))
 		uvc_delete(dev);
 }
 
@@ -1828,7 +1850,9 @@ static int uvc_register_video(struct uvc_device *dev,
 		struct uvc_streaming *stream)
 {
 	struct video_device *vdev;
+	struct uvc_stream_layer *layer;
 	int ret;
+	int i;
 
 	/* Initialize the streaming interface with default streaming
 	 * parameters.
@@ -1843,39 +1867,60 @@ static int uvc_register_video(struct uvc_device *dev,
 	uvc_debugfs_init_stream(stream);
 
 	/* Register the device with V4L. */
-	vdev = video_device_alloc();
-	if (vdev == NULL) {
-		uvc_printk(KERN_ERR, "Failed to allocate video device (%d).\n",
-			   ret);
-		return -ENOMEM;
-	}
+	for (i = 0; i < stream->num_layers; ++i) {
+		/* Register the device with V4L. */
+		vdev = video_device_alloc();
+		if (vdev == NULL) {
+			uvc_printk(KERN_ERR,
+				"Failed to allocate video device (%d).\n", ret);
+			return -ENOMEM;
+		}
 
-	/* We already hold a reference to dev->udev. The video device will be
-	 * unregistered before the reference is released, so we don't need to
-	 * get another one.
-	 */
-	vdev->v4l2_dev = &dev->vdev;
-	vdev->fops = &uvc_fops;
-	vdev->release = uvc_release;
-	vdev->prio = &stream->chain->prio;
-	set_bit(V4L2_FL_USE_FH_PRIO, &vdev->flags);
-	if (stream->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
-		vdev->vfl_dir = VFL_DIR_TX;
-	strlcpy(vdev->name, dev->name, sizeof vdev->name);
-
-	/* Set the driver data before calling video_register_device, otherwise
-	 * uvc_v4l2_open might race us.
-	 */
-	stream->vdev = vdev;
-	video_set_drvdata(vdev, stream);
+		/* We already hold a reference to dev->udev. The video device
+		 * will be unregistered before the reference is released,
+		 * so we don't need to get another one.
+		 */
+		vdev->v4l2_dev = &dev->vdev;
+		vdev->fops = &uvc_fops;
+		vdev->release = uvc_release;
+		vdev->prio = &stream->chain->prio;
+		set_bit(V4L2_FL_USE_FH_PRIO, &vdev->flags);
+		if (stream->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
+			vdev->vfl_dir = VFL_DIR_TX;
+		strlcpy(vdev->name, dev->name, sizeof(vdev->name));
+
+		/* Set the driver data before calling video_register_device,
+		 * otherwise uvc_v4l2_open might race us.
+		 */
+		stream->vdev[i] = vdev;
+		layer = &stream->layers[i];
+		layer->layer_id = i;
+		layer->stream = stream;
+		uvc_video_layer_get_resolution(layer);
+		atomic_set(&layer->active, 0);
+		video_set_drvdata(vdev, layer);
+		ret = uvc_queue_init(&layer->queue, stream->type,
+					!uvc_no_drop_param, stream);
+		if (ret < 0) {
+			uvc_printk(KERN_ERR,
+				"Failed to initialize video queue.\n");
+			stream->vdev[i] = NULL;
+			video_device_release(vdev);
+			return ret;
+		}
 
-	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
-	if (ret < 0) {
-		uvc_printk(KERN_ERR, "Failed to register video device (%d).\n",
-			   ret);
-		stream->vdev = NULL;
-		video_device_release(vdev);
-		return ret;
+		ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
+		if (ret < 0) {
+			uvc_printk(KERN_ERR,
+				"Failed to register video device (%d).\n", ret);
+			stream->vdev[i] = NULL;
+			video_device_release(vdev);
+			return ret;
+		}
+
+		atomic_inc(&dev->num_vdevs);
+		uvc_trace(UVC_TRACE_PROBE,
+			"Registered video device /dev/video%d\n", vdev->minor);
 	}
 
 	if (stream->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
@@ -1883,7 +1928,6 @@ static int uvc_register_video(struct uvc_device *dev,
 	else
 		stream->chain->caps |= V4L2_CAP_VIDEO_OUTPUT;
 
-	atomic_inc(&dev->nstreams);
 	return 0;
 }
 
@@ -1897,6 +1941,7 @@ static int uvc_register_terms(struct uvc_device *dev,
 	struct uvc_entity *term;
 	struct uvc_chain_entry *entry;
 	int ret;
+	int i;
 
 	list_for_each_entry(entry, &chain->entities, chain_entry) {
 		term = entry->entity;
@@ -1915,7 +1960,8 @@ static int uvc_register_terms(struct uvc_device *dev,
 		if (ret < 0)
 			return ret;
 
-		term->vdev = stream->vdev;
+		for (i = 0; i < stream->num_layers; ++i)
+			term->vdev[i] = stream->vdev[i];
 	}
 
 	return 0;
@@ -1924,9 +1970,30 @@ static int uvc_register_terms(struct uvc_device *dev,
 static int uvc_register_chains(struct uvc_device *dev)
 {
 	struct uvc_video_chain *chain;
+	struct uvc_control *ctrl;
 	int ret;
+	int i;
 
 	list_for_each_entry(chain, &dev->chains, list) {
+		if (chain->encoding) {
+			for (i = 0; i < chain->encoding->ncontrols; ++i) {
+				ctrl = &chain->encoding->controls[i];
+				if (!ctrl->initialized)
+					continue;
+				switch (ctrl->info.selector) {
+				case UVC_EU_SELECT_LAYER_CONTROL:
+					chain->layer_ctrl = ctrl;
+					break;
+				case UVC_EU_VIDEO_RESOLUTION_CONTROL:
+					chain->res_ctrl = ctrl;
+					break;
+				case UVC_EU_START_OR_STOP_LAYER_CONTROL:
+					chain->startstop_ctrl = ctrl;
+					break;
+				}
+			}
+		}
+
 		ret = uvc_register_terms(dev, chain);
 		if (ret < 0)
 			return ret;
@@ -1934,7 +2001,7 @@ static int uvc_register_chains(struct uvc_device *dev)
 #ifdef CONFIG_MEDIA_CONTROLLER
 		ret = uvc_mc_register_entities(chain);
 		if (ret < 0) {
-			uvc_printk(KERN_INFO, "Failed to register entites "
+			uvc_printk(KERN_INFO, "Failed to register entities "
 				"(%d).\n", ret);
 		}
 #endif
@@ -1954,13 +2021,14 @@ static int uvc_probe(struct usb_interface *intf,
 	struct uvc_device *dev;
 	int ret;
 
-	if (id->idVendor && id->idProduct)
+	if (id->idVendor && id->idProduct) {
 		uvc_trace(UVC_TRACE_PROBE, "Probing known UVC device %s "
 				"(%04x:%04x)\n", udev->devpath, id->idVendor,
 				id->idProduct);
-	else
+	} else {
 		uvc_trace(UVC_TRACE_PROBE, "Probing generic UVC device %s\n",
 				udev->devpath);
+	}
 
 	/* Allocate memory for the device and initialize it. */
 	if ((dev = kzalloc(sizeof *dev, GFP_KERNEL)) == NULL)
@@ -1969,7 +2037,7 @@ static int uvc_probe(struct usb_interface *intf,
 	INIT_LIST_HEAD(&dev->entities);
 	INIT_LIST_HEAD(&dev->chains);
 	INIT_LIST_HEAD(&dev->streams);
-	atomic_set(&dev->nstreams, 0);
+	atomic_set(&dev->num_vdevs, 0);
 	atomic_set(&dev->nmappings, 0);
 	mutex_init(&dev->lock);
 
diff --git a/drivers/media/usb/uvc/uvc_entity.c b/drivers/media/usb/uvc/uvc_entity.c
index 657f49a..e1a4afc 100644
--- a/drivers/media/usb/uvc/uvc_entity.c
+++ b/drivers/media/usb/uvc/uvc_entity.c
@@ -23,20 +23,13 @@
  * Video subdevices registration and unregistration
  */
 
-static int uvc_mc_register_entity(struct uvc_video_chain *chain,
-	struct uvc_entity *entity)
+static int uvc_mc_register_media_entity(struct uvc_entity *entity,
+					struct media_entity *sink)
 {
 	const u32 flags = MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE;
-	struct media_entity *sink;
 	unsigned int i;
 	int ret;
 
-	sink = (UVC_ENTITY_TYPE(entity) == UVC_TT_STREAMING)
-	     ? (entity->vdev ? &entity->vdev->entity : NULL)
-	     : &entity->subdev.entity;
-	if (sink == NULL)
-		return 0;
-
 	for (i = 0; i < entity->num_pads; ++i) {
 		struct media_entity *source;
 		struct uvc_entity *remote;
@@ -45,12 +38,17 @@ static int uvc_mc_register_entity(struct uvc_video_chain *chain,
 		if (!(entity->pads[i].flags & MEDIA_PAD_FL_SINK))
 			continue;
 
-		remote = uvc_entity_by_id(chain->dev, entity->baSourceID[i]);
+		remote = uvc_entity_by_id(entity->chain->dev,
+						entity->baSourceID[i]);
 		if (remote == NULL)
 			return -EINVAL;
 
+		/*
+		 * Take vdev[0], because it has to be an ITERM to be a source
+		 * and we don't create more than 1 vdev for those.
+		 */
 		source = (UVC_ENTITY_TYPE(remote) == UVC_TT_STREAMING)
-		       ? (remote->vdev ? &remote->vdev->entity : NULL)
+		       ? (remote->vdev[0] ? &remote->vdev[0]->entity : NULL)
 		       : &remote->subdev.entity;
 		if (source == NULL)
 			continue;
@@ -62,10 +60,36 @@ static int uvc_mc_register_entity(struct uvc_video_chain *chain,
 			return ret;
 	}
 
-	if (UVC_ENTITY_TYPE(entity) == UVC_TT_STREAMING)
-		return 0;
+	return 0;
+}
 
-	return v4l2_device_register_subdev(&chain->dev->vdev, &entity->subdev);
+static int uvc_mc_register_entity(struct uvc_video_chain *chain,
+					struct uvc_entity *entity)
+{
+	struct media_entity *mentity;
+	int ret = 0;
+	int i;
+
+	if (UVC_ENTITY_TYPE(entity) == UVC_TT_STREAMING) {
+		for (i = 0; i < UVC_SIMULCAST_MAX_LAYERS; ++i) {
+			if (entity->vdev[i]) {
+				mentity = &entity->vdev[i]->entity;
+				ret = uvc_mc_register_media_entity(entity,
+								   mentity);
+				if (ret < 0)
+					return ret;
+			}
+		}
+	} else {
+		mentity = &entity->subdev.entity;
+		ret = uvc_mc_register_media_entity(entity, mentity);
+		if (ret < 0)
+			return ret;
+		ret = v4l2_device_register_subdev(&chain->dev->vdev,
+						  &entity->subdev);
+	}
+
+	return ret;
 }
 
 static struct v4l2_subdev_ops uvc_subdev_ops = {
@@ -73,15 +97,26 @@ static struct v4l2_subdev_ops uvc_subdev_ops = {
 
 void uvc_mc_cleanup_entity(struct uvc_entity *entity)
 {
-	if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING)
+	int i;
+
+	if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
 		media_entity_cleanup(&entity->subdev.entity);
-	else if (entity->vdev != NULL)
-		media_entity_cleanup(&entity->vdev->entity);
+	} else {
+		for (i = 0; i < UVC_SIMULCAST_MAX_LAYERS; ++i) {
+			if (entity->vdev[i] != NULL)
+				media_entity_cleanup(&entity->vdev[i]->entity);
+		}
+	}
+
+	entity->registered = false;
 }
 
 static int uvc_mc_init_entity(struct uvc_entity *entity)
 {
-	int ret;
+	struct media_pad *pads;
+	struct device *dev = &entity->chain->dev->udev->dev;
+	int ret = 0;
+	int i;
 
 	if (UVC_ENTITY_TYPE(entity) != UVC_TT_STREAMING) {
 		v4l2_subdev_init(&entity->subdev, &uvc_subdev_ops);
@@ -90,13 +125,42 @@ static int uvc_mc_init_entity(struct uvc_entity *entity)
 
 		ret = media_entity_init(&entity->subdev.entity,
 					entity->num_pads, entity->pads, 0);
-	} else if (entity->vdev != NULL) {
-		ret = media_entity_init(&entity->vdev->entity,
-					entity->num_pads, entity->pads, 0);
-		if (entity->flags & UVC_ENTITY_FLAG_DEFAULT)
-			entity->vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
-	} else
-		ret = 0;
+	} else {
+		for (i = 0; i < UVC_SIMULCAST_MAX_LAYERS; ++i) {
+			if (entity->vdev[i] != NULL) {
+				/*
+				 * As we may have multiple media entities
+				 * (vdevs) per one STREAMING UVC entity in
+				 * simulcast situations, we have separate memory
+				 * allocated for pads for each media entity
+				 * to have its own.
+				 */
+				pads = devm_kzalloc(dev,
+					sizeof(*pads) * entity->num_pads,
+					GFP_KERNEL);
+				if (!pads) {
+					ret = -ENOMEM;
+					goto cleanup;
+				}
+				memcpy(pads, entity->pads,
+					sizeof(*pads) * entity->num_pads);
+
+				ret = media_entity_init(
+					&entity->vdev[i]->entity,
+					entity->num_pads, pads, 0);
+				if (ret)
+					goto cleanup;
+			}
+		}
+	}
+
+	return ret;
+
+cleanup:
+	for (i = i - 1; i >= 0; ++i) {
+		if (entity->vdev[i] != NULL)
+			media_entity_cleanup(&entity->vdev[i]->entity);
+	}
 
 	return ret;
 }
diff --git a/drivers/media/usb/uvc/uvc_isight.c b/drivers/media/usb/uvc/uvc_isight.c
index ab01286..c105dd7 100644
--- a/drivers/media/usb/uvc/uvc_isight.c
+++ b/drivers/media/usb/uvc/uvc_isight.c
@@ -103,8 +103,9 @@ void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream)
 {
 	int ret, i;
 	struct uvc_buffer *buf;
+	struct uvc_video_queue *queue = &stream->layers[0].queue;
 
-	buf = uvc_queue_get_first_buf(&stream->queue);
+	buf = uvc_queue_get_first_buf(queue);
 
 	for (i = 0; i < urb->number_of_packets; ++i) {
 		if (urb->iso_frame_desc[i].status < 0) {
@@ -122,7 +123,7 @@ void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream)
 		 * processes the data of the first payload of the new frame.
 		 */
 		do {
-			ret = isight_decode(&stream->queue, buf,
+			ret = isight_decode(queue, buf,
 					urb->transfer_buffer +
 					urb->iso_frame_desc[i].offset,
 					urb->iso_frame_desc[i].actual_length);
@@ -132,8 +133,7 @@ void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream)
 
 			if (buf->state == UVC_BUF_STATE_DONE ||
 			    buf->state == UVC_BUF_STATE_ERROR)
-				buf = uvc_queue_next_buffer(&stream->queue,
-							buf);
+				buf = uvc_queue_next_buffer(queue, buf);
 		} while (ret == -EAGAIN);
 	}
 }
diff --git a/drivers/media/usb/uvc/uvc_queue.c b/drivers/media/usb/uvc/uvc_queue.c
index 55d2670..379e994 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -45,15 +45,13 @@ static int uvc_queue_setup(struct vb2_queue *vq, const struct v4l2_format *fmt,
 			   unsigned int sizes[], void *alloc_ctxs[])
 {
 	struct uvc_video_queue *queue = vb2_get_drv_priv(vq);
-	struct uvc_streaming *stream =
-			container_of(queue, struct uvc_streaming, queue);
 
 	if (*nbuffers > UVC_MAX_VIDEO_BUFFERS)
 		*nbuffers = UVC_MAX_VIDEO_BUFFERS;
 
 	*nplanes = 1;
 
-	sizes[0] = stream->ctrl.dwMaxVideoFrameSize;
+	sizes[0] = queue->stream->ctrl.dwMaxVideoFrameSize;
 
 	return 0;
 }
@@ -107,11 +105,9 @@ static void uvc_buffer_queue(struct vb2_buffer *vb)
 static int uvc_buffer_finish(struct vb2_buffer *vb)
 {
 	struct uvc_video_queue *queue = vb2_get_drv_priv(vb->vb2_queue);
-	struct uvc_streaming *stream =
-			container_of(queue, struct uvc_streaming, queue);
 	struct uvc_buffer *buf = container_of(vb, struct uvc_buffer, buf);
 
-	uvc_video_clock_update(stream, &vb->v4l2_buf, buf);
+	uvc_video_clock_update(queue->stream, &vb->v4l2_buf, buf);
 	return 0;
 }
 
@@ -139,7 +135,7 @@ static struct vb2_ops uvc_queue_qops = {
 };
 
 int uvc_queue_init(struct uvc_video_queue *queue, enum v4l2_buf_type type,
-		    int drop_corrupted)
+		    int drop_corrupted, struct uvc_streaming *stream)
 {
 	int ret;
 
@@ -158,6 +154,7 @@ int uvc_queue_init(struct uvc_video_queue *queue, enum v4l2_buf_type type,
 	spin_lock_init(&queue->irqlock);
 	INIT_LIST_HEAD(&queue->irqqueue);
 	queue->flags = drop_corrupted ? UVC_QUEUE_DROP_CORRUPTED : 0;
+	queue->stream = stream;
 
 	return 0;
 }
diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
index decd65f..dbbe93b 100644
--- a/drivers/media/usb/uvc/uvc_v4l2.c
+++ b/drivers/media/usb/uvc/uvc_v4l2.c
@@ -143,12 +143,63 @@ static __u32 uvc_try_frame_interval(struct uvc_frame *frame, __u32 interval)
 	return interval;
 }
 
+static void uvc_init_vp8_probe_features(struct v4l2_format *fmt,
+		struct uvc_streaming *stream, struct uvc_frame *frame,
+		struct uvc_streaming_control *probe)
+{
+	int i;
+	__u16 layout = 0;
+	int num_streams;
+	int num_temprl_layers;
+
+	for (i = 0; i < UVC_NUM_SIMULCAST_STREAMS; ++i)
+		probe->bmLayoutPerStream[i] = 0;
+
+	/* Values to request in P&C. */
+	num_streams = fmt->fmt.pix.pixelformat == V4L2_PIX_FMT_VP8 ?
+			1 : stream->num_layers;
+
+	if (frame->bmSupportedUsages
+			& UVC_FRM_VP8_USAGE_REAL_TIME_TEMPORAL_LAYERS) {
+		/* Temporal layering supported, specify config for each
+		 * temporal layer for each stream.
+		 */
+		probe->bUsage = 2;
+		num_temprl_layers = frame->bmScalabilityCapabilities + 1;
+		num_temprl_layers = clamp(num_temprl_layers, 1,
+						UVC_VP8_MAX_TEMPORAL_LAYERS);
+
+		/* Allow all special frames for all temporal layers. */
+		for (i = 0; i < num_temprl_layers; ++i)
+			layout |= UVC_VP8_TEMPORAL_LAYOUT_WILDCARD_LAYER(i);
+		layout |= UVC_VP8_TEMPRL_LAYOUT_NUM(num_temprl_layers);
+		/* Stream enable */
+		layout |= 0x1;
+
+	} else if (frame->bmSupportedUsages & UVC_FRM_VP8_USAGE_REAL_TIME) {
+		probe->bUsage = 1;
+		/* Stream enable */
+		layout = 0x1;
+
+	} else {
+		/* Let the device decide. */
+		probe->bUsage = 0;
+		return;
+	}
+
+	for (i = 0; i < num_streams; ++i)
+		probe->bmLayoutPerStream[i] = layout;
+
+	probe->bmRateControlModes = 0x1111;
+}
+
 static int uvc_v4l2_try_format(struct uvc_streaming *stream,
 	struct v4l2_format *fmt, struct uvc_streaming_control *probe,
 	struct uvc_format **uvc_format, struct uvc_frame **uvc_frame)
 {
 	struct uvc_format *format = NULL;
 	struct uvc_frame *frame = NULL;
+	struct uvc_frame *simulcast_frame = NULL;
 	__u16 rw, rh;
 	unsigned int d, maxd;
 	unsigned int i;
@@ -218,7 +269,26 @@ static int uvc_v4l2_try_format(struct uvc_streaming *stream,
 	memset(probe, 0, sizeof *probe);
 	probe->bmHint = 1;	/* dwFrameInterval */
 	probe->bFormatIndex = format->index;
-	probe->bFrameIndex = frame->bFrameIndex;
+	/*
+	 * If we are doing simulcast, we'd like to select the highest possible
+	 * resolution available and then scale down and back up as needed
+	 * via set resolution EU control.
+	 */
+	if (fmt->fmt.pix.pixelformat == V4L2_PIX_FMT_VP8_SIMULCAST) {
+		/* TODO: resolution-framerate tradeoff negotiation logic
+		 * for simulcast would be nice to have. */
+		simulcast_frame = &format->frame[0];
+		for (i = 0; i < format->nframes; ++i) {
+			if (format->frame[i].wWidth > simulcast_frame->wWidth)
+				simulcast_frame = &format->frame[i];
+		}
+		uvc_trace(UVC_TRACE_FORMAT, "Selected %dx%d max resolution "
+				"for simulcast\n", simulcast_frame->wWidth,
+				simulcast_frame->wHeight);
+		probe->bFrameIndex = simulcast_frame->bFrameIndex;
+	} else {
+		probe->bFrameIndex = frame->bFrameIndex;
+	}
 	probe->dwFrameInterval = uvc_try_frame_interval(frame, interval);
 	/* Some webcams stall the probe control set request when the
 	 * dwMaxVideoFrameSize field is set to zero. The UVC specification
@@ -237,6 +307,12 @@ static int uvc_v4l2_try_format(struct uvc_streaming *stream,
 		probe->dwMaxVideoFrameSize =
 			stream->ctrl.dwMaxVideoFrameSize;
 
+	/* UVC 1.5 features */
+	if (uvc_get_probe_ctrl_size(stream) > 34
+		&& (fmt->fmt.pix.pixelformat == V4L2_PIX_FMT_VP8
+		|| fmt->fmt.pix.pixelformat == V4L2_PIX_FMT_VP8_SIMULCAST))
+			uvc_init_vp8_probe_features(fmt, stream, frame, probe);
+
 	/* Probe the device. */
 	ret = uvc_probe_video(stream, probe);
 	mutex_unlock(&stream->mutex);
@@ -260,11 +336,12 @@ done:
 	return ret;
 }
 
-static int uvc_v4l2_get_format(struct uvc_streaming *stream,
-	struct v4l2_format *fmt)
+static int uvc_v4l2_get_format(struct uvc_stream_layer *layer,
+				struct v4l2_format *fmt)
 {
 	struct uvc_format *format;
 	struct uvc_frame *frame;
+	struct uvc_streaming *stream = layer->stream;
 	int ret = 0;
 
 	if (fmt->type != stream->type)
@@ -280,8 +357,11 @@ static int uvc_v4l2_get_format(struct uvc_streaming *stream,
 	}
 
 	fmt->fmt.pix.pixelformat = format->fcc;
-	fmt->fmt.pix.width = frame->wWidth;
-	fmt->fmt.pix.height = frame->wHeight;
+	ret = uvc_video_layer_get_resolution(layer);
+	if (ret)
+		goto done;
+	fmt->fmt.pix.width = layer->width;
+	fmt->fmt.pix.height = layer->height;
 	fmt->fmt.pix.field = V4L2_FIELD_NONE;
 	fmt->fmt.pix.bytesperline = format->bpp * frame->wWidth / 8;
 	fmt->fmt.pix.sizeimage = stream->ctrl.dwMaxVideoFrameSize;
@@ -293,13 +373,15 @@ done:
 	return ret;
 }
 
-static int uvc_v4l2_set_format(struct uvc_streaming *stream,
-	struct v4l2_format *fmt)
+static int uvc_v4l2_set_format(struct uvc_stream_layer *layer,
+				struct v4l2_format *fmt)
 {
 	struct uvc_streaming_control probe;
 	struct uvc_format *format;
 	struct uvc_frame *frame;
+	struct uvc_streaming *stream = layer->stream;
 	int ret;
+	int i;
 
 	if (fmt->type != stream->type)
 		return -EINVAL;
@@ -309,18 +391,50 @@ static int uvc_v4l2_set_format(struct uvc_streaming *stream,
 		return ret;
 
 	mutex_lock(&stream->mutex);
+	if (stream->allocated_layers == 0) {
+		/* No queue has yet allocated buffers, so we can
+		 * set format via a commit.
+		 */
+		stream->ctrl = probe;
+		uvc_update_stream_format(stream, format, frame);
 
-	if (uvc_queue_allocated(&stream->queue)) {
-		ret = -EBUSY;
-		goto done;
-	}
+		/* Commit the streaming parameters. */
+		ret = uvc_commit_video(stream, &stream->ctrl);
+		if (ret < 0) {
+			uvc_trace(UVC_TRACE_FORMAT,
+					"Failed committing video\n");
+			goto end;
+		}
 
-	stream->ctrl = probe;
-	stream->cur_format = format;
-	stream->cur_frame = frame;
+		if (uvc_is_simulcasting(stream)) {
+			ret = uvc_video_layer_set_resolution(layer,
+				fmt->fmt.pix.width, fmt->fmt.pix.height);
+			if (ret)
+				goto end;
+			for (i = 0; i < stream->num_negotiated_layers; ++i) {
+				uvc_video_layer_get_resolution(
+							&stream->layers[i]);
+			}
+		}
+	} else {
+		/* At least one queue has already allocated buffers, so we
+		 * can only change resolution and only if simulcasting.
+		 */
+		if (fmt->fmt.pix.pixelformat == V4L2_PIX_FMT_VP8_SIMULCAST
+		    && fmt->fmt.pix.pixelformat == stream->cur_format->fcc) {
+			ret = uvc_video_layer_set_resolution(layer,
+				fmt->fmt.pix.width, fmt->fmt.pix.height);
+		} else {
+			uvc_trace(UVC_TRACE_FORMAT, "Cannot change format, "
+					"already initialized/streaming.\n");
+			ret = -EBUSY;
+		}
+	}
 
-done:
+end:
 	mutex_unlock(&stream->mutex);
+	if (ret < 0)
+		uvc_trace(UVC_TRACE_FORMAT, "Failed setting format\n");
 	return ret;
 }
 
@@ -382,7 +496,7 @@ static int uvc_v4l2_set_streamparm(struct uvc_streaming *stream,
 
 	mutex_lock(&stream->mutex);
 
-	if (uvc_queue_streaming(&stream->queue)) {
+	if (stream->allocated_layers != 0) {
 		mutex_unlock(&stream->mutex);
 		return -EBUSY;
 	}
@@ -399,8 +513,15 @@ static int uvc_v4l2_set_streamparm(struct uvc_streaming *stream,
 	}
 
 	stream->ctrl = probe;
+
+	ret = uvc_commit_video(stream, &stream->ctrl);
 	mutex_unlock(&stream->mutex);
 
+	if (ret < 0) {
+		uvc_trace(UVC_TRACE_FORMAT, "Failed committing video\n");
+		return ret;
+	}
+
 	/* Return the actual frame period. */
 	timeperframe.numerator = probe.dwFrameInterval;
 	timeperframe.denominator = 10000000;
@@ -449,8 +570,8 @@ static int uvc_acquire_privileges(struct uvc_fh *handle)
 		return 0;
 
 	/* Check if the device already has a privileged handle. */
-	if (atomic_inc_return(&handle->stream->active) != 1) {
-		atomic_dec(&handle->stream->active);
+	if (atomic_inc_return(&handle->layer->active) != 1) {
+		atomic_dec(&handle->layer->active);
 		return -EBUSY;
 	}
 
@@ -461,7 +582,7 @@ static int uvc_acquire_privileges(struct uvc_fh *handle)
 static void uvc_dismiss_privileges(struct uvc_fh *handle)
 {
 	if (handle->state == UVC_HANDLE_ACTIVE)
-		atomic_dec(&handle->stream->active);
+		atomic_dec(&handle->layer->active);
 
 	handle->state = UVC_HANDLE_PASSIVE;
 }
@@ -478,11 +599,16 @@ static int uvc_has_privileges(struct uvc_fh *handle)
 static int uvc_v4l2_open(struct file *file)
 {
 	struct uvc_streaming *stream;
+	struct uvc_stream_layer *layer;
 	struct uvc_fh *handle;
 	int ret = 0;
 
 	uvc_trace(UVC_TRACE_CALLS, "uvc_v4l2_open\n");
-	stream = video_drvdata(file);
+	layer = video_drvdata(file);
+	WARN_ON(!layer);
+	stream = layer->stream;
+	WARN_ON(!stream);
+	WARN_ON(!stream->dev);
 
 	if (stream->dev->state & UVC_DEV_DISCONNECTED)
 		return -ENODEV;
@@ -512,10 +638,10 @@ static int uvc_v4l2_open(struct file *file)
 	stream->dev->users++;
 	mutex_unlock(&stream->dev->lock);
 
-	v4l2_fh_init(&handle->vfh, stream->vdev);
+	v4l2_fh_init(&handle->vfh, stream->vdev[layer->layer_id]);
 	v4l2_fh_add(&handle->vfh);
 	handle->chain = stream->chain;
-	handle->stream = stream;
+	handle->layer = layer;
 	handle->state = UVC_HANDLE_PASSIVE;
 	file->private_data = handle;
 
@@ -525,14 +651,19 @@ static int uvc_v4l2_open(struct file *file)
 static int uvc_v4l2_release(struct file *file)
 {
 	struct uvc_fh *handle = file->private_data;
-	struct uvc_streaming *stream = handle->stream;
+	struct uvc_streaming *stream = handle->layer->stream;
 
 	uvc_trace(UVC_TRACE_CALLS, "uvc_v4l2_release\n");
 
 	/* Only free resources if this is a privileged handle. */
 	if (uvc_has_privileges(handle)) {
-		uvc_video_enable(stream, 0);
-		uvc_free_buffers(&stream->queue);
+		mutex_lock(&stream->mutex);
+
+		uvc_video_layer_disable(handle->layer);
+		clear_bit(handle->layer->layer_id, &stream->allocated_layers);
+
+		mutex_unlock(&stream->mutex);
+		uvc_free_buffers(&handle->layer->queue);
 	}
 
 	/* Release the file handle. */
@@ -556,7 +687,7 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 	struct video_device *vdev = video_devdata(file);
 	struct uvc_fh *handle = file->private_data;
 	struct uvc_video_chain *chain = handle->chain;
-	struct uvc_streaming *stream = handle->stream;
+	struct uvc_streaming *stream = handle->layer->stream;
 	long ret = 0;
 
 	switch (cmd) {
@@ -612,6 +743,12 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (ret < 0)
 			return ret;
 
+		ret = __uvc_video_layer_select(handle->layer, 0);
+		if (ret < 0) {
+			uvc_ctrl_rollback(handle);
+			return ret;
+		}
+
 		ret = uvc_ctrl_get(chain, &xctrl,
 					uvc_is_stream_streaming(stream));
 		uvc_ctrl_rollback(handle);
@@ -637,6 +774,13 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (ret < 0)
 			return ret;
 
+		/* Apply to all temporal layers. */
+		ret = __uvc_video_layer_select(handle->layer, 0x7);
+		if (ret < 0) {
+			uvc_ctrl_rollback(handle);
+			return ret;
+		}
+
 		ret = uvc_ctrl_set(chain, &xctrl,
 					uvc_is_stream_streaming(stream));
 		if (ret < 0) {
@@ -663,6 +807,13 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (ret < 0)
 			return ret;
 
+		ret = __uvc_video_layer_select(handle->layer,
+						ctrls->reserved[0]);
+		if (ret < 0) {
+			uvc_ctrl_rollback(handle);
+			return ret;
+		}
+
 		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
 			ret = uvc_ctrl_get(chain, ctrl,
 					uvc_is_stream_streaming(stream));
@@ -692,6 +843,13 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (ret < 0)
 			return ret;
 
+		ret = __uvc_video_layer_select(handle->layer,
+						ctrls->reserved[0]);
+		if (ret < 0) {
+			uvc_ctrl_rollback(handle);
+			return ret;
+		}
+
 		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
 			ret = uvc_ctrl_set(chain, ctrl,
 					uvc_is_stream_streaming(stream));
@@ -845,10 +1003,10 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if ((ret = uvc_acquire_privileges(handle)) < 0)
 			return ret;
 
-		return uvc_v4l2_set_format(stream, arg);
+		return uvc_v4l2_set_format(handle->layer, arg);
 
 	case VIDIOC_G_FMT:
-		return uvc_v4l2_get_format(stream, arg);
+		return uvc_v4l2_get_format(handle->layer, arg);
 
 	/* Frame size enumeration */
 	case VIDIOC_ENUM_FRAMESIZES:
@@ -990,15 +1148,18 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 			return ret;
 
 		mutex_lock(&stream->mutex);
-		ret = uvc_alloc_buffers(&stream->queue, arg);
-		mutex_unlock(&stream->mutex);
-		if (ret < 0)
-			return ret;
-
-		if (ret == 0)
+		ret = uvc_alloc_buffers(&handle->layer->queue, arg);
+		if (ret > 0) {
+			set_bit(handle->layer->layer_id,
+				&stream->allocated_layers);
+			ret = 0;
+		} else if (ret == 0) {
+			clear_bit(handle->layer->layer_id,
+					&stream->allocated_layers);
 			uvc_dismiss_privileges(handle);
+		}
+		mutex_unlock(&stream->mutex);
 
-		ret = 0;
 		break;
 
 	case VIDIOC_QUERYBUF:
@@ -1008,20 +1169,20 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (!uvc_has_privileges(handle))
 			return -EBUSY;
 
-		return uvc_query_buffer(&stream->queue, buf);
+		return uvc_query_buffer(&handle->layer->queue, buf);
 	}
 
 	case VIDIOC_QBUF:
 		if (!uvc_has_privileges(handle))
 			return -EBUSY;
 
-		return uvc_queue_buffer(&stream->queue, arg);
+		return uvc_queue_buffer(&handle->layer->queue, arg);
 
 	case VIDIOC_DQBUF:
 		if (!uvc_has_privileges(handle))
 			return -EBUSY;
 
-		return uvc_dequeue_buffer(&stream->queue, arg,
+		return uvc_dequeue_buffer(&handle->layer->queue, arg,
 			file->f_flags & O_NONBLOCK);
 
 	case VIDIOC_STREAMON:
@@ -1039,10 +1200,9 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 			return -EBUSY;
 
 		mutex_lock(&stream->mutex);
-		ret = uvc_video_enable(stream, 1);
+		ret = uvc_video_layer_enable(handle->layer);
 		mutex_unlock(&stream->mutex);
-		if (ret < 0)
-			return ret;
+
 		break;
 	}
 
@@ -1060,7 +1220,11 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 		if (!uvc_has_privileges(handle))
 			return -EBUSY;
 
-		return uvc_video_enable(stream, 0);
+		mutex_lock(&stream->mutex);
+		uvc_video_layer_disable(handle->layer);
+		mutex_unlock(&stream->mutex);
+
+		break;
 	}
 
 	case VIDIOC_SUBSCRIBE_EVENT:
@@ -1334,21 +1498,19 @@ static ssize_t uvc_v4l2_read(struct file *file, char __user *data,
 static int uvc_v4l2_mmap(struct file *file, struct vm_area_struct *vma)
 {
 	struct uvc_fh *handle = file->private_data;
-	struct uvc_streaming *stream = handle->stream;
 
 	uvc_trace(UVC_TRACE_CALLS, "uvc_v4l2_mmap\n");
 
-	return uvc_queue_mmap(&stream->queue, vma);
+	return uvc_queue_mmap(&handle->layer->queue, vma);
 }
 
 static unsigned int uvc_v4l2_poll(struct file *file, poll_table *wait)
 {
 	struct uvc_fh *handle = file->private_data;
-	struct uvc_streaming *stream = handle->stream;
 
 	uvc_trace(UVC_TRACE_CALLS, "uvc_v4l2_poll\n");
 
-	return uvc_queue_poll(&stream->queue, file, wait);
+	return uvc_queue_poll(&handle->layer->queue, file, wait);
 }
 
 #ifndef CONFIG_MMU
@@ -1357,11 +1519,10 @@ static unsigned long uvc_v4l2_get_unmapped_area(struct file *file,
 		unsigned long flags)
 {
 	struct uvc_fh *handle = file->private_data;
-	struct uvc_streaming *stream = handle->stream;
 
 	uvc_trace(UVC_TRACE_CALLS, "uvc_v4l2_get_unmapped_area\n");
 
-	return uvc_queue_get_unmapped_area(&stream->queue, pgoff);
+	return uvc_queue_get_unmapped_area(&handle->layer->queue, pgoff);
 }
 #endif
 
diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c
index 0291817..2d1ed2e 100644
--- a/drivers/media/usb/uvc/uvc_video.c
+++ b/drivers/media/usb/uvc/uvc_video.c
@@ -344,6 +344,14 @@ static int uvc_set_video_ctrl(struct uvc_streaming *stream,
 		ret = 0;
 	}
 
+	/* Set EU control streaming state for each layer. */
+	if (uvc_is_eu_active(stream)) {
+		for (i = 0; i < stream->num_negotiated_layers; ++i) {
+			uvc_ctrl_layer_enable(&stream->layers[i],
+				 !!test_bit(i, &stream->streaming_layers));
+		}
+	}
+
 	kfree(data);
 	return ret;
 }
@@ -404,13 +412,17 @@ int uvc_probe_video(struct uvc_streaming *stream,
 		probe->wPFrameRate = probe_min.wPFrameRate;
 		probe->wCompQuality = probe_max.wCompQuality;
 		probe->wCompWindowSize = probe_min.wCompWindowSize;
+
+		/* TODO: negotiate bmLayoutPerStream for simulcast if
+		 * adjusted by the camera.
+		 */
 	}
 
 done:
 	return ret;
 }
 
-static int uvc_commit_video(struct uvc_streaming *stream,
+int uvc_commit_video(struct uvc_streaming *stream,
 			    struct uvc_streaming_control *probe)
 {
 	return uvc_set_video_ctrl(stream, probe, 0);
@@ -808,7 +820,7 @@ static void uvc_video_stats_update(struct uvc_streaming *stream)
 	uvc_trace(UVC_TRACE_STATS, "frame %u stats: %u/%u/%u packets, "
 		  "%u/%u/%u pts (%searly %sinitial), %u/%u scr, "
 		  "last pts/stc/sof %u/%u/%u\n",
-		  stream->sequence, frame->first_data,
+		  stream->layers[stream->layer_id].sequence, frame->first_data,
 		  frame->nb_packets - frame->nb_empty, frame->nb_packets,
 		  frame->nb_pts_diffs, frame->last_pts_diff, frame->nb_pts,
 		  frame->has_early_pts ? "" : "!",
@@ -943,13 +955,14 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 		struct uvc_buffer *buf, struct uvc_payload_header *header)
 {
 	u8 fid = header->fid;
+	struct uvc_stream_layer *layer = uvc_get_curr_layer(stream);
 
 	/* Increase the sequence number regardless of any buffer states, so
 	 * that discontinuous sequence numbers always indicate lost frames.
 	 */
-	if (stream->last_fid != fid) {
-		stream->sequence++;
-		if (stream->sequence)
+	if (layer->last_fid != fid) {
+		layer->sequence++;
+		if (layer->sequence)
 			uvc_video_stats_update(stream);
 	}
 
@@ -960,7 +973,9 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 	 * NULL.
 	 */
 	if (buf == NULL) {
-		stream->last_fid = fid;
+		uvc_trace(UVC_TRACE_FRAME, "No buffers available for layer %d, "
+				"dropping payload\n", layer->layer_id);
+		layer->last_fid = fid;
 		return -ENODATA;
 	}
 
@@ -982,12 +997,13 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 	if (buf->state != UVC_BUF_STATE_ACTIVE) {
 		struct timespec ts;
 
-		if (fid == stream->last_fid) {
+		if (fid == layer->last_fid) {
 			uvc_trace(UVC_TRACE_FRAME, "Dropping payload (out of "
-				"sync).\n");
+				"sync: fid: %d, last: %d) for layer: %d.\n",
+				fid, layer->last_fid, layer->layer_id);
 			if ((stream->dev->quirks & UVC_QUIRK_STREAM_NO_FID) &&
 			    (header->has_eof))
-				stream->last_fid ^= UVC_STREAM_FID;
+				layer->last_fid ^= UVC_STREAM_FID;
 			return -ENODATA;
 		}
 
@@ -996,7 +1012,7 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 		else
 			ktime_get_real_ts(&ts);
 
-		buf->buf.v4l2_buf.sequence = stream->sequence;
+		buf->buf.v4l2_buf.sequence = layer->sequence;
 		buf->buf.v4l2_buf.timestamp.tv_sec = ts.tv_sec;
 		buf->buf.v4l2_buf.timestamp.tv_usec =
 			ts.tv_nsec / NSEC_PER_USEC;
@@ -1012,7 +1028,7 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 	 * last payload can be lost anyway). We thus must check if the FID has
 	 * been toggled.
 	 *
-	 * stream->last_fid is initialized to -1, so the first isochronous
+	 * layer->last_fid is initialized to -1, so the first isochronous
 	 * frame will never trigger an end of frame detection.
 	 *
 	 * Empty buffers (bytesused == 0) don't trigger end of frame detection
@@ -1020,14 +1036,14 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
 	 * avoids detecting end of frame conditions at FID toggling if the
 	 * previous payload had the EOF bit set.
 	 */
-	if (fid != stream->last_fid && buf->bytesused != 0) {
+	if (fid != layer->last_fid && buf->bytesused != 0) {
 		uvc_trace(UVC_TRACE_FRAME, "Frame complete (FID bit "
 				"toggled).\n");
 		buf->state = UVC_BUF_STATE_READY;
 		return -EAGAIN;
 	}
 
-	stream->last_fid = fid;
+	layer->last_fid = fid;
 
 	return 0;
 }
@@ -1060,6 +1076,8 @@ static void uvc_video_decode_data(struct uvc_streaming *stream,
 static void uvc_video_decode_end(struct uvc_streaming *stream,
 		struct uvc_buffer *buf, struct uvc_payload_header *header)
 {
+	struct uvc_stream_layer *layer = uvc_get_curr_layer(stream);
+
 	/* Mark the buffer as done if the EOF marker is set. */
 	if (header->has_eof && buf->bytesused != 0) {
 		uvc_trace(UVC_TRACE_FRAME, "Frame complete (EOF found).\n");
@@ -1067,7 +1085,7 @@ static void uvc_video_decode_end(struct uvc_streaming *stream,
 			uvc_trace(UVC_TRACE_FRAME, "EOF in empty payload.\n");
 		buf->state = UVC_BUF_STATE_READY;
 		if (stream->dev->quirks & UVC_QUIRK_STREAM_NO_FID)
-			stream->last_fid ^= UVC_STREAM_FID;
+			layer->last_fid ^= UVC_STREAM_FID;
 	}
 }
 
@@ -1087,14 +1105,14 @@ static int uvc_video_encode_header(struct uvc_streaming *stream,
 {
 	data[0] = 2;	/* Header length */
 	data[1] = UVC_STREAM_EOH | UVC_STREAM_EOF
-		| (stream->last_fid & UVC_STREAM_FID);
+		| (stream->layers[0].last_fid & UVC_STREAM_FID);
 	return 2;
 }
 
 static int uvc_video_encode_data(struct uvc_streaming *stream,
 		struct uvc_buffer *buf, __u8 *data, int len)
 {
-	struct uvc_video_queue *queue = &stream->queue;
+	struct uvc_video_queue *queue = &stream->layers[0].queue;
 	unsigned int nbytes;
 	void *mem;
 
@@ -1110,6 +1128,37 @@ static int uvc_video_encode_data(struct uvc_streaming *stream,
 	return nbytes;
 }
 
+static int uvc_video_decode_layer_id(struct uvc_streaming *stream,
+					struct uvc_payload_header *header)
+{
+	unsigned int layer_id;
+
+	if (header->has_sli) {
+		layer_id = (header->sli >> UVC_STREAM_STREAM_ID_SHIFT)
+			  & UVC_STREAM_STREAM_ID_MASK;
+		if (layer_id >= stream->num_layers) {
+			uvc_trace(UVC_TRACE_FRAME, "Invalid layer id (%d)\n",
+					layer_id);
+			return -EINVAL;
+		}
+
+		stream->prev_layer_id = stream->layer_id;
+		stream->layer_id = layer_id;
+		stream->temporal_id = (header->sli >> UVC_STREAM_TMPRL_ID_SHIFT)
+				    & UVC_STREAM_TEMPRL_ID_MASK;
+
+		header->buf_flags |= (stream->temporal_id
+					& V4L2_BUF_FLAG_LAYER_STRUCTURE_MASK)
+				  << V4L2_BUF_FLAG_LAYER_STRUCTURE_SHIFT;
+	} else {
+		stream->prev_layer_id = stream->layer_id;
+		stream->layer_id = 0;
+		stream->temporal_id = 0;
+	}
+
+	return 0;
+}
+
 static int uvc_video_parse_header(struct uvc_streaming *stream,
 		const __u8 *data, int len, struct uvc_payload_header *header)
 {
@@ -1138,7 +1187,8 @@ static int uvc_video_parse_header(struct uvc_streaming *stream,
 
 	header->buf_flags = 0;
 
-	if (stream->cur_format->fcc == V4L2_PIX_FMT_VP8) {
+	if (stream->cur_format->fcc == V4L2_PIX_FMT_VP8
+		|| stream->cur_format->fcc == V4L2_PIX_FMT_VP8_SIMULCAST) {
 		/* VP8 payload has 2 additional bytes of BFH. */
 		header->length += 2;
 		off += 2;
@@ -1203,6 +1253,7 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 	unsigned int len;
 	u8 *mem;
 	int ret, i;
+	struct uvc_video_queue *queue;
 	struct uvc_buffer *buf = NULL;
 	struct uvc_payload_header header;
 
@@ -1223,12 +1274,17 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 		if (ret < 0)
 			continue;
 
-		buf = uvc_queue_get_first_buf(&stream->queue);
+		ret = uvc_video_decode_layer_id(stream, &header);
+		if (ret < 0)
+			continue;
+
+		queue = &stream->layers[stream->layer_id].queue;
+		buf = uvc_queue_get_first_buf(queue);
+
 		do {
 			ret = uvc_video_decode_start(stream, buf, &header);
 			if (ret == -EAGAIN)
-				buf = uvc_queue_next_buffer(&stream->queue,
-							    buf);
+				buf = uvc_queue_next_buffer(queue, buf);
 		} while (ret == -EAGAIN);
 
 		if (ret < 0)
@@ -1249,7 +1305,7 @@ static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming *stream)
 			      UVC_FMT_FLAG_COMPRESSED))
 				buf->error = 1;
 
-			buf = uvc_queue_next_buffer(&stream->queue, buf);
+			buf = uvc_queue_next_buffer(queue, buf);
 		}
 	}
 }
@@ -1258,6 +1314,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 {
 	u8 *mem;
 	int len, ret;
+	struct uvc_video_queue *queue;
 	struct uvc_payload_header header;
 	struct uvc_buffer *buf;
 
@@ -1276,7 +1333,12 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 	if (ret < 0)
 		return;
 
-	buf = uvc_queue_get_first_buf(&stream->queue);
+	ret = uvc_video_decode_layer_id(stream, &header);
+	if (ret < 0)
+		return;
+
+	queue = &stream->layers[stream->layer_id].queue;
+	buf = uvc_queue_get_first_buf(queue);
 
 	/* If the URB is the first of its payload, decode and save the
 	 * header.
@@ -1285,8 +1347,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 		do {
 			ret = uvc_video_decode_start(stream, buf, &header);
 			if (ret == -EAGAIN)
-				buf = uvc_queue_next_buffer(&stream->queue,
-							    buf);
+				buf = uvc_queue_next_buffer(queue, buf);
 		} while (ret == -EAGAIN);
 
 		/* If an error occurred skip the rest of the payload. */
@@ -1320,8 +1381,7 @@ static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming *stream)
 		if (!stream->bulk.skip_payload && buf != NULL) {
 			uvc_video_decode_end(stream, buf, &header);
 			if (buf->state == UVC_BUF_STATE_READY)
-				buf = uvc_queue_next_buffer(&stream->queue,
-							    buf);
+				buf = uvc_queue_next_buffer(queue, buf);
 		}
 
 		stream->bulk.header_size = 0;
@@ -1334,9 +1394,10 @@ static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming *stream)
 {
 	u8 *mem = urb->transfer_buffer;
 	int len = stream->urb_size, ret;
+	struct uvc_stream_layer *layer = &stream->layers[0];
 	struct uvc_buffer *buf;
 
-	buf = uvc_queue_get_first_buf(&stream->queue);
+	buf = uvc_queue_get_first_buf(&layer->queue);
 	if (buf == NULL) {
 		urb->transfer_buffer_length = 0;
 		return;
@@ -1357,14 +1418,15 @@ static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming *stream)
 	stream->bulk.payload_size += ret;
 	len -= ret;
 
-	if (buf->bytesused == stream->queue.buf_used ||
+	if (buf->bytesused == layer->queue.buf_used ||
 	    stream->bulk.payload_size == stream->bulk.max_payload_size) {
-		if (buf->bytesused == stream->queue.buf_used) {
-			stream->queue.buf_used = 0;
+		if (buf->bytesused == layer->queue.buf_used) {
+			layer->queue.buf_used = 0;
 			buf->state = UVC_BUF_STATE_READY;
-			buf->buf.v4l2_buf.sequence = ++stream->sequence;
-			uvc_queue_next_buffer(&stream->queue, buf);
-			stream->last_fid ^= UVC_STREAM_FID;
+			buf->buf.v4l2_buf.sequence =
+				++layer->sequence;
+			uvc_queue_next_buffer(&layer->queue, buf);
+			layer->last_fid ^= UVC_STREAM_FID;
 		}
 
 		stream->bulk.header_size = 0;
@@ -1378,6 +1440,7 @@ static void uvc_video_complete(struct urb *urb)
 {
 	struct uvc_streaming *stream = urb->context;
 	int ret;
+	int i;
 
 	switch (urb->status) {
 	case 0:
@@ -1393,7 +1456,9 @@ static void uvc_video_complete(struct urb *urb)
 
 	case -ECONNRESET:	/* usb_unlink_urb() called. */
 	case -ESHUTDOWN:	/* The endpoint is being disabled. */
-		uvc_queue_cancel(&stream->queue, urb->status == -ESHUTDOWN);
+		for (i = 0; i < stream->num_layers; ++i)
+			uvc_queue_cancel(&stream->layers[i].queue,
+						urb->status == -ESHUTDOWN);
 		return;
 	}
 
@@ -1648,8 +1713,10 @@ static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
 	unsigned int i;
 	int ret;
 
-	stream->sequence = -1;
-	stream->last_fid = -1;
+	for (i = 0; i < stream->num_layers; ++i) {
+		stream->layers[i].sequence = -1;
+		stream->layers[i].last_fid = -1;
+	}
 	stream->bulk.header_size = 0;
 	stream->bulk.skip_payload = 0;
 	stream->bulk.payload_size = 0;
@@ -1748,7 +1815,7 @@ static int uvc_init_video(struct uvc_streaming *stream, gfp_t gfp_flags)
  */
 int uvc_video_suspend(struct uvc_streaming *stream)
 {
-	if (!uvc_queue_streaming(&stream->queue))
+	if (stream->streaming_layers == 0)
 		return 0;
 
 	stream->frozen = 1;
@@ -1761,13 +1828,14 @@ int uvc_video_suspend(struct uvc_streaming *stream)
  * Reconfigure the video interface and restart streaming if it was enabled
  * before suspend.
  *
- * If an error occurs, disable the video queue. This will wake all pending
+ * If an error occurs, disable video queues. This will wake all pending
  * buffers, making sure userspace applications are notified of the problem
  * instead of waiting forever.
  */
 int uvc_video_resume(struct uvc_streaming *stream, int reset)
 {
 	int ret;
+	int i;
 
 	/* If the bus has been reset on resume, set the alternate setting to 0.
 	 * This should be the default value, but some devices crash or otherwise
@@ -1782,18 +1850,23 @@ int uvc_video_resume(struct uvc_streaming *stream, int reset)
 	uvc_video_clock_reset(stream);
 
 	ret = uvc_commit_video(stream, &stream->ctrl);
-	if (ret < 0) {
-		uvc_queue_enable(&stream->queue, 0);
-		return ret;
-	}
+	if (ret < 0)
+		goto fail;
 
-	if (!uvc_queue_streaming(&stream->queue))
+	/* At least one queue must be streaming for us to re-enable video. */
+	if (stream->streaming_layers == 0)
 		return 0;
 
 	ret = uvc_init_video(stream, GFP_NOIO);
 	if (ret < 0)
-		uvc_queue_enable(&stream->queue, 0);
+		goto fail;
 
+	return 0;
+
+fail:
+	for (i = 0; i < stream->num_layers; ++i)
+		uvc_queue_enable(&stream->layers[i].queue, 0);
+	stream->streaming_layers = 0;
 	return ret;
 }
 
@@ -1824,13 +1897,6 @@ int uvc_video_init(struct uvc_streaming *stream)
 		return -EINVAL;
 	}
 
-	atomic_set(&stream->active, 0);
-
-	/* Initialize the video buffers queue. */
-	ret = uvc_queue_init(&stream->queue, stream->type, !uvc_no_drop_param);
-	if (ret)
-		return ret;
-
 	/* Alternate setting 0 should be the default, yet the XBox Live Vision
 	 * Cam (and possibly other devices) crash or otherwise misbehave if
 	 * they don't receive a SET_INTERFACE request before any other video
@@ -1884,9 +1950,7 @@ int uvc_video_init(struct uvc_streaming *stream)
 	probe->bFormatIndex = format->index;
 	probe->bFrameIndex = frame->bFrameIndex;
 
-	stream->def_format = format;
-	stream->cur_format = format;
-	stream->cur_frame = frame;
+	uvc_update_stream_format(stream, format, frame);
 
 	/* Select the video decoding function */
 	if (stream->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
@@ -1906,37 +1970,43 @@ int uvc_video_init(struct uvc_streaming *stream)
 		}
 	}
 
+	/* If an encoding unit is present, initialize simulcast capabilities. */
+	uvc_ctrl_simulcast_init(stream);
+
 	return 0;
 }
 
 /*
- * Enable or disable the video stream.
+ * Called on STREAMOFF/release for the last layer to be disabled.
  */
-int uvc_video_enable(struct uvc_streaming *stream, int enable)
+static void uvc_video_disable(struct uvc_streaming *stream)
+{
+	uvc_trace(UVC_TRACE_VIDEO, "Disabling video\n");
+
+	uvc_uninit_video(stream, 1);
+	usb_set_interface(stream->dev->udev, stream->intfnum, 0);
+	uvc_video_clock_cleanup(stream);
+	/*
+	 * UVC standard unfortunately specifies that after setting interface
+	 * to 0, EU state is undefined until next commit.
+	 * So commit to be able to query EU controls.
+	 */
+	uvc_commit_video(stream, &stream->ctrl);
+}
+
+/*
+ * Called on STREAMON for the first layer to be enabled.
+ */
+static int uvc_video_enable(struct uvc_streaming *stream)
 {
 	int ret;
 
-	if (!enable) {
-		uvc_uninit_video(stream, 1);
-		usb_set_interface(stream->dev->udev, stream->intfnum, 0);
-		uvc_queue_enable(&stream->queue, 0);
-		uvc_video_clock_cleanup(stream);
-		return 0;
-	}
+	uvc_trace(UVC_TRACE_VIDEO, "Enabling video\n");
 
 	ret = uvc_video_clock_init(stream);
 	if (ret < 0)
 		return ret;
 
-	ret = uvc_queue_enable(&stream->queue, 1);
-	if (ret < 0)
-		goto error_queue;
-
-	/* Commit the streaming parameters. */
-	ret = uvc_commit_video(stream, &stream->ctrl);
-	if (ret < 0)
-		goto error_commit;
-
 	ret = uvc_init_video(stream, GFP_KERNEL);
 	if (ret < 0)
 		goto error_video;
@@ -1944,11 +2014,89 @@ int uvc_video_enable(struct uvc_streaming *stream, int enable)
 	return 0;
 
 error_video:
+	uvc_printk(KERN_ERR, "Failed to enable video!\n");
 	usb_set_interface(stream->dev->udev, stream->intfnum, 0);
-error_commit:
-	uvc_queue_enable(&stream->queue, 0);
-error_queue:
 	uvc_video_clock_cleanup(stream);
 
 	return ret;
 }
+
+/* Must be called with stream->mutex taken. */
+int uvc_video_layer_enable(struct uvc_stream_layer *layer)
+{
+	struct uvc_streaming *stream = layer->stream;
+	int ret = 0;
+
+	uvc_trace(UVC_TRACE_VIDEO, "Enabling layer: %d\n", layer->layer_id);
+
+	ret = uvc_queue_enable(&layer->queue, 1);
+	/* This will also guarantee that we don't enable streaming more than
+	 * once on this layer.
+	 */
+	if (ret)
+		return ret;
+
+	/* Enable via EU control first if present/possible. This will not enable
+	 * streaming if video is not active.
+	 */
+	if (uvc_is_eu_active(stream)) {
+		ret = uvc_ctrl_layer_enable(layer, 1);
+		if (ret) {
+			uvc_queue_enable(&layer->queue, 0);
+			return ret;
+		}
+	}
+
+	if (stream->streaming_layers == 0) {
+		ret = uvc_video_enable(stream);
+		if (ret) {
+			uvc_ctrl_layer_enable(layer, 0);
+			uvc_queue_enable(&layer->queue, 0);
+			return ret;
+		}
+	}
+
+	set_bit(layer->layer_id, &stream->streaming_layers);
+	return ret;
+}
+
+/* Must be called with stream->mutex taken. */
+void uvc_video_layer_disable(struct uvc_stream_layer *layer)
+{
+	struct uvc_streaming *stream = layer->stream;
+
+	uvc_trace(UVC_TRACE_VIDEO, "Disabling layer: %d\n", layer->layer_id);
+
+	if (uvc_is_eu_active(stream))
+		uvc_ctrl_layer_enable(layer, 0);
+
+	clear_bit(layer->layer_id, &stream->streaming_layers);
+	if (stream->streaming_layers == 0)
+		uvc_video_disable(stream);
+
+	uvc_queue_enable(&layer->queue, 0);
+}
+
+void uvc_update_stream_format(struct uvc_streaming *stream,
+			struct uvc_format *format, struct uvc_frame *frame)
+{
+	int i;
+
+	stream->def_format = format;
+	stream->cur_format = format;
+	stream->cur_frame = frame;
+	stream->num_negotiated_layers = 0;
+
+	if (uvc_is_eu_active(stream)) {
+		for (i = 0; i < stream->num_layers; ++i) {
+			stream->num_negotiated_layers +=
+				uvc_is_layer_negotiated(stream, i) ? 1 : 0;
+		}
+	} else {
+		stream->num_negotiated_layers = 1;
+	}
+
+	uvc_trace(UVC_TRACE_PROBE, "Negotiated max layers: %d\n",
+			stream->num_negotiated_layers);
+}
+
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index fb21459..1df0960 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -130,6 +130,19 @@
 #define UVC_MAX_CONTROL_MAPPINGS	1024
 #define UVC_MAX_CONTROL_MENU_ENTRIES	32
 
+#define UVC_SIMULCAST_MAX_LAYERS	(UVC_NUM_SIMULCAST_STREAMS)
+#define UVC_VP8_MAX_TEMPORAL_LAYERS	4
+
+/* VP8 format/P&C features */
+#define UVC_FRM_VP8_USAGE_REAL_TIME			(1 << 1)
+#define UVC_FRM_VP8_USAGE_REAL_TIME_TEMPORAL_LAYERS	(1 << 2)
+
+/* VP8 P&C stream layout configuration (bmLayoutPerStream). */
+#define UVC_VP8_TEMPORAL_LAYOUT_WILDCARD		0x7
+#define UVC_VP8_TEMPORAL_LAYOUT_WILDCARD_LAYER(layer)	\
+	((UVC_VP8_TEMPORAL_LAYOUT_WILDCARD) << (3 + (layer) * 3))
+#define UVC_VP8_TEMPRL_LAYOUT_NUM(n)			((((n) - 1) & 0x3) << 1)
+
 /* Devices quirks */
 #define UVC_QUIRK_STATUS_INTERVAL	0x00000001
 #define UVC_QUIRK_PROBE_MINMAX		0x00000002
@@ -144,6 +157,7 @@
 /* Format flags */
 #define UVC_FMT_FLAG_COMPRESSED		0x00000001
 #define UVC_FMT_FLAG_STREAM		0x00000002
+#define UVC_FMT_FLAG_SIMULCAST		0x00000004
 
 /* v4l2_buffer codec flags */
 #define UVC_V4L2_BUFFER_CODEC_FLAGS	(V4L2_BUF_FLAG_KEYFRAME | \
@@ -251,7 +265,7 @@ struct uvc_entity {
 	char name[64];
 
 	/* Media controller-related fields. */
-	struct video_device *vdev;
+	struct video_device *vdev[UVC_SIMULCAST_MAX_LAYERS];
 	struct v4l2_subdev subdev;
 	unsigned int num_pads;
 	unsigned int num_links;
@@ -392,6 +406,8 @@ struct uvc_video_queue {
 
 	spinlock_t irqlock;			/* Protects irqqueue */
 	struct list_head irqqueue;
+
+	struct uvc_streaming *stream;
 };
 
 struct uvc_video_pipeline {
@@ -409,6 +425,10 @@ struct uvc_video_chain {
 	struct uvc_entity *selector;		/* Selector unit */
 	struct uvc_entity *encoding;		/* Encoding unit */
 
+	struct uvc_control *layer_ctrl;
+	struct uvc_control *res_ctrl;
+	struct uvc_control *startstop_ctrl;
+
 	struct uvc_video_pipeline *pipeline;    /* Pipeline this chain
 						   belongs to */
 
@@ -484,12 +504,30 @@ struct uvc_payload_header {
 	__u32 buf_flags; /* v4l2_buffer flags */
 };
 
+struct uvc_stream_layer {
+	int layer_id;
+	int width;
+	int height;
+
+	__u32 sequence;
+	__u8 last_fid;
+
+	struct uvc_video_queue queue;
+	struct uvc_streaming *stream;
+
+	atomic_t active;
+};
+
+#define UVC_SIMUL_CAP_STARTSTOP_LAYER			(1 << 0)
+#define UVC_SIMUL_CAP_STARTSTOP_LAYER_STREAMTIME	(1 << 1)
+#define UVC_SIMUL_CAP_RES				(1 << 2)
+#define UVC_SIMUL_CAP_RES_STREAMTIME			(1 << 3)
+
 struct uvc_streaming {
 	struct list_head list;
 	struct uvc_device *dev;
-	struct video_device *vdev;
+	struct video_device *vdev[UVC_SIMULCAST_MAX_LAYERS];
 	struct uvc_video_chain *chain;
-	atomic_t active;
 
 	struct usb_interface *intf;
 	int intfnum;
@@ -506,13 +544,17 @@ struct uvc_streaming {
 	struct uvc_format *cur_format;
 	struct uvc_frame *cur_frame;
 	/* Protect access to ctrl, cur_format, cur_frame and hardware video
-	 * probe control.
+	 * probe control as well as layer allocated/streaming state.
 	 */
 	struct mutex mutex;
 
 	/* Buffers queue. */
 	unsigned int frozen : 1;
-	struct uvc_video_queue queue;
+
+	/* Protected by mutex. */
+	unsigned long allocated_layers;
+	unsigned long streaming_layers;
+
 	void (*decode) (struct urb *urb, struct uvc_streaming *video);
 
 	/* Context data used by the bulk completion handler. */
@@ -529,9 +571,6 @@ struct uvc_streaming {
 	dma_addr_t urb_dma[UVC_URBS];
 	unsigned int urb_size;
 
-	__u32 sequence;
-	__u8 last_fid;
-
 	/* debugfs */
 	struct dentry *debugfs_dir;
 	struct {
@@ -557,6 +596,19 @@ struct uvc_streaming {
 
 		spinlock_t lock;
 	} clock;
+
+	/* Max number of layers this streaming interface
+	 * can potentially support.
+	 */
+	int num_layers;
+	/* Negotiated in P&C. */
+	int num_negotiated_layers;
+	struct uvc_stream_layer layers[UVC_SIMULCAST_MAX_LAYERS];
+	unsigned long simulcast_caps;
+
+	int layer_id;
+	int prev_layer_id;
+	int temporal_id;
 };
 
 enum uvc_device_state {
@@ -589,7 +641,7 @@ struct uvc_device {
 
 	/* Video Streaming interfaces */
 	struct list_head streams;
-	atomic_t nstreams;
+	atomic_t num_vdevs;
 
 	/* Status Interrupt Endpoint */
 	struct usb_host_endpoint *int_ep;
@@ -607,7 +659,7 @@ enum uvc_handle_state {
 struct uvc_fh {
 	struct v4l2_fh vfh;
 	struct uvc_video_chain *chain;
-	struct uvc_streaming *stream;
+	struct uvc_stream_layer *layer;
 	enum uvc_handle_state state;
 };
 
@@ -677,7 +729,8 @@ extern struct uvc_entity *uvc_entity_by_id(struct uvc_device *dev, int id);
 
 /* Video buffers queue management. */
 extern int uvc_queue_init(struct uvc_video_queue *queue,
-		enum v4l2_buf_type type, int drop_corrupted);
+		enum v4l2_buf_type type, int drop_corrupted,
+		struct uvc_streaming *stream);
 extern int uvc_alloc_buffers(struct uvc_video_queue *queue,
 		struct v4l2_requestbuffers *rb);
 extern void uvc_free_buffers(struct uvc_video_queue *queue);
@@ -717,17 +770,21 @@ extern void uvc_mc_cleanup_entity(struct uvc_entity *entity);
 extern int uvc_video_init(struct uvc_streaming *stream);
 extern int uvc_video_suspend(struct uvc_streaming *stream);
 extern int uvc_video_resume(struct uvc_streaming *stream, int reset);
-extern int uvc_video_enable(struct uvc_streaming *stream, int enable);
+int uvc_video_layer_enable(struct uvc_stream_layer *layer);
+void uvc_video_layer_disable(struct uvc_stream_layer *layer);
 extern int uvc_probe_video(struct uvc_streaming *stream,
 		struct uvc_streaming_control *probe);
+int uvc_commit_video(struct uvc_streaming *stream,
+			    struct uvc_streaming_control *probe);
 extern int uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
 		__u8 intfnum, __u8 cs, void *data, __u16 size);
 void uvc_video_clock_update(struct uvc_streaming *stream,
 			    struct v4l2_buffer *v4l2_buf,
 			    struct uvc_buffer *buf);
+int uvc_get_probe_ctrl_size(struct uvc_streaming *stream);
 static inline bool uvc_is_stream_streaming(struct uvc_streaming *stream)
 {
-	return vb2_is_streaming(&stream->queue.queue);
+	return stream->streaming_layers != 0;
 }
 
 /* Status */
@@ -773,6 +830,13 @@ extern int uvc_ctrl_set(struct uvc_video_chain *chain,
 extern int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
 		struct uvc_xu_control_query *xqry);
 
+int uvc_ctrl_layer_enable(struct uvc_stream_layer *layer, int enable);
+int uvc_video_layer_set_resolution(struct uvc_stream_layer *layer,
+					int width, int height);
+int uvc_video_layer_get_resolution(struct uvc_stream_layer *layer);
+void uvc_ctrl_simulcast_init(struct uvc_streaming *stream);
+int __uvc_video_layer_select(struct uvc_stream_layer *layer, int temporal_id);
+
 /* Utility functions */
 extern void uvc_simplify_fraction(uint32_t *numerator, uint32_t *denominator,
 		unsigned int n_terms, unsigned int threshold);
@@ -781,6 +845,35 @@ extern uint32_t uvc_fraction_to_interval(uint32_t numerator,
 extern struct usb_host_endpoint *uvc_find_endpoint(
 		struct usb_host_interface *alts, __u8 epaddr);
 
+void uvc_update_stream_format(struct uvc_streaming *stream,
+			struct uvc_format *format, struct uvc_frame *frame);
+bool uvc_is_layer_negotiated(struct uvc_streaming *stream, int layer_id);
+
+static inline bool uvc_is_simulcasting(struct uvc_streaming *stream)
+{
+	return stream->cur_format
+		&& (stream->cur_format->flags & UVC_FMT_FLAG_SIMULCAST);
+}
+
+static inline bool uvc_is_eu_active(struct uvc_streaming *stream)
+{
+	return stream->cur_format
+		&& (stream->cur_format->fcc == V4L2_PIX_FMT_VP8
+		    || stream->cur_format->fcc == V4L2_PIX_FMT_VP8_SIMULCAST);
+}
+
+static inline struct uvc_stream_layer *uvc_get_curr_layer(
+		struct uvc_streaming *stream)
+{
+	return &stream->layers[stream->layer_id];
+}
+
+static inline bool uvc_cur_fmt_has_tmprl_layers(struct uvc_streaming *stream)
+{
+	return stream->cur_format->fcc == V4L2_PIX_FMT_VP8
+		|| stream->cur_format->fcc == V4L2_PIX_FMT_VP8_SIMULCAST;
+}
+
 /* Quirks support */
 void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream);
 
diff --git a/include/uapi/linux/usb/video.h b/include/uapi/linux/usb/video.h
index fd1d4b0..4bb4057 100644
--- a/include/uapi/linux/usb/video.h
+++ b/include/uapi/linux/usb/video.h
@@ -56,6 +56,7 @@
 #define UVC_VS_FORMAT_STREAM_BASED			0x12
 #define UVC_VS_FORMAT_VP8				0x16
 #define UVC_VS_FRAME_VP8				0x17
+#define UVC_VS_FORMAT_VP8_SIMULCAST                     0x18
 
 /* A.7. Video Class-Specific Endpoint Descriptor Subtypes */
 #define UVC_EP_UNDEFINED				0x00
@@ -202,6 +203,11 @@
 #define UVC_STREAM_VP8_ARF				(1 << 1)
 #define UVC_STREAM_VP8_PRF				(1 << 0)
 
+#define UVC_STREAM_STREAM_ID_SHIFT			10
+#define UVC_STREAM_STREAM_ID_MASK			0x3
+#define UVC_STREAM_TMPRL_ID_SHIFT			7
+#define UVC_STREAM_TEMPRL_ID_MASK			0x3
+
 /* 4.1.2. Control Capabilities */
 #define UVC_CONTROL_CAP_GET				(1 << 0)
 #define UVC_CONTROL_CAP_SET				(1 << 1)
-- 
1.8.4

Re: [PATCH v1 10/19] uvcvideo: Support UVC 1.5 runtime control property.

[Hans Verkuil]

On 08/30/2013 04:17 AM, Pawel Osciak wrote:
> UVC 1.5 introduces the concept of runtime controls, which can be set during
> streaming. Non-runtime controls can only be changed while device is idle.

I don't know if you know this, but the V4L2_CTRL_FLAG_GRABBED flag can be set for
controls that cannot be changed while streaming. Typically drivers will set that
flag for the relevant controls when streaming starts and clear it when streaming
stops.

Toggling that flag also means that a control event has to be generated reporting
a change of the control flags.

Regards,

	Hans

> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_ctrl.c | 45 +++++++++++++++++++++++++++++++++-------
>  drivers/media/usb/uvc/uvc_v4l2.c | 18 ++++++++++------
>  drivers/media/usb/uvc/uvcvideo.h | 12 +++++++----
>  3 files changed, 57 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c b/drivers/media/usb/uvc/uvc_ctrl.c
> index d735c88..b0a19b9 100644
> --- a/drivers/media/usb/uvc/uvc_ctrl.c
> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
> @@ -1076,8 +1076,19 @@ void __uvc_ctrl_unlock(struct uvc_video_chain *chain)
>  	mutex_unlock(&chain->pipeline->ctrl_mutex);
>  }
>  
> +static int uvc_check_ctrl_runtime(struct uvc_control *ctrl, bool streaming)
> +{
> +	if (streaming && !ctrl->in_runtime) {
> +		uvc_trace(UVC_TRACE_CONTROL,
> +				"Control disabled while streaming\n");
> +		return -EBUSY;
> +	}
> +
> +	return 0;
> +}
> +
>  int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
> -	struct v4l2_queryctrl *v4l2_ctrl)
> +			struct v4l2_queryctrl *v4l2_ctrl, bool streaming)
>  {
>  	struct uvc_control *ctrl;
>  	struct uvc_control_mapping *mapping;
> @@ -1093,6 +1104,10 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
>  		goto done;
>  	}
>  
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		goto done;
> +
>  	ret = __uvc_query_v4l2_ctrl(chain, ctrl, mapping, v4l2_ctrl);
>  done:
>  	__uvc_ctrl_unlock(chain);
> @@ -1109,7 +1124,7 @@ done:
>   * manually.
>   */
>  int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
> -	struct v4l2_querymenu *query_menu)
> +	struct v4l2_querymenu *query_menu, bool streaming)
>  {
>  	struct uvc_menu_info *menu_info;
>  	struct uvc_control_mapping *mapping;
> @@ -1132,6 +1147,10 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
>  		goto done;
>  	}
>  
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		goto done;
> +
>  	if (query_menu->index >= mapping->menu_count) {
>  		ret = -EINVAL;
>  		goto done;
> @@ -1436,21 +1455,26 @@ done:
>  	return ret;
>  }
>  
> -int uvc_ctrl_get(struct uvc_video_chain *chain,
> -	struct v4l2_ext_control *xctrl)
> +int uvc_ctrl_get(struct uvc_video_chain *chain, struct v4l2_ext_control *xctrl,
> +		 bool streaming)
>  {
>  	struct uvc_control *ctrl;
>  	struct uvc_control_mapping *mapping;
> +	int ret;
>  
>  	ctrl = uvc_find_control(chain, xctrl->id, &mapping);
>  	if (ctrl == NULL)
>  		return -EINVAL;
>  
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		return ret;
> +
>  	return __uvc_ctrl_get(chain, ctrl, mapping, &xctrl->value);
>  }
>  
> -int uvc_ctrl_set(struct uvc_video_chain *chain,
> -	struct v4l2_ext_control *xctrl)
> +int uvc_ctrl_set(struct uvc_video_chain *chain, struct v4l2_ext_control *xctrl,
> +		 bool streaming)
>  {
>  	struct uvc_control *ctrl;
>  	struct uvc_control_mapping *mapping;
> @@ -1466,6 +1490,10 @@ int uvc_ctrl_set(struct uvc_video_chain *chain,
>  	if (!(ctrl->info.flags & UVC_CTRL_FLAG_SET_CUR))
>  		return -EACCES;
>  
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		return ret;
> +
>  	/* Clamp out of range values. */
>  	switch (mapping->v4l2_type) {
>  	case V4L2_CTRL_TYPE_INTEGER:
> @@ -1885,8 +1913,9 @@ static int uvc_ctrl_add_info(struct uvc_device *dev, struct uvc_control *ctrl,
>  	ctrl->initialized = 1;
>  
>  	uvc_trace(UVC_TRACE_CONTROL, "Added control %pUl/%u to device %s "
> -		"entity %u\n", ctrl->info.entity, ctrl->info.selector,
> -		dev->udev->devpath, ctrl->entity->id);
> +		"entity %u, init/runtime %d/%d\n", ctrl->info.entity,
> +		ctrl->info.selector, dev->udev->devpath, ctrl->entity->id,
> +		ctrl->on_init, ctrl->in_runtime);
>  
>  done:
>  	if (ret < 0)
> diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
> index a899159..decd65f 100644
> --- a/drivers/media/usb/uvc/uvc_v4l2.c
> +++ b/drivers/media/usb/uvc/uvc_v4l2.c
> @@ -597,7 +597,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  
>  	/* Get, Set & Query control */
>  	case VIDIOC_QUERYCTRL:
> -		return uvc_query_v4l2_ctrl(chain, arg);
> +		return uvc_query_v4l2_ctrl(chain, arg,
> +					uvc_is_stream_streaming(stream));
>  
>  	case VIDIOC_G_CTRL:
>  	{
> @@ -611,7 +612,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  		if (ret < 0)
>  			return ret;
>  
> -		ret = uvc_ctrl_get(chain, &xctrl);
> +		ret = uvc_ctrl_get(chain, &xctrl,
> +					uvc_is_stream_streaming(stream));
>  		uvc_ctrl_rollback(handle);
>  		if (ret >= 0)
>  			ctrl->value = xctrl.value;
> @@ -635,7 +637,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  		if (ret < 0)
>  			return ret;
>  
> -		ret = uvc_ctrl_set(chain, &xctrl);
> +		ret = uvc_ctrl_set(chain, &xctrl,
> +					uvc_is_stream_streaming(stream));
>  		if (ret < 0) {
>  			uvc_ctrl_rollback(handle);
>  			return ret;
> @@ -647,7 +650,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  	}
>  
>  	case VIDIOC_QUERYMENU:
> -		return uvc_query_v4l2_menu(chain, arg);
> +		return uvc_query_v4l2_menu(chain, arg,
> +					uvc_is_stream_streaming(stream));
>  
>  	case VIDIOC_G_EXT_CTRLS:
>  	{
> @@ -660,7 +664,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  			return ret;
>  
>  		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
> -			ret = uvc_ctrl_get(chain, ctrl);
> +			ret = uvc_ctrl_get(chain, ctrl,
> +					uvc_is_stream_streaming(stream));
>  			if (ret < 0) {
>  				uvc_ctrl_rollback(handle);
>  				ctrls->error_idx = i;
> @@ -688,7 +693,8 @@ static long uvc_v4l2_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  			return ret;
>  
>  		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
> -			ret = uvc_ctrl_set(chain, ctrl);
> +			ret = uvc_ctrl_set(chain, ctrl,
> +					uvc_is_stream_streaming(stream));
>  			if (ret < 0) {
>  				uvc_ctrl_rollback(handle);
>  				ctrls->error_idx = cmd == VIDIOC_S_EXT_CTRLS
> diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
> index 88f5e38..46ffd92 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -694,6 +694,10 @@ extern int uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
>  void uvc_video_clock_update(struct uvc_streaming *stream,
>  			    struct v4l2_buffer *v4l2_buf,
>  			    struct uvc_buffer *buf);
> +static inline bool uvc_is_stream_streaming(struct uvc_streaming *stream)
> +{
> +	return vb2_is_streaming(&stream->queue.queue);
> +}
>  
>  /* Status */
>  extern int uvc_status_init(struct uvc_device *dev);
> @@ -705,9 +709,9 @@ extern void uvc_status_stop(struct uvc_device *dev);
>  extern const struct v4l2_subscribed_event_ops uvc_ctrl_sub_ev_ops;
>  
>  extern int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
> -		struct v4l2_queryctrl *v4l2_ctrl);
> +		struct v4l2_queryctrl *v4l2_ctrl, bool streaming);
>  extern int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
> -		struct v4l2_querymenu *query_menu);
> +		struct v4l2_querymenu *query_menu, bool streaming);
>  
>  extern int uvc_ctrl_add_mapping(struct uvc_video_chain *chain,
>  		const struct uvc_control_mapping *mapping);
> @@ -731,9 +735,9 @@ static inline int uvc_ctrl_rollback(struct uvc_fh *handle)
>  }
>  
>  extern int uvc_ctrl_get(struct uvc_video_chain *chain,
> -		struct v4l2_ext_control *xctrl);
> +		struct v4l2_ext_control *xctrl, bool streaming);
>  extern int uvc_ctrl_set(struct uvc_video_chain *chain,
> -		struct v4l2_ext_control *xctrl);
> +		struct v4l2_ext_control *xctrl, bool streaming);
>  
>  extern int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>  		struct uvc_xu_control_query *xqry);
> 

Re: [PATCH v1 14/19] v4l: Add v4l2_buffer flags for VP8-specific special frames.

[Hans Verkuil]

On 08/30/2013 04:17 AM, Pawel Osciak wrote:
> Add bits for previous, golden and altref frame types.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>

Kamil, is this something that applies as well to your MFC driver?

> ---
>  include/uapi/linux/videodev2.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 437f1b0..c011ee0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -687,6 +687,10 @@ struct v4l2_buffer {
>  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x0000
>  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x2000
>  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x4000
> +/* VP8 special frames */
> +#define V4L2_BUF_FLAG_PREV_FRAME		0x10000  /* VP8 prev frame */
> +#define V4L2_BUF_FLAG_GOLDEN_FRAME		0x20000  /* VP8 golden frame */
> +#define V4L2_BUF_FLAG_ALTREF_FRAME		0x40000  /* VP8 altref frame */

Would it be an idea to use the same bit values as for KEYFRAME/PFRAME/BFRAME?
After all, these can never be used at the same time. I'm a bit worried that the
bits in this field are eventually all used up by different encoder flags.

Regards,

	Hans

>  
>  /**
>   * struct v4l2_exportbuffer - export of video buffer as DMABUF file descriptor
> 

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Hans Verkuil]

On 08/30/2013 04:17 AM, Pawel Osciak wrote:
> Add defines for controls found in UVC 1.5 encoding cameras.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 29 +++++++++++++++++++++++++++++
>  include/uapi/linux/v4l2-controls.h   | 31 +++++++++++++++++++++++++++++++
>  2 files changed, 60 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index c3f0803..0b3a632 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -781,6 +781,35 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_AUTO_FOCUS_STATUS:	return "Auto Focus, Status";
>  	case V4L2_CID_AUTO_FOCUS_RANGE:		return "Auto Focus, Range";
>  
> +	case V4L2_CID_ENCODER_MIN_FRAME_INTERVAL: return "Encoder, min. frame interval";
> +	case V4L2_CID_ENCODER_RATE_CONTROL_MODE: return "Encoder, rate control mode";
> +	case V4L2_CID_ENCODER_AVERAGE_BITRATE:	return "Encoder, average bitrate";
> +	case V4L2_CID_ENCODER_CPB_SIZE:		return "Encoder, CPB size";
> +	case V4L2_CID_ENCODER_PEAK_BIT_RATE:	return "Encoder, peak bit rate";
> +	case V4L2_CID_ENCODER_QP_PARAM_I:	return "Encoder, QP param for I frames";
> +	case V4L2_CID_ENCODER_QP_PARAM_P:	return "Encoder, QP param for P frames";
> +	case V4L2_CID_ENCODER_QP_PARAM_BG:	return "Encoder, QP param for B/G frames";

A lot of these exist already. E.g. V4L2_CID_MPEG_VIDEO_MPEG4_I/P/B_FRAME_QP.

Samsung added support for many of these parameters for their MFC encoder (including
VP8 support) so you should use them as well. As mentioned in v4l2-controls.h the
MPEG part of the control name is historical. Interpret it as 'CODEC', not MPEG.

> +	case V4L2_CID_ENCODER_NUM_GDR_FRAMES:	return "Encoder, number of GDR frames";
> +	case V4L2_CID_ENCODER_LTR_BUFFER_CONTROL: return "Encoder, LTR buffer control";
> +	case V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE: return "Encoder, LTR buffer trust mode";
> +	case V4L2_CID_ENCODER_LTR_PICTURE_POSITION: return "Encoder, LTR picture position";
> +	case V4L2_CID_ENCODER_LTR_PICTURE_MODE:	return "Encoder, LTR picture mode";
> +	case V4L2_CID_ENCODER_LTR_VALIDATION:	return "Encoder, LTR validation";
> +	case V4L2_CID_ENCODER_MIN_QP:		return "Encoder, minimum QP param";
> +	case V4L2_CID_ENCODER_MAX_QP:		return "Encoder, maximum QP param";
> +	case V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL: return "Encoder, sync frame interval";
> +	case V4L2_CID_ENCODER_ERROR_RESILIENCY:	return "Encoder, error resiliency";
> +	case V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE: return "Encoder, temporal layer enable";
> +
> +	case V4L2_CID_ENCODER_VP8_SLICE_MODE:	return "Encoder, VP8 slice mode";
> +	case V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE: return "Encoder, VP8 sync frame type";
> +	case V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME: return "Encoder, VP8, DCT partitions per frame";
> +
> +	case V4L2_CID_ENCODER_H264_PROFILE_TOOLSET: return "Encoder, H.264 profile and toolset";
> +	case V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT: return "Encoder, H.264 level IDC limit";
> +	case V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE: return "Encoder, H.264 SEI payload type";
> +	case V4L2_CID_ENCODER_H264_LAYER_PRIORITY: return "Encoder, H.264 layer priority";
> +
>  	/* FM Radio Modulator control */
>  	/* Keep the order of the 'case's the same as in videodev2.h! */
>  	case V4L2_CID_FM_TX_CLASS:		return "FM Radio Modulator Controls";
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 083bb5a..ef3a30d 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -729,6 +729,37 @@ enum v4l2_auto_focus_range {
>  	V4L2_AUTO_FOCUS_RANGE_INFINITY		= 3,
>  };
>  
> +/* Controls found in UVC 1.5 encoding cameras */
> +#define V4L2_CID_ENCODER_MIN_FRAME_INTERVAL	(V4L2_CID_CAMERA_CLASS_BASE+32)
> +#define V4L2_CID_ENCODER_RATE_CONTROL_MODE	(V4L2_CID_CAMERA_CLASS_BASE+33)
> +#define V4L2_CID_ENCODER_AVERAGE_BITRATE	(V4L2_CID_CAMERA_CLASS_BASE+34)
> +#define V4L2_CID_ENCODER_CPB_SIZE		(V4L2_CID_CAMERA_CLASS_BASE+35)
> +#define V4L2_CID_ENCODER_PEAK_BIT_RATE		(V4L2_CID_CAMERA_CLASS_BASE+36)
> +#define V4L2_CID_ENCODER_QP_PARAM_I		(V4L2_CID_CAMERA_CLASS_BASE+37)
> +#define V4L2_CID_ENCODER_QP_PARAM_P		(V4L2_CID_CAMERA_CLASS_BASE+38)
> +#define V4L2_CID_ENCODER_QP_PARAM_BG		(V4L2_CID_CAMERA_CLASS_BASE+39)
> +#define V4L2_CID_ENCODER_NUM_GDR_FRAMES		(V4L2_CID_CAMERA_CLASS_BASE+40)
> +#define V4L2_CID_ENCODER_LTR_BUFFER_CONTROL	(V4L2_CID_CAMERA_CLASS_BASE+41)
> +#define V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE	(V4L2_CID_CAMERA_CLASS_BASE+42)
> +#define V4L2_CID_ENCODER_LTR_PICTURE_POSITION	(V4L2_CID_CAMERA_CLASS_BASE+43)
> +#define V4L2_CID_ENCODER_LTR_PICTURE_MODE	(V4L2_CID_CAMERA_CLASS_BASE+44)
> +#define V4L2_CID_ENCODER_LTR_VALIDATION		(V4L2_CID_CAMERA_CLASS_BASE+45)
> +#define V4L2_CID_ENCODER_MIN_QP			(V4L2_CID_CAMERA_CLASS_BASE+46)
> +#define V4L2_CID_ENCODER_MAX_QP			(V4L2_CID_CAMERA_CLASS_BASE+47)
> +#define V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL	(V4L2_CID_CAMERA_CLASS_BASE+48)
> +#define V4L2_CID_ENCODER_ERROR_RESILIENCY	(V4L2_CID_CAMERA_CLASS_BASE+49)
> +#define V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE	(V4L2_CID_CAMERA_CLASS_BASE+50)
> +
> +/* VP8-specific controls */
> +#define V4L2_CID_ENCODER_VP8_SLICE_MODE		(V4L2_CID_CAMERA_CLASS_BASE+51)
> +#define V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME (V4L2_CID_CAMERA_CLASS_BASE+52)
> +#define V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE	(V4L2_CID_CAMERA_CLASS_BASE+53)
> +
> +/* H.264-specific controls */
> +#define V4L2_CID_ENCODER_H264_PROFILE_TOOLSET	(V4L2_CID_CAMERA_CLASS_BASE+54)
> +#define V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT	(V4L2_CID_CAMERA_CLASS_BASE+55)
> +#define V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE	(V4L2_CID_CAMERA_CLASS_BASE+56)
> +#define V4L2_CID_ENCODER_H264_LAYER_PRIORITY	(V4L2_CID_CAMERA_CLASS_BASE+57)

Codec controls go to the MPEG class, not the CAMERA class.

Regards,

	Hans

>  
>  /* FM Modulator class control IDs */
>  
> 

Re: [PATCH v1 14/19] v4l: Add v4l2_buffer flags for VP8-specific special frames.

[Hans Verkuil]

On Fri 30 August 2013 09:28:36 Pawel Osciak wrote:
> On Fri, Aug 30, 2013 at 3:42 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
> > On 08/30/2013 04:17 AM, Pawel Osciak wrote:
> > > Add bits for previous, golden and altref frame types.
> > >
> > > Signed-off-by: Pawel Osciak <posciak@chromium.org>
> >
> > Kamil, is this something that applies as well to your MFC driver?
> >
> > > ---
> > >  include/uapi/linux/videodev2.h | 4 ++++
> > >  1 file changed, 4 insertions(+)
> > >
> > > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > > index 437f1b0..c011ee0 100644
> > > --- a/include/uapi/linux/videodev2.h
> > > +++ b/include/uapi/linux/videodev2.h
> > > @@ -687,6 +687,10 @@ struct v4l2_buffer {
> > >  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN              0x0000
> > >  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC    0x2000
> > >  #define V4L2_BUF_FLAG_TIMESTAMP_COPY         0x4000
> > > +/* VP8 special frames */
> > > +#define V4L2_BUF_FLAG_PREV_FRAME             0x10000  /* VP8 prev frame
> > */
> > > +#define V4L2_BUF_FLAG_GOLDEN_FRAME           0x20000  /* VP8 golden
> > frame */
> > > +#define V4L2_BUF_FLAG_ALTREF_FRAME           0x40000  /* VP8 altref
> > frame */
> >
> > Would it be an idea to use the same bit values as for
> > KEYFRAME/PFRAME/BFRAME?
> > After all, these can never be used at the same time. I'm a bit worried
> > that the
> > bits in this field are eventually all used up by different encoder flags.
> >
> 
> VP8 also has a concept of I and P frames and they are orthogonal to the
> concept of Prev, Golden and Alt. There is no relationship at all, i.e. we
> can't infer I P from Prev Golden Alt and vice versa.

Ah, OK. Me no VP8 expert :-)

> For the I, P
> dimension, we need one bit, but same could be said about H264, we need 2
> bits, not 3 there (if it's not I or P, it's B), and we have 3 bits
> nevertheless.
> For Prev Golden Alt dimension, we do need 3 bits.
> Now, technically, in VP8, the first bit of the frame header indicates if
> the frame is I or P, so we could technically use that in userspace and
> overload I P B to mean Prev Golden Alt for VP8. But while I understand the
> problem with running out of bits very well, as I and P exist in VP8 as
> well, it would be pretty confusing, so it's a trade-off that should be
> carefully weighted.

Are prev/golden/altref frames mutually exclusive? If so, then perhaps we
should use a two-bit mask instead of three bits. And those two-bits can
later be expanded to more to support codecs that have more than four
different frame types.

Regards,

	Hans

> 
> Regards,
> Pawel
> 
> 
> > Regards,
> >
> >         Hans
> >
> > >
> > >  /**
> > >   * struct v4l2_exportbuffer - export of video buffer as DMABUF file
> > descriptor
> > >
> >
> >
> 

Re: [PATCH v1 14/19] v4l: Add v4l2_buffer flags for VP8-specific special frames.

[Sakari Ailus]

Hi Hans and Pawel,

On Fri, Aug 30, 2013 at 10:12:45AM +0200, Hans Verkuil wrote:
> Are prev/golden/altref frames mutually exclusive? If so, then perhaps we

Does that apply to other types of frames as well (key, p and b)? If yes, the
existing frame bits could be used for VP8 frame flags while the existing
codecs could keep using the old definitions, i.e. same bits, but different
macros.

Just my five euro cents.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v1 14/19] v4l: Add v4l2_buffer flags for VP8-specific special frames.

[Sakari Ailus]

Hi Pawel,

On Fri, Aug 30, 2013 at 11:17:13AM +0900, Pawel Osciak wrote:
> Add bits for previous, golden and altref frame types.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  include/uapi/linux/videodev2.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 437f1b0..c011ee0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -687,6 +687,10 @@ struct v4l2_buffer {
>  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x0000
>  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x2000
>  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x4000
> +/* VP8 special frames */
> +#define V4L2_BUF_FLAG_PREV_FRAME		0x10000  /* VP8 prev frame */
> +#define V4L2_BUF_FLAG_GOLDEN_FRAME		0x20000  /* VP8 golden frame */
> +#define V4L2_BUF_FLAG_ALTREF_FRAME		0x40000  /* VP8 altref frame */

Whichever way this patch is changed, could you rebased it on "v4l: Use full
32 bits for buffer flags"? It changes the definitions of the flags use
32-bit values.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v1 18/19] v4l: Add V4L2_PIX_FMT_VP8_SIMULCAST format.

[Sakari Ailus]

Hi Pawel,

Thanks for the patchset.

On Fri, Aug 30, 2013 at 11:17:17AM +0900, Pawel Osciak wrote:
> This format is used by UVC 1.5 VP8-encoding cameras. When it is used, the camera
> may encode captured frames into one or more streams, each of which may
> be configured differently. This allows simultaneous capture of streams
> with different resolutions, bitrates, and other settings, depending on the
> camera capabilities.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  include/uapi/linux/videodev2.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index c011ee0..8b0d4ad 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -402,6 +402,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_G v4l2_fourcc('V', 'C', '1', 'G') /* SMPTE 421M Annex G compliant stream */
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
> +#define V4L2_PIX_FMT_VP8_SIMULCAST v4l2_fourcc('V', 'P', '8', 'S') /* VP8 simulcast */
>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -691,6 +692,9 @@ struct v4l2_buffer {
>  #define V4L2_BUF_FLAG_PREV_FRAME		0x10000  /* VP8 prev frame */
>  #define V4L2_BUF_FLAG_GOLDEN_FRAME		0x20000  /* VP8 golden frame */
>  #define V4L2_BUF_FLAG_ALTREF_FRAME		0x40000  /* VP8 altref frame */
> +/* Simulcast layer structure. */
> +#define V4L2_BUF_FLAG_LAYER_STRUCTURE_SHIFT	19  /* Bits 19-20 for layer */
> +#define V4L2_BUF_FLAG_LAYER_STRUCTURE_MASK	0x3 /* structure information. */

What do these bits signify? It'd be also nice to document them.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v1 01/19] uvcvideo: Add UVC query tracing.

[Laurent Pinchart]

Hi Pawel,

Thank you so much for the patch (set) ! I'll probably need a couple of days to 
review it all, let's start.

On Friday 30 August 2013 11:17:00 Pawel Osciak wrote:
> Add a new trace argument enabling UVC query details and contents logging.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_video.c | 45 ++++++++++++++++++++++--------------
>  drivers/media/usb/uvc/uvcvideo.h  |  9 ++++++++
>  2 files changed, 38 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 3394c34..695f6d9 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -29,22 +29,6 @@
>  /* ------------------------------------------------------------------------
> * UVC Controls
>   */
> -
> -static int __uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
> -			__u8 intfnum, __u8 cs, void *data, __u16 size,
> -			int timeout)
> -{
> -	__u8 type = USB_TYPE_CLASS | USB_RECIP_INTERFACE;
> -	unsigned int pipe;
> -
> -	pipe = (query & 0x80) ? usb_rcvctrlpipe(dev->udev, 0)
> -			      : usb_sndctrlpipe(dev->udev, 0);
> -	type |= (query & 0x80) ? USB_DIR_IN : USB_DIR_OUT;
> -
> -	return usb_control_msg(dev->udev, pipe, query, type, cs << 8,
> -			unit << 8 | intfnum, data, size, timeout);
> -}
> -
>  static const char *uvc_query_name(__u8 query)
>  {
>  	switch (query) {
> @@ -69,6 +53,35 @@ static const char *uvc_query_name(__u8 query)
>  	}
>  }
> 
> +static int __uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
> +			__u8 intfnum, __u8 cs, void *data, __u16 size,
> +			int timeout)
> +{
> +	__u8 type = USB_TYPE_CLASS | USB_RECIP_INTERFACE;
> +	unsigned int pipe;
> +	int ret;
> +
> +	pipe = (query & 0x80) ? usb_rcvctrlpipe(dev->udev, 0)
> +			      : usb_sndctrlpipe(dev->udev, 0);
> +	type |= (query & 0x80) ? USB_DIR_IN : USB_DIR_OUT;
> +
> +	uvc_trace(UVC_TRACE_QUERY,
> +			"%s (%d): size=%d, unit=%d, cs=%d, intf=%d\n",

All the fields are unsigned, shouldn't you use %u instead of %d ?

> +			uvc_query_name(query), query, size, unit, cs, intfnum);

That's a pretty strange indentation :-)

> +	uvc_trace(UVC_TRACE_QUERY, "Sent:\n");
> +	uvc_print_hex_dump(UVC_TRACE_QUERY, data, size);
> +
> +	ret = usb_control_msg(dev->udev, pipe, query, type, cs << 8,
> +			unit << 8 | intfnum, data, size, timeout);
> +	if (ret == -EPIPE)
> +		uvc_trace(UVC_TRACE_QUERY, "Got device STALL on query!\n");
> +
> +	uvc_trace(UVC_TRACE_QUERY, "Received:\n");
> +	uvc_print_hex_dump(UVC_TRACE_QUERY, data, size);
> +
> +	return ret;
> +}
> +
>  int uvc_query_ctrl(struct uvc_device *dev, __u8 query, __u8 unit,
>  			__u8 intfnum, __u8 cs, void *data, __u16 size)
>  {
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index 9e35982..75e0153 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -574,6 +574,7 @@ struct uvc_driver {
>  #define UVC_TRACE_VIDEO		(1 << 10)
>  #define UVC_TRACE_STATS		(1 << 11)
>  #define UVC_TRACE_CLOCK		(1 << 12)
> +#define UVC_TRACE_QUERY		(1 << 13)
> 
>  #define UVC_WARN_MINMAX		0
>  #define UVC_WARN_PROBE_DEF	1
> @@ -599,6 +600,14 @@ extern unsigned int uvc_timeout_param;
>  #define uvc_printk(level, msg...) \
>  	printk(level "uvcvideo: " msg)
> 
> +#define uvc_print_hex_dump(flag, buf, len) \
> +	do { \
> +		if (uvc_trace_param & flag) { \
> +			print_hex_dump(KERN_DEBUG, "uvcvideo: ", \
> +				DUMP_PREFIX_NONE, 16, 1, buf, len, false); \
> +		} \

No need for braces around the print_hex_dump() call.

> +	} while (0)
> +
>  /*
> --------------------------------------------------------------------------
> * Internal functions.
>   */

-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 02/19] uvcvideo: Return 0 when setting probe control succeeds.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:01 Pawel Osciak wrote:
> Return 0 instead of returning size of the probe control on successful set.

This looks good, but could you update the commit message to explain why ?

> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_video.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 695f6d9..1198989 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -296,6 +296,8 @@ static int uvc_set_video_ctrl(struct uvc_streaming
> *stream, "%d (exp. %u).\n", probe ? "probe" : "commit",
>  			ret, size);
>  		ret = -EIO;
> +	} else {
> +		ret = 0;
>  	}
> 
>  	kfree(data);
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 04/19] uvcvideo: Create separate debugfs entries for each streaming interface.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:03 Pawel Osciak wrote:
> Add interface number to debugfs entry name to be able to create separate
> entries for each streaming interface for devices exposing more than one,
> instead of failing to create more than one.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_debugfs.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_debugfs.c
> b/drivers/media/usb/uvc/uvc_debugfs.c index 14561a5..0663fbd 100644
> --- a/drivers/media/usb/uvc/uvc_debugfs.c
> +++ b/drivers/media/usb/uvc/uvc_debugfs.c
> @@ -84,7 +84,8 @@ int uvc_debugfs_init_stream(struct uvc_streaming *stream)
>  	if (uvc_debugfs_root_dir == NULL)
>  		return -ENODEV;
> 
> -	sprintf(dir_name, "%u-%u", udev->bus->busnum, udev->devnum);
> +	sprintf(dir_name, "%u-%u-%u", udev->bus->busnum, udev->devnum,

What about %u-%u.%u ? The USB subsystem names devices using devnum.intfnum 
(which can be seen in /sys/bus/usb/devices/ for instance), so I believe it 
would be a good idea to follow the same naming conventions.

> +			stream->intfnum);
> 
>  	dent = debugfs_create_dir(dir_name, uvc_debugfs_root_dir);
>  	if (IS_ERR_OR_NULL(dent)) {
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 05/19] uvcvideo: Add support for UVC1.5 P&C control.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:04 Pawel Osciak wrote:
> Add support for UVC 1.5 Probe & Commit control.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_video.c | 52 +++++++++++++++++++++++++++++++++---
>  include/uapi/linux/usb/video.h    |  7 ++++++
>  2 files changed, 55 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 1198989..b4ebccd 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -168,14 +168,25 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming
> *stream, }
>  }
> 
> +int uvc_get_probe_ctrl_size(struct uvc_streaming *stream)
> +{
> +	if (stream->dev->uvc_version < 0x0110)
> +		return 26;
> +	else if (stream->dev->uvc_version < 0x0150)
> +		return 34;
> +	else
> +		return 48;
> +}
> +
>  static int uvc_get_video_ctrl(struct uvc_streaming *stream,
>  	struct uvc_streaming_control *ctrl, int probe, __u8 query)
>  {
>  	__u8 *data;
>  	__u16 size;
>  	int ret;
> +	int i;

Could you please make i unsigned, as it's used as an unsigned loop counter ?

> -	size = stream->dev->uvc_version >= 0x0110 ? 34 : 26;
> +	size = uvc_get_probe_ctrl_size(stream);
>  	if ((stream->dev->quirks & UVC_QUIRK_PROBE_DEF) &&
>  			query == UVC_GET_DEF)
>  		return -EIO;
> @@ -230,7 +241,7 @@ static int uvc_get_video_ctrl(struct uvc_streaming
> *stream, ctrl->dwMaxVideoFrameSize = get_unaligned_le32(&data[18]);
>  	ctrl->dwMaxPayloadTransferSize = get_unaligned_le32(&data[22]);
> 
> -	if (size == 34) {
> +	if (size >= 34) {
>  		ctrl->dwClockFrequency = get_unaligned_le32(&data[26]);
>  		ctrl->bmFramingInfo = data[30];
>  		ctrl->bPreferedVersion = data[31];
> @@ -244,6 +255,26 @@ static int uvc_get_video_ctrl(struct uvc_streaming
> *stream, ctrl->bMaxVersion = 0;
>  	}
> 
> +	if (size >= 48) {
> +		ctrl->bUsage = data[34];
> +		ctrl->bBitDepthLuma = data[35];
> +		ctrl->bmSetting = data[36];
> +		ctrl->bMaxNumberOfRefFramesPlus1 = data[37];
> +		ctrl->bmRateControlModes = get_unaligned_le16(&data[38]);
> +		for (i = 0; i < ARRAY_SIZE(ctrl->bmLayoutPerStream); ++i) {
> +			ctrl->bmLayoutPerStream[i] =
> +				get_unaligned_le16(&data[40 + i * 2]);
> +		}
> +	} else {
> +		ctrl->bUsage = 0;
> +		ctrl->bBitDepthLuma = 0;
> +		ctrl->bmSetting = 0;
> +		ctrl->bMaxNumberOfRefFramesPlus1 = 0;
> +		ctrl->bmRateControlModes = 0;
> +		for (i = 0; i < ARRAY_SIZE(ctrl->bmLayoutPerStream); ++i)
> +			ctrl->bmLayoutPerStream[i] = 0;
> +	}

I wonder whether we shouldn't just memset the whole structure to zero to start 
with. What do you think ?

> +
>  	/* Some broken devices return null or wrong dwMaxVideoFrameSize and
>  	 * dwMaxPayloadTransferSize fields. Try to get the value from the
>  	 * format and frame descriptors.
> @@ -262,8 +293,9 @@ static int uvc_set_video_ctrl(struct uvc_streaming
> *stream, __u8 *data;
>  	__u16 size;
>  	int ret;
> +	int i;

Unsigned here as well please.

> 
> -	size = stream->dev->uvc_version >= 0x0110 ? 34 : 26;
> +	size = uvc_get_probe_ctrl_size(stream);
>  	data = kzalloc(size, GFP_KERNEL);
>  	if (data == NULL)
>  		return -ENOMEM;
> @@ -280,7 +312,7 @@ static int uvc_set_video_ctrl(struct uvc_streaming
> *stream, put_unaligned_le32(ctrl->dwMaxVideoFrameSize, &data[18]);
>  	put_unaligned_le32(ctrl->dwMaxPayloadTransferSize, &data[22]);
> 
> -	if (size == 34) {
> +	if (size >= 34) {
>  		put_unaligned_le32(ctrl->dwClockFrequency, &data[26]);
>  		data[30] = ctrl->bmFramingInfo;
>  		data[31] = ctrl->bPreferedVersion;
> @@ -288,6 +320,18 @@ static int uvc_set_video_ctrl(struct uvc_streaming
> *stream, data[33] = ctrl->bMaxVersion;
>  	}
> 
> +	if (size >= 48) {
> +		data[34] = ctrl->bUsage;
> +		data[35] = ctrl->bBitDepthLuma;
> +		data[36] = ctrl->bmSetting;
> +		data[37] = ctrl->bMaxNumberOfRefFramesPlus1;
> +		*(__le16 *)&data[38] = cpu_to_le16(ctrl->bmRateControlModes);
> +		for (i = 0; i < ARRAY_SIZE(ctrl->bmLayoutPerStream); ++i) {
> +			*(__le16 *)&data[40 + i * 2] =
> +				cpu_to_le16(ctrl->bmLayoutPerStream[i]);
> +		}
> +	}
> +
>  	ret = __uvc_query_ctrl(stream->dev, UVC_SET_CUR, 0, stream->intfnum,
>  		probe ? UVC_VS_PROBE_CONTROL : UVC_VS_COMMIT_CONTROL, data,
>  		size, uvc_timeout_param);
> diff --git a/include/uapi/linux/usb/video.h b/include/uapi/linux/usb/video.h
> index 3b3b95e..331c071 100644
> --- a/include/uapi/linux/usb/video.h
> +++ b/include/uapi/linux/usb/video.h
> @@ -432,6 +432,7 @@ struct uvc_color_matching_descriptor {
>  #define UVC_DT_COLOR_MATCHING_SIZE			6
> 
>  /* 4.3.1.1. Video Probe and Commit Controls */
> +#define UVC_NUM_SIMULCAST_STREAMS			4
>  struct uvc_streaming_control {
>  	__u16 bmHint;
>  	__u8  bFormatIndex;
> @@ -449,6 +450,12 @@ struct uvc_streaming_control {
>  	__u8  bPreferedVersion;
>  	__u8  bMinVersion;
>  	__u8  bMaxVersion;
> +	__u8  bUsage;
> +	__u8  bBitDepthLuma;
> +	__u8  bmSetting;
> +	__u8  bMaxNumberOfRefFramesPlus1;
> +	__u16 bmRateControlModes;
> +	__u16 bmLayoutPerStream[UVC_NUM_SIMULCAST_STREAMS];
>  } __attribute__((__packed__));
> 
>  /* Uncompressed Payload - 3.1.1. Uncompressed Video Format Descriptor */
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 07/19] uvcvideo: Unify error reporting during format descriptor parsing.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:06 Pawel Osciak wrote:
> Add common error handling paths for format parsing failures.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_driver.c | 35 ++++++++++++++---------------------
> 1 file changed, 14 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index d950b40..936ddc7 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -322,13 +322,8 @@ static int uvc_parse_format(struct uvc_device *dev,
>  	case UVC_VS_FORMAT_UNCOMPRESSED:
>  	case UVC_VS_FORMAT_FRAME_BASED:
>  		n = buffer[2] == UVC_VS_FORMAT_UNCOMPRESSED ? 27 : 28;
> -		if (buflen < n) {
> -			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
> -			       "interface %d FORMAT error\n",
> -			       dev->udev->devnum,
> -			       alts->desc.bInterfaceNumber);
> -			return -EINVAL;
> -		}
> +		if (buflen < n)
> +			goto format_error;
> 
>  		/* Find the format descriptor from its GUID. */
>  		fmtdesc = uvc_format_by_guid(&buffer[5]);
> @@ -356,13 +351,8 @@ static int uvc_parse_format(struct uvc_device *dev,
>  		break;
> 
>  	case UVC_VS_FORMAT_MJPEG:
> -		if (buflen < 11) {
> -			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
> -			       "interface %d FORMAT error\n",
> -			       dev->udev->devnum,
> -			       alts->desc.bInterfaceNumber);
> -			return -EINVAL;
> -		}
> +		if (buflen < 11)
> +			goto format_error;
> 
>  		strlcpy(format->name, "MJPEG", sizeof format->name);
>  		format->fcc = V4L2_PIX_FMT_MJPEG;
> @@ -372,13 +362,8 @@ static int uvc_parse_format(struct uvc_device *dev,
>  		break;
> 
>  	case UVC_VS_FORMAT_DV:
> -		if (buflen < 9) {
> -			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
> -			       "interface %d FORMAT error\n",
> -			       dev->udev->devnum,
> -			       alts->desc.bInterfaceNumber);
> -			return -EINVAL;
> -		}
> +		if (buflen < 9)
> +			goto format_error;
> 
>  		switch (buffer[8] & 0x7f) {
>  		case 0:
> @@ -542,6 +527,14 @@ static int uvc_parse_format(struct uvc_device *dev,
>  	}
> 
>  	return buffer - start;
> +
> +format_error:
> +	uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
> +			"interface %d FORMAT error\n",
> +			dev->udev->devnum,
> +			alts->desc.bInterfaceNumber);

Could you please align the lines on UVC_TRACE_DESCR ?

> +	return -EINVAL;
> +

And remove the extra blank line ?

>  }
> 
>  static int uvc_parse_streaming(struct uvc_device *dev,
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Pawel Osciak]

Hi Hans,
Thanks for the comments, one question inline.

On Fri, Aug 30, 2013 at 3:48 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> On 08/30/2013 04:17 AM, Pawel Osciak wrote:
>> Add defines for controls found in UVC 1.5 encoding cameras.
>>
>> Signed-off-by: Pawel Osciak <posciak@chromium.org>
>> ---
>>  drivers/media/v4l2-core/v4l2-ctrls.c | 29 +++++++++++++++++++++++++++++
>>  include/uapi/linux/v4l2-controls.h   | 31 +++++++++++++++++++++++++++++++
>>  2 files changed, 60 insertions(+)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
>> index c3f0803..0b3a632 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
>> @@ -781,6 +781,35 @@ const char *v4l2_ctrl_get_name(u32 id)
>>       case V4L2_CID_AUTO_FOCUS_STATUS:        return "Auto Focus, Status";
>>       case V4L2_CID_AUTO_FOCUS_RANGE:         return "Auto Focus, Range";
>>
>> +     case V4L2_CID_ENCODER_MIN_FRAME_INTERVAL: return "Encoder, min. frame interval";
>> +     case V4L2_CID_ENCODER_RATE_CONTROL_MODE: return "Encoder, rate control mode";
>> +     case V4L2_CID_ENCODER_AVERAGE_BITRATE:  return "Encoder, average bitrate";
>> +     case V4L2_CID_ENCODER_CPB_SIZE:         return "Encoder, CPB size";
>> +     case V4L2_CID_ENCODER_PEAK_BIT_RATE:    return "Encoder, peak bit rate";
>> +     case V4L2_CID_ENCODER_QP_PARAM_I:       return "Encoder, QP param for I frames";
>> +     case V4L2_CID_ENCODER_QP_PARAM_P:       return "Encoder, QP param for P frames";
>> +     case V4L2_CID_ENCODER_QP_PARAM_BG:      return "Encoder, QP param for B/G frames";
>
> A lot of these exist already. E.g. V4L2_CID_MPEG_VIDEO_MPEG4_I/P/B_FRAME_QP.
>
> Samsung added support for many of these parameters for their MFC encoder (including
> VP8 support) so you should use them as well. As mentioned in v4l2-controls.h the
> MPEG part of the control name is historical. Interpret it as 'CODEC', not MPEG.
>

We have QP controls separately for H264, H263 and MPEG4. Why is that?
Which one should I use for VP8? Shouldn't we unify them instead?


>> +     case V4L2_CID_ENCODER_NUM_GDR_FRAMES:   return "Encoder, number of GDR frames";
>> +     case V4L2_CID_ENCODER_LTR_BUFFER_CONTROL: return "Encoder, LTR buffer control";
>> +     case V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE: return "Encoder, LTR buffer trust mode";
>> +     case V4L2_CID_ENCODER_LTR_PICTURE_POSITION: return "Encoder, LTR picture position";
>> +     case V4L2_CID_ENCODER_LTR_PICTURE_MODE: return "Encoder, LTR picture mode";
>> +     case V4L2_CID_ENCODER_LTR_VALIDATION:   return "Encoder, LTR validation";
>> +     case V4L2_CID_ENCODER_MIN_QP:           return "Encoder, minimum QP param";
>> +     case V4L2_CID_ENCODER_MAX_QP:           return "Encoder, maximum QP param";
>> +     case V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL: return "Encoder, sync frame interval";
>> +     case V4L2_CID_ENCODER_ERROR_RESILIENCY: return "Encoder, error resiliency";
>> +     case V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE: return "Encoder, temporal layer enable";
>> +
>> +     case V4L2_CID_ENCODER_VP8_SLICE_MODE:   return "Encoder, VP8 slice mode";
>> +     case V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE: return "Encoder, VP8 sync frame type";
>> +     case V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME: return "Encoder, VP8, DCT partitions per frame";
>> +
>> +     case V4L2_CID_ENCODER_H264_PROFILE_TOOLSET: return "Encoder, H.264 profile and toolset";
>> +     case V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT: return "Encoder, H.264 level IDC limit";
>> +     case V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE: return "Encoder, H.264 SEI payload type";
>> +     case V4L2_CID_ENCODER_H264_LAYER_PRIORITY: return "Encoder, H.264 layer priority";
>> +
>>       /* FM Radio Modulator control */
>>       /* Keep the order of the 'case's the same as in videodev2.h! */
>>       case V4L2_CID_FM_TX_CLASS:              return "FM Radio Modulator Controls";
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index 083bb5a..ef3a30d 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -729,6 +729,37 @@ enum v4l2_auto_focus_range {
>>       V4L2_AUTO_FOCUS_RANGE_INFINITY          = 3,
>>  };
>>
>> +/* Controls found in UVC 1.5 encoding cameras */
>> +#define V4L2_CID_ENCODER_MIN_FRAME_INTERVAL  (V4L2_CID_CAMERA_CLASS_BASE+32)
>> +#define V4L2_CID_ENCODER_RATE_CONTROL_MODE   (V4L2_CID_CAMERA_CLASS_BASE+33)
>> +#define V4L2_CID_ENCODER_AVERAGE_BITRATE     (V4L2_CID_CAMERA_CLASS_BASE+34)
>> +#define V4L2_CID_ENCODER_CPB_SIZE            (V4L2_CID_CAMERA_CLASS_BASE+35)
>> +#define V4L2_CID_ENCODER_PEAK_BIT_RATE               (V4L2_CID_CAMERA_CLASS_BASE+36)
>> +#define V4L2_CID_ENCODER_QP_PARAM_I          (V4L2_CID_CAMERA_CLASS_BASE+37)
>> +#define V4L2_CID_ENCODER_QP_PARAM_P          (V4L2_CID_CAMERA_CLASS_BASE+38)
>> +#define V4L2_CID_ENCODER_QP_PARAM_BG         (V4L2_CID_CAMERA_CLASS_BASE+39)
>> +#define V4L2_CID_ENCODER_NUM_GDR_FRAMES              (V4L2_CID_CAMERA_CLASS_BASE+40)
>> +#define V4L2_CID_ENCODER_LTR_BUFFER_CONTROL  (V4L2_CID_CAMERA_CLASS_BASE+41)
>> +#define V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE       (V4L2_CID_CAMERA_CLASS_BASE+42)
>> +#define V4L2_CID_ENCODER_LTR_PICTURE_POSITION        (V4L2_CID_CAMERA_CLASS_BASE+43)
>> +#define V4L2_CID_ENCODER_LTR_PICTURE_MODE    (V4L2_CID_CAMERA_CLASS_BASE+44)
>> +#define V4L2_CID_ENCODER_LTR_VALIDATION              (V4L2_CID_CAMERA_CLASS_BASE+45)
>> +#define V4L2_CID_ENCODER_MIN_QP                      (V4L2_CID_CAMERA_CLASS_BASE+46)
>> +#define V4L2_CID_ENCODER_MAX_QP                      (V4L2_CID_CAMERA_CLASS_BASE+47)
>> +#define V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL (V4L2_CID_CAMERA_CLASS_BASE+48)
>> +#define V4L2_CID_ENCODER_ERROR_RESILIENCY    (V4L2_CID_CAMERA_CLASS_BASE+49)
>> +#define V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE       (V4L2_CID_CAMERA_CLASS_BASE+50)
>> +
>> +/* VP8-specific controls */
>> +#define V4L2_CID_ENCODER_VP8_SLICE_MODE              (V4L2_CID_CAMERA_CLASS_BASE+51)
>> +#define V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME (V4L2_CID_CAMERA_CLASS_BASE+52)
>> +#define V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE (V4L2_CID_CAMERA_CLASS_BASE+53)
>> +
>> +/* H.264-specific controls */
>> +#define V4L2_CID_ENCODER_H264_PROFILE_TOOLSET        (V4L2_CID_CAMERA_CLASS_BASE+54)
>> +#define V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT        (V4L2_CID_CAMERA_CLASS_BASE+55)
>> +#define V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE       (V4L2_CID_CAMERA_CLASS_BASE+56)
>> +#define V4L2_CID_ENCODER_H264_LAYER_PRIORITY (V4L2_CID_CAMERA_CLASS_BASE+57)
>
> Codec controls go to the MPEG class, not the CAMERA class.
>
> Regards,
>
>         Hans
>
>>
>>  /* FM Modulator class control IDs */
>>
>>
>

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Hans Verkuil]

On 09/09/2013 05:48 AM, Pawel Osciak wrote:
> Hi Hans,
> Thanks for the comments, one question inline.
> 
> On Fri, Aug 30, 2013 at 3:48 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>> On 08/30/2013 04:17 AM, Pawel Osciak wrote:
>>> Add defines for controls found in UVC 1.5 encoding cameras.
>>>
>>> Signed-off-by: Pawel Osciak <posciak@chromium.org>
>>> ---
>>>  drivers/media/v4l2-core/v4l2-ctrls.c | 29 +++++++++++++++++++++++++++++
>>>  include/uapi/linux/v4l2-controls.h   | 31 +++++++++++++++++++++++++++++++
>>>  2 files changed, 60 insertions(+)
>>>
>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
>>> index c3f0803..0b3a632 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
>>> @@ -781,6 +781,35 @@ const char *v4l2_ctrl_get_name(u32 id)
>>>       case V4L2_CID_AUTO_FOCUS_STATUS:        return "Auto Focus, Status";
>>>       case V4L2_CID_AUTO_FOCUS_RANGE:         return "Auto Focus, Range";
>>>
>>> +     case V4L2_CID_ENCODER_MIN_FRAME_INTERVAL: return "Encoder, min. frame interval";
>>> +     case V4L2_CID_ENCODER_RATE_CONTROL_MODE: return "Encoder, rate control mode";
>>> +     case V4L2_CID_ENCODER_AVERAGE_BITRATE:  return "Encoder, average bitrate";
>>> +     case V4L2_CID_ENCODER_CPB_SIZE:         return "Encoder, CPB size";
>>> +     case V4L2_CID_ENCODER_PEAK_BIT_RATE:    return "Encoder, peak bit rate";
>>> +     case V4L2_CID_ENCODER_QP_PARAM_I:       return "Encoder, QP param for I frames";
>>> +     case V4L2_CID_ENCODER_QP_PARAM_P:       return "Encoder, QP param for P frames";
>>> +     case V4L2_CID_ENCODER_QP_PARAM_BG:      return "Encoder, QP param for B/G frames";
>>
>> A lot of these exist already. E.g. V4L2_CID_MPEG_VIDEO_MPEG4_I/P/B_FRAME_QP.
>>
>> Samsung added support for many of these parameters for their MFC encoder (including
>> VP8 support) so you should use them as well. As mentioned in v4l2-controls.h the
>> MPEG part of the control name is historical. Interpret it as 'CODEC', not MPEG.
>>
> 
> We have QP controls separately for H264, H263 and MPEG4. Why is that?
> Which one should I use for VP8? Shouldn't we unify them instead?

I can't quite remember the details, so I've CCed Kamil since he added those controls.
At least the H264 QP controls are different from the others as they have a different
range. What's the range for VP8?

I'm not sure why the H263/MPEG4 controls weren't unified: it might be that since the
H264 range was different we decided to split it up per codec. But I seem to remember
that there was another reason as well.

Regards,

	Hans

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Pawel Osciak]

On Mon, Sep 9, 2013 at 4:52 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> On 09/09/2013 05:48 AM, Pawel Osciak wrote:
>> Hi Hans,
>> Thanks for the comments, one question inline.
>>
>> On Fri, Aug 30, 2013 at 3:48 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>> On 08/30/2013 04:17 AM, Pawel Osciak wrote:
>>>> Add defines for controls found in UVC 1.5 encoding cameras.
>>>>
>>>> Signed-off-by: Pawel Osciak <posciak@chromium.org>
>>>> ---
>>>>  drivers/media/v4l2-core/v4l2-ctrls.c | 29 +++++++++++++++++++++++++++++
>>>>  include/uapi/linux/v4l2-controls.h   | 31 +++++++++++++++++++++++++++++++
>>>>  2 files changed, 60 insertions(+)
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
>>>> index c3f0803..0b3a632 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
>>>> @@ -781,6 +781,35 @@ const char *v4l2_ctrl_get_name(u32 id)
>>>>       case V4L2_CID_AUTO_FOCUS_STATUS:        return "Auto Focus, Status";
>>>>       case V4L2_CID_AUTO_FOCUS_RANGE:         return "Auto Focus, Range";
>>>>
>>>> +     case V4L2_CID_ENCODER_MIN_FRAME_INTERVAL: return "Encoder, min. frame interval";
>>>> +     case V4L2_CID_ENCODER_RATE_CONTROL_MODE: return "Encoder, rate control mode";
>>>> +     case V4L2_CID_ENCODER_AVERAGE_BITRATE:  return "Encoder, average bitrate";
>>>> +     case V4L2_CID_ENCODER_CPB_SIZE:         return "Encoder, CPB size";
>>>> +     case V4L2_CID_ENCODER_PEAK_BIT_RATE:    return "Encoder, peak bit rate";
>>>> +     case V4L2_CID_ENCODER_QP_PARAM_I:       return "Encoder, QP param for I frames";
>>>> +     case V4L2_CID_ENCODER_QP_PARAM_P:       return "Encoder, QP param for P frames";
>>>> +     case V4L2_CID_ENCODER_QP_PARAM_BG:      return "Encoder, QP param for B/G frames";
>>>
>>> A lot of these exist already. E.g. V4L2_CID_MPEG_VIDEO_MPEG4_I/P/B_FRAME_QP.
>>>
>>> Samsung added support for many of these parameters for their MFC encoder (including
>>> VP8 support) so you should use them as well. As mentioned in v4l2-controls.h the
>>> MPEG part of the control name is historical. Interpret it as 'CODEC', not MPEG.
>>>
>>
>> We have QP controls separately for H264, H263 and MPEG4. Why is that?
>> Which one should I use for VP8? Shouldn't we unify them instead?
>
> I can't quite remember the details, so I've CCed Kamil since he added those controls.
> At least the H264 QP controls are different from the others as they have a different
> range. What's the range for VP8?
>

Yes, it differs, 0-127.
But I feel this is pretty unfortunate, is it a good idea to multiply
controls to have one per format when they have different ranges
depending on the selected format in general? Perhaps a custom handler
would be better?

> I'm not sure why the H263/MPEG4 controls weren't unified: it might be that since the
> H264 range was different we decided to split it up per codec. But I seem to remember
> that there was another reason as well.
>
> Regards,
>
>         Hans

RE: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Kamil Debski]

Hi,

> From: Pawel Osciak [mailto:posciak@chromium.org]
> Sent: Monday, September 09, 2013 9:59 AM
> 
> On Mon, Sep 9, 2013 at 4:52 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> > On 09/09/2013 05:48 AM, Pawel Osciak wrote:
> >> Hi Hans,
> >> Thanks for the comments, one question inline.
> >>
> >> On Fri, Aug 30, 2013 at 3:48 PM, Hans Verkuil <hverkuil@xs4all.nl>
> wrote:
> >>> On 08/30/2013 04:17 AM, Pawel Osciak wrote:
> >>>> Add defines for controls found in UVC 1.5 encoding cameras.
> >>>>
> >>>> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> >>>> ---
> >>>>  drivers/media/v4l2-core/v4l2-ctrls.c | 29
> +++++++++++++++++++++++++++++
> >>>>  include/uapi/linux/v4l2-controls.h   | 31
> +++++++++++++++++++++++++++++++
> >>>>  2 files changed, 60 insertions(+)
> >>>>
> >>>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> >>>> b/drivers/media/v4l2-core/v4l2-ctrls.c
> >>>> index c3f0803..0b3a632 100644
> >>>> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> >>>> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> >>>> @@ -781,6 +781,35 @@ const char *v4l2_ctrl_get_name(u32 id)
> >>>>       case V4L2_CID_AUTO_FOCUS_STATUS:        return "Auto Focus,
> Status";
> >>>>       case V4L2_CID_AUTO_FOCUS_RANGE:         return "Auto Focus,
> Range";
> >>>>
> >>>> +     case V4L2_CID_ENCODER_MIN_FRAME_INTERVAL: return "Encoder,
> min. frame interval";
> >>>> +     case V4L2_CID_ENCODER_RATE_CONTROL_MODE: return "Encoder,
> rate control mode";
> >>>> +     case V4L2_CID_ENCODER_AVERAGE_BITRATE:  return "Encoder,
> average bitrate";
> >>>> +     case V4L2_CID_ENCODER_CPB_SIZE:         return "Encoder, CPB
> size";
> >>>> +     case V4L2_CID_ENCODER_PEAK_BIT_RATE:    return "Encoder,
> peak bit rate";
> >>>> +     case V4L2_CID_ENCODER_QP_PARAM_I:       return "Encoder, QP
> param for I frames";
> >>>> +     case V4L2_CID_ENCODER_QP_PARAM_P:       return "Encoder, QP
> param for P frames";
> >>>> +     case V4L2_CID_ENCODER_QP_PARAM_BG:      return "Encoder, QP
> param for B/G frames";
> >>>
> >>> A lot of these exist already. E.g.
> V4L2_CID_MPEG_VIDEO_MPEG4_I/P/B_FRAME_QP.
> >>>
> >>> Samsung added support for many of these parameters for their MFC
> >>> encoder (including
> >>> VP8 support) so you should use them as well. As mentioned in
> >>> v4l2-controls.h the MPEG part of the control name is historical.
> Interpret it as 'CODEC', not MPEG.
> >>>
> >>
> >> We have QP controls separately for H264, H263 and MPEG4. Why is that?
> >> Which one should I use for VP8? Shouldn't we unify them instead?
> >
> > I can't quite remember the details, so I've CCed Kamil since he added
> those controls.
> > At least the H264 QP controls are different from the others as they
> > have a different range. What's the range for VP8?
> >
> 
> Yes, it differs, 0-127.
> But I feel this is pretty unfortunate, is it a good idea to multiply
> controls to have one per format when they have different ranges
> depending on the selected format in general? Perhaps a custom handler
> would be better?
> 
> > I'm not sure why the H263/MPEG4 controls weren't unified: it might be
> > that since the
> > H264 range was different we decided to split it up per codec. But I
> > seem to remember that there was another reason as well.
> >

We had a discussion about this on linux-media mailing list. It can be found
here:
http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/326
06
In short, it is a mix of two reasons: one - the valid range is different for
different formats and second - implementing controls which have different
min/max
values depending on format was not easy.

On the one hand I am thinking that now, when we have more codecs, it would
be better
to have a single control, on the other hand what about backward
compatibility?
Is there a graceful way to merge H263 and H264 QP controls?

Best wishes,
Kamil

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Sylwester Nawrocki]

On 09/09/2013 11:00 AM, Kamil Debski wrote:
[...]
>>>> We have QP controls separately for H264, H263 and MPEG4. Why is that?
>>>> Which one should I use for VP8? Shouldn't we unify them instead?
>>>
>>> I can't quite remember the details, so I've CCed Kamil since he added
>> those controls.
>>> At least the H264 QP controls are different from the others as they
>>> have a different range. What's the range for VP8?
>>
>> Yes, it differs, 0-127.
>> But I feel this is pretty unfortunate, is it a good idea to multiply
>> controls to have one per format when they have different ranges
>> depending on the selected format in general? Perhaps a custom handler
>> would be better?
>>
>>> I'm not sure why the H263/MPEG4 controls weren't unified: it might be
>>> that since the
>>> H264 range was different we decided to split it up per codec. But I
>>> seem to remember that there was another reason as well.
> 
> We had a discussion about this on linux-media mailing list. It can be found
> here:
> http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/32606
> In short, it is a mix of two reasons: one - the valid range is different for
> different formats and second - implementing controls which have different
> min/max values depending on format was not easy.

Hmm, these seem pretty vague reasons. And since some time we have support
for dynamic control range update [1].

> On the one hand I am thinking that now, when we have more codecs, it would
> be better
> to have a single control, on the other hand what about backward
> compatibility?
> Is there a graceful way to merge H263 and H264 QP controls?

[1] https://patchwork.linuxtv.org/patch/16436/

--
Regards,
Sylwester

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Hans Verkuil]

On Mon 9 September 2013 11:09:57 Sylwester Nawrocki wrote:
> On 09/09/2013 11:00 AM, Kamil Debski wrote:
> [...]
> >>>> We have QP controls separately for H264, H263 and MPEG4. Why is that?
> >>>> Which one should I use for VP8? Shouldn't we unify them instead?
> >>>
> >>> I can't quite remember the details, so I've CCed Kamil since he added
> >> those controls.
> >>> At least the H264 QP controls are different from the others as they
> >>> have a different range. What's the range for VP8?
> >>
> >> Yes, it differs, 0-127.
> >> But I feel this is pretty unfortunate, is it a good idea to multiply
> >> controls to have one per format when they have different ranges
> >> depending on the selected format in general? Perhaps a custom handler
> >> would be better?
> >>
> >>> I'm not sure why the H263/MPEG4 controls weren't unified: it might be
> >>> that since the
> >>> H264 range was different we decided to split it up per codec. But I
> >>> seem to remember that there was another reason as well.
> > 
> > We had a discussion about this on linux-media mailing list. It can be found
> > here:
> > http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/32606
> > In short, it is a mix of two reasons: one - the valid range is different for
> > different formats and second - implementing controls which have different
> > min/max values depending on format was not easy.
> 
> Hmm, these seem pretty vague reasons. And since some time we have support
> for dynamic control range update [1].

I don't think we should change this. We chose to go with separate controls,
and we should stick with that. We might do it differently today, but it's
not a big deal.

Regards,

	Hans

> 
> > On the one hand I am thinking that now, when we have more codecs, it would
> > be better
> > to have a single control, on the other hand what about backward
> > compatibility?
> > Is there a graceful way to merge H263 and H264 QP controls?
> 
> [1] https://patchwork.linuxtv.org/patch/16436/
> 
> --
> Regards,
> Sylwester
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Pawel Osciak]

On Tue, Sep 10, 2013 at 6:17 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> On Mon 9 September 2013 11:09:57 Sylwester Nawrocki wrote:
>> On 09/09/2013 11:00 AM, Kamil Debski wrote:
>> [...]
>> >>>> We have QP controls separately for H264, H263 and MPEG4. Why is that?
>> >>>> Which one should I use for VP8? Shouldn't we unify them instead?
>> >>>
>> >>> I can't quite remember the details, so I've CCed Kamil since he added
>> >> those controls.
>> >>> At least the H264 QP controls are different from the others as they
>> >>> have a different range. What's the range for VP8?
>> >>
>> >> Yes, it differs, 0-127.
>> >> But I feel this is pretty unfortunate, is it a good idea to multiply
>> >> controls to have one per format when they have different ranges
>> >> depending on the selected format in general? Perhaps a custom handler
>> >> would be better?
>> >>
>> >>> I'm not sure why the H263/MPEG4 controls weren't unified: it might be
>> >>> that since the
>> >>> H264 range was different we decided to split it up per codec. But I
>> >>> seem to remember that there was another reason as well.
>> >
>> > We had a discussion about this on linux-media mailing list. It can be found
>> > here:
>> > http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/32606
>> > In short, it is a mix of two reasons: one - the valid range is different for
>> > different formats and second - implementing controls which have different
>> > min/max values depending on format was not easy.
>>
>> Hmm, these seem pretty vague reasons. And since some time we have support
>> for dynamic control range update [1].
>
> I don't think we should change this. We chose to go with separate controls,
> and we should stick with that. We might do it differently today, but it's
> not a big deal.

I guess I can't reuse MPEG controls as you suggested then. But I could
either create a new VPX class, or keep these ones in camera class and
create separate VPX class later for codecs. It's technically possible
for UVC to have different constraints on some controls though, even if
the codec is the same, potentially creating more confusion...


>
> Regards,
>
>         Hans
>
>>
>> > On the one hand I am thinking that now, when we have more codecs, it would
>> > be better
>> > to have a single control, on the other hand what about backward
>> > compatibility?
>> > Is there a graceful way to merge H263 and H264 QP controls?
>>
>> [1] https://patchwork.linuxtv.org/patch/16436/
>>
>> --
>> Regards,
>> Sylwester
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-media" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Hans Verkuil]

Hi Pawel,

On 09/12/2013 03:10 AM, Pawel Osciak wrote:
> On Tue, Sep 10, 2013 at 6:17 PM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>> On Mon 9 September 2013 11:09:57 Sylwester Nawrocki wrote:
>>> On 09/09/2013 11:00 AM, Kamil Debski wrote:
>>> [...]
>>>>>>> We have QP controls separately for H264, H263 and MPEG4. Why is that?
>>>>>>> Which one should I use for VP8? Shouldn't we unify them instead?
>>>>>>
>>>>>> I can't quite remember the details, so I've CCed Kamil since he added
>>>>> those controls.
>>>>>> At least the H264 QP controls are different from the others as they
>>>>>> have a different range. What's the range for VP8?
>>>>>
>>>>> Yes, it differs, 0-127.
>>>>> But I feel this is pretty unfortunate, is it a good idea to multiply
>>>>> controls to have one per format when they have different ranges
>>>>> depending on the selected format in general? Perhaps a custom handler
>>>>> would be better?
>>>>>
>>>>>> I'm not sure why the H263/MPEG4 controls weren't unified: it might be
>>>>>> that since the
>>>>>> H264 range was different we decided to split it up per codec. But I
>>>>>> seem to remember that there was another reason as well.
>>>>
>>>> We had a discussion about this on linux-media mailing list. It can be found
>>>> here:
>>>> http://comments.gmane.org/gmane.linux.drivers.video-input-infrastructure/32606
>>>> In short, it is a mix of two reasons: one - the valid range is different for
>>>> different formats and second - implementing controls which have different
>>>> min/max values depending on format was not easy.
>>>
>>> Hmm, these seem pretty vague reasons. And since some time we have support
>>> for dynamic control range update [1].
>>
>> I don't think we should change this. We chose to go with separate controls,
>> and we should stick with that. We might do it differently today, but it's
>> not a big deal.
> 
> I guess I can't reuse MPEG controls as you suggested then.

Sure you can. Just call it V4L2_CID_MPEG_VIDEO_VPX_MIN_QP etc. The name 'MPEG'
was an unfortunate choice, it should have been 'CODEC'. And in fact in the spec
it's now known as the Codec Control Class. The QP settings are most definitely
codec controls, not camera or vpx controls.

> But I could
> either create a new VPX class, or keep these ones in camera class and
> create separate VPX class later for codecs. It's technically possible
> for UVC to have different constraints on some controls though, even if
> the codec is the same, potentially creating more confusion...

That's nothing new for uvc and there isn't anything you can do about it.
I don't think this will cause problems in practice.

Regards,

	Hans

> 
> 
>>
>> Regards,
>>
>>         Hans
>>
>>>
>>>> On the one hand I am thinking that now, when we have more codecs, it would
>>>> be better
>>>> to have a single control, on the other hand what about backward
>>>> compatibility?
>>>> Is there a graceful way to merge H263 and H264 QP controls?
>>>
>>> [1] https://patchwork.linuxtv.org/patch/16436/
>>>
>>> --
>>> Regards,
>>> Sylwester
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-media" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>

Re: [PATCH v1 00/19] UVC 1.5 VP8 support for uvcvideo

[Paulo Assis]

Hi,
just want t know the current state on this series.

I'm currently adding h264 stream preview support to guvcview.
It's already working fine on uvc 1.1 cameras like the BCC950 but for
uvc 1.5 devices like the c930e it could really use some driver
support.

Bets Regards,
Paulo

2013/8/30 Pawel Osciak <posciak@chromium.org>:
> Hello everyone,
>
> This series adds support for UVC 1.5 and VP8 encoding cameras to the uvcvideo
> driver. The official specification for the new standard can be found here:
> http://www.usb.org/developers/devclass_docs.
>
> The main change in 1.5 is support for encoding cameras. Those cameras contain
> additional UVC entities, called Encoding Units, with their own set of controls
> governing encode parameters. Typical encoding cameras (see examples in class
> spec) expose two USB Video Streaming Interfaces (VSIs): one for raw stream
> formats and one for encoded streams. Typically, both get their source stream
> from a single sensor, producing raw and encoded versions of the video feed
> simultaneously.
> Encoding Units may also support the so-called "simulcast" formats, which allow
> additional sub-streams, or layers, used to achieve temporal scalability.
> The spec allows up to 4 simulcast layers. Those layers are encoded in the same
> format, but encoding parameters, such as resolution, bitrate, etc., may,
> depending on the camera capabilities, be changed independently for each layer,
> and their streaming state may also be controlled independently as well. The
> layers are streamed from the same USB VSI, and the information which layer
> a frame belongs to is contained in its payload header.
>
> In V4L2 API, a separate video node is created for each VSI: one for raw formats
> VSI and another for the encoded formats VSI. Both can operate completely
> independently from each other. In addition, if the Encoding Unit supports
> simulcast, one V4L2 node is created for each stream layer instead, and each
> can be controlled independently, including streamon/streamoff state, setting
> resolution and controls. Once a simulcast format is successfully set for one
> of the simulcast video nodes however, it cannot be changed, unless all connected
> nodes are idle, i.e. they are not streaming and their buffers are freed.
>
> The userspace can discover if a set of nodes belongs to one encoding unit
> by traversing media controller topology of the camera.
>
>
> I will be gradually posting documentation changes for new features after initial
> rounds of reviews. This is a relatively major change to the UVC driver and
> although I tried to keep the existing logic for UVC <1.5 cameras intact as much
> as possible, I would very much appreciate it if these patches could get some
> testing from you as well, on your own devices/systems.
>
> Thanks,
> Pawel Osciak
>
>
> Pawel Osciak (19):
>       uvcvideo: Add UVC query tracing.
>       uvcvideo: Return 0 when setting probe control succeeds.
>       uvcvideo: Add support for multiple chains with common roots.
>       uvcvideo: Create separate debugfs entries for each streaming interface.
>       uvcvideo: Add support for UVC1.5 P&C control.
>       uvcvideo: Recognize UVC 1.5 encoding units.
>       uvcvideo: Unify error reporting during format descriptor parsing.
>       uvcvideo: Add UVC1.5 VP8 format support.
>       uvcvideo: Reorganize uvc_{get,set}_le_value.
>       uvcvideo: Support UVC 1.5 runtime control property.
>       uvcvideo: Support V4L2_CTRL_TYPE_BITMASK controls.
>       uvcvideo: Reorganize next buffer handling.
>       uvcvideo: Unify UVC payload header parsing.
>       v4l: Add v4l2_buffer flags for VP8-specific special frames.
>       uvcvideo: Add support for VP8 special frame flags.
>       v4l: Add encoding camera controls.
>       uvcvideo: Add UVC 1.5 Encoding Unit controls.
>       v4l: Add V4L2_PIX_FMT_VP8_SIMULCAST format.
>       uvcvideo: Add support for UVC 1.5 VP8 simulcast.
>
>  drivers/media/usb/uvc/uvc_ctrl.c     | 960 ++++++++++++++++++++++++++++++++---
>  drivers/media/usb/uvc/uvc_debugfs.c  |   3 +-
>  drivers/media/usb/uvc/uvc_driver.c   | 604 ++++++++++++++--------
>  drivers/media/usb/uvc/uvc_entity.c   | 129 ++++-
>  drivers/media/usb/uvc/uvc_isight.c   |  12 +-
>  drivers/media/usb/uvc/uvc_queue.c    |  25 +-
>  drivers/media/usb/uvc/uvc_v4l2.c     | 284 +++++++++--
>  drivers/media/usb/uvc/uvc_video.c    | 704 ++++++++++++++++---------
>  drivers/media/usb/uvc/uvcvideo.h     | 214 +++++++-
>  drivers/media/v4l2-core/v4l2-ctrls.c |  29 ++
>  include/uapi/linux/usb/video.h       |  45 ++
>  include/uapi/linux/v4l2-controls.h   |  31 ++
>  include/uapi/linux/videodev2.h       |   8 +
>  13 files changed, 2421 insertions(+), 627 deletions(-)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v1 03/19] uvcvideo: Add support for multiple chains with common roots.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:02 Pawel Osciak wrote:
> This adds support for pipelines that fork into branches consisting of more
> than one entity, creating a chain for each fork and putting common entities
> on all chains that share them.
> 
> This requires us to share the ctrl_mutex across forked chains. Whenever we
> discover a shared part of a chain, we assign the pointer to an existing
> mutex to the sharing chain instead of creating a new one.
> 
> The "forward scan" is not needed anymore, as after scanning back from OTs,
> we go over all entities which are not on a path from an OT and accept
> single-XU branches, adding them to the existing chains.
> 
> Also extract control locking into __uvc_ctrl_{lock,unlock} functions.

This is one core piece of the UVC 1.5 rework, and I have mixed feelings about 
it.

Adding entities to multiple overlapping chains somehow doesn't feel right. 
What would you think about using a pipeline object instead, which would store 
all entities in the pipeline ?

The driver currently iterates over all entities in a chain for several 
purposes:

- registering streaming terminals as video nodes (uvc_driver.c)
- registering MC entities (uvc_entity.c)
- enumerating inputs (uvc_v4l2.c)
- finding controls and extension units (uvc_ctrl.c)

The first two uses could easily be replaced by iterations over the whole 
pipeline. Input enumeration would probably require a bit of custom code 
anyway, so we would be left with controls.

At first sight it would make sense to expose on a video node only the controls 
that can be reached from that video node moving up the pipeline (relatively to 
the video flow). However, this might break existing applications, as the 
driver currently also includes controls reacheable by forward scans of side 
branches. We thus need to carefully think about what controls to include, and 
we need to take into account output video nodes corresponding to input 
streaming terminals.

> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_ctrl.c   |  70 ++++++++-----
>  drivers/media/usb/uvc/uvc_driver.c | 210  +++++++++++++++++++--------------
>  drivers/media/usb/uvc/uvc_entity.c |  15 ++-
>  drivers/media/usb/uvc/uvc_v4l2.c   |   9 +-
>  drivers/media/usb/uvc/uvcvideo.h   |  20 +++-
>  5 files changed, 199 insertions(+), 125 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c
> b/drivers/media/usb/uvc/uvc_ctrl.c index a2f4501..ba159a4 100644
> --- a/drivers/media/usb/uvc/uvc_ctrl.c
> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
> @@ -841,6 +841,7 @@ static struct uvc_control *uvc_find_control(struct
> uvc_video_chain *chain, {
>  	struct uvc_control *ctrl = NULL;
>  	struct uvc_entity *entity;
> +	struct uvc_chain_entry *entry;
>  	int next = v4l2_id & V4L2_CTRL_FLAG_NEXT_CTRL;
> 
>  	*mapping = NULL;
> @@ -849,7 +850,8 @@ static struct uvc_control *uvc_find_control(struct
> uvc_video_chain *chain, v4l2_id &= V4L2_CTRL_ID_MASK;
> 
>  	/* Find the control. */
> -	list_for_each_entry(entity, &chain->entities, chain) {
> +	list_for_each_entry(entry, &chain->entities, chain_entry) {
> +		entity = entry->entity;
>  		__uvc_find_control(entity, v4l2_id, mapping, &ctrl, next);
>  		if (ctrl && !next)
>  			return ctrl;
> @@ -1048,6 +1050,16 @@ static int __uvc_query_v4l2_ctrl(struct
> uvc_video_chain *chain, return 0;
>  }
> 
> +int __uvc_ctrl_lock(struct uvc_video_chain *chain)
> +{
> +	return mutex_lock_interruptible(&chain->pipeline->ctrl_mutex) ?
> +					-ERESTARTSYS : 0;
> +}
> +void __uvc_ctrl_unlock(struct uvc_video_chain *chain)
> +{
> +	mutex_unlock(&chain->pipeline->ctrl_mutex);
> +}
> +
>  int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
>  	struct v4l2_queryctrl *v4l2_ctrl)
>  {
> @@ -1055,9 +1067,9 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
> struct uvc_control_mapping *mapping;
>  	int ret;
> 
> -	ret = mutex_lock_interruptible(&chain->ctrl_mutex);
> +	ret = __uvc_ctrl_lock(chain);
>  	if (ret < 0)
> -		return -ERESTARTSYS;
> +		return ret;
> 
>  	ctrl = uvc_find_control(chain, v4l2_ctrl->id, &mapping);
>  	if (ctrl == NULL) {
> @@ -1067,7 +1079,7 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
> 
>  	ret = __uvc_query_v4l2_ctrl(chain, ctrl, mapping, v4l2_ctrl);
>  done:
> -	mutex_unlock(&chain->ctrl_mutex);
> +	__uvc_ctrl_unlock(chain);
>  	return ret;
>  }
> 
> @@ -1094,9 +1106,9 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
> query_menu->id = id;
>  	query_menu->index = index;
> 
> -	ret = mutex_lock_interruptible(&chain->ctrl_mutex);
> +	ret = __uvc_ctrl_lock(chain);
>  	if (ret < 0)
> -		return -ERESTARTSYS;
> +		return ret;
> 
>  	ctrl = uvc_find_control(chain, query_menu->id, &mapping);
>  	if (ctrl == NULL || mapping->v4l2_type != V4L2_CTRL_TYPE_MENU) {
> @@ -1132,7 +1144,7 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
> strlcpy(query_menu->name, menu_info->name, sizeof query_menu->name);
> 
>  done:
> -	mutex_unlock(&chain->ctrl_mutex);
> +	__uvc_ctrl_unlock(chain);
>  	return ret;
>  }
> 
> @@ -1257,9 +1269,9 @@ static int uvc_ctrl_add_event(struct
> v4l2_subscribed_event *sev, unsigned elems) struct uvc_control *ctrl;
>  	int ret;
> 
> -	ret = mutex_lock_interruptible(&handle->chain->ctrl_mutex);
> +	ret = __uvc_ctrl_lock(handle->chain);
>  	if (ret < 0)
> -		return -ERESTARTSYS;
> +		return ret;
> 
>  	ctrl = uvc_find_control(handle->chain, sev->id, &mapping);
>  	if (ctrl == NULL) {
> @@ -1285,7 +1297,7 @@ static int uvc_ctrl_add_event(struct
> v4l2_subscribed_event *sev, unsigned elems) }
> 
>  done:
> -	mutex_unlock(&handle->chain->ctrl_mutex);
> +	__uvc_ctrl_unlock(handle->chain);
>  	return ret;
>  }
> 
> @@ -1293,9 +1305,9 @@ static void uvc_ctrl_del_event(struct
> v4l2_subscribed_event *sev) {
>  	struct uvc_fh *handle = container_of(sev->fh, struct uvc_fh, vfh);
> 
> -	mutex_lock(&handle->chain->ctrl_mutex);
> +	__uvc_ctrl_lock(handle->chain);
>  	list_del(&sev->node);
> -	mutex_unlock(&handle->chain->ctrl_mutex);
> +	__uvc_ctrl_unlock(handle->chain);
>  }
> 
>  const struct v4l2_subscribed_event_ops uvc_ctrl_sub_ev_ops = {
> @@ -1331,7 +1343,7 @@ const struct v4l2_subscribed_event_ops
> uvc_ctrl_sub_ev_ops = { */
>  int uvc_ctrl_begin(struct uvc_video_chain *chain)
>  {
> -	return mutex_lock_interruptible(&chain->ctrl_mutex) ? -ERESTARTSYS : 0;
> +	return __uvc_ctrl_lock(chain);
>  }
> 
>  static int uvc_ctrl_commit_entity(struct uvc_device *dev,
> @@ -1390,10 +1402,12 @@ int __uvc_ctrl_commit(struct uvc_fh *handle, int
> rollback, {
>  	struct uvc_video_chain *chain = handle->chain;
>  	struct uvc_entity *entity;
> +	struct uvc_chain_entry *entry;
>  	int ret = 0;
> 
>  	/* Find the control. */
> -	list_for_each_entry(entity, &chain->entities, chain) {
> +	list_for_each_entry(entry, &chain->entities, chain_entry) {
> +		entity = entry->entity;
>  		ret = uvc_ctrl_commit_entity(chain->dev, entity, rollback);
>  		if (ret < 0)
>  			goto done;
> @@ -1402,7 +1416,7 @@ int __uvc_ctrl_commit(struct uvc_fh *handle, int
> rollback, if (!rollback)
>  		uvc_ctrl_send_events(handle, xctrls, xctrls_count);
>  done:
> -	mutex_unlock(&chain->ctrl_mutex);
> +	__uvc_ctrl_unlock(chain);
>  	return ret;
>  }
> 
> @@ -1667,7 +1681,8 @@ static int uvc_ctrl_init_xu_ctrl(struct uvc_device
> *dev, int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>  	struct uvc_xu_control_query *xqry)
>  {
> -	struct uvc_entity *entity;
> +	struct uvc_entity *entity = NULL;
> +	struct uvc_chain_entry *entry;
>  	struct uvc_control *ctrl;
>  	unsigned int i, found = 0;
>  	__u32 reqflags;
> @@ -1676,13 +1691,14 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
> int ret;
> 
>  	/* Find the extension unit. */
> -	list_for_each_entry(entity, &chain->entities, chain) {
> +	list_for_each_entry(entry, &chain->entities, chain_entry) {
> +		entity = entry->entity;
>  		if (UVC_ENTITY_TYPE(entity) == UVC_VC_EXTENSION_UNIT &&
>  		    entity->id == xqry->unit)
>  			break;
>  	}
> 
> -	if (entity->id != xqry->unit) {
> +	if (!entity || entity->id != xqry->unit) {
>  		uvc_trace(UVC_TRACE_CONTROL, "Extension unit %u not found.\n",
>  			xqry->unit);
>  		return -ENOENT;
> @@ -1703,8 +1719,9 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>  		return -ENOENT;
>  	}
> 
> -	if (mutex_lock_interruptible(&chain->ctrl_mutex))
> -		return -ERESTARTSYS;
> +	ret = __uvc_ctrl_lock(chain);
> +	if (ret < 0)
> +		return ret;
> 
>  	ret = uvc_ctrl_init_xu_ctrl(chain->dev, ctrl);
>  	if (ret < 0) {
> @@ -1778,7 +1795,7 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>  		ret = -EFAULT;
>  done:
>  	kfree(data);
> -	mutex_unlock(&chain->ctrl_mutex);
> +	__uvc_ctrl_unlock(chain);
>  	return ret;
>  }
> 
> @@ -1904,6 +1921,7 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
> *chain, const struct uvc_control_mapping *mapping)
>  {
>  	struct uvc_device *dev = chain->dev;
> +	struct uvc_chain_entry *entry;
>  	struct uvc_control_mapping *map;
>  	struct uvc_entity *entity;
>  	struct uvc_control *ctrl;
> @@ -1918,8 +1936,9 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
> *chain, }
> 
>  	/* Search for the matching (GUID/CS) control on the current chain */
> -	list_for_each_entry(entity, &chain->entities, chain) {
> +	list_for_each_entry(entry, &chain->entities, chain_entry) {
>  		unsigned int i;
> +		entity = entry->entity;
> 
>  		if (UVC_ENTITY_TYPE(entity) != UVC_VC_EXTENSION_UNIT ||
>  		    !uvc_entity_match_guid(entity, mapping->entity))
> @@ -1939,8 +1958,9 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
> *chain, if (!found)
>  		return -ENOENT;
> 
> -	if (mutex_lock_interruptible(&chain->ctrl_mutex))
> -		return -ERESTARTSYS;
> +	ret = __uvc_ctrl_lock(chain);
> +	if (ret < 0)
> +		return ret;
> 
>  	/* Perform delayed initialization of XU controls */
>  	ret = uvc_ctrl_init_xu_ctrl(dev, ctrl);
> @@ -1974,7 +1994,7 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
> *chain, atomic_dec(&dev->nmappings);
> 
>  done:
> -	mutex_unlock(&chain->ctrl_mutex);
> +	__uvc_ctrl_unlock(chain);
>  	return ret;
>  }
> 
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index 81695d4..d7ff707 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -1215,6 +1215,36 @@ next_descriptor:
>   * UVC device scan
>   */
> 
> +static int uvc_add_chain_entry(struct uvc_video_chain *chain,
> +				struct uvc_entity *entity)
> +{
> +	struct uvc_chain_entry *chain_entry;
> +
> +	chain_entry = kzalloc(sizeof(struct uvc_chain_entry), GFP_KERNEL);
> +	if (!chain_entry)
> +		return -ENOMEM;
> +
> +	chain_entry->entity = entity;
> +	list_add_tail(&chain_entry->chain_entry, &chain->entities);
> +	if (!entity->chain)
> +		entity->chain = chain;
> +
> +	return 0;
> +}
> +
> +static void uvc_delete_chain(struct uvc_video_chain *chain)
> +{
> +	struct list_head *p, *n;
> +	struct uvc_chain_entry *entry;
> +
> +	list_for_each_safe(p, n, &chain->entities) {
> +		entry = list_entry(p, struct uvc_chain_entry, chain_entry);
> +		kfree(entry);
> +	}
> +
> +	kfree(chain);
> +}
> +
>  /*
>   * Scan the UVC descriptors to locate a chain starting at an Output
> Terminal * and containing the following units:
> @@ -1320,72 +1350,7 @@ static int uvc_scan_chain_entity(struct
> uvc_video_chain *chain, return -1;
>  	}
> 
> -	list_add_tail(&entity->chain, &chain->entities);
> -	return 0;
> -}
> -
> -static int uvc_scan_chain_forward(struct uvc_video_chain *chain,
> -	struct uvc_entity *entity, struct uvc_entity *prev)
> -{
> -	struct uvc_entity *forward;
> -	int found;
> -
> -	/* Forward scan */
> -	forward = NULL;
> -	found = 0;
> -
> -	while (1) {
> -		forward = uvc_entity_by_reference(chain->dev, entity->id,
> -			forward);
> -		if (forward == NULL)
> -			break;
> -		if (forward == prev)
> -			continue;
> -
> -		switch (UVC_ENTITY_TYPE(forward)) {
> -		case UVC_VC_EXTENSION_UNIT:
> -			if (forward->bNrInPins != 1) {
> -				uvc_trace(UVC_TRACE_DESCR, "Extension unit %d "
> -					  "has more than 1 input pin.\n",
> -					  entity->id);
> -				return -EINVAL;
> -			}
> -
> -			list_add_tail(&forward->chain, &chain->entities);
> -			if (uvc_trace_param & UVC_TRACE_PROBE) {
> -				if (!found)
> -					printk(" (->");
> -
> -				printk(" XU %d", forward->id);
> -				found = 1;
> -			}
> -			break;
> -
> -		case UVC_OTT_VENDOR_SPECIFIC:
> -		case UVC_OTT_DISPLAY:
> -		case UVC_OTT_MEDIA_TRANSPORT_OUTPUT:
> -		case UVC_TT_STREAMING:
> -			if (UVC_ENTITY_IS_ITERM(forward)) {
> -				uvc_trace(UVC_TRACE_DESCR, "Unsupported input "
> -					"terminal %u.\n", forward->id);
> -				return -EINVAL;
> -			}
> -
> -			list_add_tail(&forward->chain, &chain->entities);
> -			if (uvc_trace_param & UVC_TRACE_PROBE) {
> -				if (!found)
> -					printk(" (->");
> -
> -				printk(" OT %d", forward->id);
> -				found = 1;
> -			}
> -			break;
> -		}
> -	}
> -	if (found)
> -		printk(")");
> -
> -	return 0;
> +	return uvc_add_chain_entry(chain, entity);
>  }
> 
>  static int uvc_scan_chain_backward(struct uvc_video_chain *chain,
> @@ -1394,6 +1359,7 @@ static int uvc_scan_chain_backward(struct
> uvc_video_chain *chain, struct uvc_entity *entity = *_entity;
>  	struct uvc_entity *term;
>  	int id = -EINVAL, i;
> +	int ret;
> 
>  	switch (UVC_ENTITY_TYPE(entity)) {
>  	case UVC_VC_EXTENSION_UNIT:
> @@ -1425,8 +1391,9 @@ static int uvc_scan_chain_backward(struct
> uvc_video_chain *chain, if (uvc_trace_param & UVC_TRACE_PROBE)
>  				printk(" %d", term->id);
> 
> -			list_add_tail(&term->chain, &chain->entities);
> -			uvc_scan_chain_forward(chain, term, entity);
> +			ret = uvc_add_chain_entry(chain, term);
> +			if (ret)
> +				return ret;
>  		}
> 
>  		if (uvc_trace_param & UVC_TRACE_PROBE)
> @@ -1473,23 +1440,23 @@ static int uvc_scan_chain(struct uvc_video_chain
> *chain, prev = NULL;
> 
>  	while (entity != NULL) {
> -		/* Entity must not be part of an existing chain */
> -		if (entity->chain.next || entity->chain.prev) {
> -			uvc_trace(UVC_TRACE_DESCR, "Found reference to "
> -				"entity %d already in chain.\n", entity->id);
> +		if (entity->chain == chain) {
> +			uvc_trace(UVC_TRACE_DESCR, "Found a cycle in the "
> +					"chain");
>  			return -EINVAL;
>  		}
> 
> +		/* If this entity is a part of an existing chain, the
> +		 * current chain belongs to the same pipeline.
> +		 */
> +		if (entity->chain)
> +			chain->pipeline = entity->chain->pipeline;
> +
>  		/* Process entity */
>  		if (uvc_scan_chain_entity(chain, entity) < 0)
>  			return -EINVAL;
> 
> -		/* Forward scan */
> -		if (uvc_scan_chain_forward(chain, entity, prev) < 0)
> -			return -EINVAL;
> -
>  		/* Backward scan */
> -		prev = entity;
>  		if (uvc_scan_chain_backward(chain, &entity) < 0)
>  			return -EINVAL;
>  	}
> @@ -1501,10 +1468,12 @@ static unsigned int uvc_print_terms(struct list_head
> *terms, u16 dir, char *buffer)
>  {
>  	struct uvc_entity *term;
> +	struct uvc_chain_entry *entry;
>  	unsigned int nterms = 0;
>  	char *p = buffer;
> 
> -	list_for_each_entry(term, terms, chain) {
> +	list_for_each_entry(entry, terms, chain_entry) {
> +		term = entry->entity;
>  		if (!UVC_ENTITY_IS_TERM(term) ||
>  		    UVC_TERM_DIRECTION(term) != dir)
>  			continue;
> @@ -1541,39 +1510,58 @@ static const char *uvc_print_chain(struct
> uvc_video_chain *chain) static int uvc_scan_device(struct uvc_device *dev)
>  {
>  	struct uvc_video_chain *chain;
> -	struct uvc_entity *term;
> +	struct uvc_entity *entity, *source;
> +	int ret;
> 
> -	list_for_each_entry(term, &dev->entities, list) {
> -		if (!UVC_ENTITY_IS_OTERM(term))
> +	list_for_each_entry(entity, &dev->entities, list) {
> +		if (!UVC_ENTITY_IS_OTERM(entity))
>  			continue;
> 
> -		/* If the terminal is already included in a chain, skip it.
> -		 * This can happen for chains that have multiple output
> -		 * terminals, where all output terminals beside the first one
> -		 * will be inserted in the chain in forward scans.
> +		/* Allow single-unit branches of Output Terminals to reside
> +		 * on the existing chains.
>  		 */
> -		if (term->chain.next || term->chain.prev)
> -			continue;
> +		source = uvc_entity_by_id(dev, entity->baSourceID[0]);
> +		if (entity == NULL) {
> +			uvc_trace(UVC_TRACE_DESCR, "Found reference to "
> +				"unknown entity %d.\n", entity->baSourceID[0]);
> +			return -EINVAL;
> +		}
> 
>  		chain = kzalloc(sizeof(*chain), GFP_KERNEL);
>  		if (chain == NULL)
>  			return -ENOMEM;
> 
>  		INIT_LIST_HEAD(&chain->entities);
> -		mutex_init(&chain->ctrl_mutex);
>  		chain->dev = dev;
>  		v4l2_prio_init(&chain->prio);
> 
> -		term->flags |= UVC_ENTITY_FLAG_DEFAULT;
> +		entity->flags |= UVC_ENTITY_FLAG_DEFAULT;
> 
> -		if (uvc_scan_chain(chain, term) < 0) {
> -			kfree(chain);
> +		if (uvc_scan_chain(chain, entity) < 0) {
> +			uvc_delete_chain(chain);
>  			continue;
>  		}
> 
>  		uvc_trace(UVC_TRACE_PROBE, "Found a valid video chain (%s).\n",
>  			  uvc_print_chain(chain));
> 
> +		/*
> +		 * If none of the entities are shared, allocate a new pipeline.
> +		 * Otherwise, the shared pipeline is already set up.
> +		 */
> +		if (!chain->pipeline) {
> +			chain->pipeline = kzalloc(sizeof(*chain->pipeline),
> +						  GFP_KERNEL);
> +			if (!chain->pipeline) {
> +				uvc_delete_chain(chain);
> +				return -ENOMEM;
> +			}
> +			mutex_init(&chain->pipeline->ctrl_mutex);
> +			atomic_set(&chain->pipeline->num_chains, 1);
> +		} else {
> +			atomic_inc(&chain->pipeline->num_chains);
> +		}
> +
>  		list_add_tail(&chain->list, &dev->chains);
>  	}
> 
> @@ -1582,6 +1570,38 @@ static int uvc_scan_device(struct uvc_device *dev)
>  		return -1;
>  	}
> 
> +	/* Find branches with no OTERMs (if any) by looking for entities not
> +	 * on any chain. Accept only branches with a single Extension Unit.
> +	 */
> +	list_for_each_entry(entity, &dev->entities, list) {
> +		if (entity->chain)
> +			continue;
> +
> +		if (UVC_ENTITY_TYPE(entity) != UVC_VC_EXTENSION_UNIT
> +			|| entity->bNrInPins != 1
> +			|| uvc_entity_by_reference(dev, entity->id, NULL)) {
> +			uvc_printk(KERN_INFO, "Found an invalid branch "
> +				"starting at entity id %d.\n", entity->id);
> +			return -1;
> +		}
> +
> +		/* Single-unit XU branch. */
> +		source = uvc_entity_by_id(dev, entity->baSourceID[0]);
> +		if (source == NULL) {
> +			uvc_trace(UVC_TRACE_DESCR, "Found reference to "
> +				"unknown entity %d.\n", entity->baSourceID[0]);
> +			return -EINVAL;
> +		}
> +		if (!source->chain)
> +			continue;
> +
> +		ret = uvc_add_chain_entry(source->chain, entity);
> +		if (ret)
> +			return ret;
> +		uvc_trace(UVC_TRACE_DESCR, "XU %d <- (%d)\n",
> +				entity->id, source->id);
> +	}
> +
>  	return 0;
>  }
> 
> @@ -1619,7 +1639,9 @@ static void uvc_delete(struct uvc_device *dev)
>  	list_for_each_safe(p, n, &dev->chains) {
>  		struct uvc_video_chain *chain;
>  		chain = list_entry(p, struct uvc_video_chain, list);
> -		kfree(chain);
> +		if (atomic_dec_and_test(&chain->pipeline->num_chains))
> +			kfree(chain->pipeline);
> +		uvc_delete_chain(chain);
>  	}
> 
>  	list_for_each_safe(p, n, &dev->entities) {
> @@ -1763,9 +1785,11 @@ static int uvc_register_terms(struct uvc_device *dev,
> {
>  	struct uvc_streaming *stream;
>  	struct uvc_entity *term;
> +	struct uvc_chain_entry *entry;
>  	int ret;
> 
> -	list_for_each_entry(term, &chain->entities, chain) {
> +	list_for_each_entry(entry, &chain->entities, chain_entry) {
> +		term = entry->entity;
>  		if (UVC_ENTITY_TYPE(term) != UVC_TT_STREAMING)
>  			continue;
> 
> diff --git a/drivers/media/usb/uvc/uvc_entity.c
> b/drivers/media/usb/uvc/uvc_entity.c index dc56a59..657f49a 100644
> --- a/drivers/media/usb/uvc/uvc_entity.c
> +++ b/drivers/media/usb/uvc/uvc_entity.c
> @@ -104,9 +104,14 @@ static int uvc_mc_init_entity(struct uvc_entity
> *entity) int uvc_mc_register_entities(struct uvc_video_chain *chain)
>  {
>  	struct uvc_entity *entity;
> +	struct uvc_chain_entry *entry;
>  	int ret;
> 
> -	list_for_each_entry(entity, &chain->entities, chain) {
> +	list_for_each_entry(entry, &chain->entities, chain_entry) {
> +		entity = entry->entity;
> +		if (entity->registered)
> +			continue;
> +
>  		ret = uvc_mc_init_entity(entity);
>  		if (ret < 0) {
>  			uvc_printk(KERN_INFO, "Failed to initialize entity for "
> @@ -115,13 +120,19 @@ int uvc_mc_register_entities(struct uvc_video_chain
> *chain) }
>  	}
> 
> -	list_for_each_entry(entity, &chain->entities, chain) {
> +	list_for_each_entry(entry, &chain->entities, chain_entry) {
> +		entity = entry->entity;
> +		if (entity->registered)
> +			continue;
> +
>  		ret = uvc_mc_register_entity(chain, entity);
>  		if (ret < 0) {
>  			uvc_printk(KERN_INFO, "Failed to register entity for "
>  				   "entity %u\n", entity->id);
>  			return ret;
>  		}
> +
> +		entity->registered = true;
>  	}
> 
>  	return 0;
> diff --git a/drivers/media/usb/uvc/uvc_v4l2.c
> b/drivers/media/usb/uvc/uvc_v4l2.c index 3afff92..a899159 100644
> --- a/drivers/media/usb/uvc/uvc_v4l2.c
> +++ b/drivers/media/usb/uvc/uvc_v4l2.c
> @@ -713,6 +713,7 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg) const struct uvc_entity *selector =
> chain->selector;
>  		struct v4l2_input *input = arg;
>  		struct uvc_entity *iterm = NULL;
> +		struct uvc_chain_entry *entry;
>  		u32 index = input->index;
>  		int pin = 0;
> 
> @@ -720,14 +721,18 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg) (chain->dev->quirks &
> UVC_QUIRK_IGNORE_SELECTOR_UNIT)) {
>  			if (index != 0)
>  				return -EINVAL;
> -			list_for_each_entry(iterm, &chain->entities, chain) {
> +			list_for_each_entry(entry, &chain->entities,
> +						chain_entry) {
> +				iterm = entry->entity;
>  				if (UVC_ENTITY_IS_ITERM(iterm))
>  					break;
>  			}
>  			pin = iterm->id;
>  		} else if (index < selector->bNrInPins) {
>  			pin = selector->baSourceID[index];
> -			list_for_each_entry(iterm, &chain->entities, chain) {
> +			list_for_each_entry(entry, &chain->entities,
> +						chain_entry) {
> +				iterm = entry->entity;
>  				if (!UVC_ENTITY_IS_ITERM(iterm))
>  					continue;
>  				if (iterm->id == pin)
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index 75e0153..731b378 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -229,8 +229,8 @@ struct uvc_format_desc {
> 
>  struct uvc_entity {
>  	struct list_head list;		/* Entity as part of a UVC device. */
> -	struct list_head chain;		/* Entity as part of a video device
> -					 * chain. */
> +	struct uvc_video_chain *chain;  /* Entity as a part of a video device
> +					   chain. */
>  	unsigned int flags;
> 
>  	__u8 id;
> @@ -243,6 +243,7 @@ struct uvc_entity {
>  	unsigned int num_pads;
>  	unsigned int num_links;
>  	struct media_pad *pads;
> +	bool registered;		/* True if already registered with MC */
> 
>  	union {
>  		struct {
> @@ -289,6 +290,12 @@ struct uvc_entity {
>  	struct uvc_control *controls;
>  };
> 
> +struct uvc_chain_entry {
> +	struct list_head chain_entry;
> +	struct uvc_entity *entity;
> +	struct uvc_video_chain *chain;
> +};
> +
>  struct uvc_frame {
>  	__u8  bFrameIndex;
>  	__u8  bmCapabilities;
> @@ -366,6 +373,12 @@ struct uvc_video_queue {
>  	struct list_head irqqueue;
>  };
> 
> +struct uvc_video_pipeline {
> +	struct mutex ctrl_mutex;                /* Protects controls in all
> +						   chains of this pipeline */
> +	atomic_t num_chains;
> +};
> +
>  struct uvc_video_chain {
>  	struct uvc_device *dev;
>  	struct list_head list;
> @@ -374,7 +387,8 @@ struct uvc_video_chain {
>  	struct uvc_entity *processing;		/* Processing unit */
>  	struct uvc_entity *selector;		/* Selector unit */
> 
> -	struct mutex ctrl_mutex;		/* Protects ctrl.info */
> +	struct uvc_video_pipeline *pipeline;    /* Pipeline this chain
> +						   belongs to */
> 
>  	struct v4l2_prio_state prio;		/* V4L2 priority state */
>  	u32 caps;				/* V4L2 chain-wide caps */
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 06/19] uvcvideo: Recognize UVC 1.5 encoding units.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:05 Pawel Osciak wrote:
> Add encoding unit definitions and descriptor parsing code and allow them to
> be added to chains.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_ctrl.c   | 37 ++++++++++++++++++---
>  drivers/media/usb/uvc/uvc_driver.c | 67 ++++++++++++++++++++++++++++++-----
>  drivers/media/usb/uvc/uvcvideo.h   | 14 +++++++-
>  include/uapi/linux/usb/video.h     |  1 +
>  4 files changed, 105 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c
> b/drivers/media/usb/uvc/uvc_ctrl.c index ba159a4..72d6724 100644
> --- a/drivers/media/usb/uvc/uvc_ctrl.c
> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
> @@ -777,6 +777,7 @@ static const __u8 uvc_processing_guid[16] =
> UVC_GUID_UVC_PROCESSING; static const __u8 uvc_camera_guid[16] =
> UVC_GUID_UVC_CAMERA;
>  static const __u8 uvc_media_transport_input_guid[16] =
>  	UVC_GUID_UVC_MEDIA_TRANSPORT_INPUT;
> +static const __u8 uvc_encoding_guid[16] = UVC_GUID_UVC_ENCODING;
> 
>  static int uvc_entity_match_guid(const struct uvc_entity *entity,
>  	const __u8 guid[16])
> @@ -795,6 +796,9 @@ static int uvc_entity_match_guid(const struct uvc_entity
> *entity, return memcmp(entity->extension.guidExtensionCode,
>  			      guid, 16) == 0;
> 
> +	case UVC_VC_ENCODING_UNIT:
> +		return memcmp(uvc_encoding_guid, guid, 16) == 0;
> +
>  	default:
>  		return 0;
>  	}
> @@ -2105,12 +2109,13 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
>  {
>  	struct uvc_entity *entity;
>  	unsigned int i;
> +	int num_found;
> 
>  	/* Walk the entities list and instantiate controls */
>  	list_for_each_entry(entity, &dev->entities, list) {
>  		struct uvc_control *ctrl;
> -		unsigned int bControlSize = 0, ncontrols;
> -		__u8 *bmControls = NULL;
> +		unsigned int bControlSize = 0, ncontrols = 0;
> +		__u8 *bmControls = NULL, *bmControlsRuntime = NULL;
> 
>  		if (UVC_ENTITY_TYPE(entity) == UVC_VC_EXTENSION_UNIT) {
>  			bmControls = entity->extension.bmControls;
> @@ -2121,13 +2126,25 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
>  		} else if (UVC_ENTITY_TYPE(entity) == UVC_ITT_CAMERA) {
>  			bmControls = entity->camera.bmControls;
>  			bControlSize = entity->camera.bControlSize;
> +		} else if (UVC_ENTITY_TYPE(entity) == UVC_VC_ENCODING_UNIT) {
> +			bmControls = entity->encoding.bmControls;
> +			bmControlsRuntime = entity->encoding.bmControlsRuntime;
> +			bControlSize = entity->encoding.bControlSize;
>  		}
> 
>  		/* Remove bogus/blacklisted controls */
>  		uvc_ctrl_prune_entity(dev, entity);
> 
>  		/* Count supported controls and allocate the controls array */
> -		ncontrols = memweight(bmControls, bControlSize);
> +		for (i = 0; i < bControlSize; ++i) {
> +			if (bmControlsRuntime) {
> +				ncontrols += hweight8(bmControls[i]
> +						      | bmControlsRuntime[i]);

Nitpicking, could you move the | to the end of the previous line to match the 
style of the rest of the code ?

> +			} else {
> +				ncontrols += hweight8(bmControls[i]);
> +			}

The { } are not needed.

> +		}
> +
>  		if (ncontrols == 0)
>  			continue;
> 
> @@ -2139,8 +2156,17 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
> 
>  		/* Initialize all supported controls */
>  		ctrl = entity->controls;
> -		for (i = 0; i < bControlSize * 8; ++i) {
> -			if (uvc_test_bit(bmControls, i) == 0)
> +		for (i = 0, num_found = 0;
> +			i < bControlSize * 8 && num_found < ncontrols; ++i) {
> +			if (uvc_test_bit(bmControls, i) == 1)
> +				ctrl->on_init = 1;
> +			if (bmControlsRuntime &&
> +				uvc_test_bit(bmControlsRuntime, i) == 1)
> +				ctrl->in_runtime = 1;
> +			else if (!bmControlsRuntime)
> +				ctrl->in_runtime = ctrl->on_init;
> +
> +			if (ctrl->on_init == 0 && ctrl->in_runtime == 0)
>  				continue;

Wouldn't it simplify the code if you assigned bmControls to bmControlsRuntime 
when bmControlsRuntime is NULL before counting the controls above ? You could 
get rid of the if inside the count loop, and you could also simplify this 
construct.

> 
>  			ctrl->entity = entity;
> @@ -2148,6 +2174,7 @@ int uvc_ctrl_init_device(struct uvc_device *dev)
> 
>  			uvc_ctrl_init_ctrl(dev, ctrl);
>  			ctrl++;
> +			num_found++;
>  		}
>  	}
> 
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index d7ff707..d950b40 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -1155,6 +1155,37 @@ static int uvc_parse_standard_control(struct
> uvc_device *dev, list_add_tail(&unit->list, &dev->entities);
>  		break;
> 
> +	case UVC_VC_ENCODING_UNIT:
> +		n = buflen >= 7 ? buffer[6] : 0;
> +
> +		if (buflen < 7 + 2 * n) {
> +			uvc_trace(UVC_TRACE_DESCR, "device %d videocontrol "
> +				"interface %d ENCODING_UNIT error\n",
> +				udev->devnum, alts->desc.bInterfaceNumber);
> +			return -EINVAL;
> +		}
> +
> +		unit = uvc_alloc_entity(buffer[2], buffer[3], 2, 2 * n);
> +		if (unit == NULL)
> +			return -ENOMEM;
> +
> +		memcpy(unit->baSourceID, &buffer[4], 1);
> +		unit->encoding.bControlSize = buffer[6];
> +		unit->encoding.bmControls = (__u8 *)unit + sizeof(*unit);
> +		memcpy(unit->encoding.bmControls, &buffer[7], n);
> +		unit->encoding.bmControlsRuntime = unit->encoding.bmControls
> +						 + n;
> +		memcpy(unit->encoding.bmControlsRuntime, &buffer[7 + n], n);
> +
> +		if (buffer[5] != 0)
> +			usb_string(udev, buffer[5], unit->name,
> +				   sizeof(unit->name));
> +		else
> +			sprintf(unit->name, "encoding %u", buffer[3]);

Could you s/encoding/Encoding/ to match the other unit names ?

> +
> +		list_add_tail(&unit->list, &dev->entities);
> +		break;
> +
>  	default:
>  		uvc_trace(UVC_TRACE_DESCR, "Found an unknown CS_INTERFACE "
>  			"descriptor (%u)\n", buffer[2]);
> @@ -1251,25 +1282,31 @@ static void uvc_delete_chain(struct uvc_video_chain
> *chain) *
>   * - one or more Output Terminals (USB Streaming or Display)
>   * - zero or one Processing Unit
> + * - zero or one Encoding Unit
>   * - zero, one or more single-input Selector Units
>   * - zero or one multiple-input Selector Units, provided all inputs are
>   *   connected to input terminals
> - * - zero, one or mode single-input Extension Units
> + * - zero, one or more single-input Extension Units
>   * - one or more Input Terminals (Camera, External or USB Streaming)
>   *
> - * The terminal and units must match on of the following structures:
> + * The terminal and units must match one of the following structures:

While we're at it, s/terminal/terminals/ ?

>   *
> - * ITT_*(0) -> +---------+    +---------+    +---------+ -> TT_STREAMING(0)
> - * ...         | SU{0,1} | -> | PU{0,1} | -> | XU{0,n} |    ...
> - * ITT_*(n) -> +---------+    +---------+    +---------+ -> TT_STREAMING(n)
> + * ITT_*(0) -> +---------+                        -> TT_STREAMING(0) + *
> ...         | SU{0,1} | ->        (...)           ...
> + * ITT_*(n) -> +---------+                        -> TT_STREAMING(n)
> + *
> + *    Where (...), in any order:
> + *             +---------+    +---------+    +---------+
> + *             | PU{0,1} | -> | XU{0,n} | -> | EU{0,1} |
> + *             +---------+    +---------+    +---------+
>   *
>   *                 +---------+    +---------+ -> OTT_*(0)
>   * TT_STREAMING -> | PU{0,1} | -> | XU{0,n} |    ...
>   *                 +---------+    +---------+ -> OTT_*(n)
>   *
> - * The Processing Unit and Extension Units can be in any order. Additional
> - * Extension Units connected to the main chain as single-unit branches are
> - * also supported. Single-input Selector Units are ignored.
> + * The Processing Unit, the Encoding Unit and Extension Units can be in any
> + * order. Additional Extension Units connected to the main chain as
> single-unit
> + * branches are also supported. Single-input Selector Units are ignored. */
>  static int uvc_scan_chain_entity(struct uvc_video_chain *chain,
>  	struct uvc_entity *entity)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 08/19] uvcvideo: Add UVC1.5 VP8 format support.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:07 Pawel Osciak wrote:
> Add detection and parsing of VP8 format and frame descriptors and
> reorganize format parsing code.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_driver.c | 120 +++++++++++++++++++++++++---------
>  drivers/media/usb/uvc/uvcvideo.h   |   4 +-
>  include/uapi/linux/usb/video.h     |   8 +++
>  3 files changed, 104 insertions(+), 28 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_driver.c
> b/drivers/media/usb/uvc/uvc_driver.c index 936ddc7..27a7a11 100644
> --- a/drivers/media/usb/uvc/uvc_driver.c
> +++ b/drivers/media/usb/uvc/uvc_driver.c
> @@ -312,7 +312,7 @@ static int uvc_parse_format(struct uvc_device *dev,
>  	struct uvc_frame *frame;
>  	const unsigned char *start = buffer;
>  	unsigned int interval;
> -	unsigned int i, n;
> +	unsigned int i, n, intervals_off;

Could you please define the intervals_off variable on a new line, right above 
interval ?

>  	__u8 ftype;
> 
>  	format->type = buffer[2];
> @@ -401,6 +401,18 @@ static int uvc_parse_format(struct uvc_device *dev,
>  		format->nframes = 1;
>  		break;
> 
> +	case UVC_VS_FORMAT_VP8:
> +		if (buflen < 13)
> +			goto format_error;
> +
> +		format->bpp = 0;
> +		format->flags = UVC_FMT_FLAG_COMPRESSED;
> +		ftype = UVC_VS_FRAME_VP8;

Nitpicking, could you please move this line after format->fcc, to keep 
statements initializing format together ?

> +		strlcpy(format->name, "VP8", sizeof(format->name));
> +		format->fcc = V4L2_PIX_FMT_VP8;
> +
> +		break;
> +
>  	case UVC_VS_FORMAT_MPEG2TS:
>  	case UVC_VS_FORMAT_STREAM_BASED:
>  		/* Not supported yet. */
> @@ -417,44 +429,83 @@ static int uvc_parse_format(struct uvc_device *dev,
>  	buflen -= buffer[0];
>  	buffer += buffer[0];
> 
> -	/* Parse the frame descriptors. Only uncompressed, MJPEG and frame
> -	 * based formats have frame descriptors.
> +	/* Parse the frame descriptors. Only uncompressed, MJPEG, temporally
> +	 * encoded and frame based formats have frame descriptors.
>  	 */
>  	while (buflen > 2 && buffer[1] == USB_DT_CS_INTERFACE &&
>  	       buffer[2] == ftype) {
>  		frame = &format->frame[format->nframes];
> -		if (ftype != UVC_VS_FRAME_FRAME_BASED)
> -			n = buflen > 25 ? buffer[25] : 0;
> -		else
> -			n = buflen > 21 ? buffer[21] : 0;
> -
> -		n = n ? n : 3;
> 
> -		if (buflen < 26 + 4*n) {
> -			uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
> -			       "interface %d FRAME error\n", dev->udev->devnum,
> -			       alts->desc.bInterfaceNumber);
> -			return -EINVAL;
> -		}
> -
> -		frame->bFrameIndex = buffer[3];
> -		frame->bmCapabilities = buffer[4];
> -		frame->wWidth = get_unaligned_le16(&buffer[5]);
> -		frame->wHeight = get_unaligned_le16(&buffer[7]);
> -		frame->dwMinBitRate = get_unaligned_le32(&buffer[9]);
> -		frame->dwMaxBitRate = get_unaligned_le32(&buffer[13]);
> -		if (ftype != UVC_VS_FRAME_FRAME_BASED) {

I'd like to create a uvc_parse_frame function for the code below. The function 
should be created in a new patch before this one. I can do it if you would 
like me to.

> +		switch (ftype) {
> +		case UVC_VS_FRAME_UNCOMPRESSED:
> +		case UVC_VS_FRAME_MJPEG:
> +			intervals_off = 26;
> +			if (buflen < intervals_off)
> +				goto frame_error;
> +
> +			frame->bFrameIndex = buffer[3];
> +			frame->bmCapabilities = buffer[4];
> +			frame->wWidth = get_unaligned_le16(&buffer[5]);
> +			frame->wHeight = get_unaligned_le16(&buffer[7]);
> +			frame->dwMinBitRate = get_unaligned_le32(&buffer[9]);
> +			frame->dwMaxBitRate = get_unaligned_le32(&buffer[13]);
>  			frame->dwMaxVideoFrameBufferSize =
>  				get_unaligned_le32(&buffer[17]);
>  			frame->dwDefaultFrameInterval =
>  				get_unaligned_le32(&buffer[21]);
> -			frame->bFrameIntervalType = buffer[25];
> -		} else {
> +			frame->bFrameIntervalType = n = buffer[25];

One assignement per line please.

> +			break;
> +
> +		case UVC_VS_FRAME_FRAME_BASED:
> +			intervals_off = 26;
> +			if (buflen < intervals_off)
> +				goto frame_error;
> +
> +			frame->bFrameIndex = buffer[3];
> +			frame->bmCapabilities = buffer[4];
> +			frame->wWidth = get_unaligned_le16(&buffer[5]);
> +			frame->wHeight = get_unaligned_le16(&buffer[7]);
> +			frame->dwMinBitRate = get_unaligned_le32(&buffer[9]);
> +			frame->dwMaxBitRate = get_unaligned_le32(&buffer[13]);
>  			frame->dwMaxVideoFrameBufferSize = 0;
>  			frame->dwDefaultFrameInterval =
>  				get_unaligned_le32(&buffer[17]);
> -			frame->bFrameIntervalType = buffer[21];
> +			frame->bFrameIntervalType = n = buffer[21];

What about combining the 3 cases above ? Most of the code is identical.

> +			break;
> +
> +		case UVC_VS_FRAME_VP8:
> +			intervals_off = 31;
> +			if (buflen < intervals_off)
> +				goto frame_error;
> +
> +			frame->bFrameIndex = buffer[3];
> +			frame->bmSupportedUsages =
> +				get_unaligned_le32(&buffer[8]);
> +			frame->bmCapabilities =
> +				get_unaligned_le16(&buffer[12]);
> +			frame->bmScalabilityCapabilities =
> +				get_unaligned_le32(&buffer[14]);
> +			frame->wWidth = get_unaligned_le16(&buffer[4]);
> +			frame->wHeight = get_unaligned_le16(&buffer[6]);

Could you keep these sorted by the buffer offset ?

> +			frame->dwMinBitRate = get_unaligned_le32(&buffer[18]);
> +			frame->dwMaxBitRate = get_unaligned_le32(&buffer[22]);
> +			frame->dwMaxVideoFrameBufferSize = 0;
> +			frame->dwDefaultFrameInterval =
> +				get_unaligned_le32(&buffer[26]);
> +			frame->bFrameIntervalType = n = buffer[30];

One assignment per line here as well please.

VP8 doesn't seem to support continuous frame intervals. Are the number of 
frames intervals always greater than 0, or is a 0 value valid ?

> +			break;
> +
> +		default:
> +			uvc_trace(UVC_TRACE_CONTROL,
> +				"Unsupported frame descriptor %d\n", ftype);
> +			return -EINVAL;
>  		}
> +
> +		/* For n=0 - continuous intervals given, always 3 values. */
> +		n = n ? n : 3;

Maybe

		/* A zero frame interval type indicates a continuous range,
		 * always described by 3 values.
		 */
		n = frame->bFrameIntervalType ? frame->bFrameIntervalType : 3;

You could then get rid of the n assignment in the different cases.

> +		if (buflen < intervals_off + 4 * n)
> +			goto frame_error;
> +
>  		frame->dwFrameInterval = *intervals;
> 
>  		/* Several UVC chipsets screw up dwMaxVideoFrameBufferSize
> @@ -475,12 +526,14 @@ static int uvc_parse_format(struct uvc_device *dev,
>  		 * some other divisions by zero that could happen.
>  		 */
>  		for (i = 0; i < n; ++i) {
> -			interval = get_unaligned_le32(&buffer[26+4*i]);
> +			interval = get_unaligned_le32(
> +					&buffer[intervals_off + 4 * i]);
>  			*(*intervals)++ = interval ? interval : 1;
>  		}
> 
>  		/* Make sure that the default frame interval stays between
>  		 * the boundaries.
> +		 * For type = 0, the last value is interval step, so skip it.
>  		 */
>  		n -= frame->bFrameIntervalType ? 1 : 2;
>  		frame->dwDefaultFrameInterval =
> @@ -535,6 +588,11 @@ format_error:
>  			alts->desc.bInterfaceNumber);
>  	return -EINVAL;
> 
> +frame_error:
> +	uvc_trace(UVC_TRACE_DESCR, "device %d videostreaming "
> +			"interface %d FRAME error\n", dev->udev->devnum,
> +			alts->desc.bInterfaceNumber);
> +	return -EINVAL;
>  }
> 
>  static int uvc_parse_streaming(struct uvc_device *dev,


-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 09/19] uvcvideo: Reorganize uvc_{get,set}_le_value.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:08 Pawel Osciak wrote:

Could you add commit message to explain why this patch is needed ?

> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_ctrl.c | 62 ++++++++++++++++++++++---------------
>  1 file changed, 37 insertions(+), 25 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c
> b/drivers/media/usb/uvc/uvc_ctrl.c index 72d6724..d735c88 100644
> --- a/drivers/media/usb/uvc/uvc_ctrl.c
> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
> @@ -707,18 +707,12 @@ static inline void uvc_clear_bit(__u8 *data, int bit)
>  	data[bit >> 3] &= ~(1 << (bit & 7));
>  }
> 
> -/* Extract the bit string specified by mapping->offset and mapping->size
> - * from the little-endian data stored at 'data' and return the result as
> - * a signed 32bit integer. Sign extension will be performed if the mapping
> - * references a signed data type.
> - */
> -static __s32 uvc_get_le_value(struct uvc_control_mapping *mapping,
> -	__u8 query, const __u8 *data)
> +static int __uvc_get_le_value(int bits, int offset, const __u8 *data,
> +				__u32 data_type)
>  {
> -	int bits = mapping->size;
> -	int offset = mapping->offset;
>  	__s32 value = 0;
>  	__u8 mask;
> +	int size = bits;
> 
>  	data += offset / 8;
>  	offset &= 7;
> @@ -733,22 +727,49 @@ static __s32 uvc_get_le_value(struct
> uvc_control_mapping *mapping, }
> 
>  	/* Sign-extend the value if needed. */
> -	if (mapping->data_type == UVC_CTRL_DATA_TYPE_SIGNED)
> -		value |= -(value & (1 << (mapping->size - 1)));
> +	if (data_type == UVC_CTRL_DATA_TYPE_SIGNED)
> +		value |= -(value & (1 << (size - 1)));
> 
>  	return value;
>  }
> 
> +/* Extract the bit string specified by mapping->offset and mapping->size
> + * from the little-endian data stored at 'data' and return the result as
> + * a signed 32bit integer. Sign extension will be performed if the mapping
> + * references a signed data type.
> + */
> +static __s32 uvc_get_le_value(struct uvc_control_mapping *mapping,
> +	__u8 query, const __u8 *data)
> +{
> +	return __uvc_get_le_value(mapping->size, mapping->offset, data,
> +					mapping->data_type);
> +}
> +
> +static void __uvc_set_le_value(int bits, int offset, __s32 value, __u8
> *data, +				bool keep_existing)
> +{
> +	__u8 mask;
> +
> +	data += offset / 8;
> +	offset &= 7;
> +
> +	for (; bits > 0; data++) {
> +		mask = ((1LL << bits) - 1) << offset;
> +		if (!keep_existing)

keep_existing is not used by this patch, I'd remove it from here and add it to 
the patch that makes use of the feature.

> +			*data = (*data & ~mask);
> +		*data |= ((value << offset) & mask);
> +		value >>= (8 - offset);
> +		bits -= 8 - offset;
> +		offset = 0;
> +	}
> +}
> +
>  /* Set the bit string specified by mapping->offset and mapping->size
>   * in the little-endian data stored at 'data' to the value 'value'.
>   */
>  static void uvc_set_le_value(struct uvc_control_mapping *mapping,
>  	__s32 value, __u8 *data)
>  {
> -	int bits = mapping->size;
> -	int offset = mapping->offset;
> -	__u8 mask;
> -
>  	/* According to the v4l2 spec, writing any value to a button control
>  	 * should result in the action belonging to the button control being
>  	 * triggered. UVC devices however want to see a 1 written -> override
> @@ -757,16 +778,7 @@ static void uvc_set_le_value(struct uvc_control_mapping
> *mapping, if (mapping->v4l2_type == V4L2_CTRL_TYPE_BUTTON)
>  		value = -1;
> 
> -	data += offset / 8;
> -	offset &= 7;
> -
> -	for (; bits > 0; data++) {
> -		mask = ((1LL << bits) - 1) << offset;
> -		*data = (*data & ~mask) | ((value << offset) & mask);
> -		value >>= offset ? offset : 8;
> -		bits -= 8 - offset;
> -		offset = 0;
> -	}
> +	__uvc_set_le_value(mapping->size, mapping->offset, value, data, false);
>  }
> 
>  /* ------------------------------------------------------------------------
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 10/19] uvcvideo: Support UVC 1.5 runtime control property.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:09 Pawel Osciak wrote:
> UVC 1.5 introduces the concept of runtime controls, which can be set during
> streaming. Non-runtime controls can only be changed while device is idle.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_ctrl.c | 45 ++++++++++++++++++++++++++++++-------
>  drivers/media/usb/uvc/uvc_v4l2.c | 18 ++++++++++------
>  drivers/media/usb/uvc/uvcvideo.h | 12 +++++++----
>  3 files changed, 57 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c
> b/drivers/media/usb/uvc/uvc_ctrl.c index d735c88..b0a19b9 100644
> --- a/drivers/media/usb/uvc/uvc_ctrl.c
> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
> @@ -1076,8 +1076,19 @@ void __uvc_ctrl_unlock(struct uvc_video_chain *chain)
> mutex_unlock(&chain->pipeline->ctrl_mutex);
>  }
> 
> +static int uvc_check_ctrl_runtime(struct uvc_control *ctrl, bool streaming)
> +{
> +	if (streaming && !ctrl->in_runtime) {
> +		uvc_trace(UVC_TRACE_CONTROL,
> +				"Control disabled while streaming\n");
> +		return -EBUSY;
> +	}
> +
> +	return 0;
> +}
> +
>  int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
> -	struct v4l2_queryctrl *v4l2_ctrl)
> +			struct v4l2_queryctrl *v4l2_ctrl, bool streaming)
>  {
>  	struct uvc_control *ctrl;
>  	struct uvc_control_mapping *mapping;
> @@ -1093,6 +1104,10 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain
> *chain, goto done;
>  	}
> 
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		goto done;
> +

Do we really need to disallow querying controls during streaming ? Shouldn't 
we cache the runtime controls at init time instead ?

>  	ret = __uvc_query_v4l2_ctrl(chain, ctrl, mapping, v4l2_ctrl);
>  done:
>  	__uvc_ctrl_unlock(chain);
> @@ -1109,7 +1124,7 @@ done:
>   * manually.
>   */
>  int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
> -	struct v4l2_querymenu *query_menu)
> +	struct v4l2_querymenu *query_menu, bool streaming)
>  {
>  	struct uvc_menu_info *menu_info;
>  	struct uvc_control_mapping *mapping;
> @@ -1132,6 +1147,10 @@ int uvc_query_v4l2_menu(struct uvc_video_chain
> *chain, goto done;
>  	}
> 
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		goto done;
> +

Same here.

>  	if (query_menu->index >= mapping->menu_count) {
>  		ret = -EINVAL;
>  		goto done;
> @@ -1436,21 +1455,26 @@ done:
>  	return ret;
>  }
> 
> -int uvc_ctrl_get(struct uvc_video_chain *chain,
> -	struct v4l2_ext_control *xctrl)
> +int uvc_ctrl_get(struct uvc_video_chain *chain, struct v4l2_ext_control
> *xctrl,
> +		 bool streaming)
>  {
>  	struct uvc_control *ctrl;
>  	struct uvc_control_mapping *mapping;
> +	int ret;
> 
>  	ctrl = uvc_find_control(chain, xctrl->id, &mapping);
>  	if (ctrl == NULL)
>  		return -EINVAL;
> 
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		return ret;
> +

Even for the get operation, would it be possible to use the cached value ? Can 
a runtime control be also an auto-update or asynchronous control ?

>  	return __uvc_ctrl_get(chain, ctrl, mapping, &xctrl->value);
>  }
> 
> -int uvc_ctrl_set(struct uvc_video_chain *chain,
> -	struct v4l2_ext_control *xctrl)
> +int uvc_ctrl_set(struct uvc_video_chain *chain, struct v4l2_ext_control
> *xctrl,
> +		 bool streaming)
>  {
>  	struct uvc_control *ctrl;
>  	struct uvc_control_mapping *mapping;
> @@ -1466,6 +1490,10 @@ int uvc_ctrl_set(struct uvc_video_chain *chain,
>  	if (!(ctrl->info.flags & UVC_CTRL_FLAG_SET_CUR))
>  		return -EACCES;
> 
> +	ret = uvc_check_ctrl_runtime(ctrl, streaming);
> +	if (ret < 0)
> +		return ret;
> +
>  	/* Clamp out of range values. */
>  	switch (mapping->v4l2_type) {
>  	case V4L2_CTRL_TYPE_INTEGER:
> @@ -1885,8 +1913,9 @@ static int uvc_ctrl_add_info(struct uvc_device *dev,
> struct uvc_control *ctrl, ctrl->initialized = 1;
> 
>  	uvc_trace(UVC_TRACE_CONTROL, "Added control %pUl/%u to device %s "
> -		"entity %u\n", ctrl->info.entity, ctrl->info.selector,
> -		dev->udev->devpath, ctrl->entity->id);
> +		"entity %u, init/runtime %d/%d\n", ctrl->info.entity,

%u/%u ?

> +		ctrl->info.selector, dev->udev->devpath, ctrl->entity->id,
> +		ctrl->on_init, ctrl->in_runtime);
> 
>  done:
>  	if (ret < 0)
> diff --git a/drivers/media/usb/uvc/uvc_v4l2.c
> b/drivers/media/usb/uvc/uvc_v4l2.c index a899159..decd65f 100644
> --- a/drivers/media/usb/uvc/uvc_v4l2.c
> +++ b/drivers/media/usb/uvc/uvc_v4l2.c
> @@ -597,7 +597,8 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg)
> 
>  	/* Get, Set & Query control */
>  	case VIDIOC_QUERYCTRL:
> -		return uvc_query_v4l2_ctrl(chain, arg);
> +		return uvc_query_v4l2_ctrl(chain, arg,
> +					uvc_is_stream_streaming(stream));

Can't this (and all the similar constructs below) race with the 
VIDIOC_STREAMON/OFF ioctls ?

>  	case VIDIOC_G_CTRL:
>  	{
> @@ -611,7 +612,8 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg) if (ret < 0)
>  			return ret;
> 
> -		ret = uvc_ctrl_get(chain, &xctrl);
> +		ret = uvc_ctrl_get(chain, &xctrl,
> +					uvc_is_stream_streaming(stream));
>  		uvc_ctrl_rollback(handle);
>  		if (ret >= 0)
>  			ctrl->value = xctrl.value;
> @@ -635,7 +637,8 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg) if (ret < 0)
>  			return ret;
> 
> -		ret = uvc_ctrl_set(chain, &xctrl);
> +		ret = uvc_ctrl_set(chain, &xctrl,
> +					uvc_is_stream_streaming(stream));
>  		if (ret < 0) {
>  			uvc_ctrl_rollback(handle);
>  			return ret;
> @@ -647,7 +650,8 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg) }
> 
>  	case VIDIOC_QUERYMENU:
> -		return uvc_query_v4l2_menu(chain, arg);
> +		return uvc_query_v4l2_menu(chain, arg,
> +					uvc_is_stream_streaming(stream));
> 
>  	case VIDIOC_G_EXT_CTRLS:
>  	{
> @@ -660,7 +664,8 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg) return ret;
> 
>  		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
> -			ret = uvc_ctrl_get(chain, ctrl);
> +			ret = uvc_ctrl_get(chain, ctrl,
> +					uvc_is_stream_streaming(stream));
>  			if (ret < 0) {
>  				uvc_ctrl_rollback(handle);
>  				ctrls->error_idx = i;
> @@ -688,7 +693,8 @@ static long uvc_v4l2_do_ioctl(struct file *file,
> unsigned int cmd, void *arg) return ret;
> 
>  		for (i = 0; i < ctrls->count; ++ctrl, ++i) {
> -			ret = uvc_ctrl_set(chain, ctrl);
> +			ret = uvc_ctrl_set(chain, ctrl,
> +					uvc_is_stream_streaming(stream));
>  			if (ret < 0) {
>  				uvc_ctrl_rollback(handle);
>  				ctrls->error_idx = cmd == VIDIOC_S_EXT_CTRLS
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index 88f5e38..46ffd92 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -694,6 +694,10 @@ extern int uvc_query_ctrl(struct uvc_device *dev, __u8
> query, __u8 unit, void uvc_video_clock_update(struct uvc_streaming *stream,
>  			    struct v4l2_buffer *v4l2_buf,
>  			    struct uvc_buffer *buf);
> +static inline bool uvc_is_stream_streaming(struct uvc_streaming *stream)
> +{
> +	return vb2_is_streaming(&stream->queue.queue);
> +}
> 
>  /* Status */
>  extern int uvc_status_init(struct uvc_device *dev);
> @@ -705,9 +709,9 @@ extern void uvc_status_stop(struct uvc_device *dev);
>  extern const struct v4l2_subscribed_event_ops uvc_ctrl_sub_ev_ops;
> 
>  extern int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
> -		struct v4l2_queryctrl *v4l2_ctrl);
> +		struct v4l2_queryctrl *v4l2_ctrl, bool streaming);
>  extern int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
> -		struct v4l2_querymenu *query_menu);
> +		struct v4l2_querymenu *query_menu, bool streaming);
> 
>  extern int uvc_ctrl_add_mapping(struct uvc_video_chain *chain,
>  		const struct uvc_control_mapping *mapping);
> @@ -731,9 +735,9 @@ static inline int uvc_ctrl_rollback(struct uvc_fh
> *handle) }
> 
>  extern int uvc_ctrl_get(struct uvc_video_chain *chain,
> -		struct v4l2_ext_control *xctrl);
> +		struct v4l2_ext_control *xctrl, bool streaming);
>  extern int uvc_ctrl_set(struct uvc_video_chain *chain,
> -		struct v4l2_ext_control *xctrl);
> +		struct v4l2_ext_control *xctrl, bool streaming);
> 
>  extern int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>  		struct uvc_xu_control_query *xqry);
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 11/19] uvcvideo: Support V4L2_CTRL_TYPE_BITMASK controls.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patcj.

On Friday 30 August 2013 11:17:10 Pawel Osciak wrote:

Maybe a commit message here too ?

> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_ctrl.c | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c
> b/drivers/media/usb/uvc/uvc_ctrl.c index b0a19b9..a0493d6 100644
> --- a/drivers/media/usb/uvc/uvc_ctrl.c
> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
> @@ -1550,6 +1550,24 @@ int uvc_ctrl_set(struct uvc_video_chain *chain,
> struct v4l2_ext_control *xctrl,
> 
>  		break;
> 
> +	case V4L2_CTRL_TYPE_BITMASK:
> +		value = xctrl->value;
> +
> +		/* If GET_RES is supported, it will return a bitmask of bits
> +		 * that can be set. If it isn't, allow any value.
> +		 */
> +		if (ctrl->info.flags & UVC_CTRL_FLAG_GET_RES) {
> +			if (!ctrl->cached) {
> +				ret = uvc_ctrl_populate_cache(chain, ctrl);
> +				if (ret < 0)
> +					return ret;
> +			}
> +			step = mapping->get(mapping, UVC_GET_RES,
> +					uvc_ctrl_data(ctrl, UVC_CTRL_DATA_RES));
> +			if (value & ~step)
> +				return -ERANGE;
> +		}

Missing break ?

> +
>  	default:
>  		value = xctrl->value;
>  		break;
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 12/19] uvcvideo: Reorganize next buffer handling.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:11 Pawel Osciak wrote:
> Move getting the first buffer from the current queue to a uvc_queue function
> and out of the USB completion handler.

Could you please add a sentence explaining why the patch is needed ?

> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_isight.c |  6 ++++--
>  drivers/media/usb/uvc/uvc_queue.c  | 14 ++++++++++++++
>  drivers/media/usb/uvc/uvc_video.c  | 29 ++++++++++++-----------------
>  drivers/media/usb/uvc/uvcvideo.h   |  7 +++----
>  4 files changed, 33 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_isight.c
> b/drivers/media/usb/uvc/uvc_isight.c index 8510e72..ab01286 100644
> --- a/drivers/media/usb/uvc/uvc_isight.c
> +++ b/drivers/media/usb/uvc/uvc_isight.c
> @@ -99,10 +99,12 @@ static int isight_decode(struct uvc_video_queue *queue,
> struct uvc_buffer *buf, return 0;
>  }
> 
> -void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
> -		struct uvc_buffer *buf)
> +void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream)
> {
>  	int ret, i;
> +	struct uvc_buffer *buf;

Could you please move this on top of the previous line ?

> +
> +	buf = uvc_queue_get_first_buf(&stream->queue);
> 
>  	for (i = 0; i < urb->number_of_packets; ++i) {
>  		if (urb->iso_frame_desc[i].status < 0) {
> diff --git a/drivers/media/usb/uvc/uvc_queue.c
> b/drivers/media/usb/uvc/uvc_queue.c index cd962be..55d2670 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c
> @@ -352,6 +352,20 @@ void uvc_queue_cancel(struct uvc_video_queue *queue,
> int disconnect) spin_unlock_irqrestore(&queue->irqlock, flags);
>  }
> 
> +struct uvc_buffer *uvc_queue_get_first_buf(struct uvc_video_queue *queue)
> +{
> +	struct uvc_buffer *buf = NULL;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&queue->irqlock, flags);
> +	if (!list_empty(&queue->irqqueue))
> +		buf = list_first_entry(&queue->irqqueue, struct uvc_buffer,
> +					queue);
> +	spin_unlock_irqrestore(&queue->irqlock, flags);
> +
> +	return buf;
> +}
> +
>  struct uvc_buffer *uvc_queue_next_buffer(struct uvc_video_queue *queue,
>  		struct uvc_buffer *buf)
>  {
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index b4ebccd..2f9a5fa 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1193,11 +1193,11 @@ static int uvc_video_encode_data(struct
> uvc_streaming *stream, /*
>   * Completion handler for video URBs.
>   */
> -static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming
> *stream, -	struct uvc_buffer *buf)
> +static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming
> *stream) {
>  	u8 *mem;
>  	int ret, i;
> +	struct uvc_buffer *buf = NULL;

Same here (and below).

>  	for (i = 0; i < urb->number_of_packets; ++i) {
>  		if (urb->iso_frame_desc[i].status < 0) {
> @@ -1211,6 +1211,7 @@ static void uvc_video_decode_isoc(struct urb *urb,
> struct uvc_streaming *stream,
> 
>  		/* Decode the payload header. */
>  		mem = urb->transfer_buffer + urb->iso_frame_desc[i].offset;
> +		buf = uvc_queue_get_first_buf(&stream->queue);

Can't this call be moved outside of the loop ?

>  		do {
>  			ret = uvc_video_decode_start(stream, buf, mem,
>  				urb->iso_frame_desc[i].actual_length);
> @@ -1241,11 +1242,11 @@ static void uvc_video_decode_isoc(struct urb *urb,
> struct uvc_streaming *stream, }
>  }
> 
> -static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming
> *stream, -	struct uvc_buffer *buf)
> +static void uvc_video_decode_bulk(struct urb *urb, struct uvc_streaming
> *stream) {
>  	u8 *mem;
>  	int len, ret;
> +	struct uvc_buffer *buf;
> 
>  	/*
>  	 * Ignore ZLPs if they're not part of a frame, otherwise process them
> @@ -1258,6 +1259,8 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream, len = urb->actual_length;
>  	stream->bulk.payload_size += len;
> 
> +	buf = uvc_queue_get_first_buf(&stream->queue);
> +
>  	/* If the URB is the first of its payload, decode and save the
>  	 * header.
>  	 */
> @@ -1309,12 +1312,13 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream, }
>  }
> 
> -static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming
> *stream, -	struct uvc_buffer *buf)
> +static void uvc_video_encode_bulk(struct urb *urb, struct uvc_streaming
> *stream) {
>  	u8 *mem = urb->transfer_buffer;
>  	int len = stream->urb_size, ret;
> +	struct uvc_buffer *buf;
> 
> +	buf = uvc_queue_get_first_buf(&stream->queue);
>  	if (buf == NULL) {
>  		urb->transfer_buffer_length = 0;
>  		return;
> @@ -1355,9 +1359,6 @@ static void uvc_video_encode_bulk(struct urb *urb,
> struct uvc_streaming *stream, static void uvc_video_complete(struct urb
> *urb)
>  {
>  	struct uvc_streaming *stream = urb->context;
> -	struct uvc_video_queue *queue = &stream->queue;
> -	struct uvc_buffer *buf = NULL;
> -	unsigned long flags;
>  	int ret;
> 
>  	switch (urb->status) {
> @@ -1374,17 +1375,11 @@ static void uvc_video_complete(struct urb *urb)
> 
>  	case -ECONNRESET:	/* usb_unlink_urb() called. */
>  	case -ESHUTDOWN:	/* The endpoint is being disabled. */
> -		uvc_queue_cancel(queue, urb->status == -ESHUTDOWN);
> +		uvc_queue_cancel(&stream->queue, urb->status == -ESHUTDOWN);
>  		return;
>  	}
> 
> -	spin_lock_irqsave(&queue->irqlock, flags);
> -	if (!list_empty(&queue->irqqueue))
> -		buf = list_first_entry(&queue->irqqueue, struct uvc_buffer,
> -				       queue);
> -	spin_unlock_irqrestore(&queue->irqlock, flags);
> -
> -	stream->decode(urb, stream, buf);
> +	stream->decode(urb, stream);
> 
>  	if ((ret = usb_submit_urb(urb, GFP_ATOMIC)) < 0) {
>  		uvc_printk(KERN_ERR, "Failed to resubmit video URB (%d).\n",
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index 46ffd92..bca8715 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -482,8 +482,7 @@ struct uvc_streaming {
>  	/* Buffers queue. */
>  	unsigned int frozen : 1;
>  	struct uvc_video_queue queue;
> -	void (*decode) (struct urb *urb, struct uvc_streaming *video,
> -			struct uvc_buffer *buf);
> +	void (*decode) (struct urb *urb, struct uvc_streaming *video);
> 
>  	/* Context data used by the bulk completion handler. */
>  	struct {
> @@ -659,6 +658,7 @@ extern int uvc_dequeue_buffer(struct uvc_video_queue
> *queue, struct v4l2_buffer *v4l2_buf, int nonblocking);
>  extern int uvc_queue_enable(struct uvc_video_queue *queue, int enable);
>  extern void uvc_queue_cancel(struct uvc_video_queue *queue, int
> disconnect); +struct uvc_buffer *uvc_queue_get_first_buf(struct
> uvc_video_queue *queue); extern struct uvc_buffer
> *uvc_queue_next_buffer(struct uvc_video_queue *queue, struct uvc_buffer
> *buf);
>  extern int uvc_queue_mmap(struct uvc_video_queue *queue,
> @@ -751,8 +751,7 @@ extern struct usb_host_endpoint *uvc_find_endpoint(
>  		struct usb_host_interface *alts, __u8 epaddr);
> 
>  /* Quirks support */
> -void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming *stream,
> -		struct uvc_buffer *buf);
> +void uvc_video_decode_isight(struct urb *urb, struct uvc_streaming
> *stream);
> 
>  /* debugfs and statistics */
>  int uvc_debugfs_init(void);
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 13/19] uvcvideo: Unify UVC payload header parsing.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:12 Pawel Osciak wrote:
> Create a separate function for parsing UVC payload headers and extract code
> from other functions into it. Store the parsed values in a header struct.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_video.c | 270 ++++++++++++++++++-----------------
>  drivers/media/usb/uvc/uvcvideo.h  |  21 +++
>  2 files changed, 157 insertions(+), 134 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 2f9a5fa..59f57a2 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -422,40 +422,14 @@ static int uvc_commit_video(struct uvc_streaming
> *stream,
> 
>  static void
>  uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer
> *buf,
> -		       const __u8 *data, int len)
> +			struct uvc_payload_header *header)
>  {
>  	struct uvc_clock_sample *sample;
> -	unsigned int header_size;
> -	bool has_pts = false;
> -	bool has_scr = false;
>  	unsigned long flags;
>  	struct timespec ts;
>  	u16 host_sof;
>  	u16 dev_sof;
> 
> -	switch (data[1] & (UVC_STREAM_PTS | UVC_STREAM_SCR)) {
> -	case UVC_STREAM_PTS | UVC_STREAM_SCR:
> -		header_size = 12;
> -		has_pts = true;
> -		has_scr = true;
> -		break;
> -	case UVC_STREAM_PTS:
> -		header_size = 6;
> -		has_pts = true;
> -		break;
> -	case UVC_STREAM_SCR:
> -		header_size = 8;
> -		has_scr = true;
> -		break;
> -	default:
> -		header_size = 2;
> -		break;
> -	}
> -
> -	/* Check for invalid headers. */
> -	if (len < header_size)
> -		return;
> -
>  	/* Extract the timestamps:
>  	 *
>  	 * - store the frame PTS in the buffer structure
> @@ -463,17 +437,17 @@ uvc_video_clock_decode(struct uvc_streaming *stream,
> struct uvc_buffer *buf, *   kernel timestamps and store them with the SCR
> STC and SOF fields *   in the ring buffer
>  	 */
> -	if (has_pts && buf != NULL)
> -		buf->pts = get_unaligned_le32(&data[2]);
> +	if (header->has_pts && buf != NULL)
> +		buf->pts = header->pts;
> 
> -	if (!has_scr)
> +	if (!header->has_scr)
>  		return;
> 
>  	/* To limit the amount of data, drop SCRs with an SOF identical to the
>  	 * previous one.
>  	 */
> -	dev_sof = get_unaligned_le16(&data[header_size - 2]);
> -	if (dev_sof == stream->clock.last_sof)
> +	dev_sof = header->sof;
> +	if (dev_sof <= stream->clock.last_sof)

This change (== -> <=) is unrelated. If it's need it please split it to a 
separate patch.

>  		return;
> 
>  	stream->clock.last_sof = dev_sof;
> @@ -513,7 +487,7 @@ uvc_video_clock_decode(struct uvc_streaming *stream,
> struct uvc_buffer *buf, spin_lock_irqsave(&stream->clock.lock, flags);
> 
>  	sample = &stream->clock.samples[stream->clock.head];
> -	sample->dev_stc = get_unaligned_le32(&data[header_size - 6]);
> +	sample->dev_stc = header->stc;
>  	sample->dev_sof = dev_sof;
>  	sample->host_sof = host_sof;
>  	sample->host_ts = ts;
> @@ -756,114 +730,74 @@ done:
>   */
> 
>  static void uvc_video_stats_decode(struct uvc_streaming *stream,
> -		const __u8 *data, int len)
> +				    struct uvc_payload_header *header)
>  {
> -	unsigned int header_size;
> -	bool has_pts = false;
> -	bool has_scr = false;
> -	u16 uninitialized_var(scr_sof);
> -	u32 uninitialized_var(scr_stc);
> -	u32 uninitialized_var(pts);
> -
>  	if (stream->stats.stream.nb_frames == 0 &&
>  	    stream->stats.frame.nb_packets == 0)
>  		ktime_get_ts(&stream->stats.stream.start_ts);
> 
> -	switch (data[1] & (UVC_STREAM_PTS | UVC_STREAM_SCR)) {
> -	case UVC_STREAM_PTS | UVC_STREAM_SCR:
> -		header_size = 12;
> -		has_pts = true;
> -		has_scr = true;
> -		break;
> -	case UVC_STREAM_PTS:
> -		header_size = 6;
> -		has_pts = true;
> -		break;
> -	case UVC_STREAM_SCR:
> -		header_size = 8;
> -		has_scr = true;
> -		break;
> -	default:
> -		header_size = 2;
> -		break;
> -	}
> -
> -	/* Check for invalid headers. */
> -	if (len < header_size || data[0] < header_size) {
> -		stream->stats.frame.nb_invalid++;
> -		return;
> -	}
> -
> -	/* Extract the timestamps. */
> -	if (has_pts)
> -		pts = get_unaligned_le32(&data[2]);
> -
> -	if (has_scr) {
> -		scr_stc = get_unaligned_le32(&data[header_size - 6]);
> -		scr_sof = get_unaligned_le16(&data[header_size - 2]);
> -	}
> -
>  	/* Is PTS constant through the whole frame ? */
> -	if (has_pts && stream->stats.frame.nb_pts) {
> -		if (stream->stats.frame.pts != pts) {
> +	if (header->has_pts && stream->stats.frame.nb_pts) {
> +		if (stream->stats.frame.pts != header->pts) {
>  			stream->stats.frame.nb_pts_diffs++;
>  			stream->stats.frame.last_pts_diff =
>  				stream->stats.frame.nb_packets;
>  		}
>  	}
> 
> -	if (has_pts) {
> +	if (header->has_pts) {
>  		stream->stats.frame.nb_pts++;
> -		stream->stats.frame.pts = pts;
> +		stream->stats.frame.pts = header->pts;
>  	}
> 
>  	/* Do all frames have a PTS in their first non-empty packet, or before
>  	 * their first empty packet ?
>  	 */
>  	if (stream->stats.frame.size == 0) {
> -		if (len > header_size)
> -			stream->stats.frame.has_initial_pts = has_pts;
> -		if (len == header_size && has_pts)
> +		if (header->payload_size > 0)
> +			stream->stats.frame.has_initial_pts = header->has_pts;
> +		if (header->payload_size == 0 && header->has_pts)
>  			stream->stats.frame.has_early_pts = true;
>  	}
> 
>  	/* Do the SCR.STC and SCR.SOF fields vary through the frame ? */
> -	if (has_scr && stream->stats.frame.nb_scr) {
> -		if (stream->stats.frame.scr_stc != scr_stc)
> +	if (header->has_scr && stream->stats.frame.nb_scr) {
> +		if (stream->stats.frame.scr_stc != header->stc)
>  			stream->stats.frame.nb_scr_diffs++;
>  	}
> 
> -	if (has_scr) {
> +	if (header->has_scr) {
>  		/* Expand the SOF counter to 32 bits and store its value. */
>  		if (stream->stats.stream.nb_frames > 0 ||
>  		    stream->stats.frame.nb_scr > 0)
>  			stream->stats.stream.scr_sof_count +=
> -				(scr_sof - stream->stats.stream.scr_sof) % 2048;
> -		stream->stats.stream.scr_sof = scr_sof;
> +				(header->sof - stream->stats.stream.scr_sof)
> +				% 2048;
> +		stream->stats.stream.scr_sof = header->sof;
> 
>  		stream->stats.frame.nb_scr++;
> -		stream->stats.frame.scr_stc = scr_stc;
> -		stream->stats.frame.scr_sof = scr_sof;
> +		stream->stats.frame.scr_stc = header->stc;
> +		stream->stats.frame.scr_sof = header->sof;
> 
> -		if (scr_sof < stream->stats.stream.min_sof)
> -			stream->stats.stream.min_sof = scr_sof;
> -		if (scr_sof > stream->stats.stream.max_sof)
> -			stream->stats.stream.max_sof = scr_sof;
> +		if (header->sof < stream->stats.stream.min_sof)
> +			stream->stats.stream.min_sof = header->sof;
> +		if (header->sof > stream->stats.stream.max_sof)
> +			stream->stats.stream.max_sof = header->sof;
>  	}
> 
>  	/* Record the first non-empty packet number. */
> -	if (stream->stats.frame.size == 0 && len > header_size)
> +	if (stream->stats.frame.size == 0 && header->payload_size > 0)
>  		stream->stats.frame.first_data = stream->stats.frame.nb_packets;
> 
>  	/* Update the frame size. */
> -	stream->stats.frame.size += len - header_size;
> +	stream->stats.frame.size += header->payload_size;
> 
>  	/* Update the packets counters. */
>  	stream->stats.frame.nb_packets++;
> -	if (len > header_size)
> +	if (header->payload_size == 0)

This fixes a bug, could you please split it to a separate patch ? Just turn 
len > header_size into len == header_size in a bugfix patch of its own, before 
this one.

>  		stream->stats.frame.nb_empty++;
> 
> -	if (data[1] & UVC_STREAM_ERR)
> +	if (header->has_err)
>  		stream->stats.frame.nb_errors++;
>  }
> 
> @@ -1006,21 +940,9 @@ static void uvc_video_stats_stop(struct uvc_streaming
> *stream) * uvc_video_decode_end will never be called with a NULL buffer.
>   */
>  static int uvc_video_decode_start(struct uvc_streaming *stream,
> -		struct uvc_buffer *buf, const __u8 *data, int len)
> +		struct uvc_buffer *buf, struct uvc_payload_header *header)
>  {
> -	__u8 fid;
> -
> -	/* Sanity checks:
> -	 * - packet must be at least 2 bytes long
> -	 * - bHeaderLength value must be at least 2 bytes (see above)
> -	 * - bHeaderLength value can't be larger than the packet size.
> -	 */
> -	if (len < 2 || data[0] < 2 || data[0] > len) {
> -		stream->stats.frame.nb_invalid++;
> -		return -EINVAL;
> -	}
> -
> -	fid = data[1] & UVC_STREAM_FID;
> +	u8 fid = header->fid;
> 
>  	/* Increase the sequence number regardless of any buffer states, so
>  	 * that discontinuous sequence numbers always indicate lost frames.
> @@ -1031,8 +953,8 @@ static int uvc_video_decode_start(struct uvc_streaming
> *stream, uvc_video_stats_update(stream);
>  	}
> 
> -	uvc_video_clock_decode(stream, buf, data, len);
> -	uvc_video_stats_decode(stream, data, len);
> +	uvc_video_clock_decode(stream, buf, header);
> +	uvc_video_stats_decode(stream, header);
> 
>  	/* Store the payload FID bit and return immediately when the buffer is
>  	 * NULL.
> @@ -1043,7 +965,7 @@ static int uvc_video_decode_start(struct uvc_streaming
> *stream, }
> 
>  	/* Mark the buffer as bad if the error bit is set. */
> -	if (data[1] & UVC_STREAM_ERR) {
> +	if (header->has_err) {
>  		uvc_trace(UVC_TRACE_FRAME, "Marking buffer as bad (error bit "
>  			  "set).\n");
>  		buf->error = 1;
> @@ -1064,7 +986,7 @@ static int uvc_video_decode_start(struct uvc_streaming
> *stream, uvc_trace(UVC_TRACE_FRAME, "Dropping payload (out of "
>  				"sync).\n");
>  			if ((stream->dev->quirks & UVC_QUIRK_STREAM_NO_FID) &&
> -			    (data[1] & UVC_STREAM_EOF))
> +			    (header->has_eof))
>  				stream->last_fid ^= UVC_STREAM_FID;
>  			return -ENODATA;
>  		}
> @@ -1107,7 +1029,7 @@ static int uvc_video_decode_start(struct uvc_streaming
> *stream,
> 
>  	stream->last_fid = fid;
> 
> -	return data[0];
> +	return 0;
>  }
> 
>  static void uvc_video_decode_data(struct uvc_streaming *stream,
> @@ -1128,18 +1050,20 @@ static void uvc_video_decode_data(struct
> uvc_streaming *stream,
> 
>  	/* Complete the current frame if the buffer size was exceeded. */
>  	if (len > maxlen) {
> -		uvc_trace(UVC_TRACE_FRAME, "Frame complete (overflow).\n");
> +		uvc_trace(UVC_TRACE_FRAME, "Frame complete (overflow) "
> +				"len=%d, buffer size=%d used=%d\n",

The sizes can't be negative, please use %u.

> +				len, buf->length, buf->bytesused);

Do we actually need to print that extra information ? :-) Did you find it 
useful during development ?

>  		buf->state = UVC_BUF_STATE_READY;
>  	}
>  }
> 
>  static void uvc_video_decode_end(struct uvc_streaming *stream,
> -		struct uvc_buffer *buf, const __u8 *data, int len)
> +		struct uvc_buffer *buf, struct uvc_payload_header *header)
>  {
>  	/* Mark the buffer as done if the EOF marker is set. */
> -	if (data[1] & UVC_STREAM_EOF && buf->bytesused != 0) {
> +	if (header->has_eof && buf->bytesused != 0) {
>  		uvc_trace(UVC_TRACE_FRAME, "Frame complete (EOF found).\n");
> -		if (data[0] == len)
> +		if (header->payload_size == 0)
>  			uvc_trace(UVC_TRACE_FRAME, "EOF in empty payload.\n");
>  		buf->state = UVC_BUF_STATE_READY;
>  		if (stream->dev->quirks & UVC_QUIRK_STREAM_NO_FID)
> @@ -1186,6 +1110,75 @@ static int uvc_video_encode_data(struct uvc_streaming
> *stream, return nbytes;
>  }
> 
> +static int uvc_video_parse_header(struct uvc_streaming *stream,
> +		const __u8 *data, int len, struct uvc_payload_header *header)
> +{
> +	int off = 2;

The offset can't be negative, you can thus use an unsigned int type. And I 
don't think there's a need to abbreviate the variable name, you can call it 
offset. The lines below are not that long.

> +
> +	/* Sanity checks:
> +	 * - packet must be at least 2 bytes long
> +	 * - bHeaderLength value must be at least 2 bytes (see above)
> +	 */
> +	if (len < 2 || data[0] < 2)
> +		goto error;
> +
> +	header->length = 2; /* 1 byte of header length + 1 byte of BFH. */
> +
> +	header->has_sli = false;
> +	header->has_eof = data[1] & UVC_STREAM_EOF;
> +	header->has_pts = data[1] & UVC_STREAM_PTS;
> +	header->has_scr = data[1] & UVC_STREAM_SCR;
> +	header->has_err = data[1] & UVC_STREAM_ERR;
> +
> +	if (header->has_pts)
> +		header->length += 4;
> +
> +	if (header->has_scr)
> +		header->length += 6;
> +
> +	if (stream->cur_format->fcc == V4L2_PIX_FMT_VP8) {
> +		/* VP8 payload has 2 additional bytes of BFH. */
> +		header->length += 2;
> +		off += 2;
> +
> +		/* SLI always present for VP8 simulcast (at the end of header),
> +		 * allowed for VP8 non-simulcast.
> +		 */
> +		header->has_sli = data[1] & UVC_STREAM_SLI;
> +		if (header->has_sli)
> +			header->length += 2;
> +	}
> +
> +	/* - bHeaderLength value can't be larger than the packet size. */
> +	if (len < data[0] || data[0] != header->length)

Can you keep the comment and the len < data[0] check above with the other 
sanity checks as in the original code ? The data[0] != header->length check 
obviously needs to stay here.

> +		goto error;
> +
> +	/* PTS 4 bytes, STC 4 bytes, SOF 2 bytes. */
> +	if (header->has_pts) {
> +		header->pts = get_unaligned_le32(&data[off]);
> +		off += 4;
> +	}
> +
> +	if (header->has_scr) {
> +		header->stc = get_unaligned_le32(&data[off]);
> +		off += 4;
> +		header->sof = get_unaligned_le16(&data[off]);
> +		off += 2;
> +	}
> +
> +	if (header->has_sli)
> +		header->sli = get_unaligned_le16(&data[off]);
> +
> +	header->payload_size = len - header->length;
> +	header->fid = data[1] & UVC_STREAM_FID;
> +
> +	return 0;
> +
> +error:
> +	stream->stats.frame.nb_invalid++;
> +	return -EINVAL;
> +}
> +
>  /* ------------------------------------------------------------------------
> * URB handling
>   */
> @@ -1195,9 +1188,11 @@ static int uvc_video_encode_data(struct uvc_streaming
> *stream, */
>  static void uvc_video_decode_isoc(struct urb *urb, struct uvc_streaming
> *stream) {
> +	unsigned int len;
>  	u8 *mem;
>  	int ret, i;
>  	struct uvc_buffer *buf = NULL;
> +	struct uvc_payload_header header;

Could you move this line to the top of the function ? I try to keep variable 
declarations more or less sorted by line size, even though it seems I've 
failed to do so in every location in this driver :-)

> 
>  	for (i = 0; i < urb->number_of_packets; ++i) {
>  		if (urb->iso_frame_desc[i].status < 0) {
> @@ -1209,12 +1204,16 @@ static void uvc_video_decode_isoc(struct urb *urb,
> struct uvc_streaming *stream) continue;
>  		}
> 
> -		/* Decode the payload header. */
>  		mem = urb->transfer_buffer + urb->iso_frame_desc[i].offset;
> +		len = urb->iso_frame_desc[i].actual_length;
> +
> +		ret = uvc_video_parse_header(stream, mem, len, &header);
> +		if (ret < 0)
> +			continue;
> +
>  		buf = uvc_queue_get_first_buf(&stream->queue);
>  		do {
> -			ret = uvc_video_decode_start(stream, buf, mem,
> -				urb->iso_frame_desc[i].actual_length);
> +			ret = uvc_video_decode_start(stream, buf, &header);
>  			if (ret == -EAGAIN)
>  				buf = uvc_queue_next_buffer(&stream->queue,
>  							    buf);
> @@ -1224,12 +1223,11 @@ static void uvc_video_decode_isoc(struct urb *urb,
> struct uvc_streaming *stream) continue;
> 
>  		/* Decode the payload data. */
> -		uvc_video_decode_data(stream, buf, mem + ret,
> -			urb->iso_frame_desc[i].actual_length - ret);
> +		uvc_video_decode_data(stream, buf, mem + header.length,
> +			urb->iso_frame_desc[i].actual_length - header.length);
> 
>  		/* Process the header again. */
> -		uvc_video_decode_end(stream, buf, mem,
> -			urb->iso_frame_desc[i].actual_length);
> +		uvc_video_decode_end(stream, buf, &header);
> 
>  		if (buf->state == UVC_BUF_STATE_READY) {
>  			if (buf->length != buf->bytesused &&
> @@ -1246,6 +1244,7 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) {
>  	u8 *mem;
>  	int len, ret;
> +	struct uvc_payload_header header;
>  	struct uvc_buffer *buf;
> 
>  	/*
> @@ -1259,6 +1258,10 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) len = urb->actual_length;
>  	stream->bulk.payload_size += len;
> 
> +	ret = uvc_video_parse_header(stream, mem, len, &header);
> +	if (ret < 0)
> +		return;
> +
>  	buf = uvc_queue_get_first_buf(&stream->queue);
> 
>  	/* If the URB is the first of its payload, decode and save the
> @@ -1266,7 +1269,7 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) */
>  	if (stream->bulk.header_size == 0 && !stream->bulk.skip_payload) {
>  		do {
> -			ret = uvc_video_decode_start(stream, buf, mem, len);
> +			ret = uvc_video_decode_start(stream, buf, &header);
>  			if (ret == -EAGAIN)
>  				buf = uvc_queue_next_buffer(&stream->queue,
>  							    buf);
> @@ -1276,11 +1279,11 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) if (ret < 0 || buf == NULL) {
>  			stream->bulk.skip_payload = 1;
>  		} else {
> -			memcpy(stream->bulk.header, mem, ret);
> -			stream->bulk.header_size = ret;
> +			memcpy(stream->bulk.header, mem, header.length);
> +			stream->bulk.header_size = header.length;
> 
> -			mem += ret;
> -			len -= ret;
> +			mem += header.length;
> +			len -= header.length;
>  		}
>  	}
> 
> @@ -1299,8 +1302,7 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) if (urb->actual_length <
> urb->transfer_buffer_length ||
>  	    stream->bulk.payload_size >= stream->bulk.max_payload_size) {
>  		if (!stream->bulk.skip_payload && buf != NULL) {
> -			uvc_video_decode_end(stream, buf, stream->bulk.header,
> -				stream->bulk.payload_size);
> +			uvc_video_decode_end(stream, buf, &header);
>  			if (buf->state == UVC_BUF_STATE_READY)
>  				buf = uvc_queue_next_buffer(&stream->queue,
>  							    buf);
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index bca8715..b355b2c 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -453,6 +453,27 @@ struct uvc_stats_stream {
>  	unsigned int max_sof;		/* Maximum STC.SOF value */
>  };
> 
> +struct uvc_payload_header {
> +	bool has_eof;
> +
> +	bool has_pts;
> +	u32 pts;
> +
> +	bool has_scr;
> +	u16 sof;
> +	u32 stc;
> +
> +	bool has_sli;
> +	u16 sli;
> +
> +	u8 fid;
> +
> +	bool has_err;
> +
> +	int length;
> +	int payload_size;
> +};
> +
>  struct uvc_streaming {
>  	struct list_head list;
>  	struct uvc_device *dev;
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 14/19] v4l: Add v4l2_buffer flags for VP8-specific special frames.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:13 Pawel Osciak wrote:
> Add bits for previous, golden and altref frame types.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  include/uapi/linux/videodev2.h | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 437f1b0..c011ee0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -687,6 +687,10 @@ struct v4l2_buffer {
>  #define V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN		0x0000
>  #define V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC	0x2000
>  #define V4L2_BUF_FLAG_TIMESTAMP_COPY		0x4000
> +/* VP8 special frames */
> +#define V4L2_BUF_FLAG_PREV_FRAME		0x10000  /* VP8 prev frame */
> +#define V4L2_BUF_FLAG_GOLDEN_FRAME		0x20000  /* VP8 golden frame */
> +#define V4L2_BUF_FLAG_ALTREF_FRAME		0x40000  /* VP8 altref frame */

This required documentation in Documentation/DocBook/media/ :-)

>  /**
>   * struct v4l2_exportbuffer - export of video buffer as DMABUF file
> descriptor
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 13/19] uvcvideo: Unify UVC payload header parsing.

[Laurent Pinchart]

Hi Pawel,

One more comment.

On Friday 30 August 2013 11:17:12 Pawel Osciak wrote:
> Create a separate function for parsing UVC payload headers and extract code
> from other functions into it. Store the parsed values in a header struct.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_video.c | 270 +++++++++++++++++------------------
>  drivers/media/usb/uvc/uvcvideo.h  |  21 +++
>  2 files changed, 157 insertions(+), 134 deletions(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 2f9a5fa..59f57a2 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c

[snip]

> @@ -1246,6 +1244,7 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) {
>  	u8 *mem;
>  	int len, ret;
> +	struct uvc_payload_header header;
>  	struct uvc_buffer *buf;
> 
>  	/*
> @@ -1259,6 +1258,10 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) len = urb->actual_length;
>  	stream->bulk.payload_size += len;
> 
> +	ret = uvc_video_parse_header(stream, mem, len, &header);
> +	if (ret < 0)
> +		return;
> +

This won't work. UVC transmits a single header per payload and splits the 
payload to multiple URBs in the case of bulk transfers. Only the first URB 
will thus have a header. You should parse the payload inside the if { ... } in 
the next hunk, and save the header (or at least the needed fields) in stream-
>bulk for the uvc_video_decode_end() call.

>  	buf = uvc_queue_get_first_buf(&stream->queue);
> 
>  	/* If the URB is the first of its payload, decode and save the
> @@ -1266,7 +1269,7 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) */
>  	if (stream->bulk.header_size == 0 && !stream->bulk.skip_payload) {
>  		do {
> -			ret = uvc_video_decode_start(stream, buf, mem, len);
> +			ret = uvc_video_decode_start(stream, buf, &header);
>  			if (ret == -EAGAIN)
>  				buf = uvc_queue_next_buffer(&stream->queue,
>  							    buf);
> @@ -1276,11 +1279,11 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) if (ret < 0 || buf == NULL) {
>  			stream->bulk.skip_payload = 1;
>  		} else {
> -			memcpy(stream->bulk.header, mem, ret);
> -			stream->bulk.header_size = ret;
> +			memcpy(stream->bulk.header, mem, header.length);
> +			stream->bulk.header_size = header.length;
> 
> -			mem += ret;
> -			len -= ret;
> +			mem += header.length;
> +			len -= header.length;
>  		}
>  	}
> 
> @@ -1299,8 +1302,7 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) if (urb->actual_length <
> urb->transfer_buffer_length ||
>  	    stream->bulk.payload_size >= stream->bulk.max_payload_size) {
>  		if (!stream->bulk.skip_payload && buf != NULL) {
> -			uvc_video_decode_end(stream, buf, stream->bulk.header,
> -				stream->bulk.payload_size);
> +			uvc_video_decode_end(stream, buf, &header);
>  			if (buf->state == UVC_BUF_STATE_READY)
>  				buf = uvc_queue_next_buffer(&stream->queue,
>  							    buf);

-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 15/19] uvcvideo: Add support for VP8 special frame flags.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:14 Pawel Osciak wrote:

Commit message missing :-)

> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_video.c | 18 +++++++++++++++++-
>  drivers/media/usb/uvc/uvcvideo.h  | 10 ++++++++++
>  2 files changed, 27 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/usb/uvc/uvc_video.c
> b/drivers/media/usb/uvc/uvc_video.c index 59f57a2..0291817 100644
> --- a/drivers/media/usb/uvc/uvc_video.c
> +++ b/drivers/media/usb/uvc/uvc_video.c
> @@ -1136,6 +1136,8 @@ static int uvc_video_parse_header(struct uvc_streaming
> *stream, if (header->has_scr)
>  		header->length += 6;
> 
> +	header->buf_flags = 0;
> +

I would have moved this initialization line above right after header->length, 
to keep it before the conditional checks, but that's up to you.

>  	if (stream->cur_format->fcc == V4L2_PIX_FMT_VP8) {
>  		/* VP8 payload has 2 additional bytes of BFH. */
>  		header->length += 2;
> @@ -1147,6 +1149,16 @@ static int uvc_video_parse_header(struct
> uvc_streaming *stream, header->has_sli = data[1] & UVC_STREAM_SLI;
>  		if (header->has_sli)
>  			header->length += 2;
> +
> +		/* Codec-specific flags for v4l2_buffer. */
> +		header->buf_flags |= (data[1] & UVC_STREAM_STI) ?
> +					V4L2_BUF_FLAG_KEYFRAME : 0;

Is a keyframe the same thing as an intraframe ?

> +		header->buf_flags |= (data[2] & UVC_STREAM_VP8_PRF) ?
> +					V4L2_BUF_FLAG_PREV_FRAME : 0;
> +		header->buf_flags |= (data[2] & UVC_STREAM_VP8_ARF) ?
> +					V4L2_BUF_FLAG_ALTREF_FRAME : 0;
> +		header->buf_flags |= (data[2] & UVC_STREAM_VP8_GRF) ?
> +					V4L2_BUF_FLAG_GOLDEN_FRAME : 0;
>  	}
> 
>  	/* - bHeaderLength value can't be larger than the packet size. */
> @@ -1222,6 +1234,8 @@ static void uvc_video_decode_isoc(struct urb *urb,
> struct uvc_streaming *stream) if (ret < 0)
>  			continue;
> 
> +		buf->buf.v4l2_buf.flags |= header.buf_flags;

What about moving this line to the end of uvc_video_decode_data(), right 
before it returns successfully ?

> +
>  		/* Decode the payload data. */
>  		uvc_video_decode_data(stream, buf, mem + header.length,
>  			urb->iso_frame_desc[i].actual_length - header.length);
> @@ -1293,8 +1307,10 @@ static void uvc_video_decode_bulk(struct urb *urb,
> struct uvc_streaming *stream) */
> 
>  	/* Process video data. */
> -	if (!stream->bulk.skip_payload && buf != NULL)
> +	if (!stream->bulk.skip_payload && buf != NULL) {
>  		uvc_video_decode_data(stream, buf, mem, len);
> +		buf->buf.v4l2_buf.flags |= header.buf_flags;
> +	}
> 
>  	/* Detect the payload end by a URB smaller than the maximum size (or
>  	 * a payload size equal to the maximum) and process the header again.
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h index b355b2c..fb21459 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -145,6 +145,14 @@
>  #define UVC_FMT_FLAG_COMPRESSED		0x00000001
>  #define UVC_FMT_FLAG_STREAM		0x00000002
> 
> +/* v4l2_buffer codec flags */
> +#define UVC_V4L2_BUFFER_CODEC_FLAGS	(V4L2_BUF_FLAG_KEYFRAME | \
> +					 V4L2_BUF_FLAG_PFRAME | \
> +					 V4L2_BUF_FLAG_BFRAME | \
> +					 V4L2_BUF_FLAG_PREV_FRAME | \
> +					 V4L2_BUF_FLAG_GOLDEN_FRAME | \
> +					 V4L2_BUF_FLAG_ALTREF_FRAME)

This isn't used in this patch, could you move it to the patch that needs it ?

> +
>  /* ------------------------------------------------------------------------
> * Structures.
>   */
> @@ -472,6 +480,8 @@ struct uvc_payload_header {
> 
>  	int length;
>  	int payload_size;
> +
> +	__u32 buf_flags; /* v4l2_buffer flags */
>  };
> 
>  struct uvc_streaming {
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 16/19] v4l: Add encoding camera controls.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:15 Pawel Osciak wrote:
> Add defines for controls found in UVC 1.5 encoding cameras.

In addition to the comments already sent by Hans, Kamil and Sylwester, I'll 
just point out that documentation is needed.

> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 29 +++++++++++++++++++++++++++++
>  include/uapi/linux/v4l2-controls.h   | 31 +++++++++++++++++++++++++++++++
>  2 files changed, 60 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c index c3f0803..0b3a632 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -781,6 +781,35 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_AUTO_FOCUS_STATUS:	return "Auto Focus, Status";
>  	case V4L2_CID_AUTO_FOCUS_RANGE:		return "Auto Focus, Range";
> 
> +	case V4L2_CID_ENCODER_MIN_FRAME_INTERVAL: return "Encoder, min. frame
> interval";
> +	case V4L2_CID_ENCODER_RATE_CONTROL_MODE: return "Encoder, rate control
> mode";
> +	case V4L2_CID_ENCODER_AVERAGE_BITRATE:	return "Encoder, average
> bitrate";
> +	case V4L2_CID_ENCODER_CPB_SIZE:		return "Encoder, CPB size";
> +	case V4L2_CID_ENCODER_PEAK_BIT_RATE:	return "Encoder, peak bit rate";
> +	case V4L2_CID_ENCODER_QP_PARAM_I:	return "Encoder, QP param for I 
frames";
> +	case V4L2_CID_ENCODER_QP_PARAM_P:	return "Encoder, QP param for P
> frames";
> +	case V4L2_CID_ENCODER_QP_PARAM_BG:	return "Encoder, QP param for B/G
> frames";
> +	case V4L2_CID_ENCODER_NUM_GDR_FRAMES:	return "Encoder, number of GDR
> frames";
> +	case V4L2_CID_ENCODER_LTR_BUFFER_CONTROL: return "Encoder, LTR buffer
> control";
> +	case V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE: return "Encoder, LTR buffer
> trust mode";
> +	case V4L2_CID_ENCODER_LTR_PICTURE_POSITION: return "Encoder, LTR picture
> position";
> +	case V4L2_CID_ENCODER_LTR_PICTURE_MODE:	return "Encoder, LTR picture
> mode";
> +	case V4L2_CID_ENCODER_LTR_VALIDATION:	return "Encoder, LTR
> validation";
> +	case V4L2_CID_ENCODER_MIN_QP:		return "Encoder, minimum QP param";
> +	case V4L2_CID_ENCODER_MAX_QP:		return "Encoder, maximum QP param";
> +	case V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL: return "Encoder, sync frame
> interval";
> +	case V4L2_CID_ENCODER_ERROR_RESILIENCY:	return "Encoder, error
> resiliency";
> +	case V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE: return "Encoder, temporal
> layer enable";
> +
> +	case V4L2_CID_ENCODER_VP8_SLICE_MODE:	return "Encoder, VP8 slice
> mode";
> +	case V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE: return "Encoder, VP8 sync
> frame type";
> +	case V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME: return "Encoder,
> VP8, DCT partitions per frame";
> +
> +	case V4L2_CID_ENCODER_H264_PROFILE_TOOLSET: return "Encoder, H.264
> profile and toolset";
> +	case V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT: return "Encoder, H.264 level
> IDC limit";
> +	case V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE: return "Encoder, H.264 SEI
> payload type";
> +	case V4L2_CID_ENCODER_H264_LAYER_PRIORITY: return "Encoder, H.264 layer
> priority";
> +
>  	/* FM Radio Modulator control */
>  	/* Keep the order of the 'case's the same as in videodev2.h! */
>  	case V4L2_CID_FM_TX_CLASS:		return "FM Radio Modulator Controls";
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h index 083bb5a..ef3a30d 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -729,6 +729,37 @@ enum v4l2_auto_focus_range {
>  	V4L2_AUTO_FOCUS_RANGE_INFINITY		= 3,
>  };
> 
> +/* Controls found in UVC 1.5 encoding cameras */
> +#define V4L2_CID_ENCODER_MIN_FRAME_INTERVAL	
(V4L2_CID_CAMERA_CLASS_BASE+32)
> +#define V4L2_CID_ENCODER_RATE_CONTROL_MODE	(V4L2_CID_CAMERA_CLASS_BASE+33)
> +#define V4L2_CID_ENCODER_AVERAGE_BITRATE	(V4L2_CID_CAMERA_CLASS_BASE+34)
> +#define V4L2_CID_ENCODER_CPB_SIZE		(V4L2_CID_CAMERA_CLASS_BASE+35)
> +#define V4L2_CID_ENCODER_PEAK_BIT_RATE		(V4L2_CID_CAMERA_CLASS_BASE+36)
> +#define V4L2_CID_ENCODER_QP_PARAM_I		(V4L2_CID_CAMERA_CLASS_BASE+37)
> +#define V4L2_CID_ENCODER_QP_PARAM_P		(V4L2_CID_CAMERA_CLASS_BASE+38)
> +#define V4L2_CID_ENCODER_QP_PARAM_BG		(V4L2_CID_CAMERA_CLASS_BASE+39)
> +#define V4L2_CID_ENCODER_NUM_GDR_FRAMES		
(V4L2_CID_CAMERA_CLASS_BASE+40)
> +#define
> V4L2_CID_ENCODER_LTR_BUFFER_CONTROL	(V4L2_CID_CAMERA_CLASS_BASE+41)
> +#define
> V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE	(V4L2_CID_CAMERA_CLASS_BASE+42)
> +#define
> V4L2_CID_ENCODER_LTR_PICTURE_POSITION	(V4L2_CID_CAMERA_CLASS_BASE+43)
> +#define V4L2_CID_ENCODER_LTR_PICTURE_MODE	(V4L2_CID_CAMERA_CLASS_BASE+44)
> +#define V4L2_CID_ENCODER_LTR_VALIDATION		
(V4L2_CID_CAMERA_CLASS_BASE+45)
> +#define V4L2_CID_ENCODER_MIN_QP			(V4L2_CID_CAMERA_CLASS_BASE+46) 
+#define
> V4L2_CID_ENCODER_MAX_QP			(V4L2_CID_CAMERA_CLASS_BASE+47) +#define
> V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL	(V4L2_CID_CAMERA_CLASS_BASE+48)
> +#define V4L2_CID_ENCODER_ERROR_RESILIENCY	(V4L2_CID_CAMERA_CLASS_BASE+49)
> +#define
> V4L2_CID_ENCODER_TEMPORAL_LAYER_ENABLE	(V4L2_CID_CAMERA_CLASS_BASE+50) +
> +/* VP8-specific controls */
> +#define V4L2_CID_ENCODER_VP8_SLICE_MODE		
(V4L2_CID_CAMERA_CLASS_BASE+51)
> +#define V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME
> (V4L2_CID_CAMERA_CLASS_BASE+52) +#define
> V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE	(V4L2_CID_CAMERA_CLASS_BASE+53) +
> +/* H.264-specific controls */
> +#define
> V4L2_CID_ENCODER_H264_PROFILE_TOOLSET	(V4L2_CID_CAMERA_CLASS_BASE+54)
> +#define
> V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT	(V4L2_CID_CAMERA_CLASS_BASE+55)
> +#define
> V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE	(V4L2_CID_CAMERA_CLASS_BASE+56)
> +#define
> V4L2_CID_ENCODER_H264_LAYER_PRIORITY	(V4L2_CID_CAMERA_CLASS_BASE+57)
> 
>  /* FM Modulator class control IDs */
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 00/19] UVC 1.5 VP8 support for uvcvideo

[Laurent Pinchart]

Hi Paulo,

On Thursday 10 October 2013 11:27:37 Paulo Assis wrote:
> Hi,
> just want t know the current state on this series.
> 
> I'm currently adding h264 stream preview support to guvcview.
> It's already working fine on uvc 1.1 cameras like the BCC950

Great ! Is the code already available ?

> but for uvc 1.5 devices like the c930e it could really use some driver
> support.

I'm nearly done reviewing the patches (although 19/19 is a huge beast that 
will take a bit of time to tame :-)). Getting the code to mainline will still 
need some time, but it seems to be going in the right direction. We'll get 
hardware compression support for UVC 1.5 in the driver one way or another, it 
won't be left to userspace to implement (you will of course have to adapt 
existing applications to use the new features, but it should hopefully not be 
too complex).

> 2013/8/30 Pawel Osciak <posciak@chromium.org>:
> > Hello everyone,
> > 
> > This series adds support for UVC 1.5 and VP8 encoding cameras to the
> > uvcvideo driver. The official specification for the new standard can be
> > found here: http://www.usb.org/developers/devclass_docs.
> > 
> > The main change in 1.5 is support for encoding cameras. Those cameras
> > contain additional UVC entities, called Encoding Units, with their own
> > set of controls governing encode parameters. Typical encoding cameras
> > (see examples in class spec) expose two USB Video Streaming Interfaces
> > (VSIs): one for raw stream formats and one for encoded streams.
> > Typically, both get their source stream from a single sensor, producing
> > raw and encoded versions of the video feed simultaneously.
> > Encoding Units may also support the so-called "simulcast" formats, which
> > allow additional sub-streams, or layers, used to achieve temporal
> > scalability. The spec allows up to 4 simulcast layers. Those layers are
> > encoded in the same format, but encoding parameters, such as resolution,
> > bitrate, etc., may, depending on the camera capabilities, be changed
> > independently for each layer, and their streaming state may also be
> > controlled independently as well. The layers are streamed from the same
> > USB VSI, and the information which layer a frame belongs to is contained
> > in its payload header.
> > 
> > In V4L2 API, a separate video node is created for each VSI: one for raw
> > formats VSI and another for the encoded formats VSI. Both can operate
> > completely independently from each other. In addition, if the Encoding
> > Unit supports simulcast, one V4L2 node is created for each stream layer
> > instead, and each can be controlled independently, including
> > streamon/streamoff state, setting resolution and controls. Once a
> > simulcast format is successfully set for one of the simulcast video nodes
> > however, it cannot be changed, unless all connected nodes are idle, i.e.
> > they are not streaming and their buffers are freed.
> > 
> > The userspace can discover if a set of nodes belongs to one encoding unit
> > by traversing media controller topology of the camera.
> > 
> > 
> > I will be gradually posting documentation changes for new features after
> > initial rounds of reviews. This is a relatively major change to the UVC
> > driver and although I tried to keep the existing logic for UVC <1.5
> > cameras intact as much as possible, I would very much appreciate it if
> > these patches could get some testing from you as well, on your own
> > devices/systems.
> > 
> > Thanks,
> > Pawel Osciak
> > 
> > Pawel Osciak (19):
> >       uvcvideo: Add UVC query tracing.
> >       uvcvideo: Return 0 when setting probe control succeeds.
> >       uvcvideo: Add support for multiple chains with common roots.
> >       uvcvideo: Create separate debugfs entries for each streaming
> >       interface.
> >       uvcvideo: Add support for UVC1.5 P&C control.
> >       uvcvideo: Recognize UVC 1.5 encoding units.
> >       uvcvideo: Unify error reporting during format descriptor parsing.
> >       uvcvideo: Add UVC1.5 VP8 format support.
> >       uvcvideo: Reorganize uvc_{get,set}_le_value.
> >       uvcvideo: Support UVC 1.5 runtime control property.
> >       uvcvideo: Support V4L2_CTRL_TYPE_BITMASK controls.
> >       uvcvideo: Reorganize next buffer handling.
> >       uvcvideo: Unify UVC payload header parsing.
> >       v4l: Add v4l2_buffer flags for VP8-specific special frames.
> >       uvcvideo: Add support for VP8 special frame flags.
> >       v4l: Add encoding camera controls.
> >       uvcvideo: Add UVC 1.5 Encoding Unit controls.
> >       v4l: Add V4L2_PIX_FMT_VP8_SIMULCAST format.
> >       uvcvideo: Add support for UVC 1.5 VP8 simulcast.
> >  
> >  drivers/media/usb/uvc/uvc_ctrl.c     | 960 +++++++++++++++++++++++++++---
> >  drivers/media/usb/uvc/uvc_debugfs.c  |   3 +-
> >  drivers/media/usb/uvc/uvc_driver.c   | 604 ++++++++++++++--------
> >  drivers/media/usb/uvc/uvc_entity.c   | 129 ++++-
> >  drivers/media/usb/uvc/uvc_isight.c   |  12 +-
> >  drivers/media/usb/uvc/uvc_queue.c    |  25 +-
> >  drivers/media/usb/uvc/uvc_v4l2.c     | 284 +++++++++--
> >  drivers/media/usb/uvc/uvc_video.c    | 704 ++++++++++++++++---------
> >  drivers/media/usb/uvc/uvcvideo.h     | 214 +++++++-
> >  drivers/media/v4l2-core/v4l2-ctrls.c |  29 ++
> >  include/uapi/linux/usb/video.h       |  45 ++
> >  include/uapi/linux/v4l2-controls.h   |  31 ++
> >  include/uapi/linux/videodev2.h       |   8 +
> >  13 files changed, 2421 insertions(+), 627 deletions(-)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 17/19] uvcvideo: Add UVC 1.5 Encoding Unit controls.

[Laurent Pinchart]

Hi Pawel,

Thank you for the patch.

On Friday 30 August 2013 11:17:16 Pawel Osciak wrote:
> These controls allow modifying encoding parameters.
> 
> Signed-off-by: Pawel Osciak <posciak@chromium.org>
> ---
>  drivers/media/usb/uvc/uvc_ctrl.c | 445 ++++++++++++++++++++++++++++++++++++
>  include/uapi/linux/usb/video.h   |  23 ++
>  2 files changed, 468 insertions(+)
> 
> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c
> b/drivers/media/usb/uvc/uvc_ctrl.c index a0493d6..cd02c99 100644
> --- a/drivers/media/usb/uvc/uvc_ctrl.c
> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
> @@ -351,6 +351,167 @@ static struct uvc_control_info uvc_ctrls[] = {
> 
>  				| UVC_CTRL_FLAG_RESTORE
>  				| UVC_CTRL_FLAG_AUTO_UPDATE,
> 
>  	},
> +	/*
> +	 * All EU controls are marked as AUTO_UPDATE, because many are, and also
> +	 * we can't cache all of them as they are stream/layer dependent, which
> +	 * would be too slow/too much to cache.
> +	 */
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_PROFILE_TOOLSET_CONTROL,
> +		.index		= 1,
> +		.size		= 6,

Doesn't UVC 1.5 document this control size as 5 bytes ?

> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_MIN_FRAME_INTERVAL_CONTROL,
> +		.index		= 3,
> +		.size		= 4,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SLICE_MODE_CONTROL,
> +		.index		= 4,
> +		.size		= 4,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_RATE_CONTROL_MODE_CONTROL,
> +		.index		= 5,
> +		.size		= 1,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_AVERAGE_BITRATE_CONTROL,
> +		.index		= 6,
> +		.size		= 4,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR
> +				| UVC_CTRL_FLAG_GET_RANGE
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_CPB_SIZE_CONTROL,
> +		.index		= 7,
> +		.size		= 4,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR
> +				| UVC_CTRL_FLAG_GET_RANGE
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_PEAK_BIT_RATE_CONTROL,
> +		.index		= 8,
> +		.size		= 4,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR
> +				| UVC_CTRL_FLAG_GET_RANGE
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
> +		.index		= 9,
> +		.size		= 6,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,

According to the spec this one supports GET_RES as well, so you can use 
GET_RANGE.

> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
> +		.index		= 10,
> +		.size		= 4,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_BUFFER_CONTROL,
> +		.index		= 11,
> +		.size		= 2,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_MAX | UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_PICTURE_CONTROL,
> +		.index		= 12,
> +		.size		= 2,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_VALIDATION_CONTROL,
> +		.index		= 13,
> +		.size		= 2,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LEVEL_IDC_LIMIT_CONTROL,
> +		.index		= 14,
> +		.size		= 1,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_MIN | UVC_CTRL_FLAG_GET_MAX
> +				| UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SEI_PAYLOADTYPE_CONTROL,
> +		.index		= 15,
> +		.size		= 8,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_MAX | UVC_CTRL_FLAG_GET_DEF
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_QP_RANGE_CONTROL,
> +		.index		= 16,
> +		.size		= 2,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR
> +				| UVC_CTRL_FLAG_GET_RANGE
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_PRIORITY_CONTROL,
> +		.index		= 17,
> +		.size		= 1,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,
> +	},
> +	{
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_ERROR_RESILIENCY_CONTROL,
> +		.index		= 19,
> +		.size		= 2,
> +		.flags		= UVC_CTRL_FLAG_SET_CUR | UVC_CTRL_FLAG_GET_CUR
> +				| UVC_CTRL_FLAG_GET_DEF | UVC_CTRL_FLAG_GET_RES
> +				| UVC_CTRL_FLAG_AUTO_UPDATE,

The spec doesn't mention GET_LEN and GET_INFO as supported requests for this 
control. Please please please tell me it's a mistake in the documentation...

> +	},
>  };
> 
>  static struct uvc_menu_info power_line_frequency_controls[] = {
> @@ -366,6 +527,32 @@ static struct uvc_menu_info exposure_auto_controls[] =
> { { 8, "Aperture Priority Mode" },
>  };
> 
> +static struct uvc_menu_info rate_control_mode_controls[] = {
> +	{ 1, "VBR" },
> +	{ 2, "CBR" },
> +	{ 3, "Constant QP" },
> +	{ 4, "GVBR" },
> +	{ 5, "VBRN" },
> +	{ 6, "GVBRN" },
> +};
> +
> +static struct uvc_menu_info encoder_vp8_sync_frame_type_controls[] = {
> +	{ 0, "Reset" },
> +	{ 1, "Generate Intra Frame" },
> +	{ 2, "GDR" },
> +	{ 3, "GDR and Update Golden" },
> +	{ 4, "GDR and Update Alt" },
> +	{ 5, "GDR and Update Golden + Alt" },
> +	{ 6, "Update Golden" },
> +	{ 7, "Update Alt" },
> +	{ 8, "Update Golden + Alt" },
> +};
> +
> +static struct uvc_menu_info encoder_vp8_slice_mode_controls[] = {
> +	{ 0, "No Partitioning" },
> +	{ 1, "DCT Partitions Per Frame" },
> +};
> +
>  static __s32 uvc_ctrl_get_zoom(struct uvc_control_mapping *mapping,
>  	__u8 query, const __u8 *data)
>  {
> @@ -686,6 +873,264 @@ static struct uvc_control_mapping uvc_ctrl_mappings[]
> = { .v4l2_type	= V4L2_CTRL_TYPE_BOOLEAN,
>  		.data_type	= UVC_CTRL_DATA_TYPE_BOOLEAN,
>  	},
> +	/* Encoder controls. */
> +	{
> +		.id		= V4L2_CID_ENCODER_H264_PROFILE_TOOLSET,
> +		.name		= "Encoder, H.264 profile/toolset",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_PROFILE_TOOLSET_CONTROL,
> +		.size		= 40,

The get_le and set_le helpers only support up to 32 bits :-/

> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,

This UVC control is split in 3 fields. I don't think it would be a good idea 
to expose that as a single V4L2 control. We should either have 3 V4L2 controls 
(or actually 2, as wConstrainedToolset is reserved), or only expose this 
through the XU API.

> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_MIN_FRAME_INTERVAL,
> +		.name		= "Encoder, minimum frame interval",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_MIN_FRAME_INTERVAL_CONTROL,
> +		.size		= 32,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_VP8_SLICE_MODE,
> +		.name		= "Encoder, VP8 slice mode",

How nice of them, redefining the meaning of the control for VP8... :-)

> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SLICE_MODE_CONTROL,
> +		.size		= 16,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_MENU,
> +		.data_type	= UVC_CTRL_DATA_TYPE_ENUM,
> +		.menu_info	= encoder_vp8_slice_mode_controls,
> +		.menu_count	= ARRAY_SIZE(encoder_vp8_slice_mode_controls),
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_VP8_DCT_PARTS_PER_FRAME,
> +		.name		= "Encoder, VP8 DCT partns/frame",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SLICE_MODE_CONTROL,
> +		.size		= 16,
> +		.offset		= 16,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_RATE_CONTROL_MODE,
> +		.name		= "Encoder, rate control mode",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_RATE_CONTROL_MODE_CONTROL,
> +		.size		= 4,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_MENU,
> +		.data_type	= UVC_CTRL_DATA_TYPE_ENUM,
> +		.menu_info	= rate_control_mode_controls,
> +		.menu_count	= ARRAY_SIZE(rate_control_mode_controls),
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_AVERAGE_BITRATE,
> +		.name		= "Encoder, average bitrate",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_AVERAGE_BITRATE_CONTROL,
> +		.size		= 32,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_CPB_SIZE,
> +		.name		= "Encoder, CPB size",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_CPB_SIZE_CONTROL,
> +		.size		= 32,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_PEAK_BIT_RATE,
> +		.name		= "Encoder, peak bit rate",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_PEAK_BIT_RATE_CONTROL,
> +		.size		= 32,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_QP_PARAM_I,
> +		.name		= "Encoder, QP param, I frames",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
> +		.size		= 16,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_QP_PARAM_P,
> +		.name		= "Encoder, QP param, P frames",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
> +		.size		= 16,
> +		.offset		= 16,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_QP_PARAM_BG,
> +		.name		= "Encoder, QP param, B/G frames",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_QUANTIZATION_PARAMS_CONTROL,
> +		.size		= 16,
> +		.offset		= 32,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_VP8_SYNC_FRAME_TYPE,
> +		.name		= "Encoder, VP8 sync frame type",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
> +		.size		= 8,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_MENU,
> +		.data_type	= UVC_CTRL_DATA_TYPE_ENUM,
> +		.menu_info	= encoder_vp8_sync_frame_type_controls,
> +		.menu_count	=
> +			ARRAY_SIZE(encoder_vp8_sync_frame_type_controls),
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_SYNC_FRAME_INTERVAL,
> +		.name		= "Encoder, sync frame interval",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
> +		.size		= 16,
> +		.offset		= 8,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_NUM_GDR_FRAMES,
> +		.name		= "Encoder, number of GDR frames",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SYNC_REF_FRAME_CONTROL,
> +		.size		= 8,
> +		.offset		= 24,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_LTR_BUFFER_CONTROL,
> +		.name		= "Encoder, LTR buffer control",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_BUFFER_CONTROL,
> +		.size		= 8,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_LTR_BUFFER_TRUST_MODE,
> +		.name		= "Encoder, LTR buffer trust mode",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_BUFFER_CONTROL,
> +		.size		= 8,
> +		.offset		= 8,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_LTR_PICTURE_POSITION,
> +		.name		= "Encoder, LTR picture position",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_PICTURE_CONTROL,
> +		.size		= 8,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_LTR_PICTURE_MODE,
> +		.name		= "Encoder, LTR picture mode",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_PICTURE_CONTROL,
> +		.size		= 8,
> +		.offset		= 8,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_LTR_VALIDATION,
> +		.name		= "Encoder, LTR validation",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LTR_VALIDATION_CONTROL,
> +		.size		= 16,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_BITMASK,
> +		.data_type	= UVC_CTRL_DATA_TYPE_BITMASK,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_H264_LEVEL_IDC_LIMIT,
> +		.name		= "Encoder, H.264 level IDC limit",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_LEVEL_IDC_LIMIT_CONTROL,
> +		.size		= 8,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_H264_SEI_PAYLOAD_TYPE,
> +		.name		= "Encoder, H.264 SEI payload type",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_SEI_PAYLOADTYPE_CONTROL,
> +		.size		= 64,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_BITMASK,
> +		.data_type	= UVC_CTRL_DATA_TYPE_BITMASK,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_MIN_QP,
> +		.name		= "Encoder, minimum QP param",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_QP_RANGE_CONTROL,
> +		.size		= 8,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_SIGNED,

Are the QP values signed ?

> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_MAX_QP,
> +		.name		= "Encoder, maximum QP param",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_QP_RANGE_CONTROL,
> +		.size		= 8,
> +		.offset		= 8,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_SIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_H264_LAYER_PRIORITY,
> +		.name		= "Encoder, H.264 layer priority",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_PRIORITY_CONTROL,
> +		.size		= 8,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_INTEGER,
> +		.data_type	= UVC_CTRL_DATA_TYPE_UNSIGNED,
> +	},
> +	{
> +		.id		= V4L2_CID_ENCODER_ERROR_RESILIENCY,
> +		.name		= "Encoder, error resiliency",
> +		.entity		= UVC_GUID_UVC_ENCODING,
> +		.selector	= UVC_EU_ERROR_RESILIENCY_CONTROL,
> +		.size		= 16,
> +		.offset		= 0,
> +		.v4l2_type	= V4L2_CTRL_TYPE_BITMASK,
> +		.data_type	= UVC_CTRL_DATA_TYPE_BITMASK,
> +	},
>  };
> 
>  /* ------------------------------------------------------------------------
> diff --git a/include/uapi/linux/usb/video.h
> b/include/uapi/linux/usb/video.h index e09c50b..fd1d4b0 100644
> --- a/include/uapi/linux/usb/video.h
> +++ b/include/uapi/linux/usb/video.h
> @@ -127,6 +127,29 @@
>  #define UVC_PU_ANALOG_VIDEO_STANDARD_CONTROL		0x11
>  #define UVC_PU_ANALOG_LOCK_STATUS_CONTROL		0x12
> 
> +/* Encoding Unit Control Selectors */
> +#define UVC_EU_CONTROL_UNDEFINED			0x00
> +#define UVC_EU_SELECT_LAYER_CONTROL			0x01
> +#define UVC_EU_PROFILE_TOOLSET_CONTROL			0x02
> +#define UVC_EU_VIDEO_RESOLUTION_CONTROL			0x03
> +#define UVC_EU_MIN_FRAME_INTERVAL_CONTROL		0x04
> +#define UVC_EU_SLICE_MODE_CONTROL			0x05
> +#define UVC_EU_RATE_CONTROL_MODE_CONTROL		0x06
> +#define UVC_EU_AVERAGE_BITRATE_CONTROL			0x07
> +#define UVC_EU_CPB_SIZE_CONTROL				0x08
> +#define UVC_EU_PEAK_BIT_RATE_CONTROL			0x09
> +#define UVC_EU_QUANTIZATION_PARAMS_CONTROL		0x0a
> +#define UVC_EU_SYNC_REF_FRAME_CONTROL			0x0b
> +#define UVC_EU_LTR_BUFFER_CONTROL			0x0c
> +#define UVC_EU_LTR_PICTURE_CONTROL			0x0d
> +#define UVC_EU_LTR_VALIDATION_CONTROL			0x0e
> +#define UVC_EU_LEVEL_IDC_LIMIT_CONTROL			0x0f
> +#define UVC_EU_SEI_PAYLOADTYPE_CONTROL			0x10
> +#define UVC_EU_QP_RANGE_CONTROL				0x11
> +#define UVC_EU_PRIORITY_CONTROL				0x12
> +#define UVC_EU_START_OR_STOP_LAYER_CONTROL		0x13
> +#define UVC_EU_ERROR_RESILIENCY_CONTROL			0x14
> +
>  /* A.9.7. VideoStreaming Interface Control Selectors */
>  #define UVC_VS_CONTROL_UNDEFINED			0x00
>  #define UVC_VS_PROBE_CONTROL				0x01
-- 
Regards,

Laurent Pinchart

Re: [PATCH v1 00/19] UVC 1.5 VP8 support for uvcvideo

[Paulo Assis]

Hi,

2013/11/11 Laurent Pinchart <laurent.pinchart@ideasonboard.com>:
> Hi Paulo,
>
> On Thursday 10 October 2013 11:27:37 Paulo Assis wrote:
>> Hi,
>> just want t know the current state on this series.
>>
>> I'm currently adding h264 stream preview support to guvcview.
>> It's already working fine on uvc 1.1 cameras like the BCC950
>
> Great ! Is the code already available ?

Yes, the latest git version is working very well, even for the c930,
although for this one, h264, can only be muxed in the MJPG container.
>From what I understand Logitech had to remove direct H264 stream
support (no H264 descriptor) due to some issues with a very well known
OS vendor.

>
>> but for uvc 1.5 devices like the c930e it could really use some driver
>> support.
>
> I'm nearly done reviewing the patches (although 19/19 is a huge beast that
> will take a bit of time to tame :-)). Getting the code to mainline will still
> need some time, but it seems to be going in the right direction. We'll get
> hardware compression support for UVC 1.5 in the driver one way or another, it
> won't be left to userspace to implement (you will of course have to adapt
> existing applications to use the new features, but it should hopefully not be
> too complex).

I've test the patches in it's current form and for the c930, at least,
there is a regression that prevents the camera from functioning (it
doesn't create the video node).
The problem happens with patch 3 : "uvcvideo: Add support for multiple
chains with common roots", in function uvc_scan_device there is the
following check:

if (UVC_ENTITY_TYPE(entity) != UVC_VC_EXTENSION_UNIT
   || entity->bNrInPins != 1
   || uvc_entity_by_reference(dev, entity->id, NULL)) {
        uvc_printk(KERN_INFO, "Found an invalid branch "
                             "starting at entity id %d.\n", entity->id);
        return -1;
 }

for unit 12 (the h264 extension control), 'uvc_entity_by_reference'
check fails and this causes the camera initialization to also fail, so
no video node is created.
My very simple fix was to comment the 'uvc_entity_by_reference' check,
this allows the init process to carry on and the video node is
created. The camera will work fine in that case.

I could propose a more appropriate patch, but since this is still
under review I'm not sure if that's OK or if it's better to wait on
other comments.

Regards,
Paulo



>
>> 2013/8/30 Pawel Osciak <posciak@chromium.org>:
>> > Hello everyone,
>> >
>> > This series adds support for UVC 1.5 and VP8 encoding cameras to the
>> > uvcvideo driver. The official specification for the new standard can be
>> > found here: http://www.usb.org/developers/devclass_docs.
>> >
>> > The main change in 1.5 is support for encoding cameras. Those cameras
>> > contain additional UVC entities, called Encoding Units, with their own
>> > set of controls governing encode parameters. Typical encoding cameras
>> > (see examples in class spec) expose two USB Video Streaming Interfaces
>> > (VSIs): one for raw stream formats and one for encoded streams.
>> > Typically, both get their source stream from a single sensor, producing
>> > raw and encoded versions of the video feed simultaneously.
>> > Encoding Units may also support the so-called "simulcast" formats, which
>> > allow additional sub-streams, or layers, used to achieve temporal
>> > scalability. The spec allows up to 4 simulcast layers. Those layers are
>> > encoded in the same format, but encoding parameters, such as resolution,
>> > bitrate, etc., may, depending on the camera capabilities, be changed
>> > independently for each layer, and their streaming state may also be
>> > controlled independently as well. The layers are streamed from the same
>> > USB VSI, and the information which layer a frame belongs to is contained
>> > in its payload header.
>> >
>> > In V4L2 API, a separate video node is created for each VSI: one for raw
>> > formats VSI and another for the encoded formats VSI. Both can operate
>> > completely independently from each other. In addition, if the Encoding
>> > Unit supports simulcast, one V4L2 node is created for each stream layer
>> > instead, and each can be controlled independently, including
>> > streamon/streamoff state, setting resolution and controls. Once a
>> > simulcast format is successfully set for one of the simulcast video nodes
>> > however, it cannot be changed, unless all connected nodes are idle, i.e.
>> > they are not streaming and their buffers are freed.
>> >
>> > The userspace can discover if a set of nodes belongs to one encoding unit
>> > by traversing media controller topology of the camera.
>> >
>> >
>> > I will be gradually posting documentation changes for new features after
>> > initial rounds of reviews. This is a relatively major change to the UVC
>> > driver and although I tried to keep the existing logic for UVC <1.5
>> > cameras intact as much as possible, I would very much appreciate it if
>> > these patches could get some testing from you as well, on your own
>> > devices/systems.
>> >
>> > Thanks,
>> > Pawel Osciak
>> >
>> > Pawel Osciak (19):
>> >       uvcvideo: Add UVC query tracing.
>> >       uvcvideo: Return 0 when setting probe control succeeds.
>> >       uvcvideo: Add support for multiple chains with common roots.
>> >       uvcvideo: Create separate debugfs entries for each streaming
>> >       interface.
>> >       uvcvideo: Add support for UVC1.5 P&C control.
>> >       uvcvideo: Recognize UVC 1.5 encoding units.
>> >       uvcvideo: Unify error reporting during format descriptor parsing.
>> >       uvcvideo: Add UVC1.5 VP8 format support.
>> >       uvcvideo: Reorganize uvc_{get,set}_le_value.
>> >       uvcvideo: Support UVC 1.5 runtime control property.
>> >       uvcvideo: Support V4L2_CTRL_TYPE_BITMASK controls.
>> >       uvcvideo: Reorganize next buffer handling.
>> >       uvcvideo: Unify UVC payload header parsing.
>> >       v4l: Add v4l2_buffer flags for VP8-specific special frames.
>> >       uvcvideo: Add support for VP8 special frame flags.
>> >       v4l: Add encoding camera controls.
>> >       uvcvideo: Add UVC 1.5 Encoding Unit controls.
>> >       v4l: Add V4L2_PIX_FMT_VP8_SIMULCAST format.
>> >       uvcvideo: Add support for UVC 1.5 VP8 simulcast.
>> >
>> >  drivers/media/usb/uvc/uvc_ctrl.c     | 960 +++++++++++++++++++++++++++---
>> >  drivers/media/usb/uvc/uvc_debugfs.c  |   3 +-
>> >  drivers/media/usb/uvc/uvc_driver.c   | 604 ++++++++++++++--------
>> >  drivers/media/usb/uvc/uvc_entity.c   | 129 ++++-
>> >  drivers/media/usb/uvc/uvc_isight.c   |  12 +-
>> >  drivers/media/usb/uvc/uvc_queue.c    |  25 +-
>> >  drivers/media/usb/uvc/uvc_v4l2.c     | 284 +++++++++--
>> >  drivers/media/usb/uvc/uvc_video.c    | 704 ++++++++++++++++---------
>> >  drivers/media/usb/uvc/uvcvideo.h     | 214 +++++++-
>> >  drivers/media/v4l2-core/v4l2-ctrls.c |  29 ++
>> >  include/uapi/linux/usb/video.h       |  45 ++
>> >  include/uapi/linux/v4l2-controls.h   |  31 ++
>> >  include/uapi/linux/videodev2.h       |   8 +
>> >  13 files changed, 2421 insertions(+), 627 deletions(-)
>
> --
> Regards,
>
> Laurent Pinchart
>

Re: [PATCH v1 03/19] uvcvideo: Add support for multiple chains with common roots.

[Paulo Assis]

Hi,
I'm repeating myself, but this seems the proper place to post this.

This patch causes a regression with the logitech c930e.

Please see comments below:


2013/11/10 Laurent Pinchart <laurent.pinchart@ideasonboard.com>:
> Hi Pawel,
>
> Thank you for the patch.
>
> On Friday 30 August 2013 11:17:02 Pawel Osciak wrote:
>> This adds support for pipelines that fork into branches consisting of more
>> than one entity, creating a chain for each fork and putting common entities
>> on all chains that share them.
>>
>> This requires us to share the ctrl_mutex across forked chains. Whenever we
>> discover a shared part of a chain, we assign the pointer to an existing
>> mutex to the sharing chain instead of creating a new one.
>>
>> The "forward scan" is not needed anymore, as after scanning back from OTs,
>> we go over all entities which are not on a path from an OT and accept
>> single-XU branches, adding them to the existing chains.
>>
>> Also extract control locking into __uvc_ctrl_{lock,unlock} functions.
>
> This is one core piece of the UVC 1.5 rework, and I have mixed feelings about
> it.
>
> Adding entities to multiple overlapping chains somehow doesn't feel right.
> What would you think about using a pipeline object instead, which would store
> all entities in the pipeline ?
>
> The driver currently iterates over all entities in a chain for several
> purposes:
>
> - registering streaming terminals as video nodes (uvc_driver.c)
> - registering MC entities (uvc_entity.c)
> - enumerating inputs (uvc_v4l2.c)
> - finding controls and extension units (uvc_ctrl.c)
>
> The first two uses could easily be replaced by iterations over the whole
> pipeline. Input enumeration would probably require a bit of custom code
> anyway, so we would be left with controls.
>
> At first sight it would make sense to expose on a video node only the controls
> that can be reached from that video node moving up the pipeline (relatively to
> the video flow). However, this might break existing applications, as the
> driver currently also includes controls reacheable by forward scans of side
> branches. We thus need to carefully think about what controls to include, and
> we need to take into account output video nodes corresponding to input
> streaming terminals.
>
>> Signed-off-by: Pawel Osciak <posciak@chromium.org>
>> ---
>>  drivers/media/usb/uvc/uvc_ctrl.c   |  70 ++++++++-----
>>  drivers/media/usb/uvc/uvc_driver.c | 210  +++++++++++++++++++--------------
>>  drivers/media/usb/uvc/uvc_entity.c |  15 ++-
>>  drivers/media/usb/uvc/uvc_v4l2.c   |   9 +-
>>  drivers/media/usb/uvc/uvcvideo.h   |  20 +++-
>>  5 files changed, 199 insertions(+), 125 deletions(-)
>>
>> diff --git a/drivers/media/usb/uvc/uvc_ctrl.c
>> b/drivers/media/usb/uvc/uvc_ctrl.c index a2f4501..ba159a4 100644
>> --- a/drivers/media/usb/uvc/uvc_ctrl.c
>> +++ b/drivers/media/usb/uvc/uvc_ctrl.c
>> @@ -841,6 +841,7 @@ static struct uvc_control *uvc_find_control(struct
>> uvc_video_chain *chain, {
>>       struct uvc_control *ctrl = NULL;
>>       struct uvc_entity *entity;
>> +     struct uvc_chain_entry *entry;
>>       int next = v4l2_id & V4L2_CTRL_FLAG_NEXT_CTRL;
>>
>>       *mapping = NULL;
>> @@ -849,7 +850,8 @@ static struct uvc_control *uvc_find_control(struct
>> uvc_video_chain *chain, v4l2_id &= V4L2_CTRL_ID_MASK;
>>
>>       /* Find the control. */
>> -     list_for_each_entry(entity, &chain->entities, chain) {
>> +     list_for_each_entry(entry, &chain->entities, chain_entry) {
>> +             entity = entry->entity;
>>               __uvc_find_control(entity, v4l2_id, mapping, &ctrl, next);
>>               if (ctrl && !next)
>>                       return ctrl;
>> @@ -1048,6 +1050,16 @@ static int __uvc_query_v4l2_ctrl(struct
>> uvc_video_chain *chain, return 0;
>>  }
>>
>> +int __uvc_ctrl_lock(struct uvc_video_chain *chain)
>> +{
>> +     return mutex_lock_interruptible(&chain->pipeline->ctrl_mutex) ?
>> +                                     -ERESTARTSYS : 0;
>> +}
>> +void __uvc_ctrl_unlock(struct uvc_video_chain *chain)
>> +{
>> +     mutex_unlock(&chain->pipeline->ctrl_mutex);
>> +}
>> +
>>  int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
>>       struct v4l2_queryctrl *v4l2_ctrl)
>>  {
>> @@ -1055,9 +1067,9 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
>> struct uvc_control_mapping *mapping;
>>       int ret;
>>
>> -     ret = mutex_lock_interruptible(&chain->ctrl_mutex);
>> +     ret = __uvc_ctrl_lock(chain);
>>       if (ret < 0)
>> -             return -ERESTARTSYS;
>> +             return ret;
>>
>>       ctrl = uvc_find_control(chain, v4l2_ctrl->id, &mapping);
>>       if (ctrl == NULL) {
>> @@ -1067,7 +1079,7 @@ int uvc_query_v4l2_ctrl(struct uvc_video_chain *chain,
>>
>>       ret = __uvc_query_v4l2_ctrl(chain, ctrl, mapping, v4l2_ctrl);
>>  done:
>> -     mutex_unlock(&chain->ctrl_mutex);
>> +     __uvc_ctrl_unlock(chain);
>>       return ret;
>>  }
>>
>> @@ -1094,9 +1106,9 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
>> query_menu->id = id;
>>       query_menu->index = index;
>>
>> -     ret = mutex_lock_interruptible(&chain->ctrl_mutex);
>> +     ret = __uvc_ctrl_lock(chain);
>>       if (ret < 0)
>> -             return -ERESTARTSYS;
>> +             return ret;
>>
>>       ctrl = uvc_find_control(chain, query_menu->id, &mapping);
>>       if (ctrl == NULL || mapping->v4l2_type != V4L2_CTRL_TYPE_MENU) {
>> @@ -1132,7 +1144,7 @@ int uvc_query_v4l2_menu(struct uvc_video_chain *chain,
>> strlcpy(query_menu->name, menu_info->name, sizeof query_menu->name);
>>
>>  done:
>> -     mutex_unlock(&chain->ctrl_mutex);
>> +     __uvc_ctrl_unlock(chain);
>>       return ret;
>>  }
>>
>> @@ -1257,9 +1269,9 @@ static int uvc_ctrl_add_event(struct
>> v4l2_subscribed_event *sev, unsigned elems) struct uvc_control *ctrl;
>>       int ret;
>>
>> -     ret = mutex_lock_interruptible(&handle->chain->ctrl_mutex);
>> +     ret = __uvc_ctrl_lock(handle->chain);
>>       if (ret < 0)
>> -             return -ERESTARTSYS;
>> +             return ret;
>>
>>       ctrl = uvc_find_control(handle->chain, sev->id, &mapping);
>>       if (ctrl == NULL) {
>> @@ -1285,7 +1297,7 @@ static int uvc_ctrl_add_event(struct
>> v4l2_subscribed_event *sev, unsigned elems) }
>>
>>  done:
>> -     mutex_unlock(&handle->chain->ctrl_mutex);
>> +     __uvc_ctrl_unlock(handle->chain);
>>       return ret;
>>  }
>>
>> @@ -1293,9 +1305,9 @@ static void uvc_ctrl_del_event(struct
>> v4l2_subscribed_event *sev) {
>>       struct uvc_fh *handle = container_of(sev->fh, struct uvc_fh, vfh);
>>
>> -     mutex_lock(&handle->chain->ctrl_mutex);
>> +     __uvc_ctrl_lock(handle->chain);
>>       list_del(&sev->node);
>> -     mutex_unlock(&handle->chain->ctrl_mutex);
>> +     __uvc_ctrl_unlock(handle->chain);
>>  }
>>
>>  const struct v4l2_subscribed_event_ops uvc_ctrl_sub_ev_ops = {
>> @@ -1331,7 +1343,7 @@ const struct v4l2_subscribed_event_ops
>> uvc_ctrl_sub_ev_ops = { */
>>  int uvc_ctrl_begin(struct uvc_video_chain *chain)
>>  {
>> -     return mutex_lock_interruptible(&chain->ctrl_mutex) ? -ERESTARTSYS : 0;
>> +     return __uvc_ctrl_lock(chain);
>>  }
>>
>>  static int uvc_ctrl_commit_entity(struct uvc_device *dev,
>> @@ -1390,10 +1402,12 @@ int __uvc_ctrl_commit(struct uvc_fh *handle, int
>> rollback, {
>>       struct uvc_video_chain *chain = handle->chain;
>>       struct uvc_entity *entity;
>> +     struct uvc_chain_entry *entry;
>>       int ret = 0;
>>
>>       /* Find the control. */
>> -     list_for_each_entry(entity, &chain->entities, chain) {
>> +     list_for_each_entry(entry, &chain->entities, chain_entry) {
>> +             entity = entry->entity;
>>               ret = uvc_ctrl_commit_entity(chain->dev, entity, rollback);
>>               if (ret < 0)
>>                       goto done;
>> @@ -1402,7 +1416,7 @@ int __uvc_ctrl_commit(struct uvc_fh *handle, int
>> rollback, if (!rollback)
>>               uvc_ctrl_send_events(handle, xctrls, xctrls_count);
>>  done:
>> -     mutex_unlock(&chain->ctrl_mutex);
>> +     __uvc_ctrl_unlock(chain);
>>       return ret;
>>  }
>>
>> @@ -1667,7 +1681,8 @@ static int uvc_ctrl_init_xu_ctrl(struct uvc_device
>> *dev, int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>>       struct uvc_xu_control_query *xqry)
>>  {
>> -     struct uvc_entity *entity;
>> +     struct uvc_entity *entity = NULL;
>> +     struct uvc_chain_entry *entry;
>>       struct uvc_control *ctrl;
>>       unsigned int i, found = 0;
>>       __u32 reqflags;
>> @@ -1676,13 +1691,14 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>> int ret;
>>
>>       /* Find the extension unit. */
>> -     list_for_each_entry(entity, &chain->entities, chain) {
>> +     list_for_each_entry(entry, &chain->entities, chain_entry) {
>> +             entity = entry->entity;
>>               if (UVC_ENTITY_TYPE(entity) == UVC_VC_EXTENSION_UNIT &&
>>                   entity->id == xqry->unit)
>>                       break;
>>       }
>>
>> -     if (entity->id != xqry->unit) {
>> +     if (!entity || entity->id != xqry->unit) {
>>               uvc_trace(UVC_TRACE_CONTROL, "Extension unit %u not found.\n",
>>                       xqry->unit);
>>               return -ENOENT;
>> @@ -1703,8 +1719,9 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>>               return -ENOENT;
>>       }
>>
>> -     if (mutex_lock_interruptible(&chain->ctrl_mutex))
>> -             return -ERESTARTSYS;
>> +     ret = __uvc_ctrl_lock(chain);
>> +     if (ret < 0)
>> +             return ret;
>>
>>       ret = uvc_ctrl_init_xu_ctrl(chain->dev, ctrl);
>>       if (ret < 0) {
>> @@ -1778,7 +1795,7 @@ int uvc_xu_ctrl_query(struct uvc_video_chain *chain,
>>               ret = -EFAULT;
>>  done:
>>       kfree(data);
>> -     mutex_unlock(&chain->ctrl_mutex);
>> +     __uvc_ctrl_unlock(chain);
>>       return ret;
>>  }
>>
>> @@ -1904,6 +1921,7 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
>> *chain, const struct uvc_control_mapping *mapping)
>>  {
>>       struct uvc_device *dev = chain->dev;
>> +     struct uvc_chain_entry *entry;
>>       struct uvc_control_mapping *map;
>>       struct uvc_entity *entity;
>>       struct uvc_control *ctrl;
>> @@ -1918,8 +1936,9 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
>> *chain, }
>>
>>       /* Search for the matching (GUID/CS) control on the current chain */
>> -     list_for_each_entry(entity, &chain->entities, chain) {
>> +     list_for_each_entry(entry, &chain->entities, chain_entry) {
>>               unsigned int i;
>> +             entity = entry->entity;
>>
>>               if (UVC_ENTITY_TYPE(entity) != UVC_VC_EXTENSION_UNIT ||
>>                   !uvc_entity_match_guid(entity, mapping->entity))
>> @@ -1939,8 +1958,9 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
>> *chain, if (!found)
>>               return -ENOENT;
>>
>> -     if (mutex_lock_interruptible(&chain->ctrl_mutex))
>> -             return -ERESTARTSYS;
>> +     ret = __uvc_ctrl_lock(chain);
>> +     if (ret < 0)
>> +             return ret;
>>
>>       /* Perform delayed initialization of XU controls */
>>       ret = uvc_ctrl_init_xu_ctrl(dev, ctrl);
>> @@ -1974,7 +1994,7 @@ int uvc_ctrl_add_mapping(struct uvc_video_chain
>> *chain, atomic_dec(&dev->nmappings);
>>
>>  done:
>> -     mutex_unlock(&chain->ctrl_mutex);
>> +     __uvc_ctrl_unlock(chain);
>>       return ret;
>>  }
>>
>> diff --git a/drivers/media/usb/uvc/uvc_driver.c
>> b/drivers/media/usb/uvc/uvc_driver.c index 81695d4..d7ff707 100644
>> --- a/drivers/media/usb/uvc/uvc_driver.c
>> +++ b/drivers/media/usb/uvc/uvc_driver.c
>> @@ -1215,6 +1215,36 @@ next_descriptor:
>>   * UVC device scan
>>   */
>>
>> +static int uvc_add_chain_entry(struct uvc_video_chain *chain,
>> +                             struct uvc_entity *entity)
>> +{
>> +     struct uvc_chain_entry *chain_entry;
>> +
>> +     chain_entry = kzalloc(sizeof(struct uvc_chain_entry), GFP_KERNEL);
>> +     if (!chain_entry)
>> +             return -ENOMEM;
>> +
>> +     chain_entry->entity = entity;
>> +     list_add_tail(&chain_entry->chain_entry, &chain->entities);
>> +     if (!entity->chain)
>> +             entity->chain = chain;
>> +
>> +     return 0;
>> +}
>> +
>> +static void uvc_delete_chain(struct uvc_video_chain *chain)
>> +{
>> +     struct list_head *p, *n;
>> +     struct uvc_chain_entry *entry;
>> +
>> +     list_for_each_safe(p, n, &chain->entities) {
>> +             entry = list_entry(p, struct uvc_chain_entry, chain_entry);
>> +             kfree(entry);
>> +     }
>> +
>> +     kfree(chain);
>> +}
>> +
>>  /*
>>   * Scan the UVC descriptors to locate a chain starting at an Output
>> Terminal * and containing the following units:
>> @@ -1320,72 +1350,7 @@ static int uvc_scan_chain_entity(struct
>> uvc_video_chain *chain, return -1;
>>       }
>>
>> -     list_add_tail(&entity->chain, &chain->entities);
>> -     return 0;
>> -}
>> -
>> -static int uvc_scan_chain_forward(struct uvc_video_chain *chain,
>> -     struct uvc_entity *entity, struct uvc_entity *prev)
>> -{
>> -     struct uvc_entity *forward;
>> -     int found;
>> -
>> -     /* Forward scan */
>> -     forward = NULL;
>> -     found = 0;
>> -
>> -     while (1) {
>> -             forward = uvc_entity_by_reference(chain->dev, entity->id,
>> -                     forward);
>> -             if (forward == NULL)
>> -                     break;
>> -             if (forward == prev)
>> -                     continue;
>> -
>> -             switch (UVC_ENTITY_TYPE(forward)) {
>> -             case UVC_VC_EXTENSION_UNIT:
>> -                     if (forward->bNrInPins != 1) {
>> -                             uvc_trace(UVC_TRACE_DESCR, "Extension unit %d "
>> -                                       "has more than 1 input pin.\n",
>> -                                       entity->id);
>> -                             return -EINVAL;
>> -                     }
>> -
>> -                     list_add_tail(&forward->chain, &chain->entities);
>> -                     if (uvc_trace_param & UVC_TRACE_PROBE) {
>> -                             if (!found)
>> -                                     printk(" (->");
>> -
>> -                             printk(" XU %d", forward->id);
>> -                             found = 1;
>> -                     }
>> -                     break;
>> -
>> -             case UVC_OTT_VENDOR_SPECIFIC:
>> -             case UVC_OTT_DISPLAY:
>> -             case UVC_OTT_MEDIA_TRANSPORT_OUTPUT:
>> -             case UVC_TT_STREAMING:
>> -                     if (UVC_ENTITY_IS_ITERM(forward)) {
>> -                             uvc_trace(UVC_TRACE_DESCR, "Unsupported input "
>> -                                     "terminal %u.\n", forward->id);
>> -                             return -EINVAL;
>> -                     }
>> -
>> -                     list_add_tail(&forward->chain, &chain->entities);
>> -                     if (uvc_trace_param & UVC_TRACE_PROBE) {
>> -                             if (!found)
>> -                                     printk(" (->");
>> -
>> -                             printk(" OT %d", forward->id);
>> -                             found = 1;
>> -                     }
>> -                     break;
>> -             }
>> -     }
>> -     if (found)
>> -             printk(")");
>> -
>> -     return 0;
>> +     return uvc_add_chain_entry(chain, entity);
>>  }
>>
>>  static int uvc_scan_chain_backward(struct uvc_video_chain *chain,
>> @@ -1394,6 +1359,7 @@ static int uvc_scan_chain_backward(struct
>> uvc_video_chain *chain, struct uvc_entity *entity = *_entity;
>>       struct uvc_entity *term;
>>       int id = -EINVAL, i;
>> +     int ret;
>>
>>       switch (UVC_ENTITY_TYPE(entity)) {
>>       case UVC_VC_EXTENSION_UNIT:
>> @@ -1425,8 +1391,9 @@ static int uvc_scan_chain_backward(struct
>> uvc_video_chain *chain, if (uvc_trace_param & UVC_TRACE_PROBE)
>>                               printk(" %d", term->id);
>>
>> -                     list_add_tail(&term->chain, &chain->entities);
>> -                     uvc_scan_chain_forward(chain, term, entity);
>> +                     ret = uvc_add_chain_entry(chain, term);
>> +                     if (ret)
>> +                             return ret;
>>               }
>>
>>               if (uvc_trace_param & UVC_TRACE_PROBE)
>> @@ -1473,23 +1440,23 @@ static int uvc_scan_chain(struct uvc_video_chain
>> *chain, prev = NULL;
>>
>>       while (entity != NULL) {
>> -             /* Entity must not be part of an existing chain */
>> -             if (entity->chain.next || entity->chain.prev) {
>> -                     uvc_trace(UVC_TRACE_DESCR, "Found reference to "
>> -                             "entity %d already in chain.\n", entity->id);
>> +             if (entity->chain == chain) {
>> +                     uvc_trace(UVC_TRACE_DESCR, "Found a cycle in the "
>> +                                     "chain");
>>                       return -EINVAL;
>>               }
>>
>> +             /* If this entity is a part of an existing chain, the
>> +              * current chain belongs to the same pipeline.
>> +              */
>> +             if (entity->chain)
>> +                     chain->pipeline = entity->chain->pipeline;
>> +
>>               /* Process entity */
>>               if (uvc_scan_chain_entity(chain, entity) < 0)
>>                       return -EINVAL;
>>
>> -             /* Forward scan */
>> -             if (uvc_scan_chain_forward(chain, entity, prev) < 0)
>> -                     return -EINVAL;
>> -
>>               /* Backward scan */
>> -             prev = entity;
>>               if (uvc_scan_chain_backward(chain, &entity) < 0)
>>                       return -EINVAL;
>>       }
>> @@ -1501,10 +1468,12 @@ static unsigned int uvc_print_terms(struct list_head
>> *terms, u16 dir, char *buffer)
>>  {
>>       struct uvc_entity *term;
>> +     struct uvc_chain_entry *entry;
>>       unsigned int nterms = 0;
>>       char *p = buffer;
>>
>> -     list_for_each_entry(term, terms, chain) {
>> +     list_for_each_entry(entry, terms, chain_entry) {
>> +             term = entry->entity;
>>               if (!UVC_ENTITY_IS_TERM(term) ||
>>                   UVC_TERM_DIRECTION(term) != dir)
>>                       continue;
>> @@ -1541,39 +1510,58 @@ static const char *uvc_print_chain(struct
>> uvc_video_chain *chain) static int uvc_scan_device(struct uvc_device *dev)
>>  {
>>       struct uvc_video_chain *chain;
>> -     struct uvc_entity *term;
>> +     struct uvc_entity *entity, *source;
>> +     int ret;
>>
>> -     list_for_each_entry(term, &dev->entities, list) {
>> -             if (!UVC_ENTITY_IS_OTERM(term))
>> +     list_for_each_entry(entity, &dev->entities, list) {
>> +             if (!UVC_ENTITY_IS_OTERM(entity))
>>                       continue;
>>
>> -             /* If the terminal is already included in a chain, skip it.
>> -              * This can happen for chains that have multiple output
>> -              * terminals, where all output terminals beside the first one
>> -              * will be inserted in the chain in forward scans.
>> +             /* Allow single-unit branches of Output Terminals to reside
>> +              * on the existing chains.
>>                */
>> -             if (term->chain.next || term->chain.prev)
>> -                     continue;
>> +             source = uvc_entity_by_id(dev, entity->baSourceID[0]);
>> +             if (entity == NULL) {
>> +                     uvc_trace(UVC_TRACE_DESCR, "Found reference to "
>> +                             "unknown entity %d.\n", entity->baSourceID[0]);
>> +                     return -EINVAL;
>> +             }
>>
>>               chain = kzalloc(sizeof(*chain), GFP_KERNEL);
>>               if (chain == NULL)
>>                       return -ENOMEM;
>>
>>               INIT_LIST_HEAD(&chain->entities);
>> -             mutex_init(&chain->ctrl_mutex);
>>               chain->dev = dev;
>>               v4l2_prio_init(&chain->prio);
>>
>> -             term->flags |= UVC_ENTITY_FLAG_DEFAULT;
>> +             entity->flags |= UVC_ENTITY_FLAG_DEFAULT;
>>
>> -             if (uvc_scan_chain(chain, term) < 0) {
>> -                     kfree(chain);
>> +             if (uvc_scan_chain(chain, entity) < 0) {
>> +                     uvc_delete_chain(chain);
>>                       continue;
>>               }
>>
>>               uvc_trace(UVC_TRACE_PROBE, "Found a valid video chain (%s).\n",
>>                         uvc_print_chain(chain));
>>
>> +             /*
>> +              * If none of the entities are shared, allocate a new pipeline.
>> +              * Otherwise, the shared pipeline is already set up.
>> +              */
>> +             if (!chain->pipeline) {
>> +                     chain->pipeline = kzalloc(sizeof(*chain->pipeline),
>> +                                               GFP_KERNEL);
>> +                     if (!chain->pipeline) {
>> +                             uvc_delete_chain(chain);
>> +                             return -ENOMEM;
>> +                     }
>> +                     mutex_init(&chain->pipeline->ctrl_mutex);
>> +                     atomic_set(&chain->pipeline->num_chains, 1);
>> +             } else {
>> +                     atomic_inc(&chain->pipeline->num_chains);
>> +             }
>> +
>>               list_add_tail(&chain->list, &dev->chains);
>>       }
>>
>> @@ -1582,6 +1570,38 @@ static int uvc_scan_device(struct uvc_device *dev)
>>               return -1;
>>       }
>>
>> +     /* Find branches with no OTERMs (if any) by looking for entities not
>> +      * on any chain. Accept only branches with a single Extension Unit.
>> +      */
>> +     list_for_each_entry(entity, &dev->entities, list) {
>> +             if (entity->chain)
>> +                     continue;
>> +
>> +             if (UVC_ENTITY_TYPE(entity) != UVC_VC_EXTENSION_UNIT
>> +                     || entity->bNrInPins != 1
>> +                     || uvc_entity_by_reference(dev, entity->id, NULL)) {
>> +                     uvc_printk(KERN_INFO, "Found an invalid branch "
>> +                             "starting at entity id %d.\n", entity->id);
>> +                     return -1;
>> +             }


This fails for the c930e, to be more exact 'uvc_entity_by_reference'
fails for unit 12 (the h264 extension unit).
This prevents the driver from creating the video node, making the
camera unusable.
As a simple test I commented this line ( /* ||
uvc_entity_by_reference(dev, entity->id, NULL) */ ), the video node is
created and the camera works fine.

>> +
>> +             /* Single-unit XU branch. */
>> +             source = uvc_entity_by_id(dev, entity->baSourceID[0]);
>> +             if (source == NULL) {
>> +                     uvc_trace(UVC_TRACE_DESCR, "Found reference to "
>> +                             "unknown entity %d.\n", entity->baSourceID[0]);
>> +                     return -EINVAL;
>> +             }
>> +             if (!source->chain)
>> +                     continue;
>> +
>> +             ret = uvc_add_chain_entry(source->chain, entity);
>> +             if (ret)
>> +                     return ret;
>> +             uvc_trace(UVC_TRACE_DESCR, "XU %d <- (%d)\n",
>> +                             entity->id, source->id);
>> +     }
>> +
>>       return 0;
>>  }
>>
>> @@ -1619,7 +1639,9 @@ static void uvc_delete(struct uvc_device *dev)
>>       list_for_each_safe(p, n, &dev->chains) {
>>               struct uvc_video_chain *chain;
>>               chain = list_entry(p, struct uvc_video_chain, list);
>> -             kfree(chain);
>> +             if (atomic_dec_and_test(&chain->pipeline->num_chains))
>> +                     kfree(chain->pipeline);
>> +             uvc_delete_chain(chain);
>>       }
>>
>>       list_for_each_safe(p, n, &dev->entities) {
>> @@ -1763,9 +1785,11 @@ static int uvc_register_terms(struct uvc_device *dev,
>> {
>>       struct uvc_streaming *stream;
>>       struct uvc_entity *term;
>> +     struct uvc_chain_entry *entry;
>>       int ret;
>>
>> -     list_for_each_entry(term, &chain->entities, chain) {
>> +     list_for_each_entry(entry, &chain->entities, chain_entry) {
>> +             term = entry->entity;
>>               if (UVC_ENTITY_TYPE(term) != UVC_TT_STREAMING)
>>                       continue;
>>
>> diff --git a/drivers/media/usb/uvc/uvc_entity.c
>> b/drivers/media/usb/uvc/uvc_entity.c index dc56a59..657f49a 100644
>> --- a/drivers/media/usb/uvc/uvc_entity.c
>> +++ b/drivers/media/usb/uvc/uvc_entity.c
>> @@ -104,9 +104,14 @@ static int uvc_mc_init_entity(struct uvc_entity
>> *entity) int uvc_mc_register_entities(struct uvc_video_chain *chain)
>>  {
>>       struct uvc_entity *entity;
>> +     struct uvc_chain_entry *entry;
>>       int ret;
>>
>> -     list_for_each_entry(entity, &chain->entities, chain) {
>> +     list_for_each_entry(entry, &chain->entities, chain_entry) {
>> +             entity = entry->entity;
>> +             if (entity->registered)
>> +                     continue;
>> +
>>               ret = uvc_mc_init_entity(entity);
>>               if (ret < 0) {
>>                       uvc_printk(KERN_INFO, "Failed to initialize entity for "
>> @@ -115,13 +120,19 @@ int uvc_mc_register_entities(struct uvc_video_chain
>> *chain) }
>>       }
>>
>> -     list_for_each_entry(entity, &chain->entities, chain) {
>> +     list_for_each_entry(entry, &chain->entities, chain_entry) {
>> +             entity = entry->entity;
>> +             if (entity->registered)
>> +                     continue;
>> +
>>               ret = uvc_mc_register_entity(chain, entity);
>>               if (ret < 0) {
>>                       uvc_printk(KERN_INFO, "Failed to register entity for "
>>                                  "entity %u\n", entity->id);
>>                       return ret;
>>               }
>> +
>> +             entity->registered = true;
>>       }
>>
>>       return 0;
>> diff --git a/drivers/media/usb/uvc/uvc_v4l2.c
>> b/drivers/media/usb/uvc/uvc_v4l2.c index 3afff92..a899159 100644
>> --- a/drivers/media/usb/uvc/uvc_v4l2.c
>> +++ b/drivers/media/usb/uvc/uvc_v4l2.c
>> @@ -713,6 +713,7 @@ static long uvc_v4l2_do_ioctl(struct file *file,
>> unsigned int cmd, void *arg) const struct uvc_entity *selector =
>> chain->selector;
>>               struct v4l2_input *input = arg;
>>               struct uvc_entity *iterm = NULL;
>> +             struct uvc_chain_entry *entry;
>>               u32 index = input->index;
>>               int pin = 0;
>>
>> @@ -720,14 +721,18 @@ static long uvc_v4l2_do_ioctl(struct file *file,
>> unsigned int cmd, void *arg) (chain->dev->quirks &
>> UVC_QUIRK_IGNORE_SELECTOR_UNIT)) {
>>                       if (index != 0)
>>                               return -EINVAL;
>> -                     list_for_each_entry(iterm, &chain->entities, chain) {
>> +                     list_for_each_entry(entry, &chain->entities,
>> +                                             chain_entry) {
>> +                             iterm = entry->entity;
>>                               if (UVC_ENTITY_IS_ITERM(iterm))
>>                                       break;
>>                       }
>>                       pin = iterm->id;
>>               } else if (index < selector->bNrInPins) {
>>                       pin = selector->baSourceID[index];
>> -                     list_for_each_entry(iterm, &chain->entities, chain) {
>> +                     list_for_each_entry(entry, &chain->entities,
>> +                                             chain_entry) {
>> +                             iterm = entry->entity;
>>                               if (!UVC_ENTITY_IS_ITERM(iterm))
>>                                       continue;
>>                               if (iterm->id == pin)
>> diff --git a/drivers/media/usb/uvc/uvcvideo.h
>> b/drivers/media/usb/uvc/uvcvideo.h index 75e0153..731b378 100644
>> --- a/drivers/media/usb/uvc/uvcvideo.h
>> +++ b/drivers/media/usb/uvc/uvcvideo.h
>> @@ -229,8 +229,8 @@ struct uvc_format_desc {
>>
>>  struct uvc_entity {
>>       struct list_head list;          /* Entity as part of a UVC device. */
>> -     struct list_head chain;         /* Entity as part of a video device
>> -                                      * chain. */
>> +     struct uvc_video_chain *chain;  /* Entity as a part of a video device
>> +                                        chain. */
>>       unsigned int flags;
>>
>>       __u8 id;
>> @@ -243,6 +243,7 @@ struct uvc_entity {
>>       unsigned int num_pads;
>>       unsigned int num_links;
>>       struct media_pad *pads;
>> +     bool registered;                /* True if already registered with MC */
>>
>>       union {
>>               struct {
>> @@ -289,6 +290,12 @@ struct uvc_entity {
>>       struct uvc_control *controls;
>>  };
>>
>> +struct uvc_chain_entry {
>> +     struct list_head chain_entry;
>> +     struct uvc_entity *entity;
>> +     struct uvc_video_chain *chain;
>> +};
>> +
>>  struct uvc_frame {
>>       __u8  bFrameIndex;
>>       __u8  bmCapabilities;
>> @@ -366,6 +373,12 @@ struct uvc_video_queue {
>>       struct list_head irqqueue;
>>  };
>>
>> +struct uvc_video_pipeline {
>> +     struct mutex ctrl_mutex;                /* Protects controls in all
>> +                                                chains of this pipeline */
>> +     atomic_t num_chains;
>> +};
>> +
>>  struct uvc_video_chain {
>>       struct uvc_device *dev;
>>       struct list_head list;
>> @@ -374,7 +387,8 @@ struct uvc_video_chain {
>>       struct uvc_entity *processing;          /* Processing unit */
>>       struct uvc_entity *selector;            /* Selector unit */
>>
>> -     struct mutex ctrl_mutex;                /* Protects ctrl.info */
>> +     struct uvc_video_pipeline *pipeline;    /* Pipeline this chain
>> +                                                belongs to */
>>
>>       struct v4l2_prio_state prio;            /* V4L2 priority state */
>>       u32 caps;                               /* V4L2 chain-wide caps */
> --
> Regards,
>
> Laurent Pinchart
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Regards,
Paulo