[RFCv2 00/17] Request API, take three

[RFCv2 00/17] Request API, take three

[Alexandre Courbot]

This is a quickly-put together revision that includes and uses Hans' work to
use v4l2_ctrl_handler as the request state holder for V4L2 devices. Although
minor fixes have also been applied, there are still a few comments from the
previous revision that are left unaddressed. I wanted to give Hans something
to play with before he forgets what he had in mind for controls. ;)

Changelog since v1:
* Integrate Hans control framework patches so S_EXT_CTRLS and G_EXT_CTRLS now
  work with requests
* Only allow one buffer at a time for a given request in the buffer queue
* Applied comments related to documentation and document control ioctls
* Minor small fixes

I have also updated the very basic program that demonstrates the use of the
request API on vim2m:

https://gist.github.com/Gnurou/dbc3776ed97ea7d4ce6041ea15eb0438

It does not do much, but gives a practical idea of how requests should be used.

Alexandre Courbot (9):
  media: add request API core and UAPI
  media: videobuf2: add support for requests
  media: vb2: add support for requests in QBUF ioctl
  v4l2: add request API support
  videodev2.h: add request_fd field to v4l2_ext_controls
  v4l2-ctrls: support requests in EXT_CTRLS ioctls
  v4l2: document the request API interface
  media: vim2m: add media device
  media: vim2m: add request support

Hans Verkuil (7):
  videodev2.h: Add request_fd field to v4l2_buffer
  v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev
  v4l2-ctrls: prepare internal structs for request API
  v4l2-ctrls: add core request API
  v4l2-ctrls: use ref in helper instead of ctrl
  v4l2-ctrls: support g/s_ext_ctrls for requests
  v4l2-ctrls: add v4l2_ctrl_request_setup

Laurent Pinchart (1):
  media: Document the media request API

 Documentation/media/uapi/mediactl/media-funcs.rst  |   1 +
 .../media/uapi/mediactl/media-ioc-request-cmd.rst  | 141 ++++++++++
 Documentation/media/uapi/v4l/buffer.rst            |  10 +-
 Documentation/media/uapi/v4l/common.rst            |   1 +
 Documentation/media/uapi/v4l/request-api.rst       | 236 ++++++++++++++++
 .../media/uapi/v4l/vidioc-g-ext-ctrls.rst          |  16 +-
 Documentation/media/uapi/v4l/vidioc-qbuf.rst       |  21 ++
 drivers/media/Makefile                             |   3 +-
 drivers/media/dvb-frontends/rtl2832_sdr.c          |   5 +-
 drivers/media/media-device.c                       |   7 +
 drivers/media/media-request-mgr.c                  | 105 +++++++
 drivers/media/media-request.c                      | 311 +++++++++++++++++++++
 drivers/media/pci/bt8xx/bttv-driver.c              |   2 +-
 drivers/media/pci/cx23885/cx23885-417.c            |   2 +-
 drivers/media/pci/cx88/cx88-blackbird.c            |   2 +-
 drivers/media/pci/cx88/cx88-video.c                |   2 +-
 drivers/media/pci/saa7134/saa7134-empress.c        |   4 +-
 drivers/media/pci/saa7134/saa7134-video.c          |   2 +-
 drivers/media/platform/exynos4-is/fimc-capture.c   |   2 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c        |   3 +-
 drivers/media/platform/rcar_drif.c                 |   2 +-
 drivers/media/platform/soc_camera/soc_camera.c     |   3 +-
 drivers/media/platform/vim2m.c                     |  79 ++++++
 drivers/media/platform/vivid/vivid-ctrls.c         |  42 +--
 drivers/media/usb/cpia2/cpia2_v4l.c                |   2 +-
 drivers/media/usb/cx231xx/cx231xx-417.c            |   2 +-
 drivers/media/usb/cx231xx/cx231xx-video.c          |   4 +-
 drivers/media/usb/msi2500/msi2500.c                |   2 +-
 drivers/media/usb/tm6000/tm6000-video.c            |   2 +-
 drivers/media/v4l2-core/Makefile                   |   2 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c      |   7 +-
 drivers/media/v4l2-core/v4l2-ctrls.c               | 238 ++++++++++++++--
 drivers/media/v4l2-core/v4l2-device.c              |   3 +-
 drivers/media/v4l2-core/v4l2-ioctl.c               | 125 ++++++++-
 drivers/media/v4l2-core/v4l2-request.c             |  54 ++++
 drivers/media/v4l2-core/videobuf2-core.c           | 133 ++++++++-
 drivers/media/v4l2-core/videobuf2-v4l2.c           |  31 +-
 drivers/staging/media/imx/imx-media-dev.c          |   2 +-
 drivers/staging/media/imx/imx-media-fim.c          |   2 +-
 include/media/media-device.h                       |   3 +
 include/media/media-entity.h                       |   9 +
 include/media/media-request-mgr.h                  |  73 +++++
 include/media/media-request.h                      | 186 ++++++++++++
 include/media/v4l2-ctrls.h                         |  17 +-
 include/media/v4l2-request.h                       |  34 +++
 include/media/videobuf2-core.h                     |  15 +-
 include/media/videobuf2-v4l2.h                     |   2 +
 include/uapi/linux/media.h                         |  10 +
 include/uapi/linux/videodev2.h                     |   6 +-
 49 files changed, 1881 insertions(+), 85 deletions(-)
 create mode 100644 Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
 create mode 100644 Documentation/media/uapi/v4l/request-api.rst
 create mode 100644 drivers/media/media-request-mgr.c
 create mode 100644 drivers/media/media-request.c
 create mode 100644 drivers/media/v4l2-core/v4l2-request.c
 create mode 100644 include/media/media-request-mgr.h
 create mode 100644 include/media/media-request.h
 create mode 100644 include/media/v4l2-request.h

-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 01/17] media: add request API core and UAPI

[Alexandre Courbot]

The request API provides a way to group buffers and device parameters
into units of work to be queued and executed. This patch introduces the
UAPI and core framework.

This patch is based on the previous work by Laurent Pinchart. The core
has changed considerably, but the UAPI is mostly untouched.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/Makefile               |   3 +-
 drivers/media/media-device.c         |   7 +
 drivers/media/media-request-mgr.c    | 105 ++++++++++++
 drivers/media/media-request.c        | 311 +++++++++++++++++++++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c |   2 +-
 include/media/media-device.h         |   3 +
 include/media/media-entity.h         |   9 +
 include/media/media-request-mgr.h    |  73 ++++++++
 include/media/media-request.h        | 186 +++++++++++++++++++++
 include/uapi/linux/media.h           |  10 ++
 10 files changed, 707 insertions(+), 2 deletions(-)
 create mode 100644 drivers/media/media-request-mgr.c
 create mode 100644 drivers/media/media-request.c
 create mode 100644 include/media/media-request-mgr.h
 create mode 100644 include/media/media-request.h

diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index 594b462ddf0e..06c43ddb52ea 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -3,7 +3,8 @@
 # Makefile for the kernel multimedia device drivers.
 #
 
-media-objs	:= media-device.o media-devnode.o media-entity.o
+media-objs	:= media-device.o media-devnode.o media-entity.o \
+		   media-request.o media-request-mgr.o
 
 #
 # I2C drivers should come before other drivers, otherwise they'll fail
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index e79f72b8b858..024ee81a8334 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -32,6 +32,8 @@
 #include <media/media-device.h>
 #include <media/media-devnode.h>
 #include <media/media-entity.h>
+#include <media/media-request.h>
+#include <media/media-request-mgr.h>
 
 #ifdef CONFIG_MEDIA_CONTROLLER
 
@@ -407,6 +409,7 @@ static const struct media_ioctl_info ioctl_info[] = {
 	MEDIA_IOC(ENUM_LINKS, media_device_enum_links, MEDIA_IOC_FL_GRAPH_MUTEX),
 	MEDIA_IOC(SETUP_LINK, media_device_setup_link, MEDIA_IOC_FL_GRAPH_MUTEX),
 	MEDIA_IOC(G_TOPOLOGY, media_device_get_topology, MEDIA_IOC_FL_GRAPH_MUTEX),
+	MEDIA_IOC(REQUEST_CMD, media_device_request_cmd, 0),
 };
 
 static long media_device_ioctl(struct file *filp, unsigned int cmd,
@@ -688,6 +691,10 @@ EXPORT_SYMBOL_GPL(media_device_init);
 
 void media_device_cleanup(struct media_device *mdev)
 {
+	if (mdev->req_mgr) {
+		media_request_mgr_free(mdev->req_mgr);
+		mdev->req_mgr = NULL;
+	}
 	ida_destroy(&mdev->entity_internal_idx);
 	mdev->entity_internal_idx_max = 0;
 	media_graph_walk_cleanup(&mdev->pm_count_walk);
diff --git a/drivers/media/media-request-mgr.c b/drivers/media/media-request-mgr.c
new file mode 100644
index 000000000000..686e877a884b
--- /dev/null
+++ b/drivers/media/media-request-mgr.c
@@ -0,0 +1,105 @@
+/*
+ * Generic request manager implementation.
+ *
+ * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/device.h>
+#include <linux/slab.h>
+
+#include <media/media-device.h>
+#include <media/media-request.h>
+#include <media/media-request-mgr.h>
+
+static struct media_request *
+media_request_alloc(struct media_request_mgr *mgr)
+{
+	struct media_request *req;
+
+	req = kzalloc(sizeof(*req), GFP_KERNEL);
+	if (!req)
+		return ERR_PTR(-ENOMEM);
+
+	req->mgr = mgr;
+	req->state = MEDIA_REQUEST_STATE_IDLE;
+	kref_init(&req->kref);
+	INIT_LIST_HEAD(&req->data);
+	init_waitqueue_head(&req->complete_wait);
+	ATOMIC_INIT_NOTIFIER_HEAD(&req->submit_notif);
+	mutex_init(&req->lock);
+
+	mutex_lock(&mgr->mutex);
+	req->id = ++mgr->req_id;
+	list_add_tail(&req->list, &mgr->requests);
+	mutex_unlock(&mgr->mutex);
+
+	return req;
+}
+
+static void media_request_free(struct media_request *req)
+{
+	struct media_request_mgr *mgr = req->mgr;
+	struct media_request_entity_data *data, *next;
+
+	mutex_lock(&mgr->mutex);
+	list_del(&req->list);
+	mutex_unlock(&mgr->mutex);
+
+	list_for_each_entry_safe(data, next, &req->data, list) {
+		list_del(&data->list);
+		data->entity->req_ops->data_free(data);
+	}
+
+	kfree(req);
+}
+
+void media_request_mgr_free(struct media_request_mgr *mgr)
+{
+	struct media_device *mdev = mgr->mdev;
+
+	/* Just a sanity check - we should have no remaining requests */
+	while (!list_empty(&mgr->requests)) {
+		struct media_request *req;
+
+		req = list_first_entry(&mgr->requests, typeof(*req), list);
+		dev_warn(mdev->dev,
+			"%s: request %u still referenced, deleting forcibly!\n",
+			__func__, req->id);
+		mgr->ops->req_free(req);
+	}
+
+	kfree(mgr);
+}
+EXPORT_SYMBOL_GPL(media_request_mgr_free);
+
+static const struct media_request_mgr_ops request_mgr_generic_ops = {
+	.req_alloc = media_request_alloc,
+	.req_free = media_request_free,
+};
+
+struct media_request_mgr *
+media_request_mgr_alloc(struct media_device *mdev)
+{
+	struct media_request_mgr *mgr;
+
+	mgr = kzalloc(sizeof(*mgr), GFP_KERNEL);
+	if (!mgr)
+		return ERR_PTR(-ENOMEM);
+
+	mgr->mdev = mdev;
+	mutex_init(&mgr->mutex);
+	INIT_LIST_HEAD(&mgr->requests);
+	mgr->ops = &request_mgr_generic_ops;
+
+	return mgr;
+}
+EXPORT_SYMBOL_GPL(media_request_mgr_alloc);
diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
new file mode 100644
index 000000000000..30a23235b019
--- /dev/null
+++ b/drivers/media/media-request.c
@@ -0,0 +1,311 @@
+/*
+ * Request base implementation
+ *
+ * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/media.h>
+#include <linux/fs.h>
+#include <linux/file.h>
+#include <linux/slab.h>
+#include <linux/list.h>
+
+#include <media/media-device.h>
+#include <media/media-request.h>
+#include <media/media-request-mgr.h>
+
+const struct file_operations request_fops;
+
+struct media_request *media_request_get(struct media_request *req)
+{
+	kref_get(&req->kref);
+	return req;
+}
+EXPORT_SYMBOL_GPL(media_request_get);
+
+struct media_request *
+media_request_get_from_fd(int fd)
+{
+	struct file *f;
+	struct media_request *req;
+
+	f = fget(fd);
+	if (!f)
+		return NULL;
+
+	/* Not a request FD? */
+	if (f->f_op != &request_fops) {
+		fput(f);
+		return NULL;
+	}
+
+	req = f->private_data;
+	media_request_get(req);
+	fput(f);
+
+	return req;
+}
+EXPORT_SYMBOL_GPL(media_request_get_from_fd);
+
+static void media_request_release(struct kref *kref)
+{
+	struct media_request *req =
+		container_of(kref, typeof(*req), kref);
+
+	req->mgr->ops->req_free(req);
+}
+
+void media_request_put(struct media_request *req)
+{
+	if (WARN_ON(req == NULL))
+		return;
+
+	kref_put(&req->kref, media_request_release);
+}
+EXPORT_SYMBOL_GPL(media_request_put);
+
+struct media_request_entity_data *
+media_request_get_entity_data(struct media_request *req,
+			      struct media_entity *entity, void *fh)
+{
+	struct media_request_entity_data *data;
+
+	mutex_lock(&req->lock);
+
+	/* First look whether we already have entity data */
+	list_for_each_entry(data, &req->data, list) {
+		if (data->entity == entity) {
+			/*
+			 * If so, then the fh must match otherwise this means
+			 * we are trying to set the same entity through
+			 * different handles
+			 */
+			if (data->fh != fh)
+				data = ERR_PTR(-EINVAL);
+			goto done;
+		}
+	}
+
+	/* No entity data found, let's create it */
+	data = entity->req_ops->data_alloc(entity, fh);
+	if (IS_ERR(data))
+		goto done;
+	data->fh = fh;
+	data->entity = entity;
+	list_add_tail(&data->list, &req->data);
+
+done:
+	mutex_unlock(&req->lock);
+	return data;
+}
+EXPORT_SYMBOL_GPL(media_request_get_entity_data);
+
+static const char * const media_request_states[] __maybe_unused = {
+	"IDLE",
+	"SUBMITTED",
+	"COMPLETED",
+	"DELETED",
+};
+
+static const char *media_request_state(enum media_request_state state)
+{
+	return state < ARRAY_SIZE(media_request_states) ?
+		media_request_states[state] : "INVALID";
+}
+
+static int media_device_request_close(struct inode *inode, struct file *filp)
+{
+	struct media_request *req = filp->private_data;
+
+	if (req == NULL)
+		return 0;
+
+	media_request_put(req);
+
+	return 0;
+}
+
+static unsigned int media_request_poll(struct file *file, poll_table *wait)
+{
+	struct media_request *req = file->private_data;
+
+	poll_wait(file, &req->complete_wait, wait);
+
+	if (req->state == MEDIA_REQUEST_STATE_COMPLETED)
+		return POLLIN | POLLRDNORM;
+
+	return 0;
+}
+
+const struct file_operations request_fops = {
+	.owner = THIS_MODULE,
+	.poll = media_request_poll,
+	.release = media_device_request_close,
+};
+
+void media_request_complete(struct media_request *req)
+{
+	struct media_request_mgr *mgr = req->mgr;
+	struct media_device *mdev = mgr->mdev;
+
+	mutex_lock(&req->lock);
+
+	if (WARN_ON(req->state != MEDIA_REQUEST_STATE_SUBMITTED)) {
+		dev_dbg(mdev->dev, "%s: can't complete %u, state %s\n",
+			__func__, req->id, media_request_state(req->state));
+		mutex_unlock(&req->lock);
+		return;
+	}
+
+	req->state = MEDIA_REQUEST_STATE_COMPLETED;
+
+	if (mgr->ops->req_complete)
+		mgr->ops->req_complete(req);
+
+	wake_up_interruptible(&req->complete_wait);
+
+	mutex_unlock(&req->lock);
+
+	/* Release the reference acquired when we submitted the request */
+	media_request_put(req);
+}
+EXPORT_SYMBOL_GPL(media_request_complete);
+
+/*
+ * Process the MEDIA_REQ_CMD_ALLOC command
+ */
+static int media_request_cmd_alloc(struct media_request_mgr *mgr,
+				   struct media_request_cmd *cmd)
+{
+	struct media_request *req;
+	int fd;
+
+	req = mgr->ops->req_alloc(mgr);
+	if (!req)
+		return -ENOMEM;
+
+	fd = anon_inode_getfd("media_request", &request_fops, req, O_CLOEXEC);
+	if (fd < 0)
+		return fd;
+
+	cmd->fd = fd;
+
+	return 0;
+}
+
+/*
+ * Process the MEDIA_REQ_CMD_SUBMIT command
+ */
+static int media_request_cmd_submit(struct media_request *req)
+{
+	mutex_lock(&req->lock);
+
+	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
+		dev_dbg(req->mgr->mdev->dev,
+			"%s: unable to submit request in state %s\n",
+			__func__, media_request_state(req->state));
+		mutex_unlock(&req->lock);
+		return -EINVAL;
+	}
+
+	if (atomic_read(&req->buf_cpt) == 0) {
+		dev_dbg(req->mgr->mdev->dev,
+			"%s: request has no buffers!\n", __func__);
+		mutex_unlock(&req->lock);
+		return -EINVAL;
+	}
+
+	req->state = MEDIA_REQUEST_STATE_SUBMITTED;
+
+	/* Hold a reference to the request until it is completed */
+	media_request_get(req);
+
+	mutex_unlock(&req->lock);
+
+	atomic_notifier_call_chain(&req->submit_notif, req->state, req);
+
+	return 0;
+}
+
+static int media_request_cmd_reinit(struct media_request *req)
+{
+	struct media_request_entity_data *data, *next;
+
+	mutex_lock(&req->lock);
+
+	if (req->state == MEDIA_REQUEST_STATE_SUBMITTED) {
+		dev_dbg(req->mgr->mdev->dev,
+			"%s: unable to reinit submitted request\n", __func__);
+		mutex_unlock(&req->lock);
+		return -EINVAL;
+	}
+
+	/* delete all entity data */
+	list_for_each_entry_safe(data, next, &req->data, list) {
+		list_del(&data->list);
+		data->entity->req_ops->data_free(data);
+	}
+
+	/* reinitialize request to idle state */
+	req->state = MEDIA_REQUEST_STATE_IDLE;
+	atomic_set(&req->buf_cpt, 0);
+
+	mutex_unlock(&req->lock);
+
+	return 0;
+}
+
+long media_device_request_cmd(struct media_device *mdev,
+			      struct media_request_cmd *cmd)
+{
+	struct media_request *req = NULL;
+	int ret;
+
+	if (!mdev->req_mgr)
+		return -ENOTTY;
+
+	if (cmd->cmd != MEDIA_REQ_CMD_ALLOC) {
+		req = media_request_get_from_fd(cmd->fd);
+		if (IS_ERR(req))
+			return PTR_ERR(req);
+
+		/* requests must belong to this media device's manager */
+		if (req->mgr != mdev->req_mgr) {
+			media_request_put(req);
+			return -EINVAL;
+		}
+	}
+
+	switch (cmd->cmd) {
+	case MEDIA_REQ_CMD_ALLOC:
+		ret = media_request_cmd_alloc(mdev->req_mgr, cmd);
+		break;
+
+	case MEDIA_REQ_CMD_SUBMIT:
+		ret = media_request_cmd_submit(req);
+		break;
+
+	case MEDIA_REQ_CMD_REINIT:
+		ret = media_request_cmd_reinit(req);
+		break;
+
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	if (req)
+		media_request_put(req);
+
+	return ret;
+}
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b60a6b0841d1..ec4ecd5aa8bf 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -867,7 +867,7 @@ static int check_ext_ctrls(struct v4l2_ext_controls *c, int allow_priv)
 	__u32 i;
 
 	/* zero the reserved fields */
-	c->reserved[0] = c->reserved[1] = 0;
+	c->reserved[0] = 0;
 	for (i = 0; i < c->count; i++)
 		c->controls[i].reserved2[0] = 0;
 
diff --git a/include/media/media-device.h b/include/media/media-device.h
index bcc6ec434f1f..f2d471b8a53f 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -27,6 +27,7 @@
 
 struct ida;
 struct device;
+struct media_request_mgr;
 
 /**
  * struct media_entity_notify - Media Entity Notify
@@ -158,6 +159,8 @@ struct media_device {
 	void (*disable_source)(struct media_entity *entity);
 
 	const struct media_device_ops *ops;
+
+	struct media_request_mgr *req_mgr;
 };
 
 /* We don't need to include pci.h or usb.h here */
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 222d379960b7..0aa7549eb154 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -170,6 +170,8 @@ struct media_pad {
 	unsigned long flags;
 };
 
+struct media_request;
+struct v4l2_ext_controls;
 /**
  * struct media_entity_operations - Media entity operations
  * @get_fwnode_pad:	Return the pad number based on a fwnode endpoint or
@@ -195,6 +197,11 @@ struct media_entity_operations {
 	int (*link_validate)(struct media_link *link);
 };
 
+struct media_entity_request_ops {
+	struct media_request_entity_data *(*data_alloc)(struct media_entity *entity, void *fh);
+	void (*data_free)(struct media_request_entity_data *data);
+};
+
 /**
  * enum media_entity_type - Media entity type
  *
@@ -276,6 +283,8 @@ struct media_entity {
 
 	const struct media_entity_operations *ops;
 
+	const struct media_entity_request_ops *req_ops;
+
 	int stream_count;
 	int use_count;
 
diff --git a/include/media/media-request-mgr.h b/include/media/media-request-mgr.h
new file mode 100644
index 000000000000..a3161fa2add0
--- /dev/null
+++ b/include/media/media-request-mgr.h
@@ -0,0 +1,73 @@
+/*
+ * Generic request manager.
+ *
+ * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MEDIA_REQUEST_MGR_H
+#define _MEDIA_REQUEST_MGR_H
+
+#include <linux/mutex.h>
+
+struct media_request_mgr;
+
+/**
+ * struct media_request_mgr_ops - request manager operations
+ *
+ * @req_alloc:	allocate a request
+ * @req_free:	free a previously allocated request
+ * @req_complete: callback invoked when a request has completed
+ *
+ */
+struct media_request_mgr_ops {
+	struct media_request *(*req_alloc)(struct media_request_mgr *mgr);
+	void (*req_free)(struct media_request *req);
+	void (*req_complete)(struct media_request *req);
+};
+
+/**
+ * struct media_request_mgr - requests manager
+ *
+ * @mdev:	media_device owning this manager
+ * @ops:	implementation of the manager
+ * @mutex:	protects requests, active_request, req_id, and all members of
+ *		struct media_request
+ * @requests:	list of alive requests produced by this manager
+ * @req_id:	counter used to identify requests for debugging purposes
+ */
+struct media_request_mgr {
+	struct media_device *mdev;
+	const struct media_request_mgr_ops *ops;
+
+	struct mutex mutex;
+	struct list_head requests;
+	u32 req_id;
+};
+
+/**
+ * media_request_mgr_alloc() - return an instance of the generic manager
+ *
+ * @mdev:	owning media device
+ */
+struct media_request_mgr *
+media_request_mgr_alloc(struct media_device *mdev);
+
+/**
+ * media_request_mgr_free() - free a media manager
+ *
+ * This should only be called when all requests produced by this manager
+ * has been destroyed. Will warn if that is not the case.
+ *
+ */
+void media_request_mgr_free(struct media_request_mgr *mgr);
+
+#endif
diff --git a/include/media/media-request.h b/include/media/media-request.h
new file mode 100644
index 000000000000..817df13ef6e3
--- /dev/null
+++ b/include/media/media-request.h
@@ -0,0 +1,186 @@
+/*
+ * Media requests.
+ *
+ * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _MEDIA_REQUEST_H
+#define _MEDIA_REQUEST_H
+
+#include <linux/kref.h>
+#include <linux/list.h>
+#include <linux/notifier.h>
+#include <linux/wait.h>
+
+struct media_device;
+struct media_entity;
+struct media_request_cmd;
+struct media_request_entity_data;
+struct media_request_mgr;
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+
+enum media_request_state {
+	MEDIA_REQUEST_STATE_IDLE,
+	MEDIA_REQUEST_STATE_SUBMITTED,
+	MEDIA_REQUEST_STATE_COMPLETED,
+	MEDIA_REQUEST_STATE_DELETED,
+};
+
+/**
+ * struct media_request - Media request base structure
+ * @id:		request id, used internally for debugging
+ * @mgr:	manager this request belongs to
+ * @kref:	reference count
+ * @list:	list entry in the media device requests list
+ * @lock:	protects internal state against concurrent accesses
+ * @state:	current state of the request
+ * @data:	per-entity data list
+ * @complete_wait:	wait queue that signals once the request has completed
+ * @submit_notif:	notification list to call when the request is submitted
+ * @buf_cpt:	counter of queued/completed buffers used to decide when the
+ *		request is completed
+ */
+struct media_request {
+	u32 id;
+	struct media_request_mgr *mgr;
+	struct kref kref;
+	struct list_head list;
+
+	struct mutex lock;
+	enum media_request_state state;
+	struct list_head data;
+	wait_queue_head_t complete_wait;
+	struct atomic_notifier_head submit_notif;
+	atomic_t buf_cpt;
+};
+
+/**
+ * media_request_get() - increment the reference counter of a request
+ *
+ * The calling context must call media_request_put() once it does not need
+ * the reference to the request anymore.
+ *
+ * Returns the request that has been passed as argument.
+ *
+ * @req:	request to acquire a reference of
+ */
+struct media_request *media_request_get(struct media_request *req);
+
+/**
+ * media_request_get_from_fd() - lookup request by fd and acquire a reference.
+ *
+ * Look a request up from its fd, acquire a reference and return a pointer to
+ * the request. As for media_request_get(), media_request_put() must be called
+ * once the reference is not used anymore.
+ *
+ * @req:	request to lookup and acquire.
+ *
+ */
+struct media_request *media_request_get_from_fd(int fd);
+
+/**
+ * media_request_put() - decrement the reference counter of a request
+ *
+ * Mirror function of media_request_get() and media_request_get_from_fd(). Will
+ * free the request if this was the last valid reference.
+ *
+ * @req:	request to release.
+ */
+void media_request_put(struct media_request *req);
+
+/**
+ * media_request_get_entity_data() - get per-entity data for a request
+ * @req:	request to get entity data from
+ * @entity:	entity to get data of
+ * @fh:		cookie identifying the handle from which the entity is accessed
+ *
+ * Search and return the entity data associated associated to the request. If no
+ * such data exists, it is allocated as per the entity operations.
+ *
+ * The fh arguments serves as a cookie to make sure the same entity is not
+ * accessed through different opened file handles. The same handle must be
+ * passed to all calls of this function for the same entity. Failure to do so
+ * will return an error.
+ *
+ * Returns the per-entity data, or an error code if a problem happened. -EINVAL
+ * means that data for the entity already existed, but has been allocated under
+ * a different cookie.
+ */
+struct media_request_entity_data *
+media_request_get_entity_data(struct media_request *req,
+			      struct media_entity *entity, void *fh);
+
+
+/**
+ * media_request_complete() - to be invoked when the request is complete
+ *
+ * @req:	request which has completed
+ */
+void media_request_complete(struct media_request *req);
+
+/**
+ * struct media_request_entity_data - per-entity request data
+ *
+ * Base structure used to store request state data. To be extended by actual
+ * implementation.
+ *
+ * @entity:	entity this data belongs to
+ * @fh:		subsystem-dependent. For V4L2, the v4l2_fh of the opened device
+ * @list:	entry in media_request::data
+ * @applied:	whether this request has already been applied for this entity
+ */
+struct media_request_entity_data {
+	struct media_entity *entity;
+	void *fh;
+	struct list_head list;
+	bool applied;
+};
+
+/**
+ * media_device_request_cmd() - perform the REQUEST_CMD ioctl
+ *
+ * @mdev:	media device the ioctl has been called on
+ * @cmd:	user-space request command structure
+ */
+long media_device_request_cmd(struct media_device *mdev,
+			      struct media_request_cmd *cmd);
+
+#else /* CONFIG_MEDIA_CONTROLLER */
+
+static inline media_request_get(struct media_request *req)
+{
+}
+
+static inline struct media_request *media_request_get_from_fd(int fd)
+{
+	return ERR_PTR(-ENOSYS);
+}
+
+static inline void media_request_put(struct media_request *req)
+{
+}
+
+static inline struct media_request *media_request_get_entity_data(
+					  struct media_request *req,
+					  struct media_entity *entity, void *fh)
+{
+	return ERR_PTR(-ENOSYS);
+}
+
+static inline void media_request_entity_complete(struct media_request *req)
+{
+}
+
+#endif /* CONFIG_MEDIA_CONTROLLER */
+
+#endif
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index b9b9446095e9..6e20ac9848b5 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -406,6 +406,15 @@ struct media_v2_topology {
 	__u64 ptr_links;
 } __attribute__ ((packed));
 
+#define MEDIA_REQ_CMD_ALLOC		0
+#define MEDIA_REQ_CMD_SUBMIT		1
+#define MEDIA_REQ_CMD_REINIT		2
+
+struct media_request_cmd {
+	__u32 cmd;
+	__s32 fd;
+} __attribute__ ((packed));
+
 /* ioctls */
 
 #define MEDIA_IOC_DEVICE_INFO		_IOWR('|', 0x00, struct media_device_info)
@@ -413,5 +422,6 @@ struct media_v2_topology {
 #define MEDIA_IOC_ENUM_LINKS		_IOWR('|', 0x02, struct media_links_enum)
 #define MEDIA_IOC_SETUP_LINK		_IOWR('|', 0x03, struct media_link_desc)
 #define MEDIA_IOC_G_TOPOLOGY		_IOWR('|', 0x04, struct media_v2_topology)
+#define MEDIA_IOC_REQUEST_CMD		_IOWR('|', 0x05, struct media_request_cmd)
 
 #endif /* __LINUX_MEDIA_H */
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 02/17] videodev2.h: Add request_fd field to v4l2_buffer

[Alexandre Courbot]

From: Hans Verkuil <hans.verkuil@cisco.com>

When queuing buffers allow for passing the request that should
be associated with this buffer.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
[acourbot@chromium.org: make request ID 32-bit]
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/usb/cpia2/cpia2_v4l.c           | 2 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 7 ++++---
 drivers/media/v4l2-core/v4l2-ioctl.c          | 4 ++--
 drivers/media/v4l2-core/videobuf2-v4l2.c      | 3 ++-
 include/media/videobuf2-v4l2.h                | 2 ++
 include/uapi/linux/videodev2.h                | 3 ++-
 6 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/drivers/media/usb/cpia2/cpia2_v4l.c b/drivers/media/usb/cpia2/cpia2_v4l.c
index 3dedd83f0b19..7acb6807b306 100644
--- a/drivers/media/usb/cpia2/cpia2_v4l.c
+++ b/drivers/media/usb/cpia2/cpia2_v4l.c
@@ -948,7 +948,7 @@ static int cpia2_dqbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
 	buf->sequence = cam->buffers[buf->index].seq;
 	buf->m.offset = cam->buffers[buf->index].data - cam->frame_buffer;
 	buf->length = cam->frame_size;
-	buf->reserved2 = 0;
+	buf->request_fd = 0;
 	buf->reserved = 0;
 	memset(&buf->timecode, 0, sizeof(buf->timecode));
 
diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 821f2aa299ae..7e4440950c76 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -370,7 +370,7 @@ struct v4l2_buffer32 {
 		__s32		fd;
 	} m;
 	__u32			length;
-	__u32			reserved2;
+	__u32			request_fd;
 	__u32			reserved;
 };
 
@@ -438,7 +438,8 @@ static int get_v4l2_buffer32(struct v4l2_buffer *kp, struct v4l2_buffer32 __user
 		get_user(kp->type, &up->type) ||
 		get_user(kp->flags, &up->flags) ||
 		get_user(kp->memory, &up->memory) ||
-		get_user(kp->length, &up->length))
+		get_user(kp->length, &up->length) ||
+		get_user(kp->request_fd, &up->request_fd))
 			return -EFAULT;
 
 	if (V4L2_TYPE_IS_OUTPUT(kp->type))
@@ -533,7 +534,7 @@ static int put_v4l2_buffer32(struct v4l2_buffer *kp, struct v4l2_buffer32 __user
 		put_user(kp->timestamp.tv_usec, &up->timestamp.tv_usec) ||
 		copy_to_user(&up->timecode, &kp->timecode, sizeof(struct v4l2_timecode)) ||
 		put_user(kp->sequence, &up->sequence) ||
-		put_user(kp->reserved2, &up->reserved2) ||
+		put_user(kp->request_fd, &up->request_fd) ||
 		put_user(kp->reserved, &up->reserved) ||
 		put_user(kp->length, &up->length))
 			return -EFAULT;
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index ec4ecd5aa8bf..fdd2f784c264 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -437,13 +437,13 @@ static void v4l_print_buffer(const void *arg, bool write_only)
 	const struct v4l2_plane *plane;
 	int i;
 
-	pr_cont("%02ld:%02d:%02d.%08ld index=%d, type=%s, flags=0x%08x, field=%s, sequence=%d, memory=%s",
+	pr_cont("%02ld:%02d:%02d.%08ld index=%d, type=%s, request_fd=%u, flags=0x%08x, field=%s, sequence=%d, memory=%s",
 			p->timestamp.tv_sec / 3600,
 			(int)(p->timestamp.tv_sec / 60) % 60,
 			(int)(p->timestamp.tv_sec % 60),
 			(long)p->timestamp.tv_usec,
 			p->index,
-			prt_names(p->type, v4l2_type_names),
+			prt_names(p->type, v4l2_type_names), p->request_fd,
 			p->flags, prt_names(p->field, v4l2_field_names),
 			p->sequence, prt_names(p->memory, v4l2_memory_names));
 
diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
index 0c0669976bdc..0f8edbdebe30 100644
--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
@@ -203,7 +203,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
 	b->timestamp = ns_to_timeval(vb->timestamp);
 	b->timecode = vbuf->timecode;
 	b->sequence = vbuf->sequence;
-	b->reserved2 = 0;
+	b->request_fd = vbuf->request_fd;
 	b->reserved = 0;
 
 	if (q->is_multiplanar) {
@@ -320,6 +320,7 @@ static int __fill_vb2_buffer(struct vb2_buffer *vb,
 	}
 	vb->timestamp = 0;
 	vbuf->sequence = 0;
+	vbuf->request_fd = b->request_fd;
 
 	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
 		if (b->memory == VB2_MEMORY_USERPTR) {
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 036127c54bbf..d7cf4c66db38 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -31,6 +31,7 @@
  * @field:	enum v4l2_field; field order of the image in the buffer
  * @timecode:	frame timecode
  * @sequence:	sequence count of this frame
+ * @request_fd:	fd of the request used by the buffer
  *
  * Should contain enough information to be able to cover all the fields
  * of struct v4l2_buffer at videodev2.h
@@ -42,6 +43,7 @@ struct vb2_v4l2_buffer {
 	__u32			field;
 	struct v4l2_timecode	timecode;
 	__u32			sequence;
+	__u32			request_fd;
 };
 
 /*
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 1c095b5a99c5..89bd716c66a6 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -902,6 +902,7 @@ struct v4l2_plane {
  * @length:	size in bytes of the buffer (NOT its payload) for single-plane
  *		buffers (when type != *_MPLANE); number of elements in the
  *		planes array for multi-plane buffers
+ * @request_fd: fd of the request that this buffer should use
  *
  * Contains data exchanged by application and driver using one of the Streaming
  * I/O methods.
@@ -925,7 +926,7 @@ struct v4l2_buffer {
 		__s32		fd;
 	} m;
 	__u32			length;
-	__u32			reserved2;
+	__s32			request_fd;
 	__u32			reserved;
 };
 
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 03/17] media: videobuf2: add support for requests

[Alexandre Courbot]

Make vb2 aware of requests. Drivers can specify whether a given queue
can accept requests or not. Queues that accept requests will block on a
buffer that is part of a request until that request is submitted.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/videobuf2-core.c | 133 +++++++++++++++++++++++++++++--
 drivers/media/v4l2-core/videobuf2-v4l2.c |  28 ++++++-
 include/media/videobuf2-core.h           |  15 +++-
 3 files changed, 168 insertions(+), 8 deletions(-)

diff --git a/drivers/media/v4l2-core/videobuf2-core.c b/drivers/media/v4l2-core/videobuf2-core.c
index cb115ba6a1d2..4a69ac12ee88 100644
--- a/drivers/media/v4l2-core/videobuf2-core.c
+++ b/drivers/media/v4l2-core/videobuf2-core.c
@@ -26,6 +26,7 @@
 
 #include <media/videobuf2-core.h>
 #include <media/v4l2-mc.h>
+#include <media/media-request.h>
 
 #include <trace/events/vb2.h>
 
@@ -922,6 +923,17 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
 		vb->state = state;
 	}
 	atomic_dec(&q->owned_by_drv_count);
+	if (vb->request) {
+		struct media_request *req = vb->request;
+
+		if (atomic_dec_and_test(&req->buf_cpt))
+			media_request_complete(vb->request);
+
+		/* release reference acquired during qbuf */
+		vb->request = NULL;
+		media_request_put(req);
+	}
+
 	spin_unlock_irqrestore(&q->done_lock, flags);
 
 	trace_vb2_buf_done(q, vb);
@@ -1298,6 +1310,53 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
 }
 EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
 
+/**
+ * vb2_check_buf_req_status() - Validate request state of a buffer
+ * @vb:		buffer to check
+ *
+ * Returns true if a buffer is ready to be passed to the driver request-wise.
+ * This means that neither this buffer nor any previously-queued buffer is
+ * associated to a request that is not yet submitted.
+ *
+ * If this function returns false, then the buffer shall not be passed to its
+ * driver since the request state is not completely built yet. In that case,
+ * this function will register a notifier to be called when the request is
+ * submitted and the queue can be unblocked.
+ *
+ * This function must be called with req_lock held.
+ */
+static bool vb2_check_buf_req_status(struct vb2_buffer *vb)
+{
+	struct media_request *req = vb->request;
+	struct vb2_queue *q = vb->vb2_queue;
+	int ret = false;
+
+	mutex_lock(&q->req_lock);
+
+	if (!req) {
+		ret = !q->waiting_req;
+		goto done;
+	}
+
+	mutex_lock(&req->lock);
+	if (req->state == MEDIA_REQUEST_STATE_SUBMITTED) {
+		mutex_unlock(&req->lock);
+		ret = !q->waiting_req;
+		goto done;
+	}
+
+	if (!q->waiting_req) {
+		q->waiting_req = true;
+		atomic_notifier_chain_register(&req->submit_notif,
+					       &q->req_blk);
+	}
+	mutex_unlock(&req->lock);
+
+done:
+	mutex_unlock(&q->req_lock);
+	return ret;
+}
+
 /**
  * vb2_start_streaming() - Attempt to start streaming.
  * @q:		videobuf2 queue
@@ -1318,8 +1377,11 @@ static int vb2_start_streaming(struct vb2_queue *q)
 	 * If any buffers were queued before streamon,
 	 * we can now pass them to driver for processing.
 	 */
-	list_for_each_entry(vb, &q->queued_list, queued_entry)
+	list_for_each_entry(vb, &q->queued_list, queued_entry) {
+		if (!vb2_check_buf_req_status(vb))
+			break;
 		__enqueue_in_driver(vb);
+	}
 
 	/* Tell the driver to start streaming */
 	q->start_streaming_called = 1;
@@ -1361,7 +1423,46 @@ static int vb2_start_streaming(struct vb2_queue *q)
 	return ret;
 }
 
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
+/**
+ * vb2_unblock_requests() - unblock a queue waiting for a request submission
+ * @nb:		notifier block that has been registered
+ * @action:	unused
+ * @data:	request that has been submitted
+ *
+ * This is a callback function that is registered when
+ * vb2_check_buf_req_status() returns false. It is invoked when the request
+ * blocking the queue has been submitted. This means its buffers (and all
+ * following valid buffers) can be passed to drivers.
+ */
+static int vb2_unblock_requests(struct notifier_block *nb, unsigned long action,
+				void *data)
+{
+	struct vb2_queue *q = container_of(nb, struct vb2_queue, req_blk);
+	struct media_request *req = data;
+	struct vb2_buffer *vb;
+	bool found_request = false;
+
+	mutex_lock(&q->req_lock);
+	atomic_notifier_chain_unregister(&req->submit_notif, &q->req_blk);
+	q->waiting_req = false;
+	mutex_unlock(&q->req_lock);
+
+	list_for_each_entry(vb, &q->queued_list, queued_entry) {
+		/* All buffers before our request are already passed to the driver */
+		if (!found_request && vb->request != req)
+			continue;
+		found_request = true;
+
+		if (!vb2_check_buf_req_status(vb))
+			break;
+		__enqueue_in_driver(vb);
+	}
+
+	return 0;
+}
+
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index,
+		  struct media_request *req, void *pb)
 {
 	struct vb2_buffer *vb;
 	int ret;
@@ -1384,6 +1485,24 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
 		return -EINVAL;
 	}
 
+	vb->request = req;
+	if (req) {
+		struct vb2_buffer *_vb;
+
+		/* does the queue support requests? */
+		if (!q->allow_requests)
+			return -EINVAL;
+
+		/* do we already have a buffer for this request in the queue? */
+		list_for_each_entry(_vb, &q->queued_list, queued_entry)
+			if (_vb->request == req)
+				return -EBUSY;
+
+		/* make sure the request stays alive as long as we need */
+		media_request_get(req);
+		atomic_inc(&req->buf_cpt);
+	}
+
 	/*
 	 * Add to the queued buffers list, a buffer will stay on it until
 	 * dequeued in dqbuf.
@@ -1402,7 +1521,7 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
 	 * If already streaming, give the buffer to driver for processing.
 	 * If not, the buffer will be given to driver on next streamon.
 	 */
-	if (q->start_streaming_called)
+	if (q->start_streaming_called && vb2_check_buf_req_status(vb))
 		__enqueue_in_driver(vb);
 
 	/* Fill buffer information for the userspace */
@@ -1993,6 +2112,8 @@ int vb2_core_queue_init(struct vb2_queue *q)
 	spin_lock_init(&q->done_lock);
 	mutex_init(&q->mmap_lock);
 	init_waitqueue_head(&q->done_wq);
+	mutex_init(&q->req_lock);
+	q->req_blk.notifier_call = vb2_unblock_requests;
 
 	if (q->buf_struct_size == 0)
 		q->buf_struct_size = sizeof(struct vb2_buffer);
@@ -2242,7 +2363,7 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
 		 * Queue all buffers.
 		 */
 		for (i = 0; i < q->num_buffers; i++) {
-			ret = vb2_core_qbuf(q, i, NULL);
+			ret = vb2_core_qbuf(q, i, NULL, NULL);
 			if (ret)
 				goto err_reqbufs;
 			fileio->bufs[i].queued = 1;
@@ -2421,7 +2542,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
 
 		if (copy_timestamp)
 			b->timestamp = ktime_get_ns();
-		ret = vb2_core_qbuf(q, index, NULL);
+		ret = vb2_core_qbuf(q, index, NULL, NULL);
 		dprintk(5, "vb2_dbuf result: %d\n", ret);
 		if (ret)
 			return ret;
@@ -2524,7 +2645,7 @@ static int vb2_thread(void *data)
 		if (copy_timestamp)
 			vb->timestamp = ktime_get_ns();;
 		if (!threadio->stop)
-			ret = vb2_core_qbuf(q, vb->index, NULL);
+			ret = vb2_core_qbuf(q, vb->index, NULL, NULL);
 		call_void_qop(q, wait_prepare, q);
 		if (ret || threadio->stop)
 			break;
diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
index 0f8edbdebe30..267fe2d669b2 100644
--- a/drivers/media/v4l2-core/videobuf2-v4l2.c
+++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
@@ -30,6 +30,7 @@
 #include <media/v4l2-common.h>
 
 #include <media/videobuf2-v4l2.h>
+#include <media/media-request.h>
 
 static int debug;
 module_param(debug, int, 0644);
@@ -561,6 +562,7 @@ EXPORT_SYMBOL_GPL(vb2_create_bufs);
 
 int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
 {
+	struct media_request *req = NULL;
 	int ret;
 
 	if (vb2_fileio_is_active(q)) {
@@ -568,8 +570,32 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
 		return -EBUSY;
 	}
 
+	/*
+	 * The caller should have validated that the request is valid,
+	 * so we just need to look it up without further checking
+	 */
+	if (b->request_fd > 0) {
+		req = media_request_get_from_fd(b->request_fd);
+		if (!req)
+			return -EINVAL;
+
+		mutex_lock(&req->lock);
+		if (req->state != MEDIA_REQUEST_STATE_IDLE) {
+			mutex_unlock(&req->lock);
+			media_request_put(req);
+			return -EINVAL;
+		}
+		mutex_unlock(&req->lock);
+	}
+
 	ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
-	return ret ? ret : vb2_core_qbuf(q, b->index, b);
+	if (!ret)
+		ret = vb2_core_qbuf(q, b->index, req, b);
+
+	if (req)
+		media_request_put(req);
+
+	return ret;
 }
 EXPORT_SYMBOL_GPL(vb2_qbuf);
 
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index ef9b64398c8c..7bb17c842ab4 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -237,6 +237,7 @@ struct vb2_queue;
  *			on an internal driver queue
  * @planes:		private per-plane information; do not change
  * @timestamp:		frame timestamp in ns
+ * @request:		request the buffer belongs to, if any
  */
 struct vb2_buffer {
 	struct vb2_queue	*vb2_queue;
@@ -246,6 +247,7 @@ struct vb2_buffer {
 	unsigned int		num_planes;
 	struct vb2_plane	planes[VB2_MAX_PLANES];
 	u64			timestamp;
+	struct media_request	*request;
 
 	/* private: internal use only
 	 *
@@ -443,6 +445,7 @@ struct vb2_buf_ops {
  * @quirk_poll_must_check_waiting_for_buffers: Return POLLERR at poll when QBUF
  *              has not been called. This is a vb1 idiom that has been adopted
  *              also by vb2.
+ * @allow_requests:	whether requests are supported on this queue.
  * @lock:	pointer to a mutex that protects the vb2_queue struct. The
  *		driver can set this to a mutex to let the v4l2 core serialize
  *		the queuing ioctls. If the driver wants to handle locking
@@ -500,6 +503,9 @@ struct vb2_buf_ops {
  *		when a buffer with the V4L2_BUF_FLAG_LAST is dequeued.
  * @fileio:	file io emulator internal data, used only if emulator is active
  * @threadio:	thread io internal data, used only if thread is active
+ * @req_lock:	protects req_blk and waiting_req
+ * @req_blk:	notifier to be called when waiting for a request to be submitted
+ * @waiting_req:whether this queue is currently waiting on a request submission
  */
 struct vb2_queue {
 	unsigned int			type;
@@ -511,6 +517,7 @@ struct vb2_queue {
 	unsigned			fileio_write_immediately:1;
 	unsigned			allow_zero_bytesused:1;
 	unsigned		   quirk_poll_must_check_waiting_for_buffers:1;
+	unsigned			allow_requests:1;
 
 	struct mutex			*lock;
 	void				*owner;
@@ -554,6 +561,10 @@ struct vb2_queue {
 	struct vb2_fileio_data		*fileio;
 	struct vb2_threadio_data	*threadio;
 
+	struct mutex			req_lock;
+	struct notifier_block		req_blk;
+	bool				waiting_req;
+
 #ifdef CONFIG_VIDEO_ADV_DEBUG
 	/*
 	 * Counters for how often these queue-related ops are
@@ -724,6 +735,7 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
  *
  * @q:		videobuf2 queue
  * @index:	id number of the buffer
+ * @req:	request this buffer belongs to, if any
  * @pb:		buffer structure passed from userspace to vidioc_qbuf handler
  *		in driver
  *
@@ -740,7 +752,8 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
  * The return values from this function are intended to be directly returned
  * from vidioc_qbuf handler in driver.
  */
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb);
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index,
+		  struct media_request *req, void *pb);
 
 /**
  * vb2_core_dqbuf() - Dequeue a buffer to the userspace
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 04/17] media: vb2: add support for requests in QBUF ioctl

[Alexandre Courbot]

Support the request argument of the QBUF ioctl.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 83 +++++++++++++++++++++++++++++++++++-
 1 file changed, 82 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index fdd2f784c264..3b44f1fe4f23 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -29,6 +29,7 @@
 #include <media/v4l2-device.h>
 #include <media/videobuf2-v4l2.h>
 #include <media/v4l2-mc.h>
+#include <media/media-request.h>
 
 #include <trace/events/v4l2.h>
 
@@ -965,6 +966,71 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
 	return -EINVAL;
 }
 
+/*
+ * Validate that a given request can be used during an ioctl.
+ *
+ * When using the request API, request file descriptors must be matched against
+ * the actual request object. User-space can pass any file descriptor, so we
+ * need to make sure the call is valid before going further.
+ *
+ * This function looks up the request and associated data and performs the
+ * following sanity checks:
+ *
+ * * Make sure that the entity belongs to the media_device managing the passed
+ *   request,
+ * * Make sure that the entity data (if any) is associated to the current file
+ *   handler.
+ *
+ * This function returns a pointer to the valid request, or and error code in
+ * case of failure. When successful, a reference to the request is acquired and
+ * must be properly released by the caller. If _data is not NULL, then it is
+ * filled with a pointer to the corresponding entity data.
+ */
+#ifdef CONFIG_MEDIA_CONTROLLER
+static struct media_request *
+check_request(int request, struct file *file, void *fh, struct media_request_entity_data **_data)
+{
+	struct media_request *req = NULL;
+	struct video_device *vfd = video_devdata(file);
+	struct v4l2_fh *vfh =
+		test_bit(V4L2_FL_USES_V4L2_FH, &vfd->flags) ? fh : NULL;
+	struct media_entity *entity = &vfd->entity;
+	struct media_request_entity_data *data;
+
+	if (!entity)
+		return ERR_PTR(-EINVAL);
+
+	req = media_request_get_from_fd(request);
+	if (!req)
+		return ERR_PTR(-EINVAL);
+
+	/* Validate that the entity belongs to the correct media_device */
+	if (vfd->v4l2_dev->mdev->req_mgr != req->mgr) {
+		media_request_put(req);
+		return ERR_PTR(-EINVAL);
+	}
+
+	/* Validate that the entity's data belongs to the correct fh */
+	data = media_request_get_entity_data(req, entity, vfh);
+	if (IS_ERR(data)) {
+		media_request_put(req);
+		return ERR_PTR(PTR_ERR(data));
+	}
+
+	if (_data)
+		*_data = data;
+
+	return req;
+}
+#else /* CONFIG_MEDIA_CONTROLLER */
+static struct media_request *
+check_request(int request, struct file *file, void *fh)
+{
+	return ERR_PTR(-ENOSYS);
+}
+
+#endif /* CONFIG_MEDIA_CONTROLLER */
+
 static void v4l_sanitize_format(struct v4l2_format *fmt)
 {
 	unsigned int offset;
@@ -1902,10 +1968,25 @@ static int v4l_querybuf(const struct v4l2_ioctl_ops *ops,
 static int v4l_qbuf(const struct v4l2_ioctl_ops *ops,
 				struct file *file, void *fh, void *arg)
 {
+	struct media_request *req = NULL;
 	struct v4l2_buffer *p = arg;
 	int ret = check_fmt(file, p->type);
 
-	return ret ? ret : ops->vidioc_qbuf(file, fh, p);
+	if (ret)
+		return ret;
+
+	if (p->request_fd > 0) {
+		req = check_request(p->request_fd, file, fh, NULL);
+		if (IS_ERR(req))
+			return PTR_ERR(req);
+	}
+
+	ret = ops->vidioc_qbuf(file, fh, p);
+
+	if (req)
+		media_request_put(req);
+
+	return ret;
 }
 
 static int v4l_dqbuf(const struct v4l2_ioctl_ops *ops,
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 05/17] media: Document the media request API

[Alexandre Courbot]

From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

The media request API is made of a new ioctl to implement request
management. Document it.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
[acourbot@chromium.org: adapt for newest API]
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 Documentation/media/uapi/mediactl/media-funcs.rst  |   1 +
 .../media/uapi/mediactl/media-ioc-request-cmd.rst  | 141 +++++++++++++++++++++
 2 files changed, 142 insertions(+)
 create mode 100644 Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst

diff --git a/Documentation/media/uapi/mediactl/media-funcs.rst b/Documentation/media/uapi/mediactl/media-funcs.rst
index 076856501cdb..e3a45d82ffcb 100644
--- a/Documentation/media/uapi/mediactl/media-funcs.rst
+++ b/Documentation/media/uapi/mediactl/media-funcs.rst
@@ -15,4 +15,5 @@ Function Reference
     media-ioc-g-topology
     media-ioc-enum-entities
     media-ioc-enum-links
+    media-ioc-request-cmd
     media-ioc-setup-link
diff --git a/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst b/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
new file mode 100644
index 000000000000..723b422afcce
--- /dev/null
+++ b/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
@@ -0,0 +1,141 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _media_ioc_request_cmd:
+
+***************************
+ioctl MEDIA_IOC_REQUEST_CMD
+***************************
+
+Name
+====
+
+MEDIA_IOC_REQUEST_CMD - Manage media device requests
+
+
+Synopsis
+========
+
+.. c:function:: int ioctl( int fd, MEDIA_IOC_REQUEST_CMD, struct media_request_cmd *argp )
+    :name: MEDIA_IOC_REQUEST_CMD
+
+
+Arguments
+=========
+
+``fd``
+    File descriptor returned by :ref:`open() <media-func-open>`.
+
+``argp``
+
+
+Description
+===========
+
+The MEDIA_IOC_REQUEST_CMD ioctl allow applications to manage media device
+requests. A request is an object that can group media device configuration
+parameters, including subsystem-specific parameters, in order to apply all the
+parameters atomically. Applications are responsible for allocating and
+deleting requests, filling them with configuration parameters submitting them.
+
+Request operations are performed by calling the MEDIA_IOC_REQUEST_CMD ioctl
+with a pointer to a struct :c:type:`media_request_cmd` with the cmd field set
+to the appropriate command. :ref:`media-request-command` lists the commands
+supported by the ioctl.
+
+The struct :c:type:`media_request_cmd` request field contains the file
+descriptorof the request on which the command operates. For the
+``MEDIA_REQ_CMD_ALLOC`` command the field is set to zero by applications and
+filled by the driver. For all other commands the field is set by applications
+and left untouched by the driver.
+
+To allocate a new request applications use the ``MEDIA_REQ_CMD_ALLOC``
+command. The driver will allocate a new request and return its FD in the
+request field. After allocation, the request is "empty", which means that it
+does not hold any state of its own, and that the hardware's state will not be
+affected by it unless it is passed as argument to V4L2 or media controller
+commands.
+
+Requests are reference-counted. A newly allocated request is referenced
+by the returned file descriptor, and can be later referenced by
+subsystem-specific operations. Requests will thus be automatically deleted
+when they're not longer used after the returned file descriptor is closed.
+
+If a request isn't needed applications can delete it by calling ``close()``
+on it. The driver will drop the file handle reference. The request will not
+be usable through the MEDIA_IOC_REQUEST_CMD ioctl anymore, but will only be
+deleted when the last reference is released. If no other reference exists when
+``close()`` is invoked the request will be deleted immediately.
+
+After creating a request applications should fill it with configuration
+parameters. This is performed through subsystem-specific request APIs outside
+the scope of the media controller API. See the appropriate subsystem APIs for
+more information, including how they interact with the MEDIA_IOC_REQUEST_CMD
+ioctl.
+
+Once a request contains all the desired configuration parameters it can be
+submitted using the ``MEDIA_REQ_CMD_SUBMIT`` command. This will let the
+buffers queued for the request be passed to their respective drivers, which
+will then apply the request's parameters before processing them.
+
+Once a request has been queued applications are not allowed to modify its
+configuration parameters until the request has been fully processed. Any
+attempt to do so will result in the related subsystem API returning an error.
+The application that submitted the request can wait for its completion by
+polling on the request's file descriptor.
+
+Once a request has completed, it can be reused. The ``MEDIA_REQ_CMD_REINIT``
+command will bring it back to its initial state, so it can be prepared and
+submitted again.
+
+.. c:type:: media_request_cmd
+
+.. flat-table:: struct media_request_cmd
+    :header-rows:  0
+    :stub-columns: 0
+    :widths: 1 2 8
+
+    * - __u32
+      - ``cmd``
+      - Command, set by the application. See below for the list of supported
+        commands.
+    * - __u32
+      - ``fd``
+      - Request FD, set by the driver for the MEDIA_REQ_CMD_ALLOC command and
+        by the application for all other commands.
+
+
+.. _media-request-command:
+
+.. cssclass:: longtable
+
+.. flat-table:: Media request commands
+    :header-rows:  0
+    :stub-columns: 0
+
+    * .. _MEDIA-REQ-CMD-ALLOC:
+
+      - ``MEDIA_REQ_CMD_ALLOC``
+      - Allocate a new request.
+    * .. _MEDIA-REQ-CMD-SUBMIT:
+
+      - ``MEDIA_REQ_CMD_SUBMIT``
+      - Submit a request to be processed.
+    * .. _MEDIA-REQ-CMD-QUEUE:
+
+      - ``MEDIA_REQ_CMD_REINIT``
+      - Reinitializes a completed request.
+
+
+Return Value
+============
+
+On success 0 is returned, on error -1 and the ``errno`` variable is set
+appropriately. The generic error codes are described at the
+:ref:`Generic Error Codes <gen-errors>` chapter.
+
+EINVAL
+    The struct :c:type:`media_request_cmd` specifies an invalid command or
+    references a non-existing request.
+
+ENOSYS
+    Request API is not available on this device.
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 06/17] v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev

[Alexandre Courbot]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add a 'bool from_other_dev' argument: set to true if the two
handlers refer to different devices (e.g. it is true when
inheriting controls from a subdev into a main v4l2 bridge
driver).

This will be used later when implementing support for the
request API since we need to skip such controls.

TODO: check drivers/staging/media/imx/imx-media-fim.c change.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/dvb-frontends/rtl2832_sdr.c        |  5 +--
 drivers/media/pci/bt8xx/bttv-driver.c            |  2 +-
 drivers/media/pci/cx23885/cx23885-417.c          |  2 +-
 drivers/media/pci/cx88/cx88-blackbird.c          |  2 +-
 drivers/media/pci/cx88/cx88-video.c              |  2 +-
 drivers/media/pci/saa7134/saa7134-empress.c      |  4 +--
 drivers/media/pci/saa7134/saa7134-video.c        |  2 +-
 drivers/media/platform/exynos4-is/fimc-capture.c |  2 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c      |  3 +-
 drivers/media/platform/rcar_drif.c               |  2 +-
 drivers/media/platform/soc_camera/soc_camera.c   |  3 +-
 drivers/media/platform/vivid/vivid-ctrls.c       | 42 ++++++++++++------------
 drivers/media/usb/cx231xx/cx231xx-417.c          |  2 +-
 drivers/media/usb/cx231xx/cx231xx-video.c        |  4 +--
 drivers/media/usb/msi2500/msi2500.c              |  2 +-
 drivers/media/usb/tm6000/tm6000-video.c          |  2 +-
 drivers/media/v4l2-core/v4l2-ctrls.c             | 11 ++++---
 drivers/media/v4l2-core/v4l2-device.c            |  3 +-
 drivers/staging/media/imx/imx-media-dev.c        |  2 +-
 drivers/staging/media/imx/imx-media-fim.c        |  2 +-
 include/media/v4l2-ctrls.h                       |  4 ++-
 21 files changed, 56 insertions(+), 47 deletions(-)

diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.c b/drivers/media/dvb-frontends/rtl2832_sdr.c
index c6e78d870ccd..6064d28224e8 100644
--- a/drivers/media/dvb-frontends/rtl2832_sdr.c
+++ b/drivers/media/dvb-frontends/rtl2832_sdr.c
@@ -1394,7 +1394,8 @@ static int rtl2832_sdr_probe(struct platform_device *pdev)
 	case RTL2832_SDR_TUNER_E4000:
 		v4l2_ctrl_handler_init(&dev->hdl, 9);
 		if (subdev)
-			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler, NULL);
+			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler,
+					      NULL, true);
 		break;
 	case RTL2832_SDR_TUNER_R820T:
 	case RTL2832_SDR_TUNER_R828D:
@@ -1423,7 +1424,7 @@ static int rtl2832_sdr_probe(struct platform_device *pdev)
 		v4l2_ctrl_handler_init(&dev->hdl, 2);
 		if (subdev)
 			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler,
-					      NULL);
+					      NULL, true);
 		break;
 	default:
 		v4l2_ctrl_handler_init(&dev->hdl, 0);
diff --git a/drivers/media/pci/bt8xx/bttv-driver.c b/drivers/media/pci/bt8xx/bttv-driver.c
index 227086a2e99c..6fc2bb765fc6 100644
--- a/drivers/media/pci/bt8xx/bttv-driver.c
+++ b/drivers/media/pci/bt8xx/bttv-driver.c
@@ -4211,7 +4211,7 @@ static int bttv_probe(struct pci_dev *dev, const struct pci_device_id *pci_id)
 	/* register video4linux + input */
 	if (!bttv_tvcards[btv->c.type].no_video) {
 		v4l2_ctrl_add_handler(&btv->radio_ctrl_handler, hdl,
-				v4l2_ctrl_radio_filter);
+				v4l2_ctrl_radio_filter, false);
 		if (btv->radio_ctrl_handler.error) {
 			result = btv->radio_ctrl_handler.error;
 			goto fail2;
diff --git a/drivers/media/pci/cx23885/cx23885-417.c b/drivers/media/pci/cx23885/cx23885-417.c
index a71f3c7569ce..762823871c78 100644
--- a/drivers/media/pci/cx23885/cx23885-417.c
+++ b/drivers/media/pci/cx23885/cx23885-417.c
@@ -1527,7 +1527,7 @@ int cx23885_417_register(struct cx23885_dev *dev)
 	dev->cxhdl.priv = dev;
 	dev->cxhdl.func = cx23885_api_func;
 	cx2341x_handler_set_50hz(&dev->cxhdl, tsport->height == 576);
-	v4l2_ctrl_add_handler(&dev->ctrl_handler, &dev->cxhdl.hdl, NULL);
+	v4l2_ctrl_add_handler(&dev->ctrl_handler, &dev->cxhdl.hdl, NULL, false);
 
 	/* Allocate and initialize V4L video device */
 	dev->v4l_device = cx23885_video_dev_alloc(tsport,
diff --git a/drivers/media/pci/cx88/cx88-blackbird.c b/drivers/media/pci/cx88/cx88-blackbird.c
index e3101f04941c..8424fb0da90c 100644
--- a/drivers/media/pci/cx88/cx88-blackbird.c
+++ b/drivers/media/pci/cx88/cx88-blackbird.c
@@ -1184,7 +1184,7 @@ static int cx8802_blackbird_probe(struct cx8802_driver *drv)
 	err = cx2341x_handler_init(&dev->cxhdl, 36);
 	if (err)
 		goto fail_core;
-	v4l2_ctrl_add_handler(&dev->cxhdl.hdl, &core->video_hdl, NULL);
+	v4l2_ctrl_add_handler(&dev->cxhdl.hdl, &core->video_hdl, NULL, false);
 
 	/* blackbird stuff */
 	pr_info("cx23416 based mpeg encoder (blackbird reference design)\n");
diff --git a/drivers/media/pci/cx88/cx88-video.c b/drivers/media/pci/cx88/cx88-video.c
index 7d25ecd4404b..fc52d80b7472 100644
--- a/drivers/media/pci/cx88/cx88-video.c
+++ b/drivers/media/pci/cx88/cx88-video.c
@@ -1376,7 +1376,7 @@ static int cx8800_initdev(struct pci_dev *pci_dev,
 		if (vc->id == V4L2_CID_CHROMA_AGC)
 			core->chroma_agc = vc;
 	}
-	v4l2_ctrl_add_handler(&core->video_hdl, &core->audio_hdl, NULL);
+	v4l2_ctrl_add_handler(&core->video_hdl, &core->audio_hdl, NULL, false);
 
 	/* load and configure helper modules */
 
diff --git a/drivers/media/pci/saa7134/saa7134-empress.c b/drivers/media/pci/saa7134/saa7134-empress.c
index 66acfd35ffc6..fc75ce00dbf8 100644
--- a/drivers/media/pci/saa7134/saa7134-empress.c
+++ b/drivers/media/pci/saa7134/saa7134-empress.c
@@ -265,9 +265,9 @@ static int empress_init(struct saa7134_dev *dev)
 		 "%s empress (%s)", dev->name,
 		 saa7134_boards[dev->board].name);
 	v4l2_ctrl_handler_init(hdl, 21);
-	v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler, empress_ctrl_filter);
+	v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler, empress_ctrl_filter, false);
 	if (dev->empress_sd)
-		v4l2_ctrl_add_handler(hdl, dev->empress_sd->ctrl_handler, NULL);
+		v4l2_ctrl_add_handler(hdl, dev->empress_sd->ctrl_handler, NULL, true);
 	if (hdl->error) {
 		video_device_release(dev->empress_dev);
 		return hdl->error;
diff --git a/drivers/media/pci/saa7134/saa7134-video.c b/drivers/media/pci/saa7134/saa7134-video.c
index 51d42bbf969e..f8915aa3e994 100644
--- a/drivers/media/pci/saa7134/saa7134-video.c
+++ b/drivers/media/pci/saa7134/saa7134-video.c
@@ -2134,7 +2134,7 @@ int saa7134_video_init1(struct saa7134_dev *dev)
 		hdl = &dev->radio_ctrl_handler;
 		v4l2_ctrl_handler_init(hdl, 2);
 		v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler,
-				v4l2_ctrl_radio_filter);
+				v4l2_ctrl_radio_filter, false);
 		if (hdl->error)
 			return hdl->error;
 	}
diff --git a/drivers/media/platform/exynos4-is/fimc-capture.c b/drivers/media/platform/exynos4-is/fimc-capture.c
index 948fe01f6c96..8280f94400c7 100644
--- a/drivers/media/platform/exynos4-is/fimc-capture.c
+++ b/drivers/media/platform/exynos4-is/fimc-capture.c
@@ -1418,7 +1418,7 @@ static int fimc_link_setup(struct media_entity *entity,
 		return 0;
 
 	return v4l2_ctrl_add_handler(&vc->ctx->ctrls.handler,
-				     sensor->ctrl_handler, NULL);
+				     sensor->ctrl_handler, NULL, true);
 }
 
 static const struct media_entity_operations fimc_sd_media_ops = {
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index dd37ea811680..286b5564d8df 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -900,7 +900,8 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
 	if (ret < 0)
 		return ret;
 
-	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
+	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler,
+				    NULL, true);
 	if (ret < 0)
 		return ret;
 
diff --git a/drivers/media/platform/rcar_drif.c b/drivers/media/platform/rcar_drif.c
index 522364ff0d5d..869685ba635c 100644
--- a/drivers/media/platform/rcar_drif.c
+++ b/drivers/media/platform/rcar_drif.c
@@ -1167,7 +1167,7 @@ static int rcar_drif_notify_complete(struct v4l2_async_notifier *notifier)
 	}
 
 	ret = v4l2_ctrl_add_handler(&sdr->ctrl_hdl,
-				    sdr->ep.subdev->ctrl_handler, NULL);
+				    sdr->ep.subdev->ctrl_handler, NULL, true);
 	if (ret) {
 		rdrif_err(sdr, "failed: ctrl add hdlr ret %d\n", ret);
 		goto error;
diff --git a/drivers/media/platform/soc_camera/soc_camera.c b/drivers/media/platform/soc_camera/soc_camera.c
index 1f3c450c7a69..0590a1caa784 100644
--- a/drivers/media/platform/soc_camera/soc_camera.c
+++ b/drivers/media/platform/soc_camera/soc_camera.c
@@ -1180,7 +1180,8 @@ static int soc_camera_probe_finish(struct soc_camera_device *icd)
 
 	v4l2_subdev_call(sd, video, g_tvnorms, &icd->vdev->tvnorms);
 
-	ret = v4l2_ctrl_add_handler(&icd->ctrl_handler, sd->ctrl_handler, NULL);
+	ret = v4l2_ctrl_add_handler(&icd->ctrl_handler, sd->ctrl_handler,
+				    NULL, true);
 	if (ret < 0)
 		return ret;
 
diff --git a/drivers/media/platform/vivid/vivid-ctrls.c b/drivers/media/platform/vivid/vivid-ctrls.c
index 34731f71cc00..d9bc00d3328a 100644
--- a/drivers/media/platform/vivid/vivid-ctrls.c
+++ b/drivers/media/platform/vivid/vivid-ctrls.c
@@ -1652,57 +1652,57 @@ int vivid_create_controls(struct vivid_dev *dev, bool show_ccs_cap,
 		v4l2_ctrl_auto_cluster(2, &dev->autogain, 0, true);
 
 	if (dev->has_vid_cap) {
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_vid, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_aud, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_streaming, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_sdtv_cap, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_loop_cap, NULL);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_vid, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_aud, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_streaming, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_sdtv_cap, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_loop_cap, NULL, false);
 		if (hdl_vid_cap->error)
 			return hdl_vid_cap->error;
 		dev->vid_cap_dev.ctrl_handler = hdl_vid_cap;
 	}
 	if (dev->has_vid_out) {
-		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_aud, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_out, hdl_streaming, NULL);
+		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_aud, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_out, hdl_streaming, NULL, false);
 		if (hdl_vid_out->error)
 			return hdl_vid_out->error;
 		dev->vid_out_dev.ctrl_handler = hdl_vid_out;
 	}
 	if (dev->has_vbi_cap) {
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_streaming, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_sdtv_cap, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_loop_cap, NULL);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_streaming, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_sdtv_cap, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_loop_cap, NULL, false);
 		if (hdl_vbi_cap->error)
 			return hdl_vbi_cap->error;
 		dev->vbi_cap_dev.ctrl_handler = hdl_vbi_cap;
 	}
 	if (dev->has_vbi_out) {
-		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_streaming, NULL);
+		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_streaming, NULL, false);
 		if (hdl_vbi_out->error)
 			return hdl_vbi_out->error;
 		dev->vbi_out_dev.ctrl_handler = hdl_vbi_out;
 	}
 	if (dev->has_radio_rx) {
-		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_aud, NULL);
+		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_aud, NULL, false);
 		if (hdl_radio_rx->error)
 			return hdl_radio_rx->error;
 		dev->radio_rx_dev.ctrl_handler = hdl_radio_rx;
 	}
 	if (dev->has_radio_tx) {
-		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_aud, NULL);
+		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_aud, NULL, false);
 		if (hdl_radio_tx->error)
 			return hdl_radio_tx->error;
 		dev->radio_tx_dev.ctrl_handler = hdl_radio_tx;
 	}
 	if (dev->has_sdr_cap) {
-		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_streaming, NULL);
+		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_streaming, NULL, false);
 		if (hdl_sdr_cap->error)
 			return hdl_sdr_cap->error;
 		dev->sdr_cap_dev.ctrl_handler = hdl_sdr_cap;
diff --git a/drivers/media/usb/cx231xx/cx231xx-417.c b/drivers/media/usb/cx231xx/cx231xx-417.c
index d538fa407742..ba8a4072633a 100644
--- a/drivers/media/usb/cx231xx/cx231xx-417.c
+++ b/drivers/media/usb/cx231xx/cx231xx-417.c
@@ -1991,7 +1991,7 @@ int cx231xx_417_register(struct cx231xx *dev)
 	dev->mpeg_ctrl_handler.ops = &cx231xx_ops;
 	if (dev->sd_cx25840)
 		v4l2_ctrl_add_handler(&dev->mpeg_ctrl_handler.hdl,
-				dev->sd_cx25840->ctrl_handler, NULL);
+				dev->sd_cx25840->ctrl_handler, NULL, false);
 	if (dev->mpeg_ctrl_handler.hdl.error) {
 		err = dev->mpeg_ctrl_handler.hdl.error;
 		dprintk(3, "%s: can't add cx25840 controls\n", dev->name);
diff --git a/drivers/media/usb/cx231xx/cx231xx-video.c b/drivers/media/usb/cx231xx/cx231xx-video.c
index 179b8481a870..f65aba3da985 100644
--- a/drivers/media/usb/cx231xx/cx231xx-video.c
+++ b/drivers/media/usb/cx231xx/cx231xx-video.c
@@ -2202,10 +2202,10 @@ int cx231xx_register_analog_devices(struct cx231xx *dev)
 
 	if (dev->sd_cx25840) {
 		v4l2_ctrl_add_handler(&dev->ctrl_handler,
-				dev->sd_cx25840->ctrl_handler, NULL);
+				dev->sd_cx25840->ctrl_handler, NULL, true);
 		v4l2_ctrl_add_handler(&dev->radio_ctrl_handler,
 				dev->sd_cx25840->ctrl_handler,
-				v4l2_ctrl_radio_filter);
+				v4l2_ctrl_radio_filter, true);
 	}
 
 	if (dev->ctrl_handler.error)
diff --git a/drivers/media/usb/msi2500/msi2500.c b/drivers/media/usb/msi2500/msi2500.c
index a097d3dbc141..3cdcaeb4a4ce 100644
--- a/drivers/media/usb/msi2500/msi2500.c
+++ b/drivers/media/usb/msi2500/msi2500.c
@@ -1278,7 +1278,7 @@ static int msi2500_probe(struct usb_interface *intf,
 	}
 
 	/* currently all controls are from subdev */
-	v4l2_ctrl_add_handler(&dev->hdl, sd->ctrl_handler, NULL);
+	v4l2_ctrl_add_handler(&dev->hdl, sd->ctrl_handler, NULL, true);
 
 	dev->v4l2_dev.ctrl_handler = &dev->hdl;
 	dev->vdev.v4l2_dev = &dev->v4l2_dev;
diff --git a/drivers/media/usb/tm6000/tm6000-video.c b/drivers/media/usb/tm6000/tm6000-video.c
index ec8c4d2534dc..1f88c31606c9 100644
--- a/drivers/media/usb/tm6000/tm6000-video.c
+++ b/drivers/media/usb/tm6000/tm6000-video.c
@@ -1638,7 +1638,7 @@ int tm6000_v4l2_register(struct tm6000_core *dev)
 	v4l2_ctrl_new_std(&dev->ctrl_handler, &tm6000_ctrl_ops,
 			V4L2_CID_HUE, -128, 127, 1, 0);
 	v4l2_ctrl_add_handler(&dev->ctrl_handler,
-			&dev->radio_ctrl_handler, NULL);
+			&dev->radio_ctrl_handler, NULL, false);
 
 	if (dev->radio_ctrl_handler.error)
 		ret = dev->radio_ctrl_handler.error;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index dd1db678718c..54444e3b8883 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1866,7 +1866,8 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
 
 /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
 static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
-			   struct v4l2_ctrl *ctrl)
+			   struct v4l2_ctrl *ctrl,
+			   bool from_other_dev)
 {
 	struct v4l2_ctrl_ref *ref;
 	struct v4l2_ctrl_ref *new_ref;
@@ -1890,6 +1891,7 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 	if (!new_ref)
 		return handler_set_err(hdl, -ENOMEM);
 	new_ref->ctrl = ctrl;
+	new_ref->from_other_dev = from_other_dev;
 	if (ctrl->handler == hdl) {
 		/* By default each control starts in a cluster of its own.
 		   new_ref->ctrl is basically a cluster array with one
@@ -2074,7 +2076,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 		ctrl->type_ops->init(ctrl, idx, ctrl->p_new);
 	}
 
-	if (handler_new_ref(hdl, ctrl)) {
+	if (handler_new_ref(hdl, ctrl, false)) {
 		kvfree(ctrl);
 		return NULL;
 	}
@@ -2243,7 +2245,8 @@ EXPORT_SYMBOL(v4l2_ctrl_new_int_menu);
 /* Add the controls from another handler to our own. */
 int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 			  struct v4l2_ctrl_handler *add,
-			  bool (*filter)(const struct v4l2_ctrl *ctrl))
+			  bool (*filter)(const struct v4l2_ctrl *ctrl),
+			  bool from_other_dev)
 {
 	struct v4l2_ctrl_ref *ref;
 	int ret = 0;
@@ -2266,7 +2269,7 @@ int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 		/* Filter any unwanted controls */
 		if (filter && !filter(ctrl))
 			continue;
-		ret = handler_new_ref(hdl, ctrl);
+		ret = handler_new_ref(hdl, ctrl, from_other_dev);
 		if (ret)
 			break;
 	}
diff --git a/drivers/media/v4l2-core/v4l2-device.c b/drivers/media/v4l2-core/v4l2-device.c
index 937c6de85606..8391a7f0895b 100644
--- a/drivers/media/v4l2-core/v4l2-device.c
+++ b/drivers/media/v4l2-core/v4l2-device.c
@@ -178,7 +178,8 @@ int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
 
 	sd->v4l2_dev = v4l2_dev;
 	/* This just returns 0 if either of the two args is NULL */
-	err = v4l2_ctrl_add_handler(v4l2_dev->ctrl_handler, sd->ctrl_handler, NULL);
+	err = v4l2_ctrl_add_handler(v4l2_dev->ctrl_handler, sd->ctrl_handler,
+				    NULL, true);
 	if (err)
 		goto error_module;
 
diff --git a/drivers/staging/media/imx/imx-media-dev.c b/drivers/staging/media/imx/imx-media-dev.c
index b55e5ebba8b4..08a1979adbdd 100644
--- a/drivers/staging/media/imx/imx-media-dev.c
+++ b/drivers/staging/media/imx/imx-media-dev.c
@@ -459,7 +459,7 @@ static int imx_media_inherit_controls(struct imx_media_dev *imxmd,
 
 		ret = v4l2_ctrl_add_handler(vfd->ctrl_handler,
 					    sd->ctrl_handler,
-					    NULL);
+					    NULL, true);
 		if (ret)
 			return ret;
 	}
diff --git a/drivers/staging/media/imx/imx-media-fim.c b/drivers/staging/media/imx/imx-media-fim.c
index 47275ef803f3..8372acbaf042 100644
--- a/drivers/staging/media/imx/imx-media-fim.c
+++ b/drivers/staging/media/imx/imx-media-fim.c
@@ -459,7 +459,7 @@ int imx_media_fim_add_controls(struct imx_media_fim *fim)
 {
 	/* add the FIM controls to the calling subdev ctrl handler */
 	return v4l2_ctrl_add_handler(fim->sd->ctrl_handler,
-				     &fim->ctrl_handler, NULL);
+				     &fim->ctrl_handler, NULL, false);
 }
 EXPORT_SYMBOL_GPL(imx_media_fim_add_controls);
 
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index dacfe54057f8..a762f3392d90 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -250,6 +250,7 @@ struct v4l2_ctrl_ref {
 	struct v4l2_ctrl_ref *next;
 	struct v4l2_ctrl *ctrl;
 	struct v4l2_ctrl_helper *helper;
+	bool from_other_dev;
 };
 
 /**
@@ -635,7 +636,8 @@ typedef bool (*v4l2_ctrl_filter)(const struct v4l2_ctrl *ctrl);
  */
 int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 			  struct v4l2_ctrl_handler *add,
-			  v4l2_ctrl_filter filter);
+			  v4l2_ctrl_filter filter,
+			  bool from_other_dev);
 
 /**
  * v4l2_ctrl_radio_filter() - Standard filter for radio controls.
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 07/17] v4l2-ctrls: prepare internal structs for request API

[Alexandre Courbot]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add a refcount and is_request bool to struct v4l2_ctrl_handler:
this is used to refcount a handler that represents a request.

Add a p_req field to struct v4l2_ctrl_ref that will store the
request value.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 1 +
 include/media/v4l2-ctrls.h           | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 54444e3b8883..9d99679b543c 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1751,6 +1751,7 @@ int v4l2_ctrl_handler_init_class(struct v4l2_ctrl_handler *hdl,
 				      sizeof(hdl->buckets[0]),
 				      GFP_KERNEL | __GFP_ZERO);
 	hdl->error = hdl->buckets ? 0 : -ENOMEM;
+	hdl->is_request = false;
 	return hdl->error;
 }
 EXPORT_SYMBOL(v4l2_ctrl_handler_init_class);
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index a762f3392d90..a215f25a82cf 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -18,6 +18,7 @@
 #define _V4L2_CTRLS_H
 
 #include <linux/list.h>
+#include <linux/kref.h>
 #include <linux/mutex.h>
 #include <linux/videodev2.h>
 
@@ -250,6 +251,7 @@ struct v4l2_ctrl_ref {
 	struct v4l2_ctrl_ref *next;
 	struct v4l2_ctrl *ctrl;
 	struct v4l2_ctrl_helper *helper;
+	union v4l2_ctrl_ptr p_req;
 	bool from_other_dev;
 };
 
@@ -285,7 +287,9 @@ struct v4l2_ctrl_handler {
 	v4l2_ctrl_notify_fnc notify;
 	void *notify_priv;
 	u16 nr_of_buckets;
+	bool is_request;
 	int error;
+	struct kref ref;
 };
 
 /**
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 08/17] v4l2-ctrls: add core request API

[Alexandre Courbot]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add the four core request functions:

v4l2_ctrl_request_init() initializes a new (empty) request.
v4l2_ctrl_request_clone() resets a request based on another request
(or clears it if that request is NULL).
v4l2_ctrl_request_get(): increase refcount
v4l2_ctrl_request_put(): decrease refcount and delete if it reaches 0.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
[acourbot@chromium.org: turn v4l2_ctrl_request_alloc into init function]
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 106 ++++++++++++++++++++++++++++++++++-
 include/media/v4l2-ctrls.h           |   7 +++
 2 files changed, 110 insertions(+), 3 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 9d99679b543c..ca4c320f64c9 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1868,6 +1868,7 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
 /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
 static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 			   struct v4l2_ctrl *ctrl,
+			   struct v4l2_ctrl_ref **ctrl_ref,
 			   bool from_other_dev)
 {
 	struct v4l2_ctrl_ref *ref;
@@ -1875,6 +1876,10 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 	u32 id = ctrl->id;
 	u32 class_ctrl = V4L2_CTRL_ID2WHICH(id) | 1;
 	int bucket = id % hdl->nr_of_buckets;	/* which bucket to use */
+	unsigned int sz_extra = 0;
+
+	if (ctrl_ref)
+		*ctrl_ref = NULL;
 
 	/*
 	 * Automatically add the control class if it is not yet present and
@@ -1888,11 +1893,16 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 	if (hdl->error)
 		return hdl->error;
 
-	new_ref = kzalloc(sizeof(*new_ref), GFP_KERNEL);
+	if (hdl->is_request)
+		sz_extra = ctrl->elems * ctrl->elem_size;
+	new_ref = kzalloc(sizeof(*new_ref) + sz_extra, GFP_KERNEL);
 	if (!new_ref)
 		return handler_set_err(hdl, -ENOMEM);
 	new_ref->ctrl = ctrl;
 	new_ref->from_other_dev = from_other_dev;
+	if (sz_extra)
+		new_ref->p_req.p = &new_ref[1];
+
 	if (ctrl->handler == hdl) {
 		/* By default each control starts in a cluster of its own.
 		   new_ref->ctrl is basically a cluster array with one
@@ -1932,6 +1942,8 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 	/* Insert the control node in the hash */
 	new_ref->next = hdl->buckets[bucket];
 	hdl->buckets[bucket] = new_ref;
+	if (ctrl_ref)
+		*ctrl_ref = new_ref;
 
 unlock:
 	mutex_unlock(hdl->lock);
@@ -2077,7 +2089,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 		ctrl->type_ops->init(ctrl, idx, ctrl->p_new);
 	}
 
-	if (handler_new_ref(hdl, ctrl, false)) {
+	if (handler_new_ref(hdl, ctrl, NULL, false)) {
 		kvfree(ctrl);
 		return NULL;
 	}
@@ -2270,7 +2282,7 @@ int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 		/* Filter any unwanted controls */
 		if (filter && !filter(ctrl))
 			continue;
-		ret = handler_new_ref(hdl, ctrl, from_other_dev);
+		ret = handler_new_ref(hdl, ctrl, NULL, from_other_dev);
 		if (ret)
 			break;
 	}
@@ -2681,6 +2693,94 @@ int v4l2_querymenu(struct v4l2_ctrl_handler *hdl, struct v4l2_querymenu *qm)
 }
 EXPORT_SYMBOL(v4l2_querymenu);
 
+int v4l2_ctrl_request_init(struct v4l2_ctrl_handler *hdl)
+{
+	int err;
+
+	err = v4l2_ctrl_handler_init(hdl, 0);
+	if (err)
+		return err;
+	hdl->is_request = true;
+	kref_init(&hdl->ref);
+
+	return 0;
+}
+EXPORT_SYMBOL(v4l2_ctrl_request_init);
+
+int v4l2_ctrl_request_clone(struct v4l2_ctrl_handler *hdl,
+			    const struct v4l2_ctrl_handler *from,
+			    bool (*filter)(const struct v4l2_ctrl *ctrl))
+{
+	struct v4l2_ctrl_ref *ref;
+	int err;
+
+	if (WARN_ON(!hdl || hdl == from))
+		return -EINVAL;
+
+	if (hdl->error)
+		return hdl->error;
+
+	WARN_ON(hdl->lock != &hdl->_lock);
+	v4l2_ctrl_handler_free(hdl);
+	err = v4l2_ctrl_handler_init(hdl, (from->nr_of_buckets - 1) * 8);
+	hdl->is_request = true;
+	if (err)
+		return err;
+	if (!from)
+		return 0;
+
+	mutex_lock(from->lock);
+	list_for_each_entry(ref, &from->ctrl_refs, node) {
+		struct v4l2_ctrl *ctrl = ref->ctrl;
+		struct v4l2_ctrl_ref *new_ref;
+
+		/* Skip refs inherited from other devices */
+		if (ref->from_other_dev)
+			continue;
+		/* And buttons and control classes */
+		if (ctrl->type == V4L2_CTRL_TYPE_BUTTON ||
+		    ctrl->type == V4L2_CTRL_TYPE_CTRL_CLASS)
+			continue;
+		/* Filter any unwanted controls */
+		if (filter && !filter(ctrl))
+			continue;
+		err = handler_new_ref(hdl, ctrl, &new_ref, false);
+		if (err)
+			break;
+		if (from->is_request)
+			ptr_to_ptr(ctrl, ref->p_req, new_ref->p_req);
+		else
+			ptr_to_ptr(ctrl, ctrl->p_cur, new_ref->p_req);
+	}
+	mutex_unlock(from->lock);
+	return err;
+}
+EXPORT_SYMBOL(v4l2_ctrl_request_clone);
+
+void v4l2_ctrl_request_get(struct v4l2_ctrl_handler *hdl)
+{
+	if (WARN_ON(!hdl->is_request))
+		return;
+	kref_get(&hdl->ref);
+}
+EXPORT_SYMBOL(v4l2_ctrl_request_get);
+
+static void v4l2_ctrl_request_release(struct kref *kref)
+{
+	struct v4l2_ctrl_handler *hdl =
+		container_of(kref, struct v4l2_ctrl_handler, ref);
+
+	v4l2_ctrl_handler_free(hdl);
+	kfree(hdl);
+}
+
+void v4l2_ctrl_request_put(struct v4l2_ctrl_handler *hdl)
+{
+	if (WARN_ON(!hdl->is_request))
+		return;
+	kref_put(&hdl->ref, v4l2_ctrl_request_release);
+}
+EXPORT_SYMBOL(v4l2_ctrl_request_put);
 
 /* Some general notes on the atomic requirements of VIDIOC_G/TRY/S_EXT_CTRLS:
 
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index a215f25a82cf..24a60ca1a0dd 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -1045,6 +1045,13 @@ int v4l2_ctrl_subscribe_event(struct v4l2_fh *fh,
  */
 unsigned int v4l2_ctrl_poll(struct file *file, struct poll_table_struct *wait);
 
+int v4l2_ctrl_request_init(struct v4l2_ctrl_handler *hdl);
+int v4l2_ctrl_request_clone(struct v4l2_ctrl_handler *hdl,
+			    const struct v4l2_ctrl_handler *from,
+			    bool (*filter)(const struct v4l2_ctrl *ctrl));
+void v4l2_ctrl_request_get(struct v4l2_ctrl_handler *hdl);
+void v4l2_ctrl_request_put(struct v4l2_ctrl_handler *hdl);
+
 /* Helpers for ioctl_ops */
 
 /**
-- 
2.16.0.rc1.238.g530d649a79-goog

[RFCv2 09/17] v4l2-ctrls: use ref in helper instead of ctrl

[Alexandre Courbot]

From: Hans Verkuil <hans.verkuil@cisco.com>

The next patch needs the reference to a control instead of the
control itself, so change struct v4l2_ctrl_helper accordingly.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index ca4c320f64c9..0fa20c89ece1 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -37,8 +37,8 @@
 struct v4l2_ctrl_helper {
 	/* Pointer to the control reference of the master control */
 	struct v4l2_ctrl_ref *mref;
-	/* The control corresponding to the v4l2_ext_control ID field. */
-	struct v4l2_ctrl *ctrl;
+	/* The control ref corresponding to the v4l2_ext_control ID field. */
+	struct v4l2_ctrl_ref *ref;
 	/* v4l2_ext_control index of the next control belonging to the
 	   same cluster, or 0 if there isn't any. */
 	u32 next;
@@ -2852,6 +2852,7 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
 		ref = find_ref_lock(hdl, id);
 		if (ref == NULL)
 			return -EINVAL;
+		h->ref = ref;
 		ctrl = ref->ctrl;
 		if (ctrl->flags & V4L2_CTRL_FLAG_DISABLED)
 			return -EINVAL;
@@ -2874,7 +2875,6 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
 		}
 		/* Store the ref to the master control of the cluster */
 		h->mref = ref;
-		h->ctrl = ctrl;
 		/* Initially set next to 0, meaning that there is no other
 		   control in this helper array belonging to the same
 		   cluster */
@@ -2959,7 +2959,7 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
 	cs->error_idx = cs->count;
 
 	for (i = 0; !ret && i < cs->count; i++)
-		if (helpers[i].ctrl->flags & V4L2_CTRL_FLAG_WRITE_ONLY)
+		if (helpers[i].ref->ctrl->flags & V4L2_CTRL_FLAG_WRITE_ONLY)
 			ret = -EACCES;
 
 	for (i = 0; !ret && i < cs->count; i++) {
@@ -2994,7 +2994,7 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
 
 			do {
 				ret = ctrl_to_user(cs->controls + idx,
-						   helpers[idx].ctrl);
+						   helpers[idx].ref->ctrl);
 				idx = helpers[idx].next;
 			} while (!ret && idx);
 		}
@@ -3133,7 +3133,7 @@ static int validate_ctrls(struct v4l2_ext_controls *cs,
 
 	cs->error_idx = cs->count;
 	for (i = 0; i < cs->count; i++) {
-		struct v4l2_ctrl *ctrl = helpers[i].ctrl;
+		struct v4l2_ctrl *ctrl = helpers[i].ref->ctrl;
 		union v4l2_ctrl_ptr p_new;
 
 		cs->error_idx = i;
@@ -3245,7 +3245,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 			do {
 				/* Check if the auto control is part of the
 				   list, and remember the new value. */
-				if (helpers[tmp_idx].ctrl == master)
+				if (helpers[tmp_idx].ref->ctrl == master)
 					new_auto_val = cs->controls[tmp_idx].value;
 				tmp_idx = helpers[tmp_idx].next;
 			} while (tmp_idx);
@@ -3258,7 +3258,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 		/* Copy the new caller-supplied control values.
 		   user_to_new() sets 'is_new' to 1. */
 		do {
-			struct v4l2_ctrl *ctrl = helpers[idx].ctrl;
+			struct v4l2_ctrl *ctrl = helpers[idx].ref->ctrl;
 
 			ret = user_to_new(cs->controls + idx, ctrl);
 			if (!ret && ctrl->is_ptr)
@@ -3274,7 +3274,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 			idx = i;
 			do {
 				ret = new_to_user(cs->controls + idx,
-						helpers[idx].ctrl);
+						helpers[idx].ref->ctrl);
 				idx = helpers[idx].next;
 			} while (!ret && idx);
 		}
-- 
2.16.0.rc1.238.g530d649a79-goog

Re: [RFCv2 05/17] media: Document the media request API

[Randy Dunlap]

On 01/31/2018 02:24 AM, Alexandre Courbot wrote:
> From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> 
> The media request API is made of a new ioctl to implement request
> management. Document it.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> [acourbot@chromium.org: adapt for newest API]
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  Documentation/media/uapi/mediactl/media-funcs.rst  |   1 +
>  .../media/uapi/mediactl/media-ioc-request-cmd.rst  | 141 +++++++++++++++++++++
>  2 files changed, 142 insertions(+)
>  create mode 100644 Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
> 
> diff --git a/Documentation/media/uapi/mediactl/media-funcs.rst b/Documentation/media/uapi/mediactl/media-funcs.rst
> index 076856501cdb..e3a45d82ffcb 100644
> --- a/Documentation/media/uapi/mediactl/media-funcs.rst
> +++ b/Documentation/media/uapi/mediactl/media-funcs.rst
> @@ -15,4 +15,5 @@ Function Reference
>      media-ioc-g-topology
>      media-ioc-enum-entities
>      media-ioc-enum-links
> +    media-ioc-request-cmd
>      media-ioc-setup-link
> diff --git a/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst b/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
> new file mode 100644
> index 000000000000..723b422afcce
> --- /dev/null
> +++ b/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
> @@ -0,0 +1,141 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _media_ioc_request_cmd:
> +
> +***************************
> +ioctl MEDIA_IOC_REQUEST_CMD
> +***************************
> +
> +Name
> +====
> +
> +MEDIA_IOC_REQUEST_CMD - Manage media device requests
> +
> +
> +Synopsis
> +========
> +
> +.. c:function:: int ioctl( int fd, MEDIA_IOC_REQUEST_CMD, struct media_request_cmd *argp )
> +    :name: MEDIA_IOC_REQUEST_CMD
> +
> +
> +Arguments
> +=========
> +
> +``fd``
> +    File descriptor returned by :ref:`open() <media-func-open>`.
> +
> +``argp``
> +
> +
> +Description
> +===========
> +
> +The MEDIA_IOC_REQUEST_CMD ioctl allow applications to manage media device

                                   allows

> +requests. A request is an object that can group media device configuration
> +parameters, including subsystem-specific parameters, in order to apply all the
> +parameters atomically. Applications are responsible for allocating and
> +deleting requests, filling them with configuration parameters submitting them.

                                                                and submitting them.

> +
> +Request operations are performed by calling the MEDIA_IOC_REQUEST_CMD ioctl
> +with a pointer to a struct :c:type:`media_request_cmd` with the cmd field set
> +to the appropriate command. :ref:`media-request-command` lists the commands
> +supported by the ioctl.
> +
> +The struct :c:type:`media_request_cmd` request field contains the file
> +descriptorof the request on which the command operates. For the

   descriptor of

> +``MEDIA_REQ_CMD_ALLOC`` command the field is set to zero by applications and
> +filled by the driver. For all other commands the field is set by applications
> +and left untouched by the driver.
> +
> +To allocate a new request applications use the ``MEDIA_REQ_CMD_ALLOC``
> +command. The driver will allocate a new request and return its FD in the
> +request field. After allocation, the request is "empty", which means that it
> +does not hold any state of its own, and that the hardware's state will not be
> +affected by it unless it is passed as argument to V4L2 or media controller
> +commands.
> +
> +Requests are reference-counted. A newly allocated request is referenced
> +by the returned file descriptor, and can be later referenced by
> +subsystem-specific operations. Requests will thus be automatically deleted
> +when they're not longer used after the returned file descriptor is closed.

                 no longer

> +
> +If a request isn't needed applications can delete it by calling ``close()``
> +on it. The driver will drop the file handle reference. The request will not
> +be usable through the MEDIA_IOC_REQUEST_CMD ioctl anymore, but will only be
> +deleted when the last reference is released. If no other reference exists when
> +``close()`` is invoked the request will be deleted immediately.
> +
> +After creating a request applications should fill it with configuration
> +parameters. This is performed through subsystem-specific request APIs outside
> +the scope of the media controller API. See the appropriate subsystem APIs for
> +more information, including how they interact with the MEDIA_IOC_REQUEST_CMD
> +ioctl.
> +
> +Once a request contains all the desired configuration parameters it can be
> +submitted using the ``MEDIA_REQ_CMD_SUBMIT`` command. This will let the
> +buffers queued for the request be passed to their respective drivers, which
> +will then apply the request's parameters before processing them.
> +
> +Once a request has been queued applications are not allowed to modify its
> +configuration parameters until the request has been fully processed. Any
> +attempt to do so will result in the related subsystem API returning an error.
> +The application that submitted the request can wait for its completion by
> +polling on the request's file descriptor.
> +
> +Once a request has completed, it can be reused. The ``MEDIA_REQ_CMD_REINIT``
> +command will bring it back to its initial state, so it can be prepared and
> +submitted again.
> +
> +.. c:type:: media_request_cmd
> +
> +.. flat-table:: struct media_request_cmd
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths: 1 2 8
> +
> +    * - __u32
> +      - ``cmd``
> +      - Command, set by the application. See below for the list of supported
> +        commands.
> +    * - __u32
> +      - ``fd``
> +      - Request FD, set by the driver for the MEDIA_REQ_CMD_ALLOC command and
> +        by the application for all other commands.
> +
> +
> +.. _media-request-command:
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: Media request commands
> +    :header-rows:  0
> +    :stub-columns: 0
> +
> +    * .. _MEDIA-REQ-CMD-ALLOC:
> +
> +      - ``MEDIA_REQ_CMD_ALLOC``
> +      - Allocate a new request.
> +    * .. _MEDIA-REQ-CMD-SUBMIT:
> +
> +      - ``MEDIA_REQ_CMD_SUBMIT``
> +      - Submit a request to be processed.
> +    * .. _MEDIA-REQ-CMD-QUEUE:
> +
> +      - ``MEDIA_REQ_CMD_REINIT``
> +      - Reinitializes a completed request.

           Reinitialize
to be consistent with other entries above.

> +
> +
> +Return Value
> +============
> +
> +On success 0 is returned, on error -1 and the ``errno`` variable is set
> +appropriately. The generic error codes are described at the
> +:ref:`Generic Error Codes <gen-errors>` chapter.
> +
> +EINVAL
> +    The struct :c:type:`media_request_cmd` specifies an invalid command or
> +    references a non-existing request.
> +
> +ENOSYS
> +    Request API is not available on this device.
> 


-- 
~Randy

Re: [RFCv2 05/17] media: Document the media request API

[Alexandre Courbot]

Hi Randy,

On Fri, Feb 2, 2018 at 3:14 AM, Randy Dunlap <rdunlap@infradead.org> wrote:
> On 01/31/2018 02:24 AM, Alexandre Courbot wrote:
>> From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
>>
>> The media request API is made of a new ioctl to implement request
>> management. Document it.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
>> [acourbot@chromium.org: adapt for newest API]
>> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
>> ---
>>  Documentation/media/uapi/mediactl/media-funcs.rst  |   1 +
>>  .../media/uapi/mediactl/media-ioc-request-cmd.rst  | 141 +++++++++++++++++++++
>>  2 files changed, 142 insertions(+)
>>  create mode 100644 Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
>>
>> diff --git a/Documentation/media/uapi/mediactl/media-funcs.rst b/Documentation/media/uapi/mediactl/media-funcs.rst
>> index 076856501cdb..e3a45d82ffcb 100644
>> --- a/Documentation/media/uapi/mediactl/media-funcs.rst
>> +++ b/Documentation/media/uapi/mediactl/media-funcs.rst
>> @@ -15,4 +15,5 @@ Function Reference
>>      media-ioc-g-topology
>>      media-ioc-enum-entities
>>      media-ioc-enum-links
>> +    media-ioc-request-cmd
>>      media-ioc-setup-link
>> diff --git a/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst b/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
>> new file mode 100644
>> index 000000000000..723b422afcce
>> --- /dev/null
>> +++ b/Documentation/media/uapi/mediactl/media-ioc-request-cmd.rst
>> @@ -0,0 +1,141 @@
>> +.. -*- coding: utf-8; mode: rst -*-
>> +
>> +.. _media_ioc_request_cmd:
>> +
>> +***************************
>> +ioctl MEDIA_IOC_REQUEST_CMD
>> +***************************
>> +
>> +Name
>> +====
>> +
>> +MEDIA_IOC_REQUEST_CMD - Manage media device requests
>> +
>> +
>> +Synopsis
>> +========
>> +
>> +.. c:function:: int ioctl( int fd, MEDIA_IOC_REQUEST_CMD, struct media_request_cmd *argp )
>> +    :name: MEDIA_IOC_REQUEST_CMD
>> +
>> +
>> +Arguments
>> +=========
>> +
>> +``fd``
>> +    File descriptor returned by :ref:`open() <media-func-open>`.
>> +
>> +``argp``
>> +
>> +
>> +Description
>> +===========
>> +
>> +The MEDIA_IOC_REQUEST_CMD ioctl allow applications to manage media device
>
>                                    allows
>
>> +requests. A request is an object that can group media device configuration
>> +parameters, including subsystem-specific parameters, in order to apply all the
>> +parameters atomically. Applications are responsible for allocating and
>> +deleting requests, filling them with configuration parameters submitting them.
>
>                                                                 and submitting them.
>
>> +
>> +Request operations are performed by calling the MEDIA_IOC_REQUEST_CMD ioctl
>> +with a pointer to a struct :c:type:`media_request_cmd` with the cmd field set
>> +to the appropriate command. :ref:`media-request-command` lists the commands
>> +supported by the ioctl.
>> +
>> +The struct :c:type:`media_request_cmd` request field contains the file
>> +descriptorof the request on which the command operates. For the
>
>    descriptor of
>
>> +``MEDIA_REQ_CMD_ALLOC`` command the field is set to zero by applications and
>> +filled by the driver. For all other commands the field is set by applications
>> +and left untouched by the driver.
>> +
>> +To allocate a new request applications use the ``MEDIA_REQ_CMD_ALLOC``
>> +command. The driver will allocate a new request and return its FD in the
>> +request field. After allocation, the request is "empty", which means that it
>> +does not hold any state of its own, and that the hardware's state will not be
>> +affected by it unless it is passed as argument to V4L2 or media controller
>> +commands.
>> +
>> +Requests are reference-counted. A newly allocated request is referenced
>> +by the returned file descriptor, and can be later referenced by
>> +subsystem-specific operations. Requests will thus be automatically deleted
>> +when they're not longer used after the returned file descriptor is closed.
>
>                  no longer
>
>> +
>> +If a request isn't needed applications can delete it by calling ``close()``
>> +on it. The driver will drop the file handle reference. The request will not
>> +be usable through the MEDIA_IOC_REQUEST_CMD ioctl anymore, but will only be
>> +deleted when the last reference is released. If no other reference exists when
>> +``close()`` is invoked the request will be deleted immediately.
>> +
>> +After creating a request applications should fill it with configuration
>> +parameters. This is performed through subsystem-specific request APIs outside
>> +the scope of the media controller API. See the appropriate subsystem APIs for
>> +more information, including how they interact with the MEDIA_IOC_REQUEST_CMD
>> +ioctl.
>> +
>> +Once a request contains all the desired configuration parameters it can be
>> +submitted using the ``MEDIA_REQ_CMD_SUBMIT`` command. This will let the
>> +buffers queued for the request be passed to their respective drivers, which
>> +will then apply the request's parameters before processing them.
>> +
>> +Once a request has been queued applications are not allowed to modify its
>> +configuration parameters until the request has been fully processed. Any
>> +attempt to do so will result in the related subsystem API returning an error.
>> +The application that submitted the request can wait for its completion by
>> +polling on the request's file descriptor.
>> +
>> +Once a request has completed, it can be reused. The ``MEDIA_REQ_CMD_REINIT``
>> +command will bring it back to its initial state, so it can be prepared and
>> +submitted again.
>> +
>> +.. c:type:: media_request_cmd
>> +
>> +.. flat-table:: struct media_request_cmd
>> +    :header-rows:  0
>> +    :stub-columns: 0
>> +    :widths: 1 2 8
>> +
>> +    * - __u32
>> +      - ``cmd``
>> +      - Command, set by the application. See below for the list of supported
>> +        commands.
>> +    * - __u32
>> +      - ``fd``
>> +      - Request FD, set by the driver for the MEDIA_REQ_CMD_ALLOC command and
>> +        by the application for all other commands.
>> +
>> +
>> +.. _media-request-command:
>> +
>> +.. cssclass:: longtable
>> +
>> +.. flat-table:: Media request commands
>> +    :header-rows:  0
>> +    :stub-columns: 0
>> +
>> +    * .. _MEDIA-REQ-CMD-ALLOC:
>> +
>> +      - ``MEDIA_REQ_CMD_ALLOC``
>> +      - Allocate a new request.
>> +    * .. _MEDIA-REQ-CMD-SUBMIT:
>> +
>> +      - ``MEDIA_REQ_CMD_SUBMIT``
>> +      - Submit a request to be processed.
>> +    * .. _MEDIA-REQ-CMD-QUEUE:
>> +
>> +      - ``MEDIA_REQ_CMD_REINIT``
>> +      - Reinitializes a completed request.
>
>            Reinitialize
> to be consistent with other entries above.

Applied all these fixes, thanks!

Re: [RFCv2 00/17] Request API, take three

[Sakari Ailus]

Hi Alexandre,

On Wed, Jan 31, 2018 at 07:24:18PM +0900, Alexandre Courbot wrote:
> This is a quickly-put together revision that includes and uses Hans' work to
> use v4l2_ctrl_handler as the request state holder for V4L2 devices. Although
> minor fixes have also been applied, there are still a few comments from the
> previous revision that are left unaddressed. I wanted to give Hans something
> to play with before he forgets what he had in mind for controls. ;)

Could you rebase this on the current media master, please?

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

Re: [RFCv2 02/17] videodev2.h: Add request_fd field to v4l2_buffer

[Sakari Ailus]

Hi Alexandre,

On Wed, Jan 31, 2018 at 07:24:20PM +0900, Alexandre Courbot wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> When queuing buffers allow for passing the request that should
> be associated with this buffer.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> [acourbot@chromium.org: make request ID 32-bit]
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  drivers/media/usb/cpia2/cpia2_v4l.c           | 2 +-
>  drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 7 ++++---
>  drivers/media/v4l2-core/v4l2-ioctl.c          | 4 ++--
>  drivers/media/v4l2-core/videobuf2-v4l2.c      | 3 ++-
>  include/media/videobuf2-v4l2.h                | 2 ++
>  include/uapi/linux/videodev2.h                | 3 ++-
>  6 files changed, 13 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/media/usb/cpia2/cpia2_v4l.c b/drivers/media/usb/cpia2/cpia2_v4l.c
> index 3dedd83f0b19..7acb6807b306 100644
> --- a/drivers/media/usb/cpia2/cpia2_v4l.c
> +++ b/drivers/media/usb/cpia2/cpia2_v4l.c
> @@ -948,7 +948,7 @@ static int cpia2_dqbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
>  	buf->sequence = cam->buffers[buf->index].seq;
>  	buf->m.offset = cam->buffers[buf->index].data - cam->frame_buffer;
>  	buf->length = cam->frame_size;
> -	buf->reserved2 = 0;
> +	buf->request_fd = 0;
>  	buf->reserved = 0;
>  	memset(&buf->timecode, 0, sizeof(buf->timecode));
>  
> diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> index 821f2aa299ae..7e4440950c76 100644
> --- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> +++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> @@ -370,7 +370,7 @@ struct v4l2_buffer32 {
>  		__s32		fd;
>  	} m;
>  	__u32			length;
> -	__u32			reserved2;
> +	__u32			request_fd;

Please use __s32 for file descriptors.

>  	__u32			reserved;
>  };
>  
> @@ -438,7 +438,8 @@ static int get_v4l2_buffer32(struct v4l2_buffer *kp, struct v4l2_buffer32 __user
>  		get_user(kp->type, &up->type) ||
>  		get_user(kp->flags, &up->flags) ||
>  		get_user(kp->memory, &up->memory) ||
> -		get_user(kp->length, &up->length))
> +		get_user(kp->length, &up->length) ||
> +		get_user(kp->request_fd, &up->request_fd))
>  			return -EFAULT;
>  
>  	if (V4L2_TYPE_IS_OUTPUT(kp->type))
> @@ -533,7 +534,7 @@ static int put_v4l2_buffer32(struct v4l2_buffer *kp, struct v4l2_buffer32 __user
>  		put_user(kp->timestamp.tv_usec, &up->timestamp.tv_usec) ||
>  		copy_to_user(&up->timecode, &kp->timecode, sizeof(struct v4l2_timecode)) ||
>  		put_user(kp->sequence, &up->sequence) ||
> -		put_user(kp->reserved2, &up->reserved2) ||
> +		put_user(kp->request_fd, &up->request_fd) ||
>  		put_user(kp->reserved, &up->reserved) ||
>  		put_user(kp->length, &up->length))
>  			return -EFAULT;
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index ec4ecd5aa8bf..fdd2f784c264 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -437,13 +437,13 @@ static void v4l_print_buffer(const void *arg, bool write_only)
>  	const struct v4l2_plane *plane;
>  	int i;
>  
> -	pr_cont("%02ld:%02d:%02d.%08ld index=%d, type=%s, flags=0x%08x, field=%s, sequence=%d, memory=%s",
> +	pr_cont("%02ld:%02d:%02d.%08ld index=%d, type=%s, request_fd=%u, flags=0x%08x, field=%s, sequence=%d, memory=%s",
>  			p->timestamp.tv_sec / 3600,
>  			(int)(p->timestamp.tv_sec / 60) % 60,
>  			(int)(p->timestamp.tv_sec % 60),
>  			(long)p->timestamp.tv_usec,
>  			p->index,
> -			prt_names(p->type, v4l2_type_names),
> +			prt_names(p->type, v4l2_type_names), p->request_fd,
>  			p->flags, prt_names(p->field, v4l2_field_names),
>  			p->sequence, prt_names(p->memory, v4l2_memory_names));
>  
> diff --git a/drivers/media/v4l2-core/videobuf2-v4l2.c b/drivers/media/v4l2-core/videobuf2-v4l2.c
> index 0c0669976bdc..0f8edbdebe30 100644
> --- a/drivers/media/v4l2-core/videobuf2-v4l2.c
> +++ b/drivers/media/v4l2-core/videobuf2-v4l2.c
> @@ -203,7 +203,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
>  	b->timestamp = ns_to_timeval(vb->timestamp);
>  	b->timecode = vbuf->timecode;
>  	b->sequence = vbuf->sequence;
> -	b->reserved2 = 0;
> +	b->request_fd = vbuf->request_fd;
>  	b->reserved = 0;
>  
>  	if (q->is_multiplanar) {
> @@ -320,6 +320,7 @@ static int __fill_vb2_buffer(struct vb2_buffer *vb,
>  	}
>  	vb->timestamp = 0;
>  	vbuf->sequence = 0;
> +	vbuf->request_fd = b->request_fd;
>  
>  	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
>  		if (b->memory == VB2_MEMORY_USERPTR) {
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 036127c54bbf..d7cf4c66db38 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -31,6 +31,7 @@
>   * @field:	enum v4l2_field; field order of the image in the buffer
>   * @timecode:	frame timecode
>   * @sequence:	sequence count of this frame
> + * @request_fd:	fd of the request used by the buffer
>   *
>   * Should contain enough information to be able to cover all the fields
>   * of struct v4l2_buffer at videodev2.h
> @@ -42,6 +43,7 @@ struct vb2_v4l2_buffer {
>  	__u32			field;
>  	struct v4l2_timecode	timecode;
>  	__u32			sequence;
> +	__u32			request_fd;

Ditto.

>  };
>  
>  /*
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 1c095b5a99c5..89bd716c66a6 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -902,6 +902,7 @@ struct v4l2_plane {
>   * @length:	size in bytes of the buffer (NOT its payload) for single-plane
>   *		buffers (when type != *_MPLANE); number of elements in the
>   *		planes array for multi-plane buffers
> + * @request_fd: fd of the request that this buffer should use
>   *
>   * Contains data exchanged by application and driver using one of the Streaming
>   * I/O methods.
> @@ -925,7 +926,7 @@ struct v4l2_buffer {
>  		__s32		fd;
>  	} m;
>  	__u32			length;
> -	__u32			reserved2;
> +	__s32			request_fd;
>  	__u32			reserved;
>  };
>  

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv2 00/17] Request API, take three

[Hans Verkuil]

Hi Alexandre,

On 01/31/18 11:24, Alexandre Courbot wrote:
> This is a quickly-put together revision that includes and uses Hans' work to
> use v4l2_ctrl_handler as the request state holder for V4L2 devices. Although
> minor fixes have also been applied, there are still a few comments from the
> previous revision that are left unaddressed. I wanted to give Hans something
> to play with before he forgets what he had in mind for controls. ;)
> 
> Changelog since v1:
> * Integrate Hans control framework patches so S_EXT_CTRLS and G_EXT_CTRLS now
>   work with requests
> * Only allow one buffer at a time for a given request in the buffer queue
> * Applied comments related to documentation and document control ioctls
> * Minor small fixes
> 
> I have also updated the very basic program that demonstrates the use of the
> request API on vim2m:
> 
> https://gist.github.com/Gnurou/dbc3776ed97ea7d4ce6041ea15eb0438
> 
> It does not do much, but gives a practical idea of how requests should be used.

Can you rebase and repost? It doesn't apply anymore to the latest media_tree master.
Among others v4l2-core/videobuf2-* has been moved to common/videobuf2/videobuf2-*.

But there are also conflicts in v4l2-compat-ioctl32.c.

Thanks!

	Hans