[RFCv11 PATCH 00/29] Request API

[RFCv11 PATCH 00/29] Request API

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Hi all,

This is a cleaned up version of the v10 series (never posted to
the list since it was messy).

The main changes compared to v9 are in the control framework which
is (hopefully!) now in sync with the RFC. Specifically, when queueing
a request it will 'chain' itself correctly with the previously queued
request. Control values that were not explicitly set in the request
will get the value from the first request in the queue that sets it,
or the hardware value if no request in the queue ever touches it.

However, I have not yet had the opportunity to test this in
v4l2-compliance!

Sakari: I did not have the time to incorporate your comments. I'll
probably wait until I have more feedback and then post a new series
next week.

Other changes:

- various request state checks were missing (i.e. you could set
  a control in a queued request).

- a new cancel op was added to handle the corner case where a
  request was queued but never reached the driver since STREAMOFF
  was called before the buffers were ever queued in the driver.

- various random fixes.

- added the patch "videobuf2-v4l2: export request_fd" as requested
  by Sakari.

- changed some inconsistent error codes.

This has been tested with vim2m and vivid using v4l2-compliance.
The v4l-utils repo supporting requests is here:
https://git.linuxtv.org/hverkuil/v4l-utils.git/log/?h=request

TODO/Remarks:

1) missing prototype documentation in media-requests.c/h. Some
   is documented, but not everything.

2) No VIDIOC_REQUEST_ALLOC 'shortcut' ioctl. Sorry, I just ran out
   of time. Alexandre, Tomasz, feel free to add it back (it should
   be quite easy to do) and post a patch. I'll add it to my patch
   series. As mentioned before: whether or not we actually want this
   has not been decided yet.

3) vim2m: the media topology is a bit bogus, this needs to be fixed
   (i.e. a proper HW entity should be added). But for now it is
   good enough for testing.

4) I think this should slide in fairly easy after the fence support
   is merged. I made sure the request API changes in public headers
   did not clash with the changes made by the fence API.

5) I did not verify the Request API documentation patch. I did update
   it with the new buffer flags and 'which' value, but it probably is
   out of date in places.

6) More testing. I think I have fairly good coverage in v4l2-compliance,
   but no doubt I've missed a few test cases.

7) debugfs: it would be really useful to expose the number of request
   and request objects in debugfs to simplify debugging. Esp. to make
   sure everything is freed correctly.

8) Review copyright/authorship lines. I'm not sure if everything is
   correct. Alexandre, Sakari, if you see something that is not
   right in this respect, just let me know!

9) The req_queue op should likely be split into a req_validate and
   a req_queue op (based on a discussion on the ML with Sakari).
   That avoids a race condition.

Everything seemed to slip nicely into place while working on this,
so I hope this is finally an implementation that we can proceed to
upstream and build upon for complex camera pipelines in the future.

This patch series is also available here:

https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=reqv11

Regards,

	Hans


Alexandre Courbot (3):
  videodev2.h: add request_fd field to v4l2_ext_controls
  Documentation: v4l: document request API
  media: vim2m: add media device

Hans Verkuil (26):
  v4l2-device.h: always expose mdev
  uapi/linux/media.h: add request API
  media-request: allocate media requests
  media-request: core request support
  media-request: add request ioctls
  media-request: add media_request_find
  media-request: add media_request_object_find
  videodev2.h: add V4L2_CTRL_FLAG_IN_REQUEST
  v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev
  v4l2-ctrls: prepare internal structs for request API
  v4l2-ctrls: alloc memory for p_req
  v4l2-ctrls: use ref in helper instead of ctrl
  v4l2-ctrls: add core request support
  v4l2-ctrls: support g/s_ext_ctrls for requests
  videodev2.h: Add request_fd field to v4l2_buffer
  vb2: store userspace data in vb2_v4l2_buffer
  videobuf2-core: embed media_request_object
  videobuf2-core: integrate with media requests
  videobuf2-v4l2: integrate with media requests
  videobuf2-core: add vb2_core_request_has_buffers
  videobuf2-v4l2: add vb2_request_queue helper
  videobuf2-v4l2: export request_fd
  vim2m: use workqueue
  vim2m: support requests
  vivid: add mc
  vivid: add request support

 Documentation/media/uapi/v4l/buffer.rst            |  19 +-
 Documentation/media/uapi/v4l/common.rst            |   1 +
 Documentation/media/uapi/v4l/request-api.rst       | 199 +++++++++
 Documentation/media/uapi/v4l/user-func.rst         |   1 +
 .../media/uapi/v4l/vidioc-g-ext-ctrls.rst          |  22 +-
 .../media/uapi/v4l/vidioc-new-request.rst          |  64 +++
 Documentation/media/uapi/v4l/vidioc-qbuf.rst       |   8 +
 drivers/media/Makefile                             |   3 +-
 drivers/media/common/videobuf2/videobuf2-core.c    | 152 +++++--
 drivers/media/common/videobuf2/videobuf2-v4l2.c    | 449 ++++++++++++-------
 drivers/media/dvb-core/dvb_vb2.c                   |   5 +-
 drivers/media/dvb-frontends/rtl2832_sdr.c          |   5 +-
 drivers/media/media-device.c                       |  14 +
 drivers/media/media-request.c                      | 454 +++++++++++++++++++
 drivers/media/pci/bt8xx/bttv-driver.c              |   2 +-
 drivers/media/pci/cx23885/cx23885-417.c            |   2 +-
 drivers/media/pci/cx88/cx88-blackbird.c            |   2 +-
 drivers/media/pci/cx88/cx88-video.c                |   2 +-
 drivers/media/pci/saa7134/saa7134-empress.c        |   4 +-
 drivers/media/pci/saa7134/saa7134-video.c          |   2 +-
 drivers/media/platform/exynos4-is/fimc-capture.c   |   2 +-
 drivers/media/platform/omap3isp/ispvideo.c         |   4 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c        |   3 +-
 drivers/media/platform/rcar_drif.c                 |   2 +-
 drivers/media/platform/s3c-camif/camif-capture.c   |   4 +-
 drivers/media/platform/s5p-mfc/s5p_mfc_dec.c       |   4 +-
 drivers/media/platform/s5p-mfc/s5p_mfc_enc.c       |   4 +-
 drivers/media/platform/soc_camera/soc_camera.c     |   7 +-
 drivers/media/platform/vim2m.c                     |  83 +++-
 drivers/media/platform/vivid/vivid-core.c          |  68 +++
 drivers/media/platform/vivid/vivid-core.h          |   8 +
 drivers/media/platform/vivid/vivid-ctrls.c         |  46 +-
 drivers/media/platform/vivid/vivid-kthread-cap.c   |  12 +
 drivers/media/platform/vivid/vivid-kthread-out.c   |  12 +
 drivers/media/platform/vivid/vivid-sdr-cap.c       |   8 +
 drivers/media/platform/vivid/vivid-vbi-cap.c       |   2 +
 drivers/media/platform/vivid/vivid-vbi-out.c       |   2 +
 drivers/media/platform/vivid/vivid-vid-cap.c       |   2 +
 drivers/media/platform/vivid/vivid-vid-out.c       |   2 +
 drivers/media/usb/cpia2/cpia2_v4l.c                |   2 +-
 drivers/media/usb/cx231xx/cx231xx-417.c            |   2 +-
 drivers/media/usb/cx231xx/cx231xx-video.c          |   4 +-
 drivers/media/usb/msi2500/msi2500.c                |   2 +-
 drivers/media/usb/tm6000/tm6000-video.c            |   2 +-
 drivers/media/usb/uvc/uvc_queue.c                  |   5 +-
 drivers/media/usb/uvc/uvc_v4l2.c                   |   3 +-
 drivers/media/usb/uvc/uvcvideo.h                   |   1 +
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c      |  14 +-
 drivers/media/v4l2-core/v4l2-ctrls.c               | 480 +++++++++++++++++++--
 drivers/media/v4l2-core/v4l2-device.c              |   3 +-
 drivers/media/v4l2-core/v4l2-ioctl.c               |  22 +-
 drivers/media/v4l2-core/v4l2-mem2mem.c             |   7 +-
 drivers/media/v4l2-core/v4l2-subdev.c              |   9 +-
 drivers/staging/media/davinci_vpfe/vpfe_video.c    |   3 +-
 drivers/staging/media/imx/imx-media-dev.c          |   2 +-
 drivers/staging/media/imx/imx-media-fim.c          |   2 +-
 drivers/staging/media/omap4iss/iss_video.c         |   3 +-
 drivers/usb/gadget/function/uvc_queue.c            |   2 +-
 include/media/media-device.h                       |  13 +
 include/media/media-request.h                      | 213 +++++++++
 include/media/v4l2-ctrls.h                         |  45 +-
 include/media/v4l2-device.h                        |   4 +-
 include/media/videobuf2-core.h                     |  25 +-
 include/media/videobuf2-v4l2.h                     |  19 +-
 include/uapi/linux/media.h                         |   8 +
 include/uapi/linux/videodev2.h                     |  15 +-
 66 files changed, 2277 insertions(+), 318 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/request-api.rst
 create mode 100644 Documentation/media/uapi/v4l/vidioc-new-request.rst
 create mode 100644 drivers/media/media-request.c
 create mode 100644 include/media/media-request.h

-- 
2.16.3

[RFCv11 PATCH 01/29] v4l2-device.h: always expose mdev

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

The mdev field is only present if CONFIG_MEDIA_CONTROLLER is set.
But since we will need to pass the media_device to vb2 snd the
control framework it is very convenient to just make this field
available all the time. If CONFIG_MEDIA_CONTROLLER is not set,
then it will just be NULL.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 include/media/v4l2-device.h | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/include/media/v4l2-device.h b/include/media/v4l2-device.h
index 0c9e4da55499..b330e4a08a6b 100644
--- a/include/media/v4l2-device.h
+++ b/include/media/v4l2-device.h
@@ -33,7 +33,7 @@ struct v4l2_ctrl_handler;
  * struct v4l2_device - main struct to for V4L2 device drivers
  *
  * @dev: pointer to struct device.
- * @mdev: pointer to struct media_device
+ * @mdev: pointer to struct media_device, may be NULL.
  * @subdevs: used to keep track of the registered subdevs
  * @lock: lock this struct; can be used by the driver as well
  *	if this struct is embedded into a larger struct.
@@ -58,9 +58,7 @@ struct v4l2_ctrl_handler;
  */
 struct v4l2_device {
 	struct device *dev;
-#if defined(CONFIG_MEDIA_CONTROLLER)
 	struct media_device *mdev;
-#endif
 	struct list_head subdevs;
 	spinlock_t lock;
 	char name[V4L2_DEVICE_NAME_SIZE];
-- 
2.16.3

[RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Define the public request API.

This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
and two ioctls that operate on a request in order to queue the
contents of the request to the driver and to re-initialize the
request.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 include/uapi/linux/media.h | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index c7e9a5cba24e..f8769e74f847 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -342,11 +342,19 @@ struct media_v2_topology {
 
 /* ioctls */
 
+struct __attribute__ ((packed)) media_request_alloc {
+	__s32 fd;
+};
+
 #define MEDIA_IOC_DEVICE_INFO	_IOWR('|', 0x00, struct media_device_info)
 #define MEDIA_IOC_ENUM_ENTITIES	_IOWR('|', 0x01, struct media_entity_desc)
 #define MEDIA_IOC_ENUM_LINKS	_IOWR('|', 0x02, struct media_links_enum)
 #define MEDIA_IOC_SETUP_LINK	_IOWR('|', 0x03, struct media_link_desc)
 #define MEDIA_IOC_G_TOPOLOGY	_IOWR('|', 0x04, struct media_v2_topology)
+#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, struct media_request_alloc)
+
+#define MEDIA_REQUEST_IOC_QUEUE		_IO('|',  0x80)
+#define MEDIA_REQUEST_IOC_REINIT	_IO('|',  0x81)
 
 #if !defined(__KERNEL__) || defined(__NEED_MEDIA_LEGACY_API)
 
-- 
2.16.3

[RFCv11 PATCH 03/29] media-request: allocate media requests

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add support for allocating a new request. This is only supported
if mdev->ops->req_queue is set, i.e. the driver indicates that it
supports queueing requests.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/Makefile        |  3 ++-
 drivers/media/media-device.c  | 14 ++++++++++++++
 drivers/media/media-request.c | 23 +++++++++++++++++++++++
 include/media/media-device.h  | 13 +++++++++++++
 include/media/media-request.h | 22 ++++++++++++++++++++++
 5 files changed, 74 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/media-request.c
 create mode 100644 include/media/media-request.h

diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index 594b462ddf0e..985d35ec6b29 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -3,7 +3,8 @@
 # Makefile for the kernel multimedia device drivers.
 #
 
-media-objs	:= media-device.o media-devnode.o media-entity.o
+media-objs	:= media-device.o media-devnode.o media-entity.o \
+		   media-request.o
 
 #
 # I2C drivers should come before other drivers, otherwise they'll fail
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index 35e81f7c0d2f..acb583c0eacd 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -32,6 +32,7 @@
 #include <media/media-device.h>
 #include <media/media-devnode.h>
 #include <media/media-entity.h>
+#include <media/media-request.h>
 
 #ifdef CONFIG_MEDIA_CONTROLLER
 
@@ -366,6 +367,15 @@ static long media_device_get_topology(struct media_device *mdev,
 	return ret;
 }
 
+static long media_device_request_alloc(struct media_device *mdev,
+				       struct media_request_alloc *alloc)
+{
+	if (!mdev->ops || !mdev->ops->req_queue)
+		return -ENOTTY;
+
+	return media_request_alloc(mdev, alloc);
+}
+
 static long copy_arg_from_user(void *karg, void __user *uarg, unsigned int cmd)
 {
 	/* All media IOCTLs are _IOWR() */
@@ -414,6 +424,7 @@ static const struct media_ioctl_info ioctl_info[] = {
 	MEDIA_IOC(ENUM_LINKS, media_device_enum_links, MEDIA_IOC_FL_GRAPH_MUTEX),
 	MEDIA_IOC(SETUP_LINK, media_device_setup_link, MEDIA_IOC_FL_GRAPH_MUTEX),
 	MEDIA_IOC(G_TOPOLOGY, media_device_get_topology, MEDIA_IOC_FL_GRAPH_MUTEX),
+	MEDIA_IOC(REQUEST_ALLOC, media_device_request_alloc, 0),
 };
 
 static long media_device_ioctl(struct file *filp, unsigned int cmd,
@@ -686,6 +697,9 @@ void media_device_init(struct media_device *mdev)
 	INIT_LIST_HEAD(&mdev->pads);
 	INIT_LIST_HEAD(&mdev->links);
 	INIT_LIST_HEAD(&mdev->entity_notify);
+
+	spin_lock_init(&mdev->req_lock);
+	mutex_init(&mdev->req_queue_mutex);
 	mutex_init(&mdev->graph_mutex);
 	ida_init(&mdev->entity_internal_idx);
 
diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
new file mode 100644
index 000000000000..ead78613fdbe
--- /dev/null
+++ b/drivers/media/media-request.c
@@ -0,0 +1,23 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Media device request objects
+ *
+ * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
+ *
+ * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+
+#include <media/media-device.h>
+#include <media/media-request.h>
+
+int media_request_alloc(struct media_device *mdev,
+			struct media_request_alloc *alloc)
+{
+	return -ENOMEM;
+}
diff --git a/include/media/media-device.h b/include/media/media-device.h
index bcc6ec434f1f..07e323c57202 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -19,6 +19,7 @@
 #ifndef _MEDIA_DEVICE_H
 #define _MEDIA_DEVICE_H
 
+#include <linux/anon_inodes.h>
 #include <linux/list.h>
 #include <linux/mutex.h>
 
@@ -27,6 +28,7 @@
 
 struct ida;
 struct device;
+struct media_device;
 
 /**
  * struct media_entity_notify - Media Entity Notify
@@ -50,10 +52,16 @@ struct media_entity_notify {
  * struct media_device_ops - Media device operations
  * @link_notify: Link state change notification callback. This callback is
  *		 called with the graph_mutex held.
+ * @req_alloc: Allocate a request
+ * @req_free: Free a request
+ * @req_queue: Queue a request
  */
 struct media_device_ops {
 	int (*link_notify)(struct media_link *link, u32 flags,
 			   unsigned int notification);
+	struct media_request *(*req_alloc)(struct media_device *mdev);
+	void (*req_free)(struct media_request *req);
+	int (*req_queue)(struct media_request *req);
 };
 
 /**
@@ -88,6 +96,8 @@ struct media_device_ops {
  * @disable_source: Disable Source Handler function pointer
  *
  * @ops:	Operation handler callbacks
+ * @req_lock:	Serialise access to requests
+ * @req_queue_mutex: Serialise validating and queueing requests
  *
  * This structure represents an abstract high-level media device. It allows easy
  * access to entities and provides basic media device-level support. The
@@ -158,6 +168,9 @@ struct media_device {
 	void (*disable_source)(struct media_entity *entity);
 
 	const struct media_device_ops *ops;
+
+	spinlock_t req_lock;
+	struct mutex req_queue_mutex;
 };
 
 /* We don't need to include pci.h or usb.h here */
diff --git a/include/media/media-request.h b/include/media/media-request.h
new file mode 100644
index 000000000000..dae3eccd9aa7
--- /dev/null
+++ b/include/media/media-request.h
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Media device request objects
+ *
+ * Copyright (C) 2018 Intel Corporation
+ *
+ * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
+ */
+
+#ifndef MEDIA_REQUEST_H
+#define MEDIA_REQUEST_H
+
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+
+#include <media/media-device.h>
+
+int media_request_alloc(struct media_device *mdev,
+			struct media_request_alloc *alloc);
+
+#endif
-- 
2.16.3

[RFCv11 PATCH 04/29] media-request: core request support

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Implement the core of the media request processing.

Drivers can bind request objects to a request. These objects
can then be marked completed if the driver finished using them,
or just be unbound if the results do not need to be kept (e.g.
in the case of buffers).

Once all objects that were added are either unbound or completed,
the request is marked 'complete' and a POLLPRI signal is sent
via poll.

Both requests and request objects are refcounted.

While a request is queued its refcount is incremented (since it
is in use by a driver). Once it is completed the refcount is
decremented. When the user closes the request file descriptor
the refcount is also decremented. Once it reaches 0 all request
objects in the request are unbound and put() and the request
itself is freed.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/media-request.c | 284 +++++++++++++++++++++++++++++++++++++++++-
 include/media/media-request.h | 156 +++++++++++++++++++++++
 2 files changed, 439 insertions(+), 1 deletion(-)

diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index ead78613fdbe..dffc290e4ada 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -16,8 +16,290 @@
 #include <media/media-device.h>
 #include <media/media-request.h>
 
+static const char * const request_state[] = {
+	"idle",
+	"queueing",
+	"queued",
+	"complete",
+	"cleaning",
+};
+
+static const char *
+media_request_state_str(enum media_request_state state)
+{
+	if (WARN_ON(state >= ARRAY_SIZE(request_state)))
+		return "unknown";
+	return request_state[state];
+}
+
+static void media_request_clean(struct media_request *req)
+{
+	struct media_request_object *obj, *obj_safe;
+
+	WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
+
+	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
+		media_request_object_unbind(obj);
+		media_request_object_put(obj);
+	}
+
+	req->num_incomplete_objects = 0;
+	wake_up_interruptible(&req->poll_wait);
+}
+
+static void media_request_release(struct kref *kref)
+{
+	struct media_request *req =
+		container_of(kref, struct media_request, kref);
+	struct media_device *mdev = req->mdev;
+	unsigned long flags;
+
+	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
+
+	spin_lock_irqsave(&req->lock, flags);
+	req->state = MEDIA_REQUEST_STATE_CLEANING;
+	spin_unlock_irqrestore(&req->lock, flags);
+
+	media_request_clean(req);
+
+	if (mdev->ops->req_free)
+		mdev->ops->req_free(req);
+	else
+		kfree(req);
+}
+
+void media_request_put(struct media_request *req)
+{
+	kref_put(&req->kref, media_request_release);
+}
+EXPORT_SYMBOL_GPL(media_request_put);
+
+void media_request_cancel(struct media_request *req)
+{
+	struct media_request_object *obj, *obj_safe;
+
+	if (req->state != MEDIA_REQUEST_STATE_QUEUED)
+		return;
+
+	list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
+		if (obj->ops->cancel)
+			obj->ops->cancel(obj);
+}
+EXPORT_SYMBOL_GPL(media_request_cancel);
+
+static int media_request_close(struct inode *inode, struct file *filp)
+{
+	struct media_request *req = filp->private_data;
+
+	media_request_put(req);
+	return 0;
+}
+
+static unsigned int media_request_poll(struct file *filp,
+				       struct poll_table_struct *wait)
+{
+	struct media_request *req = filp->private_data;
+	unsigned long flags;
+	enum media_request_state state;
+
+	if (!(poll_requested_events(wait) & POLLPRI))
+		return 0;
+
+	spin_lock_irqsave(&req->lock, flags);
+	state = req->state;
+	spin_unlock_irqrestore(&req->lock, flags);
+
+	if (state == MEDIA_REQUEST_STATE_COMPLETE)
+		return POLLPRI;
+	if (state == MEDIA_REQUEST_STATE_IDLE)
+		return POLLERR;
+
+	poll_wait(filp, &req->poll_wait, wait);
+	return 0;
+}
+
+static long media_request_ioctl(struct file *filp, unsigned int cmd,
+				unsigned long __arg)
+{
+	return -ENOIOCTLCMD;
+}
+
+static const struct file_operations request_fops = {
+	.owner = THIS_MODULE,
+	.poll = media_request_poll,
+	.unlocked_ioctl = media_request_ioctl,
+	.release = media_request_close,
+};
+
 int media_request_alloc(struct media_device *mdev,
 			struct media_request_alloc *alloc)
 {
-	return -ENOMEM;
+	struct media_request *req;
+	struct file *filp;
+	char comm[TASK_COMM_LEN];
+	int fd;
+	int ret;
+
+	fd = get_unused_fd_flags(O_CLOEXEC);
+	if (fd < 0)
+		return fd;
+
+	filp = anon_inode_getfile("request", &request_fops, NULL, O_CLOEXEC);
+	if (IS_ERR(filp)) {
+		ret = PTR_ERR(filp);
+		goto err_put_fd;
+	}
+
+	if (mdev->ops->req_alloc)
+		req = mdev->ops->req_alloc(mdev);
+	else
+		req = kzalloc(sizeof(*req), GFP_KERNEL);
+	if (!req) {
+		ret = -ENOMEM;
+		goto err_fput;
+	}
+
+	filp->private_data = req;
+	req->mdev = mdev;
+	req->state = MEDIA_REQUEST_STATE_IDLE;
+	req->num_incomplete_objects = 0;
+	kref_init(&req->kref);
+	INIT_LIST_HEAD(&req->objects);
+	spin_lock_init(&req->lock);
+	init_waitqueue_head(&req->poll_wait);
+
+	alloc->fd = fd;
+
+	get_task_comm(comm, current);
+	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
+		 comm, fd);
+	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
+
+	fd_install(fd, filp);
+
+	return 0;
+
+err_fput:
+	fput(filp);
+
+err_put_fd:
+	put_unused_fd(fd);
+
+	return ret;
+}
+
+static void media_request_object_release(struct kref *kref)
+{
+	struct media_request_object *obj =
+		container_of(kref, struct media_request_object, kref);
+	struct media_request *req = obj->req;
+
+	if (req)
+		media_request_object_unbind(obj);
+	obj->ops->release(obj);
+}
+
+void media_request_object_put(struct media_request_object *obj)
+{
+	kref_put(&obj->kref, media_request_object_release);
+}
+EXPORT_SYMBOL_GPL(media_request_object_put);
+
+void media_request_object_init(struct media_request_object *obj)
+{
+	obj->ops = NULL;
+	obj->req = NULL;
+	obj->priv = NULL;
+	obj->completed = false;
+	INIT_LIST_HEAD(&obj->list);
+	kref_init(&obj->kref);
+}
+EXPORT_SYMBOL_GPL(media_request_object_init);
+
+void media_request_object_bind(struct media_request *req,
+			       const struct media_request_object_ops *ops,
+			       void *priv,
+			       struct media_request_object *obj)
+{
+	unsigned long flags;
+
+	if (WARN_ON(!ops->release || !ops->cancel))
+		return;
+
+	obj->req = req;
+	obj->ops = ops;
+	obj->priv = priv;
+	spin_lock_irqsave(&req->lock, flags);
+	if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
+		goto unlock;
+	list_add_tail(&obj->list, &req->objects);
+	req->num_incomplete_objects++;
+unlock:
+	spin_unlock_irqrestore(&req->lock, flags);
+}
+EXPORT_SYMBOL_GPL(media_request_object_bind);
+
+void media_request_object_unbind(struct media_request_object *obj)
+{
+	struct media_request *req = obj->req;
+	unsigned long flags;
+	bool completed = false;
+
+	if (!req)
+		return;
+
+	spin_lock_irqsave(&req->lock, flags);
+	list_del(&obj->list);
+	obj->req = NULL;
+
+	if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
+	    req->state == MEDIA_REQUEST_STATE_CLEANING)
+		goto unlock;
+
+	if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
+		goto unlock;
+
+	if (WARN_ON(!req->num_incomplete_objects))
+		goto unlock;
+
+	req->num_incomplete_objects--;
+	if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
+	    !req->num_incomplete_objects) {
+		req->state = MEDIA_REQUEST_STATE_COMPLETE;
+		completed = true;
+		wake_up_interruptible(&req->poll_wait);
+	}
+unlock:
+	spin_unlock_irqrestore(&req->lock, flags);
+	if (obj->ops->unbind)
+		obj->ops->unbind(obj);
+	if (completed)
+		media_request_put(req);
+}
+EXPORT_SYMBOL_GPL(media_request_object_unbind);
+
+void media_request_object_complete(struct media_request_object *obj)
+{
+	struct media_request *req = obj->req;
+	unsigned long flags;
+	bool completed = false;
+
+	spin_lock_irqsave(&req->lock, flags);
+	if (obj->completed)
+		goto unlock;
+	obj->completed = true;
+	if (WARN_ON(!req->num_incomplete_objects) ||
+	    WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
+		goto unlock;
+
+	if (!--req->num_incomplete_objects) {
+		req->state = MEDIA_REQUEST_STATE_COMPLETE;
+		wake_up_interruptible(&req->poll_wait);
+		completed = true;
+	}
+unlock:
+	spin_unlock_irqrestore(&req->lock, flags);
+	if (completed)
+		media_request_put(req);
 }
+EXPORT_SYMBOL_GPL(media_request_object_complete);
diff --git a/include/media/media-request.h b/include/media/media-request.h
index dae3eccd9aa7..082c3cae04ac 100644
--- a/include/media/media-request.h
+++ b/include/media/media-request.h
@@ -16,7 +16,163 @@
 
 #include <media/media-device.h>
 
+enum media_request_state {
+	MEDIA_REQUEST_STATE_IDLE,
+	MEDIA_REQUEST_STATE_QUEUEING,
+	MEDIA_REQUEST_STATE_QUEUED,
+	MEDIA_REQUEST_STATE_COMPLETE,
+	MEDIA_REQUEST_STATE_CLEANING,
+};
+
+struct media_request_object;
+
+/**
+ * struct media_request - Media device request
+ * @mdev: Media device this request belongs to
+ * @kref: Reference count
+ * @debug_prefix: Prefix for debug messages (process name:fd)
+ * @state: The state of the request
+ * @objects: List of @struct media_request_object request objects
+ * @num_objects: The number objects in the request
+ * @num_completed_objects: The number of completed objects in the request
+ * @poll_wait: Wait queue for poll
+ * @lock: Serializes access to this struct
+ */
+struct media_request {
+	struct media_device *mdev;
+	struct kref kref;
+	char debug_str[TASK_COMM_LEN + 11];
+	enum media_request_state state;
+	struct list_head objects;
+	unsigned int num_incomplete_objects;
+	struct wait_queue_head poll_wait;
+	spinlock_t lock;
+};
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+
+static inline void media_request_get(struct media_request *req)
+{
+	kref_get(&req->kref);
+}
+
+void media_request_put(struct media_request *req);
+void media_request_cancel(struct media_request *req);
+
 int media_request_alloc(struct media_device *mdev,
 			struct media_request_alloc *alloc);
+#else
+static inline void media_request_get(struct media_request *req)
+{
+}
+
+static inline void media_request_put(struct media_request *req)
+{
+}
+
+static inline void media_request_cancel(struct media_request *req)
+{
+}
+
+#endif
+
+struct media_request_object_ops {
+	int (*prepare)(struct media_request_object *object);
+	void (*unprepare)(struct media_request_object *object);
+	void (*queue)(struct media_request_object *object);
+	void (*unbind)(struct media_request_object *object);
+	void (*cancel)(struct media_request_object *object);
+	void (*release)(struct media_request_object *object);
+};
+
+/**
+ * struct media_request_object - An opaque object that belongs to a media
+ *				 request
+ *
+ * @priv: object's priv pointer
+ * @list: List entry of the object for @struct media_request
+ * @kref: Reference count of the object, acquire before releasing req->lock
+ *
+ * An object related to the request. This struct is embedded in the
+ * larger object data.
+ */
+struct media_request_object {
+	const struct media_request_object_ops *ops;
+	void *priv;
+	struct media_request *req;
+	struct list_head list;
+	struct kref kref;
+	bool completed;
+};
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+static inline void media_request_object_get(struct media_request_object *obj)
+{
+	kref_get(&obj->kref);
+}
+
+/**
+ * media_request_object_put - Put a media request object
+ *
+ * @obj: The object
+ *
+ * Put a media request object. Once all references are gone, the
+ * object's memory is released.
+ */
+void media_request_object_put(struct media_request_object *obj);
+
+/**
+ * media_request_object_init - Initialise a media request object
+ *
+ * Initialise a media request object. The object will be released using the
+ * release callback of the ops once it has no references (this function
+ * initialises references to one).
+ */
+void media_request_object_init(struct media_request_object *obj);
+
+/**
+ * media_request_object_bind - Bind a media request object to a request
+ */
+void media_request_object_bind(struct media_request *req,
+			       const struct media_request_object_ops *ops,
+			       void *priv,
+			       struct media_request_object *obj);
+
+void media_request_object_unbind(struct media_request_object *obj);
+
+/**
+ * media_request_object_complete - Mark the media request object as complete
+ */
+void media_request_object_complete(struct media_request_object *obj);
+#else
+static inline void media_request_object_get(struct media_request_object *obj)
+{
+}
+
+static inline void media_request_object_put(struct media_request_object *obj)
+{
+}
+
+static inline void media_request_object_init(struct media_request_object *obj)
+{
+	obj->ops = NULL;
+	obj->req = NULL;
+}
+
+static inline void media_request_object_bind(struct media_request *req,
+			       const struct media_request_object_ops *ops,
+			       void *priv,
+			       struct media_request_object *obj)
+{
+}
+
+static inline void media_request_object_unbind(struct media_request_object *obj)
+{
+}
+
+static inline void media_request_object_complete(struct media_request_object *obj)
+{
+}
+#endif
 
 #endif
-- 
2.16.3

[RFCv11 PATCH 05/29] media-request: add request ioctls

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
ioctls.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/media-request.c | 80 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 78 insertions(+), 2 deletions(-)

diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index dffc290e4ada..27739ff7cb09 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file *filp,
 	return 0;
 }
 
+static long media_request_ioctl_queue(struct media_request *req)
+{
+	struct media_device *mdev = req->mdev;
+	unsigned long flags;
+	int ret = 0;
+
+	dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
+
+	spin_lock_irqsave(&req->lock, flags);
+	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
+		dev_dbg(mdev->dev,
+			"request: unable to queue %s, request in state %s\n",
+			req->debug_str, media_request_state_str(req->state));
+		spin_unlock_irqrestore(&req->lock, flags);
+		return -EINVAL;
+	}
+	req->state = MEDIA_REQUEST_STATE_QUEUEING;
+
+	spin_unlock_irqrestore(&req->lock, flags);
+
+	/*
+	 * Ensure the request that is validated will be the one that gets queued
+	 * next by serialising the queueing process.
+	 */
+	mutex_lock(&mdev->req_queue_mutex);
+
+	ret = mdev->ops->req_queue(req);
+	spin_lock_irqsave(&req->lock, flags);
+	req->state = ret ? MEDIA_REQUEST_STATE_IDLE : MEDIA_REQUEST_STATE_QUEUED;
+	spin_unlock_irqrestore(&req->lock, flags);
+	mutex_unlock(&mdev->req_queue_mutex);
+
+	if (ret) {
+		dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
+			req->debug_str, ret);
+	} else {
+		media_request_get(req);
+	}
+
+	return ret;
+}
+
+static long media_request_ioctl_reinit(struct media_request *req)
+{
+	struct media_device *mdev = req->mdev;
+	unsigned long flags;
+
+	spin_lock_irqsave(&req->lock, flags);
+	if (req->state != MEDIA_REQUEST_STATE_IDLE &&
+	    req->state != MEDIA_REQUEST_STATE_COMPLETE) {
+		dev_dbg(mdev->dev,
+			"request: %s not in idle or complete state, cannot reinit\n",
+			req->debug_str);
+		spin_unlock_irqrestore(&req->lock, flags);
+		return -EINVAL;
+	}
+	req->state = MEDIA_REQUEST_STATE_CLEANING;
+	spin_unlock_irqrestore(&req->lock, flags);
+
+	media_request_clean(req);
+
+	spin_lock_irqsave(&req->lock, flags);
+	req->state = MEDIA_REQUEST_STATE_IDLE;
+	spin_unlock_irqrestore(&req->lock, flags);
+	return 0;
+}
+
 static long media_request_ioctl(struct file *filp, unsigned int cmd,
-				unsigned long __arg)
+				unsigned long arg)
 {
-	return -ENOIOCTLCMD;
+	struct media_request *req = filp->private_data;
+
+	switch (cmd) {
+	case MEDIA_REQUEST_IOC_QUEUE:
+		return media_request_ioctl_queue(req);
+	case MEDIA_REQUEST_IOC_REINIT:
+		return media_request_ioctl_reinit(req);
+	default:
+		return -ENOIOCTLCMD;
+	}
 }
 
 static const struct file_operations request_fops = {
-- 
2.16.3

[RFCv11 PATCH 06/29] media-request: add media_request_find

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add media_request_find() to find a request based on the file
descriptor.

The caller has to call media_request_put() for the returned
request since this function increments the refcount.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/media-request.c | 47 +++++++++++++++++++++++++++++++++++++++++++
 include/media/media-request.h | 10 +++++++++
 2 files changed, 57 insertions(+)

diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index 27739ff7cb09..02b620c81de5 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -207,6 +207,53 @@ static const struct file_operations request_fops = {
 	.release = media_request_close,
 };
 
+/**
+ * media_request_find - Find a request based on the file descriptor
+ * @mdev: The media device
+ * @request: The request file handle
+ *
+ * Find and return the request associated with the given file descriptor, or
+ * an error if no such request exists.
+ *
+ * When the function returns a request it increases its reference count. The
+ * caller is responsible for releasing the reference by calling
+ * media_request_put() on the request.
+ */
+struct media_request *
+media_request_find(struct media_device *mdev, int request_fd)
+{
+	struct file *filp;
+	struct media_request *req;
+
+	if (!mdev || !mdev->ops || !mdev->ops->req_queue)
+		return ERR_PTR(-EPERM);
+
+	filp = fget(request_fd);
+	if (!filp)
+		return ERR_PTR(-ENOENT);
+
+	if (filp->f_op != &request_fops)
+		goto err_fput;
+	req = filp->private_data;
+	media_request_get(req);
+
+	if (req->mdev != mdev)
+		goto err_kref_put;
+
+	fput(filp);
+
+	return req;
+
+err_kref_put:
+	media_request_put(req);
+
+err_fput:
+	fput(filp);
+
+	return ERR_PTR(-ENOENT);
+}
+EXPORT_SYMBOL_GPL(media_request_find);
+
 int media_request_alloc(struct media_device *mdev,
 			struct media_request_alloc *alloc)
 {
diff --git a/include/media/media-request.h b/include/media/media-request.h
index 082c3cae04ac..033697d493cd 100644
--- a/include/media/media-request.h
+++ b/include/media/media-request.h
@@ -59,6 +59,9 @@ static inline void media_request_get(struct media_request *req)
 void media_request_put(struct media_request *req);
 void media_request_cancel(struct media_request *req);
 
+struct media_request *
+media_request_find(struct media_device *mdev, int request_fd);
+
 int media_request_alloc(struct media_device *mdev,
 			struct media_request_alloc *alloc);
 #else
@@ -74,6 +77,12 @@ static inline void media_request_cancel(struct media_request *req)
 {
 }
 
+static inline struct media_request *
+media_request_find(struct media_device *mdev, int request_fd)
+{
+	return ERR_PTR(-ENOENT);
+}
+
 #endif
 
 struct media_request_object_ops {
@@ -173,6 +182,7 @@ static inline void media_request_object_unbind(struct media_request_object *obj)
 static inline void media_request_object_complete(struct media_request_object *obj)
 {
 }
+
 #endif
 
 #endif
-- 
2.16.3

[RFCv11 PATCH 07/29] media-request: add media_request_object_find

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add media_request_object_find to find a request object inside a
request based on ops and/or priv values.

Objects of the same type (vb2 buffer, control handler) will have
the same ops value. And objects that refer to the same 'parent'
object (e.g. the v4l2_ctrl_handler that has the current driver
state) will have the same priv value.

The caller has to call media_request_object_put() for the returned
object since this function increments the refcount.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/media-request.c | 26 ++++++++++++++++++++++++++
 include/media/media-request.h | 25 +++++++++++++++++++++++++
 2 files changed, 51 insertions(+)

diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index 02b620c81de5..415f7e31019d 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -322,6 +322,32 @@ static void media_request_object_release(struct kref *kref)
 	obj->ops->release(obj);
 }
 
+struct media_request_object *
+media_request_object_find(struct media_request *req,
+			  const struct media_request_object_ops *ops,
+			  void *priv)
+{
+	struct media_request_object *obj;
+	struct media_request_object *found = NULL;
+	unsigned long flags;
+
+	if (!ops && !priv)
+		return NULL;
+
+	spin_lock_irqsave(&req->lock, flags);
+	list_for_each_entry(obj, &req->objects, list) {
+		if ((!ops || obj->ops == ops) &&
+		    (!priv || obj->priv == priv)) {
+			media_request_object_get(obj);
+			found = obj;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&req->lock, flags);
+	return found;
+}
+EXPORT_SYMBOL_GPL(media_request_object_find);
+
 void media_request_object_put(struct media_request_object *obj)
 {
 	kref_put(&obj->kref, media_request_object_release);
diff --git a/include/media/media-request.h b/include/media/media-request.h
index 033697d493cd..ea990c8f76bc 100644
--- a/include/media/media-request.h
+++ b/include/media/media-request.h
@@ -130,6 +130,23 @@ static inline void media_request_object_get(struct media_request_object *obj)
  */
 void media_request_object_put(struct media_request_object *obj);
 
+/**
+ * media_request_object_find - Find an object in a request
+ *
+ * @ops: Find an object with this ops value, may be NULL.
+ * @priv: Find an object with this priv value, may be NULL.
+ *
+ * At least one of @ops and @priv must be non-NULL. If one of
+ * these is NULL, then skip checking for that field.
+ *
+ * Returns NULL if not found or the object (the refcount is increased
+ * in that case).
+ */
+struct media_request_object *
+media_request_object_find(struct media_request *req,
+			  const struct media_request_object_ops *ops,
+			  void *priv);
+
 /**
  * media_request_object_init - Initialise a media request object
  *
@@ -162,6 +179,14 @@ static inline void media_request_object_put(struct media_request_object *obj)
 {
 }
 
+static inline struct media_request_object *
+media_request_object_find(struct media_request *req,
+			  const struct media_request_object_ops *ops,
+			  void *priv)
+{
+	return NULL;
+}
+
 static inline void media_request_object_init(struct media_request_object *obj)
 {
 	obj->ops = NULL;
-- 
2.16.3

[RFCv11 PATCH 08/29] videodev2.h: add request_fd field to v4l2_ext_controls

[Hans Verkuil]

From: Alexandre Courbot <acourbot@chromium.org>

If which is V4L2_CTRL_WHICH_REQUEST, then the request_fd field can be
used to specify a request for the G/S/TRY_EXT_CTRLS ioctls.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 5 ++++-
 drivers/media/v4l2-core/v4l2-ioctl.c          | 6 +++---
 include/uapi/linux/videodev2.h                | 4 +++-
 3 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 5198c9eeb348..0782b3666deb 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -732,7 +732,8 @@ struct v4l2_ext_controls32 {
 	__u32 which;
 	__u32 count;
 	__u32 error_idx;
-	__u32 reserved[2];
+	__s32 request_fd;
+	__u32 reserved[1];
 	compat_caddr_t controls; /* actually struct v4l2_ext_control32 * */
 };
 
@@ -807,6 +808,7 @@ static int get_v4l2_ext_controls32(struct file *file,
 	    get_user(count, &up->count) ||
 	    put_user(count, &kp->count) ||
 	    assign_in_user(&kp->error_idx, &up->error_idx) ||
+	    assign_in_user(&kp->request_fd, &up->request_fd) ||
 	    copy_in_user(kp->reserved, up->reserved, sizeof(kp->reserved)))
 		return -EFAULT;
 
@@ -865,6 +867,7 @@ static int put_v4l2_ext_controls32(struct file *file,
 	    get_user(count, &kp->count) ||
 	    put_user(count, &up->count) ||
 	    assign_in_user(&up->error_idx, &kp->error_idx) ||
+	    assign_in_user(&up->request_fd, &kp->request_fd) ||
 	    copy_in_user(up->reserved, kp->reserved, sizeof(up->reserved)) ||
 	    get_user(kcontrols, &kp->controls))
 		return -EFAULT;
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index f48c505550e0..9ce23e23c5bf 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -553,8 +553,8 @@ static void v4l_print_ext_controls(const void *arg, bool write_only)
 	const struct v4l2_ext_controls *p = arg;
 	int i;
 
-	pr_cont("which=0x%x, count=%d, error_idx=%d",
-			p->which, p->count, p->error_idx);
+	pr_cont("which=0x%x, count=%d, error_idx=%d, request_fd=%d",
+			p->which, p->count, p->error_idx, p->request_fd);
 	for (i = 0; i < p->count; i++) {
 		if (!p->controls[i].size)
 			pr_cont(", id/val=0x%x/0x%x",
@@ -870,7 +870,7 @@ static int check_ext_ctrls(struct v4l2_ext_controls *c, int allow_priv)
 	__u32 i;
 
 	/* zero the reserved fields */
-	c->reserved[0] = c->reserved[1] = 0;
+	c->reserved[0] = 0;
 	for (i = 0; i < c->count; i++)
 		c->controls[i].reserved2[0] = 0;
 
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 600877be5c22..6f41baa53787 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1592,7 +1592,8 @@ struct v4l2_ext_controls {
 	};
 	__u32 count;
 	__u32 error_idx;
-	__u32 reserved[2];
+	__s32 request_fd;
+	__u32 reserved[1];
 	struct v4l2_ext_control *controls;
 };
 
@@ -1605,6 +1606,7 @@ struct v4l2_ext_controls {
 #define V4L2_CTRL_MAX_DIMS	  (4)
 #define V4L2_CTRL_WHICH_CUR_VAL   0
 #define V4L2_CTRL_WHICH_DEF_VAL   0x0f000000
+#define V4L2_CTRL_WHICH_REQUEST   0x0f010000
 
 enum v4l2_ctrl_type {
 	V4L2_CTRL_TYPE_INTEGER	     = 1,
-- 
2.16.3

[RFCv11 PATCH 09/29] videodev2.h: add V4L2_CTRL_FLAG_IN_REQUEST

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 include/uapi/linux/videodev2.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 6f41baa53787..d6e5c245bdcf 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1679,6 +1679,7 @@ struct v4l2_querymenu {
 #define V4L2_CTRL_FLAG_HAS_PAYLOAD	0x0100
 #define V4L2_CTRL_FLAG_EXECUTE_ON_WRITE	0x0200
 #define V4L2_CTRL_FLAG_MODIFY_LAYOUT	0x0400
+#define V4L2_CTRL_FLAG_IN_REQUEST	0x0800
 
 /*  Query flags, to be ORed with the control ID */
 #define V4L2_CTRL_FLAG_NEXT_CTRL	0x80000000
-- 
2.16.3

[RFCv11 PATCH 10/29] v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add a 'bool from_other_dev' argument: set to true if the two
handlers refer to different devices (e.g. it is true when
inheriting controls from a subdev into a main v4l2 bridge
driver).

This will be used later when implementing support for the
request API since we need to skip such controls.

TODO: check drivers/staging/media/imx/imx-media-fim.c change.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/dvb-frontends/rtl2832_sdr.c        |  5 +--
 drivers/media/pci/bt8xx/bttv-driver.c            |  2 +-
 drivers/media/pci/cx23885/cx23885-417.c          |  2 +-
 drivers/media/pci/cx88/cx88-blackbird.c          |  2 +-
 drivers/media/pci/cx88/cx88-video.c              |  2 +-
 drivers/media/pci/saa7134/saa7134-empress.c      |  4 +--
 drivers/media/pci/saa7134/saa7134-video.c        |  2 +-
 drivers/media/platform/exynos4-is/fimc-capture.c |  2 +-
 drivers/media/platform/rcar-vin/rcar-v4l2.c      |  3 +-
 drivers/media/platform/rcar_drif.c               |  2 +-
 drivers/media/platform/soc_camera/soc_camera.c   |  3 +-
 drivers/media/platform/vivid/vivid-ctrls.c       | 46 ++++++++++++------------
 drivers/media/usb/cx231xx/cx231xx-417.c          |  2 +-
 drivers/media/usb/cx231xx/cx231xx-video.c        |  4 +--
 drivers/media/usb/msi2500/msi2500.c              |  2 +-
 drivers/media/usb/tm6000/tm6000-video.c          |  2 +-
 drivers/media/v4l2-core/v4l2-ctrls.c             | 11 +++---
 drivers/media/v4l2-core/v4l2-device.c            |  3 +-
 drivers/staging/media/imx/imx-media-dev.c        |  2 +-
 drivers/staging/media/imx/imx-media-fim.c        |  2 +-
 include/media/v4l2-ctrls.h                       |  8 ++++-
 21 files changed, 62 insertions(+), 49 deletions(-)

diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.c b/drivers/media/dvb-frontends/rtl2832_sdr.c
index c6e78d870ccd..6064d28224e8 100644
--- a/drivers/media/dvb-frontends/rtl2832_sdr.c
+++ b/drivers/media/dvb-frontends/rtl2832_sdr.c
@@ -1394,7 +1394,8 @@ static int rtl2832_sdr_probe(struct platform_device *pdev)
 	case RTL2832_SDR_TUNER_E4000:
 		v4l2_ctrl_handler_init(&dev->hdl, 9);
 		if (subdev)
-			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler, NULL);
+			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler,
+					      NULL, true);
 		break;
 	case RTL2832_SDR_TUNER_R820T:
 	case RTL2832_SDR_TUNER_R828D:
@@ -1423,7 +1424,7 @@ static int rtl2832_sdr_probe(struct platform_device *pdev)
 		v4l2_ctrl_handler_init(&dev->hdl, 2);
 		if (subdev)
 			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler,
-					      NULL);
+					      NULL, true);
 		break;
 	default:
 		v4l2_ctrl_handler_init(&dev->hdl, 0);
diff --git a/drivers/media/pci/bt8xx/bttv-driver.c b/drivers/media/pci/bt8xx/bttv-driver.c
index f697698fe38d..cdcb36d8c5c3 100644
--- a/drivers/media/pci/bt8xx/bttv-driver.c
+++ b/drivers/media/pci/bt8xx/bttv-driver.c
@@ -4211,7 +4211,7 @@ static int bttv_probe(struct pci_dev *dev, const struct pci_device_id *pci_id)
 	/* register video4linux + input */
 	if (!bttv_tvcards[btv->c.type].no_video) {
 		v4l2_ctrl_add_handler(&btv->radio_ctrl_handler, hdl,
-				v4l2_ctrl_radio_filter);
+				v4l2_ctrl_radio_filter, false);
 		if (btv->radio_ctrl_handler.error) {
 			result = btv->radio_ctrl_handler.error;
 			goto fail2;
diff --git a/drivers/media/pci/cx23885/cx23885-417.c b/drivers/media/pci/cx23885/cx23885-417.c
index a71f3c7569ce..762823871c78 100644
--- a/drivers/media/pci/cx23885/cx23885-417.c
+++ b/drivers/media/pci/cx23885/cx23885-417.c
@@ -1527,7 +1527,7 @@ int cx23885_417_register(struct cx23885_dev *dev)
 	dev->cxhdl.priv = dev;
 	dev->cxhdl.func = cx23885_api_func;
 	cx2341x_handler_set_50hz(&dev->cxhdl, tsport->height == 576);
-	v4l2_ctrl_add_handler(&dev->ctrl_handler, &dev->cxhdl.hdl, NULL);
+	v4l2_ctrl_add_handler(&dev->ctrl_handler, &dev->cxhdl.hdl, NULL, false);
 
 	/* Allocate and initialize V4L video device */
 	dev->v4l_device = cx23885_video_dev_alloc(tsport,
diff --git a/drivers/media/pci/cx88/cx88-blackbird.c b/drivers/media/pci/cx88/cx88-blackbird.c
index 0e0952e60795..39f69d89a663 100644
--- a/drivers/media/pci/cx88/cx88-blackbird.c
+++ b/drivers/media/pci/cx88/cx88-blackbird.c
@@ -1183,7 +1183,7 @@ static int cx8802_blackbird_probe(struct cx8802_driver *drv)
 	err = cx2341x_handler_init(&dev->cxhdl, 36);
 	if (err)
 		goto fail_core;
-	v4l2_ctrl_add_handler(&dev->cxhdl.hdl, &core->video_hdl, NULL);
+	v4l2_ctrl_add_handler(&dev->cxhdl.hdl, &core->video_hdl, NULL, false);
 
 	/* blackbird stuff */
 	pr_info("cx23416 based mpeg encoder (blackbird reference design)\n");
diff --git a/drivers/media/pci/cx88/cx88-video.c b/drivers/media/pci/cx88/cx88-video.c
index 9be682cdb644..e35bfa03a1e2 100644
--- a/drivers/media/pci/cx88/cx88-video.c
+++ b/drivers/media/pci/cx88/cx88-video.c
@@ -1378,7 +1378,7 @@ static int cx8800_initdev(struct pci_dev *pci_dev,
 		if (vc->id == V4L2_CID_CHROMA_AGC)
 			core->chroma_agc = vc;
 	}
-	v4l2_ctrl_add_handler(&core->video_hdl, &core->audio_hdl, NULL);
+	v4l2_ctrl_add_handler(&core->video_hdl, &core->audio_hdl, NULL, false);
 
 	/* load and configure helper modules */
 
diff --git a/drivers/media/pci/saa7134/saa7134-empress.c b/drivers/media/pci/saa7134/saa7134-empress.c
index 66acfd35ffc6..fc75ce00dbf8 100644
--- a/drivers/media/pci/saa7134/saa7134-empress.c
+++ b/drivers/media/pci/saa7134/saa7134-empress.c
@@ -265,9 +265,9 @@ static int empress_init(struct saa7134_dev *dev)
 		 "%s empress (%s)", dev->name,
 		 saa7134_boards[dev->board].name);
 	v4l2_ctrl_handler_init(hdl, 21);
-	v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler, empress_ctrl_filter);
+	v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler, empress_ctrl_filter, false);
 	if (dev->empress_sd)
-		v4l2_ctrl_add_handler(hdl, dev->empress_sd->ctrl_handler, NULL);
+		v4l2_ctrl_add_handler(hdl, dev->empress_sd->ctrl_handler, NULL, true);
 	if (hdl->error) {
 		video_device_release(dev->empress_dev);
 		return hdl->error;
diff --git a/drivers/media/pci/saa7134/saa7134-video.c b/drivers/media/pci/saa7134/saa7134-video.c
index 4f1091a11e91..d478c470b975 100644
--- a/drivers/media/pci/saa7134/saa7134-video.c
+++ b/drivers/media/pci/saa7134/saa7134-video.c
@@ -2136,7 +2136,7 @@ int saa7134_video_init1(struct saa7134_dev *dev)
 		hdl = &dev->radio_ctrl_handler;
 		v4l2_ctrl_handler_init(hdl, 2);
 		v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler,
-				v4l2_ctrl_radio_filter);
+				v4l2_ctrl_radio_filter, false);
 		if (hdl->error)
 			return hdl->error;
 	}
diff --git a/drivers/media/platform/exynos4-is/fimc-capture.c b/drivers/media/platform/exynos4-is/fimc-capture.c
index a3cdac188190..2164375f0ee0 100644
--- a/drivers/media/platform/exynos4-is/fimc-capture.c
+++ b/drivers/media/platform/exynos4-is/fimc-capture.c
@@ -1424,7 +1424,7 @@ static int fimc_link_setup(struct media_entity *entity,
 		return 0;
 
 	return v4l2_ctrl_add_handler(&vc->ctx->ctrls.handler,
-				     sensor->ctrl_handler, NULL);
+				     sensor->ctrl_handler, NULL, true);
 }
 
 static const struct media_entity_operations fimc_sd_media_ops = {
diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index b479b882da12..90246113fa03 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
@@ -900,7 +900,8 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
 	if (ret < 0)
 		return ret;
 
-	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
+	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler,
+				    NULL, true);
 	if (ret < 0)
 		return ret;
 
diff --git a/drivers/media/platform/rcar_drif.c b/drivers/media/platform/rcar_drif.c
index dc7e280c91b4..159c7d2c2066 100644
--- a/drivers/media/platform/rcar_drif.c
+++ b/drivers/media/platform/rcar_drif.c
@@ -1168,7 +1168,7 @@ static int rcar_drif_notify_complete(struct v4l2_async_notifier *notifier)
 	}
 
 	ret = v4l2_ctrl_add_handler(&sdr->ctrl_hdl,
-				    sdr->ep.subdev->ctrl_handler, NULL);
+				    sdr->ep.subdev->ctrl_handler, NULL, true);
 	if (ret) {
 		rdrif_err(sdr, "failed: ctrl add hdlr ret %d\n", ret);
 		goto error;
diff --git a/drivers/media/platform/soc_camera/soc_camera.c b/drivers/media/platform/soc_camera/soc_camera.c
index 69f0d8e80bd8..e6787abc34ae 100644
--- a/drivers/media/platform/soc_camera/soc_camera.c
+++ b/drivers/media/platform/soc_camera/soc_camera.c
@@ -1180,7 +1180,8 @@ static int soc_camera_probe_finish(struct soc_camera_device *icd)
 
 	v4l2_subdev_call(sd, video, g_tvnorms, &icd->vdev->tvnorms);
 
-	ret = v4l2_ctrl_add_handler(&icd->ctrl_handler, sd->ctrl_handler, NULL);
+	ret = v4l2_ctrl_add_handler(&icd->ctrl_handler, sd->ctrl_handler,
+				    NULL, true);
 	if (ret < 0)
 		return ret;
 
diff --git a/drivers/media/platform/vivid/vivid-ctrls.c b/drivers/media/platform/vivid/vivid-ctrls.c
index 6b0bfa091592..f369b94ad7ff 100644
--- a/drivers/media/platform/vivid/vivid-ctrls.c
+++ b/drivers/media/platform/vivid/vivid-ctrls.c
@@ -1662,59 +1662,59 @@ int vivid_create_controls(struct vivid_dev *dev, bool show_ccs_cap,
 		v4l2_ctrl_auto_cluster(2, &dev->autogain, 0, true);
 
 	if (dev->has_vid_cap) {
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_vid, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_aud, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_streaming, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_sdtv_cap, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_loop_cap, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_fb, NULL);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_vid, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_aud, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_streaming, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_sdtv_cap, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_loop_cap, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_fb, NULL, false);
 		if (hdl_vid_cap->error)
 			return hdl_vid_cap->error;
 		dev->vid_cap_dev.ctrl_handler = hdl_vid_cap;
 	}
 	if (dev->has_vid_out) {
-		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_aud, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_out, hdl_streaming, NULL);
-		v4l2_ctrl_add_handler(hdl_vid_out, hdl_fb, NULL);
+		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_aud, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_out, hdl_streaming, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vid_out, hdl_fb, NULL, false);
 		if (hdl_vid_out->error)
 			return hdl_vid_out->error;
 		dev->vid_out_dev.ctrl_handler = hdl_vid_out;
 	}
 	if (dev->has_vbi_cap) {
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_streaming, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_sdtv_cap, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_loop_cap, NULL);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_streaming, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_sdtv_cap, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_loop_cap, NULL, false);
 		if (hdl_vbi_cap->error)
 			return hdl_vbi_cap->error;
 		dev->vbi_cap_dev.ctrl_handler = hdl_vbi_cap;
 	}
 	if (dev->has_vbi_out) {
-		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_streaming, NULL);
+		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_streaming, NULL, false);
 		if (hdl_vbi_out->error)
 			return hdl_vbi_out->error;
 		dev->vbi_out_dev.ctrl_handler = hdl_vbi_out;
 	}
 	if (dev->has_radio_rx) {
-		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_aud, NULL);
+		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_aud, NULL, false);
 		if (hdl_radio_rx->error)
 			return hdl_radio_rx->error;
 		dev->radio_rx_dev.ctrl_handler = hdl_radio_rx;
 	}
 	if (dev->has_radio_tx) {
-		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_aud, NULL);
+		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_aud, NULL, false);
 		if (hdl_radio_tx->error)
 			return hdl_radio_tx->error;
 		dev->radio_tx_dev.ctrl_handler = hdl_radio_tx;
 	}
 	if (dev->has_sdr_cap) {
-		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_user_gen, NULL);
-		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_streaming, NULL);
+		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_user_gen, NULL, false);
+		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_streaming, NULL, false);
 		if (hdl_sdr_cap->error)
 			return hdl_sdr_cap->error;
 		dev->sdr_cap_dev.ctrl_handler = hdl_sdr_cap;
diff --git a/drivers/media/usb/cx231xx/cx231xx-417.c b/drivers/media/usb/cx231xx/cx231xx-417.c
index b80e6857e2eb..fca16cf8b3bf 100644
--- a/drivers/media/usb/cx231xx/cx231xx-417.c
+++ b/drivers/media/usb/cx231xx/cx231xx-417.c
@@ -1991,7 +1991,7 @@ int cx231xx_417_register(struct cx231xx *dev)
 	dev->mpeg_ctrl_handler.ops = &cx231xx_ops;
 	if (dev->sd_cx25840)
 		v4l2_ctrl_add_handler(&dev->mpeg_ctrl_handler.hdl,
-				dev->sd_cx25840->ctrl_handler, NULL);
+				dev->sd_cx25840->ctrl_handler, NULL, false);
 	if (dev->mpeg_ctrl_handler.hdl.error) {
 		err = dev->mpeg_ctrl_handler.hdl.error;
 		dprintk(3, "%s: can't add cx25840 controls\n", dev->name);
diff --git a/drivers/media/usb/cx231xx/cx231xx-video.c b/drivers/media/usb/cx231xx/cx231xx-video.c
index f7fcd733a2ca..2dedb18f63a0 100644
--- a/drivers/media/usb/cx231xx/cx231xx-video.c
+++ b/drivers/media/usb/cx231xx/cx231xx-video.c
@@ -2204,10 +2204,10 @@ int cx231xx_register_analog_devices(struct cx231xx *dev)
 
 	if (dev->sd_cx25840) {
 		v4l2_ctrl_add_handler(&dev->ctrl_handler,
-				dev->sd_cx25840->ctrl_handler, NULL);
+				dev->sd_cx25840->ctrl_handler, NULL, true);
 		v4l2_ctrl_add_handler(&dev->radio_ctrl_handler,
 				dev->sd_cx25840->ctrl_handler,
-				v4l2_ctrl_radio_filter);
+				v4l2_ctrl_radio_filter, true);
 	}
 
 	if (dev->ctrl_handler.error)
diff --git a/drivers/media/usb/msi2500/msi2500.c b/drivers/media/usb/msi2500/msi2500.c
index 65ef755adfdc..4aacd77a5d58 100644
--- a/drivers/media/usb/msi2500/msi2500.c
+++ b/drivers/media/usb/msi2500/msi2500.c
@@ -1278,7 +1278,7 @@ static int msi2500_probe(struct usb_interface *intf,
 	}
 
 	/* currently all controls are from subdev */
-	v4l2_ctrl_add_handler(&dev->hdl, sd->ctrl_handler, NULL);
+	v4l2_ctrl_add_handler(&dev->hdl, sd->ctrl_handler, NULL, true);
 
 	dev->v4l2_dev.ctrl_handler = &dev->hdl;
 	dev->vdev.v4l2_dev = &dev->v4l2_dev;
diff --git a/drivers/media/usb/tm6000/tm6000-video.c b/drivers/media/usb/tm6000/tm6000-video.c
index b2399d4266da..99d6b0da8b3d 100644
--- a/drivers/media/usb/tm6000/tm6000-video.c
+++ b/drivers/media/usb/tm6000/tm6000-video.c
@@ -1622,7 +1622,7 @@ int tm6000_v4l2_register(struct tm6000_core *dev)
 	v4l2_ctrl_new_std(&dev->ctrl_handler, &tm6000_ctrl_ops,
 			V4L2_CID_HUE, -128, 127, 1, 0);
 	v4l2_ctrl_add_handler(&dev->ctrl_handler,
-			&dev->radio_ctrl_handler, NULL);
+			&dev->radio_ctrl_handler, NULL, false);
 
 	if (dev->radio_ctrl_handler.error)
 		ret = dev->radio_ctrl_handler.error;
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index d29e45516eb7..aa1dd2015e84 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1995,7 +1995,8 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
 
 /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
 static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
-			   struct v4l2_ctrl *ctrl)
+			   struct v4l2_ctrl *ctrl,
+			   bool from_other_dev)
 {
 	struct v4l2_ctrl_ref *ref;
 	struct v4l2_ctrl_ref *new_ref;
@@ -2019,6 +2020,7 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 	if (!new_ref)
 		return handler_set_err(hdl, -ENOMEM);
 	new_ref->ctrl = ctrl;
+	new_ref->from_other_dev = from_other_dev;
 	if (ctrl->handler == hdl) {
 		/* By default each control starts in a cluster of its own.
 		   new_ref->ctrl is basically a cluster array with one
@@ -2199,7 +2201,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 		ctrl->type_ops->init(ctrl, idx, ctrl->p_new);
 	}
 
-	if (handler_new_ref(hdl, ctrl)) {
+	if (handler_new_ref(hdl, ctrl, false)) {
 		kvfree(ctrl);
 		return NULL;
 	}
@@ -2368,7 +2370,8 @@ EXPORT_SYMBOL(v4l2_ctrl_new_int_menu);
 /* Add the controls from another handler to our own. */
 int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 			  struct v4l2_ctrl_handler *add,
-			  bool (*filter)(const struct v4l2_ctrl *ctrl))
+			  bool (*filter)(const struct v4l2_ctrl *ctrl),
+			  bool from_other_dev)
 {
 	struct v4l2_ctrl_ref *ref;
 	int ret = 0;
@@ -2391,7 +2394,7 @@ int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 		/* Filter any unwanted controls */
 		if (filter && !filter(ctrl))
 			continue;
-		ret = handler_new_ref(hdl, ctrl);
+		ret = handler_new_ref(hdl, ctrl, from_other_dev);
 		if (ret)
 			break;
 	}
diff --git a/drivers/media/v4l2-core/v4l2-device.c b/drivers/media/v4l2-core/v4l2-device.c
index 937c6de85606..8391a7f0895b 100644
--- a/drivers/media/v4l2-core/v4l2-device.c
+++ b/drivers/media/v4l2-core/v4l2-device.c
@@ -178,7 +178,8 @@ int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
 
 	sd->v4l2_dev = v4l2_dev;
 	/* This just returns 0 if either of the two args is NULL */
-	err = v4l2_ctrl_add_handler(v4l2_dev->ctrl_handler, sd->ctrl_handler, NULL);
+	err = v4l2_ctrl_add_handler(v4l2_dev->ctrl_handler, sd->ctrl_handler,
+				    NULL, true);
 	if (err)
 		goto error_module;
 
diff --git a/drivers/staging/media/imx/imx-media-dev.c b/drivers/staging/media/imx/imx-media-dev.c
index 289d775c4820..08799beaea42 100644
--- a/drivers/staging/media/imx/imx-media-dev.c
+++ b/drivers/staging/media/imx/imx-media-dev.c
@@ -391,7 +391,7 @@ static int imx_media_inherit_controls(struct imx_media_dev *imxmd,
 
 		ret = v4l2_ctrl_add_handler(vfd->ctrl_handler,
 					    sd->ctrl_handler,
-					    NULL);
+					    NULL, true);
 		if (ret)
 			return ret;
 	}
diff --git a/drivers/staging/media/imx/imx-media-fim.c b/drivers/staging/media/imx/imx-media-fim.c
index 6df189135db8..8cf773eef9da 100644
--- a/drivers/staging/media/imx/imx-media-fim.c
+++ b/drivers/staging/media/imx/imx-media-fim.c
@@ -463,7 +463,7 @@ int imx_media_fim_add_controls(struct imx_media_fim *fim)
 {
 	/* add the FIM controls to the calling subdev ctrl handler */
 	return v4l2_ctrl_add_handler(fim->sd->ctrl_handler,
-				     &fim->ctrl_handler, NULL);
+				     &fim->ctrl_handler, NULL, false);
 }
 EXPORT_SYMBOL_GPL(imx_media_fim_add_controls);
 
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 5b445b5654f7..f8faa54b5e7e 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -247,6 +247,8 @@ struct v4l2_ctrl {
  * @ctrl:	The actual control information.
  * @helper:	Pointer to helper struct. Used internally in
  *		``prepare_ext_ctrls`` function at ``v4l2-ctrl.c``.
+ * @from_other_dev: If true, then @ctrl was defined in another
+ *		device then the &struct v4l2_ctrl_handler.
  *
  * Each control handler has a list of these refs. The list_head is used to
  * keep a sorted-by-control-ID list of all controls, while the next pointer
@@ -257,6 +259,7 @@ struct v4l2_ctrl_ref {
 	struct v4l2_ctrl_ref *next;
 	struct v4l2_ctrl *ctrl;
 	struct v4l2_ctrl_helper *helper;
+	bool from_other_dev;
 };
 
 /**
@@ -633,6 +636,8 @@ typedef bool (*v4l2_ctrl_filter)(const struct v4l2_ctrl *ctrl);
  * @add:	The control handler whose controls you want to add to
  *		the @hdl control handler.
  * @filter:	This function will filter which controls should be added.
+ * @from_other_dev: If true, then the controls in @add were defined in another
+ *		device then @hdl.
  *
  * Does nothing if either of the two handlers is a NULL pointer.
  * If @filter is NULL, then all controls are added. Otherwise only those
@@ -642,7 +647,8 @@ typedef bool (*v4l2_ctrl_filter)(const struct v4l2_ctrl *ctrl);
  */
 int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 			  struct v4l2_ctrl_handler *add,
-			  v4l2_ctrl_filter filter);
+			  v4l2_ctrl_filter filter,
+			  bool from_other_dev);
 
 /**
  * v4l2_ctrl_radio_filter() - Standard filter for radio controls.
-- 
2.16.3

[RFCv11 PATCH 11/29] v4l2-ctrls: prepare internal structs for request API

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Embed and initialize a media_request_object in struct v4l2_ctrl_handler.

Add a p_req field to struct v4l2_ctrl_ref that will store the
request value.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 1 +
 include/media/v4l2-ctrls.h           | 7 +++++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index aa1dd2015e84..d09f49530d9e 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1880,6 +1880,7 @@ int v4l2_ctrl_handler_init_class(struct v4l2_ctrl_handler *hdl,
 				      sizeof(hdl->buckets[0]),
 				      GFP_KERNEL | __GFP_ZERO);
 	hdl->error = hdl->buckets ? 0 : -ENOMEM;
+	media_request_object_init(&hdl->req_obj);
 	return hdl->error;
 }
 EXPORT_SYMBOL(v4l2_ctrl_handler_init_class);
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index f8faa54b5e7e..89a985607126 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -20,6 +20,7 @@
 #include <linux/list.h>
 #include <linux/mutex.h>
 #include <linux/videodev2.h>
+#include <media/media-request.h>
 
 /* forward references */
 struct file;
@@ -249,6 +250,8 @@ struct v4l2_ctrl {
  *		``prepare_ext_ctrls`` function at ``v4l2-ctrl.c``.
  * @from_other_dev: If true, then @ctrl was defined in another
  *		device then the &struct v4l2_ctrl_handler.
+ * @p_req:	The request value. Only used if the control handler
+ *		is bound to a media request.
  *
  * Each control handler has a list of these refs. The list_head is used to
  * keep a sorted-by-control-ID list of all controls, while the next pointer
@@ -260,6 +263,7 @@ struct v4l2_ctrl_ref {
 	struct v4l2_ctrl *ctrl;
 	struct v4l2_ctrl_helper *helper;
 	bool from_other_dev;
+	union v4l2_ctrl_ptr p_req;
 };
 
 /**
@@ -283,6 +287,8 @@ struct v4l2_ctrl_ref {
  * @notify_priv: Passed as argument to the v4l2_ctrl notify callback.
  * @nr_of_buckets: Total number of buckets in the array.
  * @error:	The error code of the first failed control addition.
+ * @req_obj:	The &struct media_request_object, used to link into a
+ *		&struct media_request.
  */
 struct v4l2_ctrl_handler {
 	struct mutex _lock;
@@ -295,6 +301,7 @@ struct v4l2_ctrl_handler {
 	void *notify_priv;
 	u16 nr_of_buckets;
 	int error;
+	struct media_request_object req_obj;
 };
 
 /**
-- 
2.16.3

[RFCv11 PATCH 12/29] v4l2-ctrls: alloc memory for p_req

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

To store request data the handler_new_ref() allocates memory
for it if needed.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index d09f49530d9e..3c1b00baa8d0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1997,13 +1997,18 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
 /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
 static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 			   struct v4l2_ctrl *ctrl,
-			   bool from_other_dev)
+			   struct v4l2_ctrl_ref **ctrl_ref,
+			   bool from_other_dev, bool allocate_req)
 {
 	struct v4l2_ctrl_ref *ref;
 	struct v4l2_ctrl_ref *new_ref;
 	u32 id = ctrl->id;
 	u32 class_ctrl = V4L2_CTRL_ID2WHICH(id) | 1;
 	int bucket = id % hdl->nr_of_buckets;	/* which bucket to use */
+	unsigned int sz_extra = 0;
+
+	if (ctrl_ref)
+		*ctrl_ref = NULL;
 
 	/*
 	 * Automatically add the control class if it is not yet present and
@@ -2017,11 +2022,16 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 	if (hdl->error)
 		return hdl->error;
 
-	new_ref = kzalloc(sizeof(*new_ref), GFP_KERNEL);
+	if (allocate_req)
+		sz_extra = ctrl->elems * ctrl->elem_size;
+	new_ref = kzalloc(sizeof(*new_ref) + sz_extra, GFP_KERNEL);
 	if (!new_ref)
 		return handler_set_err(hdl, -ENOMEM);
 	new_ref->ctrl = ctrl;
 	new_ref->from_other_dev = from_other_dev;
+	if (sz_extra)
+		new_ref->p_req.p = &new_ref[1];
+
 	if (ctrl->handler == hdl) {
 		/* By default each control starts in a cluster of its own.
 		   new_ref->ctrl is basically a cluster array with one
@@ -2061,6 +2071,8 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
 	/* Insert the control node in the hash */
 	new_ref->next = hdl->buckets[bucket];
 	hdl->buckets[bucket] = new_ref;
+	if (ctrl_ref)
+		*ctrl_ref = new_ref;
 
 unlock:
 	mutex_unlock(hdl->lock);
@@ -2202,7 +2214,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 		ctrl->type_ops->init(ctrl, idx, ctrl->p_new);
 	}
 
-	if (handler_new_ref(hdl, ctrl, false)) {
+	if (handler_new_ref(hdl, ctrl, NULL, false, false)) {
 		kvfree(ctrl);
 		return NULL;
 	}
@@ -2395,7 +2407,7 @@ int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
 		/* Filter any unwanted controls */
 		if (filter && !filter(ctrl))
 			continue;
-		ret = handler_new_ref(hdl, ctrl, from_other_dev);
+		ret = handler_new_ref(hdl, ctrl, NULL, from_other_dev, false);
 		if (ret)
 			break;
 	}
-- 
2.16.3

[RFCv11 PATCH 13/29] v4l2-ctrls: use ref in helper instead of ctrl

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

The next patch needs the reference to a control instead of the
control itself, so change struct v4l2_ctrl_helper accordingly.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 3c1b00baa8d0..da4cc1485dc4 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -37,8 +37,8 @@
 struct v4l2_ctrl_helper {
 	/* Pointer to the control reference of the master control */
 	struct v4l2_ctrl_ref *mref;
-	/* The control corresponding to the v4l2_ext_control ID field. */
-	struct v4l2_ctrl *ctrl;
+	/* The control ref corresponding to the v4l2_ext_control ID field. */
+	struct v4l2_ctrl_ref *ref;
 	/* v4l2_ext_control index of the next control belonging to the
 	   same cluster, or 0 if there isn't any. */
 	u32 next;
@@ -2887,6 +2887,7 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
 		ref = find_ref_lock(hdl, id);
 		if (ref == NULL)
 			return -EINVAL;
+		h->ref = ref;
 		ctrl = ref->ctrl;
 		if (ctrl->flags & V4L2_CTRL_FLAG_DISABLED)
 			return -EINVAL;
@@ -2909,7 +2910,6 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
 		}
 		/* Store the ref to the master control of the cluster */
 		h->mref = ref;
-		h->ctrl = ctrl;
 		/* Initially set next to 0, meaning that there is no other
 		   control in this helper array belonging to the same
 		   cluster */
@@ -2994,7 +2994,7 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
 	cs->error_idx = cs->count;
 
 	for (i = 0; !ret && i < cs->count; i++)
-		if (helpers[i].ctrl->flags & V4L2_CTRL_FLAG_WRITE_ONLY)
+		if (helpers[i].ref->ctrl->flags & V4L2_CTRL_FLAG_WRITE_ONLY)
 			ret = -EACCES;
 
 	for (i = 0; !ret && i < cs->count; i++) {
@@ -3029,7 +3029,7 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
 
 			do {
 				ret = ctrl_to_user(cs->controls + idx,
-						   helpers[idx].ctrl);
+						   helpers[idx].ref->ctrl);
 				idx = helpers[idx].next;
 			} while (!ret && idx);
 		}
@@ -3168,7 +3168,7 @@ static int validate_ctrls(struct v4l2_ext_controls *cs,
 
 	cs->error_idx = cs->count;
 	for (i = 0; i < cs->count; i++) {
-		struct v4l2_ctrl *ctrl = helpers[i].ctrl;
+		struct v4l2_ctrl *ctrl = helpers[i].ref->ctrl;
 		union v4l2_ctrl_ptr p_new;
 
 		cs->error_idx = i;
@@ -3280,7 +3280,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 			do {
 				/* Check if the auto control is part of the
 				   list, and remember the new value. */
-				if (helpers[tmp_idx].ctrl == master)
+				if (helpers[tmp_idx].ref->ctrl == master)
 					new_auto_val = cs->controls[tmp_idx].value;
 				tmp_idx = helpers[tmp_idx].next;
 			} while (tmp_idx);
@@ -3293,7 +3293,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 		/* Copy the new caller-supplied control values.
 		   user_to_new() sets 'is_new' to 1. */
 		do {
-			struct v4l2_ctrl *ctrl = helpers[idx].ctrl;
+			struct v4l2_ctrl *ctrl = helpers[idx].ref->ctrl;
 
 			ret = user_to_new(cs->controls + idx, ctrl);
 			if (!ret && ctrl->is_ptr)
@@ -3309,7 +3309,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 			idx = i;
 			do {
 				ret = new_to_user(cs->controls + idx,
-						helpers[idx].ctrl);
+						helpers[idx].ref->ctrl);
 				idx = helpers[idx].next;
 			} while (!ret && idx);
 		}
-- 
2.16.3

[RFCv11 PATCH 14/29] v4l2-ctrls: add core request support

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Integrate the request support. This adds the v4l2_ctrl_request_complete
and v4l2_ctrl_request_setup functions to complete a request and (as a
helper function) to apply a request to the hardware.

It takes care of queuing requests and correctly chaining control values
in the request queue.

Note that when a request is marked completed it will copy control values
to the internal request state. This can be optimized in the future since
this is sub-optimal when dealing with large compound and/or array controls.

For the initial 'stateless codec' use-case the current implementation is
sufficient.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 325 ++++++++++++++++++++++++++++++++++-
 include/media/v4l2-ctrls.h           |  23 +++
 2 files changed, 342 insertions(+), 6 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index da4cc1485dc4..6e2c5e24734f 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1647,6 +1647,13 @@ static int new_to_user(struct v4l2_ext_control *c,
 	return ptr_to_user(c, ctrl, ctrl->p_new);
 }
 
+/* Helper function: copy the request value back to the caller */
+static int req_to_user(struct v4l2_ext_control *c,
+		       struct v4l2_ctrl_ref *ref)
+{
+	return ptr_to_user(c, ref->ctrl, ref->p_req);
+}
+
 /* Helper function: copy the initial control value back to the caller */
 static int def_to_user(struct v4l2_ext_control *c, struct v4l2_ctrl *ctrl)
 {
@@ -1766,6 +1773,26 @@ static void cur_to_new(struct v4l2_ctrl *ctrl)
 	ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new);
 }
 
+/* Copy the new value to the request value */
+static void new_to_req(struct v4l2_ctrl_ref *ref)
+{
+	if (!ref)
+		return;
+	ptr_to_ptr(ref->ctrl, ref->ctrl->p_new, ref->p_req);
+	ref->req = ref;
+}
+
+/* Copy the request value to the new value */
+static void req_to_new(struct v4l2_ctrl_ref *ref)
+{
+	if (!ref)
+		return;
+	if (ref->req)
+		ptr_to_ptr(ref->ctrl, ref->req->p_req, ref->ctrl->p_new);
+	else
+		ptr_to_ptr(ref->ctrl, ref->ctrl->p_cur, ref->ctrl->p_new);
+}
+
 /* Return non-zero if one or more of the controls in the cluster has a new
    value that differs from the current value. */
 static int cluster_changed(struct v4l2_ctrl *master)
@@ -1875,6 +1902,9 @@ int v4l2_ctrl_handler_init_class(struct v4l2_ctrl_handler *hdl,
 	lockdep_set_class_and_name(hdl->lock, key, name);
 	INIT_LIST_HEAD(&hdl->ctrls);
 	INIT_LIST_HEAD(&hdl->ctrl_refs);
+	INIT_LIST_HEAD(&hdl->requests);
+	INIT_LIST_HEAD(&hdl->requests_queued);
+	hdl->request_is_queued = false;
 	hdl->nr_of_buckets = 1 + nr_of_controls_hint / 8;
 	hdl->buckets = kvmalloc_array(hdl->nr_of_buckets,
 				      sizeof(hdl->buckets[0]),
@@ -1895,6 +1925,14 @@ void v4l2_ctrl_handler_free(struct v4l2_ctrl_handler *hdl)
 	if (hdl == NULL || hdl->buckets == NULL)
 		return;
 
+	if (!hdl->req_obj.req && !list_empty(&hdl->requests)) {
+		struct v4l2_ctrl_handler *req, *next_req;
+
+		list_for_each_entry_safe(req, next_req, &hdl->requests, requests) {
+			media_request_object_unbind(&req->req_obj);
+			media_request_object_put(&req->req_obj);
+		}
+	}
 	mutex_lock(hdl->lock);
 	/* Free all nodes */
 	list_for_each_entry_safe(ref, next_ref, &hdl->ctrl_refs, node) {
@@ -2816,6 +2854,126 @@ int v4l2_querymenu(struct v4l2_ctrl_handler *hdl, struct v4l2_querymenu *qm)
 }
 EXPORT_SYMBOL(v4l2_querymenu);
 
+static int v4l2_ctrl_request_clone(struct v4l2_ctrl_handler *hdl,
+				   const struct v4l2_ctrl_handler *from)
+{
+	struct v4l2_ctrl_ref *ref;
+	int err;
+
+	if (WARN_ON(!hdl || hdl == from))
+		return -EINVAL;
+
+	if (hdl->error)
+		return hdl->error;
+
+	WARN_ON(hdl->lock != &hdl->_lock);
+
+	mutex_lock(from->lock);
+	list_for_each_entry(ref, &from->ctrl_refs, node) {
+		struct v4l2_ctrl *ctrl = ref->ctrl;
+		struct v4l2_ctrl_ref *new_ref;
+
+		/* Skip refs inherited from other devices */
+		if (ref->from_other_dev)
+			continue;
+		/* And buttons */
+		if (ctrl->type == V4L2_CTRL_TYPE_BUTTON)
+			continue;
+		err = handler_new_ref(hdl, ctrl, &new_ref, false, true);
+		if (err) {
+			printk("%s: handler_new_ref on control %x (%s) returned %d\n", __func__, ctrl->id, ctrl->name, err);
+			err = 0;
+			continue;
+		}
+		if (err)
+			break;
+	}
+	mutex_unlock(from->lock);
+	return err;
+}
+
+static void v4l2_ctrl_request_queue(struct media_request_object *obj)
+{
+	struct v4l2_ctrl_handler *hdl =
+		container_of(obj, struct v4l2_ctrl_handler, req_obj);
+	struct v4l2_ctrl_handler *main_hdl = obj->priv;
+	struct v4l2_ctrl_handler *prev = NULL;
+	struct v4l2_ctrl_ref *ref_hdl, *ref_prev = NULL;
+
+	if (list_empty(&main_hdl->requests_queued))
+		goto queue;
+
+	prev = list_last_entry(&main_hdl->requests_queued,
+			       struct v4l2_ctrl_handler, requests_queued);
+	mutex_lock(prev->lock);
+	ref_prev = list_first_entry(&prev->ctrl_refs,
+				    struct v4l2_ctrl_ref, node);
+	list_for_each_entry(ref_hdl, &hdl->ctrl_refs, node) {
+		if (ref_hdl->req)
+			continue;
+		while (ref_prev->ctrl->id < ref_hdl->ctrl->id)
+			ref_prev = list_next_entry(ref_prev, node);
+		if (WARN_ON(ref_prev->ctrl->id != ref_hdl->ctrl->id))
+			break;
+		ref_hdl->req = ref_prev->req;
+	}
+	mutex_unlock(prev->lock);
+queue:
+	list_add_tail(&hdl->requests_queued, &main_hdl->requests_queued);
+	hdl->request_is_queued = true;
+}
+
+static void v4l2_ctrl_request_unbind(struct media_request_object *obj)
+{
+	struct v4l2_ctrl_handler *hdl =
+		container_of(obj, struct v4l2_ctrl_handler, req_obj);
+
+	list_del_init(&hdl->requests);
+	if (hdl->request_is_queued) {
+		list_del_init(&hdl->requests_queued);
+		hdl->request_is_queued = false;
+	}
+}
+
+static void v4l2_ctrl_request_cancel(struct media_request_object *obj)
+{
+	struct v4l2_ctrl_handler *hdl =
+		container_of(obj, struct v4l2_ctrl_handler, req_obj);
+
+	if (hdl->request_is_queued)
+		v4l2_ctrl_request_complete(hdl->req_obj.req, hdl->req_obj.priv);
+}
+
+static void v4l2_ctrl_request_release(struct media_request_object *obj)
+{
+	struct v4l2_ctrl_handler *hdl =
+		container_of(obj, struct v4l2_ctrl_handler, req_obj);
+
+	v4l2_ctrl_handler_free(hdl);
+	kfree(hdl);
+}
+
+static const struct media_request_object_ops req_ops = {
+	.queue = v4l2_ctrl_request_queue,
+	.unbind = v4l2_ctrl_request_unbind,
+	.cancel = v4l2_ctrl_request_cancel,
+	.release = v4l2_ctrl_request_release,
+};
+
+static int v4l2_ctrl_request_bind(struct media_request *req,
+			   struct v4l2_ctrl_handler *hdl,
+			   struct v4l2_ctrl_handler *from)
+{
+	int ret;
+
+	ret = v4l2_ctrl_request_clone(hdl, from);
+
+	if (!ret) {
+		list_add_tail(&hdl->requests, &from->requests);
+		media_request_object_bind(req, &req_ops, from, &hdl->req_obj);
+	}
+	return ret;
+}
 
 /* Some general notes on the atomic requirements of VIDIOC_G/TRY/S_EXT_CTRLS:
 
@@ -2877,6 +3035,7 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
 
 		if (cs->which &&
 		    cs->which != V4L2_CTRL_WHICH_DEF_VAL &&
+		    cs->which != V4L2_CTRL_WHICH_REQUEST &&
 		    V4L2_CTRL_ID2WHICH(id) != cs->which)
 			return -EINVAL;
 
@@ -2956,13 +3115,12 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
    whether there are any controls at all. */
 static int class_check(struct v4l2_ctrl_handler *hdl, u32 which)
 {
-	if (which == 0 || which == V4L2_CTRL_WHICH_DEF_VAL)
+	if (which == 0 || which == V4L2_CTRL_WHICH_DEF_VAL ||
+	    which == V4L2_CTRL_WHICH_REQUEST)
 		return 0;
 	return find_ref_lock(hdl, which | 1) ? 0 : -EINVAL;
 }
 
-
-
 /* Get extended controls. Allocates the helpers array if needed. */
 int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs)
 {
@@ -3028,8 +3186,12 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
 			u32 idx = i;
 
 			do {
-				ret = ctrl_to_user(cs->controls + idx,
-						   helpers[idx].ref->ctrl);
+				if (helpers[idx].ref->req)
+					ret = req_to_user(cs->controls + idx,
+						helpers[idx].ref->req);
+				else
+					ret = ctrl_to_user(cs->controls + idx,
+						helpers[idx].ref->ctrl);
 				idx = helpers[idx].next;
 			} while (!ret && idx);
 		}
@@ -3302,7 +3464,16 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 		} while (!ret && idx);
 
 		if (!ret)
-			ret = try_or_set_cluster(fh, master, set, 0);
+			ret = try_or_set_cluster(fh, master,
+						 !hdl->req_obj.req && set, 0);
+		if (!ret && hdl->req_obj.req && set) {
+			for (j = 0; j < master->ncontrols; j++) {
+				struct v4l2_ctrl_ref *ref =
+					find_ref(hdl, master->cluster[j]->id);
+
+				new_to_req(ref);
+			}
+		}
 
 		/* Copy the new values back to userspace. */
 		if (!ret) {
@@ -3429,6 +3600,148 @@ int __v4l2_ctrl_s_ctrl_string(struct v4l2_ctrl *ctrl, const char *s)
 }
 EXPORT_SYMBOL(__v4l2_ctrl_s_ctrl_string);
 
+void v4l2_ctrl_request_complete(struct media_request *req,
+				struct v4l2_ctrl_handler *main_hdl)
+{
+	struct media_request_object *obj;
+	struct v4l2_ctrl_handler *hdl;
+	struct v4l2_ctrl_ref *ref;
+
+	if (!req || !main_hdl)
+		return;
+
+	obj = media_request_object_find(req, &req_ops, main_hdl);
+	if (!obj)
+		return;
+	hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj);
+
+	list_for_each_entry(ref, &hdl->ctrl_refs, node) {
+		struct v4l2_ctrl *ctrl = ref->ctrl;
+		struct v4l2_ctrl *master = ctrl->cluster[0];
+		unsigned int i;
+
+		if (ctrl->flags & V4L2_CTRL_FLAG_VOLATILE) {
+			ref->req = ref;
+
+			v4l2_ctrl_lock(master);
+			/* g_volatile_ctrl will update the current control values */
+			for (i = 0; i < master->ncontrols; i++)
+				cur_to_new(master->cluster[i]);
+			call_op(master, g_volatile_ctrl);
+			new_to_req(ref);
+			v4l2_ctrl_unlock(master);
+			continue;
+		}
+		if (ref->req == ref)
+			continue;
+
+		v4l2_ctrl_lock(ctrl);
+		if (ref->req)
+			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
+		else
+			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
+		v4l2_ctrl_unlock(ctrl);
+	}
+
+	WARN_ON(!hdl->request_is_queued);
+	list_del_init(&hdl->requests_queued);
+	hdl->request_is_queued = false;
+	media_request_object_complete(obj);
+	media_request_object_put(obj);
+}
+EXPORT_SYMBOL(v4l2_ctrl_request_complete);
+
+void v4l2_ctrl_request_setup(struct media_request *req,
+			     struct v4l2_ctrl_handler *main_hdl)
+{
+	struct media_request_object *obj;
+	struct v4l2_ctrl_handler *hdl;
+	struct v4l2_ctrl_ref *ref;
+
+	if (!req || !main_hdl)
+		return;
+
+	obj = media_request_object_find(req, &req_ops, main_hdl);
+	if (!obj)
+		return;
+	if (obj->completed) {
+		media_request_object_put(obj);
+		return;
+	}
+	hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj);
+
+	mutex_lock(hdl->lock);
+
+	list_for_each_entry(ref, &hdl->ctrl_refs, node)
+		ref->done = false;
+
+	list_for_each_entry(ref, &hdl->ctrl_refs, node) {
+		struct v4l2_ctrl *ctrl = ref->ctrl;
+		struct v4l2_ctrl *master = ctrl->cluster[0];
+		bool have_new_data = false;
+		int i;
+
+		/* Skip if this control was already handled by a cluster. */
+		/* Skip button controls and read-only controls. */
+		if (ref->done || ctrl->type == V4L2_CTRL_TYPE_BUTTON ||
+		    (ctrl->flags & V4L2_CTRL_FLAG_READ_ONLY))
+			continue;
+
+		v4l2_ctrl_lock(master);
+		for (i = 0; i < master->ncontrols; i++) {
+			if (master->cluster[i]) {
+				struct v4l2_ctrl_ref *r =
+					find_ref(hdl, master->cluster[i]->id);
+
+				if (r->req && r == r->req) {
+					have_new_data = true;
+					break;
+				}
+			}
+		}
+		if (!have_new_data)
+			continue;
+
+		for (i = 0; i < master->ncontrols; i++) {
+			if (master->cluster[i]) {
+				struct v4l2_ctrl_ref *r =
+					find_ref(hdl, master->cluster[i]->id);
+
+				req_to_new(r);
+				master->cluster[i]->is_new = 1;
+				r->done = true;
+			}
+		}
+		/*
+		 * For volatile autoclusters that are currently in auto mode
+		 * we need to discover if it will be set to manual mode.
+		 * If so, then we have to copy the current volatile values
+		 * first since those will become the new manual values (which
+		 * may be overwritten by explicit new values from this set
+		 * of controls).
+		 */
+		if (master->is_auto && master->has_volatiles &&
+		    !is_cur_manual(master)) {
+			s32 new_auto_val = *master->p_new.p_s32;
+
+			/*
+			 * If the new value == the manual value, then copy
+			 * the current volatile values.
+			 */
+			if (new_auto_val == master->manual_mode_value)
+				update_from_auto_cluster(master);
+		}
+
+		try_or_set_cluster(NULL, master, true, 0);
+
+		v4l2_ctrl_unlock(master);
+	}
+
+	mutex_unlock(hdl->lock);
+	media_request_object_put(obj);
+}
+EXPORT_SYMBOL(v4l2_ctrl_request_setup);
+
 void v4l2_ctrl_notify(struct v4l2_ctrl *ctrl, v4l2_ctrl_notify_fnc notify, void *priv)
 {
 	if (ctrl == NULL)
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 89a985607126..9499846aa1d1 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -250,6 +250,10 @@ struct v4l2_ctrl {
  *		``prepare_ext_ctrls`` function at ``v4l2-ctrl.c``.
  * @from_other_dev: If true, then @ctrl was defined in another
  *		device then the &struct v4l2_ctrl_handler.
+ * @done:	If true, then this control reference is part of a
+ *		control cluster that was already set while applying
+ *		the controls in this media request object.
+ * @req:	If set, this refers to another request that sets this control.
  * @p_req:	The request value. Only used if the control handler
  *		is bound to a media request.
  *
@@ -263,6 +267,8 @@ struct v4l2_ctrl_ref {
 	struct v4l2_ctrl *ctrl;
 	struct v4l2_ctrl_helper *helper;
 	bool from_other_dev;
+	bool done;
+	struct v4l2_ctrl_ref *req;
 	union v4l2_ctrl_ptr p_req;
 };
 
@@ -287,6 +293,15 @@ struct v4l2_ctrl_ref {
  * @notify_priv: Passed as argument to the v4l2_ctrl notify callback.
  * @nr_of_buckets: Total number of buckets in the array.
  * @error:	The error code of the first failed control addition.
+ * @request_is_queued: True if the request was queued.
+ * @requests:	List to keep track of open control handler request objects.
+ *		For the parent control handler (@req_obj.req == NULL) this
+ *		is the list header. When the parent control handler is
+ *		removed, it has to unbind and put all these requests since
+ *		they refer to the parent.
+ * @requests_queued: List of the queued requests. This determines the order
+ *		in which these controls are applied. Once the request is
+ *		completed it is removed from this list.
  * @req_obj:	The &struct media_request_object, used to link into a
  *		&struct media_request.
  */
@@ -301,6 +316,9 @@ struct v4l2_ctrl_handler {
 	void *notify_priv;
 	u16 nr_of_buckets;
 	int error;
+	bool request_is_queued;
+	struct list_head requests;
+	struct list_head requests_queued;
 	struct media_request_object req_obj;
 };
 
@@ -1059,6 +1077,11 @@ int v4l2_ctrl_subscribe_event(struct v4l2_fh *fh,
  */
 __poll_t v4l2_ctrl_poll(struct file *file, struct poll_table_struct *wait);
 
+void v4l2_ctrl_request_setup(struct media_request *req,
+			     struct v4l2_ctrl_handler *hdl);
+void v4l2_ctrl_request_complete(struct media_request *req,
+				struct v4l2_ctrl_handler *hdl);
+
 /* Helpers for ioctl_ops */
 
 /**
-- 
2.16.3

[RFCv11 PATCH 15/29] v4l2-ctrls: support g/s_ext_ctrls for requests

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

The v4l2_g/s_ext_ctrls functions now support control handlers that
represent requests.

The v4l2_ctrls_find_req_obj() function is responsible for finding the
request from the fd.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/omap3isp/ispvideo.c |   2 +-
 drivers/media/v4l2-core/v4l2-ctrls.c       | 113 +++++++++++++++++++++++++++--
 drivers/media/v4l2-core/v4l2-ioctl.c       |  12 +--
 drivers/media/v4l2-core/v4l2-subdev.c      |   9 ++-
 include/media/v4l2-ctrls.h                 |   7 +-
 5 files changed, 124 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index a751c89a3ea8..bd564c2e767f 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -1028,7 +1028,7 @@ static int isp_video_check_external_subdevs(struct isp_video *video,
 	ctrls.count = 1;
 	ctrls.controls = &ctrl;
 
-	ret = v4l2_g_ext_ctrls(pipe->external->ctrl_handler, &ctrls);
+	ret = v4l2_g_ext_ctrls(pipe->external->ctrl_handler, NULL, &ctrls);
 	if (ret < 0) {
 		dev_warn(isp->dev, "no pixel rate control in subdev %s\n",
 			 pipe->external->name);
diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 6e2c5e24734f..317a374b6ea6 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -3122,7 +3122,8 @@ static int class_check(struct v4l2_ctrl_handler *hdl, u32 which)
 }
 
 /* Get extended controls. Allocates the helpers array if needed. */
-int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs)
+int __v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl,
+		       struct v4l2_ext_controls *cs)
 {
 	struct v4l2_ctrl_helper helper[4];
 	struct v4l2_ctrl_helper *helpers = helper;
@@ -3202,6 +3203,73 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
 		kvfree(helpers);
 	return ret;
 }
+
+static struct media_request_object *
+v4l2_ctrls_find_req_obj(struct v4l2_ctrl_handler *hdl,
+			struct media_device *mdev, s32 fd, bool set)
+{
+	struct media_request *req = media_request_find(mdev, fd);
+	struct media_request_object *obj;
+	struct v4l2_ctrl_handler *new_hdl;
+	int ret;
+
+	if (IS_ERR(req))
+		return ERR_CAST(req);
+
+	if (set && req->state != MEDIA_REQUEST_STATE_IDLE) {
+		media_request_put(req);
+		return ERR_PTR(-EBUSY);
+	}
+
+	obj = media_request_object_find(req, &req_ops, hdl);
+	if (obj) {
+		media_request_put(req);
+		return obj;
+	}
+
+	new_hdl = kzalloc(sizeof(*new_hdl), GFP_KERNEL);
+	if (!new_hdl) {
+		ret = -ENOMEM;
+		goto put;
+	}
+	obj = &new_hdl->req_obj;
+	ret = v4l2_ctrl_handler_init(new_hdl, (hdl->nr_of_buckets - 1) * 8);
+	if (!ret)
+		ret = v4l2_ctrl_request_bind(req, new_hdl, hdl);
+	if (!ret) {
+		media_request_object_get(obj);
+		media_request_put(req);
+		return obj;
+	}
+	kfree(new_hdl);
+
+put:
+	media_request_put(req);
+	return ERR_PTR(ret);
+}
+
+int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
+		     struct v4l2_ext_controls *cs)
+{
+	struct media_request_object *obj = NULL;
+	int ret;
+
+	if (cs->which == V4L2_CTRL_WHICH_REQUEST) {
+		if (!mdev || cs->request_fd < 0)
+			return -EINVAL;
+		obj = v4l2_ctrls_find_req_obj(hdl, mdev, cs->request_fd, false);
+		if (IS_ERR(obj))
+			return PTR_ERR(obj);
+		hdl = container_of(obj, struct v4l2_ctrl_handler,
+				   req_obj);
+	}
+
+	ret = __v4l2_g_ext_ctrls(hdl, cs);
+
+	if (obj)
+		media_request_object_put(obj);
+	return ret;
+}
 EXPORT_SYMBOL(v4l2_g_ext_ctrls);
 
 /* Helper function to get a single control */
@@ -3377,9 +3445,9 @@ static void update_from_auto_cluster(struct v4l2_ctrl *master)
 }
 
 /* Try or try-and-set controls */
-static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
-			     struct v4l2_ext_controls *cs,
-			     bool set)
+static int __try_set_ext_ctrls(struct v4l2_fh *fh,
+			       struct v4l2_ctrl_handler *hdl,
+			       struct v4l2_ext_controls *cs, bool set)
 {
 	struct v4l2_ctrl_helper helper[4];
 	struct v4l2_ctrl_helper *helpers = helper;
@@ -3492,16 +3560,45 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
 	return ret;
 }
 
-int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs)
+static int try_set_ext_ctrls(struct v4l2_fh *fh,
+			     struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
+			     struct v4l2_ext_controls *cs, bool set)
+{
+	struct media_request_object *obj = NULL;
+	int ret;
+
+	if (cs->which == V4L2_CTRL_WHICH_REQUEST) {
+		if (!mdev || cs->request_fd < 0)
+			return -EINVAL;
+		obj = v4l2_ctrls_find_req_obj(hdl, mdev, cs->request_fd, true);
+		if (IS_ERR(obj))
+			return PTR_ERR(obj);
+		if (obj->req->state != MEDIA_REQUEST_STATE_IDLE) {
+			media_request_object_put(obj);
+			return -EBUSY;
+		}
+		hdl = container_of(obj, struct v4l2_ctrl_handler,
+				   req_obj);
+	}
+
+	ret = __try_set_ext_ctrls(fh, hdl, cs, set);
+
+	if (obj)
+		media_request_object_put(obj);
+	return ret;
+}
+
+int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
+		       struct v4l2_ext_controls *cs)
 {
-	return try_set_ext_ctrls(NULL, hdl, cs, false);
+	return try_set_ext_ctrls(NULL, hdl, mdev, cs, false);
 }
 EXPORT_SYMBOL(v4l2_try_ext_ctrls);
 
 int v4l2_s_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
-					struct v4l2_ext_controls *cs)
+		     struct media_device *mdev, struct v4l2_ext_controls *cs)
 {
-	return try_set_ext_ctrls(fh, hdl, cs, true);
+	return try_set_ext_ctrls(fh, hdl, mdev, cs, true);
 }
 EXPORT_SYMBOL(v4l2_s_ext_ctrls);
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 9ce23e23c5bf..56741c4a48fc 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2079,9 +2079,9 @@ static int v4l_g_ext_ctrls(const struct v4l2_ioctl_ops *ops,
 
 	p->error_idx = p->count;
 	if (vfh && vfh->ctrl_handler)
-		return v4l2_g_ext_ctrls(vfh->ctrl_handler, p);
+		return v4l2_g_ext_ctrls(vfh->ctrl_handler, vfd->v4l2_dev->mdev, p);
 	if (vfd->ctrl_handler)
-		return v4l2_g_ext_ctrls(vfd->ctrl_handler, p);
+		return v4l2_g_ext_ctrls(vfd->ctrl_handler, vfd->v4l2_dev->mdev, p);
 	if (ops->vidioc_g_ext_ctrls == NULL)
 		return -ENOTTY;
 	return check_ext_ctrls(p, 0) ? ops->vidioc_g_ext_ctrls(file, fh, p) :
@@ -2098,9 +2098,9 @@ static int v4l_s_ext_ctrls(const struct v4l2_ioctl_ops *ops,
 
 	p->error_idx = p->count;
 	if (vfh && vfh->ctrl_handler)
-		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler, p);
+		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler, vfd->v4l2_dev->mdev, p);
 	if (vfd->ctrl_handler)
-		return v4l2_s_ext_ctrls(NULL, vfd->ctrl_handler, p);
+		return v4l2_s_ext_ctrls(NULL, vfd->ctrl_handler, vfd->v4l2_dev->mdev, p);
 	if (ops->vidioc_s_ext_ctrls == NULL)
 		return -ENOTTY;
 	return check_ext_ctrls(p, 0) ? ops->vidioc_s_ext_ctrls(file, fh, p) :
@@ -2117,9 +2117,9 @@ static int v4l_try_ext_ctrls(const struct v4l2_ioctl_ops *ops,
 
 	p->error_idx = p->count;
 	if (vfh && vfh->ctrl_handler)
-		return v4l2_try_ext_ctrls(vfh->ctrl_handler, p);
+		return v4l2_try_ext_ctrls(vfh->ctrl_handler, vfd->v4l2_dev->mdev, p);
 	if (vfd->ctrl_handler)
-		return v4l2_try_ext_ctrls(vfd->ctrl_handler, p);
+		return v4l2_try_ext_ctrls(vfd->ctrl_handler, vfd->v4l2_dev->mdev, p);
 	if (ops->vidioc_try_ext_ctrls == NULL)
 		return -ENOTTY;
 	return check_ext_ctrls(p, 0) ? ops->vidioc_try_ext_ctrls(file, fh, p) :
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index f9eed938d348..ce8c133e0ccd 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -222,17 +222,20 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
 	case VIDIOC_G_EXT_CTRLS:
 		if (!vfh->ctrl_handler)
 			return -ENOTTY;
-		return v4l2_g_ext_ctrls(vfh->ctrl_handler, arg);
+		return v4l2_g_ext_ctrls(vfh->ctrl_handler,
+					sd->v4l2_dev->mdev, arg);
 
 	case VIDIOC_S_EXT_CTRLS:
 		if (!vfh->ctrl_handler)
 			return -ENOTTY;
-		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler, arg);
+		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler,
+					sd->v4l2_dev->mdev, arg);
 
 	case VIDIOC_TRY_EXT_CTRLS:
 		if (!vfh->ctrl_handler)
 			return -ENOTTY;
-		return v4l2_try_ext_ctrls(vfh->ctrl_handler, arg);
+		return v4l2_try_ext_ctrls(vfh->ctrl_handler,
+					  sd->v4l2_dev->mdev, arg);
 
 	case VIDIOC_DQEVENT:
 		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_EVENTS))
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index 9499846aa1d1..be27835334a2 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -1148,11 +1148,12 @@ int v4l2_s_ctrl(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
  *	:ref:`VIDIOC_G_EXT_CTRLS <vidioc_g_ext_ctrls>` ioctl
  *
  * @hdl: pointer to &struct v4l2_ctrl_handler
+ * @mdev: pointer to &struct media_device
  * @c: pointer to &struct v4l2_ext_controls
  *
  * If hdl == NULL then they will all return -EINVAL.
  */
-int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl,
+int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
 		     struct v4l2_ext_controls *c);
 
 /**
@@ -1160,11 +1161,13 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl,
  *	:ref:`VIDIOC_TRY_EXT_CTRLS <vidioc_g_ext_ctrls>` ioctl
  *
  * @hdl: pointer to &struct v4l2_ctrl_handler
+ * @mdev: pointer to &struct media_device
  * @c: pointer to &struct v4l2_ext_controls
  *
  * If hdl == NULL then they will all return -EINVAL.
  */
 int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl,
+		       struct media_device *mdev,
 		       struct v4l2_ext_controls *c);
 
 /**
@@ -1173,11 +1176,13 @@ int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl,
  *
  * @fh: pointer to &struct v4l2_fh
  * @hdl: pointer to &struct v4l2_ctrl_handler
+ * @mdev: pointer to &struct media_device
  * @c: pointer to &struct v4l2_ext_controls
  *
  * If hdl == NULL then they will all return -EINVAL.
  */
 int v4l2_s_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
+		     struct media_device *mdev,
 		     struct v4l2_ext_controls *c);
 
 /**
-- 
2.16.3

[RFCv11 PATCH 16/29] videodev2.h: Add request_fd field to v4l2_buffer

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

When queuing buffers allow for passing the request that should
be associated with this buffer.

If V4L2_BUF_FLAG_REQUEST_FD is set, then request_fd is used as
the file descriptor.

If a buffer is stored in a request, but not yet queued to the
driver, then V4L2_BUF_FLAG_IN_REQUEST is set.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/common/videobuf2/videobuf2-v4l2.c |  2 +-
 drivers/media/usb/cpia2/cpia2_v4l.c             |  2 +-
 drivers/media/v4l2-core/v4l2-compat-ioctl32.c   |  9 ++++++---
 drivers/media/v4l2-core/v4l2-ioctl.c            |  4 ++--
 include/uapi/linux/videodev2.h                  | 10 +++++++++-
 5 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 886a2d8d5c6c..4e9c77f21858 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -204,7 +204,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
 	b->timecode = vbuf->timecode;
 	b->sequence = vbuf->sequence;
 	b->reserved2 = 0;
-	b->reserved = 0;
+	b->request_fd = 0;
 
 	if (q->is_multiplanar) {
 		/*
diff --git a/drivers/media/usb/cpia2/cpia2_v4l.c b/drivers/media/usb/cpia2/cpia2_v4l.c
index 99f106b13280..13aee9f67d05 100644
--- a/drivers/media/usb/cpia2/cpia2_v4l.c
+++ b/drivers/media/usb/cpia2/cpia2_v4l.c
@@ -949,7 +949,7 @@ static int cpia2_dqbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
 	buf->m.offset = cam->buffers[buf->index].data - cam->frame_buffer;
 	buf->length = cam->frame_size;
 	buf->reserved2 = 0;
-	buf->reserved = 0;
+	buf->request_fd = 0;
 	memset(&buf->timecode, 0, sizeof(buf->timecode));
 
 	DBG("DQBUF #%d status:%d seq:%d length:%d\n", buf->index,
diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
index 0782b3666deb..7e27d0feac94 100644
--- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
+++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
@@ -387,7 +387,7 @@ struct v4l2_buffer32 {
 	} m;
 	__u32			length;
 	__u32			reserved2;
-	__u32			reserved;
+	__s32			request_fd;
 };
 
 static int get_v4l2_plane32(struct v4l2_plane __user *up,
@@ -486,6 +486,7 @@ static int get_v4l2_buffer32(struct v4l2_buffer __user *kp,
 {
 	u32 type;
 	u32 length;
+	s32 request_fd;
 	enum v4l2_memory memory;
 	struct v4l2_plane32 __user *uplane32;
 	struct v4l2_plane __user *uplane;
@@ -500,7 +501,9 @@ static int get_v4l2_buffer32(struct v4l2_buffer __user *kp,
 	    get_user(memory, &up->memory) ||
 	    put_user(memory, &kp->memory) ||
 	    get_user(length, &up->length) ||
-	    put_user(length, &kp->length))
+	    put_user(length, &kp->length) ||
+	    get_user(request_fd, &up->request_fd) ||
+	    put_user(request_fd, &kp->request_fd))
 		return -EFAULT;
 
 	if (V4L2_TYPE_IS_OUTPUT(type))
@@ -605,7 +608,7 @@ static int put_v4l2_buffer32(struct v4l2_buffer __user *kp,
 	    copy_in_user(&up->timecode, &kp->timecode, sizeof(kp->timecode)) ||
 	    assign_in_user(&up->sequence, &kp->sequence) ||
 	    assign_in_user(&up->reserved2, &kp->reserved2) ||
-	    assign_in_user(&up->reserved, &kp->reserved) ||
+	    assign_in_user(&up->request_fd, &kp->request_fd) ||
 	    get_user(length, &kp->length) ||
 	    put_user(length, &up->length))
 		return -EFAULT;
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 56741c4a48fc..561a1fe3160b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -437,13 +437,13 @@ static void v4l_print_buffer(const void *arg, bool write_only)
 	const struct v4l2_plane *plane;
 	int i;
 
-	pr_cont("%02ld:%02d:%02d.%08ld index=%d, type=%s, flags=0x%08x, field=%s, sequence=%d, memory=%s",
+	pr_cont("%02ld:%02d:%02d.%08ld index=%d, type=%s, request_fd=%d, flags=0x%08x, field=%s, sequence=%d, memory=%s",
 			p->timestamp.tv_sec / 3600,
 			(int)(p->timestamp.tv_sec / 60) % 60,
 			(int)(p->timestamp.tv_sec % 60),
 			(long)p->timestamp.tv_usec,
 			p->index,
-			prt_names(p->type, v4l2_type_names),
+			prt_names(p->type, v4l2_type_names), p->request_fd,
 			p->flags, prt_names(p->field, v4l2_field_names),
 			p->sequence, prt_names(p->memory, v4l2_memory_names));
 
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index d6e5c245bdcf..9c65d890a5f2 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -910,6 +910,7 @@ struct v4l2_plane {
  * @length:	size in bytes of the buffer (NOT its payload) for single-plane
  *		buffers (when type != *_MPLANE); number of elements in the
  *		planes array for multi-plane buffers
+ * @request_fd: fd of the request that this buffer should use
  *
  * Contains data exchanged by application and driver using one of the Streaming
  * I/O methods.
@@ -934,7 +935,10 @@ struct v4l2_buffer {
 	} m;
 	__u32			length;
 	__u32			reserved2;
-	__u32			reserved;
+	union {
+		__s32		request_fd;
+		__u32		reserved;
+	};
 };
 
 /*  Flags for 'flags' field */
@@ -952,6 +956,8 @@ struct v4l2_buffer {
 #define V4L2_BUF_FLAG_BFRAME			0x00000020
 /* Buffer is ready, but the data contained within is corrupted. */
 #define V4L2_BUF_FLAG_ERROR			0x00000040
+/* Buffer is added to an unqueued request */
+#define V4L2_BUF_FLAG_IN_REQUEST		0x00000080
 /* timecode field is valid */
 #define V4L2_BUF_FLAG_TIMECODE			0x00000100
 /* Buffer is prepared for queuing */
@@ -970,6 +976,8 @@ struct v4l2_buffer {
 #define V4L2_BUF_FLAG_TSTAMP_SRC_SOE		0x00010000
 /* mem2mem encoder/decoder */
 #define V4L2_BUF_FLAG_LAST			0x00100000
+/* request_fd is valid */
+#define V4L2_BUF_FLAG_REQUEST_FD		0x00800000
 
 /**
  * struct v4l2_exportbuffer - export of video buffer as DMABUF file descriptor
-- 
2.16.3

[RFCv11 PATCH 17/29] vb2: store userspace data in vb2_v4l2_buffer

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

The userspace-provided plane data needs to be stored in
vb2_v4l2_buffer. Currently this information is applied by
__fill_vb2_buffer() which is called by the core prepare_buf
and qbuf functions, but when using requests these functions
aren't called yet since the buffer won't be prepared until
the media request is actually queued.

In the meantime this information has to be stored somewhere
and vb2_v4l2_buffer is a good place for it.

The __fill_vb2_buffer callback now just copies the relevant
information from vb2_v4l2_buffer into the planes array.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c |  25 +-
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 324 +++++++++++++-----------
 drivers/media/dvb-core/dvb_vb2.c                |   3 +-
 include/media/videobuf2-core.h                  |   3 +-
 include/media/videobuf2-v4l2.h                  |   2 +
 5 files changed, 197 insertions(+), 160 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index d3f7bb33a54d..3d436ccb61f8 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -968,9 +968,8 @@ static int __prepare_mmap(struct vb2_buffer *vb, const void *pb)
 {
 	int ret = 0;
 
-	if (pb)
-		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
-				 vb, pb, vb->planes);
+	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
+			 vb, vb->planes);
 	return ret ? ret : call_vb_qop(vb, buf_prepare, vb);
 }
 
@@ -988,12 +987,10 @@ static int __prepare_userptr(struct vb2_buffer *vb, const void *pb)
 
 	memset(planes, 0, sizeof(planes[0]) * vb->num_planes);
 	/* Copy relevant information provided by the userspace */
-	if (pb) {
-		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
-				 vb, pb, planes);
-		if (ret)
-			return ret;
-	}
+	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
+			 vb, planes);
+	if (ret)
+		return ret;
 
 	for (plane = 0; plane < vb->num_planes; ++plane) {
 		/* Skip the plane if already verified */
@@ -1104,12 +1101,10 @@ static int __prepare_dmabuf(struct vb2_buffer *vb, const void *pb)
 
 	memset(planes, 0, sizeof(planes[0]) * vb->num_planes);
 	/* Copy relevant information provided by the userspace */
-	if (pb) {
-		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
-				 vb, pb, planes);
-		if (ret)
-			return ret;
-	}
+	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
+			 vb, planes);
+	if (ret)
+		return ret;
 
 	for (plane = 0; plane < vb->num_planes; ++plane) {
 		struct dma_buf *dbuf = dma_buf_get(planes[plane].m.fd);
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 4e9c77f21858..bf7a3ba9fed0 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -154,9 +154,177 @@ static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
 		pr_warn("use the actual size instead.\n");
 }
 
+static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b)
+{
+	struct vb2_queue *q = vb->vb2_queue;
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct vb2_plane *planes = vbuf->planes;
+	unsigned int plane;
+	int ret;
+
+	ret = __verify_length(vb, b);
+	if (ret < 0) {
+		dprintk(1, "plane parameters verification failed: %d\n", ret);
+		return ret;
+	}
+	if (b->field == V4L2_FIELD_ALTERNATE && q->is_output) {
+		/*
+		 * If the format's field is ALTERNATE, then the buffer's field
+		 * should be either TOP or BOTTOM, not ALTERNATE since that
+		 * makes no sense. The driver has to know whether the
+		 * buffer represents a top or a bottom field in order to
+		 * program any DMA correctly. Using ALTERNATE is wrong, since
+		 * that just says that it is either a top or a bottom field,
+		 * but not which of the two it is.
+		 */
+		dprintk(1, "the field is incorrectly set to ALTERNATE for an output buffer\n");
+		return -EINVAL;
+	}
+	vbuf->sequence = 0;
+
+	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
+		switch (b->memory) {
+		case VB2_MEMORY_USERPTR:
+			for (plane = 0; plane < vb->num_planes; ++plane) {
+				planes[plane].m.userptr =
+					b->m.planes[plane].m.userptr;
+				planes[plane].length =
+					b->m.planes[plane].length;
+			}
+			break;
+		case VB2_MEMORY_DMABUF:
+			for (plane = 0; plane < vb->num_planes; ++plane) {
+				planes[plane].m.fd =
+					b->m.planes[plane].m.fd;
+				planes[plane].length =
+					b->m.planes[plane].length;
+			}
+			break;
+		default:
+			for (plane = 0; plane < vb->num_planes; ++plane) {
+				planes[plane].m.offset =
+					vb->planes[plane].m.offset;
+				planes[plane].length =
+					vb->planes[plane].length;
+			}
+			break;
+		}
+
+		/* Fill in driver-provided information for OUTPUT types */
+		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
+			/*
+			 * Will have to go up to b->length when API starts
+			 * accepting variable number of planes.
+			 *
+			 * If bytesused == 0 for the output buffer, then fall
+			 * back to the full buffer size. In that case
+			 * userspace clearly never bothered to set it and
+			 * it's a safe assumption that they really meant to
+			 * use the full plane sizes.
+			 *
+			 * Some drivers, e.g. old codec drivers, use bytesused == 0
+			 * as a way to indicate that streaming is finished.
+			 * In that case, the driver should use the
+			 * allow_zero_bytesused flag to keep old userspace
+			 * applications working.
+			 */
+			for (plane = 0; plane < vb->num_planes; ++plane) {
+				struct vb2_plane *pdst = &planes[plane];
+				struct v4l2_plane *psrc = &b->m.planes[plane];
+
+				if (psrc->bytesused == 0)
+					vb2_warn_zero_bytesused(vb);
+
+				if (vb->vb2_queue->allow_zero_bytesused)
+					pdst->bytesused = psrc->bytesused;
+				else
+					pdst->bytesused = psrc->bytesused ?
+						psrc->bytesused : pdst->length;
+				pdst->data_offset = psrc->data_offset;
+			}
+		}
+	} else {
+		/*
+		 * Single-planar buffers do not use planes array,
+		 * so fill in relevant v4l2_buffer struct fields instead.
+		 * In videobuf we use our internal V4l2_planes struct for
+		 * single-planar buffers as well, for simplicity.
+		 *
+		 * If bytesused == 0 for the output buffer, then fall back
+		 * to the full buffer size as that's a sensible default.
+		 *
+		 * Some drivers, e.g. old codec drivers, use bytesused == 0 as
+		 * a way to indicate that streaming is finished. In that case,
+		 * the driver should use the allow_zero_bytesused flag to keep
+		 * old userspace applications working.
+		 */
+		switch (b->memory) {
+		case VB2_MEMORY_USERPTR:
+			planes[0].m.userptr = b->m.userptr;
+			planes[0].length = b->length;
+			break;
+		case VB2_MEMORY_DMABUF:
+			planes[0].m.fd = b->m.fd;
+			planes[0].length = b->length;
+			break;
+		default:
+			planes[0].m.offset = vb->planes[0].m.offset;
+			planes[0].length = vb->planes[0].length;
+			break;
+		}
+
+		planes[0].data_offset = 0;
+		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
+			if (b->bytesused == 0)
+				vb2_warn_zero_bytesused(vb);
+
+			if (vb->vb2_queue->allow_zero_bytesused)
+				planes[0].bytesused = b->bytesused;
+			else
+				planes[0].bytesused = b->bytesused ?
+					b->bytesused : planes[0].length;
+		} else
+			planes[0].bytesused = 0;
+
+	}
+
+	/* Zero flags that we handle */
+	vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
+	if (!vb->vb2_queue->copy_timestamp || !V4L2_TYPE_IS_OUTPUT(b->type)) {
+		/*
+		 * Non-COPY timestamps and non-OUTPUT queues will get
+		 * their timestamp and timestamp source flags from the
+		 * queue.
+		 */
+		vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
+	}
+
+	if (V4L2_TYPE_IS_OUTPUT(b->type)) {
+		/*
+		 * For output buffers mask out the timecode flag:
+		 * this will be handled later in vb2_qbuf().
+		 * The 'field' is valid metadata for this output buffer
+		 * and so that needs to be copied here.
+		 */
+		vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
+		vbuf->field = b->field;
+	} else {
+		/* Zero any output buffer flags as this is a capture buffer */
+		vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
+		/* Zero last flag, this is a signal from driver to userspace */
+		vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
+	}
+
+	return 0;
+}
+
 static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
 				    const char *opname)
 {
+	struct vb2_v4l2_buffer *vbuf;
+	struct vb2_buffer *vb;
+	int ret;
+
 	if (b->type != q->type) {
 		dprintk(1, "%s: invalid buffer type\n", opname);
 		return -EINVAL;
@@ -178,7 +346,15 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
 		return -EINVAL;
 	}
 
-	return __verify_planes_array(q->bufs[b->index], b);
+	vb = q->bufs[b->index];
+	vbuf = to_vb2_v4l2_buffer(vb);
+	ret = __verify_planes_array(vb, b);
+	if (ret)
+		return ret;
+
+	/* Copy relevant information provided by the userspace */
+	memset(vbuf->planes, 0, sizeof(vbuf->planes[0]) * vb->num_planes);
+	return vb2_fill_vb2_v4l2_buffer(vb, b);
 }
 
 /*
@@ -291,153 +467,19 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
  * v4l2_buffer by the userspace. It also verifies that struct
  * v4l2_buffer has a valid number of planes.
  */
-static int __fill_vb2_buffer(struct vb2_buffer *vb,
-		const void *pb, struct vb2_plane *planes)
+static int __fill_vb2_buffer(struct vb2_buffer *vb, struct vb2_plane *planes)
 {
-	struct vb2_queue *q = vb->vb2_queue;
-	const struct v4l2_buffer *b = pb;
 	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
 	unsigned int plane;
-	int ret;
 
-	ret = __verify_length(vb, b);
-	if (ret < 0) {
-		dprintk(1, "plane parameters verification failed: %d\n", ret);
-		return ret;
-	}
-	if (b->field == V4L2_FIELD_ALTERNATE && q->is_output) {
-		/*
-		 * If the format's field is ALTERNATE, then the buffer's field
-		 * should be either TOP or BOTTOM, not ALTERNATE since that
-		 * makes no sense. The driver has to know whether the
-		 * buffer represents a top or a bottom field in order to
-		 * program any DMA correctly. Using ALTERNATE is wrong, since
-		 * that just says that it is either a top or a bottom field,
-		 * but not which of the two it is.
-		 */
-		dprintk(1, "the field is incorrectly set to ALTERNATE for an output buffer\n");
-		return -EINVAL;
-	}
 	vb->timestamp = 0;
-	vbuf->sequence = 0;
-
-	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
-		if (b->memory == VB2_MEMORY_USERPTR) {
-			for (plane = 0; plane < vb->num_planes; ++plane) {
-				planes[plane].m.userptr =
-					b->m.planes[plane].m.userptr;
-				planes[plane].length =
-					b->m.planes[plane].length;
-			}
-		}
-		if (b->memory == VB2_MEMORY_DMABUF) {
-			for (plane = 0; plane < vb->num_planes; ++plane) {
-				planes[plane].m.fd =
-					b->m.planes[plane].m.fd;
-				planes[plane].length =
-					b->m.planes[plane].length;
-			}
-		}
-
-		/* Fill in driver-provided information for OUTPUT types */
-		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
-			/*
-			 * Will have to go up to b->length when API starts
-			 * accepting variable number of planes.
-			 *
-			 * If bytesused == 0 for the output buffer, then fall
-			 * back to the full buffer size. In that case
-			 * userspace clearly never bothered to set it and
-			 * it's a safe assumption that they really meant to
-			 * use the full plane sizes.
-			 *
-			 * Some drivers, e.g. old codec drivers, use bytesused == 0
-			 * as a way to indicate that streaming is finished.
-			 * In that case, the driver should use the
-			 * allow_zero_bytesused flag to keep old userspace
-			 * applications working.
-			 */
-			for (plane = 0; plane < vb->num_planes; ++plane) {
-				struct vb2_plane *pdst = &planes[plane];
-				struct v4l2_plane *psrc = &b->m.planes[plane];
-
-				if (psrc->bytesused == 0)
-					vb2_warn_zero_bytesused(vb);
 
-				if (vb->vb2_queue->allow_zero_bytesused)
-					pdst->bytesused = psrc->bytesused;
-				else
-					pdst->bytesused = psrc->bytesused ?
-						psrc->bytesused : pdst->length;
-				pdst->data_offset = psrc->data_offset;
-			}
-		}
-	} else {
-		/*
-		 * Single-planar buffers do not use planes array,
-		 * so fill in relevant v4l2_buffer struct fields instead.
-		 * In videobuf we use our internal V4l2_planes struct for
-		 * single-planar buffers as well, for simplicity.
-		 *
-		 * If bytesused == 0 for the output buffer, then fall back
-		 * to the full buffer size as that's a sensible default.
-		 *
-		 * Some drivers, e.g. old codec drivers, use bytesused == 0 as
-		 * a way to indicate that streaming is finished. In that case,
-		 * the driver should use the allow_zero_bytesused flag to keep
-		 * old userspace applications working.
-		 */
-		if (b->memory == VB2_MEMORY_USERPTR) {
-			planes[0].m.userptr = b->m.userptr;
-			planes[0].length = b->length;
-		}
-
-		if (b->memory == VB2_MEMORY_DMABUF) {
-			planes[0].m.fd = b->m.fd;
-			planes[0].length = b->length;
-		}
-
-		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
-			if (b->bytesused == 0)
-				vb2_warn_zero_bytesused(vb);
-
-			if (vb->vb2_queue->allow_zero_bytesused)
-				planes[0].bytesused = b->bytesused;
-			else
-				planes[0].bytesused = b->bytesused ?
-					b->bytesused : planes[0].length;
-		} else
-			planes[0].bytesused = 0;
-
-	}
-
-	/* Zero flags that the vb2 core handles */
-	vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
-	if (!vb->vb2_queue->copy_timestamp || !V4L2_TYPE_IS_OUTPUT(b->type)) {
-		/*
-		 * Non-COPY timestamps and non-OUTPUT queues will get
-		 * their timestamp and timestamp source flags from the
-		 * queue.
-		 */
-		vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
+	for (plane = 0; plane < vb->num_planes; ++plane) {
+		planes[plane].m = vbuf->planes[plane].m;
+		planes[plane].length = vbuf->planes[plane].length;
+		planes[plane].bytesused = vbuf->planes[plane].bytesused;
+		planes[plane].data_offset = vbuf->planes[plane].data_offset;
 	}
-
-	if (V4L2_TYPE_IS_OUTPUT(b->type)) {
-		/*
-		 * For output buffers mask out the timecode flag:
-		 * this will be handled later in vb2_qbuf().
-		 * The 'field' is valid metadata for this output buffer
-		 * and so that needs to be copied here.
-		 */
-		vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
-		vbuf->field = b->field;
-	} else {
-		/* Zero any output buffer flags as this is a capture buffer */
-		vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
-		/* Zero last flag, this is a signal from driver to userspace */
-		vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
-	}
-
 	return 0;
 }
 
diff --git a/drivers/media/dvb-core/dvb_vb2.c b/drivers/media/dvb-core/dvb_vb2.c
index b811adf88afa..da6a8cec7d42 100644
--- a/drivers/media/dvb-core/dvb_vb2.c
+++ b/drivers/media/dvb-core/dvb_vb2.c
@@ -146,8 +146,7 @@ static void _fill_dmx_buffer(struct vb2_buffer *vb, void *pb)
 	dprintk(3, "[%s]\n", ctx->name);
 }
 
-static int _fill_vb2_buffer(struct vb2_buffer *vb,
-			    const void *pb, struct vb2_plane *planes)
+static int _fill_vb2_buffer(struct vb2_buffer *vb, struct vb2_plane *planes)
 {
 	struct dvb_vb2_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
 
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f6818f732f34..224c4820a044 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -417,8 +417,7 @@ struct vb2_ops {
 struct vb2_buf_ops {
 	int (*verify_planes_array)(struct vb2_buffer *vb, const void *pb);
 	void (*fill_user_buffer)(struct vb2_buffer *vb, void *pb);
-	int (*fill_vb2_buffer)(struct vb2_buffer *vb, const void *pb,
-				struct vb2_plane *planes);
+	int (*fill_vb2_buffer)(struct vb2_buffer *vb, struct vb2_plane *planes);
 	void (*copy_timestamp)(struct vb2_buffer *vb, const void *pb);
 };
 
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 3d5e2d739f05..097bf3e6951d 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -32,6 +32,7 @@
  *		&enum v4l2_field.
  * @timecode:	frame timecode.
  * @sequence:	sequence count of this frame.
+ * @planes:	plane information (userptr/fd, length, bytesused, data_offset).
  *
  * Should contain enough information to be able to cover all the fields
  * of &struct v4l2_buffer at ``videodev2.h``.
@@ -43,6 +44,7 @@ struct vb2_v4l2_buffer {
 	__u32			field;
 	struct v4l2_timecode	timecode;
 	__u32			sequence;
+	struct vb2_plane	planes[VB2_MAX_PLANES];
 };
 
 /*
-- 
2.16.3

[RFCv11 PATCH 18/29] videobuf2-core: embed media_request_object

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Make vb2_buffer a request object.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 include/media/videobuf2-core.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 224c4820a044..3d54654c3cd4 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -17,6 +17,7 @@
 #include <linux/poll.h>
 #include <linux/dma-buf.h>
 #include <linux/bitops.h>
+#include <media/media-request.h>
 
 #define VB2_MAX_FRAME	(32)
 #define VB2_MAX_PLANES	(8)
@@ -238,6 +239,7 @@ struct vb2_queue;
  * @num_planes:		number of planes in the buffer
  *			on an internal driver queue.
  * @timestamp:		frame timestamp in ns.
+ * @req_obj:		used to bind this buffer to a request
  */
 struct vb2_buffer {
 	struct vb2_queue	*vb2_queue;
@@ -246,6 +248,7 @@ struct vb2_buffer {
 	unsigned int		memory;
 	unsigned int		num_planes;
 	u64			timestamp;
+	struct media_request_object	req_obj;
 
 	/* private: internal use only
 	 *
-- 
2.16.3

[RFCv11 PATCH 19/29] videobuf2-core: integrate with media requests

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Buffers can now be prepared or queued for a request.

A buffer is unbound from the request at vb2_buffer_done time or
when the queue is cancelled.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 115 +++++++++++++++++++++---
 drivers/media/common/videobuf2/videobuf2-v4l2.c |   4 +-
 drivers/media/dvb-core/dvb_vb2.c                |   2 +-
 include/media/videobuf2-core.h                  |  17 +++-
 4 files changed, 122 insertions(+), 16 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 3d436ccb61f8..4f6c0b2d7d1a 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -930,6 +930,14 @@ void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
 		vb->state = state;
 	}
 	atomic_dec(&q->owned_by_drv_count);
+
+	if (vb->req_obj.req) {
+		/* This is not supported at the moment */
+		WARN_ON(state == VB2_BUF_STATE_REQUEUEING);
+		media_request_object_unbind(&vb->req_obj);
+		media_request_object_put(&vb->req_obj);
+	}
+
 	spin_unlock_irqrestore(&q->done_lock, flags);
 
 	trace_vb2_buf_done(q, vb);
@@ -1276,11 +1284,66 @@ static int __buf_prepare(struct vb2_buffer *vb, const void *pb)
 	return 0;
 }
 
-int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
+static int vb2_req_prepare(struct media_request_object *obj)
 {
-	struct vb2_buffer *vb;
+	struct vb2_buffer *vb = container_of(obj, struct vb2_buffer, req_obj);
 	int ret;
 
+	if (WARN_ON(vb->state != VB2_BUF_STATE_IN_REQUEST))
+		return -EINVAL;
+
+	ret = __buf_prepare(vb, NULL);
+	if (ret)
+		vb->state = VB2_BUF_STATE_IN_REQUEST;
+	return ret;
+}
+
+static void __vb2_dqbuf(struct vb2_buffer *vb);
+static void vb2_req_unprepare(struct media_request_object *obj)
+{
+	struct vb2_buffer *vb = container_of(obj, struct vb2_buffer, req_obj);
+
+	__vb2_dqbuf(vb);
+	vb->state = VB2_BUF_STATE_IN_REQUEST;
+	WARN_ON(!vb->req_obj.req);
+}
+
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+		  struct media_request *req);
+
+static void vb2_req_queue(struct media_request_object *obj)
+{
+	struct vb2_buffer *vb = container_of(obj, struct vb2_buffer, req_obj);
+
+	vb2_core_qbuf(vb->vb2_queue, vb->index, NULL, NULL);
+}
+
+static void vb2_req_release(struct media_request_object *obj)
+{
+	struct vb2_buffer *vb = container_of(obj, struct vb2_buffer, req_obj);
+
+	if (vb->state == VB2_BUF_STATE_IN_REQUEST)
+		vb->state = VB2_BUF_STATE_DEQUEUED;
+}
+
+static void vb2_req_cancel(struct media_request_object *obj)
+{
+	/* Nothing to do here, buffers are cancelled in __vb2_queue_cancel() */
+}
+
+static const struct media_request_object_ops vb2_core_req_ops = {
+	.prepare = vb2_req_prepare,
+	.unprepare = vb2_req_unprepare,
+	.queue = vb2_req_queue,
+	.cancel = vb2_req_cancel,
+	.release = vb2_req_release,
+};
+
+int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
+			 struct media_request *req)
+{
+	struct vb2_buffer *vb;
+
 	vb = q->bufs[index];
 	if (vb->state != VB2_BUF_STATE_DEQUEUED) {
 		dprintk(1, "invalid buffer state %d\n",
@@ -1288,16 +1351,24 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb)
 		return -EINVAL;
 	}
 
-	ret = __buf_prepare(vb, pb);
-	if (ret)
-		return ret;
+	if (req) {
+		vb->state = VB2_BUF_STATE_IN_REQUEST;
+		media_request_object_init(&vb->req_obj);
+		media_request_object_bind(req, &vb2_core_req_ops,
+					  q, &vb->req_obj);
+	} else {
+		int ret = __buf_prepare(vb, pb);
+
+		if (ret)
+			return ret;
+	}
 
 	/* Fill buffer information for the userspace */
 	call_void_bufop(q, fill_user_buffer, vb, pb);
 
 	dprintk(2, "prepare of buffer %d succeeded\n", vb->index);
 
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL_GPL(vb2_core_prepare_buf);
 
@@ -1364,13 +1435,27 @@ static int vb2_start_streaming(struct vb2_queue *q)
 	return ret;
 }
 
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+		  struct media_request *req)
 {
 	struct vb2_buffer *vb;
 	int ret;
 
 	vb = q->bufs[index];
 
+	if (vb->state == VB2_BUF_STATE_DEQUEUED && req) {
+		vb->state = VB2_BUF_STATE_IN_REQUEST;
+		media_request_object_init(&vb->req_obj);
+		media_request_object_bind(req, &vb2_core_req_ops,
+					  q, &vb->req_obj);
+		/* Fill buffer information for the userspace */
+		if (pb)
+			call_void_bufop(q, fill_user_buffer, vb, pb);
+
+		dprintk(2, "qbuf of buffer %d succeeded\n", vb->index);
+		return 0;
+	}
+
 	switch (vb->state) {
 	case VB2_BUF_STATE_DEQUEUED:
 		ret = __buf_prepare(vb, pb);
@@ -1578,6 +1663,10 @@ static void __vb2_dqbuf(struct vb2_buffer *vb)
 			call_void_memop(vb, unmap_dmabuf, vb->planes[i].mem_priv);
 			vb->planes[i].dbuf_mapped = 0;
 		}
+	if (vb->req_obj.req) {
+		media_request_object_unbind(&vb->req_obj);
+		media_request_object_put(&vb->req_obj);
+	}
 }
 
 int vb2_core_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
@@ -1691,6 +1780,9 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
 	for (i = 0; i < q->num_buffers; ++i) {
 		struct vb2_buffer *vb = q->bufs[i];
 
+		if (vb->req_obj.req)
+			media_request_cancel(vb->req_obj.req);
+
 		if (vb->state == VB2_BUF_STATE_PREPARED ||
 		    vb->state == VB2_BUF_STATE_QUEUED) {
 			unsigned int plane;
@@ -1700,7 +1792,8 @@ static void __vb2_queue_cancel(struct vb2_queue *q)
 						vb->planes[plane].mem_priv);
 		}
 
-		if (vb->state != VB2_BUF_STATE_DEQUEUED) {
+		if (vb->state != VB2_BUF_STATE_DEQUEUED &&
+		    vb->state != VB2_BUF_STATE_IN_REQUEST) {
 			vb->state = VB2_BUF_STATE_PREPARED;
 			call_void_vb_qop(vb, buf_finish, vb);
 		}
@@ -2259,7 +2352,7 @@ static int __vb2_init_fileio(struct vb2_queue *q, int read)
 		 * Queue all buffers.
 		 */
 		for (i = 0; i < q->num_buffers; i++) {
-			ret = vb2_core_qbuf(q, i, NULL);
+			ret = vb2_core_qbuf(q, i, NULL, NULL);
 			if (ret)
 				goto err_reqbufs;
 			fileio->bufs[i].queued = 1;
@@ -2438,7 +2531,7 @@ static size_t __vb2_perform_fileio(struct vb2_queue *q, char __user *data, size_
 
 		if (copy_timestamp)
 			b->timestamp = ktime_get_ns();
-		ret = vb2_core_qbuf(q, index, NULL);
+		ret = vb2_core_qbuf(q, index, NULL, NULL);
 		dprintk(5, "vb2_dbuf result: %d\n", ret);
 		if (ret)
 			return ret;
@@ -2541,7 +2634,7 @@ static int vb2_thread(void *data)
 		if (copy_timestamp)
 			vb->timestamp = ktime_get_ns();
 		if (!threadio->stop)
-			ret = vb2_core_qbuf(q, vb->index, NULL);
+			ret = vb2_core_qbuf(q, vb->index, NULL, NULL);
 		call_void_qop(q, wait_prepare, q);
 		if (ret || threadio->stop)
 			break;
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index bf7a3ba9fed0..b8d370b97cca 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -544,7 +544,7 @@ int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b)
 
 	ret = vb2_queue_or_prepare_buf(q, b, "prepare_buf");
 
-	return ret ? ret : vb2_core_prepare_buf(q, b->index, b);
+	return ret ? ret : vb2_core_prepare_buf(q, b->index, b, NULL);
 }
 EXPORT_SYMBOL_GPL(vb2_prepare_buf);
 
@@ -612,7 +612,7 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
 	}
 
 	ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
-	return ret ? ret : vb2_core_qbuf(q, b->index, b);
+	return ret ? ret : vb2_core_qbuf(q, b->index, b, NULL);
 }
 EXPORT_SYMBOL_GPL(vb2_qbuf);
 
diff --git a/drivers/media/dvb-core/dvb_vb2.c b/drivers/media/dvb-core/dvb_vb2.c
index da6a8cec7d42..f1e7f0536028 100644
--- a/drivers/media/dvb-core/dvb_vb2.c
+++ b/drivers/media/dvb-core/dvb_vb2.c
@@ -384,7 +384,7 @@ int dvb_vb2_qbuf(struct dvb_vb2_ctx *ctx, struct dmx_buffer *b)
 {
 	int ret;
 
-	ret = vb2_core_qbuf(&ctx->vb_q, b->index, b);
+	ret = vb2_core_qbuf(&ctx->vb_q, b->index, b, NULL);
 	if (ret) {
 		dprintk(1, "[%s] index=%d errno=%d\n", ctx->name,
 			b->index, ret);
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 3d54654c3cd4..72663c2a3ba3 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -204,6 +204,7 @@ enum vb2_io_modes {
 /**
  * enum vb2_buffer_state - current video buffer state.
  * @VB2_BUF_STATE_DEQUEUED:	buffer under userspace control.
+ * @VB2_BUF_STATE_IN_REQUEST:	buffer is queued in media request.
  * @VB2_BUF_STATE_PREPARING:	buffer is being prepared in videobuf.
  * @VB2_BUF_STATE_PREPARED:	buffer prepared in videobuf and by the driver.
  * @VB2_BUF_STATE_QUEUED:	buffer queued in videobuf, but not in driver.
@@ -218,6 +219,7 @@ enum vb2_io_modes {
  */
 enum vb2_buffer_state {
 	VB2_BUF_STATE_DEQUEUED,
+	VB2_BUF_STATE_IN_REQUEST,
 	VB2_BUF_STATE_PREPARING,
 	VB2_BUF_STATE_PREPARED,
 	VB2_BUF_STATE_QUEUED,
@@ -732,6 +734,7 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
  * @index:	id number of the buffer.
  * @pb:		buffer structure passed from userspace to
  *		&v4l2_ioctl_ops->vidioc_prepare_buf handler in driver.
+ * @req:	pointer to &struct media_request, may be NULL.
  *
  * Videobuf2 core helper to implement VIDIOC_PREPARE_BUF() operation. It is
  * called internally by VB2 by an API-specific handler, like
@@ -743,9 +746,13 @@ int vb2_core_create_bufs(struct vb2_queue *q, enum vb2_memory memory,
  * (if provided), in which driver-specific buffer initialization can
  * be performed.
  *
+ * If @req is non-NULL, then the prepared buffer will be bound to this
+ * media request.
+ *
  * Return: returns zero on success; an error code otherwise.
  */
-int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
+int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
+			 struct media_request *req);
 
 /**
  * vb2_core_qbuf() - Queue a buffer from userspace
@@ -754,12 +761,17 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
  * @index:	id number of the buffer
  * @pb:		buffer structure passed from userspace to
  *		v4l2_ioctl_ops->vidioc_qbuf handler in driver
+ * @req:	pointer to &struct media_request, may be NULL.
  *
  * Videobuf2 core helper to implement VIDIOC_QBUF() operation. It is called
  * internally by VB2 by an API-specific handler, like ``videobuf2-v4l2.h``.
  *
  * This function:
  *
+ * #) If @req is non-NULL, then the buffer will be bound to this
+ *    media request and it returns. The buffer will be prepared and
+ *    queued to the driver (i.e. the next two steps) when the request
+ *    itself is queued.
  * #) if necessary, calls &vb2_ops->buf_prepare callback in the driver
  *    (if provided), in which driver-specific buffer initialization can
  *    be performed;
@@ -768,7 +780,8 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
  *
  * Return: returns zero on success; an error code otherwise.
  */
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb);
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+		  struct media_request *req);
 
 /**
  * vb2_core_dqbuf() - Dequeue a buffer to the userspace
-- 
2.16.3

[RFCv11 PATCH 20/29] videobuf2-v4l2: integrate with media requests

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

This implements the V4L2 part of the request support. The main
change is that vb2_qbuf and vb2_prepare_buf now have a new
media_device pointer. This required changes to several drivers
that did not use the vb2_ioctl_qbuf/prepare_buf helper functions.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/common/videobuf2/videobuf2-v4l2.c  | 84 ++++++++++++++++++++----
 drivers/media/platform/omap3isp/ispvideo.c       |  2 +-
 drivers/media/platform/s3c-camif/camif-capture.c |  4 +-
 drivers/media/platform/s5p-mfc/s5p_mfc_dec.c     |  4 +-
 drivers/media/platform/s5p-mfc/s5p_mfc_enc.c     |  4 +-
 drivers/media/platform/soc_camera/soc_camera.c   |  4 +-
 drivers/media/usb/uvc/uvc_queue.c                |  5 +-
 drivers/media/usb/uvc/uvc_v4l2.c                 |  3 +-
 drivers/media/usb/uvc/uvcvideo.h                 |  1 +
 drivers/media/v4l2-core/v4l2-mem2mem.c           |  7 +-
 drivers/staging/media/davinci_vpfe/vpfe_video.c  |  3 +-
 drivers/staging/media/omap4iss/iss_video.c       |  3 +-
 drivers/usb/gadget/function/uvc_queue.c          |  2 +-
 include/media/videobuf2-v4l2.h                   | 12 +++-
 14 files changed, 106 insertions(+), 32 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index b8d370b97cca..73c1fd4da58a 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -25,6 +25,7 @@
 #include <linux/kthread.h>
 
 #include <media/v4l2-dev.h>
+#include <media/v4l2-device.h>
 #include <media/v4l2-fh.h>
 #include <media/v4l2-event.h>
 #include <media/v4l2-common.h>
@@ -40,10 +41,12 @@ module_param(debug, int, 0644);
 			pr_info("vb2-v4l2: %s: " fmt, __func__, ## arg); \
 	} while (0)
 
-/* Flags that are set by the vb2 core */
+/* Flags that are set by us */
 #define V4L2_BUFFER_MASK_FLAGS	(V4L2_BUF_FLAG_MAPPED | V4L2_BUF_FLAG_QUEUED | \
 				 V4L2_BUF_FLAG_DONE | V4L2_BUF_FLAG_ERROR | \
 				 V4L2_BUF_FLAG_PREPARED | \
+				 V4L2_BUF_FLAG_IN_REQUEST | \
+				 V4L2_BUF_FLAG_REQUEST_FD | \
 				 V4L2_BUF_FLAG_TIMESTAMP_MASK)
 /* Output buffer flags that should be passed on to the driver */
 #define V4L2_BUFFER_OUT_FLAGS	(V4L2_BUF_FLAG_PFRAME | V4L2_BUF_FLAG_BFRAME | \
@@ -318,13 +321,17 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
 	return 0;
 }
 
-static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
-				    const char *opname)
+static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
+				    struct v4l2_buffer *b,
+				    const char *opname,
+				    struct media_request **p_req)
 {
+	struct media_request *req;
 	struct vb2_v4l2_buffer *vbuf;
 	struct vb2_buffer *vb;
 	int ret;
 
+	*p_req = NULL;
 	if (b->type != q->type) {
 		dprintk(1, "%s: invalid buffer type\n", opname);
 		return -EINVAL;
@@ -354,7 +361,38 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
 
 	/* Copy relevant information provided by the userspace */
 	memset(vbuf->planes, 0, sizeof(vbuf->planes[0]) * vb->num_planes);
-	return vb2_fill_vb2_v4l2_buffer(vb, b);
+	ret = vb2_fill_vb2_v4l2_buffer(vb, b);
+	if (ret)
+		return ret;
+
+	if (!(b->flags & V4L2_BUF_FLAG_REQUEST_FD))
+		return 0;
+
+	if (vb->state != VB2_BUF_STATE_DEQUEUED) {
+		dprintk(1, "%s: buffer is not in dequeued state\n", opname);
+		return -EINVAL;
+	}
+
+	if (b->request_fd < 0) {
+		dprintk(1, "%s: request_fd < 0\n", opname);
+		return -EINVAL;
+	}
+
+	req = media_request_find(mdev, b->request_fd);
+	if (IS_ERR(req)) {
+		dprintk(1, "%s: invalid request_fd\n", opname);
+		return PTR_ERR(req);
+	}
+
+	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
+		dprintk(1, "%s: request is not idle\n", opname);
+		media_request_put(req);
+		return -EBUSY;
+	}
+
+	*p_req = req;
+
+	return 0;
 }
 
 /*
@@ -437,6 +475,9 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
 	case VB2_BUF_STATE_ACTIVE:
 		b->flags |= V4L2_BUF_FLAG_QUEUED;
 		break;
+	case VB2_BUF_STATE_IN_REQUEST:
+		b->flags |= V4L2_BUF_FLAG_IN_REQUEST;
+		break;
 	case VB2_BUF_STATE_ERROR:
 		b->flags |= V4L2_BUF_FLAG_ERROR;
 		/* fall through */
@@ -455,6 +496,10 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
 
 	if (vb2_buffer_in_use(q, vb))
 		b->flags |= V4L2_BUF_FLAG_MAPPED;
+	if (vb->req_obj.req) {
+		b->flags |= V4L2_BUF_FLAG_REQUEST_FD;
+		b->request_fd = -1;
+	}
 
 	if (!q->is_output &&
 		b->flags & V4L2_BUF_FLAG_DONE &&
@@ -533,8 +578,10 @@ int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
 }
 EXPORT_SYMBOL_GPL(vb2_reqbufs);
 
-int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b)
+int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
+		    struct v4l2_buffer *b)
 {
+	struct media_request *req;
 	int ret;
 
 	if (vb2_fileio_is_active(q)) {
@@ -542,9 +589,13 @@ int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b)
 		return -EBUSY;
 	}
 
-	ret = vb2_queue_or_prepare_buf(q, b, "prepare_buf");
-
-	return ret ? ret : vb2_core_prepare_buf(q, b->index, b, NULL);
+	ret = vb2_queue_or_prepare_buf(q, mdev, b, "prepare_buf", &req);
+	if (ret)
+		return ret;
+	ret = vb2_core_prepare_buf(q, b->index, b, req);
+	if (req)
+		media_request_put(req);
+	return ret;
 }
 EXPORT_SYMBOL_GPL(vb2_prepare_buf);
 
@@ -602,8 +653,10 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
 }
 EXPORT_SYMBOL_GPL(vb2_create_bufs);
 
-int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
+int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
+	     struct v4l2_buffer *b)
 {
+	struct media_request *req;
 	int ret;
 
 	if (vb2_fileio_is_active(q)) {
@@ -611,8 +664,13 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
 		return -EBUSY;
 	}
 
-	ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
-	return ret ? ret : vb2_core_qbuf(q, b->index, b, NULL);
+	ret = vb2_queue_or_prepare_buf(q, mdev, b, "qbuf", &req);
+	if (ret)
+		return ret;
+	ret = vb2_core_qbuf(q, b->index, b, req);
+	if (req)
+		media_request_put(req);
+	return ret;
 }
 EXPORT_SYMBOL_GPL(vb2_qbuf);
 
@@ -802,7 +860,7 @@ int vb2_ioctl_prepare_buf(struct file *file, void *priv,
 
 	if (vb2_queue_is_busy(vdev, file))
 		return -EBUSY;
-	return vb2_prepare_buf(vdev->queue, p);
+	return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p);
 }
 EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf);
 
@@ -821,7 +879,7 @@ int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p)
 
 	if (vb2_queue_is_busy(vdev, file))
 		return -EBUSY;
-	return vb2_qbuf(vdev->queue, p);
+	return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p);
 }
 EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf);
 
diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
index bd564c2e767f..9fecbd8c6edd 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -940,7 +940,7 @@ isp_video_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
 	int ret;
 
 	mutex_lock(&video->queue_lock);
-	ret = vb2_qbuf(&vfh->queue, b);
+	ret = vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
 	mutex_unlock(&video->queue_lock);
 
 	return ret;
diff --git a/drivers/media/platform/s3c-camif/camif-capture.c b/drivers/media/platform/s3c-camif/camif-capture.c
index 9ab8e7ee2e1e..fafb6da3e804 100644
--- a/drivers/media/platform/s3c-camif/camif-capture.c
+++ b/drivers/media/platform/s3c-camif/camif-capture.c
@@ -941,7 +941,7 @@ static int s3c_camif_qbuf(struct file *file, void *priv,
 	if (vp->owner && vp->owner != priv)
 		return -EBUSY;
 
-	return vb2_qbuf(&vp->vb_queue, buf);
+	return vb2_qbuf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, buf);
 }
 
 static int s3c_camif_dqbuf(struct file *file, void *priv,
@@ -979,7 +979,7 @@ static int s3c_camif_prepare_buf(struct file *file, void *priv,
 				 struct v4l2_buffer *b)
 {
 	struct camif_vp *vp = video_drvdata(file);
-	return vb2_prepare_buf(&vp->vb_queue, b);
+	return vb2_prepare_buf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, b);
 }
 
 static int s3c_camif_g_selection(struct file *file, void *priv,
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
index 5cf4d9921264..3d863e3f5798 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
@@ -632,9 +632,9 @@ static int vidioc_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 		return -EIO;
 	}
 	if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
-		return vb2_qbuf(&ctx->vq_src, buf);
+		return vb2_qbuf(&ctx->vq_src, NULL, buf);
 	else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
-		return vb2_qbuf(&ctx->vq_dst, buf);
+		return vb2_qbuf(&ctx->vq_dst, NULL, buf);
 	return -EINVAL;
 }
 
diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
index 5c0462ca9993..ed12d5d062e8 100644
--- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
+++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
@@ -1621,9 +1621,9 @@ static int vidioc_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
 			mfc_err("Call on QBUF after EOS command\n");
 			return -EIO;
 		}
-		return vb2_qbuf(&ctx->vq_src, buf);
+		return vb2_qbuf(&ctx->vq_src, NULL, buf);
 	} else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
-		return vb2_qbuf(&ctx->vq_dst, buf);
+		return vb2_qbuf(&ctx->vq_dst, NULL, buf);
 	}
 	return -EINVAL;
 }
diff --git a/drivers/media/platform/soc_camera/soc_camera.c b/drivers/media/platform/soc_camera/soc_camera.c
index e6787abc34ae..08adf9a79420 100644
--- a/drivers/media/platform/soc_camera/soc_camera.c
+++ b/drivers/media/platform/soc_camera/soc_camera.c
@@ -394,7 +394,7 @@ static int soc_camera_qbuf(struct file *file, void *priv,
 	if (icd->streamer != file)
 		return -EBUSY;
 
-	return vb2_qbuf(&icd->vb2_vidq, p);
+	return vb2_qbuf(&icd->vb2_vidq, NULL, p);
 }
 
 static int soc_camera_dqbuf(struct file *file, void *priv,
@@ -430,7 +430,7 @@ static int soc_camera_prepare_buf(struct file *file, void *priv,
 {
 	struct soc_camera_device *icd = file->private_data;
 
-	return vb2_prepare_buf(&icd->vb2_vidq, b);
+	return vb2_prepare_buf(&icd->vb2_vidq, NULL, b);
 }
 
 static int soc_camera_expbuf(struct file *file, void *priv,
diff --git a/drivers/media/usb/uvc/uvc_queue.c b/drivers/media/usb/uvc/uvc_queue.c
index fecccb5e7628..8964e16f2b22 100644
--- a/drivers/media/usb/uvc/uvc_queue.c
+++ b/drivers/media/usb/uvc/uvc_queue.c
@@ -300,12 +300,13 @@ int uvc_create_buffers(struct uvc_video_queue *queue,
 	return ret;
 }
 
-int uvc_queue_buffer(struct uvc_video_queue *queue, struct v4l2_buffer *buf)
+int uvc_queue_buffer(struct uvc_video_queue *queue,
+		     struct media_device *mdev, struct v4l2_buffer *buf)
 {
 	int ret;
 
 	mutex_lock(&queue->mutex);
-	ret = vb2_qbuf(&queue->queue, buf);
+	ret = vb2_qbuf(&queue->queue, mdev, buf);
 	mutex_unlock(&queue->mutex);
 
 	return ret;
diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
index bd32914259ae..3da5fdc002ac 100644
--- a/drivers/media/usb/uvc/uvc_v4l2.c
+++ b/drivers/media/usb/uvc/uvc_v4l2.c
@@ -751,7 +751,8 @@ static int uvc_ioctl_qbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
 	if (!uvc_has_privileges(handle))
 		return -EBUSY;
 
-	return uvc_queue_buffer(&stream->queue, buf);
+	return uvc_queue_buffer(&stream->queue,
+				stream->vdev.v4l2_dev->mdev, buf);
 }
 
 static int uvc_ioctl_expbuf(struct file *file, void *fh,
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index be5cf179228b..bc9ed18f043c 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -680,6 +680,7 @@ int uvc_query_buffer(struct uvc_video_queue *queue,
 int uvc_create_buffers(struct uvc_video_queue *queue,
 		       struct v4l2_create_buffers *v4l2_cb);
 int uvc_queue_buffer(struct uvc_video_queue *queue,
+		     struct media_device *mdev,
 		     struct v4l2_buffer *v4l2_buf);
 int uvc_export_buffer(struct uvc_video_queue *queue,
 		      struct v4l2_exportbuffer *exp);
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index c4f963d96a79..438f1b869319 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -20,6 +20,7 @@
 #include <media/videobuf2-v4l2.h>
 #include <media/v4l2-mem2mem.h>
 #include <media/v4l2-dev.h>
+#include <media/v4l2-device.h>
 #include <media/v4l2-fh.h>
 #include <media/v4l2-event.h>
 
@@ -388,11 +389,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_querybuf);
 int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
 		  struct v4l2_buffer *buf)
 {
+	struct video_device *vdev = video_devdata(file);
 	struct vb2_queue *vq;
 	int ret;
 
 	vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
-	ret = vb2_qbuf(vq, buf);
+	ret = vb2_qbuf(vq, vdev->v4l2_dev->mdev, buf);
 	if (!ret)
 		v4l2_m2m_try_schedule(m2m_ctx);
 
@@ -413,11 +415,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_dqbuf);
 int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
 			 struct v4l2_buffer *buf)
 {
+	struct video_device *vdev = video_devdata(file);
 	struct vb2_queue *vq;
 	int ret;
 
 	vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
-	ret = vb2_prepare_buf(vq, buf);
+	ret = vb2_prepare_buf(vq, vdev->v4l2_dev->mdev, buf);
 	if (!ret)
 		v4l2_m2m_try_schedule(m2m_ctx);
 
diff --git a/drivers/staging/media/davinci_vpfe/vpfe_video.c b/drivers/staging/media/davinci_vpfe/vpfe_video.c
index 588743a6fd8a..00bf28e830d4 100644
--- a/drivers/staging/media/davinci_vpfe/vpfe_video.c
+++ b/drivers/staging/media/davinci_vpfe/vpfe_video.c
@@ -1426,7 +1426,8 @@ static int vpfe_qbuf(struct file *file, void *priv,
 		return -EACCES;
 	}
 
-	return vb2_qbuf(&video->buffer_queue, p);
+	return vb2_qbuf(&video->buffer_queue,
+			video->video_dev.v4l2_dev->mdev, p);
 }
 
 /*
diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
index a3a83424a926..a35d1004b522 100644
--- a/drivers/staging/media/omap4iss/iss_video.c
+++ b/drivers/staging/media/omap4iss/iss_video.c
@@ -805,9 +805,10 @@ iss_video_querybuf(struct file *file, void *fh, struct v4l2_buffer *b)
 static int
 iss_video_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
 {
+	struct iss_video *video = video_drvdata(file);
 	struct iss_video_fh *vfh = to_iss_video_fh(fh);
 
-	return vb2_qbuf(&vfh->queue, b);
+	return vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
 }
 
 static int
diff --git a/drivers/usb/gadget/function/uvc_queue.c b/drivers/usb/gadget/function/uvc_queue.c
index 9e33d5206d54..f2497cb96abb 100644
--- a/drivers/usb/gadget/function/uvc_queue.c
+++ b/drivers/usb/gadget/function/uvc_queue.c
@@ -166,7 +166,7 @@ int uvcg_queue_buffer(struct uvc_video_queue *queue, struct v4l2_buffer *buf)
 	unsigned long flags;
 	int ret;
 
-	ret = vb2_qbuf(&queue->queue, buf);
+	ret = vb2_qbuf(&queue->queue, NULL, buf);
 	if (ret < 0)
 		return ret;
 
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 097bf3e6951d..cf312ab4e7e8 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -79,6 +79,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
  * vb2_prepare_buf() - Pass ownership of a buffer from userspace to the kernel
  *
  * @q:		pointer to &struct vb2_queue with videobuf2 queue.
+ * @mdev:	pointer to &struct media_device, may be NULL.
  * @b:		buffer structure passed from userspace to
  *		&v4l2_ioctl_ops->vidioc_prepare_buf handler in driver
  *
@@ -90,15 +91,19 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
  * #) verifies the passed buffer,
  * #) calls &vb2_ops->buf_prepare callback in the driver (if provided),
  *    in which driver-specific buffer initialization can be performed.
+ * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
+ *    then bind the prepared buffer to the request.
  *
  * The return values from this function are intended to be directly returned
  * from &v4l2_ioctl_ops->vidioc_prepare_buf handler in driver.
  */
-int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
+int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
+		    struct v4l2_buffer *b);
 
 /**
  * vb2_qbuf() - Queue a buffer from userspace
  * @q:		pointer to &struct vb2_queue with videobuf2 queue.
+ * @mdev:	pointer to &struct media_device, may be NULL.
  * @b:		buffer structure passed from userspace to
  *		&v4l2_ioctl_ops->vidioc_qbuf handler in driver
  *
@@ -107,6 +112,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
  * This function:
  *
  * #) verifies the passed buffer;
+ * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
+ *    then bind the buffer to the request.
  * #) if necessary, calls &vb2_ops->buf_prepare callback in the driver
  *    (if provided), in which driver-specific buffer initialization can
  *    be performed;
@@ -116,7 +123,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
  * The return values from this function are intended to be directly returned
  * from &v4l2_ioctl_ops->vidioc_qbuf handler in driver.
  */
-int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b);
+int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
+	     struct v4l2_buffer *b);
 
 /**
  * vb2_expbuf() - Export a buffer as a file descriptor
-- 
2.16.3

[RFCv11 PATCH 21/29] videobuf2-core: add vb2_core_request_has_buffers

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add a new helper function that returns true if a media_request
contains buffers.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 12 ++++++++++++
 include/media/videobuf2-core.h                  |  2 ++
 2 files changed, 14 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 4f6c0b2d7d1a..13c9d9e243dd 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1339,6 +1339,18 @@ static const struct media_request_object_ops vb2_core_req_ops = {
 	.release = vb2_req_release,
 };
 
+bool vb2_core_request_has_buffers(struct media_request *req)
+{
+	struct media_request_object *obj;
+
+	list_for_each_entry(obj, &req->objects, list) {
+		if (obj->ops == &vb2_core_req_ops)
+			return true;
+	}
+	return false;
+}
+EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
+
 int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
 			 struct media_request *req)
 {
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 72663c2a3ba3..e23dc028aee7 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -1158,4 +1158,6 @@ bool vb2_buffer_in_use(struct vb2_queue *q, struct vb2_buffer *vb);
  */
 int vb2_verify_memory_type(struct vb2_queue *q,
 		enum vb2_memory memory, unsigned int type);
+
+bool vb2_core_request_has_buffers(struct media_request *req);
 #endif /* _MEDIA_VIDEOBUF2_CORE_H */
-- 
2.16.3

[RFCv11 PATCH 22/29] videobuf2-v4l2: add vb2_request_queue helper

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Generic helper function that checks if there are buffers in
the request and if so, prepares and queues all objects in the
request.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 39 +++++++++++++++++++++++++
 include/media/videobuf2-v4l2.h                  |  3 ++
 2 files changed, 42 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 73c1fd4da58a..3d0c74bb4220 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -1061,6 +1061,45 @@ void vb2_ops_wait_finish(struct vb2_queue *vq)
 }
 EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
 
+int vb2_request_queue(struct media_request *req)
+{
+	struct media_request_object *obj;
+	struct media_request_object *failed_obj = NULL;
+	int ret = 0;
+
+	if (!vb2_core_request_has_buffers(req))
+		return -ENOENT;
+
+	list_for_each_entry(obj, &req->objects, list) {
+		if (!obj->ops->prepare)
+			continue;
+
+		ret = obj->ops->prepare(obj);
+
+		if (ret) {
+			failed_obj = obj;
+			break;
+		}
+	}
+
+	if (ret) {
+		list_for_each_entry(obj, &req->objects, list) {
+			if (obj == failed_obj)
+				break;
+			if (obj->ops->unprepare)
+				obj->ops->unprepare(obj);
+		}
+		return ret;
+	}
+
+	list_for_each_entry(obj, &req->objects, list) {
+		if (obj->ops->queue)
+			obj->ops->queue(obj);
+	}
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vb2_request_queue);
+
 MODULE_DESCRIPTION("Driver helper framework for Video for Linux 2");
 MODULE_AUTHOR("Pawel Osciak <pawel@osciak.com>, Marek Szyprowski");
 MODULE_LICENSE("GPL");
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index cf312ab4e7e8..0baa3023d7ad 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -301,4 +301,7 @@ void vb2_ops_wait_prepare(struct vb2_queue *vq);
  */
 void vb2_ops_wait_finish(struct vb2_queue *vq);
 
+struct media_request;
+int vb2_request_queue(struct media_request *req);
+
 #endif /* _MEDIA_VIDEOBUF2_V4L2_H */
-- 
2.16.3

[RFCv11 PATCH 23/29] videobuf2-v4l2: export request_fd

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Requested by Sakari

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 6 ++++--
 include/media/videobuf2-v4l2.h                  | 2 ++
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 3d0c74bb4220..7b79149b7fae 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -184,6 +184,7 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
 		return -EINVAL;
 	}
 	vbuf->sequence = 0;
+	vbuf->request_fd = -1;
 
 	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
 		switch (b->memory) {
@@ -391,6 +392,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
 	}
 
 	*p_req = req;
+	vbuf->request_fd = b->request_fd;
 
 	return 0;
 }
@@ -496,9 +498,9 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
 
 	if (vb2_buffer_in_use(q, vb))
 		b->flags |= V4L2_BUF_FLAG_MAPPED;
-	if (vb->req_obj.req) {
+	if (vbuf->request_fd >= 0) {
 		b->flags |= V4L2_BUF_FLAG_REQUEST_FD;
-		b->request_fd = -1;
+		b->request_fd = vbuf->request_fd;
 	}
 
 	if (!q->is_output &&
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 0baa3023d7ad..d3ee1f28e197 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -32,6 +32,7 @@
  *		&enum v4l2_field.
  * @timecode:	frame timecode.
  * @sequence:	sequence count of this frame.
+ * @request_fd:	the request_fd associated with this buffer
  * @planes:	plane information (userptr/fd, length, bytesused, data_offset).
  *
  * Should contain enough information to be able to cover all the fields
@@ -44,6 +45,7 @@ struct vb2_v4l2_buffer {
 	__u32			field;
 	struct v4l2_timecode	timecode;
 	__u32			sequence;
+	__s32			request_fd;
 	struct vb2_plane	planes[VB2_MAX_PLANES];
 };
 
-- 
2.16.3

[RFCv11 PATCH 24/29] Documentation: v4l: document request API

[Hans Verkuil]

From: Alexandre Courbot <acourbot@chromium.org>

Document the request API for V4L2 devices, and amend the documentation
of system calls influenced by it.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 Documentation/media/uapi/v4l/buffer.rst            |  19 +-
 Documentation/media/uapi/v4l/common.rst            |   1 +
 Documentation/media/uapi/v4l/request-api.rst       | 199 +++++++++++++++++++++
 Documentation/media/uapi/v4l/user-func.rst         |   1 +
 .../media/uapi/v4l/vidioc-g-ext-ctrls.rst          |  22 ++-
 .../media/uapi/v4l/vidioc-new-request.rst          |  64 +++++++
 Documentation/media/uapi/v4l/vidioc-qbuf.rst       |   8 +
 7 files changed, 308 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/media/uapi/v4l/request-api.rst
 create mode 100644 Documentation/media/uapi/v4l/vidioc-new-request.rst

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index e2c85ddc990b..e23eae12905c 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -306,10 +306,13 @@ struct v4l2_buffer
       - A place holder for future extensions. Drivers and applications
 	must set this to 0.
     * - __u32
-      - ``reserved``
+      - ``request_fd``
       -
-      - A place holder for future extensions. Drivers and applications
-	must set this to 0.
+      - The file descriptor of the request to queue the buffer to. If specified
+        and flag ``V4L2_BUF_FLAG_REQUEST_FD`` is set, then the buffer will be
+	queued to that request. This is set by the user when calling
+	:ref:`VIDIOC_QBUF` and :ref:`VIDIOC_PREPARE_BUF` and ignored by other
+	ioctls.
 
 
 
@@ -514,6 +517,11 @@ Buffer Flags
 	streaming may continue as normal and the buffer may be reused
 	normally. Drivers set this flag when the ``VIDIOC_DQBUF`` ioctl is
 	called.
+    * .. _`V4L2-BUF-FLAG-IN-REQUEST`:
+
+      - ``V4L2_BUF_FLAG_IN_REQUEST``
+      - 0x00000080
+      - This buffer is part of a request the hasn't been queued yet.
     * .. _`V4L2-BUF-FLAG-KEYFRAME`:
 
       - ``V4L2_BUF_FLAG_KEYFRAME``
@@ -589,6 +597,11 @@ Buffer Flags
 	the format. Any Any subsequent call to the
 	:ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl will not block anymore,
 	but return an ``EPIPE`` error code.
+    * .. _`V4L2-BUF-FLAG-REQUEST-FD`:
+
+      - ``V4L2_BUF_FLAG_REQUEST_FD``
+      - 0x00800000
+      - The ``request_fd`` field contains a valid file descriptor.
     * .. _`V4L2-BUF-FLAG-TIMESTAMP-MASK`:
 
       - ``V4L2_BUF_FLAG_TIMESTAMP_MASK``
diff --git a/Documentation/media/uapi/v4l/common.rst b/Documentation/media/uapi/v4l/common.rst
index 13f2ed3fc5a6..a4aa0059d45a 100644
--- a/Documentation/media/uapi/v4l/common.rst
+++ b/Documentation/media/uapi/v4l/common.rst
@@ -44,3 +44,4 @@ applicable to all devices.
     crop
     selection-api
     streaming-par
+    request-api
diff --git a/Documentation/media/uapi/v4l/request-api.rst b/Documentation/media/uapi/v4l/request-api.rst
new file mode 100644
index 000000000000..0c1f2896e197
--- /dev/null
+++ b/Documentation/media/uapi/v4l/request-api.rst
@@ -0,0 +1,199 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _media-request-api:
+
+Request API
+===========
+
+The Request API has been designed to allow V4L2 to deal with requirements of
+modern devices (stateless codecs, MIPI cameras, ...) and APIs (Android Codec
+v2). One such requirement is the ability for devices belonging to the same
+pipeline to reconfigure and collaborate closely on a per-frame basis. Another is
+efficient support of stateless codecs, which need per-frame controls to be set
+asynchronously in order to be efficiently used.
+
+Supporting these features without the Request API is possible but terribly
+inefficient: user-space would have to flush all activity on the media pipeline,
+reconfigure it for the next frame, queue the buffers to be processed with that
+configuration, and wait until they are all available for dequeing before
+considering the next frame. This defeats the purpose of having buffer queues
+since in practice only one buffer would be queued at a time.
+
+The Request API allows a specific configuration of the pipeline (media
+controller topology + controls for each device) to be associated with specific
+buffers. The parameters are applied by each participating device as buffers
+associated to a request flow in. This allows user-space to schedule several
+tasks ("requests") with different parameters in advance, knowing that the
+parameters will be applied when needed to get the expected result. Controls
+values at the time of request completion are also available for reading.
+
+Usage
+=====
+
+The Request API is used on top of standard media controller and V4L2 calls,
+which are augmented with an extra ``request_fd`` parameter. Request themselves
+are allocated from either a supporting V4L2 device node, or a supporting media
+controller node. The origin of requests determine their scope: requests
+allocated from a V4L2 device node can only act on that device, whereas requests
+allocated from a media controller node can control the whole pipeline of the
+controller.
+
+Request Allocation
+------------------
+
+User-space allocates requests using the ``VIDIOC_NEW_REQUEST`` (for V4L2 device
+requests) or ``MEDIA_IOC_NEW_REQUEST`` (for media controller requests) on an
+opened device or media node. This returns a file descriptor representing the
+request. Typically, several such requests will be allocated.
+
+Request Preparation
+-------------------
+
+Standard V4L2 ioctls can then receive a request file descriptor to express the
+fact that the ioctl is part of said request, and is not to be applied
+immediately. V4L2 ioctls supporting this are :c:func:`VIDIOC_S_EXT_CTRLS` and
+:c:func:`VIDIOC_QBUF`. Controls set with a request parameter are stored instead
+of being immediately applied, and queued buffers not enter the regular buffer
+queue until the request is submitted. Only one buffer can be queued to a given
+queue for a given request.
+
+Request Submission
+------------------
+
+Once the parameters and buffers of the request are specified, it can be
+submitted by calling the ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl on the request FD.
+This will make the buffers associated to the request available to their driver,
+which can then apply the saved controls as buffers are processed. A submitted
+request cannot be modified anymore.
+
+If several devices are part of the request, individual drivers may synchronize
+so the requested pipeline's topology is applied before the buffers are
+processed. This is at the discretion of media controller drivers and is not a
+requirement.
+
+Buffers queued without an associated request after a request-bound buffer will
+be processed using the state of the hardware at the time of the request
+completion. All the same, controls set without a request are applied
+immediately, regardless of whether a request is in use or not.
+
+User-space can ``poll()`` a request FD in order to wait until the request
+completes. A request is considered complete once all its associated buffers are
+available for dequeing. Note that user-space does not need to wait for the
+request to complete to dequeue its buffers: buffers that are available halfway
+through a request can be dequeued independently of the request's state.
+
+A completed request includes the state of all devices that had queued buffers
+associated with it at the time of the request completion. User-space can query
+that state by calling :c:func:`VIDIOC_G_EXT_CTRLS` with the request FD.
+
+Recycling and Destruction
+-------------------------
+
+Finally, completed request can either be discarded or be reused. Calling
+``close()`` on a request FD will make that FD unusable, freeing the request if
+it is not referenced elsewhere. The ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl will
+clear a request's state and make it available again. No state is retained by
+this operation: the request is as if it had just been allocated.
+
+Example for a M2M Device
+------------------------
+
+M2M devices are single-node V4L2 devices providing one OUTPUT queue (for
+user-space
+to provide input buffers) and one CAPTURE queue (to retrieve processed data).
+They are perfectly symetric, i.e. one buffer of input will produce one buffer of
+output. These devices are commonly used for frame processors or stateless
+codecs.
+
+In this use-case, the request API can be used to associate specific controls to
+be applied by the driver before processing an OUTPUT buffer, allowing user-space
+to queue many such buffers in advance. It can also take advantage of requests'
+ability to capture the state of controls when the request completes to read back
+information that may be subject to change.
+
+Put into code, after obtaining a request, user-space can assign controls and one
+OUTPUT buffer to it:
+
+	struct v4l2_buf buf;
+	struct v4l2_ext_controls ctrls;
+	struct media_request_new new = { 0 };
+	int req_fd;
+	...
+	ioctl(media_fd, VIDIOC_NEW_REQUEST, &new);
+	req_fd = new.fd;
+	...
+	ctrls.request_fd = req_fd;
+	ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls);
+	...
+	buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
+	buf.request_fd = req_fd;
+	ioctl(codec_fd, VIDIOC_QBUF, &buf);
+
+Note that request_fd does not need to be specified for CAPTURE buffers: since
+there is symetry between the OUTPUT and CAPTURE queues, and requests are
+processed in order of submission, we can know which CAPTURE buffer corresponds
+to which request.
+
+Once the request is fully prepared, it can be submitted to the driver:
+
+	ioctl(request_fd, MEDIA_REQUEST_IOC_SUBMIT, NULL);
+
+User-space can then either wait for the request to complete by calling poll() on
+its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will
+want to get CAPTURE buffers as soon as possible and this can be done using a
+regular DQBUF:
+
+	struct v4l2_buf buf;
+
+	memset(&buf, 0, sizeof(buf));
+	buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	ioctl(codec_fd, VIDIOC_DQBUF, &buf);
+
+We can then, after ensuring that the request is completed via polling the
+request FD, query control values at the time of its completion via an
+annotated call to G_EXT_CTRLS. This is particularly useful for volatile controls
+for which we want to query values as soon as the capture buffer is produced.
+
+	struct pollfd pfd = { .events = POLLIN, .fd = request_fd };
+	poll(&pfd, 1, -1);
+	...
+	ctrls.request_fd = req_fd;
+	ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls);
+
+Once we don't need the request anymore, we can either recycle it for reuse with
+MEDIA_REQUEST_IOC_REINIT...
+
+	ioctl(request, MEDIA_REQUEST_IOC_REINIT, NULL);
+
+... or close its file descriptor to completely dispose of it.
+
+	close(request_fd);
+
+Example for a Simple Capture Device
+-----------------------------------
+
+With a simple capture device, requests can be used to specify controls to apply
+to a given CAPTURE buffer. The driver will apply these controls before producing
+the marked CAPTURE buffer.
+
+	struct v4l2_buf buf;
+	struct v4l2_ext_controls ctrls;
+	struct media_request_new new = { 0 };
+	int req_fd;
+	...
+	ioctl(camera_fd, VIDIOC_NEW_REQUEST, &new);
+	req_fd = new.fd;
+	...
+	ctrls.request_fd = req_fd;
+	ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls);
+	...
+	buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	buf.request_fd = req_fd;
+	ioctl(camera_fd, VIDIOC_QBUF, &buf);
+
+Once the request is fully prepared, it can be submitted to the driver:
+
+	ioctl(req_fd, MEDIA_REQUEST_IOC_SUBMIT, &cmd);
+
+User-space can then dequeue buffers, wait for the request completion, query
+controls and recycle the request as in the M2M example above.
diff --git a/Documentation/media/uapi/v4l/user-func.rst b/Documentation/media/uapi/v4l/user-func.rst
index 3e0413b83a33..2c8238a2b188 100644
--- a/Documentation/media/uapi/v4l/user-func.rst
+++ b/Documentation/media/uapi/v4l/user-func.rst
@@ -53,6 +53,7 @@ Function Reference
     vidioc-g-std
     vidioc-g-tuner
     vidioc-log-status
+    vidioc-new-request
     vidioc-overlay
     vidioc-prepare-buf
     vidioc-qbuf
diff --git a/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst b/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
index 2011c2b2ee67..d31ef86c7caa 100644
--- a/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
+++ b/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
@@ -95,6 +95,17 @@ appropriate. In the first case the new value is set in struct
 is inappropriate (e.g. the given menu index is not supported by the menu
 control), then this will also result in an ``EINVAL`` error code error.
 
+If ``request_fd`` is set to a not-submitted request file descriptor, then the
+controls are not applied immediately when calling
+:ref:`VIDIOC_S_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`, but instead are applied right
+before the driver starts processing a buffer associated to the same request.
+
+If ``request_fd`` is specified during a call to
+:ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`, then the returned values will
+be the values currently set for the request (or the hardware value if none is
+set) if the request has not yet completed, or the values of the controls at the
+time of request completion if it has already completed.
+
 The driver will only set/get these controls if all control values are
 correct. This prevents the situation where only some of the controls
 were set/get. Only low-level errors (e. g. a failed i2c command) can
@@ -209,8 +220,10 @@ still cause this situation.
       - ``which``
       - Which value of the control to get/set/try.
 	``V4L2_CTRL_WHICH_CUR_VAL`` will return the current value of the
-	control and ``V4L2_CTRL_WHICH_DEF_VAL`` will return the default
-	value of the control.
+	control, ``V4L2_CTRL_WHICH_DEF_VAL`` will return the default
+	value of the control and ``V4L2_CTRL_WHICH_REQUEST`` indicates that
+	these controls have to be retrieved from a request or tried/set for
+	a request.
 
 	.. note::
 
@@ -272,8 +285,11 @@ still cause this situation.
 	then you can call :ref:`VIDIOC_TRY_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` to try to discover the
 	actual control that failed the validation step. Unfortunately,
 	there is no ``TRY`` equivalent for :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`.
+    * - __s32
+      - ``request_fd``
+	File descriptor of the request to be used by this operation (0 if none).
     * - __u32
-      - ``reserved``\ [2]
+      - ``reserved``\ [1]
       - Reserved for future extensions.
 
 	Drivers and applications must set the array to zero.
diff --git a/Documentation/media/uapi/v4l/vidioc-new-request.rst b/Documentation/media/uapi/v4l/vidioc-new-request.rst
new file mode 100644
index 000000000000..0038287f7d16
--- /dev/null
+++ b/Documentation/media/uapi/v4l/vidioc-new-request.rst
@@ -0,0 +1,64 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _VIDIOC_NEW_REQUEST:
+
+************************
+ioctl VIDIOC_NEW_REQUEST
+************************
+
+Name
+====
+
+VIDIOC_NEW_REQUEST - Allocate a request for given video device.
+
+
+Synopsis
+========
+
+.. c:function:: int ioctl( int fd, VIDIOC_NEW_REQUEST, struct media_request_new *argp )
+    :name: VIDIOC_NEW_REQUEST
+
+Arguments
+=========
+
+``fd``
+    File descriptor returned by :ref:`open() <func-open>`.
+
+``argp``
+    Pointer to struct :c:type:`media_request_new`.
+
+
+Description
+===========
+
+Applications call the ``VIDIOC_NEW_REQUEST`` ioctl to allocate a new request for a given V4L2 video device. The request will only be valid in the scope of the device that allocated it and cannot be used to coordinate multiple devices.
+
+Applications can also check whether requests are supported by a given device by calling this ioctl with the MEDIA_REQUEST_FLAG_TEST bit of :c:type:`media_request_new`'s ``flags`` set. Doing so will not allocate a new request, but will return 0 is request allocation is supported by the device, or -1 and set ``errno`` to ENOTTY if they are not.
+
+.. c:type:: media_request_new
+
+.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
+
+.. flat-table:: struct media_request_new
+    :header-rows:  0
+    :stub-columns: 0
+    :widths:       1 1 2
+
+    * - __u32
+      - ``flags``
+      - Flags for this request creation. If ``MEDIA_REQUEST_FLAG_TEST`` is set, then no request is created and the call only checks for request availability. Written by the application.
+    * - __s32
+      - ``fd``
+      - File descriptor referencing the created request. Written by the kernel.
+
+Return Value
+============
+
+On success 0 is returned, and the ``fd`` field of ``argp`` is set to a file descriptor referencing the request. User-space can use this file descriptor to mention the request in other system calls, perform ``MEDIA_REQUEST_IOC_SUBMIT`` and ``MEDIA_REQUEST_IOC_REINIT`` ioctls on it, and close it to discard the request.
+
+On error -1 is returned and the ``errno`` variable is set appropriately.  The
+generic error codes are described in the :ref:`Generic Error Codes <gen-errors>`
+chapter.
+
+ENOTTY
+    The device does not support the use of requests or request support is not built into the kernel.
diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
index 9e448a4aa3aa..6246f1888583 100644
--- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
+++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
@@ -98,6 +98,14 @@ dequeued, until the :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` or
 :ref:`VIDIOC_REQBUFS` ioctl is called, or until the
 device is closed.
 
+The ``request_fd`` field can be used when queuing to specify the file
+descriptor of a request, if requests are in use. Setting it means that the
+buffer will not be passed to the driver until the request itself is submitted.
+Also, the driver will apply any setting associated with the request before
+processing the buffer. Only one buffer per queue can be assigned that way to
+a request. This field will be ignored unless the ``V4L2_BUF_FLAG_REQUEST_FD``
+flag is set.
+
 Applications call the ``VIDIOC_DQBUF`` ioctl to dequeue a filled
 (capturing) or displayed (output) buffer from the driver's outgoing
 queue. They just set the ``type``, ``memory`` and ``reserved`` fields of
-- 
2.16.3

[RFCv11 PATCH 25/29] media: vim2m: add media device

[Hans Verkuil]

From: Alexandre Courbot <acourbot@chromium.org>

Request API requires a media node. Add one to the vim2m driver so we can
use requests with it.

This probably needs a bit more work to correctly represent m2m
hardware in the media topology.

Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/platform/vim2m.c | 43 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 38 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index 065483e62db4..ef970434af13 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -140,6 +140,10 @@ static struct vim2m_fmt *find_format(struct v4l2_format *f)
 struct vim2m_dev {
 	struct v4l2_device	v4l2_dev;
 	struct video_device	vfd;
+#ifdef CONFIG_MEDIA_CONTROLLER
+	struct media_device	mdev;
+	struct media_pad	pad[2];
+#endif
 
 	atomic_t		num_inst;
 	struct mutex		dev_mutex;
@@ -1000,11 +1004,6 @@ static int vim2m_probe(struct platform_device *pdev)
 		return -ENOMEM;
 
 	spin_lock_init(&dev->irqlock);
-
-	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
-	if (ret)
-		return ret;
-
 	atomic_set(&dev->num_inst, 0);
 	mutex_init(&dev->dev_mutex);
 
@@ -1013,6 +1012,22 @@ static int vim2m_probe(struct platform_device *pdev)
 	vfd->lock = &dev->dev_mutex;
 	vfd->v4l2_dev = &dev->v4l2_dev;
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+	dev->mdev.dev = &pdev->dev;
+	strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
+	media_device_init(&dev->mdev);
+	dev->v4l2_dev.mdev = &dev->mdev;
+	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
+	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
+	ret = media_entity_pads_init(&vfd->entity, 2, dev->pad);
+	if (ret)
+		return ret;
+#endif
+
+	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+	if (ret)
+		goto unreg_media;
+
 	ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
 	if (ret) {
 		v4l2_err(&dev->v4l2_dev, "Failed to register video device\n");
@@ -1034,6 +1049,13 @@ static int vim2m_probe(struct platform_device *pdev)
 		goto err_m2m;
 	}
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+	/* Register the media device node */
+	ret = media_device_register(&dev->mdev);
+	if (ret)
+		goto err_m2m;
+#endif
+
 	return 0;
 
 err_m2m:
@@ -1041,6 +1063,10 @@ static int vim2m_probe(struct platform_device *pdev)
 	video_unregister_device(&dev->vfd);
 unreg_dev:
 	v4l2_device_unregister(&dev->v4l2_dev);
+unreg_media:
+#ifdef CONFIG_MEDIA_CONTROLLER
+	media_device_unregister(&dev->mdev);
+#endif
 
 	return ret;
 }
@@ -1050,6 +1076,13 @@ static int vim2m_remove(struct platform_device *pdev)
 	struct vim2m_dev *dev = platform_get_drvdata(pdev);
 
 	v4l2_info(&dev->v4l2_dev, "Removing " MEM2MEM_NAME);
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+	if (media_devnode_is_registered(dev->mdev.devnode))
+		media_device_unregister(&dev->mdev);
+	media_device_cleanup(&dev->mdev);
+#endif
+
 	v4l2_m2m_release(dev->m2m_dev);
 	del_timer_sync(&dev->timer);
 	video_unregister_device(&dev->vfd);
-- 
2.16.3

[RFCv11 PATCH 26/29] vim2m: use workqueue

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

v4l2_ctrl uses mutexes, so we can't setup a ctrl_handler in
interrupt context. Switch to a workqueue instead.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vim2m.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index ef970434af13..9b18b32c255d 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -150,6 +150,7 @@ struct vim2m_dev {
 	spinlock_t		irqlock;
 
 	struct timer_list	timer;
+	struct work_struct	work_run;
 
 	struct v4l2_m2m_dev	*m2m_dev;
 };
@@ -392,9 +393,10 @@ static void device_run(void *priv)
 	schedule_irq(dev, ctx->transtime);
 }
 
-static void device_isr(struct timer_list *t)
+static void device_work(struct work_struct *w)
 {
-	struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t, timer);
+	struct vim2m_dev *vim2m_dev =
+		container_of(w, struct vim2m_dev, work_run);
 	struct vim2m_ctx *curr_ctx;
 	struct vb2_v4l2_buffer *src_vb, *dst_vb;
 	unsigned long flags;
@@ -426,6 +428,13 @@ static void device_isr(struct timer_list *t)
 	}
 }
 
+static void device_isr(struct timer_list *t)
+{
+	struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t, timer);
+
+	schedule_work(&vim2m_dev->work_run);
+}
+
 /*
  * video ioctls
  */
@@ -806,6 +815,7 @@ static void vim2m_stop_streaming(struct vb2_queue *q)
 	struct vb2_v4l2_buffer *vbuf;
 	unsigned long flags;
 
+	flush_scheduled_work();
 	for (;;) {
 		if (V4L2_TYPE_IS_OUTPUT(q->type))
 			vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
@@ -1011,6 +1021,7 @@ static int vim2m_probe(struct platform_device *pdev)
 	vfd = &dev->vfd;
 	vfd->lock = &dev->dev_mutex;
 	vfd->v4l2_dev = &dev->v4l2_dev;
+	INIT_WORK(&dev->work_run, device_work);
 
 #ifdef CONFIG_MEDIA_CONTROLLER
 	dev->mdev.dev = &pdev->dev;
-- 
2.16.3

[RFCv11 PATCH 27/29] vim2m: support requests

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add support for requests to vim2m.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index 9b18b32c255d..2dcf0ea85705 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -387,8 +387,26 @@ static void device_run(void *priv)
 	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
 	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
 
+	/* Apply request if needed */
+	if (src_buf->vb2_buf.req_obj.req)
+		v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
+					&ctx->hdl);
+	if (dst_buf->vb2_buf.req_obj.req &&
+	    dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
+		v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
+					&ctx->hdl);
+
 	device_process(ctx, src_buf, dst_buf);
 
+	/* Complete request if needed */
+	if (src_buf->vb2_buf.req_obj.req)
+		v4l2_ctrl_request_complete(src_buf->vb2_buf.req_obj.req,
+					&ctx->hdl);
+	if (dst_buf->vb2_buf.req_obj.req &&
+	    dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
+		v4l2_ctrl_request_complete(dst_buf->vb2_buf.req_obj.req,
+					&ctx->hdl);
+
 	/* Run a timer, which simulates a hardware irq  */
 	schedule_irq(dev, ctx->transtime);
 }
@@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue *q)
 			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
 		if (vbuf == NULL)
 			return;
+		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
+					   &ctx->hdl);
 		spin_lock_irqsave(&ctx->dev->irqlock, flags);
 		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
 		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
@@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
 	.job_abort	= job_abort,
 };
 
+static const struct media_device_ops m2m_media_ops = {
+	.req_queue = vb2_request_queue,
+};
+
 static int vim2m_probe(struct platform_device *pdev)
 {
 	struct vim2m_dev *dev;
@@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device *pdev)
 	dev->mdev.dev = &pdev->dev;
 	strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
 	media_device_init(&dev->mdev);
+	dev->mdev.ops = &m2m_media_ops;
 	dev->v4l2_dev.mdev = &dev->mdev;
 	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
 	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
-- 
2.16.3

[RFCv11 PATCH 28/29] vivid: add mc

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add support for the media_device to vivid. This is a prerequisite
for request support.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vivid/vivid-core.c | 61 +++++++++++++++++++++++++++++++
 drivers/media/platform/vivid/vivid-core.h |  8 ++++
 2 files changed, 69 insertions(+)

diff --git a/drivers/media/platform/vivid/vivid-core.c b/drivers/media/platform/vivid/vivid-core.c
index 82ec216f2ad8..69386b26d5dd 100644
--- a/drivers/media/platform/vivid/vivid-core.c
+++ b/drivers/media/platform/vivid/vivid-core.c
@@ -657,6 +657,15 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 
 	dev->inst = inst;
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+	dev->v4l2_dev.mdev = &dev->mdev;
+
+	/* Initialize media device */
+	strlcpy(dev->mdev.model, VIVID_MODULE_NAME, sizeof(dev->mdev.model));
+	dev->mdev.dev = &pdev->dev;
+	media_device_init(&dev->mdev);
+#endif
+
 	/* register v4l2_device */
 	snprintf(dev->v4l2_dev.name, sizeof(dev->v4l2_dev.name),
 			"%s-%03d", VIVID_MODULE_NAME, inst);
@@ -1173,6 +1182,13 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 		vfd->lock = &dev->mutex;
 		video_set_drvdata(vfd, dev);
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+		dev->vid_cap_pad.flags = MEDIA_PAD_FL_SINK;
+		ret = media_entity_pads_init(&vfd->entity, 1, &dev->vid_cap_pad);
+		if (ret)
+			goto unreg_dev;
+#endif
+
 #ifdef CONFIG_VIDEO_VIVID_CEC
 		if (in_type_counter[HDMI]) {
 			struct cec_adapter *adap;
@@ -1225,6 +1241,13 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 		vfd->lock = &dev->mutex;
 		video_set_drvdata(vfd, dev);
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+		dev->vid_out_pad.flags = MEDIA_PAD_FL_SOURCE;
+		ret = media_entity_pads_init(&vfd->entity, 1, &dev->vid_out_pad);
+		if (ret)
+			goto unreg_dev;
+#endif
+
 #ifdef CONFIG_VIDEO_VIVID_CEC
 		for (i = 0; i < dev->num_outputs; i++) {
 			struct cec_adapter *adap;
@@ -1274,6 +1297,13 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 		vfd->tvnorms = tvnorms_cap;
 		video_set_drvdata(vfd, dev);
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+		dev->vbi_cap_pad.flags = MEDIA_PAD_FL_SINK;
+		ret = media_entity_pads_init(&vfd->entity, 1, &dev->vbi_cap_pad);
+		if (ret)
+			goto unreg_dev;
+#endif
+
 		ret = video_register_device(vfd, VFL_TYPE_VBI, vbi_cap_nr[inst]);
 		if (ret < 0)
 			goto unreg_dev;
@@ -1299,6 +1329,13 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 		vfd->tvnorms = tvnorms_out;
 		video_set_drvdata(vfd, dev);
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+		dev->vbi_out_pad.flags = MEDIA_PAD_FL_SOURCE;
+		ret = media_entity_pads_init(&vfd->entity, 1, &dev->vbi_out_pad);
+		if (ret)
+			goto unreg_dev;
+#endif
+
 		ret = video_register_device(vfd, VFL_TYPE_VBI, vbi_out_nr[inst]);
 		if (ret < 0)
 			goto unreg_dev;
@@ -1322,6 +1359,13 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 		vfd->lock = &dev->mutex;
 		video_set_drvdata(vfd, dev);
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+		dev->sdr_cap_pad.flags = MEDIA_PAD_FL_SINK;
+		ret = media_entity_pads_init(&vfd->entity, 1, &dev->sdr_cap_pad);
+		if (ret)
+			goto unreg_dev;
+#endif
+
 		ret = video_register_device(vfd, VFL_TYPE_SDR, sdr_cap_nr[inst]);
 		if (ret < 0)
 			goto unreg_dev;
@@ -1368,12 +1412,25 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 					  video_device_node_name(vfd));
 	}
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+	/* Register the media device */
+	ret = media_device_register(&dev->mdev);
+	if (ret) {
+		dev_err(dev->mdev.dev,
+			"media device register failed (err=%d)\n", ret);
+		goto unreg_dev;
+	}
+#endif
+
 	/* Now that everything is fine, let's add it to device list */
 	vivid_devs[inst] = dev;
 
 	return 0;
 
 unreg_dev:
+#ifdef CONFIG_MEDIA_CONTROLLER
+	media_device_unregister(&dev->mdev);
+#endif
 	video_unregister_device(&dev->radio_tx_dev);
 	video_unregister_device(&dev->radio_rx_dev);
 	video_unregister_device(&dev->sdr_cap_dev);
@@ -1444,6 +1501,10 @@ static int vivid_remove(struct platform_device *pdev)
 		if (!dev)
 			continue;
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+		media_device_unregister(&dev->mdev);
+#endif
+
 		if (dev->has_vid_cap) {
 			v4l2_info(&dev->v4l2_dev, "unregistering %s\n",
 				video_device_node_name(&dev->vid_cap_dev));
diff --git a/drivers/media/platform/vivid/vivid-core.h b/drivers/media/platform/vivid/vivid-core.h
index 477c80a4d44c..6ccd1f5c1d91 100644
--- a/drivers/media/platform/vivid/vivid-core.h
+++ b/drivers/media/platform/vivid/vivid-core.h
@@ -136,6 +136,14 @@ struct vivid_cec_work {
 struct vivid_dev {
 	unsigned			inst;
 	struct v4l2_device		v4l2_dev;
+#ifdef CONFIG_MEDIA_CONTROLLER
+	struct media_device		mdev;
+	struct media_pad		vid_cap_pad;
+	struct media_pad		vid_out_pad;
+	struct media_pad		vbi_cap_pad;
+	struct media_pad		vbi_out_pad;
+	struct media_pad		sdr_cap_pad;
+#endif
 	struct v4l2_ctrl_handler	ctrl_hdl_user_gen;
 	struct v4l2_ctrl_handler	ctrl_hdl_user_vid;
 	struct v4l2_ctrl_handler	ctrl_hdl_user_aud;
-- 
2.16.3

[RFCv11 PATCH 29/29] vivid: add request support

[Hans Verkuil]

From: Hans Verkuil <hans.verkuil@cisco.com>

Add support for requests to vivid.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vivid/vivid-core.c        |  7 +++++++
 drivers/media/platform/vivid/vivid-kthread-cap.c | 12 ++++++++++++
 drivers/media/platform/vivid/vivid-kthread-out.c | 12 ++++++++++++
 drivers/media/platform/vivid/vivid-sdr-cap.c     |  8 ++++++++
 drivers/media/platform/vivid/vivid-vbi-cap.c     |  2 ++
 drivers/media/platform/vivid/vivid-vbi-out.c     |  2 ++
 drivers/media/platform/vivid/vivid-vid-cap.c     |  2 ++
 drivers/media/platform/vivid/vivid-vid-out.c     |  2 ++
 8 files changed, 47 insertions(+)

diff --git a/drivers/media/platform/vivid/vivid-core.c b/drivers/media/platform/vivid/vivid-core.c
index 69386b26d5dd..20e74d36b673 100644
--- a/drivers/media/platform/vivid/vivid-core.c
+++ b/drivers/media/platform/vivid/vivid-core.c
@@ -627,6 +627,12 @@ static void vivid_dev_release(struct v4l2_device *v4l2_dev)
 	kfree(dev);
 }
 
+#ifdef CONFIG_MEDIA_CONTROLLER
+static const struct media_device_ops vivid_media_ops = {
+	.req_queue = vb2_request_queue,
+};
+#endif
+
 static int vivid_create_instance(struct platform_device *pdev, int inst)
 {
 	static const struct v4l2_dv_timings def_dv_timings =
@@ -664,6 +670,7 @@ static int vivid_create_instance(struct platform_device *pdev, int inst)
 	strlcpy(dev->mdev.model, VIVID_MODULE_NAME, sizeof(dev->mdev.model));
 	dev->mdev.dev = &pdev->dev;
 	media_device_init(&dev->mdev);
+	dev->mdev.ops = &vivid_media_ops;
 #endif
 
 	/* register v4l2_device */
diff --git a/drivers/media/platform/vivid/vivid-kthread-cap.c b/drivers/media/platform/vivid/vivid-kthread-cap.c
index 3fdb280c36ca..c192b4b1b9de 100644
--- a/drivers/media/platform/vivid/vivid-kthread-cap.c
+++ b/drivers/media/platform/vivid/vivid-kthread-cap.c
@@ -703,6 +703,8 @@ static void vivid_thread_vid_cap_tick(struct vivid_dev *dev, int dropped_bufs)
 		goto update_mv;
 
 	if (vid_cap_buf) {
+		v4l2_ctrl_request_setup(vid_cap_buf->vb.vb2_buf.req_obj.req,
+					&dev->ctrl_hdl_vid_cap);
 		/* Fill buffer */
 		vivid_fillbuff(dev, vid_cap_buf);
 		dprintk(dev, 1, "filled buffer %d\n",
@@ -713,6 +715,8 @@ static void vivid_thread_vid_cap_tick(struct vivid_dev *dev, int dropped_bufs)
 			dev->fb_cap.fmt.pixelformat == dev->fmt_cap->fourcc)
 			vivid_overlay(dev, vid_cap_buf);
 
+		v4l2_ctrl_request_complete(vid_cap_buf->vb.vb2_buf.req_obj.req,
+					   &dev->ctrl_hdl_vid_cap);
 		vb2_buffer_done(&vid_cap_buf->vb.vb2_buf, dev->dqbuf_error ?
 				VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
 		dprintk(dev, 2, "vid_cap buffer %d done\n",
@@ -720,10 +724,14 @@ static void vivid_thread_vid_cap_tick(struct vivid_dev *dev, int dropped_bufs)
 	}
 
 	if (vbi_cap_buf) {
+		v4l2_ctrl_request_setup(vbi_cap_buf->vb.vb2_buf.req_obj.req,
+					&dev->ctrl_hdl_vbi_cap);
 		if (dev->stream_sliced_vbi_cap)
 			vivid_sliced_vbi_cap_process(dev, vbi_cap_buf);
 		else
 			vivid_raw_vbi_cap_process(dev, vbi_cap_buf);
+		v4l2_ctrl_request_complete(vbi_cap_buf->vb.vb2_buf.req_obj.req,
+					   &dev->ctrl_hdl_vbi_cap);
 		vb2_buffer_done(&vbi_cap_buf->vb.vb2_buf, dev->dqbuf_error ?
 				VB2_BUF_STATE_ERROR : VB2_BUF_STATE_DONE);
 		dprintk(dev, 2, "vbi_cap %d done\n",
@@ -891,6 +899,8 @@ void vivid_stop_generating_vid_cap(struct vivid_dev *dev, bool *pstreaming)
 			buf = list_entry(dev->vid_cap_active.next,
 					 struct vivid_buffer, list);
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vid_cap);
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 			dprintk(dev, 2, "vid_cap buffer %d done\n",
 				buf->vb.vb2_buf.index);
@@ -904,6 +914,8 @@ void vivid_stop_generating_vid_cap(struct vivid_dev *dev, bool *pstreaming)
 			buf = list_entry(dev->vbi_cap_active.next,
 					 struct vivid_buffer, list);
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vbi_cap);
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 			dprintk(dev, 2, "vbi_cap buffer %d done\n",
 				buf->vb.vb2_buf.index);
diff --git a/drivers/media/platform/vivid/vivid-kthread-out.c b/drivers/media/platform/vivid/vivid-kthread-out.c
index 9981e7548019..5a14810eeb69 100644
--- a/drivers/media/platform/vivid/vivid-kthread-out.c
+++ b/drivers/media/platform/vivid/vivid-kthread-out.c
@@ -75,6 +75,10 @@ static void vivid_thread_vid_out_tick(struct vivid_dev *dev)
 		return;
 
 	if (vid_out_buf) {
+		v4l2_ctrl_request_setup(vid_out_buf->vb.vb2_buf.req_obj.req,
+					&dev->ctrl_hdl_vid_out);
+		v4l2_ctrl_request_complete(vid_out_buf->vb.vb2_buf.req_obj.req,
+					   &dev->ctrl_hdl_vid_out);
 		vid_out_buf->vb.sequence = dev->vid_out_seq_count;
 		if (dev->field_out == V4L2_FIELD_ALTERNATE) {
 			/*
@@ -92,6 +96,10 @@ static void vivid_thread_vid_out_tick(struct vivid_dev *dev)
 	}
 
 	if (vbi_out_buf) {
+		v4l2_ctrl_request_setup(vbi_out_buf->vb.vb2_buf.req_obj.req,
+					&dev->ctrl_hdl_vbi_out);
+		v4l2_ctrl_request_complete(vbi_out_buf->vb.vb2_buf.req_obj.req,
+					   &dev->ctrl_hdl_vbi_out);
 		if (dev->stream_sliced_vbi_out)
 			vivid_sliced_vbi_out_process(dev, vbi_out_buf);
 
@@ -262,6 +270,8 @@ void vivid_stop_generating_vid_out(struct vivid_dev *dev, bool *pstreaming)
 			buf = list_entry(dev->vid_out_active.next,
 					 struct vivid_buffer, list);
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vid_out);
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 			dprintk(dev, 2, "vid_out buffer %d done\n",
 				buf->vb.vb2_buf.index);
@@ -275,6 +285,8 @@ void vivid_stop_generating_vid_out(struct vivid_dev *dev, bool *pstreaming)
 			buf = list_entry(dev->vbi_out_active.next,
 					 struct vivid_buffer, list);
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vbi_out);
 			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 			dprintk(dev, 2, "vbi_out buffer %d done\n",
 				buf->vb.vb2_buf.index);
diff --git a/drivers/media/platform/vivid/vivid-sdr-cap.c b/drivers/media/platform/vivid/vivid-sdr-cap.c
index cfb7cb4d37a8..2c466cc4080d 100644
--- a/drivers/media/platform/vivid/vivid-sdr-cap.c
+++ b/drivers/media/platform/vivid/vivid-sdr-cap.c
@@ -102,6 +102,10 @@ static void vivid_thread_sdr_cap_tick(struct vivid_dev *dev)
 
 	if (sdr_cap_buf) {
 		sdr_cap_buf->vb.sequence = dev->sdr_cap_seq_count;
+		v4l2_ctrl_request_setup(sdr_cap_buf->vb.vb2_buf.req_obj.req,
+					&dev->ctrl_hdl_sdr_cap);
+		v4l2_ctrl_request_complete(sdr_cap_buf->vb.vb2_buf.req_obj.req,
+					   &dev->ctrl_hdl_sdr_cap);
 		vivid_sdr_cap_process(dev, sdr_cap_buf);
 		sdr_cap_buf->vb.vb2_buf.timestamp =
 			ktime_get_ns() + dev->time_wrap_offset;
@@ -272,6 +276,8 @@ static int sdr_cap_start_streaming(struct vb2_queue *vq, unsigned count)
 
 		list_for_each_entry_safe(buf, tmp, &dev->sdr_cap_active, list) {
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_sdr_cap);
 			vb2_buffer_done(&buf->vb.vb2_buf,
 					VB2_BUF_STATE_QUEUED);
 		}
@@ -293,6 +299,8 @@ static void sdr_cap_stop_streaming(struct vb2_queue *vq)
 		buf = list_entry(dev->sdr_cap_active.next,
 				struct vivid_buffer, list);
 		list_del(&buf->list);
+		v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+					   &dev->ctrl_hdl_sdr_cap);
 		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
 	}
 
diff --git a/drivers/media/platform/vivid/vivid-vbi-cap.c b/drivers/media/platform/vivid/vivid-vbi-cap.c
index 92a852955173..c5601639a0f3 100644
--- a/drivers/media/platform/vivid/vivid-vbi-cap.c
+++ b/drivers/media/platform/vivid/vivid-vbi-cap.c
@@ -204,6 +204,8 @@ static int vbi_cap_start_streaming(struct vb2_queue *vq, unsigned count)
 
 		list_for_each_entry_safe(buf, tmp, &dev->vbi_cap_active, list) {
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vbi_cap);
 			vb2_buffer_done(&buf->vb.vb2_buf,
 					VB2_BUF_STATE_QUEUED);
 		}
diff --git a/drivers/media/platform/vivid/vivid-vbi-out.c b/drivers/media/platform/vivid/vivid-vbi-out.c
index 69486c130a7e..7c06bbf5fc7a 100644
--- a/drivers/media/platform/vivid/vivid-vbi-out.c
+++ b/drivers/media/platform/vivid/vivid-vbi-out.c
@@ -96,6 +96,8 @@ static int vbi_out_start_streaming(struct vb2_queue *vq, unsigned count)
 
 		list_for_each_entry_safe(buf, tmp, &dev->vbi_out_active, list) {
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vbi_out);
 			vb2_buffer_done(&buf->vb.vb2_buf,
 					VB2_BUF_STATE_QUEUED);
 		}
diff --git a/drivers/media/platform/vivid/vivid-vid-cap.c b/drivers/media/platform/vivid/vivid-vid-cap.c
index 01c703683657..73cdfa5349a6 100644
--- a/drivers/media/platform/vivid/vivid-vid-cap.c
+++ b/drivers/media/platform/vivid/vivid-vid-cap.c
@@ -240,6 +240,8 @@ static int vid_cap_start_streaming(struct vb2_queue *vq, unsigned count)
 
 		list_for_each_entry_safe(buf, tmp, &dev->vid_cap_active, list) {
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vid_cap);
 			vb2_buffer_done(&buf->vb.vb2_buf,
 					VB2_BUF_STATE_QUEUED);
 		}
diff --git a/drivers/media/platform/vivid/vivid-vid-out.c b/drivers/media/platform/vivid/vivid-vid-out.c
index 51fec66d8d45..b53805fca7b4 100644
--- a/drivers/media/platform/vivid/vivid-vid-out.c
+++ b/drivers/media/platform/vivid/vivid-vid-out.c
@@ -162,6 +162,8 @@ static int vid_out_start_streaming(struct vb2_queue *vq, unsigned count)
 
 		list_for_each_entry_safe(buf, tmp, &dev->vid_out_active, list) {
 			list_del(&buf->list);
+			v4l2_ctrl_request_complete(buf->vb.vb2_buf.req_obj.req,
+						   &dev->ctrl_hdl_vid_out);
 			vb2_buffer_done(&buf->vb.vb2_buf,
 					VB2_BUF_STATE_QUEUED);
 		}
-- 
2.16.3

Re: [RFCv11 PATCH 17/29] vb2: store userspace data in vb2_v4l2_buffer

[Kieran Bingham]

Hi Hans,

Thank you for the patch series !

I'm looking forwards to finding some time to try out this work.

Just briefly scanning through the series, and I saw the minor issue below.

Regards

Kieran


On 09/04/18 15:20, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> The userspace-provided plane data needs to be stored in
> vb2_v4l2_buffer. Currently this information is applied by
> __fill_vb2_buffer() which is called by the core prepare_buf
> and qbuf functions, but when using requests these functions
> aren't called yet since the buffer won't be prepared until
> the media request is actually queued.
> 
> In the meantime this information has to be stored somewhere
> and vb2_v4l2_buffer is a good place for it.
> 
> The __fill_vb2_buffer callback now just copies the relevant
> information from vb2_v4l2_buffer into the planes array.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c |  25 +-
>  drivers/media/common/videobuf2/videobuf2-v4l2.c | 324 +++++++++++++-----------
>  drivers/media/dvb-core/dvb_vb2.c                |   3 +-
>  include/media/videobuf2-core.h                  |   3 +-
>  include/media/videobuf2-v4l2.h                  |   2 +
>  5 files changed, 197 insertions(+), 160 deletions(-)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index d3f7bb33a54d..3d436ccb61f8 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -968,9 +968,8 @@ static int __prepare_mmap(struct vb2_buffer *vb, const void *pb)

Now that pb is unused here, should it be removed from the function arguments ?

>  {
>  	int ret = 0;
>  
> -	if (pb)
> -		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> -				 vb, pb, vb->planes);
> +	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> +			 vb, vb->planes);
>  	return ret ? ret : call_vb_qop(vb, buf_prepare, vb);
>  }
>  
> @@ -988,12 +987,10 @@ static int __prepare_userptr(struct vb2_buffer *vb, const void *pb)

Same comment here

>  
>  	memset(planes, 0, sizeof(planes[0]) * vb->num_planes);
>  	/* Copy relevant information provided by the userspace */
> -	if (pb) {
> -		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> -				 vb, pb, planes);
> -		if (ret)
> -			return ret;
> -	}
> +	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> +			 vb, planes);
> +	if (ret)
> +		return ret;
>  
>  	for (plane = 0; plane < vb->num_planes; ++plane) {
>  		/* Skip the plane if already verified */
> @@ -1104,12 +1101,10 @@ static int __prepare_dmabuf(struct vb2_buffer *vb, const void *pb)

And here :D

>  
>  	memset(planes, 0, sizeof(planes[0]) * vb->num_planes);
>  	/* Copy relevant information provided by the userspace */
> -	if (pb) {
> -		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> -				 vb, pb, planes);
> -		if (ret)
> -			return ret;
> -	}
> +	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> +			 vb, planes);
> +	if (ret)
> +		return ret;
>  
>  	for (plane = 0; plane < vb->num_planes; ++plane) {
>  		struct dma_buf *dbuf = dma_buf_get(planes[plane].m.fd);
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 4e9c77f21858..bf7a3ba9fed0 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -154,9 +154,177 @@ static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
>  		pr_warn("use the actual size instead.\n");
>  }
>  
> +static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b)
> +{
> +	struct vb2_queue *q = vb->vb2_queue;
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct vb2_plane *planes = vbuf->planes;
> +	unsigned int plane;
> +	int ret;
> +
> +	ret = __verify_length(vb, b);
> +	if (ret < 0) {
> +		dprintk(1, "plane parameters verification failed: %d\n", ret);
> +		return ret;
> +	}
> +	if (b->field == V4L2_FIELD_ALTERNATE && q->is_output) {
> +		/*
> +		 * If the format's field is ALTERNATE, then the buffer's field
> +		 * should be either TOP or BOTTOM, not ALTERNATE since that
> +		 * makes no sense. The driver has to know whether the
> +		 * buffer represents a top or a bottom field in order to
> +		 * program any DMA correctly. Using ALTERNATE is wrong, since
> +		 * that just says that it is either a top or a bottom field,
> +		 * but not which of the two it is.
> +		 */
> +		dprintk(1, "the field is incorrectly set to ALTERNATE for an output buffer\n");
> +		return -EINVAL;
> +	}
> +	vbuf->sequence = 0;
> +
> +	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
> +		switch (b->memory) {
> +		case VB2_MEMORY_USERPTR:
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				planes[plane].m.userptr =
> +					b->m.planes[plane].m.userptr;
> +				planes[plane].length =
> +					b->m.planes[plane].length;
> +			}
> +			break;
> +		case VB2_MEMORY_DMABUF:
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				planes[plane].m.fd =
> +					b->m.planes[plane].m.fd;
> +				planes[plane].length =
> +					b->m.planes[plane].length;
> +			}
> +			break;
> +		default:
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				planes[plane].m.offset =
> +					vb->planes[plane].m.offset;
> +				planes[plane].length =
> +					vb->planes[plane].length;
> +			}
> +			break;
> +		}
> +
> +		/* Fill in driver-provided information for OUTPUT types */
> +		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> +			/*
> +			 * Will have to go up to b->length when API starts
> +			 * accepting variable number of planes.
> +			 *
> +			 * If bytesused == 0 for the output buffer, then fall
> +			 * back to the full buffer size. In that case
> +			 * userspace clearly never bothered to set it and
> +			 * it's a safe assumption that they really meant to
> +			 * use the full plane sizes.
> +			 *
> +			 * Some drivers, e.g. old codec drivers, use bytesused == 0
> +			 * as a way to indicate that streaming is finished.
> +			 * In that case, the driver should use the
> +			 * allow_zero_bytesused flag to keep old userspace
> +			 * applications working.
> +			 */
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				struct vb2_plane *pdst = &planes[plane];
> +				struct v4l2_plane *psrc = &b->m.planes[plane];
> +
> +				if (psrc->bytesused == 0)
> +					vb2_warn_zero_bytesused(vb);
> +
> +				if (vb->vb2_queue->allow_zero_bytesused)
> +					pdst->bytesused = psrc->bytesused;
> +				else
> +					pdst->bytesused = psrc->bytesused ?
> +						psrc->bytesused : pdst->length;
> +				pdst->data_offset = psrc->data_offset;
> +			}
> +		}
> +	} else {
> +		/*
> +		 * Single-planar buffers do not use planes array,
> +		 * so fill in relevant v4l2_buffer struct fields instead.
> +		 * In videobuf we use our internal V4l2_planes struct for
> +		 * single-planar buffers as well, for simplicity.
> +		 *
> +		 * If bytesused == 0 for the output buffer, then fall back
> +		 * to the full buffer size as that's a sensible default.
> +		 *
> +		 * Some drivers, e.g. old codec drivers, use bytesused == 0 as
> +		 * a way to indicate that streaming is finished. In that case,
> +		 * the driver should use the allow_zero_bytesused flag to keep
> +		 * old userspace applications working.
> +		 */
> +		switch (b->memory) {
> +		case VB2_MEMORY_USERPTR:
> +			planes[0].m.userptr = b->m.userptr;
> +			planes[0].length = b->length;
> +			break;
> +		case VB2_MEMORY_DMABUF:
> +			planes[0].m.fd = b->m.fd;
> +			planes[0].length = b->length;
> +			break;
> +		default:
> +			planes[0].m.offset = vb->planes[0].m.offset;
> +			planes[0].length = vb->planes[0].length;
> +			break;
> +		}
> +
> +		planes[0].data_offset = 0;
> +		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> +			if (b->bytesused == 0)
> +				vb2_warn_zero_bytesused(vb);
> +
> +			if (vb->vb2_queue->allow_zero_bytesused)
> +				planes[0].bytesused = b->bytesused;
> +			else
> +				planes[0].bytesused = b->bytesused ?
> +					b->bytesused : planes[0].length;
> +		} else
> +			planes[0].bytesused = 0;
> +
> +	}
> +
> +	/* Zero flags that we handle */
> +	vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
> +	if (!vb->vb2_queue->copy_timestamp || !V4L2_TYPE_IS_OUTPUT(b->type)) {
> +		/*
> +		 * Non-COPY timestamps and non-OUTPUT queues will get
> +		 * their timestamp and timestamp source flags from the
> +		 * queue.
> +		 */
> +		vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
> +	}
> +
> +	if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> +		/*
> +		 * For output buffers mask out the timecode flag:
> +		 * this will be handled later in vb2_qbuf().
> +		 * The 'field' is valid metadata for this output buffer
> +		 * and so that needs to be copied here.
> +		 */
> +		vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
> +		vbuf->field = b->field;
> +	} else {
> +		/* Zero any output buffer flags as this is a capture buffer */
> +		vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
> +		/* Zero last flag, this is a signal from driver to userspace */
> +		vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
> +	}
> +
> +	return 0;
> +}
> +
>  static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
>  				    const char *opname)
>  {
> +	struct vb2_v4l2_buffer *vbuf;
> +	struct vb2_buffer *vb;
> +	int ret;
> +
>  	if (b->type != q->type) {
>  		dprintk(1, "%s: invalid buffer type\n", opname);
>  		return -EINVAL;
> @@ -178,7 +346,15 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
>  		return -EINVAL;
>  	}
>  
> -	return __verify_planes_array(q->bufs[b->index], b);
> +	vb = q->bufs[b->index];
> +	vbuf = to_vb2_v4l2_buffer(vb);
> +	ret = __verify_planes_array(vb, b);
> +	if (ret)
> +		return ret;
> +
> +	/* Copy relevant information provided by the userspace */
> +	memset(vbuf->planes, 0, sizeof(vbuf->planes[0]) * vb->num_planes);
> +	return vb2_fill_vb2_v4l2_buffer(vb, b);
>  }
>  
>  /*
> @@ -291,153 +467,19 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
>   * v4l2_buffer by the userspace. It also verifies that struct
>   * v4l2_buffer has a valid number of planes.
>   */
> -static int __fill_vb2_buffer(struct vb2_buffer *vb,
> -		const void *pb, struct vb2_plane *planes)
> +static int __fill_vb2_buffer(struct vb2_buffer *vb, struct vb2_plane *planes)
>  {
> -	struct vb2_queue *q = vb->vb2_queue;
> -	const struct v4l2_buffer *b = pb;
>  	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>  	unsigned int plane;
> -	int ret;
>  
> -	ret = __verify_length(vb, b);
> -	if (ret < 0) {
> -		dprintk(1, "plane parameters verification failed: %d\n", ret);
> -		return ret;
> -	}
> -	if (b->field == V4L2_FIELD_ALTERNATE && q->is_output) {
> -		/*
> -		 * If the format's field is ALTERNATE, then the buffer's field
> -		 * should be either TOP or BOTTOM, not ALTERNATE since that
> -		 * makes no sense. The driver has to know whether the
> -		 * buffer represents a top or a bottom field in order to
> -		 * program any DMA correctly. Using ALTERNATE is wrong, since
> -		 * that just says that it is either a top or a bottom field,
> -		 * but not which of the two it is.
> -		 */
> -		dprintk(1, "the field is incorrectly set to ALTERNATE for an output buffer\n");
> -		return -EINVAL;
> -	}
>  	vb->timestamp = 0;
> -	vbuf->sequence = 0;
> -
> -	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
> -		if (b->memory == VB2_MEMORY_USERPTR) {
> -			for (plane = 0; plane < vb->num_planes; ++plane) {
> -				planes[plane].m.userptr =
> -					b->m.planes[plane].m.userptr;
> -				planes[plane].length =
> -					b->m.planes[plane].length;
> -			}
> -		}
> -		if (b->memory == VB2_MEMORY_DMABUF) {
> -			for (plane = 0; plane < vb->num_planes; ++plane) {
> -				planes[plane].m.fd =
> -					b->m.planes[plane].m.fd;
> -				planes[plane].length =
> -					b->m.planes[plane].length;
> -			}
> -		}
> -
> -		/* Fill in driver-provided information for OUTPUT types */
> -		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> -			/*
> -			 * Will have to go up to b->length when API starts
> -			 * accepting variable number of planes.
> -			 *
> -			 * If bytesused == 0 for the output buffer, then fall
> -			 * back to the full buffer size. In that case
> -			 * userspace clearly never bothered to set it and
> -			 * it's a safe assumption that they really meant to
> -			 * use the full plane sizes.
> -			 *
> -			 * Some drivers, e.g. old codec drivers, use bytesused == 0
> -			 * as a way to indicate that streaming is finished.
> -			 * In that case, the driver should use the
> -			 * allow_zero_bytesused flag to keep old userspace
> -			 * applications working.
> -			 */
> -			for (plane = 0; plane < vb->num_planes; ++plane) {
> -				struct vb2_plane *pdst = &planes[plane];
> -				struct v4l2_plane *psrc = &b->m.planes[plane];
> -
> -				if (psrc->bytesused == 0)
> -					vb2_warn_zero_bytesused(vb);
>  
> -				if (vb->vb2_queue->allow_zero_bytesused)
> -					pdst->bytesused = psrc->bytesused;
> -				else
> -					pdst->bytesused = psrc->bytesused ?
> -						psrc->bytesused : pdst->length;
> -				pdst->data_offset = psrc->data_offset;
> -			}
> -		}
> -	} else {
> -		/*
> -		 * Single-planar buffers do not use planes array,
> -		 * so fill in relevant v4l2_buffer struct fields instead.
> -		 * In videobuf we use our internal V4l2_planes struct for
> -		 * single-planar buffers as well, for simplicity.
> -		 *
> -		 * If bytesused == 0 for the output buffer, then fall back
> -		 * to the full buffer size as that's a sensible default.
> -		 *
> -		 * Some drivers, e.g. old codec drivers, use bytesused == 0 as
> -		 * a way to indicate that streaming is finished. In that case,
> -		 * the driver should use the allow_zero_bytesused flag to keep
> -		 * old userspace applications working.
> -		 */
> -		if (b->memory == VB2_MEMORY_USERPTR) {
> -			planes[0].m.userptr = b->m.userptr;
> -			planes[0].length = b->length;
> -		}
> -
> -		if (b->memory == VB2_MEMORY_DMABUF) {
> -			planes[0].m.fd = b->m.fd;
> -			planes[0].length = b->length;
> -		}
> -
> -		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> -			if (b->bytesused == 0)
> -				vb2_warn_zero_bytesused(vb);
> -
> -			if (vb->vb2_queue->allow_zero_bytesused)
> -				planes[0].bytesused = b->bytesused;
> -			else
> -				planes[0].bytesused = b->bytesused ?
> -					b->bytesused : planes[0].length;
> -		} else
> -			planes[0].bytesused = 0;
> -
> -	}
> -
> -	/* Zero flags that the vb2 core handles */
> -	vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
> -	if (!vb->vb2_queue->copy_timestamp || !V4L2_TYPE_IS_OUTPUT(b->type)) {
> -		/*
> -		 * Non-COPY timestamps and non-OUTPUT queues will get
> -		 * their timestamp and timestamp source flags from the
> -		 * queue.
> -		 */
> -		vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
> +	for (plane = 0; plane < vb->num_planes; ++plane) {
> +		planes[plane].m = vbuf->planes[plane].m;
> +		planes[plane].length = vbuf->planes[plane].length;
> +		planes[plane].bytesused = vbuf->planes[plane].bytesused;
> +		planes[plane].data_offset = vbuf->planes[plane].data_offset;
>  	}
> -
> -	if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> -		/*
> -		 * For output buffers mask out the timecode flag:
> -		 * this will be handled later in vb2_qbuf().
> -		 * The 'field' is valid metadata for this output buffer
> -		 * and so that needs to be copied here.
> -		 */
> -		vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
> -		vbuf->field = b->field;
> -	} else {
> -		/* Zero any output buffer flags as this is a capture buffer */
> -		vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
> -		/* Zero last flag, this is a signal from driver to userspace */
> -		vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
> -	}
> -
>  	return 0;
>  }
>  
> diff --git a/drivers/media/dvb-core/dvb_vb2.c b/drivers/media/dvb-core/dvb_vb2.c
> index b811adf88afa..da6a8cec7d42 100644
> --- a/drivers/media/dvb-core/dvb_vb2.c
> +++ b/drivers/media/dvb-core/dvb_vb2.c
> @@ -146,8 +146,7 @@ static void _fill_dmx_buffer(struct vb2_buffer *vb, void *pb)
>  	dprintk(3, "[%s]\n", ctx->name);
>  }
>  
> -static int _fill_vb2_buffer(struct vb2_buffer *vb,
> -			    const void *pb, struct vb2_plane *planes)
> +static int _fill_vb2_buffer(struct vb2_buffer *vb, struct vb2_plane *planes)
>  {
>  	struct dvb_vb2_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
>  
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index f6818f732f34..224c4820a044 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -417,8 +417,7 @@ struct vb2_ops {
>  struct vb2_buf_ops {
>  	int (*verify_planes_array)(struct vb2_buffer *vb, const void *pb);
>  	void (*fill_user_buffer)(struct vb2_buffer *vb, void *pb);
> -	int (*fill_vb2_buffer)(struct vb2_buffer *vb, const void *pb,
> -				struct vb2_plane *planes);
> +	int (*fill_vb2_buffer)(struct vb2_buffer *vb, struct vb2_plane *planes);
>  	void (*copy_timestamp)(struct vb2_buffer *vb, const void *pb);
>  };
>  
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 3d5e2d739f05..097bf3e6951d 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -32,6 +32,7 @@
>   *		&enum v4l2_field.
>   * @timecode:	frame timecode.
>   * @sequence:	sequence count of this frame.
> + * @planes:	plane information (userptr/fd, length, bytesused, data_offset).
>   *
>   * Should contain enough information to be able to cover all the fields
>   * of &struct v4l2_buffer at ``videodev2.h``.
> @@ -43,6 +44,7 @@ struct vb2_v4l2_buffer {
>  	__u32			field;
>  	struct v4l2_timecode	timecode;
>  	__u32			sequence;
> +	struct vb2_plane	planes[VB2_MAX_PLANES];
>  };
>  
>  /*
> 

Re: [RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Define the public request API.

> This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
> and two ioctls that operate on a request in order to queue the
> contents of the request to the driver and to re-initialize the
> request.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   include/uapi/linux/media.h | 8 ++++++++
>   1 file changed, 8 insertions(+)

> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index c7e9a5cba24e..f8769e74f847 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -342,11 +342,19 @@ struct media_v2_topology {

>   /* ioctls */

> +struct __attribute__ ((packed)) media_request_alloc {
> +       __s32 fd;
> +};
> +
>   #define MEDIA_IOC_DEVICE_INFO  _IOWR('|', 0x00, struct media_device_info)
>   #define MEDIA_IOC_ENUM_ENTITIES        _IOWR('|', 0x01, struct
media_entity_desc)
>   #define MEDIA_IOC_ENUM_LINKS   _IOWR('|', 0x02, struct media_links_enum)
>   #define MEDIA_IOC_SETUP_LINK   _IOWR('|', 0x03, struct media_link_desc)
>   #define MEDIA_IOC_G_TOPOLOGY   _IOWR('|', 0x04, struct media_v2_topology)
> +#define MEDIA_IOC_REQUEST_ALLOC        _IOWR('|', 0x05, struct
media_request_alloc)
> +
> +#define MEDIA_REQUEST_IOC_QUEUE                _IO('|',  0x80)
> +#define MEDIA_REQUEST_IOC_REINIT       _IO('|',  0x81)

I wonder if it wouldn't make sense to add a comment here saying that these
are called on request FD, as opposed to the others above, which are called
on the media FD.

Best regards,
Tomasz

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
[snip]
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> new file mode 100644
> index 000000000000..ead78613fdbe
> --- /dev/null
> +++ b/drivers/media/media-request.c
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Media device request objects
> + *
> + * Copyright (C) 2018 Intel Corporation
> + * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.

I'm not sure about the origin of this line, but it's not a correct
copyright for kernel code produced as a part of Chrome OS project. It would
normally be something like

Copyright (C) 2018 Google, Inc.

> + *
> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> + */
> +
> +#include <linux/anon_inodes.h>
> +#include <linux/file.h>
> +#include <linux/mm.h>
> +#include <linux/string.h>
> +
> +#include <media/media-device.h>
> +#include <media/media-request.h>
> +
> +int media_request_alloc(struct media_device *mdev,
> +                       struct media_request_alloc *alloc)
> +{
> +       return -ENOMEM;
> +}
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index bcc6ec434f1f..07e323c57202 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -19,6 +19,7 @@
>   #ifndef _MEDIA_DEVICE_H
>   #define _MEDIA_DEVICE_H

> +#include <linux/anon_inodes.h>

What is the need for anon_inodes in this header?

>   #include <linux/list.h>
>   #include <linux/mutex.h>

> @@ -27,6 +28,7 @@

>   struct ida;
>   struct device;
> +struct media_device;

>   /**
>    * struct media_entity_notify - Media Entity Notify
> @@ -50,10 +52,16 @@ struct media_entity_notify {
>    * struct media_device_ops - Media device operations
>    * @link_notify: Link state change notification callback. This callback
is
>    *              called with the graph_mutex held.
> + * @req_alloc: Allocate a request
> + * @req_free: Free a request
> + * @req_queue: Queue a request
>    */
>   struct media_device_ops {
>          int (*link_notify)(struct media_link *link, u32 flags,
>                             unsigned int notification);
> +       struct media_request *(*req_alloc)(struct media_device *mdev);
> +       void (*req_free)(struct media_request *req);
> +       int (*req_queue)(struct media_request *req);
>   };

>   /**
> @@ -88,6 +96,8 @@ struct media_device_ops {
>    * @disable_source: Disable Source Handler function pointer
>    *
>    * @ops:       Operation handler callbacks
> + * @req_lock:  Serialise access to requests
> + * @req_queue_mutex: Serialise validating and queueing requests

Let's bikeshed a bit! "access" sounds like a superset of "validating and
queuing" to me. Perhaps it could make sense to be a bit more specific on
what type of access the spinlock is used for?

Best regards,
Tomasz

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
[snip]
> +static void media_request_clean(struct media_request *req)
> +{
> +       struct media_request_object *obj, *obj_safe;
> +
> +       WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
> +
> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> +               media_request_object_unbind(obj);
> +               media_request_object_put(obj);
> +       }
> +
> +       req->num_incomplete_objects = 0;
> +       wake_up_interruptible(&req->poll_wait);

Do we intentionally wake up only one waiter here (as opposed to calling
wake_up_interruptible_all())?

> +}
> +
> +static void media_request_release(struct kref *kref)
> +{
> +       struct media_request *req =
> +               container_of(kref, struct media_request, kref);
> +       struct media_device *mdev = req->mdev;
> +       unsigned long flags;
> +
> +       dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       req->state = MEDIA_REQUEST_STATE_CLEANING;
> +       spin_unlock_irqrestore(&req->lock, flags);
> +
> +       media_request_clean(req);
> +
> +       if (mdev->ops->req_free)
> +               mdev->ops->req_free(req);
> +       else
> +               kfree(req);
> +}
> +
> +void media_request_put(struct media_request *req)
> +{
> +       kref_put(&req->kref, media_request_release);
> +}
> +EXPORT_SYMBOL_GPL(media_request_put);
> +
> +void media_request_cancel(struct media_request *req)
> +{
> +       struct media_request_object *obj, *obj_safe;
> +
> +       if (req->state != MEDIA_REQUEST_STATE_QUEUED)
> +               return;

I can see that media_request_release() guards the state change with
req->lock. However we access the state here without holding the spinlock.

Also, should we perhaps have MEDIA_REQUEST_STATE_CANCELLED (or maybe just
reset to MEDIA_REQUEST_STATE_IDLE?), so that we can guard against calling
this multiple times?

Or perhaps we're expecting this to be called with req_queue_mutex held?

> +
> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
> +               if (obj->ops->cancel)
> +                       obj->ops->cancel(obj);
> +}
> +EXPORT_SYMBOL_GPL(media_request_cancel);
[snip]
> +void media_request_object_init(struct media_request_object *obj)
> +{
> +       obj->ops = NULL;
> +       obj->req = NULL;
> +       obj->priv = NULL;

Perhaps it could make sense to pass ops and priv as arguments here? There
is probably not much value in having a request object with both set to NULL.

> +       obj->completed = false;
> +       INIT_LIST_HEAD(&obj->list);
> +       kref_init(&obj->kref);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_init);
> +
> +void media_request_object_bind(struct media_request *req,
> +                              const struct media_request_object_ops *ops,
> +                              void *priv,
> +                              struct media_request_object *obj)
> +{
> +       unsigned long flags;
> +
> +       if (WARN_ON(!ops->release || !ops->cancel))
> +               return;
> +
> +       obj->req = req;
> +       obj->ops = ops;
> +       obj->priv = priv;

Ah, I see, they're being set here. I guess it's just the matter of how we
define the semantics.

I wonder, though, what happens with a media_request_object struct that was
allocated, initialized, but not bound to a request, because the client
terminated. We would want to call media_request_object_put(obj), but
obj->ops would be NULL.

> +       spin_lock_irqsave(&req->lock, flags);
> +       if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
> +               goto unlock;
> +       list_add_tail(&obj->list, &req->objects);
> +       req->num_incomplete_objects++;
> +unlock:
> +       spin_unlock_irqrestore(&req->lock, flags);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_bind);
> +
> +void media_request_object_unbind(struct media_request_object *obj)
> +{
> +       struct media_request *req = obj->req;
> +       unsigned long flags;
> +       bool completed = false;
> +
> +       if (!req)
> +               return;

No need for any locking here?

> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       list_del(&obj->list);
> +       obj->req = NULL;
> +
> +       if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
> +           req->state == MEDIA_REQUEST_STATE_CLEANING)
> +               goto unlock;
> +
> +       if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
> +               goto unlock;
> +
> +       if (WARN_ON(!req->num_incomplete_objects))
> +               goto unlock;
> +
> +       req->num_incomplete_objects--;
> +       if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
> +           !req->num_incomplete_objects) {
> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +               completed = true;
> +               wake_up_interruptible(&req->poll_wait);

Is this intentionally waking up only 1 waiter?

> +       }
> +unlock:
> +       spin_unlock_irqrestore(&req->lock, flags);
> +       if (obj->ops->unbind)
> +               obj->ops->unbind(obj);
> +       if (completed)
> +               media_request_put(req);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_unbind);
> +
> +void media_request_object_complete(struct media_request_object *obj)
> +{
> +       struct media_request *req = obj->req;
> +       unsigned long flags;
> +       bool completed = false;
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       if (obj->completed)
> +               goto unlock;
> +       obj->completed = true;
> +       if (WARN_ON(!req->num_incomplete_objects) ||
> +           WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
> +               goto unlock;
> +
> +       if (!--req->num_incomplete_objects) {
> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +               wake_up_interruptible(&req->poll_wait);

Is this intentionally waking up only 1 waiter?

> +               completed = true;
> +       }
> +unlock:
> +       spin_unlock_irqrestore(&req->lock, flags);
> +       if (completed)
> +               media_request_put(req);
>   }
> +EXPORT_SYMBOL_GPL(media_request_object_complete);
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index dae3eccd9aa7..082c3cae04ac 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -16,7 +16,163 @@

>   #include <media/media-device.h>

> +enum media_request_state {
> +       MEDIA_REQUEST_STATE_IDLE,
> +       MEDIA_REQUEST_STATE_QUEUEING,
> +       MEDIA_REQUEST_STATE_QUEUED,
> +       MEDIA_REQUEST_STATE_COMPLETE,
> +       MEDIA_REQUEST_STATE_CLEANING,
> +};
> +
> +struct media_request_object;
> +
> +/**
> + * struct media_request - Media device request
> + * @mdev: Media device this request belongs to
> + * @kref: Reference count
> + * @debug_prefix: Prefix for debug messages (process name:fd)

debug_str?

> + * @state: The state of the request
> + * @objects: List of @struct media_request_object request objects
> + * @num_objects: The number objects in the request
> + * @num_completed_objects: The number of completed objects in the request

num_incomplete_objects?

> + * @poll_wait: Wait queue for poll
> + * @lock: Serializes access to this struct

Does it have to be acquired to access any field?

> + */
> +struct media_request {
> +       struct media_device *mdev;
> +       struct kref kref;
> +       char debug_str[TASK_COMM_LEN + 11];
> +       enum media_request_state state;
> +       struct list_head objects;
> +       unsigned int num_incomplete_objects;
> +       struct wait_queue_head poll_wait;
> +       spinlock_t lock;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +
> +static inline void media_request_get(struct media_request *req)
> +{
> +       kref_get(&req->kref);
> +}
> +
> +void media_request_put(struct media_request *req);
> +void media_request_cancel(struct media_request *req);
> +
>   int media_request_alloc(struct media_device *mdev,
>                          struct media_request_alloc *alloc);
> +#else
> +static inline void media_request_get(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_put(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_cancel(struct media_request *req)
> +{
> +}
> +
> +#endif
> +

Documentation. (Disclaimer from cover letter acknowledged, though. Just
marking.)

> +struct media_request_object_ops {
> +       int (*prepare)(struct media_request_object *object);
> +       void (*unprepare)(struct media_request_object *object);
> +       void (*queue)(struct media_request_object *object);
> +       void (*unbind)(struct media_request_object *object);
> +       void (*cancel)(struct media_request_object *object);
> +       void (*release)(struct media_request_object *object);
> +};
> +
> +/**
> + * struct media_request_object - An opaque object that belongs to a media
> + *                              request
> + *

ops

> + * @priv: object's priv pointer

req

> + * @list: List entry of the object for @struct media_request
> + * @kref: Reference count of the object, acquire before releasing
req->lock

completed

> + *
> + * An object related to the request. This struct is embedded in the
> + * larger object data.
> + */
> +struct media_request_object {
> +       const struct media_request_object_ops *ops;
> +       void *priv;
> +       struct media_request *req;
> +       struct list_head list;
> +       struct kref kref;
> +       bool completed;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +static inline void media_request_object_get(struct media_request_object
*obj)
> +{
> +       kref_get(&obj->kref);
> +}
> +
> +/**
> + * media_request_object_put - Put a media request object
> + *
> + * @obj: The object
> + *
> + * Put a media request object. Once all references are gone, the
> + * object's memory is released.
> + */
> +void media_request_object_put(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_init - Initialise a media request object
> + *
> + * Initialise a media request object. The object will be released using
the
> + * release callback of the ops once it has no references (this function
> + * initialises references to one).
> + */
> +void media_request_object_init(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_bind - Bind a media request object to a request
> + */
> +void media_request_object_bind(struct media_request *req,
> +                              const struct media_request_object_ops *ops,
> +                              void *priv,
> +                              struct media_request_object *obj);
> +
> +void media_request_object_unbind(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_complete - Mark the media request object as
complete
> + */
> +void media_request_object_complete(struct media_request_object *obj);
> +#else
> +static inline void media_request_object_get(struct media_request_object
*obj)
> +{
> +}
> +
> +static inline void media_request_object_put(struct media_request_object
*obj)
> +{
> +}
> +
> +static inline void media_request_object_init(struct media_request_object
*obj)
> +{
> +       obj->ops = NULL;
> +       obj->req = NULL;

Hmm. Do we need to do anything here if !defined(CONFIG_MEDIA_CONTROLLER)?

Best regards,
Tomasz

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
> ioctls.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/media-request.c | 80
+++++++++++++++++++++++++++++++++++++++++--
>   1 file changed, 78 insertions(+), 2 deletions(-)

> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index dffc290e4ada..27739ff7cb09 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file
*filp,
>          return 0;
>   }

> +static long media_request_ioctl_queue(struct media_request *req)
> +{
> +       struct media_device *mdev = req->mdev;
> +       unsigned long flags;
> +       int ret = 0;
> +
> +       dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       if (req->state != MEDIA_REQUEST_STATE_IDLE) {
> +               dev_dbg(mdev->dev,
> +                       "request: unable to queue %s, request in state
%s\n",
> +                       req->debug_str,
media_request_state_str(req->state));
> +               spin_unlock_irqrestore(&req->lock, flags);
> +               return -EINVAL;

nit: Perhaps -EBUSY? (vb2 returns -EINVAL, though, but IMHO it doesn't
really represent the real error too closely.)

> +       }
> +       req->state = MEDIA_REQUEST_STATE_QUEUEING;
> +
> +       spin_unlock_irqrestore(&req->lock, flags);
> +
> +       /*
> +        * Ensure the request that is validated will be the one that gets
queued
> +        * next by serialising the queueing process.
> +        */
> +       mutex_lock(&mdev->req_queue_mutex);
> +
> +       ret = mdev->ops->req_queue(req);
> +       spin_lock_irqsave(&req->lock, flags);
> +       req->state = ret ? MEDIA_REQUEST_STATE_IDLE :
MEDIA_REQUEST_STATE_QUEUED;
> +       spin_unlock_irqrestore(&req->lock, flags);
> +       mutex_unlock(&mdev->req_queue_mutex);
> +
> +       if (ret) {
> +               dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
> +                       req->debug_str, ret);
> +       } else {
> +               media_request_get(req);

I'm not convinced that this is the right place to take a reference. IMHO
whoever saves a pointer to the request in its own internal data (the
->req_queue() callback?), should also grab a reference before doing so. Not
a strong objection, though, if we clearly document this, so that whoever
implements ->req_queue() callback can do the right thing.

> +       }
> +
> +       return ret;
> +}
> +
> +static long media_request_ioctl_reinit(struct media_request *req)
> +{
> +       struct media_device *mdev = req->mdev;
> +       unsigned long flags;
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       if (req->state != MEDIA_REQUEST_STATE_IDLE &&
> +           req->state != MEDIA_REQUEST_STATE_COMPLETE) {
> +               dev_dbg(mdev->dev,
> +                       "request: %s not in idle or complete state,
cannot reinit\n",
> +                       req->debug_str);
> +               spin_unlock_irqrestore(&req->lock, flags);
> +               return -EINVAL;

nit: Perhaps -EBUSY? (Again vb2 would return -EINVAL...)

Best regards,
Tomasz

Re: [RFCv11 PATCH 12/29] v4l2-ctrls: alloc memory for p_req

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> To store request data the handler_new_ref() allocates memory
> for it if needed.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/v4l2-core/v4l2-ctrls.c | 20 ++++++++++++++++----
>   1 file changed, 16 insertions(+), 4 deletions(-)

> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
b/drivers/media/v4l2-core/v4l2-ctrls.c
> index d09f49530d9e..3c1b00baa8d0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -1997,13 +1997,18 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
>   /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
>   static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
>                             struct v4l2_ctrl *ctrl,
> -                          bool from_other_dev)
> +                          struct v4l2_ctrl_ref **ctrl_ref,
> +                          bool from_other_dev, bool allocate_req)
>   {
>          struct v4l2_ctrl_ref *ref;
>          struct v4l2_ctrl_ref *new_ref;
>          u32 id = ctrl->id;
>          u32 class_ctrl = V4L2_CTRL_ID2WHICH(id) | 1;
>          int bucket = id % hdl->nr_of_buckets;   /* which bucket to use */
> +       unsigned int sz_extra = 0;
> +
> +       if (ctrl_ref)
> +               *ctrl_ref = NULL;

>          /*
>           * Automatically add the control class if it is not yet present
and
> @@ -2017,11 +2022,16 @@ static int handler_new_ref(struct
v4l2_ctrl_handler *hdl,
>          if (hdl->error)
>                  return hdl->error;

> -       new_ref = kzalloc(sizeof(*new_ref), GFP_KERNEL);
> +       if (allocate_req)
> +               sz_extra = ctrl->elems * ctrl->elem_size;
> +       new_ref = kzalloc(sizeof(*new_ref) + sz_extra, GFP_KERNEL);

I think we might want to use kvzalloc() here to cover allocation of big
elements and/or large arrays, without order>0 allocations killing the
systems.

I'm actually also wondering if it wouldn't be cleaner to just allocate the
extra data separately, since we store a pointer in new_ref->p_req.p anyway.

Best regards,
Tomasz

Re: [RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:19:59 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Define the public request API.
> 
> This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
> and two ioctls that operate on a request in order to queue the
> contents of the request to the driver and to re-initialize the
> request.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  include/uapi/linux/media.h | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index c7e9a5cba24e..f8769e74f847 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -342,11 +342,19 @@ struct media_v2_topology {
>  
>  /* ioctls */
>  
> +struct __attribute__ ((packed)) media_request_alloc {
> +	__s32 fd;
> +};
> +
>  #define MEDIA_IOC_DEVICE_INFO	_IOWR('|', 0x00, struct media_device_info)
>  #define MEDIA_IOC_ENUM_ENTITIES	_IOWR('|', 0x01, struct media_entity_desc)
>  #define MEDIA_IOC_ENUM_LINKS	_IOWR('|', 0x02, struct media_links_enum)
>  #define MEDIA_IOC_SETUP_LINK	_IOWR('|', 0x03, struct media_link_desc)
>  #define MEDIA_IOC_G_TOPOLOGY	_IOWR('|', 0x04, struct media_v2_topology)
> +#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, struct media_request_alloc)
> +

Why use a struct here? Just declare it as:

	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)

> +#define MEDIA_REQUEST_IOC_QUEUE		_IO('|',  0x80)
> +#define MEDIA_REQUEST_IOC_REINIT	_IO('|',  0x81)
>  
>  #if !defined(__KERNEL__) || defined(__NEED_MEDIA_LEGACY_API)
>  

Thanks,
Mauro

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:00 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add support for allocating a new request. This is only supported
> if mdev->ops->req_queue is set, i.e. the driver indicates that it
> supports queueing requests.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/Makefile        |  3 ++-
>  drivers/media/media-device.c  | 14 ++++++++++++++
>  drivers/media/media-request.c | 23 +++++++++++++++++++++++
>  include/media/media-device.h  | 13 +++++++++++++
>  include/media/media-request.h | 22 ++++++++++++++++++++++
>  5 files changed, 74 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/media/media-request.c
>  create mode 100644 include/media/media-request.h
> 
> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> index 594b462ddf0e..985d35ec6b29 100644
> --- a/drivers/media/Makefile
> +++ b/drivers/media/Makefile
> @@ -3,7 +3,8 @@
>  # Makefile for the kernel multimedia device drivers.
>  #
>  
> -media-objs	:= media-device.o media-devnode.o media-entity.o
> +media-objs	:= media-device.o media-devnode.o media-entity.o \
> +		   media-request.o
>  
>  #
>  # I2C drivers should come before other drivers, otherwise they'll fail
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index 35e81f7c0d2f..acb583c0eacd 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -32,6 +32,7 @@
>  #include <media/media-device.h>
>  #include <media/media-devnode.h>
>  #include <media/media-entity.h>
> +#include <media/media-request.h>
>  
>  #ifdef CONFIG_MEDIA_CONTROLLER
>  
> @@ -366,6 +367,15 @@ static long media_device_get_topology(struct media_device *mdev,
>  	return ret;
>  }
>  
> +static long media_device_request_alloc(struct media_device *mdev,
> +				       struct media_request_alloc *alloc)
> +{
> +	if (!mdev->ops || !mdev->ops->req_queue)
> +		return -ENOTTY;
> +
> +	return media_request_alloc(mdev, alloc);
> +}
> +
>  static long copy_arg_from_user(void *karg, void __user *uarg, unsigned int cmd)
>  {
>  	/* All media IOCTLs are _IOWR() */
> @@ -414,6 +424,7 @@ static const struct media_ioctl_info ioctl_info[] = {
>  	MEDIA_IOC(ENUM_LINKS, media_device_enum_links, MEDIA_IOC_FL_GRAPH_MUTEX),
>  	MEDIA_IOC(SETUP_LINK, media_device_setup_link, MEDIA_IOC_FL_GRAPH_MUTEX),
>  	MEDIA_IOC(G_TOPOLOGY, media_device_get_topology, MEDIA_IOC_FL_GRAPH_MUTEX),
> +	MEDIA_IOC(REQUEST_ALLOC, media_device_request_alloc, 0),
>  };
>  
>  static long media_device_ioctl(struct file *filp, unsigned int cmd,
> @@ -686,6 +697,9 @@ void media_device_init(struct media_device *mdev)
>  	INIT_LIST_HEAD(&mdev->pads);
>  	INIT_LIST_HEAD(&mdev->links);
>  	INIT_LIST_HEAD(&mdev->entity_notify);
> +
> +	spin_lock_init(&mdev->req_lock);
> +	mutex_init(&mdev->req_queue_mutex);
>  	mutex_init(&mdev->graph_mutex);
>  	ida_init(&mdev->entity_internal_idx);
>  
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> new file mode 100644
> index 000000000000..ead78613fdbe
> --- /dev/null
> +++ b/drivers/media/media-request.c
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Media device request objects
> + *
> + * Copyright (C) 2018 Intel Corporation
> + * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
> + *
> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> + */
> +
> +#include <linux/anon_inodes.h>

Not needed. You already included it at media_device.h.

> +#include <linux/file.h>
> +#include <linux/mm.h>
> +#include <linux/string.h>

Why do you need so many includes for a stub function?

> +
> +#include <media/media-device.h>
> +#include <media/media-request.h>
> +
> +int media_request_alloc(struct media_device *mdev,
> +			struct media_request_alloc *alloc)
> +{
> +	return -ENOMEM;
> +}
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index bcc6ec434f1f..07e323c57202 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -19,6 +19,7 @@
>  #ifndef _MEDIA_DEVICE_H
>  #define _MEDIA_DEVICE_H
>  
> +#include <linux/anon_inodes.h>

Why do you need it? I don't see anything below needing it.

>  #include <linux/list.h>
>  #include <linux/mutex.h>
>  
> @@ -27,6 +28,7 @@
>  
>  struct ida;
>  struct device;
> +struct media_device;
>  
>  /**
>   * struct media_entity_notify - Media Entity Notify
> @@ -50,10 +52,16 @@ struct media_entity_notify {
>   * struct media_device_ops - Media device operations
>   * @link_notify: Link state change notification callback. This callback is
>   *		 called with the graph_mutex held.
> + * @req_alloc: Allocate a request
> + * @req_free: Free a request
> + * @req_queue: Queue a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
>  			   unsigned int notification);
> +	struct media_request *(*req_alloc)(struct media_device *mdev);
> +	void (*req_free)(struct media_request *req);
> +	int (*req_queue)(struct media_request *req);
>  };
>  
>  /**
> @@ -88,6 +96,8 @@ struct media_device_ops {
>   * @disable_source: Disable Source Handler function pointer
>   *
>   * @ops:	Operation handler callbacks
> + * @req_lock:	Serialise access to requests
> + * @req_queue_mutex: Serialise validating and queueing requests

IMHO, this would better describe it:
	Serialise validate and queue requests

Yet, IMO, it doesn't let it clear when the spin lock should be
used and when the mutex should be used.

I mean, what of them protect what variable?

>   *
>   * This structure represents an abstract high-level media device. It allows easy
>   * access to entities and provides basic media device-level support. The
> @@ -158,6 +168,9 @@ struct media_device {
>  	void (*disable_source)(struct media_entity *entity);
>  
>  	const struct media_device_ops *ops;
> +
> +	spinlock_t req_lock;
> +	struct mutex req_queue_mutex;
>  };
>  
>  /* We don't need to include pci.h or usb.h here */
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> new file mode 100644
> index 000000000000..dae3eccd9aa7
> --- /dev/null
> +++ b/include/media/media-request.h
> @@ -0,0 +1,22 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Media device request objects
> + *
> + * Copyright (C) 2018 Intel Corporation
> + *
> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> + */
> +
> +#ifndef MEDIA_REQUEST_H
> +#define MEDIA_REQUEST_H
> +
> +#include <linux/list.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>

Why those includes are needed?

> +
> +#include <media/media-device.h>
> +
> +int media_request_alloc(struct media_device *mdev,
> +			struct media_request_alloc *alloc);
> +
> +#endif



Thanks,
Mauro

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Mauro Carvalho Chehab]

Em Tue, 10 Apr 2018 05:35:37 +0000
Tomasz Figa <tfiga@google.com> escreveu:

> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> [snip]
> > diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> > new file mode 100644
> > index 000000000000..ead78613fdbe
> > --- /dev/null
> > +++ b/drivers/media/media-request.c
> > @@ -0,0 +1,23 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Media device request objects
> > + *
> > + * Copyright (C) 2018 Intel Corporation
> > + * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.  
> 
> I'm not sure about the origin of this line, but it's not a correct
> copyright for kernel code produced as a part of Chrome OS project. It would
> normally be something like
> 
> Copyright (C) 2018 Google, Inc.

Also, it sounds a lot of copyright for a file with just stub :-)

> 
> > + *
> > + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> > + */
> > +
> > +#include <linux/anon_inodes.h>
> > +#include <linux/file.h>
> > +#include <linux/mm.h>
> > +#include <linux/string.h>
> > +
> > +#include <media/media-device.h>
> > +#include <media/media-request.h>
> > +
> > +int media_request_alloc(struct media_device *mdev,
> > +                       struct media_request_alloc *alloc)
> > +{
> > +       return -ENOMEM;
> > +}
> > diff --git a/include/media/media-device.h b/include/media/media-device.h
> > index bcc6ec434f1f..07e323c57202 100644
> > --- a/include/media/media-device.h
> > +++ b/include/media/media-device.h
> > @@ -19,6 +19,7 @@
> >   #ifndef _MEDIA_DEVICE_H
> >   #define _MEDIA_DEVICE_H  
> 
> > +#include <linux/anon_inodes.h>  
> 
> What is the need for anon_inodes in this header?
> 
> >   #include <linux/list.h>
> >   #include <linux/mutex.h>  
> 
> > @@ -27,6 +28,7 @@  
> 
> >   struct ida;
> >   struct device;
> > +struct media_device;  
> 
> >   /**
> >    * struct media_entity_notify - Media Entity Notify
> > @@ -50,10 +52,16 @@ struct media_entity_notify {
> >    * struct media_device_ops - Media device operations
> >    * @link_notify: Link state change notification callback. This callback  
> is
> >    *              called with the graph_mutex held.
> > + * @req_alloc: Allocate a request
> > + * @req_free: Free a request
> > + * @req_queue: Queue a request
> >    */
> >   struct media_device_ops {
> >          int (*link_notify)(struct media_link *link, u32 flags,
> >                             unsigned int notification);
> > +       struct media_request *(*req_alloc)(struct media_device *mdev);
> > +       void (*req_free)(struct media_request *req);
> > +       int (*req_queue)(struct media_request *req);
> >   };  
> 
> >   /**
> > @@ -88,6 +96,8 @@ struct media_device_ops {
> >    * @disable_source: Disable Source Handler function pointer
> >    *
> >    * @ops:       Operation handler callbacks
> > + * @req_lock:  Serialise access to requests
> > + * @req_queue_mutex: Serialise validating and queueing requests  
> 
> Let's bikeshed a bit! "access" sounds like a superset of "validating and
> queuing" to me. Perhaps it could make sense to be a bit more specific on
> what type of access the spinlock is used for?
> 
> Best regards,
> Tomasz



Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:01 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Implement the core of the media request processing.
> 
> Drivers can bind request objects to a request. These objects
> can then be marked completed if the driver finished using them,
> or just be unbound if the results do not need to be kept (e.g.
> in the case of buffers).
> 
> Once all objects that were added are either unbound or completed,
> the request is marked 'complete' and a POLLPRI signal is sent
> via poll.
> 
> Both requests and request objects are refcounted.
> 
> While a request is queued its refcount is incremented (since it
> is in use by a driver). Once it is completed the refcount is
> decremented. When the user closes the request file descriptor
> the refcount is also decremented. Once it reaches 0 all request
> objects in the request are unbound and put() and the request
> itself is freed.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 284 +++++++++++++++++++++++++++++++++++++++++-
>  include/media/media-request.h | 156 +++++++++++++++++++++++
>  2 files changed, 439 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index ead78613fdbe..dffc290e4ada 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -16,8 +16,290 @@
>  #include <media/media-device.h>
>  #include <media/media-request.h>
>  
> +static const char * const request_state[] = {
> +	"idle",
> +	"queueing",
> +	"queued",
> +	"complete",
> +	"cleaning",
> +};

Please use c99 designated inits here, e. g.:

static const char * const request_state[] = {
	[MEDIA_REQUEST_STATE_IDLE] = "idle",
	...
};

> +
> +static const char *
> +media_request_state_str(enum media_request_state state)
> +{
> +	if (WARN_ON(state >= ARRAY_SIZE(request_state)))
> +		return "unknown";
> +	return request_state[state];
> +}
> +
> +static void media_request_clean(struct media_request *req)
> +{
> +	struct media_request_object *obj, *obj_safe;
> +
> +	WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
> +
> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> +		media_request_object_unbind(obj);
> +		media_request_object_put(obj);
> +	}
> +
> +	req->num_incomplete_objects = 0;
> +	wake_up_interruptible(&req->poll_wait);
> +}
> +
> +static void media_request_release(struct kref *kref)
> +{
> +	struct media_request *req =
> +		container_of(kref, struct media_request, kref);
> +	struct media_device *mdev = req->mdev;
> +	unsigned long flags;
> +
> +	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> +	spin_unlock_irqrestore(&req->lock, flags);
> +
> +	media_request_clean(req);
> +
> +	if (mdev->ops->req_free)
> +		mdev->ops->req_free(req);
> +	else
> +		kfree(req);

Without looking at req_free() implementation, I would actually prefer
to always do a kfree(req) here. So, a req_free() function would only
free "extra" allocations, and not the request itself. e. g.:

...
	if (mdev->ops->req_free)
		mdev->ops->req_free(req);

	kfree(req);
}


> +}
> +
> +void media_request_put(struct media_request *req)
> +{
> +	kref_put(&req->kref, media_request_release);
> +}
> +EXPORT_SYMBOL_GPL(media_request_put);
> +
> +void media_request_cancel(struct media_request *req)
> +{
> +	struct media_request_object *obj, *obj_safe;
> +
> +	if (req->state != MEDIA_REQUEST_STATE_QUEUED)
> +		return;
> +
> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
> +		if (obj->ops->cancel)
> +			obj->ops->cancel(obj);
> +}
> +EXPORT_SYMBOL_GPL(media_request_cancel);
> +
> +static int media_request_close(struct inode *inode, struct file *filp)
> +{
> +	struct media_request *req = filp->private_data;
> +
> +	media_request_put(req);
> +	return 0;
> +}
> +
> +static unsigned int media_request_poll(struct file *filp,
> +				       struct poll_table_struct *wait)
> +{
> +	struct media_request *req = filp->private_data;
> +	unsigned long flags;
> +	enum media_request_state state;
> +
> +	if (!(poll_requested_events(wait) & POLLPRI))
> +		return 0;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	state = req->state;
> +	spin_unlock_irqrestore(&req->lock, flags);

IMO, it would be better to use an atomic var for state, having a
lockless access to it.

> +
> +	if (state == MEDIA_REQUEST_STATE_COMPLETE)
> +		return POLLPRI;
> +	if (state == MEDIA_REQUEST_STATE_IDLE)
> +		return POLLERR;
> +
> +	poll_wait(filp, &req->poll_wait, wait);
> +	return 0;
> +}
> +
> +static long media_request_ioctl(struct file *filp, unsigned int cmd,
> +				unsigned long __arg)
> +{
> +	return -ENOIOCTLCMD;
> +}
> +
> +static const struct file_operations request_fops = {
> +	.owner = THIS_MODULE,
> +	.poll = media_request_poll,
> +	.unlocked_ioctl = media_request_ioctl,
> +	.release = media_request_close,
> +};
> +
>  int media_request_alloc(struct media_device *mdev,
>  			struct media_request_alloc *alloc)
>  {
> -	return -ENOMEM;
> +	struct media_request *req;
> +	struct file *filp;
> +	char comm[TASK_COMM_LEN];
> +	int fd;
> +	int ret;
> +
> +	fd = get_unused_fd_flags(O_CLOEXEC);
> +	if (fd < 0)
> +		return fd;
> +
> +	filp = anon_inode_getfile("request", &request_fops, NULL, O_CLOEXEC);
> +	if (IS_ERR(filp)) {
> +		ret = PTR_ERR(filp);
> +		goto err_put_fd;
> +	}
> +
> +	if (mdev->ops->req_alloc)
> +		req = mdev->ops->req_alloc(mdev);
> +	else
> +		req = kzalloc(sizeof(*req), GFP_KERNEL);
> +	if (!req) {
> +		ret = -ENOMEM;
> +		goto err_fput;
> +	}
> +
> +	filp->private_data = req;
> +	req->mdev = mdev;
> +	req->state = MEDIA_REQUEST_STATE_IDLE;
> +	req->num_incomplete_objects = 0;
> +	kref_init(&req->kref);
> +	INIT_LIST_HEAD(&req->objects);
> +	spin_lock_init(&req->lock);
> +	init_waitqueue_head(&req->poll_wait);
> +
> +	alloc->fd = fd;

Btw, this is a very good reason why you should define the ioctl to
have an integer argument instead of a struct with a __s32 field
on it, as per my comment to patch 02/29:

	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)

At 64 bit architectures, you're truncating the file descriptor!

> +	get_task_comm(comm, current);
> +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> +		 comm, fd);

Not sure if it is a good idea to store the task that allocated
the request. While it makes sense for the dev_dbg() below, it
may not make sense anymore on other dev_dbg() you would be
using it.

> +	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
> +
> +	fd_install(fd, filp);
> +
> +	return 0;
> +
> +err_fput:
> +	fput(filp);
> +
> +err_put_fd:
> +	put_unused_fd(fd);
> +
> +	return ret;
> +}
> +
> +static void media_request_object_release(struct kref *kref)
> +{
> +	struct media_request_object *obj =
> +		container_of(kref, struct media_request_object, kref);
> +	struct media_request *req = obj->req;
> +
> +	if (req)
> +		media_request_object_unbind(obj);
> +	obj->ops->release(obj);
> +}
> +
> +void media_request_object_put(struct media_request_object *obj)
> +{
> +	kref_put(&obj->kref, media_request_object_release);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_put);
> +
> +void media_request_object_init(struct media_request_object *obj)
> +{
> +	obj->ops = NULL;
> +	obj->req = NULL;
> +	obj->priv = NULL;
> +	obj->completed = false;
> +	INIT_LIST_HEAD(&obj->list);
> +	kref_init(&obj->kref);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_init);
> +
> +void media_request_object_bind(struct media_request *req,
> +			       const struct media_request_object_ops *ops,
> +			       void *priv,
> +			       struct media_request_object *obj)
> +{
> +	unsigned long flags;
> +
> +	if (WARN_ON(!ops->release || !ops->cancel))
> +		return;
> +
> +	obj->req = req;
> +	obj->ops = ops;
> +	obj->priv = priv;
> +	spin_lock_irqsave(&req->lock, flags);
> +	if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
> +		goto unlock;
> +	list_add_tail(&obj->list, &req->objects);
> +	req->num_incomplete_objects++;
> +unlock:
> +	spin_unlock_irqrestore(&req->lock, flags);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_bind);
> +
> +void media_request_object_unbind(struct media_request_object *obj)
> +{
> +	struct media_request *req = obj->req;
> +	unsigned long flags;
> +	bool completed = false;
> +
> +	if (!req)
> +		return;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	list_del(&obj->list);
> +	obj->req = NULL;
> +
> +	if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
> +	    req->state == MEDIA_REQUEST_STATE_CLEANING)
> +		goto unlock;
> +
> +	if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
> +		goto unlock;
> +
> +	if (WARN_ON(!req->num_incomplete_objects))
> +		goto unlock;
> +
> +	req->num_incomplete_objects--;
> +	if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
> +	    !req->num_incomplete_objects) {
> +		req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +		completed = true;
> +		wake_up_interruptible(&req->poll_wait);
> +	}
> +unlock:
> +	spin_unlock_irqrestore(&req->lock, flags);
> +	if (obj->ops->unbind)
> +		obj->ops->unbind(obj);
> +	if (completed)
> +		media_request_put(req);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_unbind);
> +
> +void media_request_object_complete(struct media_request_object *obj)
> +{
> +	struct media_request *req = obj->req;
> +	unsigned long flags;
> +	bool completed = false;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	if (obj->completed)
> +		goto unlock;
> +	obj->completed = true;
> +	if (WARN_ON(!req->num_incomplete_objects) ||
> +	    WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
> +		goto unlock;
> +
> +	if (!--req->num_incomplete_objects) {
> +		req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +		wake_up_interruptible(&req->poll_wait);
> +		completed = true;
> +	}
> +unlock:
> +	spin_unlock_irqrestore(&req->lock, flags);
> +	if (completed)
> +		media_request_put(req);
>  }
> +EXPORT_SYMBOL_GPL(media_request_object_complete);
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index dae3eccd9aa7..082c3cae04ac 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -16,7 +16,163 @@
>  
>  #include <media/media-device.h>
>  
> +enum media_request_state {
> +	MEDIA_REQUEST_STATE_IDLE,
> +	MEDIA_REQUEST_STATE_QUEUEING,
> +	MEDIA_REQUEST_STATE_QUEUED,
> +	MEDIA_REQUEST_STATE_COMPLETE,
> +	MEDIA_REQUEST_STATE_CLEANING,
> +};
> +
> +struct media_request_object;
> +
> +/**
> + * struct media_request - Media device request
> + * @mdev: Media device this request belongs to
> + * @kref: Reference count
> + * @debug_prefix: Prefix for debug messages (process name:fd)
> + * @state: The state of the request
> + * @objects: List of @struct media_request_object request objects
> + * @num_objects: The number objects in the request
> + * @num_completed_objects: The number of completed objects in the request

This doesn't match any var...

> + * @poll_wait: Wait queue for poll
> + * @lock: Serializes access to this struct
> + */
> +struct media_request {
> +	struct media_device *mdev;
> +	struct kref kref;
> +	char debug_str[TASK_COMM_LEN + 11];
> +	enum media_request_state state;
> +	struct list_head objects;
> +	unsigned int num_incomplete_objects;

... as it seems that you decide to count incomplete objects, instead
of complete ones.

> +	struct wait_queue_head poll_wait;
> +	spinlock_t lock;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +
> +static inline void media_request_get(struct media_request *req)
> +{
> +	kref_get(&req->kref);
> +}
> +
> +void media_request_put(struct media_request *req);
> +void media_request_cancel(struct media_request *req);
> +
>  int media_request_alloc(struct media_device *mdev,
>  			struct media_request_alloc *alloc);
> +#else
> +static inline void media_request_get(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_put(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_cancel(struct media_request *req)
> +{
> +}
> +
> +#endif
> +
> +struct media_request_object_ops {
> +	int (*prepare)(struct media_request_object *object);
> +	void (*unprepare)(struct media_request_object *object);
> +	void (*queue)(struct media_request_object *object);
> +	void (*unbind)(struct media_request_object *object);
> +	void (*cancel)(struct media_request_object *object);
> +	void (*release)(struct media_request_object *object);
> +};

-ENODESCRIPTION. Please describe the struct and its fields.

> +
> +/**
> + * struct media_request_object - An opaque object that belongs to a media
> + *				 request
> + *
> + * @priv: object's priv pointer
> + * @list: List entry of the object for @struct media_request
> + * @kref: Reference count of the object, acquire before releasing req->lock

Field descriptions missing.

> + *
> + * An object related to the request. This struct is embedded in the
> + * larger object data.
> + */
> +struct media_request_object {
> +	const struct media_request_object_ops *ops;
> +	void *priv;
> +	struct media_request *req;
> +	struct list_head list;
> +	struct kref kref;
> +	bool completed;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +static inline void media_request_object_get(struct media_request_object *obj)
> +{
> +	kref_get(&obj->kref);
> +}

Why do you need a function? Just use kref_get() were needed, instead of
aliasing it for no good reason.

> +
> +/**
> + * media_request_object_put - Put a media request object
> + *
> + * @obj: The object
> + *
> + * Put a media request object. Once all references are gone, the
> + * object's memory is released.
> + */
> +void media_request_object_put(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_init - Initialise a media request object
> + *
> + * Initialise a media request object. The object will be released using the
> + * release callback of the ops once it has no references (this function
> + * initialises references to one).
> + */
> +void media_request_object_init(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_bind - Bind a media request object to a request
> + */
> +void media_request_object_bind(struct media_request *req,
> +			       const struct media_request_object_ops *ops,
> +			       void *priv,
> +			       struct media_request_object *obj);
> +
> +void media_request_object_unbind(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_complete - Mark the media request object as complete
> + */
> +void media_request_object_complete(struct media_request_object *obj);

The kernel-doc tags there are wrong: They don't describe the function
arguments.

Please validate it with sphinx to be sure that everything is ok.

> +#else
> +static inline void media_request_object_get(struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_put(struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_init(struct media_request_object *obj)
> +{
> +	obj->ops = NULL;
> +	obj->req = NULL;
> +}
> +
> +static inline void media_request_object_bind(struct media_request *req,
> +			       const struct media_request_object_ops *ops,
> +			       void *priv,
> +			       struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_unbind(struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_complete(struct media_request_object *obj)
> +{
> +}
> +#endif
>  
>  #endif



Thanks,
Mauro

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:02 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
> ioctls.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 80 +++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 78 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index dffc290e4ada..27739ff7cb09 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file *filp,
>  	return 0;
>  }
>  
> +static long media_request_ioctl_queue(struct media_request *req)
> +{
> +	struct media_device *mdev = req->mdev;
> +	unsigned long flags;
> +	int ret = 0;
> +
> +	dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
> +		dev_dbg(mdev->dev,
> +			"request: unable to queue %s, request in state %s\n",
> +			req->debug_str, media_request_state_str(req->state));
> +		spin_unlock_irqrestore(&req->lock, flags);
> +		return -EINVAL;
> +	}
> +	req->state = MEDIA_REQUEST_STATE_QUEUEING;
> +
> +	spin_unlock_irqrestore(&req->lock, flags);
> +
> +	/*
> +	 * Ensure the request that is validated will be the one that gets queued
> +	 * next by serialising the queueing process.
> +	 */
> +	mutex_lock(&mdev->req_queue_mutex);

The locking here seems really weird. IMHO, it should lock before
touching state, as otherwise race conditions may happen.

As I suggested before, I would use an atomic type for state, and get rid
of the spin lock (as it seems that it is meant to be used just
for state).

> +
> +	ret = mdev->ops->req_queue(req);
> +	spin_lock_irqsave(&req->lock, flags);
> +	req->state = ret ? MEDIA_REQUEST_STATE_IDLE : MEDIA_REQUEST_STATE_QUEUED;
> +	spin_unlock_irqrestore(&req->lock, flags);
> +	mutex_unlock(&mdev->req_queue_mutex);
> +

Here, you have both mutex and spin locked. This is a strong indication
that locks are not well designed, are you're using two different locks
to protect the same data.

> +	if (ret) {
> +		dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
> +			req->debug_str, ret);
> +	} else {
> +		media_request_get(req);
> +	}
> +
> +	return ret;
> +}

IMHO, the above construction hides the main code inside an if due
to an error condition. Doing this makes it clearer that, under normal
circumstances, you're doing a kref_get() call:

	if (ret) {
		dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
			req->debug_str, ret);
		return ret;
	}

	kref_get(&req->kref);	// This is a way more easier to read than media_request_get(req);
	return 0;
}

Another related issue: IMHO, kref_get() should be called here with the
mutex hold. 

> +static long media_request_ioctl_reinit(struct media_request *req)
> +{
> +	struct media_device *mdev = req->mdev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	if (req->state != MEDIA_REQUEST_STATE_IDLE &&
> +	    req->state != MEDIA_REQUEST_STATE_COMPLETE) {
> +		dev_dbg(mdev->dev,
> +			"request: %s not in idle or complete state, cannot reinit\n",
> +			req->debug_str);
> +		spin_unlock_irqrestore(&req->lock, flags);
> +		return -EINVAL;
> +	}
> +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> +	spin_unlock_irqrestore(&req->lock, flags);
> +
> +	media_request_clean(req);
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	req->state = MEDIA_REQUEST_STATE_IDLE;
> +	spin_unlock_irqrestore(&req->lock, flags);

This code should be called with the mutex hold.

> +	return 0;
> +}
> +
>  static long media_request_ioctl(struct file *filp, unsigned int cmd,
> -				unsigned long __arg)
> +				unsigned long arg)
>  {
> -	return -ENOIOCTLCMD;
> +	struct media_request *req = filp->private_data;
> +
> +	switch (cmd) {
> +	case MEDIA_REQUEST_IOC_QUEUE:
> +		return media_request_ioctl_queue(req);
> +	case MEDIA_REQUEST_IOC_REINIT:
> +		return media_request_ioctl_reinit(req);
> +	default:
> +		return -ENOIOCTLCMD;
> +	}
>  }
>  
>  static const struct file_operations request_fops = {



Thanks,
Mauro

Re: [RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Sakari Ailus]

On Tue, Apr 10, 2018 at 06:38:56AM -0300, Mauro Carvalho Chehab wrote:
> Em Mon,  9 Apr 2018 16:19:59 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> 
> > From: Hans Verkuil <hans.verkuil@cisco.com>
> > 
> > Define the public request API.
> > 
> > This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
> > and two ioctls that operate on a request in order to queue the
> > contents of the request to the driver and to re-initialize the
> > request.
> > 
> > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  include/uapi/linux/media.h | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> > 
> > diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> > index c7e9a5cba24e..f8769e74f847 100644
> > --- a/include/uapi/linux/media.h
> > +++ b/include/uapi/linux/media.h
> > @@ -342,11 +342,19 @@ struct media_v2_topology {
> >  
> >  /* ioctls */
> >  
> > +struct __attribute__ ((packed)) media_request_alloc {
> > +	__s32 fd;
> > +};
> > +
> >  #define MEDIA_IOC_DEVICE_INFO	_IOWR('|', 0x00, struct media_device_info)
> >  #define MEDIA_IOC_ENUM_ENTITIES	_IOWR('|', 0x01, struct media_entity_desc)
> >  #define MEDIA_IOC_ENUM_LINKS	_IOWR('|', 0x02, struct media_links_enum)
> >  #define MEDIA_IOC_SETUP_LINK	_IOWR('|', 0x03, struct media_link_desc)
> >  #define MEDIA_IOC_G_TOPOLOGY	_IOWR('|', 0x04, struct media_v2_topology)
> > +#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, struct media_request_alloc)
> > +
> 
> Why use a struct here? Just declare it as:
> 
> 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)

I'd say it's easier to extend it if it's a struct. All other IOCTLs also
have a struct as an argument. As a struct member, the parameter (fd) also
has a name; this is a plus.

> 
> > +#define MEDIA_REQUEST_IOC_QUEUE		_IO('|',  0x80)
> > +#define MEDIA_REQUEST_IOC_REINIT	_IO('|',  0x81)
> >  
> >  #if !defined(__KERNEL__) || defined(__NEED_MEDIA_LEGACY_API)
> >  
> 
> Thanks,
> Mauro

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 06/29] media-request: add media_request_find

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:03 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add media_request_find() to find a request based on the file
> descriptor.
> 
> The caller has to call media_request_put() for the returned
> request since this function increments the refcount.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 47 +++++++++++++++++++++++++++++++++++++++++++
>  include/media/media-request.h | 10 +++++++++
>  2 files changed, 57 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 27739ff7cb09..02b620c81de5 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -207,6 +207,53 @@ static const struct file_operations request_fops = {
>  	.release = media_request_close,
>  };
>  
> +/**
> + * media_request_find - Find a request based on the file descriptor
> + * @mdev: The media device
> + * @request: The request file handle

request_id.

> + *
> + * Find and return the request associated with the given file descriptor, or
> + * an error if no such request exists.
> + *
> + * When the function returns a request it increases its reference count. The
> + * caller is responsible for releasing the reference by calling
> + * media_request_put() on the request.

IMHO, this can only be called with mutex held. Please add such note
here.

> + */
> +struct media_request *
> +media_request_find(struct media_device *mdev, int request_fd)
> +{
> +	struct file *filp;
> +	struct media_request *req;
> +
> +	if (!mdev || !mdev->ops || !mdev->ops->req_queue)
> +		return ERR_PTR(-EPERM);
> +
> +	filp = fget(request_fd);
> +	if (!filp)
> +		return ERR_PTR(-ENOENT);
> +
> +	if (filp->f_op != &request_fops)
> +		goto err_fput;
> +	req = filp->private_data;
> +	media_request_get(req);
> +
> +	if (req->mdev != mdev)
> +		goto err_kref_put;
> +
> +	fput(filp);
> +
> +	return req;
> +
> +err_kref_put:
> +	media_request_put(req);
> +
> +err_fput:
> +	fput(filp);
> +
> +	return ERR_PTR(-ENOENT);
> +}
> +EXPORT_SYMBOL_GPL(media_request_find);
> +
>  int media_request_alloc(struct media_device *mdev,
>  			struct media_request_alloc *alloc)
>  {
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index 082c3cae04ac..033697d493cd 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -59,6 +59,9 @@ static inline void media_request_get(struct media_request *req)
>  void media_request_put(struct media_request *req);
>  void media_request_cancel(struct media_request *req);
>  
> +struct media_request *
> +media_request_find(struct media_device *mdev, int request_fd);
> +
>  int media_request_alloc(struct media_device *mdev,
>  			struct media_request_alloc *alloc);
>  #else
> @@ -74,6 +77,12 @@ static inline void media_request_cancel(struct media_request *req)
>  {
>  }
>  
> +static inline struct media_request *
> +media_request_find(struct media_device *mdev, int request_fd)
> +{
> +	return ERR_PTR(-ENOENT);
> +}
> +
>  #endif
>  
>  struct media_request_object_ops {
> @@ -173,6 +182,7 @@ static inline void media_request_object_unbind(struct media_request_object *obj)
>  static inline void media_request_object_complete(struct media_request_object *obj)
>  {
>  }
> +
>  #endif
>  
>  #endif



Thanks,
Mauro

Re: [RFCv11 PATCH 07/29] media-request: add media_request_object_find

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:04 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add media_request_object_find to find a request object inside a
> request based on ops and/or priv values.
> 
> Objects of the same type (vb2 buffer, control handler) will have
> the same ops value. And objects that refer to the same 'parent'
> object (e.g. the v4l2_ctrl_handler that has the current driver
> state) will have the same priv value.
> 
> The caller has to call media_request_object_put() for the returned
> object since this function increments the refcount.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 26 ++++++++++++++++++++++++++
>  include/media/media-request.h | 25 +++++++++++++++++++++++++
>  2 files changed, 51 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 02b620c81de5..415f7e31019d 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -322,6 +322,32 @@ static void media_request_object_release(struct kref *kref)
>  	obj->ops->release(obj);
>  }
>  
> +struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +			  const struct media_request_object_ops *ops,
> +			  void *priv)
> +{
> +	struct media_request_object *obj;
> +	struct media_request_object *found = NULL;
> +	unsigned long flags;
> +
> +	if (!ops && !priv)
> +		return NULL;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	list_for_each_entry(obj, &req->objects, list) {
> +		if ((!ops || obj->ops == ops) &&
> +		    (!priv || obj->priv == priv)) {
> +			media_request_object_get(obj);
> +			found = obj;
> +			break;
> +		}
> +	}
> +	spin_unlock_irqrestore(&req->lock, flags);

Huh? The spin lock were used before only to protect the req->state,
while the mutex is the one that protects the request itself.

So, here, it should be doing mutex_lock/unlock() instead.

> +	return found;
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_find);
> +
>  void media_request_object_put(struct media_request_object *obj)
>  {
>  	kref_put(&obj->kref, media_request_object_release);
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index 033697d493cd..ea990c8f76bc 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -130,6 +130,23 @@ static inline void media_request_object_get(struct media_request_object *obj)
>   */
>  void media_request_object_put(struct media_request_object *obj);
>  
> +/**
> + * media_request_object_find - Find an object in a request
> + *
> + * @ops: Find an object with this ops value, may be NULL.
> + * @priv: Find an object with this priv value, may be NULL.

@req ?

> + *
> + * At least one of @ops and @priv must be non-NULL. If one of
> + * these is NULL, then skip checking for that field.
> + *
> + * Returns NULL if not found or the object (the refcount is increased
> + * in that case).
> + */
> +struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +			  const struct media_request_object_ops *ops,
> +			  void *priv);
> +
>  /**
>   * media_request_object_init - Initialise a media request object
>   *
> @@ -162,6 +179,14 @@ static inline void media_request_object_put(struct media_request_object *obj)
>  {
>  }
>  
> +static inline struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +			  const struct media_request_object_ops *ops,
> +			  void *priv)
> +{
> +	return NULL;
> +}
> +
>  static inline void media_request_object_init(struct media_request_object *obj)
>  {
>  	obj->ops = NULL;



Thanks,
Mauro

Re: [RFCv11 PATCH 07/29] media-request: add media_request_object_find

[Hans Verkuil]

On 04/10/18 13:07, Mauro Carvalho Chehab wrote:
> Em Mon,  9 Apr 2018 16:20:04 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> Add media_request_object_find to find a request object inside a
>> request based on ops and/or priv values.
>>
>> Objects of the same type (vb2 buffer, control handler) will have
>> the same ops value. And objects that refer to the same 'parent'
>> object (e.g. the v4l2_ctrl_handler that has the current driver
>> state) will have the same priv value.
>>
>> The caller has to call media_request_object_put() for the returned
>> object since this function increments the refcount.
>>
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/media-request.c | 26 ++++++++++++++++++++++++++
>>  include/media/media-request.h | 25 +++++++++++++++++++++++++
>>  2 files changed, 51 insertions(+)
>>
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> index 02b620c81de5..415f7e31019d 100644
>> --- a/drivers/media/media-request.c
>> +++ b/drivers/media/media-request.c
>> @@ -322,6 +322,32 @@ static void media_request_object_release(struct kref *kref)
>>  	obj->ops->release(obj);
>>  }
>>  
>> +struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +			  const struct media_request_object_ops *ops,
>> +			  void *priv)
>> +{
>> +	struct media_request_object *obj;
>> +	struct media_request_object *found = NULL;
>> +	unsigned long flags;
>> +
>> +	if (!ops && !priv)
>> +		return NULL;
>> +
>> +	spin_lock_irqsave(&req->lock, flags);
>> +	list_for_each_entry(obj, &req->objects, list) {
>> +		if ((!ops || obj->ops == ops) &&
>> +		    (!priv || obj->priv == priv)) {
>> +			media_request_object_get(obj);
>> +			found = obj;
>> +			break;
>> +		}
>> +	}
>> +	spin_unlock_irqrestore(&req->lock, flags);
> 
> Huh? The spin lock were used before only to protect the req->state,
> while the mutex is the one that protects the request itself.
> 
> So, here, it should be doing mutex_lock/unlock() instead.

The mutex only serializes the actual queuing operation where you queue a
request (and will likely have to sleep etc.). The spinlock is for the the
other fields of the media_request struct.

Regards,

	Hans

> 
>> +	return found;
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_find);
>> +
>>  void media_request_object_put(struct media_request_object *obj)
>>  {
>>  	kref_put(&obj->kref, media_request_object_release);
>> diff --git a/include/media/media-request.h b/include/media/media-request.h
>> index 033697d493cd..ea990c8f76bc 100644
>> --- a/include/media/media-request.h
>> +++ b/include/media/media-request.h
>> @@ -130,6 +130,23 @@ static inline void media_request_object_get(struct media_request_object *obj)
>>   */
>>  void media_request_object_put(struct media_request_object *obj);
>>  
>> +/**
>> + * media_request_object_find - Find an object in a request
>> + *
>> + * @ops: Find an object with this ops value, may be NULL.
>> + * @priv: Find an object with this priv value, may be NULL.
> 
> @req ?
> 
>> + *
>> + * At least one of @ops and @priv must be non-NULL. If one of
>> + * these is NULL, then skip checking for that field.
>> + *
>> + * Returns NULL if not found or the object (the refcount is increased
>> + * in that case).
>> + */
>> +struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +			  const struct media_request_object_ops *ops,
>> +			  void *priv);
>> +
>>  /**
>>   * media_request_object_init - Initialise a media request object
>>   *
>> @@ -162,6 +179,14 @@ static inline void media_request_object_put(struct media_request_object *obj)
>>  {
>>  }
>>  
>> +static inline struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +			  const struct media_request_object_ops *ops,
>> +			  void *priv)
>> +{
>> +	return NULL;
>> +}
>> +
>>  static inline void media_request_object_init(struct media_request_object *obj)
>>  {
>>  	obj->ops = NULL;
> 
> 
> 
> Thanks,
> Mauro
> 

Re: [RFCv11 PATCH 08/29] videodev2.h: add request_fd field to v4l2_ext_controls

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:05 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Alexandre Courbot <acourbot@chromium.org>
> 
> If which is V4L2_CTRL_WHICH_REQUEST, then the request_fd field can be
> used to specify a request for the G/S/TRY_EXT_CTRLS ioctls.

That's a general comment that applies not only to this patch, but to
all patches touching include/uapi:

Please, for v12, add also the required Documentation/ changes together
with the patch that introduces the new field.

That makes a lot easier to understand why this would be needed. Also,
in the future, if someone needs to look at the git history, it makes
a way easier to identify what parts of the API got introduced by each
patch.

> 
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 5 ++++-
>  drivers/media/v4l2-core/v4l2-ioctl.c          | 6 +++---
>  include/uapi/linux/videodev2.h                | 4 +++-
>  3 files changed, 10 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> index 5198c9eeb348..0782b3666deb 100644
> --- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> +++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> @@ -732,7 +732,8 @@ struct v4l2_ext_controls32 {
>  	__u32 which;
>  	__u32 count;
>  	__u32 error_idx;
> -	__u32 reserved[2];
> +	__s32 request_fd;
> +	__u32 reserved[1];
>  	compat_caddr_t controls; /* actually struct v4l2_ext_control32 * */
>  };
>  
> @@ -807,6 +808,7 @@ static int get_v4l2_ext_controls32(struct file *file,
>  	    get_user(count, &up->count) ||
>  	    put_user(count, &kp->count) ||
>  	    assign_in_user(&kp->error_idx, &up->error_idx) ||
> +	    assign_in_user(&kp->request_fd, &up->request_fd) ||
>  	    copy_in_user(kp->reserved, up->reserved, sizeof(kp->reserved)))
>  		return -EFAULT;
>  
> @@ -865,6 +867,7 @@ static int put_v4l2_ext_controls32(struct file *file,
>  	    get_user(count, &kp->count) ||
>  	    put_user(count, &up->count) ||
>  	    assign_in_user(&up->error_idx, &kp->error_idx) ||
> +	    assign_in_user(&up->request_fd, &kp->request_fd) ||
>  	    copy_in_user(up->reserved, kp->reserved, sizeof(up->reserved)) ||
>  	    get_user(kcontrols, &kp->controls))
>  		return -EFAULT;
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index f48c505550e0..9ce23e23c5bf 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -553,8 +553,8 @@ static void v4l_print_ext_controls(const void *arg, bool write_only)
>  	const struct v4l2_ext_controls *p = arg;
>  	int i;
>  
> -	pr_cont("which=0x%x, count=%d, error_idx=%d",
> -			p->which, p->count, p->error_idx);
> +	pr_cont("which=0x%x, count=%d, error_idx=%d, request_fd=%d",
> +			p->which, p->count, p->error_idx, p->request_fd);
>  	for (i = 0; i < p->count; i++) {
>  		if (!p->controls[i].size)
>  			pr_cont(", id/val=0x%x/0x%x",
> @@ -870,7 +870,7 @@ static int check_ext_ctrls(struct v4l2_ext_controls *c, int allow_priv)
>  	__u32 i;
>  
>  	/* zero the reserved fields */
> -	c->reserved[0] = c->reserved[1] = 0;
> +	c->reserved[0] = 0;
>  	for (i = 0; i < c->count; i++)
>  		c->controls[i].reserved2[0] = 0;
>  
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 600877be5c22..6f41baa53787 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -1592,7 +1592,8 @@ struct v4l2_ext_controls {
>  	};
>  	__u32 count;
>  	__u32 error_idx;
> -	__u32 reserved[2];
> +	__s32 request_fd;
> +	__u32 reserved[1];
>  	struct v4l2_ext_control *controls;
>  };
>  
> @@ -1605,6 +1606,7 @@ struct v4l2_ext_controls {
>  #define V4L2_CTRL_MAX_DIMS	  (4)
>  #define V4L2_CTRL_WHICH_CUR_VAL   0
>  #define V4L2_CTRL_WHICH_DEF_VAL   0x0f000000
> +#define V4L2_CTRL_WHICH_REQUEST   0x0f010000
>  
>  enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_INTEGER	     = 1,



Thanks,
Mauro

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Sakari Ailus]

Hi Mauro and Hans,

On Tue, Apr 10, 2018 at 06:52:39AM -0300, Mauro Carvalho Chehab wrote:
> > diff --git a/include/media/media-device.h b/include/media/media-device.h
> > index bcc6ec434f1f..07e323c57202 100644
> > --- a/include/media/media-device.h
> > +++ b/include/media/media-device.h
> > @@ -19,6 +19,7 @@
> >  #ifndef _MEDIA_DEVICE_H
> >  #define _MEDIA_DEVICE_H
> >  
> > +#include <linux/anon_inodes.h>
> 
> Why do you need it? I don't see anything below needing it.

This should be on the 4th patch actually. The .c file should suffice.

> 
> >  #include <linux/list.h>
> >  #include <linux/mutex.h>
> >  
> > @@ -27,6 +28,7 @@
> >  
> >  struct ida;
> >  struct device;
> > +struct media_device;
> >  
> >  /**
> >   * struct media_entity_notify - Media Entity Notify
> > @@ -50,10 +52,16 @@ struct media_entity_notify {
> >   * struct media_device_ops - Media device operations
> >   * @link_notify: Link state change notification callback. This callback is
> >   *		 called with the graph_mutex held.
> > + * @req_alloc: Allocate a request
> > + * @req_free: Free a request
> > + * @req_queue: Queue a request
> >   */
> >  struct media_device_ops {
> >  	int (*link_notify)(struct media_link *link, u32 flags,
> >  			   unsigned int notification);
> > +	struct media_request *(*req_alloc)(struct media_device *mdev);
> > +	void (*req_free)(struct media_request *req);
> > +	int (*req_queue)(struct media_request *req);
> >  };
> >  
> >  /**
> > @@ -88,6 +96,8 @@ struct media_device_ops {
> >   * @disable_source: Disable Source Handler function pointer
> >   *
> >   * @ops:	Operation handler callbacks
> > + * @req_lock:	Serialise access to requests
> > + * @req_queue_mutex: Serialise validating and queueing requests
> 
> IMHO, this would better describe it:
> 	Serialise validate and queue requests
> 
> Yet, IMO, it doesn't let it clear when the spin lock should be
> used and when the mutex should be used.
> 
> I mean, what of them protect what variable?

It might not be obvious, but the purpose of this mutex is to prevent
queueing multiple requests simultaneously in order to serialise access to
the top of the queue. How about this instead:

	Serialise access to accessing device state on the tail of the
	request queue.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 10/29] v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:07 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add a 'bool from_other_dev' argument: set to true if the two
> handlers refer to different devices (e.g. it is true when
> inheriting controls from a subdev into a main v4l2 bridge
> driver).

Hmm... wouldn't it be clearer to call it as "is_inherited_control"?

> 
> This will be used later when implementing support for the
> request API since we need to skip such controls.
> 
> TODO: check drivers/staging/media/imx/imx-media-fim.c change.

I'm assuming that you'll be solving the TODO before the final
version of this patchset.

> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  drivers/media/dvb-frontends/rtl2832_sdr.c        |  5 +--
>  drivers/media/pci/bt8xx/bttv-driver.c            |  2 +-
>  drivers/media/pci/cx23885/cx23885-417.c          |  2 +-
>  drivers/media/pci/cx88/cx88-blackbird.c          |  2 +-
>  drivers/media/pci/cx88/cx88-video.c              |  2 +-
>  drivers/media/pci/saa7134/saa7134-empress.c      |  4 +--
>  drivers/media/pci/saa7134/saa7134-video.c        |  2 +-
>  drivers/media/platform/exynos4-is/fimc-capture.c |  2 +-
>  drivers/media/platform/rcar-vin/rcar-v4l2.c      |  3 +-
>  drivers/media/platform/rcar_drif.c               |  2 +-
>  drivers/media/platform/soc_camera/soc_camera.c   |  3 +-
>  drivers/media/platform/vivid/vivid-ctrls.c       | 46 ++++++++++++------------
>  drivers/media/usb/cx231xx/cx231xx-417.c          |  2 +-
>  drivers/media/usb/cx231xx/cx231xx-video.c        |  4 +--
>  drivers/media/usb/msi2500/msi2500.c              |  2 +-
>  drivers/media/usb/tm6000/tm6000-video.c          |  2 +-
>  drivers/media/v4l2-core/v4l2-ctrls.c             | 11 +++---
>  drivers/media/v4l2-core/v4l2-device.c            |  3 +-
>  drivers/staging/media/imx/imx-media-dev.c        |  2 +-
>  drivers/staging/media/imx/imx-media-fim.c        |  2 +-
>  include/media/v4l2-ctrls.h                       |  8 ++++-
>  21 files changed, 62 insertions(+), 49 deletions(-)
> 
> diff --git a/drivers/media/dvb-frontends/rtl2832_sdr.c b/drivers/media/dvb-frontends/rtl2832_sdr.c
> index c6e78d870ccd..6064d28224e8 100644
> --- a/drivers/media/dvb-frontends/rtl2832_sdr.c
> +++ b/drivers/media/dvb-frontends/rtl2832_sdr.c
> @@ -1394,7 +1394,8 @@ static int rtl2832_sdr_probe(struct platform_device *pdev)
>  	case RTL2832_SDR_TUNER_E4000:
>  		v4l2_ctrl_handler_init(&dev->hdl, 9);
>  		if (subdev)
> -			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler, NULL);
> +			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler,
> +					      NULL, true);
>  		break;
>  	case RTL2832_SDR_TUNER_R820T:
>  	case RTL2832_SDR_TUNER_R828D:
> @@ -1423,7 +1424,7 @@ static int rtl2832_sdr_probe(struct platform_device *pdev)
>  		v4l2_ctrl_handler_init(&dev->hdl, 2);
>  		if (subdev)
>  			v4l2_ctrl_add_handler(&dev->hdl, subdev->ctrl_handler,
> -					      NULL);
> +					      NULL, true);
>  		break;
>  	default:
>  		v4l2_ctrl_handler_init(&dev->hdl, 0);
> diff --git a/drivers/media/pci/bt8xx/bttv-driver.c b/drivers/media/pci/bt8xx/bttv-driver.c
> index f697698fe38d..cdcb36d8c5c3 100644
> --- a/drivers/media/pci/bt8xx/bttv-driver.c
> +++ b/drivers/media/pci/bt8xx/bttv-driver.c
> @@ -4211,7 +4211,7 @@ static int bttv_probe(struct pci_dev *dev, const struct pci_device_id *pci_id)
>  	/* register video4linux + input */
>  	if (!bttv_tvcards[btv->c.type].no_video) {
>  		v4l2_ctrl_add_handler(&btv->radio_ctrl_handler, hdl,
> -				v4l2_ctrl_radio_filter);
> +				v4l2_ctrl_radio_filter, false);
>  		if (btv->radio_ctrl_handler.error) {
>  			result = btv->radio_ctrl_handler.error;
>  			goto fail2;
> diff --git a/drivers/media/pci/cx23885/cx23885-417.c b/drivers/media/pci/cx23885/cx23885-417.c
> index a71f3c7569ce..762823871c78 100644
> --- a/drivers/media/pci/cx23885/cx23885-417.c
> +++ b/drivers/media/pci/cx23885/cx23885-417.c
> @@ -1527,7 +1527,7 @@ int cx23885_417_register(struct cx23885_dev *dev)
>  	dev->cxhdl.priv = dev;
>  	dev->cxhdl.func = cx23885_api_func;
>  	cx2341x_handler_set_50hz(&dev->cxhdl, tsport->height == 576);
> -	v4l2_ctrl_add_handler(&dev->ctrl_handler, &dev->cxhdl.hdl, NULL);
> +	v4l2_ctrl_add_handler(&dev->ctrl_handler, &dev->cxhdl.hdl, NULL, false);
>  
>  	/* Allocate and initialize V4L video device */
>  	dev->v4l_device = cx23885_video_dev_alloc(tsport,
> diff --git a/drivers/media/pci/cx88/cx88-blackbird.c b/drivers/media/pci/cx88/cx88-blackbird.c
> index 0e0952e60795..39f69d89a663 100644
> --- a/drivers/media/pci/cx88/cx88-blackbird.c
> +++ b/drivers/media/pci/cx88/cx88-blackbird.c
> @@ -1183,7 +1183,7 @@ static int cx8802_blackbird_probe(struct cx8802_driver *drv)
>  	err = cx2341x_handler_init(&dev->cxhdl, 36);
>  	if (err)
>  		goto fail_core;
> -	v4l2_ctrl_add_handler(&dev->cxhdl.hdl, &core->video_hdl, NULL);
> +	v4l2_ctrl_add_handler(&dev->cxhdl.hdl, &core->video_hdl, NULL, false);
>  
>  	/* blackbird stuff */
>  	pr_info("cx23416 based mpeg encoder (blackbird reference design)\n");
> diff --git a/drivers/media/pci/cx88/cx88-video.c b/drivers/media/pci/cx88/cx88-video.c
> index 9be682cdb644..e35bfa03a1e2 100644
> --- a/drivers/media/pci/cx88/cx88-video.c
> +++ b/drivers/media/pci/cx88/cx88-video.c
> @@ -1378,7 +1378,7 @@ static int cx8800_initdev(struct pci_dev *pci_dev,
>  		if (vc->id == V4L2_CID_CHROMA_AGC)
>  			core->chroma_agc = vc;
>  	}
> -	v4l2_ctrl_add_handler(&core->video_hdl, &core->audio_hdl, NULL);
> +	v4l2_ctrl_add_handler(&core->video_hdl, &core->audio_hdl, NULL, false);
>  
>  	/* load and configure helper modules */
>  
> diff --git a/drivers/media/pci/saa7134/saa7134-empress.c b/drivers/media/pci/saa7134/saa7134-empress.c
> index 66acfd35ffc6..fc75ce00dbf8 100644
> --- a/drivers/media/pci/saa7134/saa7134-empress.c
> +++ b/drivers/media/pci/saa7134/saa7134-empress.c
> @@ -265,9 +265,9 @@ static int empress_init(struct saa7134_dev *dev)
>  		 "%s empress (%s)", dev->name,
>  		 saa7134_boards[dev->board].name);
>  	v4l2_ctrl_handler_init(hdl, 21);
> -	v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler, empress_ctrl_filter);
> +	v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler, empress_ctrl_filter, false);
>  	if (dev->empress_sd)
> -		v4l2_ctrl_add_handler(hdl, dev->empress_sd->ctrl_handler, NULL);
> +		v4l2_ctrl_add_handler(hdl, dev->empress_sd->ctrl_handler, NULL, true);
>  	if (hdl->error) {
>  		video_device_release(dev->empress_dev);
>  		return hdl->error;
> diff --git a/drivers/media/pci/saa7134/saa7134-video.c b/drivers/media/pci/saa7134/saa7134-video.c
> index 4f1091a11e91..d478c470b975 100644
> --- a/drivers/media/pci/saa7134/saa7134-video.c
> +++ b/drivers/media/pci/saa7134/saa7134-video.c
> @@ -2136,7 +2136,7 @@ int saa7134_video_init1(struct saa7134_dev *dev)
>  		hdl = &dev->radio_ctrl_handler;
>  		v4l2_ctrl_handler_init(hdl, 2);
>  		v4l2_ctrl_add_handler(hdl, &dev->ctrl_handler,
> -				v4l2_ctrl_radio_filter);
> +				v4l2_ctrl_radio_filter, false);
>  		if (hdl->error)
>  			return hdl->error;
>  	}
> diff --git a/drivers/media/platform/exynos4-is/fimc-capture.c b/drivers/media/platform/exynos4-is/fimc-capture.c
> index a3cdac188190..2164375f0ee0 100644
> --- a/drivers/media/platform/exynos4-is/fimc-capture.c
> +++ b/drivers/media/platform/exynos4-is/fimc-capture.c
> @@ -1424,7 +1424,7 @@ static int fimc_link_setup(struct media_entity *entity,
>  		return 0;
>  
>  	return v4l2_ctrl_add_handler(&vc->ctx->ctrls.handler,
> -				     sensor->ctrl_handler, NULL);
> +				     sensor->ctrl_handler, NULL, true);
>  }
>  
>  static const struct media_entity_operations fimc_sd_media_ops = {
> diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> index b479b882da12..90246113fa03 100644
> --- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
> +++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c
> @@ -900,7 +900,8 @@ int rvin_v4l2_probe(struct rvin_dev *vin)
>  	if (ret < 0)
>  		return ret;
>  
> -	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler, NULL);
> +	ret = v4l2_ctrl_add_handler(&vin->ctrl_handler, sd->ctrl_handler,
> +				    NULL, true);
>  	if (ret < 0)
>  		return ret;
>  
> diff --git a/drivers/media/platform/rcar_drif.c b/drivers/media/platform/rcar_drif.c
> index dc7e280c91b4..159c7d2c2066 100644
> --- a/drivers/media/platform/rcar_drif.c
> +++ b/drivers/media/platform/rcar_drif.c
> @@ -1168,7 +1168,7 @@ static int rcar_drif_notify_complete(struct v4l2_async_notifier *notifier)
>  	}
>  
>  	ret = v4l2_ctrl_add_handler(&sdr->ctrl_hdl,
> -				    sdr->ep.subdev->ctrl_handler, NULL);
> +				    sdr->ep.subdev->ctrl_handler, NULL, true);
>  	if (ret) {
>  		rdrif_err(sdr, "failed: ctrl add hdlr ret %d\n", ret);
>  		goto error;
> diff --git a/drivers/media/platform/soc_camera/soc_camera.c b/drivers/media/platform/soc_camera/soc_camera.c
> index 69f0d8e80bd8..e6787abc34ae 100644
> --- a/drivers/media/platform/soc_camera/soc_camera.c
> +++ b/drivers/media/platform/soc_camera/soc_camera.c
> @@ -1180,7 +1180,8 @@ static int soc_camera_probe_finish(struct soc_camera_device *icd)
>  
>  	v4l2_subdev_call(sd, video, g_tvnorms, &icd->vdev->tvnorms);
>  
> -	ret = v4l2_ctrl_add_handler(&icd->ctrl_handler, sd->ctrl_handler, NULL);
> +	ret = v4l2_ctrl_add_handler(&icd->ctrl_handler, sd->ctrl_handler,
> +				    NULL, true);
>  	if (ret < 0)
>  		return ret;
>  
> diff --git a/drivers/media/platform/vivid/vivid-ctrls.c b/drivers/media/platform/vivid/vivid-ctrls.c
> index 6b0bfa091592..f369b94ad7ff 100644
> --- a/drivers/media/platform/vivid/vivid-ctrls.c
> +++ b/drivers/media/platform/vivid/vivid-ctrls.c
> @@ -1662,59 +1662,59 @@ int vivid_create_controls(struct vivid_dev *dev, bool show_ccs_cap,
>  		v4l2_ctrl_auto_cluster(2, &dev->autogain, 0, true);
>  
>  	if (dev->has_vid_cap) {
> -		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_gen, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_vid, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_aud, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_streaming, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_sdtv_cap, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_loop_cap, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_fb, NULL);
> +		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_gen, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_vid, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_user_aud, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_streaming, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_sdtv_cap, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_loop_cap, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_cap, hdl_fb, NULL, false);
>  		if (hdl_vid_cap->error)
>  			return hdl_vid_cap->error;
>  		dev->vid_cap_dev.ctrl_handler = hdl_vid_cap;
>  	}
>  	if (dev->has_vid_out) {
> -		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_gen, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_aud, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_out, hdl_streaming, NULL);
> -		v4l2_ctrl_add_handler(hdl_vid_out, hdl_fb, NULL);
> +		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_gen, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_out, hdl_user_aud, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_out, hdl_streaming, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vid_out, hdl_fb, NULL, false);
>  		if (hdl_vid_out->error)
>  			return hdl_vid_out->error;
>  		dev->vid_out_dev.ctrl_handler = hdl_vid_out;
>  	}
>  	if (dev->has_vbi_cap) {
> -		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_user_gen, NULL);
> -		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_streaming, NULL);
> -		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_sdtv_cap, NULL);
> -		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_loop_cap, NULL);
> +		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_user_gen, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_streaming, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_sdtv_cap, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vbi_cap, hdl_loop_cap, NULL, false);
>  		if (hdl_vbi_cap->error)
>  			return hdl_vbi_cap->error;
>  		dev->vbi_cap_dev.ctrl_handler = hdl_vbi_cap;
>  	}
>  	if (dev->has_vbi_out) {
> -		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_user_gen, NULL);
> -		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_streaming, NULL);
> +		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_user_gen, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_vbi_out, hdl_streaming, NULL, false);
>  		if (hdl_vbi_out->error)
>  			return hdl_vbi_out->error;
>  		dev->vbi_out_dev.ctrl_handler = hdl_vbi_out;
>  	}
>  	if (dev->has_radio_rx) {
> -		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_gen, NULL);
> -		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_aud, NULL);
> +		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_gen, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_radio_rx, hdl_user_aud, NULL, false);
>  		if (hdl_radio_rx->error)
>  			return hdl_radio_rx->error;
>  		dev->radio_rx_dev.ctrl_handler = hdl_radio_rx;
>  	}
>  	if (dev->has_radio_tx) {
> -		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_gen, NULL);
> -		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_aud, NULL);
> +		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_gen, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_radio_tx, hdl_user_aud, NULL, false);
>  		if (hdl_radio_tx->error)
>  			return hdl_radio_tx->error;
>  		dev->radio_tx_dev.ctrl_handler = hdl_radio_tx;
>  	}
>  	if (dev->has_sdr_cap) {
> -		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_user_gen, NULL);
> -		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_streaming, NULL);
> +		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_user_gen, NULL, false);
> +		v4l2_ctrl_add_handler(hdl_sdr_cap, hdl_streaming, NULL, false);
>  		if (hdl_sdr_cap->error)
>  			return hdl_sdr_cap->error;
>  		dev->sdr_cap_dev.ctrl_handler = hdl_sdr_cap;
> diff --git a/drivers/media/usb/cx231xx/cx231xx-417.c b/drivers/media/usb/cx231xx/cx231xx-417.c
> index b80e6857e2eb..fca16cf8b3bf 100644
> --- a/drivers/media/usb/cx231xx/cx231xx-417.c
> +++ b/drivers/media/usb/cx231xx/cx231xx-417.c
> @@ -1991,7 +1991,7 @@ int cx231xx_417_register(struct cx231xx *dev)
>  	dev->mpeg_ctrl_handler.ops = &cx231xx_ops;
>  	if (dev->sd_cx25840)
>  		v4l2_ctrl_add_handler(&dev->mpeg_ctrl_handler.hdl,
> -				dev->sd_cx25840->ctrl_handler, NULL);
> +				dev->sd_cx25840->ctrl_handler, NULL, false);
>  	if (dev->mpeg_ctrl_handler.hdl.error) {
>  		err = dev->mpeg_ctrl_handler.hdl.error;
>  		dprintk(3, "%s: can't add cx25840 controls\n", dev->name);
> diff --git a/drivers/media/usb/cx231xx/cx231xx-video.c b/drivers/media/usb/cx231xx/cx231xx-video.c
> index f7fcd733a2ca..2dedb18f63a0 100644
> --- a/drivers/media/usb/cx231xx/cx231xx-video.c
> +++ b/drivers/media/usb/cx231xx/cx231xx-video.c
> @@ -2204,10 +2204,10 @@ int cx231xx_register_analog_devices(struct cx231xx *dev)
>  
>  	if (dev->sd_cx25840) {
>  		v4l2_ctrl_add_handler(&dev->ctrl_handler,
> -				dev->sd_cx25840->ctrl_handler, NULL);
> +				dev->sd_cx25840->ctrl_handler, NULL, true);
>  		v4l2_ctrl_add_handler(&dev->radio_ctrl_handler,
>  				dev->sd_cx25840->ctrl_handler,
> -				v4l2_ctrl_radio_filter);
> +				v4l2_ctrl_radio_filter, true);
>  	}
>  
>  	if (dev->ctrl_handler.error)
> diff --git a/drivers/media/usb/msi2500/msi2500.c b/drivers/media/usb/msi2500/msi2500.c
> index 65ef755adfdc..4aacd77a5d58 100644
> --- a/drivers/media/usb/msi2500/msi2500.c
> +++ b/drivers/media/usb/msi2500/msi2500.c
> @@ -1278,7 +1278,7 @@ static int msi2500_probe(struct usb_interface *intf,
>  	}
>  
>  	/* currently all controls are from subdev */
> -	v4l2_ctrl_add_handler(&dev->hdl, sd->ctrl_handler, NULL);
> +	v4l2_ctrl_add_handler(&dev->hdl, sd->ctrl_handler, NULL, true);
>  
>  	dev->v4l2_dev.ctrl_handler = &dev->hdl;
>  	dev->vdev.v4l2_dev = &dev->v4l2_dev;
> diff --git a/drivers/media/usb/tm6000/tm6000-video.c b/drivers/media/usb/tm6000/tm6000-video.c
> index b2399d4266da..99d6b0da8b3d 100644
> --- a/drivers/media/usb/tm6000/tm6000-video.c
> +++ b/drivers/media/usb/tm6000/tm6000-video.c
> @@ -1622,7 +1622,7 @@ int tm6000_v4l2_register(struct tm6000_core *dev)
>  	v4l2_ctrl_new_std(&dev->ctrl_handler, &tm6000_ctrl_ops,
>  			V4L2_CID_HUE, -128, 127, 1, 0);
>  	v4l2_ctrl_add_handler(&dev->ctrl_handler,
> -			&dev->radio_ctrl_handler, NULL);
> +			&dev->radio_ctrl_handler, NULL, false);
>  
>  	if (dev->radio_ctrl_handler.error)
>  		ret = dev->radio_ctrl_handler.error;
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index d29e45516eb7..aa1dd2015e84 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -1995,7 +1995,8 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
>  
>  /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
>  static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
> -			   struct v4l2_ctrl *ctrl)
> +			   struct v4l2_ctrl *ctrl,
> +			   bool from_other_dev)
>  {
>  	struct v4l2_ctrl_ref *ref;
>  	struct v4l2_ctrl_ref *new_ref;
> @@ -2019,6 +2020,7 @@ static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
>  	if (!new_ref)
>  		return handler_set_err(hdl, -ENOMEM);
>  	new_ref->ctrl = ctrl;
> +	new_ref->from_other_dev = from_other_dev;
>  	if (ctrl->handler == hdl) {
>  		/* By default each control starts in a cluster of its own.
>  		   new_ref->ctrl is basically a cluster array with one
> @@ -2199,7 +2201,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  		ctrl->type_ops->init(ctrl, idx, ctrl->p_new);
>  	}
>  
> -	if (handler_new_ref(hdl, ctrl)) {
> +	if (handler_new_ref(hdl, ctrl, false)) {
>  		kvfree(ctrl);
>  		return NULL;
>  	}
> @@ -2368,7 +2370,8 @@ EXPORT_SYMBOL(v4l2_ctrl_new_int_menu);
>  /* Add the controls from another handler to our own. */
>  int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
>  			  struct v4l2_ctrl_handler *add,
> -			  bool (*filter)(const struct v4l2_ctrl *ctrl))
> +			  bool (*filter)(const struct v4l2_ctrl *ctrl),
> +			  bool from_other_dev)
>  {
>  	struct v4l2_ctrl_ref *ref;
>  	int ret = 0;
> @@ -2391,7 +2394,7 @@ int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
>  		/* Filter any unwanted controls */
>  		if (filter && !filter(ctrl))
>  			continue;
> -		ret = handler_new_ref(hdl, ctrl);
> +		ret = handler_new_ref(hdl, ctrl, from_other_dev);
>  		if (ret)
>  			break;
>  	}
> diff --git a/drivers/media/v4l2-core/v4l2-device.c b/drivers/media/v4l2-core/v4l2-device.c
> index 937c6de85606..8391a7f0895b 100644
> --- a/drivers/media/v4l2-core/v4l2-device.c
> +++ b/drivers/media/v4l2-core/v4l2-device.c
> @@ -178,7 +178,8 @@ int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
>  
>  	sd->v4l2_dev = v4l2_dev;
>  	/* This just returns 0 if either of the two args is NULL */
> -	err = v4l2_ctrl_add_handler(v4l2_dev->ctrl_handler, sd->ctrl_handler, NULL);
> +	err = v4l2_ctrl_add_handler(v4l2_dev->ctrl_handler, sd->ctrl_handler,
> +				    NULL, true);
>  	if (err)
>  		goto error_module;
>  
> diff --git a/drivers/staging/media/imx/imx-media-dev.c b/drivers/staging/media/imx/imx-media-dev.c
> index 289d775c4820..08799beaea42 100644
> --- a/drivers/staging/media/imx/imx-media-dev.c
> +++ b/drivers/staging/media/imx/imx-media-dev.c
> @@ -391,7 +391,7 @@ static int imx_media_inherit_controls(struct imx_media_dev *imxmd,
>  
>  		ret = v4l2_ctrl_add_handler(vfd->ctrl_handler,
>  					    sd->ctrl_handler,
> -					    NULL);
> +					    NULL, true);
>  		if (ret)
>  			return ret;
>  	}
> diff --git a/drivers/staging/media/imx/imx-media-fim.c b/drivers/staging/media/imx/imx-media-fim.c
> index 6df189135db8..8cf773eef9da 100644
> --- a/drivers/staging/media/imx/imx-media-fim.c
> +++ b/drivers/staging/media/imx/imx-media-fim.c
> @@ -463,7 +463,7 @@ int imx_media_fim_add_controls(struct imx_media_fim *fim)
>  {
>  	/* add the FIM controls to the calling subdev ctrl handler */
>  	return v4l2_ctrl_add_handler(fim->sd->ctrl_handler,
> -				     &fim->ctrl_handler, NULL);
> +				     &fim->ctrl_handler, NULL, false);
>  }
>  EXPORT_SYMBOL_GPL(imx_media_fim_add_controls);
>  
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 5b445b5654f7..f8faa54b5e7e 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -247,6 +247,8 @@ struct v4l2_ctrl {
>   * @ctrl:	The actual control information.
>   * @helper:	Pointer to helper struct. Used internally in
>   *		``prepare_ext_ctrls`` function at ``v4l2-ctrl.c``.
> + * @from_other_dev: If true, then @ctrl was defined in another
> + *		device then the &struct v4l2_ctrl_handler.

typo:
	device then -> device than

>   *
>   * Each control handler has a list of these refs. The list_head is used to
>   * keep a sorted-by-control-ID list of all controls, while the next pointer
> @@ -257,6 +259,7 @@ struct v4l2_ctrl_ref {
>  	struct v4l2_ctrl_ref *next;
>  	struct v4l2_ctrl *ctrl;
>  	struct v4l2_ctrl_helper *helper;
> +	bool from_other_dev;
>  };
>  
>  /**
> @@ -633,6 +636,8 @@ typedef bool (*v4l2_ctrl_filter)(const struct v4l2_ctrl *ctrl);
>   * @add:	The control handler whose controls you want to add to
>   *		the @hdl control handler.
>   * @filter:	This function will filter which controls should be added.
> + * @from_other_dev: If true, then the controls in @add were defined in another
> + *		device then @hdl.

typo:
	device then -> device than

>   *
>   * Does nothing if either of the two handlers is a NULL pointer.
>   * If @filter is NULL, then all controls are added. Otherwise only those
> @@ -642,7 +647,8 @@ typedef bool (*v4l2_ctrl_filter)(const struct v4l2_ctrl *ctrl);
>   */
>  int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
>  			  struct v4l2_ctrl_handler *add,
> -			  v4l2_ctrl_filter filter);
> +			  v4l2_ctrl_filter filter,
> +			  bool from_other_dev);
>  
>  /**
>   * v4l2_ctrl_radio_filter() - Standard filter for radio controls.



Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

Hi Hans,

Thanks for the update.

On Mon, Apr 09, 2018 at 04:20:01PM +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Implement the core of the media request processing.
> 
> Drivers can bind request objects to a request. These objects
> can then be marked completed if the driver finished using them,
> or just be unbound if the results do not need to be kept (e.g.
> in the case of buffers).
> 
> Once all objects that were added are either unbound or completed,
> the request is marked 'complete' and a POLLPRI signal is sent
> via poll.
> 
> Both requests and request objects are refcounted.
> 
> While a request is queued its refcount is incremented (since it
> is in use by a driver). Once it is completed the refcount is
> decremented. When the user closes the request file descriptor
> the refcount is also decremented. Once it reaches 0 all request
> objects in the request are unbound and put() and the request
> itself is freed.

I think this would be also good for kAPI documentation.

> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 284 +++++++++++++++++++++++++++++++++++++++++-
>  include/media/media-request.h | 156 +++++++++++++++++++++++
>  2 files changed, 439 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index ead78613fdbe..dffc290e4ada 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -16,8 +16,290 @@
>  #include <media/media-device.h>
>  #include <media/media-request.h>
>  
> +static const char * const request_state[] = {
> +	"idle",
> +	"queueing",
> +	"queued",
> +	"complete",
> +	"cleaning",
> +};
> +
> +static const char *
> +media_request_state_str(enum media_request_state state)
> +{
> +	if (WARN_ON(state >= ARRAY_SIZE(request_state)))
> +		return "unknown";
> +	return request_state[state];
> +}
> +
> +static void media_request_clean(struct media_request *req)
> +{
> +	struct media_request_object *obj, *obj_safe;
> +
> +	WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
> +
> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> +		media_request_object_unbind(obj);
> +		media_request_object_put(obj);
> +	}
> +
> +	req->num_incomplete_objects = 0;
> +	wake_up_interruptible(&req->poll_wait);

What's the purpose to wake up poll here?

The request must have been in an IDLE state (REINIT) or in CLEANING state
if it is being released. Neither could have been polled.

> +}
> +
> +static void media_request_release(struct kref *kref)
> +{
> +	struct media_request *req =
> +		container_of(kref, struct media_request, kref);
> +	struct media_device *mdev = req->mdev;
> +	unsigned long flags;
> +
> +	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> +	spin_unlock_irqrestore(&req->lock, flags);
> +
> +	media_request_clean(req);
> +
> +	if (mdev->ops->req_free)
> +		mdev->ops->req_free(req);
> +	else
> +		kfree(req);
> +}
> +
> +void media_request_put(struct media_request *req)
> +{
> +	kref_put(&req->kref, media_request_release);
> +}
> +EXPORT_SYMBOL_GPL(media_request_put);
> +
> +void media_request_cancel(struct media_request *req)
> +{
> +	struct media_request_object *obj, *obj_safe;
> +
> +	if (req->state != MEDIA_REQUEST_STATE_QUEUED)
> +		return;
> +
> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
> +		if (obj->ops->cancel)
> +			obj->ops->cancel(obj);
> +}
> +EXPORT_SYMBOL_GPL(media_request_cancel);
> +
> +static int media_request_close(struct inode *inode, struct file *filp)
> +{
> +	struct media_request *req = filp->private_data;
> +
> +	media_request_put(req);

Newline?

> +	return 0;
> +}
> +
> +static unsigned int media_request_poll(struct file *filp,
> +				       struct poll_table_struct *wait)
> +{
> +	struct media_request *req = filp->private_data;
> +	unsigned long flags;
> +	enum media_request_state state;
> +
> +	if (!(poll_requested_events(wait) & POLLPRI))
> +		return 0;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	state = req->state;
> +	spin_unlock_irqrestore(&req->lock, flags);
> +
> +	if (state == MEDIA_REQUEST_STATE_COMPLETE)
> +		return POLLPRI;
> +	if (state == MEDIA_REQUEST_STATE_IDLE)
> +		return POLLERR;
> +
> +	poll_wait(filp, &req->poll_wait, wait);

Newline.

> +	return 0;
> +}
> +
> +static long media_request_ioctl(struct file *filp, unsigned int cmd,
> +				unsigned long __arg)
> +{
> +	return -ENOIOCTLCMD;
> +}
> +
> +static const struct file_operations request_fops = {
> +	.owner = THIS_MODULE,
> +	.poll = media_request_poll,
> +	.unlocked_ioctl = media_request_ioctl,
> +	.release = media_request_close,
> +};
> +
>  int media_request_alloc(struct media_device *mdev,
>  			struct media_request_alloc *alloc)
>  {
> -	return -ENOMEM;
> +	struct media_request *req;
> +	struct file *filp;
> +	char comm[TASK_COMM_LEN];
> +	int fd;
> +	int ret;
> +
> +	fd = get_unused_fd_flags(O_CLOEXEC);
> +	if (fd < 0)
> +		return fd;
> +
> +	filp = anon_inode_getfile("request", &request_fops, NULL, O_CLOEXEC);
> +	if (IS_ERR(filp)) {
> +		ret = PTR_ERR(filp);
> +		goto err_put_fd;
> +	}
> +
> +	if (mdev->ops->req_alloc)
> +		req = mdev->ops->req_alloc(mdev);
> +	else
> +		req = kzalloc(sizeof(*req), GFP_KERNEL);
> +	if (!req) {
> +		ret = -ENOMEM;
> +		goto err_fput;
> +	}
> +
> +	filp->private_data = req;
> +	req->mdev = mdev;
> +	req->state = MEDIA_REQUEST_STATE_IDLE;
> +	req->num_incomplete_objects = 0;
> +	kref_init(&req->kref);
> +	INIT_LIST_HEAD(&req->objects);
> +	spin_lock_init(&req->lock);
> +	init_waitqueue_head(&req->poll_wait);
> +
> +	alloc->fd = fd;
> +
> +	get_task_comm(comm, current);
> +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> +		 comm, fd);
> +	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
> +
> +	fd_install(fd, filp);
> +
> +	return 0;
> +
> +err_fput:
> +	fput(filp);
> +
> +err_put_fd:
> +	put_unused_fd(fd);
> +
> +	return ret;
> +}
> +
> +static void media_request_object_release(struct kref *kref)
> +{
> +	struct media_request_object *obj =
> +		container_of(kref, struct media_request_object, kref);
> +	struct media_request *req = obj->req;
> +
> +	if (req)
> +		media_request_object_unbind(obj);
> +	obj->ops->release(obj);
> +}
> +
> +void media_request_object_put(struct media_request_object *obj)
> +{
> +	kref_put(&obj->kref, media_request_object_release);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_put);
> +
> +void media_request_object_init(struct media_request_object *obj)
> +{
> +	obj->ops = NULL;
> +	obj->req = NULL;
> +	obj->priv = NULL;
> +	obj->completed = false;
> +	INIT_LIST_HEAD(&obj->list);
> +	kref_init(&obj->kref);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_init);
> +
> +void media_request_object_bind(struct media_request *req,
> +			       const struct media_request_object_ops *ops,
> +			       void *priv,
> +			       struct media_request_object *obj)
> +{
> +	unsigned long flags;
> +
> +	if (WARN_ON(!ops->release || !ops->cancel))
> +		return;
> +
> +	obj->req = req;
> +	obj->ops = ops;
> +	obj->priv = priv;

What would be the purpose of setting ops and priv fields here rather than
in object init above? They're not supposed to change during a lifetime of
an object, are they?

The fact that releasing the object is done through the release callback in
ops suggests that setting ops (as well as priv) belongs to object init.

> +	spin_lock_irqsave(&req->lock, flags);
> +	if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
> +		goto unlock;
> +	list_add_tail(&obj->list, &req->objects);
> +	req->num_incomplete_objects++;
> +unlock:
> +	spin_unlock_irqrestore(&req->lock, flags);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_bind);
> +
> +void media_request_object_unbind(struct media_request_object *obj)
> +{
> +	struct media_request *req = obj->req;
> +	unsigned long flags;
> +	bool completed = false;
> +
> +	if (!req)
> +		return;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	list_del(&obj->list);
> +	obj->req = NULL;
> +
> +	if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
> +	    req->state == MEDIA_REQUEST_STATE_CLEANING)
> +		goto unlock;
> +
> +	if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
> +		goto unlock;
> +
> +	if (WARN_ON(!req->num_incomplete_objects))
> +		goto unlock;
> +
> +	req->num_incomplete_objects--;
> +	if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
> +	    !req->num_incomplete_objects) {
> +		req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +		completed = true;
> +		wake_up_interruptible(&req->poll_wait);
> +	}

Newline?

> +unlock:
> +	spin_unlock_irqrestore(&req->lock, flags);
> +	if (obj->ops->unbind)
> +		obj->ops->unbind(obj);
> +	if (completed)
> +		media_request_put(req);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_unbind);
> +
> +void media_request_object_complete(struct media_request_object *obj)
> +{
> +	struct media_request *req = obj->req;
> +	unsigned long flags;
> +	bool completed = false;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	if (obj->completed)
> +		goto unlock;
> +	obj->completed = true;
> +	if (WARN_ON(!req->num_incomplete_objects) ||
> +	    WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
> +		goto unlock;
> +
> +	if (!--req->num_incomplete_objects) {
> +		req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +		wake_up_interruptible(&req->poll_wait);
> +		completed = true;
> +	}

Newline.

> +unlock:
> +	spin_unlock_irqrestore(&req->lock, flags);
> +	if (completed)
> +		media_request_put(req);
>  }
> +EXPORT_SYMBOL_GPL(media_request_object_complete);
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index dae3eccd9aa7..082c3cae04ac 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -16,7 +16,163 @@
>  
>  #include <media/media-device.h>
>  
> +enum media_request_state {
> +	MEDIA_REQUEST_STATE_IDLE,
> +	MEDIA_REQUEST_STATE_QUEUEING,
> +	MEDIA_REQUEST_STATE_QUEUED,
> +	MEDIA_REQUEST_STATE_COMPLETE,
> +	MEDIA_REQUEST_STATE_CLEANING,
> +};
> +
> +struct media_request_object;
> +
> +/**
> + * struct media_request - Media device request
> + * @mdev: Media device this request belongs to
> + * @kref: Reference count
> + * @debug_prefix: Prefix for debug messages (process name:fd)
> + * @state: The state of the request
> + * @objects: List of @struct media_request_object request objects
> + * @num_objects: The number objects in the request
> + * @num_completed_objects: The number of completed objects in the request
> + * @poll_wait: Wait queue for poll
> + * @lock: Serializes access to this struct
> + */
> +struct media_request {
> +	struct media_device *mdev;
> +	struct kref kref;
> +	char debug_str[TASK_COMM_LEN + 11];
> +	enum media_request_state state;
> +	struct list_head objects;
> +	unsigned int num_incomplete_objects;
> +	struct wait_queue_head poll_wait;
> +	spinlock_t lock;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +
> +static inline void media_request_get(struct media_request *req)
> +{
> +	kref_get(&req->kref);
> +}
> +
> +void media_request_put(struct media_request *req);
> +void media_request_cancel(struct media_request *req);
> +
>  int media_request_alloc(struct media_device *mdev,
>  			struct media_request_alloc *alloc);
> +#else
> +static inline void media_request_get(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_put(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_cancel(struct media_request *req)
> +{
> +}
> +
> +#endif
> +

Some documentation would be nice.

> +struct media_request_object_ops {
> +	int (*prepare)(struct media_request_object *object);
> +	void (*unprepare)(struct media_request_object *object);
> +	void (*queue)(struct media_request_object *object);
> +	void (*unbind)(struct media_request_object *object);
> +	void (*cancel)(struct media_request_object *object);
> +	void (*release)(struct media_request_object *object);
> +};
> +
> +/**
> + * struct media_request_object - An opaque object that belongs to a media
> + *				 request
> + *
> + * @priv: object's priv pointer
> + * @list: List entry of the object for @struct media_request
> + * @kref: Reference count of the object, acquire before releasing req->lock
> + *
> + * An object related to the request. This struct is embedded in the
> + * larger object data.
> + */
> +struct media_request_object {
> +	const struct media_request_object_ops *ops;
> +	void *priv;
> +	struct media_request *req;
> +	struct list_head list;
> +	struct kref kref;
> +	bool completed;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +static inline void media_request_object_get(struct media_request_object *obj)
> +{
> +	kref_get(&obj->kref);
> +}
> +
> +/**
> + * media_request_object_put - Put a media request object
> + *
> + * @obj: The object
> + *
> + * Put a media request object. Once all references are gone, the
> + * object's memory is released.
> + */
> +void media_request_object_put(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_init - Initialise a media request object
> + *
> + * Initialise a media request object. The object will be released using the
> + * release callback of the ops once it has no references (this function
> + * initialises references to one).
> + */
> +void media_request_object_init(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_bind - Bind a media request object to a request
> + */
> +void media_request_object_bind(struct media_request *req,
> +			       const struct media_request_object_ops *ops,
> +			       void *priv,
> +			       struct media_request_object *obj);
> +
> +void media_request_object_unbind(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_complete - Mark the media request object as complete
> + */

I think we'll need to document bind, unbind and complete very well; exactly
when can they be used and for what purpose.

> +void media_request_object_complete(struct media_request_object *obj);
> +#else
> +static inline void media_request_object_get(struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_put(struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_init(struct media_request_object *obj)
> +{
> +	obj->ops = NULL;
> +	obj->req = NULL;
> +}
> +
> +static inline void media_request_object_bind(struct media_request *req,
> +			       const struct media_request_object_ops *ops,
> +			       void *priv,
> +			       struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_unbind(struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_complete(struct media_request_object *obj)
> +{
> +}
> +#endif
>  
>  #endif

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

Hi Mauro and Hans,

On Tue, Apr 10, 2018 at 07:32:06AM -0300, Mauro Carvalho Chehab wrote:
...
> > +static void media_request_release(struct kref *kref)
> > +{
> > +	struct media_request *req =
> > +		container_of(kref, struct media_request, kref);
> > +	struct media_device *mdev = req->mdev;
> > +	unsigned long flags;
> > +
> > +	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> > +
> > +	spin_lock_irqsave(&req->lock, flags);
> > +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> > +	spin_unlock_irqrestore(&req->lock, flags);
> > +
> > +	media_request_clean(req);
> > +
> > +	if (mdev->ops->req_free)
> > +		mdev->ops->req_free(req);
> > +	else
> > +		kfree(req);
> 
> Without looking at req_free() implementation, I would actually prefer
> to always do a kfree(req) here. So, a req_free() function would only
> free "extra" allocations, and not the request itself. e. g.:
> 
> ...
> 	if (mdev->ops->req_free)
> 		mdev->ops->req_free(req);
> 
> 	kfree(req);
> }

The idea is that you can embed the request object in a driver specific
struct. The drivers can store information related to that request rather
easily that way, without requiring to be aware of two objects with
references pointing to each other. I rather prefer the current
implementation. It same pattern is actually used on videobuf2 buffers.

...

> > +static unsigned int media_request_poll(struct file *filp,
> > +				       struct poll_table_struct *wait)
> > +{
> > +	struct media_request *req = filp->private_data;
> > +	unsigned long flags;
> > +	enum media_request_state state;
> > +
> > +	if (!(poll_requested_events(wait) & POLLPRI))
> > +		return 0;
> > +
> > +	spin_lock_irqsave(&req->lock, flags);
> > +	state = req->state;
> > +	spin_unlock_irqrestore(&req->lock, flags);
> 
> IMO, it would be better to use an atomic var for state, having a
> lockless access to it.

The lock serialises access to the whole request, not just to its state.
While it doesn't matter here as you're just reading the state, writing it
would still require taking the lock. Using atomic_t might suggest
otherwise, and could end up being a source of bugs.

> 
> > +
> > +	if (state == MEDIA_REQUEST_STATE_COMPLETE)
> > +		return POLLPRI;
> > +	if (state == MEDIA_REQUEST_STATE_IDLE)
> > +		return POLLERR;
> > +
> > +	poll_wait(filp, &req->poll_wait, wait);
> > +	return 0;
> > +}
> > +
> > +static long media_request_ioctl(struct file *filp, unsigned int cmd,
> > +				unsigned long __arg)
> > +{
> > +	return -ENOIOCTLCMD;
> > +}
> > +
> > +static const struct file_operations request_fops = {
> > +	.owner = THIS_MODULE,
> > +	.poll = media_request_poll,
> > +	.unlocked_ioctl = media_request_ioctl,
> > +	.release = media_request_close,
> > +};
> > +
> >  int media_request_alloc(struct media_device *mdev,
> >  			struct media_request_alloc *alloc)
> >  {
> > -	return -ENOMEM;
> > +	struct media_request *req;
> > +	struct file *filp;
> > +	char comm[TASK_COMM_LEN];
> > +	int fd;
> > +	int ret;
> > +
> > +	fd = get_unused_fd_flags(O_CLOEXEC);
> > +	if (fd < 0)
> > +		return fd;
> > +
> > +	filp = anon_inode_getfile("request", &request_fops, NULL, O_CLOEXEC);
> > +	if (IS_ERR(filp)) {
> > +		ret = PTR_ERR(filp);
> > +		goto err_put_fd;
> > +	}
> > +
> > +	if (mdev->ops->req_alloc)
> > +		req = mdev->ops->req_alloc(mdev);
> > +	else
> > +		req = kzalloc(sizeof(*req), GFP_KERNEL);
> > +	if (!req) {
> > +		ret = -ENOMEM;
> > +		goto err_fput;
> > +	}
> > +
> > +	filp->private_data = req;
> > +	req->mdev = mdev;
> > +	req->state = MEDIA_REQUEST_STATE_IDLE;
> > +	req->num_incomplete_objects = 0;
> > +	kref_init(&req->kref);
> > +	INIT_LIST_HEAD(&req->objects);
> > +	spin_lock_init(&req->lock);
> > +	init_waitqueue_head(&req->poll_wait);
> > +
> > +	alloc->fd = fd;
> 
> Btw, this is a very good reason why you should define the ioctl to
> have an integer argument instead of a struct with a __s32 field
> on it, as per my comment to patch 02/29:
> 
> 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> 
> At 64 bit architectures, you're truncating the file descriptor!

I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
well.

We actually replaced int by __s32 or __u32 everywhere in uAPI (apart from
one particular struct; time to send a patch) about five years ago.

> 
> > +	get_task_comm(comm, current);
> > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > +		 comm, fd);
> 
> Not sure if it is a good idea to store the task that allocated
> the request. While it makes sense for the dev_dbg() below, it
> may not make sense anymore on other dev_dbg() you would be
> using it.

The lifetime of the file handle roughly matches that of the request. It's
for debug only anyway.

Better proposals are always welcome of course. But I think we should have
something here that helps debugging by meaningfully making the requests
identifiable from logs.

...

> > +#ifdef CONFIG_MEDIA_CONTROLLER
> > +static inline void media_request_object_get(struct media_request_object *obj)
> > +{
> > +	kref_get(&obj->kref);
> > +}
> 
> Why do you need a function? Just use kref_get() were needed, instead of
> aliasing it for no good reason.

We have similar functions for get and put in many places in the kernel. If
this were a static function only used in a single .c file, I'd most likely
agree. Here it avoids having to know the internals of request objects
elsewhere.

I wonder if this could be moved to the .c file actually; I haven't looked
at the other patches.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 13/29] v4l2-ctrls: use ref in helper instead of ctrl

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:10 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> The next patch needs the reference to a control instead of the
> control itself, so change struct v4l2_ctrl_helper accordingly.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index 3c1b00baa8d0..da4cc1485dc4 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -37,8 +37,8 @@
>  struct v4l2_ctrl_helper {
>  	/* Pointer to the control reference of the master control */
>  	struct v4l2_ctrl_ref *mref;
> -	/* The control corresponding to the v4l2_ext_control ID field. */
> -	struct v4l2_ctrl *ctrl;

Why are you removing it in this patch? 

I mean: if this is something that it is not used, it should be 
removed on a separate one. Otherwise, please explain it at the
patch description.

> +	/* The control ref corresponding to the v4l2_ext_control ID field. */
> +	struct v4l2_ctrl_ref *ref;
>  	/* v4l2_ext_control index of the next control belonging to the
>  	   same cluster, or 0 if there isn't any. */
>  	u32 next;
> @@ -2887,6 +2887,7 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
>  		ref = find_ref_lock(hdl, id);
>  		if (ref == NULL)
>  			return -EINVAL;
> +		h->ref = ref;
>  		ctrl = ref->ctrl;
>  		if (ctrl->flags & V4L2_CTRL_FLAG_DISABLED)
>  			return -EINVAL;
> @@ -2909,7 +2910,6 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
>  		}
>  		/* Store the ref to the master control of the cluster */
>  		h->mref = ref;
> -		h->ctrl = ctrl;
>  		/* Initially set next to 0, meaning that there is no other
>  		   control in this helper array belonging to the same
>  		   cluster */
> @@ -2994,7 +2994,7 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
>  	cs->error_idx = cs->count;
>  
>  	for (i = 0; !ret && i < cs->count; i++)
> -		if (helpers[i].ctrl->flags & V4L2_CTRL_FLAG_WRITE_ONLY)
> +		if (helpers[i].ref->ctrl->flags & V4L2_CTRL_FLAG_WRITE_ONLY)

Shouldn't it be checking if ref is not NULL?

>  			ret = -EACCES;
>  
>  	for (i = 0; !ret && i < cs->count; i++) {
> @@ -3029,7 +3029,7 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
>  
>  			do {
>  				ret = ctrl_to_user(cs->controls + idx,
> -						   helpers[idx].ctrl);
> +						   helpers[idx].ref->ctrl);
>  				idx = helpers[idx].next;
>  			} while (!ret && idx);
>  		}
> @@ -3168,7 +3168,7 @@ static int validate_ctrls(struct v4l2_ext_controls *cs,
>  
>  	cs->error_idx = cs->count;
>  	for (i = 0; i < cs->count; i++) {
> -		struct v4l2_ctrl *ctrl = helpers[i].ctrl;
> +		struct v4l2_ctrl *ctrl = helpers[i].ref->ctrl;
>  		union v4l2_ctrl_ptr p_new;
>  
>  		cs->error_idx = i;
> @@ -3280,7 +3280,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
>  			do {
>  				/* Check if the auto control is part of the
>  				   list, and remember the new value. */
> -				if (helpers[tmp_idx].ctrl == master)
> +				if (helpers[tmp_idx].ref->ctrl == master)
>  					new_auto_val = cs->controls[tmp_idx].value;
>  				tmp_idx = helpers[tmp_idx].next;
>  			} while (tmp_idx);
> @@ -3293,7 +3293,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
>  		/* Copy the new caller-supplied control values.
>  		   user_to_new() sets 'is_new' to 1. */
>  		do {
> -			struct v4l2_ctrl *ctrl = helpers[idx].ctrl;
> +			struct v4l2_ctrl *ctrl = helpers[idx].ref->ctrl;
>  
>  			ret = user_to_new(cs->controls + idx, ctrl);
>  			if (!ret && ctrl->is_ptr)
> @@ -3309,7 +3309,7 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
>  			idx = i;
>  			do {
>  				ret = new_to_user(cs->controls + idx,
> -						helpers[idx].ctrl);
> +						helpers[idx].ref->ctrl);
>  				idx = helpers[idx].next;
>  			} while (!ret && idx);
>  		}



Thanks,
Mauro

Re: [RFCv11 PATCH 15/29] v4l2-ctrls: support g/s_ext_ctrls for requests

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:12 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> The v4l2_g/s_ext_ctrls functions now support control handlers that
> represent requests.
> 
> The v4l2_ctrls_find_req_obj() function is responsible for finding the
> request from the fd.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/omap3isp/ispvideo.c |   2 +-
>  drivers/media/v4l2-core/v4l2-ctrls.c       | 113 +++++++++++++++++++++++++++--
>  drivers/media/v4l2-core/v4l2-ioctl.c       |  12 +--
>  drivers/media/v4l2-core/v4l2-subdev.c      |   9 ++-
>  include/media/v4l2-ctrls.h                 |   7 +-
>  5 files changed, 124 insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
> index a751c89a3ea8..bd564c2e767f 100644
> --- a/drivers/media/platform/omap3isp/ispvideo.c
> +++ b/drivers/media/platform/omap3isp/ispvideo.c
> @@ -1028,7 +1028,7 @@ static int isp_video_check_external_subdevs(struct isp_video *video,
>  	ctrls.count = 1;
>  	ctrls.controls = &ctrl;
>  
> -	ret = v4l2_g_ext_ctrls(pipe->external->ctrl_handler, &ctrls);
> +	ret = v4l2_g_ext_ctrls(pipe->external->ctrl_handler, NULL, &ctrls);
>  	if (ret < 0) {
>  		dev_warn(isp->dev, "no pixel rate control in subdev %s\n",
>  			 pipe->external->name);
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index 6e2c5e24734f..317a374b6ea6 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -3122,7 +3122,8 @@ static int class_check(struct v4l2_ctrl_handler *hdl, u32 which)
>  }
>  
>  /* Get extended controls. Allocates the helpers array if needed. */
> -int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs)
> +int __v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl,
> +		       struct v4l2_ext_controls *cs)
>  {
>  	struct v4l2_ctrl_helper helper[4];
>  	struct v4l2_ctrl_helper *helpers = helper;
> @@ -3202,6 +3203,73 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs
>  		kvfree(helpers);
>  	return ret;
>  }
> +
> +static struct media_request_object *
> +v4l2_ctrls_find_req_obj(struct v4l2_ctrl_handler *hdl,
> +			struct media_device *mdev, s32 fd, bool set)
> +{
> +	struct media_request *req = media_request_find(mdev, fd);
> +	struct media_request_object *obj;
> +	struct v4l2_ctrl_handler *new_hdl;
> +	int ret;
> +
> +	if (IS_ERR(req))
> +		return ERR_CAST(req);
> +
> +	if (set && req->state != MEDIA_REQUEST_STATE_IDLE) {
> +		media_request_put(req);
> +		return ERR_PTR(-EBUSY);
> +	}
> +
> +	obj = media_request_object_find(req, &req_ops, hdl);
> +	if (obj) {
> +		media_request_put(req);
> +		return obj;
> +	}

This routine has lock issues: it access state directly and
doesn't use mutex_lock().

> +
> +	new_hdl = kzalloc(sizeof(*new_hdl), GFP_KERNEL);
> +	if (!new_hdl) {
> +		ret = -ENOMEM;
> +		goto put;
> +	}
> +	obj = &new_hdl->req_obj;
> +	ret = v4l2_ctrl_handler_init(new_hdl, (hdl->nr_of_buckets - 1) * 8);
> +	if (!ret)
> +		ret = v4l2_ctrl_request_bind(req, new_hdl, hdl);
> +	if (!ret) {
> +		media_request_object_get(obj);
> +		media_request_put(req);
> +		return obj;
> +	}
> +	kfree(new_hdl);
> +
> +put:
> +	media_request_put(req);
> +	return ERR_PTR(ret);
> +}
> +
> +int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
> +		     struct v4l2_ext_controls *cs)
> +{
> +	struct media_request_object *obj = NULL;
> +	int ret;
> +
> +	if (cs->which == V4L2_CTRL_WHICH_REQUEST) {
> +		if (!mdev || cs->request_fd < 0)
> +			return -EINVAL;
> +		obj = v4l2_ctrls_find_req_obj(hdl, mdev, cs->request_fd, false);
> +		if (IS_ERR(obj))
> +			return PTR_ERR(obj);
> +		hdl = container_of(obj, struct v4l2_ctrl_handler,
> +				   req_obj);
> +	}
> +
> +	ret = __v4l2_g_ext_ctrls(hdl, cs);
> +
> +	if (obj)
> +		media_request_object_put(obj);
> +	return ret;
> +}
>  EXPORT_SYMBOL(v4l2_g_ext_ctrls);
>  
>  /* Helper function to get a single control */
> @@ -3377,9 +3445,9 @@ static void update_from_auto_cluster(struct v4l2_ctrl *master)
>  }
>  
>  /* Try or try-and-set controls */
> -static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
> -			     struct v4l2_ext_controls *cs,
> -			     bool set)
> +static int __try_set_ext_ctrls(struct v4l2_fh *fh,
> +			       struct v4l2_ctrl_handler *hdl,
> +			       struct v4l2_ext_controls *cs, bool set)
>  {
>  	struct v4l2_ctrl_helper helper[4];
>  	struct v4l2_ctrl_helper *helpers = helper;
> @@ -3492,16 +3560,45 @@ static int try_set_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
>  	return ret;
>  }
>  
> -int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct v4l2_ext_controls *cs)
> +static int try_set_ext_ctrls(struct v4l2_fh *fh,
> +			     struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
> +			     struct v4l2_ext_controls *cs, bool set)
> +{
> +	struct media_request_object *obj = NULL;
> +	int ret;
> +
> +	if (cs->which == V4L2_CTRL_WHICH_REQUEST) {
> +		if (!mdev || cs->request_fd < 0)
> +			return -EINVAL;
> +		obj = v4l2_ctrls_find_req_obj(hdl, mdev, cs->request_fd, true);
> +		if (IS_ERR(obj))
> +			return PTR_ERR(obj);
> +		if (obj->req->state != MEDIA_REQUEST_STATE_IDLE) {
> +			media_request_object_put(obj);
> +			return -EBUSY;
> +		}
> +		hdl = container_of(obj, struct v4l2_ctrl_handler,
> +				   req_obj);
> +	}
> +
> +	ret = __try_set_ext_ctrls(fh, hdl, cs, set);
> +
> +	if (obj)
> +		media_request_object_put(obj);
> +	return ret;
> +}
> +
> +int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
> +		       struct v4l2_ext_controls *cs)
>  {
> -	return try_set_ext_ctrls(NULL, hdl, cs, false);
> +	return try_set_ext_ctrls(NULL, hdl, mdev, cs, false);
>  }
>  EXPORT_SYMBOL(v4l2_try_ext_ctrls);
>  
>  int v4l2_s_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
> -					struct v4l2_ext_controls *cs)
> +		     struct media_device *mdev, struct v4l2_ext_controls *cs)
>  {
> -	return try_set_ext_ctrls(fh, hdl, cs, true);
> +	return try_set_ext_ctrls(fh, hdl, mdev, cs, true);
>  }
>  EXPORT_SYMBOL(v4l2_s_ext_ctrls);
>  
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 9ce23e23c5bf..56741c4a48fc 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -2079,9 +2079,9 @@ static int v4l_g_ext_ctrls(const struct v4l2_ioctl_ops *ops,
>  
>  	p->error_idx = p->count;
>  	if (vfh && vfh->ctrl_handler)
> -		return v4l2_g_ext_ctrls(vfh->ctrl_handler, p);
> +		return v4l2_g_ext_ctrls(vfh->ctrl_handler, vfd->v4l2_dev->mdev, p);
>  	if (vfd->ctrl_handler)
> -		return v4l2_g_ext_ctrls(vfd->ctrl_handler, p);
> +		return v4l2_g_ext_ctrls(vfd->ctrl_handler, vfd->v4l2_dev->mdev, p);
>  	if (ops->vidioc_g_ext_ctrls == NULL)
>  		return -ENOTTY;
>  	return check_ext_ctrls(p, 0) ? ops->vidioc_g_ext_ctrls(file, fh, p) :
> @@ -2098,9 +2098,9 @@ static int v4l_s_ext_ctrls(const struct v4l2_ioctl_ops *ops,
>  
>  	p->error_idx = p->count;
>  	if (vfh && vfh->ctrl_handler)
> -		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler, p);
> +		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler, vfd->v4l2_dev->mdev, p);
>  	if (vfd->ctrl_handler)
> -		return v4l2_s_ext_ctrls(NULL, vfd->ctrl_handler, p);
> +		return v4l2_s_ext_ctrls(NULL, vfd->ctrl_handler, vfd->v4l2_dev->mdev, p);
>  	if (ops->vidioc_s_ext_ctrls == NULL)
>  		return -ENOTTY;
>  	return check_ext_ctrls(p, 0) ? ops->vidioc_s_ext_ctrls(file, fh, p) :
> @@ -2117,9 +2117,9 @@ static int v4l_try_ext_ctrls(const struct v4l2_ioctl_ops *ops,
>  
>  	p->error_idx = p->count;
>  	if (vfh && vfh->ctrl_handler)
> -		return v4l2_try_ext_ctrls(vfh->ctrl_handler, p);
> +		return v4l2_try_ext_ctrls(vfh->ctrl_handler, vfd->v4l2_dev->mdev, p);
>  	if (vfd->ctrl_handler)
> -		return v4l2_try_ext_ctrls(vfd->ctrl_handler, p);
> +		return v4l2_try_ext_ctrls(vfd->ctrl_handler, vfd->v4l2_dev->mdev, p);
>  	if (ops->vidioc_try_ext_ctrls == NULL)
>  		return -ENOTTY;
>  	return check_ext_ctrls(p, 0) ? ops->vidioc_try_ext_ctrls(file, fh, p) :
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index f9eed938d348..ce8c133e0ccd 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -222,17 +222,20 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
>  	case VIDIOC_G_EXT_CTRLS:
>  		if (!vfh->ctrl_handler)
>  			return -ENOTTY;
> -		return v4l2_g_ext_ctrls(vfh->ctrl_handler, arg);
> +		return v4l2_g_ext_ctrls(vfh->ctrl_handler,
> +					sd->v4l2_dev->mdev, arg);
>  
>  	case VIDIOC_S_EXT_CTRLS:
>  		if (!vfh->ctrl_handler)
>  			return -ENOTTY;
> -		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler, arg);
> +		return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler,
> +					sd->v4l2_dev->mdev, arg);
>  
>  	case VIDIOC_TRY_EXT_CTRLS:
>  		if (!vfh->ctrl_handler)
>  			return -ENOTTY;
> -		return v4l2_try_ext_ctrls(vfh->ctrl_handler, arg);
> +		return v4l2_try_ext_ctrls(vfh->ctrl_handler,
> +					  sd->v4l2_dev->mdev, arg);
>  
>  	case VIDIOC_DQEVENT:
>  		if (!(sd->flags & V4L2_SUBDEV_FL_HAS_EVENTS))
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 9499846aa1d1..be27835334a2 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -1148,11 +1148,12 @@ int v4l2_s_ctrl(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
>   *	:ref:`VIDIOC_G_EXT_CTRLS <vidioc_g_ext_ctrls>` ioctl
>   *
>   * @hdl: pointer to &struct v4l2_ctrl_handler
> + * @mdev: pointer to &struct media_device
>   * @c: pointer to &struct v4l2_ext_controls
>   *
>   * If hdl == NULL then they will all return -EINVAL.
>   */
> -int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl,
> +int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct media_device *mdev,
>  		     struct v4l2_ext_controls *c);
>  
>  /**
> @@ -1160,11 +1161,13 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl,
>   *	:ref:`VIDIOC_TRY_EXT_CTRLS <vidioc_g_ext_ctrls>` ioctl
>   *
>   * @hdl: pointer to &struct v4l2_ctrl_handler
> + * @mdev: pointer to &struct media_device
>   * @c: pointer to &struct v4l2_ext_controls
>   *
>   * If hdl == NULL then they will all return -EINVAL.
>   */
>  int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl,
> +		       struct media_device *mdev,
>  		       struct v4l2_ext_controls *c);
>  
>  /**
> @@ -1173,11 +1176,13 @@ int v4l2_try_ext_ctrls(struct v4l2_ctrl_handler *hdl,
>   *
>   * @fh: pointer to &struct v4l2_fh
>   * @hdl: pointer to &struct v4l2_ctrl_handler
> + * @mdev: pointer to &struct media_device
>   * @c: pointer to &struct v4l2_ext_controls
>   *
>   * If hdl == NULL then they will all return -EINVAL.
>   */
>  int v4l2_s_ext_ctrls(struct v4l2_fh *fh, struct v4l2_ctrl_handler *hdl,
> +		     struct media_device *mdev,
>  		     struct v4l2_ext_controls *c);
>  
>  /**



Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

Hi Tomasz and Hans,

On Tue, Apr 10, 2018 at 08:21:27AM +0000, Tomasz Figa wrote:
> > +void media_request_cancel(struct media_request *req)
> > +{
> > +       struct media_request_object *obj, *obj_safe;
> > +
> > +       if (req->state != MEDIA_REQUEST_STATE_QUEUED)
> > +               return;
> 
> I can see that media_request_release() guards the state change with
> req->lock. However we access the state here without holding the spinlock.

The driver is supposed to own a queued request so any serialisation is the
responsibility of the driver here. But I wouldn't mind taking a lock
either, it's safer.

> 
> Also, should we perhaps have MEDIA_REQUEST_STATE_CANCELLED (or maybe just
> reset to MEDIA_REQUEST_STATE_IDLE?), so that we can guard against calling
> this multiple times?
> 
> Or perhaps we're expecting this to be called with req_queue_mutex held?
> 
> > +
> > +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
> > +               if (obj->ops->cancel)
> > +                       obj->ops->cancel(obj);

Hans: how is cancel different from simply unbinding the object? I guess
we'll need precise definitions of what these ops mean.

Same question; should the request return to IDLE state here as well? Or an
error? I'd presume this is for a driver to cancel a request due to an error
of some sort.

...

> > +       spin_lock_irqsave(&req->lock, flags);
> > +       if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
> > +               goto unlock;
> > +       list_add_tail(&obj->list, &req->objects);
> > +       req->num_incomplete_objects++;
> > +unlock:
> > +       spin_unlock_irqrestore(&req->lock, flags);
> > +}
> > +EXPORT_SYMBOL_GPL(media_request_object_bind);
> > +
> > +void media_request_object_unbind(struct media_request_object *obj)
> > +{
> > +       struct media_request *req = obj->req;
> > +       unsigned long flags;
> > +       bool completed = false;
> > +
> > +       if (!req)
> > +               return;
> 
> No need for any locking here?

I wonder if WARN_ON() would be appropriate above.

Request objects, such as video buffers, can be bound and unbound multiple
times over the lifetime of the object. Video buffer state in this case
provides information whether the object is bound to a request (or not).

For objects the lifetime of which depends on the request only (controls?),
the bound request won't change until the object is unbound. The caller must
be aware of the state of the object, otherwise it could not meaningfully
unbind the object from a request.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 17/29] vb2: store userspace data in vb2_v4l2_buffer

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:14 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> The userspace-provided plane data needs to be stored in
> vb2_v4l2_buffer. Currently this information is applied by
> __fill_vb2_buffer() which is called by the core prepare_buf
> and qbuf functions, but when using requests these functions
> aren't called yet since the buffer won't be prepared until
> the media request is actually queued.
> 
> In the meantime this information has to be stored somewhere
> and vb2_v4l2_buffer is a good place for it.
> 
> The __fill_vb2_buffer callback now just copies the relevant
> information from vb2_v4l2_buffer into the planes array.

This patch is to hairy and do a lot more than what's
described. Please split it into one patch per change,
as otherwise we won't be able to properly review it.

> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c |  25 +-
>  drivers/media/common/videobuf2/videobuf2-v4l2.c | 324 +++++++++++++-----------
>  drivers/media/dvb-core/dvb_vb2.c                |   3 +-
>  include/media/videobuf2-core.h                  |   3 +-
>  include/media/videobuf2-v4l2.h                  |   2 +
>  5 files changed, 197 insertions(+), 160 deletions(-)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index d3f7bb33a54d..3d436ccb61f8 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -968,9 +968,8 @@ static int __prepare_mmap(struct vb2_buffer *vb, const void *pb)
>  {
>  	int ret = 0;
>  
> -	if (pb)
> -		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> -				 vb, pb, vb->planes);
> +	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> +			 vb, vb->planes);
>  	return ret ? ret : call_vb_qop(vb, buf_prepare, vb);
>  }
>  
> @@ -988,12 +987,10 @@ static int __prepare_userptr(struct vb2_buffer *vb, const void *pb)
>  
>  	memset(planes, 0, sizeof(planes[0]) * vb->num_planes);
>  	/* Copy relevant information provided by the userspace */
> -	if (pb) {
> -		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> -				 vb, pb, planes);
> -		if (ret)
> -			return ret;
> -	}
> +	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> +			 vb, planes);
> +	if (ret)
> +		return ret;
>  
>  	for (plane = 0; plane < vb->num_planes; ++plane) {
>  		/* Skip the plane if already verified */
> @@ -1104,12 +1101,10 @@ static int __prepare_dmabuf(struct vb2_buffer *vb, const void *pb)
>  
>  	memset(planes, 0, sizeof(planes[0]) * vb->num_planes);
>  	/* Copy relevant information provided by the userspace */
> -	if (pb) {
> -		ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> -				 vb, pb, planes);
> -		if (ret)
> -			return ret;
> -	}
> +	ret = call_bufop(vb->vb2_queue, fill_vb2_buffer,
> +			 vb, planes);
> +	if (ret)
> +		return ret;
>  
>  	for (plane = 0; plane < vb->num_planes; ++plane) {
>  		struct dma_buf *dbuf = dma_buf_get(planes[plane].m.fd);

On all the above: why are you removing the check if (pb)? 
Stripping a check like that is really scary!

I don't remember why those checks are/were needed, but the patch
description doesn't give any hint why this change is required, and,
on a quick glance, pb can be NULL, for instance when this is called
while setting buffers for file io, at __vb2_init_fileio():

	ret = vb2_core_qbuf(q, i, NULL);


	int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb)
	{
	...
	        switch (vb->state) {
        	case VB2_BUF_STATE_DEQUEUED:
                	ret = __buf_prepare(vb, pb);

If this is really needed, please do such change on a separate patch,
in order to make easier to verify if everything is covered.

> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 4e9c77f21858..bf7a3ba9fed0 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -154,9 +154,177 @@ static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
>  		pr_warn("use the actual size instead.\n");
>  }
>  
> +static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b)
> +{
> +	struct vb2_queue *q = vb->vb2_queue;
> +	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> +	struct vb2_plane *planes = vbuf->planes;
> +	unsigned int plane;
> +	int ret;
> +
> +	ret = __verify_length(vb, b);
> +	if (ret < 0) {
> +		dprintk(1, "plane parameters verification failed: %d\n", ret);
> +		return ret;
> +	}
> +	if (b->field == V4L2_FIELD_ALTERNATE && q->is_output) {
> +		/*
> +		 * If the format's field is ALTERNATE, then the buffer's field
> +		 * should be either TOP or BOTTOM, not ALTERNATE since that
> +		 * makes no sense. The driver has to know whether the
> +		 * buffer represents a top or a bottom field in order to
> +		 * program any DMA correctly. Using ALTERNATE is wrong, since
> +		 * that just says that it is either a top or a bottom field,
> +		 * but not which of the two it is.
> +		 */
> +		dprintk(1, "the field is incorrectly set to ALTERNATE for an output buffer\n");
> +		return -EINVAL;
> +	}
> +	vbuf->sequence = 0;
> +
> +	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
> +		switch (b->memory) {
> +		case VB2_MEMORY_USERPTR:
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				planes[plane].m.userptr =
> +					b->m.planes[plane].m.userptr;
> +				planes[plane].length =
> +					b->m.planes[plane].length;
> +			}
> +			break;
> +		case VB2_MEMORY_DMABUF:
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				planes[plane].m.fd =
> +					b->m.planes[plane].m.fd;
> +				planes[plane].length =
> +					b->m.planes[plane].length;
> +			}
> +			break;
> +		default:
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				planes[plane].m.offset =
> +					vb->planes[plane].m.offset;
> +				planes[plane].length =
> +					vb->planes[plane].length;
> +			}
> +			break;
> +		}
> +
> +		/* Fill in driver-provided information for OUTPUT types */
> +		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> +			/*
> +			 * Will have to go up to b->length when API starts
> +			 * accepting variable number of planes.
> +			 *
> +			 * If bytesused == 0 for the output buffer, then fall
> +			 * back to the full buffer size. In that case
> +			 * userspace clearly never bothered to set it and
> +			 * it's a safe assumption that they really meant to
> +			 * use the full plane sizes.
> +			 *
> +			 * Some drivers, e.g. old codec drivers, use bytesused == 0
> +			 * as a way to indicate that streaming is finished.
> +			 * In that case, the driver should use the
> +			 * allow_zero_bytesused flag to keep old userspace
> +			 * applications working.
> +			 */
> +			for (plane = 0; plane < vb->num_planes; ++plane) {
> +				struct vb2_plane *pdst = &planes[plane];
> +				struct v4l2_plane *psrc = &b->m.planes[plane];
> +
> +				if (psrc->bytesused == 0)
> +					vb2_warn_zero_bytesused(vb);
> +
> +				if (vb->vb2_queue->allow_zero_bytesused)
> +					pdst->bytesused = psrc->bytesused;
> +				else
> +					pdst->bytesused = psrc->bytesused ?
> +						psrc->bytesused : pdst->length;
> +				pdst->data_offset = psrc->data_offset;
> +			}
> +		}
> +	} else {
> +		/*
> +		 * Single-planar buffers do not use planes array,
> +		 * so fill in relevant v4l2_buffer struct fields instead.
> +		 * In videobuf we use our internal V4l2_planes struct for
> +		 * single-planar buffers as well, for simplicity.
> +		 *
> +		 * If bytesused == 0 for the output buffer, then fall back
> +		 * to the full buffer size as that's a sensible default.
> +		 *
> +		 * Some drivers, e.g. old codec drivers, use bytesused == 0 as
> +		 * a way to indicate that streaming is finished. In that case,
> +		 * the driver should use the allow_zero_bytesused flag to keep
> +		 * old userspace applications working.
> +		 */
> +		switch (b->memory) {
> +		case VB2_MEMORY_USERPTR:
> +			planes[0].m.userptr = b->m.userptr;
> +			planes[0].length = b->length;
> +			break;
> +		case VB2_MEMORY_DMABUF:
> +			planes[0].m.fd = b->m.fd;
> +			planes[0].length = b->length;
> +			break;
> +		default:
> +			planes[0].m.offset = vb->planes[0].m.offset;
> +			planes[0].length = vb->planes[0].length;
> +			break;
> +		}
> +
> +		planes[0].data_offset = 0;
> +		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> +			if (b->bytesused == 0)
> +				vb2_warn_zero_bytesused(vb);
> +
> +			if (vb->vb2_queue->allow_zero_bytesused)
> +				planes[0].bytesused = b->bytesused;
> +			else
> +				planes[0].bytesused = b->bytesused ?
> +					b->bytesused : planes[0].length;
> +		} else
> +			planes[0].bytesused = 0;
> +
> +	}
> +
> +	/* Zero flags that we handle */
> +	vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
> +	if (!vb->vb2_queue->copy_timestamp || !V4L2_TYPE_IS_OUTPUT(b->type)) {
> +		/*
> +		 * Non-COPY timestamps and non-OUTPUT queues will get
> +		 * their timestamp and timestamp source flags from the
> +		 * queue.
> +		 */
> +		vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
> +	}
> +
> +	if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> +		/*
> +		 * For output buffers mask out the timecode flag:
> +		 * this will be handled later in vb2_qbuf().
> +		 * The 'field' is valid metadata for this output buffer
> +		 * and so that needs to be copied here.
> +		 */
> +		vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
> +		vbuf->field = b->field;
> +	} else {
> +		/* Zero any output buffer flags as this is a capture buffer */
> +		vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
> +		/* Zero last flag, this is a signal from driver to userspace */
> +		vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
> +	}
> +
> +	return 0;
> +}
> +

As I said at the beginning, just this change, with apparently moves part of
the logic from __fill_vb2_buffer() into a separate function should be done
in a separate patch, and then a followup patch doing the required changes.

That would help us to check what changed in this part of the code 
were changed due to the request API needs.

>  static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
>  				    const char *opname)
>  {
> +	struct vb2_v4l2_buffer *vbuf;
> +	struct vb2_buffer *vb;
> +	int ret;
> +
>  	if (b->type != q->type) {
>  		dprintk(1, "%s: invalid buffer type\n", opname);
>  		return -EINVAL;
> @@ -178,7 +346,15 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
>  		return -EINVAL;
>  	}
>  
> -	return __verify_planes_array(q->bufs[b->index], b);
> +	vb = q->bufs[b->index];
> +	vbuf = to_vb2_v4l2_buffer(vb);
> +	ret = __verify_planes_array(vb, b);
> +	if (ret)
> +		return ret;
> +
> +	/* Copy relevant information provided by the userspace */
> +	memset(vbuf->planes, 0, sizeof(vbuf->planes[0]) * vb->num_planes);
> +	return vb2_fill_vb2_v4l2_buffer(vb, b);
>  }
>  
>  /*
> @@ -291,153 +467,19 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
>   * v4l2_buffer by the userspace. It also verifies that struct
>   * v4l2_buffer has a valid number of planes.
>   */
> -static int __fill_vb2_buffer(struct vb2_buffer *vb,
> -		const void *pb, struct vb2_plane *planes)
> +static int __fill_vb2_buffer(struct vb2_buffer *vb, struct vb2_plane *planes)
>  {
> -	struct vb2_queue *q = vb->vb2_queue;
> -	const struct v4l2_buffer *b = pb;
>  	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
>  	unsigned int plane;
> -	int ret;
>  
> -	ret = __verify_length(vb, b);
> -	if (ret < 0) {
> -		dprintk(1, "plane parameters verification failed: %d\n", ret);
> -		return ret;
> -	}
> -	if (b->field == V4L2_FIELD_ALTERNATE && q->is_output) {
> -		/*
> -		 * If the format's field is ALTERNATE, then the buffer's field
> -		 * should be either TOP or BOTTOM, not ALTERNATE since that
> -		 * makes no sense. The driver has to know whether the
> -		 * buffer represents a top or a bottom field in order to
> -		 * program any DMA correctly. Using ALTERNATE is wrong, since
> -		 * that just says that it is either a top or a bottom field,
> -		 * but not which of the two it is.
> -		 */
> -		dprintk(1, "the field is incorrectly set to ALTERNATE for an output buffer\n");
> -		return -EINVAL;
> -	}
>  	vb->timestamp = 0;
> -	vbuf->sequence = 0;
> -
> -	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
> -		if (b->memory == VB2_MEMORY_USERPTR) {
> -			for (plane = 0; plane < vb->num_planes; ++plane) {
> -				planes[plane].m.userptr =
> -					b->m.planes[plane].m.userptr;
> -				planes[plane].length =
> -					b->m.planes[plane].length;
> -			}
> -		}
> -		if (b->memory == VB2_MEMORY_DMABUF) {
> -			for (plane = 0; plane < vb->num_planes; ++plane) {
> -				planes[plane].m.fd =
> -					b->m.planes[plane].m.fd;
> -				planes[plane].length =
> -					b->m.planes[plane].length;
> -			}
> -		}
> -
> -		/* Fill in driver-provided information for OUTPUT types */
> -		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> -			/*
> -			 * Will have to go up to b->length when API starts
> -			 * accepting variable number of planes.
> -			 *
> -			 * If bytesused == 0 for the output buffer, then fall
> -			 * back to the full buffer size. In that case
> -			 * userspace clearly never bothered to set it and
> -			 * it's a safe assumption that they really meant to
> -			 * use the full plane sizes.
> -			 *
> -			 * Some drivers, e.g. old codec drivers, use bytesused == 0
> -			 * as a way to indicate that streaming is finished.
> -			 * In that case, the driver should use the
> -			 * allow_zero_bytesused flag to keep old userspace
> -			 * applications working.
> -			 */
> -			for (plane = 0; plane < vb->num_planes; ++plane) {
> -				struct vb2_plane *pdst = &planes[plane];
> -				struct v4l2_plane *psrc = &b->m.planes[plane];
> -
> -				if (psrc->bytesused == 0)
> -					vb2_warn_zero_bytesused(vb);
>  
> -				if (vb->vb2_queue->allow_zero_bytesused)
> -					pdst->bytesused = psrc->bytesused;
> -				else
> -					pdst->bytesused = psrc->bytesused ?
> -						psrc->bytesused : pdst->length;
> -				pdst->data_offset = psrc->data_offset;
> -			}
> -		}
> -	} else {
> -		/*
> -		 * Single-planar buffers do not use planes array,
> -		 * so fill in relevant v4l2_buffer struct fields instead.
> -		 * In videobuf we use our internal V4l2_planes struct for
> -		 * single-planar buffers as well, for simplicity.
> -		 *
> -		 * If bytesused == 0 for the output buffer, then fall back
> -		 * to the full buffer size as that's a sensible default.
> -		 *
> -		 * Some drivers, e.g. old codec drivers, use bytesused == 0 as
> -		 * a way to indicate that streaming is finished. In that case,
> -		 * the driver should use the allow_zero_bytesused flag to keep
> -		 * old userspace applications working.
> -		 */
> -		if (b->memory == VB2_MEMORY_USERPTR) {
> -			planes[0].m.userptr = b->m.userptr;
> -			planes[0].length = b->length;
> -		}
> -
> -		if (b->memory == VB2_MEMORY_DMABUF) {
> -			planes[0].m.fd = b->m.fd;
> -			planes[0].length = b->length;
> -		}
> -
> -		if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> -			if (b->bytesused == 0)
> -				vb2_warn_zero_bytesused(vb);
> -
> -			if (vb->vb2_queue->allow_zero_bytesused)
> -				planes[0].bytesused = b->bytesused;
> -			else
> -				planes[0].bytesused = b->bytesused ?
> -					b->bytesused : planes[0].length;
> -		} else
> -			planes[0].bytesused = 0;
> -
> -	}
> -
> -	/* Zero flags that the vb2 core handles */
> -	vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
> -	if (!vb->vb2_queue->copy_timestamp || !V4L2_TYPE_IS_OUTPUT(b->type)) {
> -		/*
> -		 * Non-COPY timestamps and non-OUTPUT queues will get
> -		 * their timestamp and timestamp source flags from the
> -		 * queue.
> -		 */
> -		vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
> +	for (plane = 0; plane < vb->num_planes; ++plane) {
> +		planes[plane].m = vbuf->planes[plane].m;
> +		planes[plane].length = vbuf->planes[plane].length;
> +		planes[plane].bytesused = vbuf->planes[plane].bytesused;
> +		planes[plane].data_offset = vbuf->planes[plane].data_offset;
>  	}
> -
> -	if (V4L2_TYPE_IS_OUTPUT(b->type)) {
> -		/*
> -		 * For output buffers mask out the timecode flag:
> -		 * this will be handled later in vb2_qbuf().
> -		 * The 'field' is valid metadata for this output buffer
> -		 * and so that needs to be copied here.
> -		 */
> -		vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
> -		vbuf->field = b->field;
> -	} else {
> -		/* Zero any output buffer flags as this is a capture buffer */
> -		vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
> -		/* Zero last flag, this is a signal from driver to userspace */
> -		vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
> -	}
> -
>  	return 0;
>  }

I got lost with that huge code churn that started with the addition of
a new vb2_fill_vb2_v4l2_buffer() function. Is it replacing __fill_vb2_buffer()?
Why?

Please rework on it in order to minimize the diffstat, as it is very hard
to see if the code is an exact copy or if something else got changed.

> diff --git a/drivers/media/dvb-core/dvb_vb2.c b/drivers/media/dvb-core/dvb_vb2.c
> index b811adf88afa..da6a8cec7d42 100644
> --- a/drivers/media/dvb-core/dvb_vb2.c
> +++ b/drivers/media/dvb-core/dvb_vb2.c
> @@ -146,8 +146,7 @@ static void _fill_dmx_buffer(struct vb2_buffer *vb, void *pb)
>  	dprintk(3, "[%s]\n", ctx->name);
>  }
>  
> -static int _fill_vb2_buffer(struct vb2_buffer *vb,
> -			    const void *pb, struct vb2_plane *planes)
> +static int _fill_vb2_buffer(struct vb2_buffer *vb, struct vb2_plane *planes)
>  {
>  	struct dvb_vb2_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
>  
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index f6818f732f34..224c4820a044 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -417,8 +417,7 @@ struct vb2_ops {
>  struct vb2_buf_ops {
>  	int (*verify_planes_array)(struct vb2_buffer *vb, const void *pb);
>  	void (*fill_user_buffer)(struct vb2_buffer *vb, void *pb);
> -	int (*fill_vb2_buffer)(struct vb2_buffer *vb, const void *pb,
> -				struct vb2_plane *planes);
> +	int (*fill_vb2_buffer)(struct vb2_buffer *vb, struct vb2_plane *planes);

Hmm... what happens if pb is NULL - with can happen for read()?

>  	void (*copy_timestamp)(struct vb2_buffer *vb, const void *pb);
>  };
>  
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 3d5e2d739f05..097bf3e6951d 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -32,6 +32,7 @@
>   *		&enum v4l2_field.
>   * @timecode:	frame timecode.
>   * @sequence:	sequence count of this frame.
> + * @planes:	plane information (userptr/fd, length, bytesused, data_offset).
>   *
>   * Should contain enough information to be able to cover all the fields
>   * of &struct v4l2_buffer at ``videodev2.h``.
> @@ -43,6 +44,7 @@ struct vb2_v4l2_buffer {
>  	__u32			field;
>  	struct v4l2_timecode	timecode;
>  	__u32			sequence;
> +	struct vb2_plane	planes[VB2_MAX_PLANES];
>  };
>  
>  /*



Thanks,
Mauro

Re: [RFCv11 PATCH 20/29] videobuf2-v4l2: integrate with media requests

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:17 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> This implements the V4L2 part of the request support. The main
> change is that vb2_qbuf and vb2_prepare_buf now have a new
> media_device pointer. This required changes to several drivers
> that did not use the vb2_ioctl_qbuf/prepare_buf helper functions.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-v4l2.c  | 84 ++++++++++++++++++++----
>  drivers/media/platform/omap3isp/ispvideo.c       |  2 +-
>  drivers/media/platform/s3c-camif/camif-capture.c |  4 +-
>  drivers/media/platform/s5p-mfc/s5p_mfc_dec.c     |  4 +-
>  drivers/media/platform/s5p-mfc/s5p_mfc_enc.c     |  4 +-
>  drivers/media/platform/soc_camera/soc_camera.c   |  4 +-
>  drivers/media/usb/uvc/uvc_queue.c                |  5 +-
>  drivers/media/usb/uvc/uvc_v4l2.c                 |  3 +-
>  drivers/media/usb/uvc/uvcvideo.h                 |  1 +
>  drivers/media/v4l2-core/v4l2-mem2mem.c           |  7 +-
>  drivers/staging/media/davinci_vpfe/vpfe_video.c  |  3 +-
>  drivers/staging/media/omap4iss/iss_video.c       |  3 +-
>  drivers/usb/gadget/function/uvc_queue.c          |  2 +-
>  include/media/videobuf2-v4l2.h                   | 12 +++-
>  14 files changed, 106 insertions(+), 32 deletions(-)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index b8d370b97cca..73c1fd4da58a 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -25,6 +25,7 @@
>  #include <linux/kthread.h>
>  
>  #include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
>  #include <media/v4l2-fh.h>
>  #include <media/v4l2-event.h>
>  #include <media/v4l2-common.h>
> @@ -40,10 +41,12 @@ module_param(debug, int, 0644);
>  			pr_info("vb2-v4l2: %s: " fmt, __func__, ## arg); \
>  	} while (0)
>  
> -/* Flags that are set by the vb2 core */
> +/* Flags that are set by us */
>  #define V4L2_BUFFER_MASK_FLAGS	(V4L2_BUF_FLAG_MAPPED | V4L2_BUF_FLAG_QUEUED | \
>  				 V4L2_BUF_FLAG_DONE | V4L2_BUF_FLAG_ERROR | \
>  				 V4L2_BUF_FLAG_PREPARED | \
> +				 V4L2_BUF_FLAG_IN_REQUEST | \
> +				 V4L2_BUF_FLAG_REQUEST_FD | \
>  				 V4L2_BUF_FLAG_TIMESTAMP_MASK)
>  /* Output buffer flags that should be passed on to the driver */
>  #define V4L2_BUFFER_OUT_FLAGS	(V4L2_BUF_FLAG_PFRAME | V4L2_BUF_FLAG_BFRAME | \
> @@ -318,13 +321,17 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
>  	return 0;
>  }
>  
> -static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
> -				    const char *opname)
> +static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> +				    struct v4l2_buffer *b,
> +				    const char *opname,
> +				    struct media_request **p_req)
>  {
> +	struct media_request *req;
>  	struct vb2_v4l2_buffer *vbuf;
>  	struct vb2_buffer *vb;
>  	int ret;
>  
> +	*p_req = NULL;
>  	if (b->type != q->type) {
>  		dprintk(1, "%s: invalid buffer type\n", opname);
>  		return -EINVAL;
> @@ -354,7 +361,38 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b,
>  
>  	/* Copy relevant information provided by the userspace */
>  	memset(vbuf->planes, 0, sizeof(vbuf->planes[0]) * vb->num_planes);
> -	return vb2_fill_vb2_v4l2_buffer(vb, b);
> +	ret = vb2_fill_vb2_v4l2_buffer(vb, b);
> +	if (ret)
> +		return ret;
> +
> +	if (!(b->flags & V4L2_BUF_FLAG_REQUEST_FD))
> +		return 0;
> +
> +	if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> +		dprintk(1, "%s: buffer is not in dequeued state\n", opname);
> +		return -EINVAL;
> +	}
> +
> +	if (b->request_fd < 0) {
> +		dprintk(1, "%s: request_fd < 0\n", opname);
> +		return -EINVAL;
> +	}
> +
> +	req = media_request_find(mdev, b->request_fd);
> +	if (IS_ERR(req)) {
> +		dprintk(1, "%s: invalid request_fd\n", opname);
> +		return PTR_ERR(req);
> +	}
> +
> +	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
> +		dprintk(1, "%s: request is not idle\n", opname);
> +		media_request_put(req);
> +		return -EBUSY;
> +	}

It is accessing req->state without locking. As I said before,
IMHO, it should really be an atomic var.

Also, on the changes introduced in this patch, it should be
doing the proper lock before using req (The way it is, it should
be locking the mutex - except that it won't work at IRQ),
or the entire logic should switch to some other locking type
or to use something like RCU.

> +
> +	*p_req = req;
> +
> +	return 0;
>  }
>  
>  /*
> @@ -437,6 +475,9 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
>  	case VB2_BUF_STATE_ACTIVE:
>  		b->flags |= V4L2_BUF_FLAG_QUEUED;
>  		break;
> +	case VB2_BUF_STATE_IN_REQUEST:
> +		b->flags |= V4L2_BUF_FLAG_IN_REQUEST;
> +		break;
>  	case VB2_BUF_STATE_ERROR:
>  		b->flags |= V4L2_BUF_FLAG_ERROR;
>  		/* fall through */
> @@ -455,6 +496,10 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
>  
>  	if (vb2_buffer_in_use(q, vb))
>  		b->flags |= V4L2_BUF_FLAG_MAPPED;
> +	if (vb->req_obj.req) {
> +		b->flags |= V4L2_BUF_FLAG_REQUEST_FD;
> +		b->request_fd = -1;
> +	}
>  
>  	if (!q->is_output &&
>  		b->flags & V4L2_BUF_FLAG_DONE &&
> @@ -533,8 +578,10 @@ int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_reqbufs);
>  
> -int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> +int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> +		    struct v4l2_buffer *b)
>  {
> +	struct media_request *req;
>  	int ret;
>  
>  	if (vb2_fileio_is_active(q)) {
> @@ -542,9 +589,13 @@ int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>  		return -EBUSY;
>  	}
>  
> -	ret = vb2_queue_or_prepare_buf(q, b, "prepare_buf");
> -
> -	return ret ? ret : vb2_core_prepare_buf(q, b->index, b, NULL);
> +	ret = vb2_queue_or_prepare_buf(q, mdev, b, "prepare_buf", &req);
> +	if (ret)
> +		return ret;
> +	ret = vb2_core_prepare_buf(q, b->index, b, req);
> +	if (req)
> +		media_request_put(req);
> +	return ret;
>  }
>  EXPORT_SYMBOL_GPL(vb2_prepare_buf);
>  
> @@ -602,8 +653,10 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create)
>  }
>  EXPORT_SYMBOL_GPL(vb2_create_bufs);
>  
> -int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
> +int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
> +	     struct v4l2_buffer *b)
>  {
> +	struct media_request *req;
>  	int ret;
>  
>  	if (vb2_fileio_is_active(q)) {
> @@ -611,8 +664,13 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
>  		return -EBUSY;
>  	}
>  
> -	ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
> -	return ret ? ret : vb2_core_qbuf(q, b->index, b, NULL);
> +	ret = vb2_queue_or_prepare_buf(q, mdev, b, "qbuf", &req);
> +	if (ret)
> +		return ret;
> +	ret = vb2_core_qbuf(q, b->index, b, req);
> +	if (req)
> +		media_request_put(req);
> +	return ret;
>  }
>  EXPORT_SYMBOL_GPL(vb2_qbuf);
>  
> @@ -802,7 +860,7 @@ int vb2_ioctl_prepare_buf(struct file *file, void *priv,
>  
>  	if (vb2_queue_is_busy(vdev, file))
>  		return -EBUSY;
> -	return vb2_prepare_buf(vdev->queue, p);
> +	return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p);
>  }
>  EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf);
>  
> @@ -821,7 +879,7 @@ int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p)
>  
>  	if (vb2_queue_is_busy(vdev, file))
>  		return -EBUSY;
> -	return vb2_qbuf(vdev->queue, p);
> +	return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p);
>  }
>  EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf);
>  
> diff --git a/drivers/media/platform/omap3isp/ispvideo.c b/drivers/media/platform/omap3isp/ispvideo.c
> index bd564c2e767f..9fecbd8c6edd 100644
> --- a/drivers/media/platform/omap3isp/ispvideo.c
> +++ b/drivers/media/platform/omap3isp/ispvideo.c
> @@ -940,7 +940,7 @@ isp_video_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
>  	int ret;
>  
>  	mutex_lock(&video->queue_lock);
> -	ret = vb2_qbuf(&vfh->queue, b);
> +	ret = vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
>  	mutex_unlock(&video->queue_lock);
>  
>  	return ret;
> diff --git a/drivers/media/platform/s3c-camif/camif-capture.c b/drivers/media/platform/s3c-camif/camif-capture.c
> index 9ab8e7ee2e1e..fafb6da3e804 100644
> --- a/drivers/media/platform/s3c-camif/camif-capture.c
> +++ b/drivers/media/platform/s3c-camif/camif-capture.c
> @@ -941,7 +941,7 @@ static int s3c_camif_qbuf(struct file *file, void *priv,
>  	if (vp->owner && vp->owner != priv)
>  		return -EBUSY;
>  
> -	return vb2_qbuf(&vp->vb_queue, buf);
> +	return vb2_qbuf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, buf);
>  }
>  
>  static int s3c_camif_dqbuf(struct file *file, void *priv,
> @@ -979,7 +979,7 @@ static int s3c_camif_prepare_buf(struct file *file, void *priv,
>  				 struct v4l2_buffer *b)
>  {
>  	struct camif_vp *vp = video_drvdata(file);
> -	return vb2_prepare_buf(&vp->vb_queue, b);
> +	return vb2_prepare_buf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, b);
>  }
>  
>  static int s3c_camif_g_selection(struct file *file, void *priv,
> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
> index 5cf4d9921264..3d863e3f5798 100644
> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
> @@ -632,9 +632,9 @@ static int vidioc_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
>  		return -EIO;
>  	}
>  	if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> -		return vb2_qbuf(&ctx->vq_src, buf);
> +		return vb2_qbuf(&ctx->vq_src, NULL, buf);
>  	else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> -		return vb2_qbuf(&ctx->vq_dst, buf);
> +		return vb2_qbuf(&ctx->vq_dst, NULL, buf);
>  	return -EINVAL;
>  }
>  
> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> index 5c0462ca9993..ed12d5d062e8 100644
> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> @@ -1621,9 +1621,9 @@ static int vidioc_qbuf(struct file *file, void *priv, struct v4l2_buffer *buf)
>  			mfc_err("Call on QBUF after EOS command\n");
>  			return -EIO;
>  		}
> -		return vb2_qbuf(&ctx->vq_src, buf);
> +		return vb2_qbuf(&ctx->vq_src, NULL, buf);
>  	} else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> -		return vb2_qbuf(&ctx->vq_dst, buf);
> +		return vb2_qbuf(&ctx->vq_dst, NULL, buf);
>  	}
>  	return -EINVAL;
>  }
> diff --git a/drivers/media/platform/soc_camera/soc_camera.c b/drivers/media/platform/soc_camera/soc_camera.c
> index e6787abc34ae..08adf9a79420 100644
> --- a/drivers/media/platform/soc_camera/soc_camera.c
> +++ b/drivers/media/platform/soc_camera/soc_camera.c
> @@ -394,7 +394,7 @@ static int soc_camera_qbuf(struct file *file, void *priv,
>  	if (icd->streamer != file)
>  		return -EBUSY;
>  
> -	return vb2_qbuf(&icd->vb2_vidq, p);
> +	return vb2_qbuf(&icd->vb2_vidq, NULL, p);
>  }
>  
>  static int soc_camera_dqbuf(struct file *file, void *priv,
> @@ -430,7 +430,7 @@ static int soc_camera_prepare_buf(struct file *file, void *priv,
>  {
>  	struct soc_camera_device *icd = file->private_data;
>  
> -	return vb2_prepare_buf(&icd->vb2_vidq, b);
> +	return vb2_prepare_buf(&icd->vb2_vidq, NULL, b);
>  }
>  
>  static int soc_camera_expbuf(struct file *file, void *priv,
> diff --git a/drivers/media/usb/uvc/uvc_queue.c b/drivers/media/usb/uvc/uvc_queue.c
> index fecccb5e7628..8964e16f2b22 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c
> @@ -300,12 +300,13 @@ int uvc_create_buffers(struct uvc_video_queue *queue,
>  	return ret;
>  }
>  
> -int uvc_queue_buffer(struct uvc_video_queue *queue, struct v4l2_buffer *buf)
> +int uvc_queue_buffer(struct uvc_video_queue *queue,
> +		     struct media_device *mdev, struct v4l2_buffer *buf)
>  {
>  	int ret;
>  
>  	mutex_lock(&queue->mutex);
> -	ret = vb2_qbuf(&queue->queue, buf);
> +	ret = vb2_qbuf(&queue->queue, mdev, buf);
>  	mutex_unlock(&queue->mutex);
>  
>  	return ret;
> diff --git a/drivers/media/usb/uvc/uvc_v4l2.c b/drivers/media/usb/uvc/uvc_v4l2.c
> index bd32914259ae..3da5fdc002ac 100644
> --- a/drivers/media/usb/uvc/uvc_v4l2.c
> +++ b/drivers/media/usb/uvc/uvc_v4l2.c
> @@ -751,7 +751,8 @@ static int uvc_ioctl_qbuf(struct file *file, void *fh, struct v4l2_buffer *buf)
>  	if (!uvc_has_privileges(handle))
>  		return -EBUSY;
>  
> -	return uvc_queue_buffer(&stream->queue, buf);
> +	return uvc_queue_buffer(&stream->queue,
> +				stream->vdev.v4l2_dev->mdev, buf);
>  }
>  
>  static int uvc_ioctl_expbuf(struct file *file, void *fh,
> diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
> index be5cf179228b..bc9ed18f043c 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -680,6 +680,7 @@ int uvc_query_buffer(struct uvc_video_queue *queue,
>  int uvc_create_buffers(struct uvc_video_queue *queue,
>  		       struct v4l2_create_buffers *v4l2_cb);
>  int uvc_queue_buffer(struct uvc_video_queue *queue,
> +		     struct media_device *mdev,
>  		     struct v4l2_buffer *v4l2_buf);
>  int uvc_export_buffer(struct uvc_video_queue *queue,
>  		      struct v4l2_exportbuffer *exp);
> diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
> index c4f963d96a79..438f1b869319 100644
> --- a/drivers/media/v4l2-core/v4l2-mem2mem.c
> +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
> @@ -20,6 +20,7 @@
>  #include <media/videobuf2-v4l2.h>
>  #include <media/v4l2-mem2mem.h>
>  #include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
>  #include <media/v4l2-fh.h>
>  #include <media/v4l2-event.h>
>  
> @@ -388,11 +389,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_querybuf);
>  int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>  		  struct v4l2_buffer *buf)
>  {
> +	struct video_device *vdev = video_devdata(file);
>  	struct vb2_queue *vq;
>  	int ret;
>  
>  	vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
> -	ret = vb2_qbuf(vq, buf);
> +	ret = vb2_qbuf(vq, vdev->v4l2_dev->mdev, buf);
>  	if (!ret)
>  		v4l2_m2m_try_schedule(m2m_ctx);
>  
> @@ -413,11 +415,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_dqbuf);
>  int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>  			 struct v4l2_buffer *buf)
>  {
> +	struct video_device *vdev = video_devdata(file);
>  	struct vb2_queue *vq;
>  	int ret;
>  
>  	vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
> -	ret = vb2_prepare_buf(vq, buf);
> +	ret = vb2_prepare_buf(vq, vdev->v4l2_dev->mdev, buf);
>  	if (!ret)
>  		v4l2_m2m_try_schedule(m2m_ctx);
>  
> diff --git a/drivers/staging/media/davinci_vpfe/vpfe_video.c b/drivers/staging/media/davinci_vpfe/vpfe_video.c
> index 588743a6fd8a..00bf28e830d4 100644
> --- a/drivers/staging/media/davinci_vpfe/vpfe_video.c
> +++ b/drivers/staging/media/davinci_vpfe/vpfe_video.c
> @@ -1426,7 +1426,8 @@ static int vpfe_qbuf(struct file *file, void *priv,
>  		return -EACCES;
>  	}
>  
> -	return vb2_qbuf(&video->buffer_queue, p);
> +	return vb2_qbuf(&video->buffer_queue,
> +			video->video_dev.v4l2_dev->mdev, p);
>  }
>  
>  /*
> diff --git a/drivers/staging/media/omap4iss/iss_video.c b/drivers/staging/media/omap4iss/iss_video.c
> index a3a83424a926..a35d1004b522 100644
> --- a/drivers/staging/media/omap4iss/iss_video.c
> +++ b/drivers/staging/media/omap4iss/iss_video.c
> @@ -805,9 +805,10 @@ iss_video_querybuf(struct file *file, void *fh, struct v4l2_buffer *b)
>  static int
>  iss_video_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
>  {
> +	struct iss_video *video = video_drvdata(file);
>  	struct iss_video_fh *vfh = to_iss_video_fh(fh);
>  
> -	return vb2_qbuf(&vfh->queue, b);
> +	return vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
>  }
>  
>  static int
> diff --git a/drivers/usb/gadget/function/uvc_queue.c b/drivers/usb/gadget/function/uvc_queue.c
> index 9e33d5206d54..f2497cb96abb 100644
> --- a/drivers/usb/gadget/function/uvc_queue.c
> +++ b/drivers/usb/gadget/function/uvc_queue.c
> @@ -166,7 +166,7 @@ int uvcg_queue_buffer(struct uvc_video_queue *queue, struct v4l2_buffer *buf)
>  	unsigned long flags;
>  	int ret;
>  
> -	ret = vb2_qbuf(&queue->queue, buf);
> +	ret = vb2_qbuf(&queue->queue, NULL, buf);
>  	if (ret < 0)
>  		return ret;
>  
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 097bf3e6951d..cf312ab4e7e8 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -79,6 +79,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
>   * vb2_prepare_buf() - Pass ownership of a buffer from userspace to the kernel
>   *
>   * @q:		pointer to &struct vb2_queue with videobuf2 queue.
> + * @mdev:	pointer to &struct media_device, may be NULL.
>   * @b:		buffer structure passed from userspace to
>   *		&v4l2_ioctl_ops->vidioc_prepare_buf handler in driver
>   *
> @@ -90,15 +91,19 @@ int vb2_create_bufs(struct vb2_queue *q, struct v4l2_create_buffers *create);
>   * #) verifies the passed buffer,
>   * #) calls &vb2_ops->buf_prepare callback in the driver (if provided),
>   *    in which driver-specific buffer initialization can be performed.
> + * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
> + *    then bind the prepared buffer to the request.
>   *
>   * The return values from this function are intended to be directly returned
>   * from &v4l2_ioctl_ops->vidioc_prepare_buf handler in driver.
>   */
> -int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> +int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> +		    struct v4l2_buffer *b);
>  
>  /**
>   * vb2_qbuf() - Queue a buffer from userspace
>   * @q:		pointer to &struct vb2_queue with videobuf2 queue.
> + * @mdev:	pointer to &struct media_device, may be NULL.
>   * @b:		buffer structure passed from userspace to
>   *		&v4l2_ioctl_ops->vidioc_qbuf handler in driver
>   *
> @@ -107,6 +112,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>   * This function:
>   *
>   * #) verifies the passed buffer;
> + * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
> + *    then bind the buffer to the request.
>   * #) if necessary, calls &vb2_ops->buf_prepare callback in the driver
>   *    (if provided), in which driver-specific buffer initialization can
>   *    be performed;
> @@ -116,7 +123,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>   * The return values from this function are intended to be directly returned
>   * from &v4l2_ioctl_ops->vidioc_qbuf handler in driver.
>   */
> -int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b);
> +int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
> +	     struct v4l2_buffer *b);
>  
>  /**
>   * vb2_expbuf() - Export a buffer as a file descriptor



Thanks,
Mauro

Re: [RFCv11 PATCH 21/29] videobuf2-core: add vb2_core_request_has_buffers

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:18 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add a new helper function that returns true if a media_request
> contains buffers.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 12 ++++++++++++
>  include/media/videobuf2-core.h                  |  2 ++
>  2 files changed, 14 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 4f6c0b2d7d1a..13c9d9e243dd 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1339,6 +1339,18 @@ static const struct media_request_object_ops vb2_core_req_ops = {
>  	.release = vb2_req_release,
>  };
>  
> +bool vb2_core_request_has_buffers(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +
> +	list_for_each_entry(obj, &req->objects, list) {
> +		if (obj->ops == &vb2_core_req_ops)
> +			return true;
> +	}
> +	return false;

-ENOLOCK

> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 72663c2a3ba3..e23dc028aee7 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1158,4 +1158,6 @@ bool vb2_buffer_in_use(struct vb2_queue *q, struct vb2_buffer *vb);
>   */
>  int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
> +
> +bool vb2_core_request_has_buffers(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */



Thanks,
Mauro

Re: [RFCv11 PATCH 12/29] v4l2-ctrls: alloc memory for p_req

[Mauro Carvalho Chehab]

Em Tue, 10 Apr 2018 09:32:00 +0000
Tomasz Figa <tfiga@google.com> escreveu:

> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
> > From: Hans Verkuil <hans.verkuil@cisco.com>  
> 
> > To store request data the handler_new_ref() allocates memory
> > for it if needed.  
> 
> > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >   drivers/media/v4l2-core/v4l2-ctrls.c | 20 ++++++++++++++++----
> >   1 file changed, 16 insertions(+), 4 deletions(-)  
> 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c  
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index d09f49530d9e..3c1b00baa8d0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -1997,13 +1997,18 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
> >   /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
> >   static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
> >                             struct v4l2_ctrl *ctrl,
> > -                          bool from_other_dev)
> > +                          struct v4l2_ctrl_ref **ctrl_ref,
> > +                          bool from_other_dev, bool allocate_req)
> >   {
> >          struct v4l2_ctrl_ref *ref;
> >          struct v4l2_ctrl_ref *new_ref;
> >          u32 id = ctrl->id;
> >          u32 class_ctrl = V4L2_CTRL_ID2WHICH(id) | 1;
> >          int bucket = id % hdl->nr_of_buckets;   /* which bucket to use */
> > +       unsigned int sz_extra = 0;
> > +
> > +       if (ctrl_ref)
> > +               *ctrl_ref = NULL;  
> 
> >          /*
> >           * Automatically add the control class if it is not yet present  
> and
> > @@ -2017,11 +2022,16 @@ static int handler_new_ref(struct  
> v4l2_ctrl_handler *hdl,
> >          if (hdl->error)
> >                  return hdl->error;  
> 
> > -       new_ref = kzalloc(sizeof(*new_ref), GFP_KERNEL);
> > +       if (allocate_req)

You don't need a flag here. Instead, it could simply be:

	if (ctrl_ref)


> > +               sz_extra = ctrl->elems * ctrl->elem_size;
> > +       new_ref = kzalloc(sizeof(*new_ref) + sz_extra, GFP_KERNEL);  
> 
> I think we might want to use kvzalloc() here to cover allocation of big
> elements and/or large arrays, without order>0 allocations killing the
> systems.
> 
> I'm actually also wondering if it wouldn't be cleaner to just allocate the
> extra data separately, since we store a pointer in new_ref->p_req.p anyway.

Yeah, me too.

> 
> Best regards,
> Tomasz



Thanks,
Mauro

Re: [RFCv11 PATCH 23/29] videobuf2-v4l2: export request_fd

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:20 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Requested by Sakari

Huh? What kind of description is that?

Why is it needed?

It is even harder to analyze this as documentation for the new
field is not there.

> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-v4l2.c | 6 ++++--
>  include/media/videobuf2-v4l2.h                  | 2 ++
>  2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 3d0c74bb4220..7b79149b7fae 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -184,6 +184,7 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
>  		return -EINVAL;
>  	}
>  	vbuf->sequence = 0;
> +	vbuf->request_fd = -1;
>  
>  	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
>  		switch (b->memory) {
> @@ -391,6 +392,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
>  	}
>  
>  	*p_req = req;
> +	vbuf->request_fd = b->request_fd;
>  
>  	return 0;
>  }
> @@ -496,9 +498,9 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
>  
>  	if (vb2_buffer_in_use(q, vb))
>  		b->flags |= V4L2_BUF_FLAG_MAPPED;
> -	if (vb->req_obj.req) {
> +	if (vbuf->request_fd >= 0) {
>  		b->flags |= V4L2_BUF_FLAG_REQUEST_FD;
> -		b->request_fd = -1;
> +		b->request_fd = vbuf->request_fd;
>  	}
>  
>  	if (!q->is_output &&
> diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
> index 0baa3023d7ad..d3ee1f28e197 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -32,6 +32,7 @@
>   *		&enum v4l2_field.
>   * @timecode:	frame timecode.
>   * @sequence:	sequence count of this frame.
> + * @request_fd:	the request_fd associated with this buffer
>   * @planes:	plane information (userptr/fd, length, bytesused, data_offset).
>   *
>   * Should contain enough information to be able to cover all the fields
> @@ -44,6 +45,7 @@ struct vb2_v4l2_buffer {
>  	__u32			field;
>  	struct v4l2_timecode	timecode;
>  	__u32			sequence;
> +	__s32			request_fd;
>  	struct vb2_plane	planes[VB2_MAX_PLANES];
>  };
>  



Thanks,
Mauro

Re: [RFCv11 PATCH 24/29] Documentation: v4l: document request API

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:20:21 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Alexandre Courbot <acourbot@chromium.org>
> 
> Document the request API for V4L2 devices, and amend the documentation
> of system calls influenced by it.

As said before, I would be expecting this patch to be split,
with the initial contents coming together with patch 01/29,
adding things incrementally together with the code.

I can't really analyze it the way it is, as it is hard to identify
if all changes at the V4L2 API were covered here.

> 
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  Documentation/media/uapi/v4l/buffer.rst            |  19 +-
>  Documentation/media/uapi/v4l/common.rst            |   1 +
>  Documentation/media/uapi/v4l/request-api.rst       | 199 +++++++++++++++++++++
>  Documentation/media/uapi/v4l/user-func.rst         |   1 +
>  .../media/uapi/v4l/vidioc-g-ext-ctrls.rst          |  22 ++-
>  .../media/uapi/v4l/vidioc-new-request.rst          |  64 +++++++
>  Documentation/media/uapi/v4l/vidioc-qbuf.rst       |   8 +
>  7 files changed, 308 insertions(+), 6 deletions(-)
>  create mode 100644 Documentation/media/uapi/v4l/request-api.rst
>  create mode 100644 Documentation/media/uapi/v4l/vidioc-new-request.rst
> 
> diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
> index e2c85ddc990b..e23eae12905c 100644
> --- a/Documentation/media/uapi/v4l/buffer.rst
> +++ b/Documentation/media/uapi/v4l/buffer.rst
> @@ -306,10 +306,13 @@ struct v4l2_buffer
>        - A place holder for future extensions. Drivers and applications
>  	must set this to 0.
>      * - __u32
> -      - ``reserved``
> +      - ``request_fd``

As I said before, on my comments to patch 02/29, u32 doesn't seem big enough.
The get_unused_fd_flags() function returns an integer, with can have up to
63 bits + sign.

>        -
> -      - A place holder for future extensions. Drivers and applications
> -	must set this to 0.
> +      - The file descriptor of the request to queue the buffer to. If specified
> +        and flag ``V4L2_BUF_FLAG_REQUEST_FD`` is set, then the buffer will be
> +	queued to that request. This is set by the user when calling
> +	:ref:`VIDIOC_QBUF` and :ref:`VIDIOC_PREPARE_BUF` and ignored by other
> +	ioctls.
>  
>  
>  
> @@ -514,6 +517,11 @@ Buffer Flags
>  	streaming may continue as normal and the buffer may be reused
>  	normally. Drivers set this flag when the ``VIDIOC_DQBUF`` ioctl is
>  	called.
> +    * .. _`V4L2-BUF-FLAG-IN-REQUEST`:
> +
> +      - ``V4L2_BUF_FLAG_IN_REQUEST``
> +      - 0x00000080
> +      - This buffer is part of a request the hasn't been queued yet.
>      * .. _`V4L2-BUF-FLAG-KEYFRAME`:
>  
>        - ``V4L2_BUF_FLAG_KEYFRAME``
> @@ -589,6 +597,11 @@ Buffer Flags
>  	the format. Any Any subsequent call to the
>  	:ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl will not block anymore,
>  	but return an ``EPIPE`` error code.
> +    * .. _`V4L2-BUF-FLAG-REQUEST-FD`:
> +
> +      - ``V4L2_BUF_FLAG_REQUEST_FD``
> +      - 0x00800000
> +      - The ``request_fd`` field contains a valid file descriptor.
>      * .. _`V4L2-BUF-FLAG-TIMESTAMP-MASK`:
>  
>        - ``V4L2_BUF_FLAG_TIMESTAMP_MASK``
> diff --git a/Documentation/media/uapi/v4l/common.rst b/Documentation/media/uapi/v4l/common.rst
> index 13f2ed3fc5a6..a4aa0059d45a 100644
> --- a/Documentation/media/uapi/v4l/common.rst
> +++ b/Documentation/media/uapi/v4l/common.rst
> @@ -44,3 +44,4 @@ applicable to all devices.
>      crop
>      selection-api
>      streaming-par
> +    request-api
> diff --git a/Documentation/media/uapi/v4l/request-api.rst b/Documentation/media/uapi/v4l/request-api.rst
> new file mode 100644
> index 000000000000..0c1f2896e197
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/request-api.rst
> @@ -0,0 +1,199 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _media-request-api:
> +
> +Request API
> +===========
> +
> +The Request API has been designed to allow V4L2 to deal with requirements of
> +modern devices (stateless codecs, MIPI cameras, ...) and APIs (Android Codec
> +v2). One such requirement is the ability for devices belonging to the same
> +pipeline to reconfigure and collaborate closely on a per-frame basis. Another is

Another one

> +efficient support of stateless codecs, which need per-frame controls to be set
> +asynchronously in order to be efficiently used.
> +
> +Supporting these features without the Request API is possible but terribly
> +inefficient: user-space would have to flush all activity on the media pipeline,
> +reconfigure it for the next frame, queue the buffers to be processed with that
> +configuration, and wait until they are all available for dequeing before
> +considering the next frame. This defeats the purpose of having buffer queues
> +since in practice only one buffer would be queued at a time.
> +
> +The Request API allows a specific configuration of the pipeline (media
> +controller topology + controls for each device) to be associated with specific
> +buffers. The parameters are applied by each participating device as buffers
> +associated to a request flow in. This allows user-space to schedule several
> +tasks ("requests") with different parameters in advance, knowing that the
> +parameters will be applied when needed to get the expected result. Controls
> +values at the time of request completion are also available for reading.
> +
> +Usage
> +=====
> +
> +The Request API is used on top of standard media controller and V4L2 calls,
> +which are augmented with an extra ``request_fd`` parameter. Request themselves
> +are allocated from either a supporting V4L2 device node, or a supporting media
> +controller node. The origin of requests determine their scope: requests
> +allocated from a V4L2 device node can only act on that device, whereas requests
> +allocated from a media controller node can control the whole pipeline of the
> +controller.
> +
> +Request Allocation
> +------------------
> +
> +User-space allocates requests using the ``VIDIOC_NEW_REQUEST`` (for V4L2 device
> +requests) or ``MEDIA_IOC_NEW_REQUEST`` (for media controller requests) on an
> +opened device or media node. This returns a file descriptor representing the
> +request. Typically, several such requests will be allocated.
> +
> +Request Preparation
> +-------------------
> +
> +Standard V4L2 ioctls can then receive a request file descriptor to express the
> +fact that the ioctl is part of said request, and is not to be applied
> +immediately. V4L2 ioctls supporting this are :c:func:`VIDIOC_S_EXT_CTRLS` and
> +:c:func:`VIDIOC_QBUF`. Controls set with a request parameter are stored instead
> +of being immediately applied, and queued buffers not enter the regular buffer
> +queue until the request is submitted. Only one buffer can be queued to a given
> +queue for a given request.
> +
> +Request Submission
> +------------------
> +
> +Once the parameters and buffers of the request are specified, it can be
> +submitted by calling the ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl on the request FD.
> +This will make the buffers associated to the request available to their driver,
> +which can then apply the saved controls as buffers are processed. A submitted
> +request cannot be modified anymore.
> +
> +If several devices are part of the request, individual drivers may synchronize
> +so the requested pipeline's topology is applied before the buffers are
> +processed. This is at the discretion of media controller drivers and is not a
> +requirement.
> +
> +Buffers queued without an associated request after a request-bound buffer will
> +be processed using the state of the hardware at the time of the request
> +completion. All the same, controls set without a request are applied
> +immediately, regardless of whether a request is in use or not.
> +
> +User-space can ``poll()`` a request FD in order to wait until the request
> +completes. A request is considered complete once all its associated buffers are
> +available for dequeing. Note that user-space does not need to wait for the
> +request to complete to dequeue its buffers: buffers that are available halfway
> +through a request can be dequeued independently of the request's state.
> +
> +A completed request includes the state of all devices that had queued buffers
> +associated with it at the time of the request completion. User-space can query
> +that state by calling :c:func:`VIDIOC_G_EXT_CTRLS` with the request FD.
> +
> +Recycling and Destruction
> +-------------------------
> +
> +Finally, completed request can either be discarded or be reused. Calling
> +``close()`` on a request FD will make that FD unusable, freeing the request if
> +it is not referenced elsewhere. The ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl will
> +clear a request's state and make it available again. No state is retained by
> +this operation: the request is as if it had just been allocated.
> +
> +Example for a M2M Device
> +------------------------
> +
> +M2M devices are single-node V4L2 devices providing one OUTPUT queue (for
> +user-space
> +to provide input buffers) and one CAPTURE queue (to retrieve processed data).
> +They are perfectly symetric, i.e. one buffer of input will produce one buffer of
> +output. These devices are commonly used for frame processors or stateless
> +codecs.
> +
> +In this use-case, the request API can be used to associate specific controls to
> +be applied by the driver before processing an OUTPUT buffer, allowing user-space
> +to queue many such buffers in advance. It can also take advantage of requests'
> +ability to capture the state of controls when the request completes to read back
> +information that may be subject to change.
> +
> +Put into code, after obtaining a request, user-space can assign controls and one
> +OUTPUT buffer to it:
> +
> +	struct v4l2_buf buf;
> +	struct v4l2_ext_controls ctrls;
> +	struct media_request_new new = { 0 };

Why are you zeroing just "new"?

Also, I guess the more portable way to zero an array on is, instead:

	struct media_request_new new = {};

As the above could be interpreted by the C compiler to zero just the first
field, but I didn't actually investigate how the several compilers handle
it (or how compilation directives gcc's -std=c89 would affect it).

so, perhaps, it would be better to write, instead:

	struct v4l2_buf buf;
	struct v4l2_ext_controls ctrls;
	struct media_request_new new;

	memset (&buf, 0, sizeof(buf));
	memset (&new, 0, sizeof(new));
	memset (&ctrls, 0, sizeof(ctrls));

> +	int req_fd;
> +	...
> +	ioctl(media_fd, VIDIOC_NEW_REQUEST, &new);
> +	req_fd = new.fd;
> +	...
> +	ctrls.request_fd = req_fd;
> +	ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls);
> +	...
> +	buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
> +	buf.request_fd = req_fd;
> +	ioctl(codec_fd, VIDIOC_QBUF, &buf);
> +
> +Note that request_fd does not need to be specified for CAPTURE buffers: since
> +there is symetry between the OUTPUT and CAPTURE queues, and requests are
> +processed in order of submission, we can know which CAPTURE buffer corresponds
> +to which request.
> +
> +Once the request is fully prepared, it can be submitted to the driver:
> +
> +	ioctl(request_fd, MEDIA_REQUEST_IOC_SUBMIT, NULL);
> +
> +User-space can then either wait for the request to complete by calling poll() on
> +its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will
> +want to get CAPTURE buffers as soon as possible and this can be done using a
> +regular DQBUF:
> +
> +	struct v4l2_buf buf;
> +
> +	memset(&buf, 0, sizeof(buf));
> +	buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +	ioctl(codec_fd, VIDIOC_DQBUF, &buf);
> +
> +We can then, after ensuring that the request is completed via polling the
> +request FD, query control values at the time of its completion via an
> +annotated call to G_EXT_CTRLS. This is particularly useful for volatile controls
> +for which we want to query values as soon as the capture buffer is produced.
> +
> +	struct pollfd pfd = { .events = POLLIN, .fd = request_fd };
> +	poll(&pfd, 1, -1);
> +	...
> +	ctrls.request_fd = req_fd;
> +	ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls);
> +
> +Once we don't need the request anymore, we can either recycle it for reuse with
> +MEDIA_REQUEST_IOC_REINIT...
> +
> +	ioctl(request, MEDIA_REQUEST_IOC_REINIT, NULL);
> +
> +... or close its file descriptor to completely dispose of it.
> +
> +	close(request_fd);
> +
> +Example for a Simple Capture Device
> +-----------------------------------
> +
> +With a simple capture device, requests can be used to specify controls to apply
> +to a given CAPTURE buffer. The driver will apply these controls before producing
> +the marked CAPTURE buffer.
> +
> +	struct v4l2_buf buf;
> +	struct v4l2_ext_controls ctrls;
> +	struct media_request_new new = { 0 };

Same comments as before.

> +	int req_fd;
> +	...
> +	ioctl(camera_fd, VIDIOC_NEW_REQUEST, &new);
> +	req_fd = new.fd;
> +	...
> +	ctrls.request_fd = req_fd;
> +	ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls);
> +	...
> +	buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +	buf.request_fd = req_fd;
> +	ioctl(camera_fd, VIDIOC_QBUF, &buf);
> +
> +Once the request is fully prepared, it can be submitted to the driver:
> +
> +	ioctl(req_fd, MEDIA_REQUEST_IOC_SUBMIT, &cmd);
> +
> +User-space can then dequeue buffers, wait for the request completion, query
> +controls and recycle the request as in the M2M example above.
> diff --git a/Documentation/media/uapi/v4l/user-func.rst b/Documentation/media/uapi/v4l/user-func.rst
> index 3e0413b83a33..2c8238a2b188 100644
> --- a/Documentation/media/uapi/v4l/user-func.rst
> +++ b/Documentation/media/uapi/v4l/user-func.rst
> @@ -53,6 +53,7 @@ Function Reference
>      vidioc-g-std
>      vidioc-g-tuner
>      vidioc-log-status
> +    vidioc-new-request
>      vidioc-overlay
>      vidioc-prepare-buf
>      vidioc-qbuf
> diff --git a/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst b/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
> index 2011c2b2ee67..d31ef86c7caa 100644
> --- a/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
> @@ -95,6 +95,17 @@ appropriate. In the first case the new value is set in struct
>  is inappropriate (e.g. the given menu index is not supported by the menu
>  control), then this will also result in an ``EINVAL`` error code error.
>  
> +If ``request_fd`` is set to a not-submitted request file descriptor, then the
> +controls are not applied immediately when calling
> +:ref:`VIDIOC_S_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`, but instead are applied right
> +before the driver starts processing a buffer associated to the same request.
> +
> +If ``request_fd`` is specified during a call to
> +:ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`, then the returned values will
> +be the values currently set for the request (or the hardware value if none is
> +set) if the request has not yet completed, or the values of the controls at the
> +time of request completion if it has already completed.
> +
>  The driver will only set/get these controls if all control values are
>  correct. This prevents the situation where only some of the controls
>  were set/get. Only low-level errors (e. g. a failed i2c command) can
> @@ -209,8 +220,10 @@ still cause this situation.
>        - ``which``
>        - Which value of the control to get/set/try.
>  	``V4L2_CTRL_WHICH_CUR_VAL`` will return the current value of the
> -	control and ``V4L2_CTRL_WHICH_DEF_VAL`` will return the default
> -	value of the control.
> +	control, ``V4L2_CTRL_WHICH_DEF_VAL`` will return the default
> +	value of the control and ``V4L2_CTRL_WHICH_REQUEST`` indicates that
> +	these controls have to be retrieved from a request or tried/set for
> +	a request.
>  
>  	.. note::
>  
> @@ -272,8 +285,11 @@ still cause this situation.
>  	then you can call :ref:`VIDIOC_TRY_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` to try to discover the
>  	actual control that failed the validation step. Unfortunately,
>  	there is no ``TRY`` equivalent for :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`.
> +    * - __s32
> +      - ``request_fd``
> +	File descriptor of the request to be used by this operation (0 if none).

Same comment as before: _s32 is too short on 64-bit systems. Won't repeat
it again, but other parts of the doc/code has the same issue.

>      * - __u32
> -      - ``reserved``\ [2]
> +      - ``reserved``\ [1]
>        - Reserved for future extensions.
>  
>  	Drivers and applications must set the array to zero.
> diff --git a/Documentation/media/uapi/v4l/vidioc-new-request.rst b/Documentation/media/uapi/v4l/vidioc-new-request.rst
> new file mode 100644
> index 000000000000..0038287f7d16
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/vidioc-new-request.rst
> @@ -0,0 +1,64 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _VIDIOC_NEW_REQUEST:
> +
> +************************
> +ioctl VIDIOC_NEW_REQUEST
> +************************
> +
> +Name
> +====
> +
> +VIDIOC_NEW_REQUEST - Allocate a request for given video device.
> +
> +
> +Synopsis
> +========
> +
> +.. c:function:: int ioctl( int fd, VIDIOC_NEW_REQUEST, struct media_request_new *argp )
> +    :name: VIDIOC_NEW_REQUEST
> +
> +Arguments
> +=========
> +
> +``fd``
> +    File descriptor returned by :ref:`open() <func-open>`.
> +
> +``argp``
> +    Pointer to struct :c:type:`media_request_new`.
> +
> +
> +Description
> +===========
> +
> +Applications call the ``VIDIOC_NEW_REQUEST`` ioctl to allocate a new request for a given V4L2 video device. The request will only be valid in the scope of the device that allocated it and cannot be used to coordinate multiple devices.
> +
> +Applications can also check whether requests are supported by a given device by calling this ioctl with the MEDIA_REQUEST_FLAG_TEST bit of :c:type:`media_request_new`'s ``flags`` set. Doing so will not allocate a new request, but will return 0 is request allocation is supported by the device, or -1 and set ``errno`` to ENOTTY if they are not.
> +
> +.. c:type:: media_request_new
> +
> +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.7cm}|
> +
> +.. flat-table:: struct media_request_new
> +    :header-rows:  0
> +    :stub-columns: 0
> +    :widths:       1 1 2
> +
> +    * - __u32
> +      - ``flags``
> +      - Flags for this request creation. If ``MEDIA_REQUEST_FLAG_TEST`` is set, then no request is created and the call only checks for request availability. Written by the application.
> +    * - __s32
> +      - ``fd``
> +      - File descriptor referencing the created request. Written by the kernel.
> +
> +Return Value
> +============
> +
> +On success 0 is returned, and the ``fd`` field of ``argp`` is set to a file descriptor referencing the request. User-space can use this file descriptor to mention the request in other system calls, perform ``MEDIA_REQUEST_IOC_SUBMIT`` and ``MEDIA_REQUEST_IOC_REINIT`` ioctls on it, and close it to discard the request.
> +
> +On error -1 is returned and the ``errno`` variable is set appropriately.  The
> +generic error codes are described in the :ref:`Generic Error Codes <gen-errors>`
> +chapter.
> +
> +ENOTTY
> +    The device does not support the use of requests or request support is not built into the kernel.
> diff --git a/Documentation/media/uapi/v4l/vidioc-qbuf.rst b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> index 9e448a4aa3aa..6246f1888583 100644
> --- a/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-qbuf.rst
> @@ -98,6 +98,14 @@ dequeued, until the :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` or
>  :ref:`VIDIOC_REQBUFS` ioctl is called, or until the
>  device is closed.
>  
> +The ``request_fd`` field can be used when queuing to specify the file
> +descriptor of a request, if requests are in use. Setting it means that the
> +buffer will not be passed to the driver until the request itself is submitted.
> +Also, the driver will apply any setting associated with the request before
> +processing the buffer. Only one buffer per queue can be assigned that way to
> +a request. This field will be ignored unless the ``V4L2_BUF_FLAG_REQUEST_FD``
> +flag is set.
> +
>  Applications call the ``VIDIOC_DQBUF`` ioctl to dequeue a filled
>  (capturing) or displayed (output) buffer from the driver's outgoing
>  queue. They just set the ``type``, ``memory`` and ``reserved`` fields of



Thanks,
Mauro

Re: [RFCv11 PATCH 00/29] Request API

[Mauro Carvalho Chehab]

Em Mon,  9 Apr 2018 16:19:57 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Hi all,
> 
> This is a cleaned up version of the v10 series (never posted to
> the list since it was messy).

Just reviewed the full series.

I found v11 still a little messy, specially at the part that
touches VB2 core. I wasn't capable of understanding the changes
there on a reasonable time. As mentioned there, please split it
into more palatable patches :-)

> 
> The main changes compared to v9 are in the control framework which
> is (hopefully!) now in sync with the RFC. Specifically, when queueing
> a request it will 'chain' itself correctly with the previously queued
> request. Control values that were not explicitly set in the request
> will get the value from the first request in the queue that sets it,
> or the hardware value if no request in the queue ever touches it.
> 
> However, I have not yet had the opportunity to test this in
> v4l2-compliance!

You should. I'd expect some troubles, expecially when read()
interface is used.

> 
> Sakari: I did not have the time to incorporate your comments. I'll
> probably wait until I have more feedback and then post a new series
> next week.
> 
> Other changes:
> 
> - various request state checks were missing (i.e. you could set
>   a control in a queued request).
> 
> - a new cancel op was added to handle the corner case where a
>   request was queued but never reached the driver since STREAMOFF
>   was called before the buffers were ever queued in the driver.
> 
> - various random fixes.
> 
> - added the patch "videobuf2-v4l2: export request_fd" as requested
>   by Sakari.
> 
> - changed some inconsistent error codes.

- Review locks.

IMHO, they're currently broken.

> 
> This has been tested with vim2m and vivid using v4l2-compliance.
> The v4l-utils repo supporting requests is here:
> https://git.linuxtv.org/hverkuil/v4l-utils.git/log/?h=request
> 
> TODO/Remarks:
> 
> 1) missing prototype documentation in media-requests.c/h. Some
>    is documented, but not everything.
> 
> 2) No VIDIOC_REQUEST_ALLOC 'shortcut' ioctl. Sorry, I just ran out
>    of time. Alexandre, Tomasz, feel free to add it back (it should
>    be quite easy to do) and post a patch. I'll add it to my patch
>    series. As mentioned before: whether or not we actually want this
>    has not been decided yet.
> 
> 3) vim2m: the media topology is a bit bogus, this needs to be fixed
>    (i.e. a proper HW entity should be added). But for now it is
>    good enough for testing.
> 
> 4) I think this should slide in fairly easy after the fence support
>    is merged. I made sure the request API changes in public headers
>    did not clash with the changes made by the fence API.
> 
> 5) I did not verify the Request API documentation patch. I did update
>    it with the new buffer flags and 'which' value, but it probably is
>    out of date in places.

That's why I think documentation patch should come first, with the "basic"
stuff, and, as new features/API headers are touched, more stuff gets added
there.

> 
> 6) More testing. I think I have fairly good coverage in v4l2-compliance,
>    but no doubt I've missed a few test cases.
> 
> 7) debugfs: it would be really useful to expose the number of request
>    and request objects in debugfs to simplify debugging. Esp. to make
>    sure everything is freed correctly.
> 
> 8) Review copyright/authorship lines. I'm not sure if everything is
>    correct. Alexandre, Sakari, if you see something that is not
>    right in this respect, just let me know!
> 
> 9) The req_queue op should likely be split into a req_validate and
>    a req_queue op (based on a discussion on the ML with Sakari).
>    That avoids a race condition.
> 
> Everything seemed to slip nicely into place while working on this,
> so I hope this is finally an implementation that we can proceed to
> upstream and build upon for complex camera pipelines in the future.
> 
> This patch series is also available here:
> 
> https://git.linuxtv.org/hverkuil/media_tree.git/log/?h=reqv11
> 
> Regards,
> 
> 	Hans
> 
> 
> Alexandre Courbot (3):
>   videodev2.h: add request_fd field to v4l2_ext_controls
>   Documentation: v4l: document request API
>   media: vim2m: add media device
> 
> Hans Verkuil (26):
>   v4l2-device.h: always expose mdev
>   uapi/linux/media.h: add request API
>   media-request: allocate media requests
>   media-request: core request support
>   media-request: add request ioctls
>   media-request: add media_request_find
>   media-request: add media_request_object_find
>   videodev2.h: add V4L2_CTRL_FLAG_IN_REQUEST
>   v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev
>   v4l2-ctrls: prepare internal structs for request API
>   v4l2-ctrls: alloc memory for p_req
>   v4l2-ctrls: use ref in helper instead of ctrl
>   v4l2-ctrls: add core request support
>   v4l2-ctrls: support g/s_ext_ctrls for requests
>   videodev2.h: Add request_fd field to v4l2_buffer
>   vb2: store userspace data in vb2_v4l2_buffer
>   videobuf2-core: embed media_request_object
>   videobuf2-core: integrate with media requests
>   videobuf2-v4l2: integrate with media requests
>   videobuf2-core: add vb2_core_request_has_buffers
>   videobuf2-v4l2: add vb2_request_queue helper
>   videobuf2-v4l2: export request_fd
>   vim2m: use workqueue
>   vim2m: support requests
>   vivid: add mc
>   vivid: add request support
> 
>  Documentation/media/uapi/v4l/buffer.rst            |  19 +-
>  Documentation/media/uapi/v4l/common.rst            |   1 +
>  Documentation/media/uapi/v4l/request-api.rst       | 199 +++++++++
>  Documentation/media/uapi/v4l/user-func.rst         |   1 +
>  .../media/uapi/v4l/vidioc-g-ext-ctrls.rst          |  22 +-
>  .../media/uapi/v4l/vidioc-new-request.rst          |  64 +++
>  Documentation/media/uapi/v4l/vidioc-qbuf.rst       |   8 +
>  drivers/media/Makefile                             |   3 +-
>  drivers/media/common/videobuf2/videobuf2-core.c    | 152 +++++--
>  drivers/media/common/videobuf2/videobuf2-v4l2.c    | 449 ++++++++++++-------
>  drivers/media/dvb-core/dvb_vb2.c                   |   5 +-
>  drivers/media/dvb-frontends/rtl2832_sdr.c          |   5 +-
>  drivers/media/media-device.c                       |  14 +
>  drivers/media/media-request.c                      | 454 +++++++++++++++++++
>  drivers/media/pci/bt8xx/bttv-driver.c              |   2 +-
>  drivers/media/pci/cx23885/cx23885-417.c            |   2 +-
>  drivers/media/pci/cx88/cx88-blackbird.c            |   2 +-
>  drivers/media/pci/cx88/cx88-video.c                |   2 +-
>  drivers/media/pci/saa7134/saa7134-empress.c        |   4 +-
>  drivers/media/pci/saa7134/saa7134-video.c          |   2 +-
>  drivers/media/platform/exynos4-is/fimc-capture.c   |   2 +-
>  drivers/media/platform/omap3isp/ispvideo.c         |   4 +-
>  drivers/media/platform/rcar-vin/rcar-v4l2.c        |   3 +-
>  drivers/media/platform/rcar_drif.c                 |   2 +-
>  drivers/media/platform/s3c-camif/camif-capture.c   |   4 +-
>  drivers/media/platform/s5p-mfc/s5p_mfc_dec.c       |   4 +-
>  drivers/media/platform/s5p-mfc/s5p_mfc_enc.c       |   4 +-
>  drivers/media/platform/soc_camera/soc_camera.c     |   7 +-
>  drivers/media/platform/vim2m.c                     |  83 +++-
>  drivers/media/platform/vivid/vivid-core.c          |  68 +++
>  drivers/media/platform/vivid/vivid-core.h          |   8 +
>  drivers/media/platform/vivid/vivid-ctrls.c         |  46 +-
>  drivers/media/platform/vivid/vivid-kthread-cap.c   |  12 +
>  drivers/media/platform/vivid/vivid-kthread-out.c   |  12 +
>  drivers/media/platform/vivid/vivid-sdr-cap.c       |   8 +
>  drivers/media/platform/vivid/vivid-vbi-cap.c       |   2 +
>  drivers/media/platform/vivid/vivid-vbi-out.c       |   2 +
>  drivers/media/platform/vivid/vivid-vid-cap.c       |   2 +
>  drivers/media/platform/vivid/vivid-vid-out.c       |   2 +
>  drivers/media/usb/cpia2/cpia2_v4l.c                |   2 +-
>  drivers/media/usb/cx231xx/cx231xx-417.c            |   2 +-
>  drivers/media/usb/cx231xx/cx231xx-video.c          |   4 +-
>  drivers/media/usb/msi2500/msi2500.c                |   2 +-
>  drivers/media/usb/tm6000/tm6000-video.c            |   2 +-
>  drivers/media/usb/uvc/uvc_queue.c                  |   5 +-
>  drivers/media/usb/uvc/uvc_v4l2.c                   |   3 +-
>  drivers/media/usb/uvc/uvcvideo.h                   |   1 +
>  drivers/media/v4l2-core/v4l2-compat-ioctl32.c      |  14 +-
>  drivers/media/v4l2-core/v4l2-ctrls.c               | 480 +++++++++++++++++++--
>  drivers/media/v4l2-core/v4l2-device.c              |   3 +-
>  drivers/media/v4l2-core/v4l2-ioctl.c               |  22 +-
>  drivers/media/v4l2-core/v4l2-mem2mem.c             |   7 +-
>  drivers/media/v4l2-core/v4l2-subdev.c              |   9 +-
>  drivers/staging/media/davinci_vpfe/vpfe_video.c    |   3 +-
>  drivers/staging/media/imx/imx-media-dev.c          |   2 +-
>  drivers/staging/media/imx/imx-media-fim.c          |   2 +-
>  drivers/staging/media/omap4iss/iss_video.c         |   3 +-
>  drivers/usb/gadget/function/uvc_queue.c            |   2 +-
>  include/media/media-device.h                       |  13 +
>  include/media/media-request.h                      | 213 +++++++++
>  include/media/v4l2-ctrls.h                         |  45 +-
>  include/media/v4l2-device.h                        |   4 +-
>  include/media/videobuf2-core.h                     |  25 +-
>  include/media/videobuf2-v4l2.h                     |  19 +-
>  include/uapi/linux/media.h                         |   8 +
>  include/uapi/linux/videodev2.h                     |  15 +-
>  66 files changed, 2277 insertions(+), 318 deletions(-)
>  create mode 100644 Documentation/media/uapi/v4l/request-api.rst
>  create mode 100644 Documentation/media/uapi/v4l/vidioc-new-request.rst
>  create mode 100644 drivers/media/media-request.c
>  create mode 100644 include/media/media-request.h
> 



Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Mauro Carvalho Chehab]

Em Tue, 10 Apr 2018 15:32:34 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:

> Hi Mauro and Hans,
> 
> On Tue, Apr 10, 2018 at 07:32:06AM -0300, Mauro Carvalho Chehab wrote:
> ...
> > > +static void media_request_release(struct kref *kref)
> > > +{
> > > +	struct media_request *req =
> > > +		container_of(kref, struct media_request, kref);
> > > +	struct media_device *mdev = req->mdev;
> > > +	unsigned long flags;
> > > +
> > > +	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> > > +
> > > +	spin_lock_irqsave(&req->lock, flags);
> > > +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> > > +	spin_unlock_irqrestore(&req->lock, flags);
> > > +
> > > +	media_request_clean(req);
> > > +
> > > +	if (mdev->ops->req_free)
> > > +		mdev->ops->req_free(req);
> > > +	else
> > > +		kfree(req);  
> > 
> > Without looking at req_free() implementation, I would actually prefer
> > to always do a kfree(req) here. So, a req_free() function would only
> > free "extra" allocations, and not the request itself. e. g.:
> > 
> > ...
> > 	if (mdev->ops->req_free)
> > 		mdev->ops->req_free(req);
> > 
> > 	kfree(req);
> > }  
> 
> The idea is that you can embed the request object in a driver specific
> struct. The drivers can store information related to that request rather
> easily that way, without requiring to be aware of two objects with
> references pointing to each other. I rather prefer the current
> implementation. It same pattern is actually used on videobuf2 buffers.

Ok, then document it so.

Btw, one of the things it is missing is a kAPI documentation patch,
describing things like that.

> 
> ...
> 
> > > +static unsigned int media_request_poll(struct file *filp,
> > > +				       struct poll_table_struct *wait)
> > > +{
> > > +	struct media_request *req = filp->private_data;
> > > +	unsigned long flags;
> > > +	enum media_request_state state;
> > > +
> > > +	if (!(poll_requested_events(wait) & POLLPRI))
> > > +		return 0;
> > > +
> > > +	spin_lock_irqsave(&req->lock, flags);
> > > +	state = req->state;
> > > +	spin_unlock_irqrestore(&req->lock, flags);  
> > 
> > IMO, it would be better to use an atomic var for state, having a
> > lockless access to it.  
> 
> The lock serialises access to the whole request, not just to its state.

>From what I understood at the code is that the spin lock
is used *sometimes* to protect just the state. I didn't
see it used to protect the hole data struct.

Instead, the mutex is used for that purpose, but, again,
it is *sometimes* used, but on several parts, neither the
spin lock nor the mutex is used.

It should be noticed that the data, itself, should be protected
by *either* mutex or spin lock.

> While it doesn't matter here as you're just reading the state, writing it
> would still require taking the lock. Using atomic_t might suggest
> otherwise, and could end up being a source of bugs.

There are already a lot of bugs at the locking, from what I noticed.

We should do it right: it should use *just one* kind of memory
protection for struct media_request. On the places where it is
safe to just read the status without locking, atomic_t() should
be used. Where it doesn't, probably the entire code should be
protected by the lock.

> >   
> > > +
> > > +	if (state == MEDIA_REQUEST_STATE_COMPLETE)
> > > +		return POLLPRI;
> > > +	if (state == MEDIA_REQUEST_STATE_IDLE)
> > > +		return POLLERR;
> > > +
> > > +	poll_wait(filp, &req->poll_wait, wait);
> > > +	return 0;
> > > +}
> > > +
> > > +static long media_request_ioctl(struct file *filp, unsigned int cmd,
> > > +				unsigned long __arg)
> > > +{
> > > +	return -ENOIOCTLCMD;
> > > +}
> > > +
> > > +static const struct file_operations request_fops = {
> > > +	.owner = THIS_MODULE,
> > > +	.poll = media_request_poll,
> > > +	.unlocked_ioctl = media_request_ioctl,
> > > +	.release = media_request_close,
> > > +};
> > > +
> > >  int media_request_alloc(struct media_device *mdev,
> > >  			struct media_request_alloc *alloc)
> > >  {
> > > -	return -ENOMEM;
> > > +	struct media_request *req;
> > > +	struct file *filp;
> > > +	char comm[TASK_COMM_LEN];
> > > +	int fd;
> > > +	int ret;
> > > +
> > > +	fd = get_unused_fd_flags(O_CLOEXEC);
> > > +	if (fd < 0)
> > > +		return fd;
> > > +
> > > +	filp = anon_inode_getfile("request", &request_fops, NULL, O_CLOEXEC);
> > > +	if (IS_ERR(filp)) {
> > > +		ret = PTR_ERR(filp);
> > > +		goto err_put_fd;
> > > +	}
> > > +
> > > +	if (mdev->ops->req_alloc)
> > > +		req = mdev->ops->req_alloc(mdev);
> > > +	else
> > > +		req = kzalloc(sizeof(*req), GFP_KERNEL);
> > > +	if (!req) {
> > > +		ret = -ENOMEM;
> > > +		goto err_fput;
> > > +	}
> > > +
> > > +	filp->private_data = req;
> > > +	req->mdev = mdev;
> > > +	req->state = MEDIA_REQUEST_STATE_IDLE;
> > > +	req->num_incomplete_objects = 0;
> > > +	kref_init(&req->kref);
> > > +	INIT_LIST_HEAD(&req->objects);
> > > +	spin_lock_init(&req->lock);
> > > +	init_waitqueue_head(&req->poll_wait);
> > > +
> > > +	alloc->fd = fd;  
> > 
> > Btw, this is a very good reason why you should define the ioctl to
> > have an integer argument instead of a struct with a __s32 field
> > on it, as per my comment to patch 02/29:
> > 
> > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > 
> > At 64 bit architectures, you're truncating the file descriptor!  
> 
> I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> well.

Hmm.. you're right. I was thinking that it could be 64 bits on some
archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
according with:

	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html

This is not the case on gcc.

> We actually replaced int by __s32 or __u32 everywhere in uAPI (apart from
> one particular struct; time to send a patch) about five years ago.
> 
> >   
> > > +	get_task_comm(comm, current);
> > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > +		 comm, fd);  
> > 
> > Not sure if it is a good idea to store the task that allocated
> > the request. While it makes sense for the dev_dbg() below, it
> > may not make sense anymore on other dev_dbg() you would be
> > using it.  
> 
> The lifetime of the file handle roughly matches that of the request. It's
> for debug only anyway.
> 
> Better proposals are always welcome of course. But I think we should have
> something here that helps debugging by meaningfully making the requests
> identifiable from logs.

What I meant to say is that one PID could be allocating the
request, while some other one could be actually doing Q/DQ_BUF.
On such scenario, the debug string could provide mislead prints.

> 
> ...
> 
> > > +#ifdef CONFIG_MEDIA_CONTROLLER
> > > +static inline void media_request_object_get(struct media_request_object *obj)
> > > +{
> > > +	kref_get(&obj->kref);
> > > +}  
> > 
> > Why do you need a function? Just use kref_get() were needed, instead of
> > aliasing it for no good reason.  
> 
> We have similar functions for get and put in many places in the kernel. If
> this were a static function only used in a single .c file, I'd most likely
> agree. Here it avoids having to know the internals of request objects
> elsewhere.
> 
> I wonder if this could be moved to the .c file actually; I haven't looked
> at the other patches.
> 



Thanks,
Mauro

Re: [RFCv11 PATCH 14/29] v4l2-ctrls: add core request support

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 17197 bytes --]

Hi,

On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Integrate the request support. This adds the
> v4l2_ctrl_request_complete
> and v4l2_ctrl_request_setup functions to complete a request and (as a
> helper function) to apply a request to the hardware.
> 
> It takes care of queuing requests and correctly chaining control
> values
> in the request queue.
> 
> Note that when a request is marked completed it will copy control
> values
> to the internal request state. This can be optimized in the future
> since
> this is sub-optimal when dealing with large compound and/or array
> controls.
> 
> For the initial 'stateless codec' use-case the current implementation
> is
> sufficient.

See one comment about a missing unlock below.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 325
> ++++++++++++++++++++++++++++++++++-
>  include/media/v4l2-ctrls.h           |  23 +++
>  2 files changed, 342 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index da4cc1485dc4..6e2c5e24734f 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -1647,6 +1647,13 @@ static int new_to_user(struct v4l2_ext_control
> *c,
>  	return ptr_to_user(c, ctrl, ctrl->p_new);
>  }
>  
> +/* Helper function: copy the request value back to the caller */
> +static int req_to_user(struct v4l2_ext_control *c,
> +		       struct v4l2_ctrl_ref *ref)
> +{
> +	return ptr_to_user(c, ref->ctrl, ref->p_req);
> +}
> +
>  /* Helper function: copy the initial control value back to the caller
> */
>  static int def_to_user(struct v4l2_ext_control *c, struct v4l2_ctrl
> *ctrl)
>  {
> @@ -1766,6 +1773,26 @@ static void cur_to_new(struct v4l2_ctrl *ctrl)
>  	ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new);
>  }
>  
> +/* Copy the new value to the request value */
> +static void new_to_req(struct v4l2_ctrl_ref *ref)
> +{
> +	if (!ref)
> +		return;
> +	ptr_to_ptr(ref->ctrl, ref->ctrl->p_new, ref->p_req);
> +	ref->req = ref;
> +}
> +
> +/* Copy the request value to the new value */
> +static void req_to_new(struct v4l2_ctrl_ref *ref)
> +{
> +	if (!ref)
> +		return;
> +	if (ref->req)
> +		ptr_to_ptr(ref->ctrl, ref->req->p_req, ref->ctrl-
> >p_new);
> +	else
> +		ptr_to_ptr(ref->ctrl, ref->ctrl->p_cur, ref->ctrl-
> >p_new);
> +}
> +
>  /* Return non-zero if one or more of the controls in the cluster has
> a new
>     value that differs from the current value. */
>  static int cluster_changed(struct v4l2_ctrl *master)
> @@ -1875,6 +1902,9 @@ int v4l2_ctrl_handler_init_class(struct
> v4l2_ctrl_handler *hdl,
>  	lockdep_set_class_and_name(hdl->lock, key, name);
>  	INIT_LIST_HEAD(&hdl->ctrls);
>  	INIT_LIST_HEAD(&hdl->ctrl_refs);
> +	INIT_LIST_HEAD(&hdl->requests);
> +	INIT_LIST_HEAD(&hdl->requests_queued);
> +	hdl->request_is_queued = false;
>  	hdl->nr_of_buckets = 1 + nr_of_controls_hint / 8;
>  	hdl->buckets = kvmalloc_array(hdl->nr_of_buckets,
>  				      sizeof(hdl->buckets[0]),
> @@ -1895,6 +1925,14 @@ void v4l2_ctrl_handler_free(struct
> v4l2_ctrl_handler *hdl)
>  	if (hdl == NULL || hdl->buckets == NULL)
>  		return;
>  
> +	if (!hdl->req_obj.req && !list_empty(&hdl->requests)) {
> +		struct v4l2_ctrl_handler *req, *next_req;
> +
> +		list_for_each_entry_safe(req, next_req, &hdl-
> >requests, requests) {
> +			media_request_object_unbind(&req->req_obj);
> +			media_request_object_put(&req->req_obj);
> +		}
> +	}
>  	mutex_lock(hdl->lock);
>  	/* Free all nodes */
>  	list_for_each_entry_safe(ref, next_ref, &hdl->ctrl_refs,
> node) {
> @@ -2816,6 +2854,126 @@ int v4l2_querymenu(struct v4l2_ctrl_handler
> *hdl, struct v4l2_querymenu *qm)
>  }
>  EXPORT_SYMBOL(v4l2_querymenu);
>  
> +static int v4l2_ctrl_request_clone(struct v4l2_ctrl_handler *hdl,
> +				   const struct v4l2_ctrl_handler
> *from)
> +{
> +	struct v4l2_ctrl_ref *ref;
> +	int err;
> +
> +	if (WARN_ON(!hdl || hdl == from))
> +		return -EINVAL;
> +
> +	if (hdl->error)
> +		return hdl->error;
> +
> +	WARN_ON(hdl->lock != &hdl->_lock);
> +
> +	mutex_lock(from->lock);
> +	list_for_each_entry(ref, &from->ctrl_refs, node) {
> +		struct v4l2_ctrl *ctrl = ref->ctrl;
> +		struct v4l2_ctrl_ref *new_ref;
> +
> +		/* Skip refs inherited from other devices */
> +		if (ref->from_other_dev)
> +			continue;
> +		/* And buttons */
> +		if (ctrl->type == V4L2_CTRL_TYPE_BUTTON)
> +			continue;
> +		err = handler_new_ref(hdl, ctrl, &new_ref, false,
> true);
> +		if (err) {
> +			printk("%s: handler_new_ref on control %x
> (%s) returned %d\n", __func__, ctrl->id, ctrl->name, err);
> +			err = 0;
> +			continue;
> +		}
> +		if (err)
> +			break;
> +	}
> +	mutex_unlock(from->lock);
> +	return err;
> +}
> +
> +static void v4l2_ctrl_request_queue(struct media_request_object *obj)
> +{
> +	struct v4l2_ctrl_handler *hdl =
> +		container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +	struct v4l2_ctrl_handler *main_hdl = obj->priv;
> +	struct v4l2_ctrl_handler *prev = NULL;
> +	struct v4l2_ctrl_ref *ref_hdl, *ref_prev = NULL;
> +
> +	if (list_empty(&main_hdl->requests_queued))
> +		goto queue;
> +
> +	prev = list_last_entry(&main_hdl->requests_queued,
> +			       struct v4l2_ctrl_handler,
> requests_queued);
> +	mutex_lock(prev->lock);
> +	ref_prev = list_first_entry(&prev->ctrl_refs,
> +				    struct v4l2_ctrl_ref, node);
> +	list_for_each_entry(ref_hdl, &hdl->ctrl_refs, node) {
> +		if (ref_hdl->req)
> +			continue;
> +		while (ref_prev->ctrl->id < ref_hdl->ctrl->id)
> +			ref_prev = list_next_entry(ref_prev, node);
> +		if (WARN_ON(ref_prev->ctrl->id != ref_hdl->ctrl->id))
> +			break;
> +		ref_hdl->req = ref_prev->req;
> +	}
> +	mutex_unlock(prev->lock);
> +queue:
> +	list_add_tail(&hdl->requests_queued, &main_hdl-
> >requests_queued);
> +	hdl->request_is_queued = true;
> +}
> +
> +static void v4l2_ctrl_request_unbind(struct media_request_object
> *obj)
> +{
> +	struct v4l2_ctrl_handler *hdl =
> +		container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +
> +	list_del_init(&hdl->requests);
> +	if (hdl->request_is_queued) {
> +		list_del_init(&hdl->requests_queued);
> +		hdl->request_is_queued = false;
> +	}
> +}
> +
> +static void v4l2_ctrl_request_cancel(struct media_request_object
> *obj)
> +{
> +	struct v4l2_ctrl_handler *hdl =
> +		container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +
> +	if (hdl->request_is_queued)
> +		v4l2_ctrl_request_complete(hdl->req_obj.req, hdl-
> >req_obj.priv);
> +}
> +
> +static void v4l2_ctrl_request_release(struct media_request_object
> *obj)
> +{
> +	struct v4l2_ctrl_handler *hdl =
> +		container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +
> +	v4l2_ctrl_handler_free(hdl);
> +	kfree(hdl);
> +}
> +
> +static const struct media_request_object_ops req_ops = {
> +	.queue = v4l2_ctrl_request_queue,
> +	.unbind = v4l2_ctrl_request_unbind,
> +	.cancel = v4l2_ctrl_request_cancel,
> +	.release = v4l2_ctrl_request_release,
> +};
> +
> +static int v4l2_ctrl_request_bind(struct media_request *req,
> +			   struct v4l2_ctrl_handler *hdl,
> +			   struct v4l2_ctrl_handler *from)
> +{
> +	int ret;
> +
> +	ret = v4l2_ctrl_request_clone(hdl, from);
> +
> +	if (!ret) {
> +		list_add_tail(&hdl->requests, &from->requests);
> +		media_request_object_bind(req, &req_ops, from, &hdl-
> >req_obj);
> +	}
> +	return ret;
> +}
>  
>  /* Some general notes on the atomic requirements of
> VIDIOC_G/TRY/S_EXT_CTRLS:
>  
> @@ -2877,6 +3035,7 @@ static int prepare_ext_ctrls(struct
> v4l2_ctrl_handler *hdl,
>  
>  		if (cs->which &&
>  		    cs->which != V4L2_CTRL_WHICH_DEF_VAL &&
> +		    cs->which != V4L2_CTRL_WHICH_REQUEST &&
>  		    V4L2_CTRL_ID2WHICH(id) != cs->which)
>  			return -EINVAL;
>  
> @@ -2956,13 +3115,12 @@ static int prepare_ext_ctrls(struct
> v4l2_ctrl_handler *hdl,
>     whether there are any controls at all. */
>  static int class_check(struct v4l2_ctrl_handler *hdl, u32 which)
>  {
> -	if (which == 0 || which == V4L2_CTRL_WHICH_DEF_VAL)
> +	if (which == 0 || which == V4L2_CTRL_WHICH_DEF_VAL ||
> +	    which == V4L2_CTRL_WHICH_REQUEST)
>  		return 0;
>  	return find_ref_lock(hdl, which | 1) ? 0 : -EINVAL;
>  }
>  
> -
> -
>  /* Get extended controls. Allocates the helpers array if needed. */
>  int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct
> v4l2_ext_controls *cs)
>  {
> @@ -3028,8 +3186,12 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler
> *hdl, struct v4l2_ext_controls *cs
>  			u32 idx = i;
>  
>  			do {
> -				ret = ctrl_to_user(cs->controls +
> idx,
> -						   helpers[idx].ref-
> >ctrl);
> +				if (helpers[idx].ref->req)
> +					ret = req_to_user(cs-
> >controls + idx,
> +						helpers[idx].ref-
> >req);
> +				else
> +					ret = ctrl_to_user(cs-
> >controls + idx,
> +						helpers[idx].ref-
> >ctrl);
>  				idx = helpers[idx].next;
>  			} while (!ret && idx);
>  		}
> @@ -3302,7 +3464,16 @@ static int try_set_ext_ctrls(struct v4l2_fh
> *fh, struct v4l2_ctrl_handler *hdl,
>  		} while (!ret && idx);
>  
>  		if (!ret)
> -			ret = try_or_set_cluster(fh, master, set, 0);
> +			ret = try_or_set_cluster(fh, master,
> +						 !hdl->req_obj.req &&
> set, 0);
> +		if (!ret && hdl->req_obj.req && set) {
> +			for (j = 0; j < master->ncontrols; j++) {
> +				struct v4l2_ctrl_ref *ref =
> +					find_ref(hdl, master-
> >cluster[j]->id);
> +
> +				new_to_req(ref);
> +			}
> +		}
>  
>  		/* Copy the new values back to userspace. */
>  		if (!ret) {
> @@ -3429,6 +3600,148 @@ int __v4l2_ctrl_s_ctrl_string(struct v4l2_ctrl
> *ctrl, const char *s)
>  }
>  EXPORT_SYMBOL(__v4l2_ctrl_s_ctrl_string);
>  
> +void v4l2_ctrl_request_complete(struct media_request *req,
> +				struct v4l2_ctrl_handler *main_hdl)
> +{
> +	struct media_request_object *obj;
> +	struct v4l2_ctrl_handler *hdl;
> +	struct v4l2_ctrl_ref *ref;
> +
> +	if (!req || !main_hdl)
> +		return;
> +
> +	obj = media_request_object_find(req, &req_ops, main_hdl);
> +	if (!obj)
> +		return;
> +	hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +
> +	list_for_each_entry(ref, &hdl->ctrl_refs, node) {
> +		struct v4l2_ctrl *ctrl = ref->ctrl;
> +		struct v4l2_ctrl *master = ctrl->cluster[0];
> +		unsigned int i;
> +
> +		if (ctrl->flags & V4L2_CTRL_FLAG_VOLATILE) {
> +			ref->req = ref;
> +
> +			v4l2_ctrl_lock(master);
> +			/* g_volatile_ctrl will update the current
> control values */
> +			for (i = 0; i < master->ncontrols; i++)
> +				cur_to_new(master->cluster[i]);
> +			call_op(master, g_volatile_ctrl);
> +			new_to_req(ref);
> +			v4l2_ctrl_unlock(master);
> +			continue;
> +		}
> +		if (ref->req == ref)
> +			continue;
> +
> +		v4l2_ctrl_lock(ctrl);
> +		if (ref->req)
> +			ptr_to_ptr(ctrl, ref->req->p_req, ref-
> >p_req);
> +		else
> +			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
> +		v4l2_ctrl_unlock(ctrl);
> +	}
> +
> +	WARN_ON(!hdl->request_is_queued);
> +	list_del_init(&hdl->requests_queued);
> +	hdl->request_is_queued = false;
> +	media_request_object_complete(obj);
> +	media_request_object_put(obj);
> +}
> +EXPORT_SYMBOL(v4l2_ctrl_request_complete);
> +
> +void v4l2_ctrl_request_setup(struct media_request *req,
> +			     struct v4l2_ctrl_handler *main_hdl)
> +{
> +	struct media_request_object *obj;
> +	struct v4l2_ctrl_handler *hdl;
> +	struct v4l2_ctrl_ref *ref;
> +
> +	if (!req || !main_hdl)
> +		return;
> +
> +	obj = media_request_object_find(req, &req_ops, main_hdl);
> +	if (!obj)
> +		return;
> +	if (obj->completed) {
> +		media_request_object_put(obj);
> +		return;
> +	}
> +	hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +
> +	mutex_lock(hdl->lock);
> +
> +	list_for_each_entry(ref, &hdl->ctrl_refs, node)
> +		ref->done = false;
> +
> +	list_for_each_entry(ref, &hdl->ctrl_refs, node) {
> +		struct v4l2_ctrl *ctrl = ref->ctrl;
> +		struct v4l2_ctrl *master = ctrl->cluster[0];
> +		bool have_new_data = false;
> +		int i;
> +
> +		/* Skip if this control was already handled by a
> cluster. */
> +		/* Skip button controls and read-only controls. */
> +		if (ref->done || ctrl->type == V4L2_CTRL_TYPE_BUTTON
> ||
> +		    (ctrl->flags & V4L2_CTRL_FLAG_READ_ONLY))
> +			continue;
> +
> +		v4l2_ctrl_lock(master);
> +		for (i = 0; i < master->ncontrols; i++) {
> +			if (master->cluster[i]) {
> +				struct v4l2_ctrl_ref *r =
> +					find_ref(hdl, master-
> >cluster[i]->id);
> +
> +				if (r->req && r == r->req) {
> +					have_new_data = true;
> +					break;
> +				}
> +			}
> +		}
> +		if (!have_new_data)

The v4l2 control lock has not been unlocked here, so a call to
v4l2_ctrl_unlock is required before continue.

> +			continue;
> +
> +		for (i = 0; i < master->ncontrols; i++) {
> +			if (master->cluster[i]) {
> +				struct v4l2_ctrl_ref *r =
> +					find_ref(hdl, master-
> >cluster[i]->id);
> +
> +				req_to_new(r);
> +				master->cluster[i]->is_new = 1;
> +				r->done = true;
> +			}
> +		}
> +		/*
> +		 * For volatile autoclusters that are currently in
> auto mode
> +		 * we need to discover if it will be set to manual
> mode.
> +		 * If so, then we have to copy the current volatile
> values
> +		 * first since those will become the new manual
> values (which
> +		 * may be overwritten by explicit new values from
> this set
> +		 * of controls).
> +		 */
> +		if (master->is_auto && master->has_volatiles &&
> +		    !is_cur_manual(master)) {
> +			s32 new_auto_val = *master->p_new.p_s32;
> +
> +			/*
> +			 * If the new value == the manual value, then
> copy
> +			 * the current volatile values.
> +			 */
> +			if (new_auto_val == master-
> >manual_mode_value)
> +				update_from_auto_cluster(master);
> +		}
> +
> +		try_or_set_cluster(NULL, master, true, 0);
> +
> +		v4l2_ctrl_unlock(master);
> +	}
> +
> +	mutex_unlock(hdl->lock);
> +	media_request_object_put(obj);
> +}
> +EXPORT_SYMBOL(v4l2_ctrl_request_setup);
> +
>  void v4l2_ctrl_notify(struct v4l2_ctrl *ctrl, v4l2_ctrl_notify_fnc
> notify, void *priv)
>  {
>  	if (ctrl == NULL)
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 89a985607126..9499846aa1d1 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -250,6 +250,10 @@ struct v4l2_ctrl {
>   *		``prepare_ext_ctrls`` function at ``v4l2-ctrl.c``.
>   * @from_other_dev: If true, then @ctrl was defined in another
>   *		device then the &struct v4l2_ctrl_handler.
> + * @done:	If true, then this control reference is part of a
> + *		control cluster that was already set while applying
> + *		the controls in this media request object.
> + * @req:	If set, this refers to another request that sets this
> control.
>   * @p_req:	The request value. Only used if the control handler
>   *		is bound to a media request.
>   *
> @@ -263,6 +267,8 @@ struct v4l2_ctrl_ref {
>  	struct v4l2_ctrl *ctrl;
>  	struct v4l2_ctrl_helper *helper;
>  	bool from_other_dev;
> +	bool done;
> +	struct v4l2_ctrl_ref *req;
>  	union v4l2_ctrl_ptr p_req;
>  };
>  
> @@ -287,6 +293,15 @@ struct v4l2_ctrl_ref {
>   * @notify_priv: Passed as argument to the v4l2_ctrl notify callback.
>   * @nr_of_buckets: Total number of buckets in the array.
>   * @error:	The error code of the first failed control
> addition.
> + * @request_is_queued: True if the request was queued.
> + * @requests:	List to keep track of open control handler
> request objects.
> + *		For the parent control handler (@req_obj.req ==
> NULL) this
> + *		is the list header. When the parent control handler
> is
> + *		removed, it has to unbind and put all these
> requests since
> + *		they refer to the parent.
> + * @requests_queued: List of the queued requests. This determines the
> order
> + *		in which these controls are applied. Once the
> request is
> + *		completed it is removed from this list.
>   * @req_obj:	The &struct media_request_object, used to link
> into a
>   *		&struct media_request.
>   */
> @@ -301,6 +316,9 @@ struct v4l2_ctrl_handler {
>  	void *notify_priv;
>  	u16 nr_of_buckets;
>  	int error;
> +	bool request_is_queued;
> +	struct list_head requests;
> +	struct list_head requests_queued;
>  	struct media_request_object req_obj;
>  };
>  
> @@ -1059,6 +1077,11 @@ int v4l2_ctrl_subscribe_event(struct v4l2_fh
> *fh,
>   */
>  __poll_t v4l2_ctrl_poll(struct file *file, struct poll_table_struct
> *wait);
>  
> +void v4l2_ctrl_request_setup(struct media_request *req,
> +			     struct v4l2_ctrl_handler *hdl);
> +void v4l2_ctrl_request_complete(struct media_request *req,
> +				struct v4l2_ctrl_handler *hdl);
> +
>  /* Helpers for ioctl_ops */
>  
>  /**
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFCv11 PATCH 14/29] v4l2-ctrls: add core request support

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Integrate the request support. This adds the v4l2_ctrl_request_complete
> and v4l2_ctrl_request_setup functions to complete a request and (as a
> helper function) to apply a request to the hardware.

Please see my comments inline.

[snip]
> +static int v4l2_ctrl_request_clone(struct v4l2_ctrl_handler *hdl,
> +                                  const struct v4l2_ctrl_handler *from)
> +{
> +       struct v4l2_ctrl_ref *ref;
> +       int err;
> +
> +       if (WARN_ON(!hdl || hdl == from))
> +               return -EINVAL;
> +
> +       if (hdl->error)
> +               return hdl->error;
> +
> +       WARN_ON(hdl->lock != &hdl->_lock);
> +
> +       mutex_lock(from->lock);
> +       list_for_each_entry(ref, &from->ctrl_refs, node) {
> +               struct v4l2_ctrl *ctrl = ref->ctrl;
> +               struct v4l2_ctrl_ref *new_ref;
> +
> +               /* Skip refs inherited from other devices */
> +               if (ref->from_other_dev)
> +                       continue;
> +               /* And buttons */
> +               if (ctrl->type == V4L2_CTRL_TYPE_BUTTON)
> +                       continue;
> +               err = handler_new_ref(hdl, ctrl, &new_ref, false, true);
> +               if (err) {
> +                       printk("%s: handler_new_ref on control %x (%s)
returned %d\n", __func__, ctrl->id, ctrl->name, err);

Perhaps pr_err()?

> +                       err = 0;
> +                       continue;

Hmm, is it really fine to ignore the error and continue here?

> +               }
> +               if (err)
> +                       break;
> +       }
> +       mutex_unlock(from->lock);
> +       return err;
> +}
> +
> +static void v4l2_ctrl_request_queue(struct media_request_object *obj)
> +{
> +       struct v4l2_ctrl_handler *hdl =
> +               container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +       struct v4l2_ctrl_handler *main_hdl = obj->priv;
> +       struct v4l2_ctrl_handler *prev = NULL;
> +       struct v4l2_ctrl_ref *ref_hdl, *ref_prev = NULL;
> +
> +       if (list_empty(&main_hdl->requests_queued))
> +               goto queue;
> +
> +       prev = list_last_entry(&main_hdl->requests_queued,
> +                              struct v4l2_ctrl_handler, requests_queued);
> +       mutex_lock(prev->lock);
> +       ref_prev = list_first_entry(&prev->ctrl_refs,
> +                                   struct v4l2_ctrl_ref, node);
> +       list_for_each_entry(ref_hdl, &hdl->ctrl_refs, node) {
> +               if (ref_hdl->req)
> +                       continue;
> +               while (ref_prev->ctrl->id < ref_hdl->ctrl->id)
> +                       ref_prev = list_next_entry(ref_prev, node);

Is this really safe? The only stop condition here is the control id.
Perhaps the code below could be better?

list_for_each_entry_from(ref_prev, &prev->ctrl_refs, node)
         if (ref_prev->ctrl->id >= ref_hdl->ctrl->id)
                 break;

> +               if (WARN_ON(ref_prev->ctrl->id != ref_hdl->ctrl->id))
> +                       break;
> +               ref_hdl->req = ref_prev->req;
> +       }
> +       mutex_unlock(prev->lock);
> +queue:
> +       list_add_tail(&hdl->requests_queued, &main_hdl->requests_queued);
> +       hdl->request_is_queued = true;
> +}
> +

[snip]
> +void v4l2_ctrl_request_complete(struct media_request *req,
> +                               struct v4l2_ctrl_handler *main_hdl)
> +{
> +       struct media_request_object *obj;
> +       struct v4l2_ctrl_handler *hdl;
> +       struct v4l2_ctrl_ref *ref;
> +
> +       if (!req || !main_hdl)
> +               return;

Can this happen normally? Perhaps WARN_ON() would make sense?

[snip]
> +void v4l2_ctrl_request_setup(struct media_request *req,
> +                            struct v4l2_ctrl_handler *main_hdl)
> +{
> +       struct media_request_object *obj;
> +       struct v4l2_ctrl_handler *hdl;
> +       struct v4l2_ctrl_ref *ref;
> +
> +       if (!req || !main_hdl)

Can this happen normally? Perhaps WARN_ON() would make sense?

> +               return;
> +
> +       obj = media_request_object_find(req, &req_ops, main_hdl);
> +       if (!obj)
> +               return;
> +       if (obj->completed) {
> +               media_request_object_put(obj);
> +               return;
> +       }
> +       hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj);
> +
> +       mutex_lock(hdl->lock);
> +
> +       list_for_each_entry(ref, &hdl->ctrl_refs, node)
> +               ref->done = false;
> +
> +       list_for_each_entry(ref, &hdl->ctrl_refs, node) {
> +               struct v4l2_ctrl *ctrl = ref->ctrl;
> +               struct v4l2_ctrl *master = ctrl->cluster[0];
> +               bool have_new_data = false;
> +               int i;
> +
> +               /* Skip if this control was already handled by a cluster.
*/
> +               /* Skip button controls and read-only controls. */
> +               if (ref->done || ctrl->type == V4L2_CTRL_TYPE_BUTTON ||
> +                   (ctrl->flags & V4L2_CTRL_FLAG_READ_ONLY))
> +                       continue;
> +
> +               v4l2_ctrl_lock(master);
> +               for (i = 0; i < master->ncontrols; i++) {
> +                       if (master->cluster[i]) {
> +                               struct v4l2_ctrl_ref *r =
> +                                       find_ref(hdl,
master->cluster[i]->id);
> +
> +                               if (r->req && r == r->req) {
> +                                       have_new_data = true;
> +                                       break;
> +                               }
> +                       }
> +               }
> +               if (!have_new_data)
> +                       continue;

No need to call v4l2_ctrl_unlock() here?

[snip]
> @@ -263,6 +267,8 @@ struct v4l2_ctrl_ref {
>          struct v4l2_ctrl *ctrl;
>          struct v4l2_ctrl_helper *helper;
>          bool from_other_dev;
> +       bool done;
> +       struct v4l2_ctrl_ref *req;

Perhaps it could make sense to call this ref_req, which would use the same
convention as p_req below?

>          union v4l2_ctrl_ptr p_req;
>   };

Best regards,
Tomasz

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Mauro Carvalho Chehab]

Em Tue, 10 Apr 2018 14:14:30 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:

> > >  /**
> > > @@ -88,6 +96,8 @@ struct media_device_ops {
> > >   * @disable_source: Disable Source Handler function pointer
> > >   *
> > >   * @ops:	Operation handler callbacks
> > > + * @req_lock:	Serialise access to requests
> > > + * @req_queue_mutex: Serialise validating and queueing requests  
> > 
> > IMHO, this would better describe it:
> > 	Serialise validate and queue requests
> > 
> > Yet, IMO, it doesn't let it clear when the spin lock should be
> > used and when the mutex should be used.
> > 
> > I mean, what of them protect what variable?  
> 
> It might not be obvious, but the purpose of this mutex is to prevent
> queueing multiple requests simultaneously in order to serialise access to
> the top of the queue.

It is not obvious at all. The purpose of a lock is to prevent
multiple acesses to the content of a data structure.

>From what I see, here we have a single structure with two locks.
To make it worse, you're introducing first the lock and then,
on patch 04/29, the actual structs to be protected.

Well, a house with two doors is only closed if both doors are
closed. The same happens with a data structure protected by
two locks. It is only locked if both locks are locked.

So, every time I see two locks meant to protect the same struct, it
sounds poorly designed or wrong. 

There's one exception, though: if you have an independent
guest house and a main house, you could have two locks at the
same place, one for the main house and another one for the
guest house, but in this case, you should clearly tag with lock
protects each house.

So, in this case, two new locks that are being proposed to be added
at for struct media_device. So, I would be expecting a comment
like:

	@req_lock: serialize access to &struct media_request 
	@req_queue_mutex: serialize access to &struct media_request_object

Clearly stating what data structures each lock protects.

That was my concern when I pointed it. After looking at the entire
patchset, what I saw was a non-consistent locking model.

It sounded to me that the original design to be:

1) req_queue_mutex was designed to protect struct media_request
   instances;

2) req_lock was designed to protect just one field inside
   struct media_request (the state field).

There is a big issue on that:

As state is part of media_request, assuming that the data struct
won't disappear while in use (with is warranted by kref), before
changing its value and touching other fields at req, the code should 
be locking both req_queue_mutex and req_lock, but I didn't see that
behavior on several places.

Also, I noticed several locking inconsistencies, as, on several places,
the content of a media_request instance and/or its state was 
accessed/altered without either locks.

> How about this instead:
> 
> 	Serialise access to accessing device state on the tail of the
> 	request queue.

It still doesn't mention what struct each of the new locks protect.
IMHO, this patch should either be bound with patch 04/29 or come after
that, and explicitly mention what data is protected by each lock.

Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

Hi Mauro,

On Tue, Apr 10, 2018 at 11:51:43AM -0300, Mauro Carvalho Chehab wrote:
> Em Tue, 10 Apr 2018 15:32:34 +0300
> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> 
> > Hi Mauro and Hans,
> > 
> > On Tue, Apr 10, 2018 at 07:32:06AM -0300, Mauro Carvalho Chehab wrote:
> > ...
> > > > +static void media_request_release(struct kref *kref)
> > > > +{
> > > > +	struct media_request *req =
> > > > +		container_of(kref, struct media_request, kref);
> > > > +	struct media_device *mdev = req->mdev;
> > > > +	unsigned long flags;
> > > > +
> > > > +	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> > > > +
> > > > +	spin_lock_irqsave(&req->lock, flags);
> > > > +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> > > > +	spin_unlock_irqrestore(&req->lock, flags);
> > > > +
> > > > +	media_request_clean(req);
> > > > +
> > > > +	if (mdev->ops->req_free)
> > > > +		mdev->ops->req_free(req);
> > > > +	else
> > > > +		kfree(req);  
> > > 
> > > Without looking at req_free() implementation, I would actually prefer
> > > to always do a kfree(req) here. So, a req_free() function would only
> > > free "extra" allocations, and not the request itself. e. g.:
> > > 
> > > ...
> > > 	if (mdev->ops->req_free)
> > > 		mdev->ops->req_free(req);
> > > 
> > > 	kfree(req);
> > > }  
> > 
> > The idea is that you can embed the request object in a driver specific
> > struct. The drivers can store information related to that request rather
> > easily that way, without requiring to be aware of two objects with
> > references pointing to each other. I rather prefer the current
> > implementation. It same pattern is actually used on videobuf2 buffers.
> 
> Ok, then document it so.
> 
> Btw, one of the things it is missing is a kAPI documentation patch,
> describing things like that.

I agree. Documentation will need to be added and improved.

> 
> > 
> > ...
> > 
> > > > +static unsigned int media_request_poll(struct file *filp,
> > > > +				       struct poll_table_struct *wait)
> > > > +{
> > > > +	struct media_request *req = filp->private_data;
> > > > +	unsigned long flags;
> > > > +	enum media_request_state state;
> > > > +
> > > > +	if (!(poll_requested_events(wait) & POLLPRI))
> > > > +		return 0;
> > > > +
> > > > +	spin_lock_irqsave(&req->lock, flags);
> > > > +	state = req->state;
> > > > +	spin_unlock_irqrestore(&req->lock, flags);  
> > > 
> > > IMO, it would be better to use an atomic var for state, having a
> > > lockless access to it.  
> > 
> > The lock serialises access to the whole request, not just to its state.
> 
> From what I understood at the code is that the spin lock
> is used *sometimes* to protect just the state. I didn't
> see it used to protect the hole data struct.
> 
> Instead, the mutex is used for that purpose, but, again,
> it is *sometimes* used, but on several parts, neither the
> spin lock nor the mutex is used.
> 
> It should be noticed that the data, itself, should be protected
> by *either* mutex or spin lock.
> 
> > While it doesn't matter here as you're just reading the state, writing it
> > would still require taking the lock. Using atomic_t might suggest
> > otherwise, and could end up being a source of bugs.
> 
> There are already a lot of bugs at the locking, from what I noticed.
> 
> We should do it right: it should use *just one* kind of memory
> protection for struct media_request. On the places where it is
> safe to just read the status without locking, atomic_t() should
> be used. Where it doesn't, probably the entire code should be
> protected by the lock.

If done that way, it needs to be documented because it isn't obvious.
Keeping the code simple helps a lot; I don't see the core code growing a
lot so this looks good.

> 
> > >   
> > > > +
> > > > +	if (state == MEDIA_REQUEST_STATE_COMPLETE)
> > > > +		return POLLPRI;
> > > > +	if (state == MEDIA_REQUEST_STATE_IDLE)
> > > > +		return POLLERR;
> > > > +
> > > > +	poll_wait(filp, &req->poll_wait, wait);
> > > > +	return 0;
> > > > +}
> > > > +
> > > > +static long media_request_ioctl(struct file *filp, unsigned int cmd,
> > > > +				unsigned long __arg)
> > > > +{
> > > > +	return -ENOIOCTLCMD;
> > > > +}
> > > > +
> > > > +static const struct file_operations request_fops = {
> > > > +	.owner = THIS_MODULE,
> > > > +	.poll = media_request_poll,
> > > > +	.unlocked_ioctl = media_request_ioctl,
> > > > +	.release = media_request_close,
> > > > +};
> > > > +
> > > >  int media_request_alloc(struct media_device *mdev,
> > > >  			struct media_request_alloc *alloc)
> > > >  {
> > > > -	return -ENOMEM;
> > > > +	struct media_request *req;
> > > > +	struct file *filp;
> > > > +	char comm[TASK_COMM_LEN];
> > > > +	int fd;
> > > > +	int ret;
> > > > +
> > > > +	fd = get_unused_fd_flags(O_CLOEXEC);
> > > > +	if (fd < 0)
> > > > +		return fd;
> > > > +
> > > > +	filp = anon_inode_getfile("request", &request_fops, NULL, O_CLOEXEC);
> > > > +	if (IS_ERR(filp)) {
> > > > +		ret = PTR_ERR(filp);
> > > > +		goto err_put_fd;
> > > > +	}
> > > > +
> > > > +	if (mdev->ops->req_alloc)
> > > > +		req = mdev->ops->req_alloc(mdev);
> > > > +	else
> > > > +		req = kzalloc(sizeof(*req), GFP_KERNEL);
> > > > +	if (!req) {
> > > > +		ret = -ENOMEM;
> > > > +		goto err_fput;
> > > > +	}
> > > > +
> > > > +	filp->private_data = req;
> > > > +	req->mdev = mdev;
> > > > +	req->state = MEDIA_REQUEST_STATE_IDLE;
> > > > +	req->num_incomplete_objects = 0;
> > > > +	kref_init(&req->kref);
> > > > +	INIT_LIST_HEAD(&req->objects);
> > > > +	spin_lock_init(&req->lock);
> > > > +	init_waitqueue_head(&req->poll_wait);
> > > > +
> > > > +	alloc->fd = fd;  
> > > 
> > > Btw, this is a very good reason why you should define the ioctl to
> > > have an integer argument instead of a struct with a __s32 field
> > > on it, as per my comment to patch 02/29:
> > > 
> > > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > > 
> > > At 64 bit architectures, you're truncating the file descriptor!  
> > 
> > I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> > well.
> 
> Hmm.. you're right. I was thinking that it could be 64 bits on some
> archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> according with:
> 
> 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> 
> This is not the case on gcc.

Ok. The reasoning back then was that what "int" means varies across
compilers and languages. And the intent was to codify this to __s32 which
is what the kernel effectively uses.

The rest of the kernel uses int rather liberally in the uAPI so I'm not
sure in the end whether something desirable was achieved. Perhaps it'd be
good to go back to the original discussion to find out for sure.

Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
due to that difference.

Well, I stop here for this begins to be off-topic. :-)

> 
> > We actually replaced int by __s32 or __u32 everywhere in uAPI (apart from
> > one particular struct; time to send a patch) about five years ago.
> > 
> > >   
> > > > +	get_task_comm(comm, current);
> > > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > > +		 comm, fd);  
> > > 
> > > Not sure if it is a good idea to store the task that allocated
> > > the request. While it makes sense for the dev_dbg() below, it
> > > may not make sense anymore on other dev_dbg() you would be
> > > using it.  
> > 
> > The lifetime of the file handle roughly matches that of the request. It's
> > for debug only anyway.
> > 
> > Better proposals are always welcome of course. But I think we should have
> > something here that helps debugging by meaningfully making the requests
> > identifiable from logs.
> 
> What I meant to say is that one PID could be allocating the
> request, while some other one could be actually doing Q/DQ_BUF.
> On such scenario, the debug string could provide mislead prints.

Um, yes, indeed it would no longer match the process. But the request is
still the same. That's actually a positive thing since it allows you to
identify the request.

With a global ID space this was trivial; you could just print the request
ID and that was all that was ever needed. (I'm not proposing to consider
that though.)

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Mauro Carvalho Chehab]

Em Wed, 11 Apr 2018 16:21:16 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:


> > > > Btw, this is a very good reason why you should define the ioctl to
> > > > have an integer argument instead of a struct with a __s32 field
> > > > on it, as per my comment to patch 02/29:
> > > > 
> > > > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > > > 
> > > > At 64 bit architectures, you're truncating the file descriptor!    
> > > 
> > > I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> > > well.  
> > 
> > Hmm.. you're right. I was thinking that it could be 64 bits on some
> > archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> > according with:
> > 
> > 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> > 
> > This is not the case on gcc.  
> 
> Ok. The reasoning back then was that what "int" means varies across
> compilers and languages. And the intent was to codify this to __s32 which
> is what the kernel effectively uses.

...

> The rest of the kernel uses int rather liberally in the uAPI so I'm not
> sure in the end whether something desirable was achieved. Perhaps it'd be
> good to go back to the original discussion to find out for sure.
> 
> Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
> due to that difference.
> 
> Well, I stop here for this begins to be off-topic. :-)

Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
about that :-)

> > > > > +	get_task_comm(comm, current);
> > > > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > > > +		 comm, fd);    
> > > > 
> > > > Not sure if it is a good idea to store the task that allocated
> > > > the request. While it makes sense for the dev_dbg() below, it
> > > > may not make sense anymore on other dev_dbg() you would be
> > > > using it.    
> > > 
> > > The lifetime of the file handle roughly matches that of the request. It's
> > > for debug only anyway.
> > > 
> > > Better proposals are always welcome of course. But I think we should have
> > > something here that helps debugging by meaningfully making the requests
> > > identifiable from logs.  
> > 
> > What I meant to say is that one PID could be allocating the
> > request, while some other one could be actually doing Q/DQ_BUF.
> > On such scenario, the debug string could provide mislead prints.  
> 
> Um, yes, indeed it would no longer match the process. But the request is
> still the same. That's actually a positive thing since it allows you to
> identify the request.
> 
> With a global ID space this was trivial; you could just print the request
> ID and that was all that was ever needed. (I'm not proposing to consider
> that though.)
> 

IMO, a global ID number would work better than get_task_comm().

Just add a static int monotonic counter and use it for the debug purposes,
e. g.:

{
	static unsigned int req_count = 0;

	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
		req_count++, fd);    

Ok, eventually, it will overflow, but, it will be unique within
a reasonable timeframe to be good enough for debugging purposes.


Thanks,
Mauro

Re: [RFCv11 PATCH 27/29] vim2m: support requests

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 3886 bytes --]

Hi,

On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add support for requests to vim2m.

Depending on where STREAMON happens in the sequence, there is a
possibility that v4l2_m2m_try_schedule is never called and thus that
device_run never runs when submitting the request.

More specifically, the m2m fashion of the STREAMON ioctl handler will
normally call v4l2_m2m_try_schedule. Hence, there is no issue if it's
called after submitting the request. On the other hand, if the request
was not submitted when calling STREAMON (which is a valid use case),
buffers are not available and v4l2_m2m_try_schedule won't schedule
anything. Once the request is submitted, the internal plumbing will
detect that STREAMON was requested but did not take effect, so it will
call vb2_streamon. And of course, vb2_streamon has no provision for
trying to schedule a m2m device run.

One way to fix this is to call v4l2_m2m_try_schedule from a workqueue
triggered in the driver's start_streaming qop. The same is also probably
necessary in buf_queue and buf_prepare since those qops are not called
from the m2m ioctl handler either.

What do you think?

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
> index 9b18b32c255d..2dcf0ea85705 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -387,8 +387,26 @@ static void device_run(void *priv)
>  	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
>  	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
>  
> +	/* Apply request if needed */
> +	if (src_buf->vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +	if (dst_buf->vb2_buf.req_obj.req &&
> +	    dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +
>  	device_process(ctx, src_buf, dst_buf);
>  
> +	/* Complete request if needed */
> +	if (src_buf->vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_complete(src_buf-
> >vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +	if (dst_buf->vb2_buf.req_obj.req &&
> +	    dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_complete(dst_buf-
> >vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +
>  	/* Run a timer, which simulates a hardware irq  */
>  	schedule_irq(dev, ctx->transtime);
>  }
> @@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue
> *q)
>  			vbuf = v4l2_m2m_dst_buf_remove(ctx-
> >fh.m2m_ctx);
>  		if (vbuf == NULL)
>  			return;
> +		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
> +					   &ctx->hdl);
>  		spin_lock_irqsave(&ctx->dev->irqlock, flags);
>  		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
>  		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> @@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
>  	.job_abort	= job_abort,
>  };
>  
> +static const struct media_device_ops m2m_media_ops = {
> +	.req_queue = vb2_request_queue,
> +};
> +
>  static int vim2m_probe(struct platform_device *pdev)
>  {
>  	struct vim2m_dev *dev;
> @@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device
> *pdev)
>  	dev->mdev.dev = &pdev->dev;
>  	strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
>  	media_device_init(&dev->mdev);
> +	dev->mdev.ops = &m2m_media_ops;
>  	dev->v4l2_dev.mdev = &dev->mdev;
>  	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
>  	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFCv11 PATCH 26/29] vim2m: use workqueue

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2409 bytes --]

Hi,

On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> v4l2_ctrl uses mutexes, so we can't setup a ctrl_handler in
> interrupt context. Switch to a workqueue instead.

See one comment below.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/vim2m.c | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
> index ef970434af13..9b18b32c255d 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -150,6 +150,7 @@ struct vim2m_dev {
>  	spinlock_t		irqlock;
>  
>  	struct timer_list	timer;
> +	struct work_struct	work_run;

Wouldn't it make more sense to move this to vim2m_ctx instead (since
this is heavily m2m-specific)?

>  	struct v4l2_m2m_dev	*m2m_dev;
>  };
> @@ -392,9 +393,10 @@ static void device_run(void *priv)
>  	schedule_irq(dev, ctx->transtime);
>  }
>  
> -static void device_isr(struct timer_list *t)
> +static void device_work(struct work_struct *w)
>  {
> -	struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t,
> timer);
> +	struct vim2m_dev *vim2m_dev =
> +		container_of(w, struct vim2m_dev, work_run);
>  	struct vim2m_ctx *curr_ctx;
>  	struct vb2_v4l2_buffer *src_vb, *dst_vb;
>  	unsigned long flags;
> @@ -426,6 +428,13 @@ static void device_isr(struct timer_list *t)
>  	}
>  }
>  
> +static void device_isr(struct timer_list *t)
> +{
> +	struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t,
> timer);
> +
> +	schedule_work(&vim2m_dev->work_run);
> +}
> +
>  /*
>   * video ioctls
>   */
> @@ -806,6 +815,7 @@ static void vim2m_stop_streaming(struct vb2_queue
> *q)
>  	struct vb2_v4l2_buffer *vbuf;
>  	unsigned long flags;
>  
> +	flush_scheduled_work();
>  	for (;;) {
>  		if (V4L2_TYPE_IS_OUTPUT(q->type))
>  			vbuf = v4l2_m2m_src_buf_remove(ctx-
> >fh.m2m_ctx);
> @@ -1011,6 +1021,7 @@ static int vim2m_probe(struct platform_device
> *pdev)
>  	vfd = &dev->vfd;
>  	vfd->lock = &dev->dev_mutex;
>  	vfd->v4l2_dev = &dev->v4l2_dev;
> +	INIT_WORK(&dev->work_run, device_work);
>  
>  #ifdef CONFIG_MEDIA_CONTROLLER
>  	dev->mdev.dev = &pdev->dev;
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

On Wed, Apr 11, 2018 at 10:49:35AM -0300, Mauro Carvalho Chehab wrote:
> Em Wed, 11 Apr 2018 16:21:16 +0300
> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> 
> 
> > > > > Btw, this is a very good reason why you should define the ioctl to
> > > > > have an integer argument instead of a struct with a __s32 field
> > > > > on it, as per my comment to patch 02/29:
> > > > > 
> > > > > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > > > > 
> > > > > At 64 bit architectures, you're truncating the file descriptor!    
> > > > 
> > > > I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> > > > well.  
> > > 
> > > Hmm.. you're right. I was thinking that it could be 64 bits on some
> > > archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> > > according with:
> > > 
> > > 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> > > 
> > > This is not the case on gcc.  
> > 
> > Ok. The reasoning back then was that what "int" means varies across
> > compilers and languages. And the intent was to codify this to __s32 which
> > is what the kernel effectively uses.
> 
> ...
> 
> > The rest of the kernel uses int rather liberally in the uAPI so I'm not
> > sure in the end whether something desirable was achieved. Perhaps it'd be
> > good to go back to the original discussion to find out for sure.
> > 
> > Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
> > due to that difference.
> > 
> > Well, I stop here for this begins to be off-topic. :-)
> 
> Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
> about that :-)
> 
> > > > > > +	get_task_comm(comm, current);
> > > > > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > > > > +		 comm, fd);    
> > > > > 
> > > > > Not sure if it is a good idea to store the task that allocated
> > > > > the request. While it makes sense for the dev_dbg() below, it
> > > > > may not make sense anymore on other dev_dbg() you would be
> > > > > using it.    
> > > > 
> > > > The lifetime of the file handle roughly matches that of the request. It's
> > > > for debug only anyway.
> > > > 
> > > > Better proposals are always welcome of course. But I think we should have
> > > > something here that helps debugging by meaningfully making the requests
> > > > identifiable from logs.  
> > > 
> > > What I meant to say is that one PID could be allocating the
> > > request, while some other one could be actually doing Q/DQ_BUF.
> > > On such scenario, the debug string could provide mislead prints.  
> > 
> > Um, yes, indeed it would no longer match the process. But the request is
> > still the same. That's actually a positive thing since it allows you to
> > identify the request.
> > 
> > With a global ID space this was trivial; you could just print the request
> > ID and that was all that was ever needed. (I'm not proposing to consider
> > that though.)
> > 
> 
> IMO, a global ID number would work better than get_task_comm().
> 
> Just add a static int monotonic counter and use it for the debug purposes,
> e. g.:
> 
> {
> 	static unsigned int req_count = 0;
> 
> 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
> 		req_count++, fd);    
> 
> Ok, eventually, it will overflow, but, it will be unique within
> a reasonable timeframe to be good enough for debugging purposes.

Yes, but you can't figure out which process allocated it anymore, making
associating kernel debug logs with user space process logs harder.

How about process id + file handle? That still doesn't tell which process
operated on the request though, but I'm not sure whether that's really a
crucial piece of information.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Mauro Carvalho Chehab]

Em Wed, 11 Apr 2018 18:02:19 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:

> On Wed, Apr 11, 2018 at 10:49:35AM -0300, Mauro Carvalho Chehab wrote:
> > Em Wed, 11 Apr 2018 16:21:16 +0300
> > Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> > 
> >   
> > > > > > Btw, this is a very good reason why you should define the ioctl to
> > > > > > have an integer argument instead of a struct with a __s32 field
> > > > > > on it, as per my comment to patch 02/29:
> > > > > > 
> > > > > > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > > > > > 
> > > > > > At 64 bit architectures, you're truncating the file descriptor!      
> > > > > 
> > > > > I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> > > > > well.    
> > > > 
> > > > Hmm.. you're right. I was thinking that it could be 64 bits on some
> > > > archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> > > > according with:
> > > > 
> > > > 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> > > > 
> > > > This is not the case on gcc.    
> > > 
> > > Ok. The reasoning back then was that what "int" means varies across
> > > compilers and languages. And the intent was to codify this to __s32 which
> > > is what the kernel effectively uses.  
> > 
> > ...
> >   
> > > The rest of the kernel uses int rather liberally in the uAPI so I'm not
> > > sure in the end whether something desirable was achieved. Perhaps it'd be
> > > good to go back to the original discussion to find out for sure.
> > > 
> > > Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
> > > due to that difference.
> > > 
> > > Well, I stop here for this begins to be off-topic. :-)  
> > 
> > Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
> > about that :-)
> >   
> > > > > > > +	get_task_comm(comm, current);
> > > > > > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > > > > > +		 comm, fd);      
> > > > > > 
> > > > > > Not sure if it is a good idea to store the task that allocated
> > > > > > the request. While it makes sense for the dev_dbg() below, it
> > > > > > may not make sense anymore on other dev_dbg() you would be
> > > > > > using it.      
> > > > > 
> > > > > The lifetime of the file handle roughly matches that of the request. It's
> > > > > for debug only anyway.
> > > > > 
> > > > > Better proposals are always welcome of course. But I think we should have
> > > > > something here that helps debugging by meaningfully making the requests
> > > > > identifiable from logs.    
> > > > 
> > > > What I meant to say is that one PID could be allocating the
> > > > request, while some other one could be actually doing Q/DQ_BUF.
> > > > On such scenario, the debug string could provide mislead prints.    
> > > 
> > > Um, yes, indeed it would no longer match the process. But the request is
> > > still the same. That's actually a positive thing since it allows you to
> > > identify the request.
> > > 
> > > With a global ID space this was trivial; you could just print the request
> > > ID and that was all that was ever needed. (I'm not proposing to consider
> > > that though.)
> > >   
> > 
> > IMO, a global ID number would work better than get_task_comm().
> > 
> > Just add a static int monotonic counter and use it for the debug purposes,
> > e. g.:
> > 
> > {
> > 	static unsigned int req_count = 0;
> > 
> > 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
> > 		req_count++, fd);    
> > 
> > Ok, eventually, it will overflow, but, it will be unique within
> > a reasonable timeframe to be good enough for debugging purposes.  
> 
> Yes, but you can't figure out which process allocated it anymore, making
> associating kernel debug logs with user space process logs harder.
> 
> How about process id + file handle? That still doesn't tell which process
> operated on the request though, but I'm not sure whether that's really a
> crucial piece of information.

You don't need that. With dev_dbg() - and other *_dbg() macros - you can
enable process ID for all debug messages.

Basically, if the user needs the PID, all it needs is to use "+pt",
e. g. something like:

	echo "file drivers/media/* +pt" > /sys/kernel/debug/dynamic_debug/control


[1] see:
	https://www.kernel.org/doc/html/v4.11/admin-guide/dynamic-debug-howto.html

Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

On Wed, Apr 11, 2018 at 12:17:27PM -0300, Mauro Carvalho Chehab wrote:
> Em Wed, 11 Apr 2018 18:02:19 +0300
> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> 
> > On Wed, Apr 11, 2018 at 10:49:35AM -0300, Mauro Carvalho Chehab wrote:
> > > Em Wed, 11 Apr 2018 16:21:16 +0300
> > > Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> > > 
> > >   
> > > > > > > Btw, this is a very good reason why you should define the ioctl to
> > > > > > > have an integer argument instead of a struct with a __s32 field
> > > > > > > on it, as per my comment to patch 02/29:
> > > > > > > 
> > > > > > > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > > > > > > 
> > > > > > > At 64 bit architectures, you're truncating the file descriptor!      
> > > > > > 
> > > > > > I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> > > > > > well.    
> > > > > 
> > > > > Hmm.. you're right. I was thinking that it could be 64 bits on some
> > > > > archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> > > > > according with:
> > > > > 
> > > > > 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> > > > > 
> > > > > This is not the case on gcc.    
> > > > 
> > > > Ok. The reasoning back then was that what "int" means varies across
> > > > compilers and languages. And the intent was to codify this to __s32 which
> > > > is what the kernel effectively uses.  
> > > 
> > > ...
> > >   
> > > > The rest of the kernel uses int rather liberally in the uAPI so I'm not
> > > > sure in the end whether something desirable was achieved. Perhaps it'd be
> > > > good to go back to the original discussion to find out for sure.
> > > > 
> > > > Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
> > > > due to that difference.
> > > > 
> > > > Well, I stop here for this begins to be off-topic. :-)  
> > > 
> > > Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
> > > about that :-)
> > >   
> > > > > > > > +	get_task_comm(comm, current);
> > > > > > > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > > > > > > +		 comm, fd);      
> > > > > > > 
> > > > > > > Not sure if it is a good idea to store the task that allocated
> > > > > > > the request. While it makes sense for the dev_dbg() below, it
> > > > > > > may not make sense anymore on other dev_dbg() you would be
> > > > > > > using it.      
> > > > > > 
> > > > > > The lifetime of the file handle roughly matches that of the request. It's
> > > > > > for debug only anyway.
> > > > > > 
> > > > > > Better proposals are always welcome of course. But I think we should have
> > > > > > something here that helps debugging by meaningfully making the requests
> > > > > > identifiable from logs.    
> > > > > 
> > > > > What I meant to say is that one PID could be allocating the
> > > > > request, while some other one could be actually doing Q/DQ_BUF.
> > > > > On such scenario, the debug string could provide mislead prints.    
> > > > 
> > > > Um, yes, indeed it would no longer match the process. But the request is
> > > > still the same. That's actually a positive thing since it allows you to
> > > > identify the request.
> > > > 
> > > > With a global ID space this was trivial; you could just print the request
> > > > ID and that was all that was ever needed. (I'm not proposing to consider
> > > > that though.)
> > > >   
> > > 
> > > IMO, a global ID number would work better than get_task_comm().
> > > 
> > > Just add a static int monotonic counter and use it for the debug purposes,
> > > e. g.:
> > > 
> > > {
> > > 	static unsigned int req_count = 0;
> > > 
> > > 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
> > > 		req_count++, fd);    
> > > 
> > > Ok, eventually, it will overflow, but, it will be unique within
> > > a reasonable timeframe to be good enough for debugging purposes.  
> > 
> > Yes, but you can't figure out which process allocated it anymore, making
> > associating kernel debug logs with user space process logs harder.
> > 
> > How about process id + file handle? That still doesn't tell which process
> > operated on the request though, but I'm not sure whether that's really a
> > crucial piece of information.
> 
> You don't need that. With dev_dbg() - and other *_dbg() macros - you can
> enable process ID for all debug messages.

With this, the problem again is that it does not uniquely identify the
request: the request is the same request independently of which process
would operate on it. Or whether it is being processed in an interrupt
context.

AFAICT, the allocator PID (or process name) + file handle are both required
to match a request between user and kernel space logs.

> 
> Basically, if the user needs the PID, all it needs is to use "+pt",
> e. g. something like:
> 
> 	echo "file drivers/media/* +pt" > /sys/kernel/debug/dynamic_debug/control

Thanks for the info. I didn't know this.

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Mauro Carvalho Chehab]

Em Wed, 11 Apr 2018 18:35:14 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:

> On Wed, Apr 11, 2018 at 12:17:27PM -0300, Mauro Carvalho Chehab wrote:
> > Em Wed, 11 Apr 2018 18:02:19 +0300
> > Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> >   
> > > On Wed, Apr 11, 2018 at 10:49:35AM -0300, Mauro Carvalho Chehab wrote:  
> > > > Em Wed, 11 Apr 2018 16:21:16 +0300
> > > > Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> > > > 
> > > >     
> > > > > > > > Btw, this is a very good reason why you should define the ioctl to
> > > > > > > > have an integer argument instead of a struct with a __s32 field
> > > > > > > > on it, as per my comment to patch 02/29:
> > > > > > > > 
> > > > > > > > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > > > > > > > 
> > > > > > > > At 64 bit architectures, you're truncating the file descriptor!        
> > > > > > > 
> > > > > > > I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> > > > > > > well.      
> > > > > > 
> > > > > > Hmm.. you're right. I was thinking that it could be 64 bits on some
> > > > > > archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> > > > > > according with:
> > > > > > 
> > > > > > 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> > > > > > 
> > > > > > This is not the case on gcc.      
> > > > > 
> > > > > Ok. The reasoning back then was that what "int" means varies across
> > > > > compilers and languages. And the intent was to codify this to __s32 which
> > > > > is what the kernel effectively uses.    
> > > > 
> > > > ...
> > > >     
> > > > > The rest of the kernel uses int rather liberally in the uAPI so I'm not
> > > > > sure in the end whether something desirable was achieved. Perhaps it'd be
> > > > > good to go back to the original discussion to find out for sure.
> > > > > 
> > > > > Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
> > > > > due to that difference.
> > > > > 
> > > > > Well, I stop here for this begins to be off-topic. :-)    
> > > > 
> > > > Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
> > > > about that :-)
> > > >     
> > > > > > > > > +	get_task_comm(comm, current);
> > > > > > > > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > > > > > > > +		 comm, fd);        
> > > > > > > > 
> > > > > > > > Not sure if it is a good idea to store the task that allocated
> > > > > > > > the request. While it makes sense for the dev_dbg() below, it
> > > > > > > > may not make sense anymore on other dev_dbg() you would be
> > > > > > > > using it.        
> > > > > > > 
> > > > > > > The lifetime of the file handle roughly matches that of the request. It's
> > > > > > > for debug only anyway.
> > > > > > > 
> > > > > > > Better proposals are always welcome of course. But I think we should have
> > > > > > > something here that helps debugging by meaningfully making the requests
> > > > > > > identifiable from logs.      
> > > > > > 
> > > > > > What I meant to say is that one PID could be allocating the
> > > > > > request, while some other one could be actually doing Q/DQ_BUF.
> > > > > > On such scenario, the debug string could provide mislead prints.      
> > > > > 
> > > > > Um, yes, indeed it would no longer match the process. But the request is
> > > > > still the same. That's actually a positive thing since it allows you to
> > > > > identify the request.
> > > > > 
> > > > > With a global ID space this was trivial; you could just print the request
> > > > > ID and that was all that was ever needed. (I'm not proposing to consider
> > > > > that though.)
> > > > >     
> > > > 
> > > > IMO, a global ID number would work better than get_task_comm().
> > > > 
> > > > Just add a static int monotonic counter and use it for the debug purposes,
> > > > e. g.:
> > > > 
> > > > {
> > > > 	static unsigned int req_count = 0;
> > > > 
> > > > 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
> > > > 		req_count++, fd);    
> > > > 
> > > > Ok, eventually, it will overflow, but, it will be unique within
> > > > a reasonable timeframe to be good enough for debugging purposes.    
> > > 
> > > Yes, but you can't figure out which process allocated it anymore, making
> > > associating kernel debug logs with user space process logs harder.
> > > 
> > > How about process id + file handle? That still doesn't tell which process
> > > operated on the request though, but I'm not sure whether that's really a
> > > crucial piece of information.  
> > 
> > You don't need that. With dev_dbg() - and other *_dbg() macros - you can
> > enable process ID for all debug messages.  
> 
> With this, the problem again is that it does not uniquely identify the
> request: the request is the same request independently of which process
> would operate on it. Or whether it is being processed in an interrupt
> context.
> 
> AFAICT, the allocator PID (or process name) + file handle are both required
> to match a request between user and kernel space logs.

Sorry, I was unable to understand what you're saying.

If you set the debug string with:

	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d", req_count++, fd);  

With the remaining stuff at patch 04/29, e. g. those two printks:

	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);

And use "+pt" to enable those debug messages, for the request #1 with fd #45, 
created by PID 16613 you would have a log like:

	[  269.021116] [16613] request: allocated 1:45
	[  269.024118] [16613] request: release 1:45

(assuming that the same PID would create and release)

The "1:45" is an unique global ID that would allow tracking it, even
if Q/DQ_BUF is done by some other PID.

E. g. if a PID#16618 were responsible for Q/DQ_BUF, you would have
something like:

	[  269.021116] [16613] request: allocated 1:45
	[  269.021117] [16618] request: Q_BUF 1:45
	[  269.021118] [16618] request: DQ_BUF 1:45
	[  269.024118] [16613] request: release 1:45

(assuming that you would have a Q_BUF/DQ_BUF similar dev_dbg())

That seems good enough to track it.

Yet, in order to make easier to track, I would actually change the
dev_dbg() parameter order everywhere to something like:

	dev_dbg(mdev->dev, "request#%s: allocated\n", req->debug_str)
	dev_dbg(mdev->dev, "request#%s: release\n", req->debug_str);

In order to print something like:

	[  269.021116] [16613] request#1:45: allocated 
	[  269.021117] [16618] request#1:45: Q_BUF
	[  269.021118] [16618] request#1:45: DQ_BUF
	[  269.024118] [16613] request#1:45: release

Then, getting everything related to the first request would be as simple as:

	$ dmesg|grep request#1:

That will provide the PID for both processes: the one that
created/released and the one that queued/dequeued.

Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

On Wed, Apr 11, 2018 at 01:13:44PM -0300, Mauro Carvalho Chehab wrote:
> Em Wed, 11 Apr 2018 18:35:14 +0300
> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> 
> > On Wed, Apr 11, 2018 at 12:17:27PM -0300, Mauro Carvalho Chehab wrote:
> > > Em Wed, 11 Apr 2018 18:02:19 +0300
> > > Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> > >   
> > > > On Wed, Apr 11, 2018 at 10:49:35AM -0300, Mauro Carvalho Chehab wrote:  
> > > > > Em Wed, 11 Apr 2018 16:21:16 +0300
> > > > > Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> > > > > 
> > > > >     
> > > > > > > > > Btw, this is a very good reason why you should define the ioctl to
> > > > > > > > > have an integer argument instead of a struct with a __s32 field
> > > > > > > > > on it, as per my comment to patch 02/29:
> > > > > > > > > 
> > > > > > > > > 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> > > > > > > > > 
> > > > > > > > > At 64 bit architectures, you're truncating the file descriptor!        
> > > > > > > > 
> > > > > > > > I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> > > > > > > > well.      
> > > > > > > 
> > > > > > > Hmm.. you're right. I was thinking that it could be 64 bits on some
> > > > > > > archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> > > > > > > according with:
> > > > > > > 
> > > > > > > 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> > > > > > > 
> > > > > > > This is not the case on gcc.      
> > > > > > 
> > > > > > Ok. The reasoning back then was that what "int" means varies across
> > > > > > compilers and languages. And the intent was to codify this to __s32 which
> > > > > > is what the kernel effectively uses.    
> > > > > 
> > > > > ...
> > > > >     
> > > > > > The rest of the kernel uses int rather liberally in the uAPI so I'm not
> > > > > > sure in the end whether something desirable was achieved. Perhaps it'd be
> > > > > > good to go back to the original discussion to find out for sure.
> > > > > > 
> > > > > > Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
> > > > > > due to that difference.
> > > > > > 
> > > > > > Well, I stop here for this begins to be off-topic. :-)    
> > > > > 
> > > > > Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
> > > > > about that :-)
> > > > >     
> > > > > > > > > > +	get_task_comm(comm, current);
> > > > > > > > > > +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> > > > > > > > > > +		 comm, fd);        
> > > > > > > > > 
> > > > > > > > > Not sure if it is a good idea to store the task that allocated
> > > > > > > > > the request. While it makes sense for the dev_dbg() below, it
> > > > > > > > > may not make sense anymore on other dev_dbg() you would be
> > > > > > > > > using it.        
> > > > > > > > 
> > > > > > > > The lifetime of the file handle roughly matches that of the request. It's
> > > > > > > > for debug only anyway.
> > > > > > > > 
> > > > > > > > Better proposals are always welcome of course. But I think we should have
> > > > > > > > something here that helps debugging by meaningfully making the requests
> > > > > > > > identifiable from logs.      
> > > > > > > 
> > > > > > > What I meant to say is that one PID could be allocating the
> > > > > > > request, while some other one could be actually doing Q/DQ_BUF.
> > > > > > > On such scenario, the debug string could provide mislead prints.      
> > > > > > 
> > > > > > Um, yes, indeed it would no longer match the process. But the request is
> > > > > > still the same. That's actually a positive thing since it allows you to
> > > > > > identify the request.
> > > > > > 
> > > > > > With a global ID space this was trivial; you could just print the request
> > > > > > ID and that was all that was ever needed. (I'm not proposing to consider
> > > > > > that though.)
> > > > > >     
> > > > > 
> > > > > IMO, a global ID number would work better than get_task_comm().
> > > > > 
> > > > > Just add a static int monotonic counter and use it for the debug purposes,
> > > > > e. g.:
> > > > > 
> > > > > {
> > > > > 	static unsigned int req_count = 0;
> > > > > 
> > > > > 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
> > > > > 		req_count++, fd);    
> > > > > 
> > > > > Ok, eventually, it will overflow, but, it will be unique within
> > > > > a reasonable timeframe to be good enough for debugging purposes.    
> > > > 
> > > > Yes, but you can't figure out which process allocated it anymore, making
> > > > associating kernel debug logs with user space process logs harder.
> > > > 
> > > > How about process id + file handle? That still doesn't tell which process
> > > > operated on the request though, but I'm not sure whether that's really a
> > > > crucial piece of information.  
> > > 
> > > You don't need that. With dev_dbg() - and other *_dbg() macros - you can
> > > enable process ID for all debug messages.  
> > 
> > With this, the problem again is that it does not uniquely identify the
> > request: the request is the same request independently of which process
> > would operate on it. Or whether it is being processed in an interrupt
> > context.
> > 
> > AFAICT, the allocator PID (or process name) + file handle are both required
> > to match a request between user and kernel space logs.
> 
> Sorry, I was unable to understand what you're saying.
> 
> If you set the debug string with:
> 
> 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d", req_count++, fd);  
> 
> With the remaining stuff at patch 04/29, e. g. those two printks:
> 
> 	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
> 	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> 
> And use "+pt" to enable those debug messages, for the request #1 with fd #45, 
> created by PID 16613 you would have a log like:
> 
> 	[  269.021116] [16613] request: allocated 1:45
> 	[  269.024118] [16613] request: release 1:45
> 
> (assuming that the same PID would create and release)
> 
> The "1:45" is an unique global ID that would allow tracking it, even
> if Q/DQ_BUF is done by some other PID.
> 
> E. g. if a PID#16618 were responsible for Q/DQ_BUF, you would have
> something like:
> 
> 	[  269.021116] [16613] request: allocated 1:45
> 	[  269.021117] [16618] request: Q_BUF 1:45
> 	[  269.021118] [16618] request: DQ_BUF 1:45
> 	[  269.024118] [16613] request: release 1:45
> 
> (assuming that you would have a Q_BUF/DQ_BUF similar dev_dbg())
> 
> That seems good enough to track it.
> 
> Yet, in order to make easier to track, I would actually change the
> dev_dbg() parameter order everywhere to something like:
> 
> 	dev_dbg(mdev->dev, "request#%s: allocated\n", req->debug_str)
> 	dev_dbg(mdev->dev, "request#%s: release\n", req->debug_str);
> 
> In order to print something like:
> 
> 	[  269.021116] [16613] request#1:45: allocated 
> 	[  269.021117] [16618] request#1:45: Q_BUF
> 	[  269.021118] [16618] request#1:45: DQ_BUF
> 	[  269.024118] [16613] request#1:45: release
> 
> Then, getting everything related to the first request would be as simple as:
> 
> 	$ dmesg|grep request#1:
> 
> That will provide the PID for both processes: the one that
> created/released and the one that queued/dequeued.

Ah, right; yes, then you can. It's still a bit more complicated as you have
one more piece of information to follow (the ID) vs. just PID and FD. For
instance, you can't grep for requests created by a given process. Note that
you can still print the PID of the process that operates on the request
through dyndbg.

I'd like to hear what Hans thinks.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Sakari Ailus]

Hi Hans,

On Mon, Apr 09, 2018 at 04:20:02PM +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
> ioctls.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 80 +++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 78 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index dffc290e4ada..27739ff7cb09 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file *filp,
>  	return 0;
>  }
>  
> +static long media_request_ioctl_queue(struct media_request *req)
> +{
> +	struct media_device *mdev = req->mdev;
> +	unsigned long flags;
> +	int ret = 0;
> +
> +	dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
> +		dev_dbg(mdev->dev,
> +			"request: unable to queue %s, request in state %s\n",
> +			req->debug_str, media_request_state_str(req->state));
> +		spin_unlock_irqrestore(&req->lock, flags);
> +		return -EINVAL;
> +	}
> +	req->state = MEDIA_REQUEST_STATE_QUEUEING;
> +
> +	spin_unlock_irqrestore(&req->lock, flags);
> +
> +	/*
> +	 * Ensure the request that is validated will be the one that gets queued
> +	 * next by serialising the queueing process.
> +	 */
> +	mutex_lock(&mdev->req_queue_mutex);
> +
> +	ret = mdev->ops->req_queue(req);
> +	spin_lock_irqsave(&req->lock, flags);
> +	req->state = ret ? MEDIA_REQUEST_STATE_IDLE : MEDIA_REQUEST_STATE_QUEUED;
> +	spin_unlock_irqrestore(&req->lock, flags);
> +	mutex_unlock(&mdev->req_queue_mutex);
> +
> +	if (ret) {
> +		dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
> +			req->debug_str, ret);
> +	} else {
> +		media_request_get(req);
> +	}
> +
> +	return ret;
> +}
> +
> +static long media_request_ioctl_reinit(struct media_request *req)
> +{
> +	struct media_device *mdev = req->mdev;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	if (req->state != MEDIA_REQUEST_STATE_IDLE &&
> +	    req->state != MEDIA_REQUEST_STATE_COMPLETE) {
> +		dev_dbg(mdev->dev,
> +			"request: %s not in idle or complete state, cannot reinit\n",
> +			req->debug_str);
> +		spin_unlock_irqrestore(&req->lock, flags);
> +		return -EINVAL;
> +	}
> +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> +	spin_unlock_irqrestore(&req->lock, flags);
> +
> +	media_request_clean(req);
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	req->state = MEDIA_REQUEST_STATE_IDLE;
> +	spin_unlock_irqrestore(&req->lock, flags);

Newline here.

> +	return 0;
> +}
> +
>  static long media_request_ioctl(struct file *filp, unsigned int cmd,
> -				unsigned long __arg)
> +				unsigned long arg)

This change should be in the patch that added the function.

>  {
> -	return -ENOIOCTLCMD;
> +	struct media_request *req = filp->private_data;
> +
> +	switch (cmd) {
> +	case MEDIA_REQUEST_IOC_QUEUE:
> +		return media_request_ioctl_queue(req);
> +	case MEDIA_REQUEST_IOC_REINIT:
> +		return media_request_ioctl_reinit(req);
> +	default:
> +		return -ENOIOCTLCMD;
> +	}
>  }
>  
>  static const struct file_operations request_fops = {

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Sakari Ailus]

Hi Tomasz,

On Tue, Apr 10, 2018 at 08:59:23AM +0000, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
> > From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> > Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
> > ioctls.
> 
> > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >   drivers/media/media-request.c | 80
> +++++++++++++++++++++++++++++++++++++++++--
> >   1 file changed, 78 insertions(+), 2 deletions(-)
> 
> > diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> > index dffc290e4ada..27739ff7cb09 100644
> > --- a/drivers/media/media-request.c
> > +++ b/drivers/media/media-request.c
> > @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file
> *filp,
> >          return 0;
> >   }
> 
> > +static long media_request_ioctl_queue(struct media_request *req)
> > +{
> > +       struct media_device *mdev = req->mdev;
> > +       unsigned long flags;
> > +       int ret = 0;
> > +
> > +       dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
> > +
> > +       spin_lock_irqsave(&req->lock, flags);
> > +       if (req->state != MEDIA_REQUEST_STATE_IDLE) {
> > +               dev_dbg(mdev->dev,
> > +                       "request: unable to queue %s, request in state
> %s\n",
> > +                       req->debug_str,
> media_request_state_str(req->state));
> > +               spin_unlock_irqrestore(&req->lock, flags);
> > +               return -EINVAL;
> 
> nit: Perhaps -EBUSY? (vb2 returns -EINVAL, though, but IMHO it doesn't
> really represent the real error too closely.)
> 
> > +       }
> > +       req->state = MEDIA_REQUEST_STATE_QUEUEING;
> > +
> > +       spin_unlock_irqrestore(&req->lock, flags);
> > +
> > +       /*
> > +        * Ensure the request that is validated will be the one that gets
> queued
> > +        * next by serialising the queueing process.
> > +        */
> > +       mutex_lock(&mdev->req_queue_mutex);
> > +
> > +       ret = mdev->ops->req_queue(req);
> > +       spin_lock_irqsave(&req->lock, flags);
> > +       req->state = ret ? MEDIA_REQUEST_STATE_IDLE :
> MEDIA_REQUEST_STATE_QUEUED;
> > +       spin_unlock_irqrestore(&req->lock, flags);
> > +       mutex_unlock(&mdev->req_queue_mutex);
> > +
> > +       if (ret) {
> > +               dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
> > +                       req->debug_str, ret);
> > +       } else {
> > +               media_request_get(req);
> 
> I'm not convinced that this is the right place to take a reference. IMHO
> whoever saves a pointer to the request in its own internal data (the
> ->req_queue() callback?), should also grab a reference before doing so. Not
> a strong objection, though, if we clearly document this, so that whoever
> implements ->req_queue() callback can do the right thing.

The framework will also release that reference once the request is
complete. In my opinion it's perfectly reasonable to do this in the
framework; moving things to frameworks that do not need to be done in
drivers generally reduces bugs and makes drivers more maintainable. Albeit
this is a minor matter in this respect.

Hans: the reference must be taken before the request is queued: otherwise
it is possible that the driver completes the request before the reference
is taken here. That would mean the request might have been already released
by the time of getting a refenrece to it.

> 
> > +       }
> > +
> > +       return ret;
> > +}
> > +
> > +static long media_request_ioctl_reinit(struct media_request *req)
> > +{
> > +       struct media_device *mdev = req->mdev;
> > +       unsigned long flags;
> > +
> > +       spin_lock_irqsave(&req->lock, flags);
> > +       if (req->state != MEDIA_REQUEST_STATE_IDLE &&
> > +           req->state != MEDIA_REQUEST_STATE_COMPLETE) {
> > +               dev_dbg(mdev->dev,
> > +                       "request: %s not in idle or complete state,
> cannot reinit\n",
> > +                       req->debug_str);
> > +               spin_unlock_irqrestore(&req->lock, flags);
> > +               return -EINVAL;
> 
> nit: Perhaps -EBUSY? (Again vb2 would return -EINVAL...)

I agree on both return values.

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 19/29] videobuf2-core: integrate with media requests

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Buffers can now be prepared or queued for a request.

> A buffer is unbound from the request at vb2_buffer_done time or
> when the queue is cancelled.

Please see my comments inline.

[snip]
> -int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void
*pb)
> +static int vb2_req_prepare(struct media_request_object *obj)
>   {
> -       struct vb2_buffer *vb;
> +       struct vb2_buffer *vb = container_of(obj, struct vb2_buffer,
req_obj);
>          int ret;

> +       if (WARN_ON(vb->state != VB2_BUF_STATE_IN_REQUEST))
> +               return -EINVAL;
> +
> +       ret = __buf_prepare(vb, NULL);
> +       if (ret)
> +               vb->state = VB2_BUF_STATE_IN_REQUEST;

Hmm, I suppose this is here because __buf_prepare() changes the state to
VB2_BUF_STATE_DEQUEUED on error (other than q->error)? I guess it's
harmless, but perhaps we could have a comment explaining this?

Best regards,
Tomasz

Re: [RFCv11 PATCH 22/29] videobuf2-v4l2: add vb2_request_queue helper

[Tomasz Figa]

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Generic helper function that checks if there are buffers in
> the request and if so, prepares and queues all objects in the
> request.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/common/videobuf2/videobuf2-v4l2.c | 39
+++++++++++++++++++++++++
>   include/media/videobuf2-v4l2.h                  |  3 ++
>   2 files changed, 42 insertions(+)

> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index 73c1fd4da58a..3d0c74bb4220 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -1061,6 +1061,45 @@ void vb2_ops_wait_finish(struct vb2_queue *vq)
>   }
>   EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);

> +int vb2_request_queue(struct media_request *req)
> +{
> +       struct media_request_object *obj;
> +       struct media_request_object *failed_obj = NULL;
> +       int ret = 0;
> +
> +       if (!vb2_core_request_has_buffers(req))
> +               return -ENOENT;
> +
> +       list_for_each_entry(obj, &req->objects, list) {
> +               if (!obj->ops->prepare)
> +                       continue;
> +
> +               ret = obj->ops->prepare(obj);
> +
> +               if (ret) {
> +                       failed_obj = obj;
> +                       break;
> +               }
> +       }
> +
> +       if (ret) {
> +               list_for_each_entry(obj, &req->objects, list) {
> +                       if (obj == failed_obj)
> +                               break;

nit: If we use list_for_each_entry_continue_reverse() here, we wouldn't
need failed_obj.

Best regards,
Tomasz

Re: [RFCv11 PATCH 24/29] Documentation: v4l: document request API

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
[snip]
> @@ -514,6 +517,11 @@ Buffer Flags
>          streaming may continue as normal and the buffer may be reused
>          normally. Drivers set this flag when the ``VIDIOC_DQBUF`` ioctl is
>          called.
> +    * .. _`V4L2-BUF-FLAG-IN-REQUEST`:
> +
> +      - ``V4L2_BUF_FLAG_IN_REQUEST``
> +      - 0x00000080
> +      - This buffer is part of a request the hasn't been queued yet.

typo: s/the hasn't/that hasn't/

[snip]
> +Usage
> +=====
> +
> +The Request API is used on top of standard media controller and V4L2
calls,
> +which are augmented with an extra ``request_fd`` parameter. Request
themselves
> +are allocated from either a supporting V4L2 device node, or a supporting
media
> +controller node. The origin of requests determine their scope: requests
> +allocated from a V4L2 device node can only act on that device, whereas
requests
> +allocated from a media controller node can control the whole pipeline of
the
> +controller.

Current patch set doesn't support allocating requests from V4L2 device
nodes.

> +
> +Request Allocation
> +------------------
> +
> +User-space allocates requests using the ``VIDIOC_NEW_REQUEST`` (for V4L2
device

VIDIOC_NEW_REQUEST doesn't exist in current patch set. (We should add this
bit to documentation, when we add respective functionality.)

> +requests) or ``MEDIA_IOC_NEW_REQUEST`` (for media controller requests)
on an

s/MEDIA_IOC_NEW_REQUEST/MEDIA_IOC_REQUEST_ALLOC/

> +opened device or media node. This returns a file descriptor representing
the
> +request. Typically, several such requests will be allocated.
> +
> +Request Preparation
> +-------------------
> +
> +Standard V4L2 ioctls can then receive a request file descriptor to
express the
> +fact that the ioctl is part of said request, and is not to be applied
> +immediately. V4L2 ioctls supporting this are
:c:func:`VIDIOC_S_EXT_CTRLS` and
> +:c:func:`VIDIOC_QBUF`. Controls set with a request parameter are stored
instead
> +of being immediately applied, and queued buffers not enter the regular
buffer
> +queue until the request is submitted. Only one buffer can be queued to a
given
> +queue for a given request.
> +
> +Request Submission
> +------------------
> +
> +Once the parameters and buffers of the request are specified, it can be
> +submitted by calling the ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl on the
request FD.

s/MEDIA_REQUEST_IOC_SUBMIT/MEDIA_REQUEST_IOC_QUEUE/

> +This will make the buffers associated to the request available to their
driver,
> +which can then apply the saved controls as buffers are processed. A
submitted
> +request cannot be modified anymore.
> +
> +If several devices are part of the request, individual drivers may
synchronize
> +so the requested pipeline's topology is applied before the buffers are
> +processed. This is at the discretion of media controller drivers and is
not a
> +requirement.
> +
> +Buffers queued without an associated request after a request-bound
buffer will
> +be processed using the state of the hardware at the time of the request
> +completion. All the same, controls set without a request are applied
> +immediately, regardless of whether a request is in use or not.
> +
> +User-space can ``poll()`` a request FD in order to wait until the request
> +completes. A request is considered complete once all its associated
buffers are
> +available for dequeing. Note that user-space does not need to wait for
the
> +request to complete to dequeue its buffers: buffers that are available
halfway
> +through a request can be dequeued independently of the request's state.
> +
> +A completed request includes the state of all devices that had queued
buffers

Why storing state is gated on queued buffers? Perhaps more like "all
devices, which had their state affected by the request"?

> +associated with it at the time of the request completion. User-space can
query
> +that state by calling :c:func:`VIDIOC_G_EXT_CTRLS` with the request FD.
> +
> +Recycling and Destruction
> +-------------------------
> +
> +Finally, completed request can either be discarded or be reused. Calling
> +``close()`` on a request FD will make that FD unusable, freeing the
request if
> +it is not referenced elsewhere. The ``MEDIA_REQUEST_IOC_SUBMIT`` ioctl
will
> +clear a request's state and make it available again. No state is
retained by
> +this operation: the request is as if it had just been allocated.
> +
> +Example for a M2M Device
> +------------------------
> +
> +M2M devices are single-node V4L2 devices providing one OUTPUT queue (for
> +user-space
> +to provide input buffers) and one CAPTURE queue (to retrieve processed
data).
> +They are perfectly symetric, i.e. one buffer of input will produce one
buffer of
> +output. These devices are commonly used for frame processors or stateless
> +codecs.
> +
> +In this use-case, the request API can be used to associate specific
controls to
> +be applied by the driver before processing an OUTPUT buffer, allowing
user-space
> +to queue many such buffers in advance. It can also take advantage of
requests'
> +ability to capture the state of controls when the request completes to
read back
> +information that may be subject to change.
> +
> +Put into code, after obtaining a request, user-space can assign controls
and one
> +OUTPUT buffer to it:
> +
> +       struct v4l2_buf buf;

struct v4l2_buffer

> +       struct v4l2_ext_controls ctrls;
> +       struct media_request_new new = { 0 };

struct media_request_alloc

> +       int req_fd;
> +       ...
> +       ioctl(media_fd, VIDIOC_NEW_REQUEST, &new);

s/VIDIOC_NEW_REQUEST/MEDIA_IOC_REQUEST_ALLOC/

> +       req_fd = new.fd;
> +       ...
> +       ctrls.request_fd = req_fd;

ctrls.which = V4L2_CTRL_WHICH_REQUEST;

> +       ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls);
> +       ...
> +       buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;

buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;

> +       buf.request_fd = req_fd;
> +       ioctl(codec_fd, VIDIOC_QBUF, &buf);
> +
> +Note that request_fd does not need to be specified for CAPTURE buffers:
since
> +there is symetry between the OUTPUT and CAPTURE queues, and requests are
> +processed in order of submission, we can know which CAPTURE buffer
corresponds
> +to which request.
> +
> +Once the request is fully prepared, it can be submitted to the driver:
> +
> +       ioctl(request_fd, MEDIA_REQUEST_IOC_SUBMIT, NULL);

s/MEDIA_REQUEST_IOC_SUBMIT/MEDIA_REQUEST_IOC_QUEUE/

> +
> +User-space can then either wait for the request to complete by calling
poll() on
> +its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it
will
> +want to get CAPTURE buffers as soon as possible and this can be done
using a
> +regular DQBUF:
> +
> +       struct v4l2_buf buf;

struct v4l2_buffer

> +
> +       memset(&buf, 0, sizeof(buf));
> +       buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +       ioctl(codec_fd, VIDIOC_DQBUF, &buf);
> +
> +We can then, after ensuring that the request is completed via polling the
> +request FD, query control values at the time of its completion via an
> +annotated call to G_EXT_CTRLS. This is particularly useful for volatile
controls
> +for which we want to query values as soon as the capture buffer is
produced.
> +
> +       struct pollfd pfd = { .events = POLLIN, .fd = request_fd };
> +       poll(&pfd, 1, -1);
> +       ...
> +       ctrls.request_fd = req_fd;

ctrls.which = V4L2_CTRL_WHICH_REQUEST;

> +       ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls);
> +
> +Once we don't need the request anymore, we can either recycle it for
reuse with
> +MEDIA_REQUEST_IOC_REINIT...
> +
> +       ioctl(request, MEDIA_REQUEST_IOC_REINIT, NULL);
> +
> +... or close its file descriptor to completely dispose of it.
> +
> +       close(request_fd);
> +
> +Example for a Simple Capture Device
> +-----------------------------------
> +
> +With a simple capture device, requests can be used to specify controls
to apply
> +to a given CAPTURE buffer. The driver will apply these controls before
producing
> +the marked CAPTURE buffer.
> +
> +       struct v4l2_buf buf;

struct v4l2_buffer

> +       struct v4l2_ext_controls ctrls;
> +       struct media_request_new new = { 0 };

struct media_request_alloc

> +       int req_fd;
> +       ...
> +       ioctl(camera_fd, VIDIOC_NEW_REQUEST, &new);
> +       req_fd = new.fd;
> +       ...
> +       ctrls.request_fd = req_fd;
> +       ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls);
> +       ...
> +       buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +       buf.request_fd = req_fd;

buf.flags |= V4L2_BUF_FLAG_REQUEST_FD;

> +       ioctl(camera_fd, VIDIOC_QBUF, &buf);
> +
> +Once the request is fully prepared, it can be submitted to the driver:
> +
> +       ioctl(req_fd, MEDIA_REQUEST_IOC_SUBMIT, &cmd);

s/MEDIA_REQUEST_IOC_SUBMIT/MEDIA_REQUEST_IOC_QUEUE/

> +
> +User-space can then dequeue buffers, wait for the request completion,
query
> +controls and recycle the request as in the M2M example above.
> diff --git a/Documentation/media/uapi/v4l/user-func.rst
b/Documentation/media/uapi/v4l/user-func.rst
> index 3e0413b83a33..2c8238a2b188 100644
> --- a/Documentation/media/uapi/v4l/user-func.rst
> +++ b/Documentation/media/uapi/v4l/user-func.rst
> @@ -53,6 +53,7 @@ Function Reference
>       vidioc-g-std
>       vidioc-g-tuner
>       vidioc-log-status
> +    vidioc-new-request
>       vidioc-overlay
>       vidioc-prepare-buf
>       vidioc-qbuf
> diff --git a/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
b/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
> index 2011c2b2ee67..d31ef86c7caa 100644
> --- a/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
> +++ b/Documentation/media/uapi/v4l/vidioc-g-ext-ctrls.rst
> @@ -95,6 +95,17 @@ appropriate. In the first case the new value is set in
struct
>   is inappropriate (e.g. the given menu index is not supported by the menu
>   control), then this will also result in an ``EINVAL`` error code error.

> +If ``request_fd`` is set to a not-submitted request file descriptor,
then the

...and ``which`` is set to V4L2_CTRL_WHICH_REQUEST

> +controls are not applied immediately when calling
> +:ref:`VIDIOC_S_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`, but instead are applied
right
> +before the driver starts processing a buffer associated to the same
request.
> +
> +If ``request_fd`` is specified during a call to
> +:ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`, then the returned values
will

...and ``which`` is set to V4L2_CTRL_WHICH_REQUEST

> +be the values currently set for the request (or the hardware value if
none is
> +set) if the request has not yet completed, or the values of the controls
at the
> +time of request completion if it has already completed.
> +
>   The driver will only set/get these controls if all control values are
>   correct. This prevents the situation where only some of the controls
>   were set/get. Only low-level errors (e. g. a failed i2c command) can
> @@ -209,8 +220,10 @@ still cause this situation.
>         - ``which``
>         - Which value of the control to get/set/try.
>          ``V4L2_CTRL_WHICH_CUR_VAL`` will return the current value of the
> -       control and ``V4L2_CTRL_WHICH_DEF_VAL`` will return the default
> -       value of the control.
> +       control, ``V4L2_CTRL_WHICH_DEF_VAL`` will return the default
> +       value of the control and ``V4L2_CTRL_WHICH_REQUEST`` indicates
that
> +       these controls have to be retrieved from a request or tried/set
for
> +       a request.

>          .. note::

> @@ -272,8 +285,11 @@ still cause this situation.
>          then you can call :ref:`VIDIOC_TRY_EXT_CTRLS
<VIDIOC_G_EXT_CTRLS>` to try to discover the
>          actual control that failed the validation step. Unfortunately,
>          there is no ``TRY`` equivalent for :ref:`VIDIOC_G_EXT_CTRLS
<VIDIOC_G_EXT_CTRLS>`.
> +    * - __s32
> +      - ``request_fd``
> +       File descriptor of the request to be used by this operation (0 if
none).

-1 if none? Or maybe just say, valid only if ``which`` is
V4L2_CTRL_WHICH_REQUEST?

>       * - __u32
> -      - ``reserved``\ [2]
> +      - ``reserved``\ [1]
>         - Reserved for future extensions.

>          Drivers and applications must set the array to zero.
> diff --git a/Documentation/media/uapi/v4l/vidioc-new-request.rst
b/Documentation/media/uapi/v4l/vidioc-new-request.rst
> new file mode 100644
> index 000000000000..0038287f7d16
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/vidioc-new-request.rst

This whole file added here should probably be moved out to a separate patch
that adds documentation after we add the functionality.

Best regards,
Tomasz

Re: [RFCv11 PATCH 25/29] media: vim2m: add media device

[Tomasz Figa]

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Alexandre Courbot <acourbot@chromium.org>

> Request API requires a media node. Add one to the vim2m driver so we can
> use requests with it.

> This probably needs a bit more work to correctly represent m2m
> hardware in the media topology.
[snip]
> @@ -1013,6 +1012,22 @@ static int vim2m_probe(struct platform_device
*pdev)
>          vfd->lock = &dev->dev_mutex;
>          vfd->v4l2_dev = &dev->v4l2_dev;

> +#ifdef CONFIG_MEDIA_CONTROLLER
> +       dev->mdev.dev = &pdev->dev;
> +       strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
> +       media_device_init(&dev->mdev);
> +       dev->v4l2_dev.mdev = &dev->mdev;
> +       dev->pad[0].flags = MEDIA_PAD_FL_SINK;
> +       dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
> +       ret = media_entity_pads_init(&vfd->entity, 2, dev->pad);
> +       if (ret)
> +               return ret;

Hmm, what are these pads linked to?

[snip]
> @@ -1050,6 +1076,13 @@ static int vim2m_remove(struct platform_device
*pdev)
>          struct vim2m_dev *dev = platform_get_drvdata(pdev);

>          v4l2_info(&dev->v4l2_dev, "Removing " MEM2MEM_NAME);
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +       if (media_devnode_is_registered(dev->mdev.devnode))

Do we need to check this? Probe seems to fail if media device fails to
register.

Best regards,
Tomasz

Re: [RFCv11 PATCH 26/29] vim2m: use workqueue

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> v4l2_ctrl uses mutexes, so we can't setup a ctrl_handler in
> interrupt context. Switch to a workqueue instead.

Could it make more sense to just replace the old (non-hr) timer used in
this driver with delayed work?

Best regards,
Tomasz

Re: [RFCv11 PATCH 27/29] vim2m: support requests

[Tomasz Figa]

Hi Hans,

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Add support for requests to vim2m.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
>   1 file changed, 25 insertions(+)

> diff --git a/drivers/media/platform/vim2m.c
b/drivers/media/platform/vim2m.c
> index 9b18b32c255d..2dcf0ea85705 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -387,8 +387,26 @@ static void device_run(void *priv)
>          src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
>          dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);

> +       /* Apply request if needed */
> +       if (src_buf->vb2_buf.req_obj.req)
> +               v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
> +                                       &ctx->hdl);
> +       if (dst_buf->vb2_buf.req_obj.req &&
> +           dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
> +               v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
> +                                       &ctx->hdl);

I'm not sure I understand what's going on here. How is it possible that we
have 2 different requests?

Best regards,
Tomasz

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Sakari Ailus]

Hi Mauro,

On Tue, Apr 10, 2018 at 07:57:56AM -0300, Mauro Carvalho Chehab wrote:
> Em Mon,  9 Apr 2018 16:20:02 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> 
> > From: Hans Verkuil <hans.verkuil@cisco.com>
> > 
> > Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
> > ioctls.
> > 
> > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/media-request.c | 80 +++++++++++++++++++++++++++++++++++++++++--
> >  1 file changed, 78 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> > index dffc290e4ada..27739ff7cb09 100644
> > --- a/drivers/media/media-request.c
> > +++ b/drivers/media/media-request.c
> > @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file *filp,
> >  	return 0;
> >  }
> >  
> > +static long media_request_ioctl_queue(struct media_request *req)
> > +{
> > +	struct media_device *mdev = req->mdev;
> > +	unsigned long flags;
> > +	int ret = 0;
> > +
> > +	dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
> > +
> > +	spin_lock_irqsave(&req->lock, flags);
> > +	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
> > +		dev_dbg(mdev->dev,
> > +			"request: unable to queue %s, request in state %s\n",
> > +			req->debug_str, media_request_state_str(req->state));
> > +		spin_unlock_irqrestore(&req->lock, flags);
> > +		return -EINVAL;
> > +	}
> > +	req->state = MEDIA_REQUEST_STATE_QUEUEING;
> > +
> > +	spin_unlock_irqrestore(&req->lock, flags);
> > +
> > +	/*
> > +	 * Ensure the request that is validated will be the one that gets queued
> > +	 * next by serialising the queueing process.
> > +	 */
> > +	mutex_lock(&mdev->req_queue_mutex);
> 
> The locking here seems really weird. IMHO, it should lock before
> touching state, as otherwise race conditions may happen.
> 
> As I suggested before, I would use an atomic type for state, and get rid
> of the spin lock (as it seems that it is meant to be used just
> for state).

The new request state depends on the old state; I don't think you can
meaningfully do that using the atomic API.

> 
> > +
> > +	ret = mdev->ops->req_queue(req);
> > +	spin_lock_irqsave(&req->lock, flags);
> > +	req->state = ret ? MEDIA_REQUEST_STATE_IDLE : MEDIA_REQUEST_STATE_QUEUED;
> > +	spin_unlock_irqrestore(&req->lock, flags);
> > +	mutex_unlock(&mdev->req_queue_mutex);
> > +
> 
> Here, you have both mutex and spin locked. This is a strong indication
> that locks are not well designed, are you're using two different locks
> to protect the same data.

Not the same, it's different data. What is no longer visible in this
patchset is how request objects are referenced in a request. Effectively
that part is missing altogether. It will be needed when adding support for
objects that are not managed through the V4L2 framework such as pixel
formats or selection rectangles.

You could move the serialisation of queueing requests to drivers altogether
but I don't think that'd be a wise choice: the device state at request
queue head will need to be maintained so that queued requests can be
validated against it (right now validation is embedded in queueing).
Failing validation will result into restoring the previous state.

I had an implementation of that in my previous request set here:

<URL:https://www.spinics.net/lists/linux-media/msg130994.html>

We'll implement it but not yet: right now there's just a need for buffers
and controls. Still, knowing where we're going I'd keep the mutex where it
is.

> > +static long media_request_ioctl_reinit(struct media_request *req)
> > +{
> > +	struct media_device *mdev = req->mdev;
> > +	unsigned long flags;
> > +
> > +	spin_lock_irqsave(&req->lock, flags);
> > +	if (req->state != MEDIA_REQUEST_STATE_IDLE &&
> > +	    req->state != MEDIA_REQUEST_STATE_COMPLETE) {
> > +		dev_dbg(mdev->dev,
> > +			"request: %s not in idle or complete state, cannot reinit\n",
> > +			req->debug_str);
> > +		spin_unlock_irqrestore(&req->lock, flags);
> > +		return -EINVAL;
> > +	}
> > +	req->state = MEDIA_REQUEST_STATE_CLEANING;
> > +	spin_unlock_irqrestore(&req->lock, flags);
> > +
> > +	media_request_clean(req);
> > +
> > +	spin_lock_irqsave(&req->lock, flags);
> > +	req->state = MEDIA_REQUEST_STATE_IDLE;
> > +	spin_unlock_irqrestore(&req->lock, flags);
> 
> This code should be called with the mutex hold.

That's not necessary. A request which is being re-initialised was in an
idle or complete state; nothing of that request is bound to the current
device state in any way. Therefore the objects in that request can be
simply thrown out.

The state change to CLEANING is there to prevent the request being e.g.
queued during the cleaning.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 06/29] media-request: add media_request_find

[Sakari Ailus]

Hi Mauro,

On Tue, Apr 10, 2018 at 08:04:09AM -0300, Mauro Carvalho Chehab wrote:
> Em Mon,  9 Apr 2018 16:20:03 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> 
> > From: Hans Verkuil <hans.verkuil@cisco.com>
> > 
> > Add media_request_find() to find a request based on the file
> > descriptor.
> > 
> > The caller has to call media_request_put() for the returned
> > request since this function increments the refcount.
> > 
> > Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> > ---
> >  drivers/media/media-request.c | 47 +++++++++++++++++++++++++++++++++++++++++++
> >  include/media/media-request.h | 10 +++++++++
> >  2 files changed, 57 insertions(+)
> > 
> > diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> > index 27739ff7cb09..02b620c81de5 100644
> > --- a/drivers/media/media-request.c
> > +++ b/drivers/media/media-request.c
> > @@ -207,6 +207,53 @@ static const struct file_operations request_fops = {
> >  	.release = media_request_close,
> >  };
> >  
> > +/**
> > + * media_request_find - Find a request based on the file descriptor
> > + * @mdev: The media device
> > + * @request: The request file handle
> 
> request_id.

I believe you meant "request_fd".

> 
> > + *
> > + * Find and return the request associated with the given file descriptor, or
> > + * an error if no such request exists.
> > + *
> > + * When the function returns a request it increases its reference count. The
> > + * caller is responsible for releasing the reference by calling
> > + * media_request_put() on the request.
> 
> IMHO, this can only be called with mutex held. Please add such note
> here.

Why?

The kernel already handles this for file descriptors, so if we get a struct
file reference here, we do already have a reference to the media request
--- struct file always does, this reference is only released when the file
is released.

> 
> > + */
> > +struct media_request *
> > +media_request_find(struct media_device *mdev, int request_fd)
> > +{
> > +	struct file *filp;
> > +	struct media_request *req;
> > +
> > +	if (!mdev || !mdev->ops || !mdev->ops->req_queue)
> > +		return ERR_PTR(-EPERM);
> > +
> > +	filp = fget(request_fd);
> > +	if (!filp)
> > +		return ERR_PTR(-ENOENT);
> > +
> > +	if (filp->f_op != &request_fops)
> > +		goto err_fput;
> > +	req = filp->private_data;
> > +	media_request_get(req);
> > +
> > +	if (req->mdev != mdev)
> > +		goto err_kref_put;
> > +
> > +	fput(filp);
> > +
> > +	return req;
> > +
> > +err_kref_put:
> > +	media_request_put(req);
> > +
> > +err_fput:
> > +	fput(filp);
> > +
> > +	return ERR_PTR(-ENOENT);
> > +}
> > +EXPORT_SYMBOL_GPL(media_request_find);
> > +
> >  int media_request_alloc(struct media_device *mdev,
> >  			struct media_request_alloc *alloc)
> >  {
> > diff --git a/include/media/media-request.h b/include/media/media-request.h
> > index 082c3cae04ac..033697d493cd 100644
> > --- a/include/media/media-request.h
> > +++ b/include/media/media-request.h
> > @@ -59,6 +59,9 @@ static inline void media_request_get(struct media_request *req)
> >  void media_request_put(struct media_request *req);
> >  void media_request_cancel(struct media_request *req);
> >  
> > +struct media_request *
> > +media_request_find(struct media_device *mdev, int request_fd);
> > +
> >  int media_request_alloc(struct media_device *mdev,
> >  			struct media_request_alloc *alloc);
> >  #else
> > @@ -74,6 +77,12 @@ static inline void media_request_cancel(struct media_request *req)
> >  {
> >  }
> >  
> > +static inline struct media_request *
> > +media_request_find(struct media_device *mdev, int request_fd)
> > +{
> > +	return ERR_PTR(-ENOENT);
> > +}
> > +
> >  #endif
> >  
> >  struct media_request_object_ops {
> > @@ -173,6 +182,7 @@ static inline void media_request_object_unbind(struct media_request_object *obj)
> >  static inline void media_request_object_complete(struct media_request_object *obj)
> >  {
> >  }
> > +
> >  #endif
> >  
> >  #endif
> 
> 
> 
> Thanks,
> Mauro

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 06/29] media-request: add media_request_find

[Sakari Ailus]

Hi Hans,

On Mon, Apr 09, 2018 at 04:20:03PM +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add media_request_find() to find a request based on the file
> descriptor.
> 
> The caller has to call media_request_put() for the returned
> request since this function increments the refcount.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 47 +++++++++++++++++++++++++++++++++++++++++++
>  include/media/media-request.h | 10 +++++++++
>  2 files changed, 57 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 27739ff7cb09..02b620c81de5 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -207,6 +207,53 @@ static const struct file_operations request_fops = {
>  	.release = media_request_close,
>  };
>  
> +/**
> + * media_request_find - Find a request based on the file descriptor
> + * @mdev: The media device
> + * @request: The request file handle
> + *
> + * Find and return the request associated with the given file descriptor, or
> + * an error if no such request exists.
> + *
> + * When the function returns a request it increases its reference count. The
> + * caller is responsible for releasing the reference by calling
> + * media_request_put() on the request.
> + */
> +struct media_request *
> +media_request_find(struct media_device *mdev, int request_fd)
> +{
> +	struct file *filp;
> +	struct media_request *req;
> +
> +	if (!mdev || !mdev->ops || !mdev->ops->req_queue)
> +		return ERR_PTR(-EPERM);
> +
> +	filp = fget(request_fd);
> +	if (!filp)
> +		return ERR_PTR(-ENOENT);
> +
> +	if (filp->f_op != &request_fops)
> +		goto err_fput;
> +	req = filp->private_data;
> +	media_request_get(req);

As you're holding a reference to filp, you also indirectly have reference
to ref --- you otherwise couldn't access file->private_data either.

Therefore you could move the check for the right mdev before getting a
reference to req, thus simplifying error handling as a result.

> +
> +	if (req->mdev != mdev)
> +		goto err_kref_put;
> +
> +	fput(filp);
> +
> +	return req;
> +
> +err_kref_put:
> +	media_request_put(req);
> +
> +err_fput:
> +	fput(filp);
> +
> +	return ERR_PTR(-ENOENT);
> +}
> +EXPORT_SYMBOL_GPL(media_request_find);
> +
>  int media_request_alloc(struct media_device *mdev,
>  			struct media_request_alloc *alloc)
>  {

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 07/29] media-request: add media_request_object_find

[Sakari Ailus]

Hi Hans,

On Mon, Apr 09, 2018 at 04:20:04PM +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add media_request_object_find to find a request object inside a
> request based on ops and/or priv values.
> 
> Objects of the same type (vb2 buffer, control handler) will have
> the same ops value. And objects that refer to the same 'parent'
> object (e.g. the v4l2_ctrl_handler that has the current driver
> state) will have the same priv value.
> 
> The caller has to call media_request_object_put() for the returned
> object since this function increments the refcount.
> 
> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/media-request.c | 26 ++++++++++++++++++++++++++
>  include/media/media-request.h | 25 +++++++++++++++++++++++++
>  2 files changed, 51 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 02b620c81de5..415f7e31019d 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -322,6 +322,32 @@ static void media_request_object_release(struct kref *kref)
>  	obj->ops->release(obj);
>  }
>  
> +struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +			  const struct media_request_object_ops *ops,
> +			  void *priv)
> +{
> +	struct media_request_object *obj;
> +	struct media_request_object *found = NULL;
> +	unsigned long flags;
> +
> +	if (!ops && !priv)
> +		return NULL;
> +
> +	spin_lock_irqsave(&req->lock, flags);
> +	list_for_each_entry(obj, &req->objects, list) {
> +		if ((!ops || obj->ops == ops) &&
> +		    (!priv || obj->priv == priv)) {

I think I may have given the similar comment on an earlier version but I
don't remember a reply to that at least --- is there a case where the
object wouldn't be uniquely identified by req together with priv?

It'd be still prudent to check that ops match, but failing that check, I'd
add WARN_ON() because it's a sign something is wrong.

> +			media_request_object_get(obj);
> +			found = obj;
> +			break;
> +		}
> +	}
> +	spin_unlock_irqrestore(&req->lock, flags);
> +	return found;
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_find);
> +
>  void media_request_object_put(struct media_request_object *obj)
>  {
>  	kref_put(&obj->kref, media_request_object_release);
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index 033697d493cd..ea990c8f76bc 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -130,6 +130,23 @@ static inline void media_request_object_get(struct media_request_object *obj)
>   */
>  void media_request_object_put(struct media_request_object *obj);
>  
> +/**
> + * media_request_object_find - Find an object in a request
> + *
> + * @ops: Find an object with this ops value, may be NULL.
> + * @priv: Find an object with this priv value, may be NULL.
> + *
> + * At least one of @ops and @priv must be non-NULL. If one of
> + * these is NULL, then skip checking for that field.
> + *
> + * Returns NULL if not found or the object (the refcount is increased
> + * in that case).
> + */
> +struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +			  const struct media_request_object_ops *ops,
> +			  void *priv);
> +
>  /**
>   * media_request_object_init - Initialise a media request object
>   *
> @@ -162,6 +179,14 @@ static inline void media_request_object_put(struct media_request_object *obj)
>  {
>  }
>  
> +static inline struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +			  const struct media_request_object_ops *ops,
> +			  void *priv)
> +{
> +	return NULL;
> +}
> +
>  static inline void media_request_object_init(struct media_request_object *obj)
>  {
>  	obj->ops = NULL;

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 08/29] videodev2.h: add request_fd field to v4l2_ext_controls

[Sakari Ailus]

On Mon, Apr 09, 2018 at 04:20:05PM +0200, Hans Verkuil wrote:
> From: Alexandre Courbot <acourbot@chromium.org>
> 
> If which is V4L2_CTRL_WHICH_REQUEST, then the request_fd field can be
> used to specify a request for the G/S/TRY_EXT_CTRLS ioctls.
> 
> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 5 ++++-
>  drivers/media/v4l2-core/v4l2-ioctl.c          | 6 +++---
>  include/uapi/linux/videodev2.h                | 4 +++-
>  3 files changed, 10 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> index 5198c9eeb348..0782b3666deb 100644
> --- a/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> +++ b/drivers/media/v4l2-core/v4l2-compat-ioctl32.c
> @@ -732,7 +732,8 @@ struct v4l2_ext_controls32 {
>  	__u32 which;
>  	__u32 count;
>  	__u32 error_idx;
> -	__u32 reserved[2];
> +	__s32 request_fd;
> +	__u32 reserved[1];

No need for an array.

>  	compat_caddr_t controls; /* actually struct v4l2_ext_control32 * */
>  };
>  
> @@ -807,6 +808,7 @@ static int get_v4l2_ext_controls32(struct file *file,
>  	    get_user(count, &up->count) ||
>  	    put_user(count, &kp->count) ||
>  	    assign_in_user(&kp->error_idx, &up->error_idx) ||
> +	    assign_in_user(&kp->request_fd, &up->request_fd) ||
>  	    copy_in_user(kp->reserved, up->reserved, sizeof(kp->reserved)))
>  		return -EFAULT;
>  
> @@ -865,6 +867,7 @@ static int put_v4l2_ext_controls32(struct file *file,
>  	    get_user(count, &kp->count) ||
>  	    put_user(count, &up->count) ||
>  	    assign_in_user(&up->error_idx, &kp->error_idx) ||
> +	    assign_in_user(&up->request_fd, &kp->request_fd) ||
>  	    copy_in_user(up->reserved, kp->reserved, sizeof(up->reserved)) ||
>  	    get_user(kcontrols, &kp->controls))
>  		return -EFAULT;
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index f48c505550e0..9ce23e23c5bf 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -553,8 +553,8 @@ static void v4l_print_ext_controls(const void *arg, bool write_only)
>  	const struct v4l2_ext_controls *p = arg;
>  	int i;
>  
> -	pr_cont("which=0x%x, count=%d, error_idx=%d",
> -			p->which, p->count, p->error_idx);
> +	pr_cont("which=0x%x, count=%d, error_idx=%d, request_fd=%d",
> +			p->which, p->count, p->error_idx, p->request_fd);
>  	for (i = 0; i < p->count; i++) {
>  		if (!p->controls[i].size)
>  			pr_cont(", id/val=0x%x/0x%x",
> @@ -870,7 +870,7 @@ static int check_ext_ctrls(struct v4l2_ext_controls *c, int allow_priv)
>  	__u32 i;
>  
>  	/* zero the reserved fields */
> -	c->reserved[0] = c->reserved[1] = 0;
> +	c->reserved[0] = 0;
>  	for (i = 0; i < c->count; i++)
>  		c->controls[i].reserved2[0] = 0;
>  
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 600877be5c22..6f41baa53787 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -1592,7 +1592,8 @@ struct v4l2_ext_controls {
>  	};
>  	__u32 count;
>  	__u32 error_idx;
> -	__u32 reserved[2];
> +	__s32 request_fd;
> +	__u32 reserved[1];
>  	struct v4l2_ext_control *controls;
>  };
>  
> @@ -1605,6 +1606,7 @@ struct v4l2_ext_controls {
>  #define V4L2_CTRL_MAX_DIMS	  (4)
>  #define V4L2_CTRL_WHICH_CUR_VAL   0
>  #define V4L2_CTRL_WHICH_DEF_VAL   0x0f000000
> +#define V4L2_CTRL_WHICH_REQUEST   0x0f010000
>  
>  enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_INTEGER	     = 1,
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 10/29] v4l2-ctrls: v4l2_ctrl_add_handler: add from_other_dev

[Sakari Ailus]

Hi Hans,

I don't really have an opinion on the patch yet; a few trivial comments
below.

On Mon, Apr 09, 2018 at 04:20:07PM +0200, Hans Verkuil wrote:
...
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index 5b445b5654f7..f8faa54b5e7e 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -247,6 +247,8 @@ struct v4l2_ctrl {
>   * @ctrl:	The actual control information.
>   * @helper:	Pointer to helper struct. Used internally in
>   *		``prepare_ext_ctrls`` function at ``v4l2-ctrl.c``.
> + * @from_other_dev: If true, then @ctrl was defined in another
> + *		device then the &struct v4l2_ctrl_handler.

s/then/than/

>   *
>   * Each control handler has a list of these refs. The list_head is used to
>   * keep a sorted-by-control-ID list of all controls, while the next pointer
> @@ -257,6 +259,7 @@ struct v4l2_ctrl_ref {
>  	struct v4l2_ctrl_ref *next;
>  	struct v4l2_ctrl *ctrl;
>  	struct v4l2_ctrl_helper *helper;
> +	bool from_other_dev;
>  };
>  
>  /**
> @@ -633,6 +636,8 @@ typedef bool (*v4l2_ctrl_filter)(const struct v4l2_ctrl *ctrl);
>   * @add:	The control handler whose controls you want to add to
>   *		the @hdl control handler.
>   * @filter:	This function will filter which controls should be added.
> + * @from_other_dev: If true, then the controls in @add were defined in another
> + *		device then @hdl.

s/then/than/

>   *
>   * Does nothing if either of the two handlers is a NULL pointer.
>   * If @filter is NULL, then all controls are added. Otherwise only those
> @@ -642,7 +647,8 @@ typedef bool (*v4l2_ctrl_filter)(const struct v4l2_ctrl *ctrl);
>   */
>  int v4l2_ctrl_add_handler(struct v4l2_ctrl_handler *hdl,
>  			  struct v4l2_ctrl_handler *add,
> -			  v4l2_ctrl_filter filter);
> +			  v4l2_ctrl_filter filter,
> +			  bool from_other_dev);
>  
>  /**
>   * v4l2_ctrl_radio_filter() - Standard filter for radio controls.
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Re: [RFCv11 PATCH 00/29] Request API

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Hi all,

> This is a cleaned up version of the v10 series (never posted to
> the list since it was messy).

Hi Hans,

It took me a while to test and review this, but finally have been able to
do it.

First the result of the test: I have tried porting my dummy vim2m test
program
(https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a42 for
reference),
and am getting a hang when trying to queue the second OUTPUT buffer (right
after
queuing the first request). If I move the calls the VIDIOC_STREAMON after
the
requests are queued, the hang seems to happen at that moment. Probably a
deadlock, haven't looked in detail yet.

I have a few other comments, will follow up per-patch.

Re: [RFCv11 PATCH 01/29] v4l2-device.h: always expose mdev

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> The mdev field is only present if CONFIG_MEDIA_CONTROLLER is set.
> But since we will need to pass the media_device to vb2 snd the

Typo: s/snd/and

> control framework it is very convenient to just make this field
> available all the time. If CONFIG_MEDIA_CONTROLLER is not set,
> then it will just be NULL.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   include/media/v4l2-device.h | 4 +---
>   1 file changed, 1 insertion(+), 3 deletions(-)

> diff --git a/include/media/v4l2-device.h b/include/media/v4l2-device.h
> index 0c9e4da55499..b330e4a08a6b 100644
> --- a/include/media/v4l2-device.h
> +++ b/include/media/v4l2-device.h
> @@ -33,7 +33,7 @@ struct v4l2_ctrl_handler;
>    * struct v4l2_device - main struct to for V4L2 device drivers
>    *
>    * @dev: pointer to struct device.
> - * @mdev: pointer to struct media_device
> + * @mdev: pointer to struct media_device, may be NULL.
>    * @subdevs: used to keep track of the registered subdevs
>    * @lock: lock this struct; can be used by the driver as well
>    *     if this struct is embedded into a larger struct.
> @@ -58,9 +58,7 @@ struct v4l2_ctrl_handler;
>    */
>   struct v4l2_device {
>          struct device *dev;
> -#if defined(CONFIG_MEDIA_CONTROLLER)
>          struct media_device *mdev;
> -#endif
>          struct list_head subdevs;
>          spinlock_t lock;
>          char name[V4L2_DEVICE_NAME_SIZE];
> --
> 2.16.3

Re: [RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Define the public request API.

> This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
> and two ioctls that operate on a request in order to queue the
> contents of the request to the driver and to re-initialize the
> request.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>

Like it that the API stays simple, with request ioctls tied to request FDs.

Acked-by: Alexandre Courbot <acourbot@chromium.org>

> ---
>   include/uapi/linux/media.h | 8 ++++++++
>   1 file changed, 8 insertions(+)

> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index c7e9a5cba24e..f8769e74f847 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -342,11 +342,19 @@ struct media_v2_topology {

>   /* ioctls */

> +struct __attribute__ ((packed)) media_request_alloc {
> +       __s32 fd;
> +};
> +
>   #define MEDIA_IOC_DEVICE_INFO  _IOWR('|', 0x00, struct media_device_info)
>   #define MEDIA_IOC_ENUM_ENTITIES        _IOWR('|', 0x01, struct
media_entity_desc)
>   #define MEDIA_IOC_ENUM_LINKS   _IOWR('|', 0x02, struct media_links_enum)
>   #define MEDIA_IOC_SETUP_LINK   _IOWR('|', 0x03, struct media_link_desc)
>   #define MEDIA_IOC_G_TOPOLOGY   _IOWR('|', 0x04, struct media_v2_topology)
> +#define MEDIA_IOC_REQUEST_ALLOC        _IOWR('|', 0x05, struct
media_request_alloc)
> +
> +#define MEDIA_REQUEST_IOC_QUEUE                _IO('|',  0x80)
> +#define MEDIA_REQUEST_IOC_REINIT       _IO('|',  0x81)

>   #if !defined(__KERNEL__) || defined(__NEED_MEDIA_LEGACY_API)

> --
> 2.16.3

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Add support for allocating a new request. This is only supported
> if mdev->ops->req_queue is set, i.e. the driver indicates that it
> supports queueing requests.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/Makefile        |  3 ++-
>   drivers/media/media-device.c  | 14 ++++++++++++++
>   drivers/media/media-request.c | 23 +++++++++++++++++++++++
>   include/media/media-device.h  | 13 +++++++++++++
>   include/media/media-request.h | 22 ++++++++++++++++++++++
>   5 files changed, 74 insertions(+), 1 deletion(-)
>   create mode 100644 drivers/media/media-request.c
>   create mode 100644 include/media/media-request.h

> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> index 594b462ddf0e..985d35ec6b29 100644
> --- a/drivers/media/Makefile
> +++ b/drivers/media/Makefile
> @@ -3,7 +3,8 @@
>   # Makefile for the kernel multimedia device drivers.
>   #

> -media-objs     := media-device.o media-devnode.o media-entity.o
> +media-objs     := media-device.o media-devnode.o media-entity.o \
> +                  media-request.o

>   #
>   # I2C drivers should come before other drivers, otherwise they'll fail
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index 35e81f7c0d2f..acb583c0eacd 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -32,6 +32,7 @@
>   #include <media/media-device.h>
>   #include <media/media-devnode.h>
>   #include <media/media-entity.h>
> +#include <media/media-request.h>

>   #ifdef CONFIG_MEDIA_CONTROLLER

> @@ -366,6 +367,15 @@ static long media_device_get_topology(struct
media_device *mdev,
>          return ret;
>   }

> +static long media_device_request_alloc(struct media_device *mdev,
> +                                      struct media_request_alloc *alloc)
> +{
> +       if (!mdev->ops || !mdev->ops->req_queue)
> +               return -ENOTTY;
> +
> +       return media_request_alloc(mdev, alloc);
> +}
> +
>   static long copy_arg_from_user(void *karg, void __user *uarg, unsigned
int cmd)
>   {
>          /* All media IOCTLs are _IOWR() */
> @@ -414,6 +424,7 @@ static const struct media_ioctl_info ioctl_info[] = {
>          MEDIA_IOC(ENUM_LINKS, media_device_enum_links,
MEDIA_IOC_FL_GRAPH_MUTEX),
>          MEDIA_IOC(SETUP_LINK, media_device_setup_link,
MEDIA_IOC_FL_GRAPH_MUTEX),
>          MEDIA_IOC(G_TOPOLOGY, media_device_get_topology,
MEDIA_IOC_FL_GRAPH_MUTEX),
> +       MEDIA_IOC(REQUEST_ALLOC, media_device_request_alloc, 0),
>   };

>   static long media_device_ioctl(struct file *filp, unsigned int cmd,
> @@ -686,6 +697,9 @@ void media_device_init(struct media_device *mdev)
>          INIT_LIST_HEAD(&mdev->pads);
>          INIT_LIST_HEAD(&mdev->links);
>          INIT_LIST_HEAD(&mdev->entity_notify);
> +
> +       spin_lock_init(&mdev->req_lock);
> +       mutex_init(&mdev->req_queue_mutex);
>          mutex_init(&mdev->graph_mutex);
>          ida_init(&mdev->entity_internal_idx);

> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> new file mode 100644
> index 000000000000..ead78613fdbe
> --- /dev/null
> +++ b/drivers/media/media-request.c
> @@ -0,0 +1,23 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Media device request objects
> + *
> + * Copyright (C) 2018 Intel Corporation
> + * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.

Seconding Tomasz' comment - Chromium authorship should be something like

Copyright (C) 2018 Google, Inc.

(see
https://www.chromium.org/chromium-os/how-tos-and-troubleshooting/kernel-faq#TOC-Which-copyright-header-should-I-use-
)

Sorry since I think I am the one who introduced this line in my early
patches.

> + *
> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> + */
> +
> +#include <linux/anon_inodes.h>
> +#include <linux/file.h>
> +#include <linux/mm.h>
> +#include <linux/string.h>
> +
> +#include <media/media-device.h>
> +#include <media/media-request.h>
> +
> +int media_request_alloc(struct media_device *mdev,
> +                       struct media_request_alloc *alloc)
> +{
> +       return -ENOMEM;

Probably not the best error code for now? And this function can probably be
declared later (i.e. along with its full definition) in the series anyway.

> +}
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index bcc6ec434f1f..07e323c57202 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -19,6 +19,7 @@
>   #ifndef _MEDIA_DEVICE_H
>   #define _MEDIA_DEVICE_H

> +#include <linux/anon_inodes.h>
>   #include <linux/list.h>
>   #include <linux/mutex.h>

> @@ -27,6 +28,7 @@

>   struct ida;
>   struct device;
> +struct media_device;

>   /**
>    * struct media_entity_notify - Media Entity Notify
> @@ -50,10 +52,16 @@ struct media_entity_notify {
>    * struct media_device_ops - Media device operations
>    * @link_notify: Link state change notification callback. This callback
is
>    *              called with the graph_mutex held.
> + * @req_alloc: Allocate a request
> + * @req_free: Free a request
> + * @req_queue: Queue a request
>    */
>   struct media_device_ops {
>          int (*link_notify)(struct media_link *link, u32 flags,
>                             unsigned int notification);
> +       struct media_request *(*req_alloc)(struct media_device *mdev);
> +       void (*req_free)(struct media_request *req);
> +       int (*req_queue)(struct media_request *req);

Same as above - these ops are introduced but not used at all. The glue code
could already be here, it would actually be helpful to understand the patch.

>   };

>   /**
> @@ -88,6 +96,8 @@ struct media_device_ops {
>    * @disable_source: Disable Source Handler function pointer
>    *
>    * @ops:       Operation handler callbacks
> + * @req_lock:  Serialise access to requests
> + * @req_queue_mutex: Serialise validating and queueing requests
>    *
>    * This structure represents an abstract high-level media device. It
allows easy
>    * access to entities and provides basic media device-level support. The
> @@ -158,6 +168,9 @@ struct media_device {
>          void (*disable_source)(struct media_entity *entity);

>          const struct media_device_ops *ops;
> +
> +       spinlock_t req_lock;
> +       struct mutex req_queue_mutex;
>   };

>   /* We don't need to include pci.h or usb.h here */
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> new file mode 100644
> index 000000000000..dae3eccd9aa7
> --- /dev/null
> +++ b/include/media/media-request.h
> @@ -0,0 +1,22 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Media device request objects
> + *
> + * Copyright (C) 2018 Intel Corporation
> + *
> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> + */
> +
> +#ifndef MEDIA_REQUEST_H
> +#define MEDIA_REQUEST_H
> +
> +#include <linux/list.h>
> +#include <linux/slab.h>
> +#include <linux/spinlock.h>
> +
> +#include <media/media-device.h>
> +
> +int media_request_alloc(struct media_device *mdev,
> +                       struct media_request_alloc *alloc);
> +
> +#endif
> --
> 2.16.3

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Implement the core of the media request processing.

> Drivers can bind request objects to a request. These objects
> can then be marked completed if the driver finished using them,
> or just be unbound if the results do not need to be kept (e.g.
> in the case of buffers).

> Once all objects that were added are either unbound or completed,
> the request is marked 'complete' and a POLLPRI signal is sent
> via poll.

> Both requests and request objects are refcounted.

> While a request is queued its refcount is incremented (since it
> is in use by a driver). Once it is completed the refcount is
> decremented. When the user closes the request file descriptor
> the refcount is also decremented. Once it reaches 0 all request
> objects in the request are unbound and put() and the request
> itself is freed.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/media-request.c | 284
+++++++++++++++++++++++++++++++++++++++++-
>   include/media/media-request.h | 156 +++++++++++++++++++++++
>   2 files changed, 439 insertions(+), 1 deletion(-)

> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index ead78613fdbe..dffc290e4ada 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -16,8 +16,290 @@
>   #include <media/media-device.h>
>   #include <media/media-request.h>

> +static const char * const request_state[] = {
> +       "idle",
> +       "queueing",
> +       "queued",
> +       "complete",
> +       "cleaning",
> +};
> +
> +static const char *
> +media_request_state_str(enum media_request_state state)
> +{
> +       if (WARN_ON(state >= ARRAY_SIZE(request_state)))
> +               return "unknown";
> +       return request_state[state];
> +}
> +
> +static void media_request_clean(struct media_request *req)
> +{
> +       struct media_request_object *obj, *obj_safe;
> +
> +       WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
> +
> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
> +               media_request_object_unbind(obj);
> +               media_request_object_put(obj);
> +       }
> +
> +       req->num_incomplete_objects = 0;
> +       wake_up_interruptible(&req->poll_wait);
> +}
> +
> +static void media_request_release(struct kref *kref)
> +{
> +       struct media_request *req =
> +               container_of(kref, struct media_request, kref);
> +       struct media_device *mdev = req->mdev;
> +       unsigned long flags;
> +
> +       dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       req->state = MEDIA_REQUEST_STATE_CLEANING;
> +       spin_unlock_irqrestore(&req->lock, flags);
> +
> +       media_request_clean(req);
> +
> +       if (mdev->ops->req_free)
> +               mdev->ops->req_free(req);
> +       else
> +               kfree(req);

Adding a third (different) opinion on this: if requests are to be embedded
into
other struct, then shouldn't we mandate an implementation for req_free
anyway?
Making it optional sounds error-prone to me.

> +}
> +
> +void media_request_put(struct media_request *req)
> +{
> +       kref_put(&req->kref, media_request_release);
> +}
> +EXPORT_SYMBOL_GPL(media_request_put);
> +
> +void media_request_cancel(struct media_request *req)
> +{
> +       struct media_request_object *obj, *obj_safe;
> +
> +       if (req->state != MEDIA_REQUEST_STATE_QUEUED)
> +               return;
> +
> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
> +               if (obj->ops->cancel)
> +                       obj->ops->cancel(obj);
> +}
> +EXPORT_SYMBOL_GPL(media_request_cancel);
> +
> +static int media_request_close(struct inode *inode, struct file *filp)
> +{
> +       struct media_request *req = filp->private_data;
> +
> +       media_request_put(req);
> +       return 0;
> +}
> +
> +static unsigned int media_request_poll(struct file *filp,
> +                                      struct poll_table_struct *wait)
> +{
> +       struct media_request *req = filp->private_data;
> +       unsigned long flags;
> +       enum media_request_state state;
> +
> +       if (!(poll_requested_events(wait) & POLLPRI))
> +               return 0;
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       state = req->state;
> +       spin_unlock_irqrestore(&req->lock, flags);
> +
> +       if (state == MEDIA_REQUEST_STATE_COMPLETE)
> +               return POLLPRI;
> +       if (state == MEDIA_REQUEST_STATE_IDLE)
> +               return POLLERR;
> +
> +       poll_wait(filp, &req->poll_wait, wait);
> +       return 0;
> +}
> +
> +static long media_request_ioctl(struct file *filp, unsigned int cmd,
> +                               unsigned long __arg)
> +{
> +       return -ENOIOCTLCMD;
> +}
> +
> +static const struct file_operations request_fops = {
> +       .owner = THIS_MODULE,
> +       .poll = media_request_poll,
> +       .unlocked_ioctl = media_request_ioctl,
> +       .release = media_request_close,
> +};
> +
>   int media_request_alloc(struct media_device *mdev,
>                          struct media_request_alloc *alloc)
>   {
> -       return -ENOMEM;
> +       struct media_request *req;
> +       struct file *filp;
> +       char comm[TASK_COMM_LEN];
> +       int fd;
> +       int ret;
> +
> +       fd = get_unused_fd_flags(O_CLOEXEC);
> +       if (fd < 0)
> +               return fd;
> +
> +       filp = anon_inode_getfile("request", &request_fops, NULL,
O_CLOEXEC);
> +       if (IS_ERR(filp)) {
> +               ret = PTR_ERR(filp);
> +               goto err_put_fd;
> +       }
> +
> +       if (mdev->ops->req_alloc)
> +               req = mdev->ops->req_alloc(mdev);
> +       else
> +               req = kzalloc(sizeof(*req), GFP_KERNEL);
> +       if (!req) {
> +               ret = -ENOMEM;
> +               goto err_fput;
> +       }
> +
> +       filp->private_data = req;
> +       req->mdev = mdev;
> +       req->state = MEDIA_REQUEST_STATE_IDLE;
> +       req->num_incomplete_objects = 0;
> +       kref_init(&req->kref);
> +       INIT_LIST_HEAD(&req->objects);
> +       spin_lock_init(&req->lock);
> +       init_waitqueue_head(&req->poll_wait);
> +
> +       alloc->fd = fd;
> +
> +       get_task_comm(comm, current);
> +       snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> +                comm, fd);
> +       dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
> +
> +       fd_install(fd, filp);
> +
> +       return 0;
> +
> +err_fput:
> +       fput(filp);
> +
> +err_put_fd:
> +       put_unused_fd(fd);
> +
> +       return ret;
> +}
> +
> +static void media_request_object_release(struct kref *kref)
> +{
> +       struct media_request_object *obj =
> +               container_of(kref, struct media_request_object, kref);
> +       struct media_request *req = obj->req;
> +
> +       if (req)
> +               media_request_object_unbind(obj);
> +       obj->ops->release(obj);
> +}
> +
> +void media_request_object_put(struct media_request_object *obj)
> +{
> +       kref_put(&obj->kref, media_request_object_release);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_put);
> +
> +void media_request_object_init(struct media_request_object *obj)
> +{
> +       obj->ops = NULL;
> +       obj->req = NULL;
> +       obj->priv = NULL;
> +       obj->completed = false;
> +       INIT_LIST_HEAD(&obj->list);
> +       kref_init(&obj->kref);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_init);
> +
> +void media_request_object_bind(struct media_request *req,
> +                              const struct media_request_object_ops *ops,

As suggested elsewhere, I think the ops would better be set at init() time.
We
probably don't want them to change during the object's lifetime. Probably
the
same for priv.

Actually, looking closer at how these functions are used, I can always see
media_request_object_init() being called in pair with
media_request_object_bind(). Same thing with media_request_object_unbind()
and
media_request_object_put(). It looks like media_request_object_init() and
media_request_object_bind() could be merged, and
media_request_object_unbind()
could include a call to media_request_object_put().

> +                              void *priv,
> +                              struct media_request_object *obj)
> +{
> +       unsigned long flags;
> +
> +       if (WARN_ON(!ops->release || !ops->cancel))
> +               return;
> +
> +       obj->req = req;
> +       obj->ops = ops;
> +       obj->priv = priv;
> +       spin_lock_irqsave(&req->lock, flags);
> +       if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
> +               goto unlock;
> +       list_add_tail(&obj->list, &req->objects);
> +       req->num_incomplete_objects++;
> +unlock:
> +       spin_unlock_irqrestore(&req->lock, flags);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_bind);
> +
> +void media_request_object_unbind(struct media_request_object *obj)
> +{
> +       struct media_request *req = obj->req;
> +       unsigned long flags;
> +       bool completed = false;
> +
> +       if (!req)
> +               return;
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       list_del(&obj->list);
> +       obj->req = NULL;
> +
> +       if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
> +           req->state == MEDIA_REQUEST_STATE_CLEANING)
> +               goto unlock;
> +
> +       if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
> +               goto unlock;
> +
> +       if (WARN_ON(!req->num_incomplete_objects))
> +               goto unlock;
> +
> +       req->num_incomplete_objects--;
> +       if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
> +           !req->num_incomplete_objects) {
> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +               completed = true;
> +               wake_up_interruptible(&req->poll_wait);
> +       }
> +unlock:
> +       spin_unlock_irqrestore(&req->lock, flags);
> +       if (obj->ops->unbind)
> +               obj->ops->unbind(obj);
> +       if (completed)
> +               media_request_put(req);
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_unbind);
> +
> +void media_request_object_complete(struct media_request_object *obj)
> +{
> +       struct media_request *req = obj->req;
> +       unsigned long flags;
> +       bool completed = false;
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       if (obj->completed)
> +               goto unlock;
> +       obj->completed = true;
> +       if (WARN_ON(!req->num_incomplete_objects) ||
> +           WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
> +               goto unlock;
> +
> +       if (!--req->num_incomplete_objects) {
> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
> +               wake_up_interruptible(&req->poll_wait);
> +               completed = true;
> +       }
> +unlock:
> +       spin_unlock_irqrestore(&req->lock, flags);
> +       if (completed)
> +               media_request_put(req);
>   }
> +EXPORT_SYMBOL_GPL(media_request_object_complete);
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index dae3eccd9aa7..082c3cae04ac 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -16,7 +16,163 @@

>   #include <media/media-device.h>

> +enum media_request_state {
> +       MEDIA_REQUEST_STATE_IDLE,
> +       MEDIA_REQUEST_STATE_QUEUEING,
> +       MEDIA_REQUEST_STATE_QUEUED,
> +       MEDIA_REQUEST_STATE_COMPLETE,
> +       MEDIA_REQUEST_STATE_CLEANING,
> +};
> +
> +struct media_request_object;
> +
> +/**
> + * struct media_request - Media device request
> + * @mdev: Media device this request belongs to
> + * @kref: Reference count
> + * @debug_prefix: Prefix for debug messages (process name:fd)
> + * @state: The state of the request
> + * @objects: List of @struct media_request_object request objects
> + * @num_objects: The number objects in the request
> + * @num_completed_objects: The number of completed objects in the request
> + * @poll_wait: Wait queue for poll
> + * @lock: Serializes access to this struct
> + */
> +struct media_request {
> +       struct media_device *mdev;
> +       struct kref kref;

I thought we wanted to use a struct file to manage the request reference
count?
That was even your idea IIRC.

> +       char debug_str[TASK_COMM_LEN + 11];
> +       enum media_request_state state;

If possible, using an atomic here would probably simplify locking
considerably.
Might require adding a few extra intermediate states though.

> +       struct list_head objects;
> +       unsigned int num_incomplete_objects;

Same here. I did that in one of my previous versions (albeit with a bit
mask)
and it did help simplify locking.

By doing both, I think the only operation that needs lock protection would
be
manipulation of the objects list (and maybe serialization of ops on
individual
objects, which could be done either by the same request lock or individual
objects locks).

> +       struct wait_queue_head poll_wait;
> +       spinlock_t lock;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +
> +static inline void media_request_get(struct media_request *req)
> +{
> +       kref_get(&req->kref);
> +}

I personally think it is a good idea to have this function - makes the
intent
clearer than a direct kref_get.

> +
> +void media_request_put(struct media_request *req);
> +void media_request_cancel(struct media_request *req);
> +
>   int media_request_alloc(struct media_device *mdev,
>                          struct media_request_alloc *alloc);
> +#else
> +static inline void media_request_get(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_put(struct media_request *req)
> +{
> +}
> +
> +static inline void media_request_cancel(struct media_request *req)
> +{
> +}
> +
> +#endif
> +
> +struct media_request_object_ops {
> +       int (*prepare)(struct media_request_object *object);
> +       void (*unprepare)(struct media_request_object *object);
> +       void (*queue)(struct media_request_object *object);
> +       void (*unbind)(struct media_request_object *object);
> +       void (*cancel)(struct media_request_object *object);
> +       void (*release)(struct media_request_object *object);
> +};
> +
> +/**
> + * struct media_request_object - An opaque object that belongs to a media
> + *                              request
> + *
> + * @priv: object's priv pointer
> + * @list: List entry of the object for @struct media_request
> + * @kref: Reference count of the object, acquire before releasing
req->lock
> + *
> + * An object related to the request. This struct is embedded in the
> + * larger object data.
> + */
> +struct media_request_object {
> +       const struct media_request_object_ops *ops;
> +       void *priv;

Is the priv member needed? Since it seems that we are embedding this struct
into other structs and setting priv to these objects address, so we could
just
use container_of() here. Although this may still be required for
media_request_object_find().

> +       struct media_request *req;
> +       struct list_head list;
> +       struct kref kref;
> +       bool completed;
> +};
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +static inline void media_request_object_get(struct media_request_object
*obj)
> +{
> +       kref_get(&obj->kref);
> +}
> +
> +/**
> + * media_request_object_put - Put a media request object
> + *
> + * @obj: The object
> + *
> + * Put a media request object. Once all references are gone, the
> + * object's memory is released.
> + */
> +void media_request_object_put(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_init - Initialise a media request object
> + *
> + * Initialise a media request object. The object will be released using
the
> + * release callback of the ops once it has no references (this function
> + * initialises references to one).
> + */
> +void media_request_object_init(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_bind - Bind a media request object to a request
> + */
> +void media_request_object_bind(struct media_request *req,
> +                              const struct media_request_object_ops *ops,
> +                              void *priv,
> +                              struct media_request_object *obj);
> +
> +void media_request_object_unbind(struct media_request_object *obj);
> +
> +/**
> + * media_request_object_complete - Mark the media request object as
complete
> + */
> +void media_request_object_complete(struct media_request_object *obj);
> +#else
> +static inline void media_request_object_get(struct media_request_object
*obj)
> +{
> +}
> +
> +static inline void media_request_object_put(struct media_request_object
*obj)
> +{
> +}
> +
> +static inline void media_request_object_init(struct media_request_object
*obj)
> +{
> +       obj->ops = NULL;
> +       obj->req = NULL;
> +}
> +
> +static inline void media_request_object_bind(struct media_request *req,
> +                              const struct media_request_object_ops *ops,
> +                              void *priv,
> +                              struct media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_unbind(struct
media_request_object *obj)
> +{
> +}
> +
> +static inline void media_request_object_complete(struct
media_request_object *obj)
> +{
> +}
> +#endif

>   #endif
> --
> 2.16.3

Re: [RFCv11 PATCH 07/29] media-request: add media_request_object_find

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Add media_request_object_find to find a request object inside a
> request based on ops and/or priv values.

> Objects of the same type (vb2 buffer, control handler) will have
> the same ops value. And objects that refer to the same 'parent'
> object (e.g. the v4l2_ctrl_handler that has the current driver
> state) will have the same priv value.

> The caller has to call media_request_object_put() for the returned
> object since this function increments the refcount.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/media-request.c | 26 ++++++++++++++++++++++++++
>   include/media/media-request.h | 25 +++++++++++++++++++++++++
>   2 files changed, 51 insertions(+)

> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 02b620c81de5..415f7e31019d 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -322,6 +322,32 @@ static void media_request_object_release(struct kref
*kref)
>          obj->ops->release(obj);
>   }

> +struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +                         const struct media_request_object_ops *ops,
> +                         void *priv)
> +{
> +       struct media_request_object *obj;
> +       struct media_request_object *found = NULL;
> +       unsigned long flags;
> +
> +       if (!ops && !priv)
> +               return NULL;
> +
> +       spin_lock_irqsave(&req->lock, flags);
> +       list_for_each_entry(obj, &req->objects, list) {
> +               if ((!ops || obj->ops == ops) &&
> +                   (!priv || obj->priv == priv)) {
> +                       media_request_object_get(obj);
> +                       found = obj;
> +                       break;
> +               }
> +       }
> +       spin_unlock_irqrestore(&req->lock, flags);
> +       return found;
> +}
> +EXPORT_SYMBOL_GPL(media_request_object_find);
> +
>   void media_request_object_put(struct media_request_object *obj)
>   {
>          kref_put(&obj->kref, media_request_object_release);
> diff --git a/include/media/media-request.h b/include/media/media-request.h
> index 033697d493cd..ea990c8f76bc 100644
> --- a/include/media/media-request.h
> +++ b/include/media/media-request.h
> @@ -130,6 +130,23 @@ static inline void media_request_object_get(struct
media_request_object *obj)
>    */
>   void media_request_object_put(struct media_request_object *obj);

> +/**
> + * media_request_object_find - Find an object in a request
> + *
> + * @ops: Find an object with this ops value, may be NULL.
> + * @priv: Find an object with this priv value, may be NULL.
> + *
> + * At least one of @ops and @priv must be non-NULL. If one of
> + * these is NULL, then skip checking for that field.
> + *
> + * Returns NULL if not found or the object (the refcount is increased
> + * in that case).
> + */
> +struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +                         const struct media_request_object_ops *ops,
> +                         void *priv);

Mm, that signature is weird. I don't yet know how this function is going to
be
called, but wouldn't priv be enough? If we look for ops, this means we are
looking for the first object of a given class (IIUC the class/objects
mechanism
here), and I cannot see where we would want to do that.

> +
>   /**
>    * media_request_object_init - Initialise a media request object
>    *
> @@ -162,6 +179,14 @@ static inline void media_request_object_put(struct
media_request_object *obj)
>   {
>   }

> +static inline struct media_request_object *
> +media_request_object_find(struct media_request *req,
> +                         const struct media_request_object_ops *ops,
> +                         void *priv)
> +{
> +       return NULL;
> +}
> +
>   static inline void media_request_object_init(struct media_request_object
*obj)
>   {
>          obj->ops = NULL;
> --
> 2.16.3

Re: [RFCv11 PATCH 20/29] videobuf2-v4l2: integrate with media requests

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> This implements the V4L2 part of the request support. The main
> change is that vb2_qbuf and vb2_prepare_buf now have a new
> media_device pointer. This required changes to several drivers
> that did not use the vb2_ioctl_qbuf/prepare_buf helper functions.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/common/videobuf2/videobuf2-v4l2.c  | 84
++++++++++++++++++++----
>   drivers/media/platform/omap3isp/ispvideo.c       |  2 +-
>   drivers/media/platform/s3c-camif/camif-capture.c |  4 +-
>   drivers/media/platform/s5p-mfc/s5p_mfc_dec.c     |  4 +-
>   drivers/media/platform/s5p-mfc/s5p_mfc_enc.c     |  4 +-
>   drivers/media/platform/soc_camera/soc_camera.c   |  4 +-
>   drivers/media/usb/uvc/uvc_queue.c                |  5 +-
>   drivers/media/usb/uvc/uvc_v4l2.c                 |  3 +-
>   drivers/media/usb/uvc/uvcvideo.h                 |  1 +
>   drivers/media/v4l2-core/v4l2-mem2mem.c           |  7 +-
>   drivers/staging/media/davinci_vpfe/vpfe_video.c  |  3 +-
>   drivers/staging/media/omap4iss/iss_video.c       |  3 +-
>   drivers/usb/gadget/function/uvc_queue.c          |  2 +-
>   include/media/videobuf2-v4l2.h                   | 12 +++-
>   14 files changed, 106 insertions(+), 32 deletions(-)

> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> index b8d370b97cca..73c1fd4da58a 100644
> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
> @@ -25,6 +25,7 @@
>   #include <linux/kthread.h>

>   #include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
>   #include <media/v4l2-fh.h>
>   #include <media/v4l2-event.h>
>   #include <media/v4l2-common.h>
> @@ -40,10 +41,12 @@ module_param(debug, int, 0644);
>                          pr_info("vb2-v4l2: %s: " fmt, __func__, ## arg); \
>          } while (0)

> -/* Flags that are set by the vb2 core */
> +/* Flags that are set by us */
>   #define V4L2_BUFFER_MASK_FLAGS (V4L2_BUF_FLAG_MAPPED |
V4L2_BUF_FLAG_QUEUED | \
>                                   V4L2_BUF_FLAG_DONE | V4L2_BUF_FLAG_ERROR
| \
>                                   V4L2_BUF_FLAG_PREPARED | \
> +                                V4L2_BUF_FLAG_IN_REQUEST | \
> +                                V4L2_BUF_FLAG_REQUEST_FD | \
>                                   V4L2_BUF_FLAG_TIMESTAMP_MASK)
>   /* Output buffer flags that should be passed on to the driver */
>   #define V4L2_BUFFER_OUT_FLAGS  (V4L2_BUF_FLAG_PFRAME |
V4L2_BUF_FLAG_BFRAME | \
> @@ -318,13 +321,17 @@ static int vb2_fill_vb2_v4l2_buffer(struct
vb2_buffer *vb, struct v4l2_buffer *b
>          return 0;
>   }

> -static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct
v4l2_buffer *b,
> -                                   const char *opname)
> +static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct
media_device *mdev,
> +                                   struct v4l2_buffer *b,
> +                                   const char *opname,
> +                                   struct media_request **p_req)
>   {
> +       struct media_request *req;
>          struct vb2_v4l2_buffer *vbuf;
>          struct vb2_buffer *vb;
>          int ret;

> +       *p_req = NULL;
>          if (b->type != q->type) {
>                  dprintk(1, "%s: invalid buffer type\n", opname);
>                  return -EINVAL;
> @@ -354,7 +361,38 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue
*q, struct v4l2_buffer *b,

>          /* Copy relevant information provided by the userspace */
>          memset(vbuf->planes, 0, sizeof(vbuf->planes[0]) * vb->num_planes);
> -       return vb2_fill_vb2_v4l2_buffer(vb, b);
> +       ret = vb2_fill_vb2_v4l2_buffer(vb, b);
> +       if (ret)
> +               return ret;
> +
> +       if (!(b->flags & V4L2_BUF_FLAG_REQUEST_FD))
> +               return 0;
> +
> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
> +               dprintk(1, "%s: buffer is not in dequeued state\n",
opname);
> +               return -EINVAL;
> +       }
> +
> +       if (b->request_fd < 0) {
> +               dprintk(1, "%s: request_fd < 0\n", opname);
> +               return -EINVAL;
> +       }
> +
> +       req = media_request_find(mdev, b->request_fd);
> +       if (IS_ERR(req)) {
> +               dprintk(1, "%s: invalid request_fd\n", opname);
> +               return PTR_ERR(req);
> +       }
> +
> +       if (req->state != MEDIA_REQUEST_STATE_IDLE) {
> +               dprintk(1, "%s: request is not idle\n", opname);
> +               media_request_put(req);
> +               return -EBUSY;
> +       }
> +
> +       *p_req = req;
> +
> +       return 0;
>   }

>   /*
> @@ -437,6 +475,9 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb,
void *pb)
>          case VB2_BUF_STATE_ACTIVE:
>                  b->flags |= V4L2_BUF_FLAG_QUEUED;
>                  break;
> +       case VB2_BUF_STATE_IN_REQUEST:
> +               b->flags |= V4L2_BUF_FLAG_IN_REQUEST;
> +               break;
>          case VB2_BUF_STATE_ERROR:
>                  b->flags |= V4L2_BUF_FLAG_ERROR;
>                  /* fall through */
> @@ -455,6 +496,10 @@ static void __fill_v4l2_buffer(struct vb2_buffer
*vb, void *pb)

>          if (vb2_buffer_in_use(q, vb))
>                  b->flags |= V4L2_BUF_FLAG_MAPPED;
> +       if (vb->req_obj.req) {
> +               b->flags |= V4L2_BUF_FLAG_REQUEST_FD;
> +               b->request_fd = -1;
> +       }

>          if (!q->is_output &&
>                  b->flags & V4L2_BUF_FLAG_DONE &&
> @@ -533,8 +578,10 @@ int vb2_reqbufs(struct vb2_queue *q, struct
v4l2_requestbuffers *req)
>   }
>   EXPORT_SYMBOL_GPL(vb2_reqbufs);

> -int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b)
> +int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> +                   struct v4l2_buffer *b)
>   {
> +       struct media_request *req;
>          int ret;

>          if (vb2_fileio_is_active(q)) {
> @@ -542,9 +589,13 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
v4l2_buffer *b)
>                  return -EBUSY;
>          }

> -       ret = vb2_queue_or_prepare_buf(q, b, "prepare_buf");
> -
> -       return ret ? ret : vb2_core_prepare_buf(q, b->index, b, NULL);
> +       ret = vb2_queue_or_prepare_buf(q, mdev, b, "prepare_buf", &req);
> +       if (ret)
> +               return ret;
> +       ret = vb2_core_prepare_buf(q, b->index, b, req);
> +       if (req)
> +               media_request_put(req);
> +       return ret;
>   }
>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);

> @@ -602,8 +653,10 @@ int vb2_create_bufs(struct vb2_queue *q, struct
v4l2_create_buffers *create)
>   }
>   EXPORT_SYMBOL_GPL(vb2_create_bufs);

> -int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
> +int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
> +            struct v4l2_buffer *b)

That extra mdev argument is request-specific and not very nice to have here.
Isn't it possible instead to obtain the media_device by looking up through
the
queue, or at least to cache a pointer to the media_device inside vb2_queue
when
requests are in use? After all a queue can only be used with one
media_device.

Btw, that's also one of the reasons why I used to put request management
data
into its own structure instead of having it directly in media_device.
Non-request users would wonder what is a media_device doing here. With a
dedicated struct to manage requests, the intent is at least clear. Even if
we
mandate the use of media_device, it's never a bad idea to clearly sort your
data members by purpose.

>   {
> +       struct media_request *req;
>          int ret;

>          if (vb2_fileio_is_active(q)) {
> @@ -611,8 +664,13 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer
*b)
>                  return -EBUSY;
>          }

> -       ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
> -       return ret ? ret : vb2_core_qbuf(q, b->index, b, NULL);
> +       ret = vb2_queue_or_prepare_buf(q, mdev, b, "qbuf", &req);
> +       if (ret)
> +               return ret;
> +       ret = vb2_core_qbuf(q, b->index, b, req);
> +       if (req)
> +               media_request_put(req);
> +       return ret;
>   }
>   EXPORT_SYMBOL_GPL(vb2_qbuf);

> @@ -802,7 +860,7 @@ int vb2_ioctl_prepare_buf(struct file *file, void
*priv,

>          if (vb2_queue_is_busy(vdev, file))
>                  return -EBUSY;
> -       return vb2_prepare_buf(vdev->queue, p);
> +       return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p);
>   }
>   EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf);

> @@ -821,7 +879,7 @@ int vb2_ioctl_qbuf(struct file *file, void *priv,
struct v4l2_buffer *p)

>          if (vb2_queue_is_busy(vdev, file))
>                  return -EBUSY;
> -       return vb2_qbuf(vdev->queue, p);
> +       return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p);
>   }
>   EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf);

> diff --git a/drivers/media/platform/omap3isp/ispvideo.c
b/drivers/media/platform/omap3isp/ispvideo.c
> index bd564c2e767f..9fecbd8c6edd 100644
> --- a/drivers/media/platform/omap3isp/ispvideo.c
> +++ b/drivers/media/platform/omap3isp/ispvideo.c
> @@ -940,7 +940,7 @@ isp_video_qbuf(struct file *file, void *fh, struct
v4l2_buffer *b)
>          int ret;

>          mutex_lock(&video->queue_lock);
> -       ret = vb2_qbuf(&vfh->queue, b);
> +       ret = vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
>          mutex_unlock(&video->queue_lock);

>          return ret;
> diff --git a/drivers/media/platform/s3c-camif/camif-capture.c
b/drivers/media/platform/s3c-camif/camif-capture.c
> index 9ab8e7ee2e1e..fafb6da3e804 100644
> --- a/drivers/media/platform/s3c-camif/camif-capture.c
> +++ b/drivers/media/platform/s3c-camif/camif-capture.c
> @@ -941,7 +941,7 @@ static int s3c_camif_qbuf(struct file *file, void
*priv,
>          if (vp->owner && vp->owner != priv)
>                  return -EBUSY;

> -       return vb2_qbuf(&vp->vb_queue, buf);
> +       return vb2_qbuf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, buf);
>   }

>   static int s3c_camif_dqbuf(struct file *file, void *priv,
> @@ -979,7 +979,7 @@ static int s3c_camif_prepare_buf(struct file *file,
void *priv,
>                                   struct v4l2_buffer *b)
>   {
>          struct camif_vp *vp = video_drvdata(file);
> -       return vb2_prepare_buf(&vp->vb_queue, b);
> +       return vb2_prepare_buf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, b);
>   }

>   static int s3c_camif_g_selection(struct file *file, void *priv,
> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
> index 5cf4d9921264..3d863e3f5798 100644
> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
> @@ -632,9 +632,9 @@ static int vidioc_qbuf(struct file *file, void *priv,
struct v4l2_buffer *buf)
>                  return -EIO;
>          }
>          if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
> -               return vb2_qbuf(&ctx->vq_src, buf);
> +               return vb2_qbuf(&ctx->vq_src, NULL, buf);
>          else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
> -               return vb2_qbuf(&ctx->vq_dst, buf);
> +               return vb2_qbuf(&ctx->vq_dst, NULL, buf);
>          return -EINVAL;
>   }

> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> index 5c0462ca9993..ed12d5d062e8 100644
> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> @@ -1621,9 +1621,9 @@ static int vidioc_qbuf(struct file *file, void
*priv, struct v4l2_buffer *buf)
>                          mfc_err("Call on QBUF after EOS command\n");
>                          return -EIO;
>                  }
> -               return vb2_qbuf(&ctx->vq_src, buf);
> +               return vb2_qbuf(&ctx->vq_src, NULL, buf);
>          } else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
> -               return vb2_qbuf(&ctx->vq_dst, buf);
> +               return vb2_qbuf(&ctx->vq_dst, NULL, buf);
>          }
>          return -EINVAL;
>   }
> diff --git a/drivers/media/platform/soc_camera/soc_camera.c
b/drivers/media/platform/soc_camera/soc_camera.c
> index e6787abc34ae..08adf9a79420 100644
> --- a/drivers/media/platform/soc_camera/soc_camera.c
> +++ b/drivers/media/platform/soc_camera/soc_camera.c
> @@ -394,7 +394,7 @@ static int soc_camera_qbuf(struct file *file, void
*priv,
>          if (icd->streamer != file)
>                  return -EBUSY;

> -       return vb2_qbuf(&icd->vb2_vidq, p);
> +       return vb2_qbuf(&icd->vb2_vidq, NULL, p);
>   }

>   static int soc_camera_dqbuf(struct file *file, void *priv,
> @@ -430,7 +430,7 @@ static int soc_camera_prepare_buf(struct file *file,
void *priv,
>   {
>          struct soc_camera_device *icd = file->private_data;

> -       return vb2_prepare_buf(&icd->vb2_vidq, b);
> +       return vb2_prepare_buf(&icd->vb2_vidq, NULL, b);
>   }

>   static int soc_camera_expbuf(struct file *file, void *priv,
> diff --git a/drivers/media/usb/uvc/uvc_queue.c
b/drivers/media/usb/uvc/uvc_queue.c
> index fecccb5e7628..8964e16f2b22 100644
> --- a/drivers/media/usb/uvc/uvc_queue.c
> +++ b/drivers/media/usb/uvc/uvc_queue.c
> @@ -300,12 +300,13 @@ int uvc_create_buffers(struct uvc_video_queue
*queue,
>          return ret;
>   }

> -int uvc_queue_buffer(struct uvc_video_queue *queue, struct v4l2_buffer
*buf)
> +int uvc_queue_buffer(struct uvc_video_queue *queue,
> +                    struct media_device *mdev, struct v4l2_buffer *buf)
>   {
>          int ret;

>          mutex_lock(&queue->mutex);
> -       ret = vb2_qbuf(&queue->queue, buf);
> +       ret = vb2_qbuf(&queue->queue, mdev, buf);
>          mutex_unlock(&queue->mutex);

>          return ret;
> diff --git a/drivers/media/usb/uvc/uvc_v4l2.c
b/drivers/media/usb/uvc/uvc_v4l2.c
> index bd32914259ae..3da5fdc002ac 100644
> --- a/drivers/media/usb/uvc/uvc_v4l2.c
> +++ b/drivers/media/usb/uvc/uvc_v4l2.c
> @@ -751,7 +751,8 @@ static int uvc_ioctl_qbuf(struct file *file, void
*fh, struct v4l2_buffer *buf)
>          if (!uvc_has_privileges(handle))
>                  return -EBUSY;

> -       return uvc_queue_buffer(&stream->queue, buf);
> +       return uvc_queue_buffer(&stream->queue,
> +                               stream->vdev.v4l2_dev->mdev, buf);
>   }

>   static int uvc_ioctl_expbuf(struct file *file, void *fh,
> diff --git a/drivers/media/usb/uvc/uvcvideo.h
b/drivers/media/usb/uvc/uvcvideo.h
> index be5cf179228b..bc9ed18f043c 100644
> --- a/drivers/media/usb/uvc/uvcvideo.h
> +++ b/drivers/media/usb/uvc/uvcvideo.h
> @@ -680,6 +680,7 @@ int uvc_query_buffer(struct uvc_video_queue *queue,
>   int uvc_create_buffers(struct uvc_video_queue *queue,
>                         struct v4l2_create_buffers *v4l2_cb);
>   int uvc_queue_buffer(struct uvc_video_queue *queue,
> +                    struct media_device *mdev,
>                       struct v4l2_buffer *v4l2_buf);
>   int uvc_export_buffer(struct uvc_video_queue *queue,
>                        struct v4l2_exportbuffer *exp);
> diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
> index c4f963d96a79..438f1b869319 100644
> --- a/drivers/media/v4l2-core/v4l2-mem2mem.c
> +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
> @@ -20,6 +20,7 @@
>   #include <media/videobuf2-v4l2.h>
>   #include <media/v4l2-mem2mem.h>
>   #include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
>   #include <media/v4l2-fh.h>
>   #include <media/v4l2-event.h>

> @@ -388,11 +389,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_querybuf);
>   int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>                    struct v4l2_buffer *buf)
>   {
> +       struct video_device *vdev = video_devdata(file);
>          struct vb2_queue *vq;
>          int ret;

>          vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
> -       ret = vb2_qbuf(vq, buf);
> +       ret = vb2_qbuf(vq, vdev->v4l2_dev->mdev, buf);
>          if (!ret)
>                  v4l2_m2m_try_schedule(m2m_ctx);

> @@ -413,11 +415,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_dqbuf);
>   int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>                           struct v4l2_buffer *buf)
>   {
> +       struct video_device *vdev = video_devdata(file);
>          struct vb2_queue *vq;
>          int ret;

>          vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
> -       ret = vb2_prepare_buf(vq, buf);
> +       ret = vb2_prepare_buf(vq, vdev->v4l2_dev->mdev, buf);
>          if (!ret)
>                  v4l2_m2m_try_schedule(m2m_ctx);

> diff --git a/drivers/staging/media/davinci_vpfe/vpfe_video.c
b/drivers/staging/media/davinci_vpfe/vpfe_video.c
> index 588743a6fd8a..00bf28e830d4 100644
> --- a/drivers/staging/media/davinci_vpfe/vpfe_video.c
> +++ b/drivers/staging/media/davinci_vpfe/vpfe_video.c
> @@ -1426,7 +1426,8 @@ static int vpfe_qbuf(struct file *file, void *priv,
>                  return -EACCES;
>          }

> -       return vb2_qbuf(&video->buffer_queue, p);
> +       return vb2_qbuf(&video->buffer_queue,
> +                       video->video_dev.v4l2_dev->mdev, p);
>   }

>   /*
> diff --git a/drivers/staging/media/omap4iss/iss_video.c
b/drivers/staging/media/omap4iss/iss_video.c
> index a3a83424a926..a35d1004b522 100644
> --- a/drivers/staging/media/omap4iss/iss_video.c
> +++ b/drivers/staging/media/omap4iss/iss_video.c
> @@ -805,9 +805,10 @@ iss_video_querybuf(struct file *file, void *fh,
struct v4l2_buffer *b)
>   static int
>   iss_video_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
>   {
> +       struct iss_video *video = video_drvdata(file);
>          struct iss_video_fh *vfh = to_iss_video_fh(fh);

> -       return vb2_qbuf(&vfh->queue, b);
> +       return vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
>   }

>   static int
> diff --git a/drivers/usb/gadget/function/uvc_queue.c
b/drivers/usb/gadget/function/uvc_queue.c
> index 9e33d5206d54..f2497cb96abb 100644
> --- a/drivers/usb/gadget/function/uvc_queue.c
> +++ b/drivers/usb/gadget/function/uvc_queue.c
> @@ -166,7 +166,7 @@ int uvcg_queue_buffer(struct uvc_video_queue *queue,
struct v4l2_buffer *buf)
>          unsigned long flags;
>          int ret;

> -       ret = vb2_qbuf(&queue->queue, buf);
> +       ret = vb2_qbuf(&queue->queue, NULL, buf);
>          if (ret < 0)
>                  return ret;

> diff --git a/include/media/videobuf2-v4l2.h
b/include/media/videobuf2-v4l2.h
> index 097bf3e6951d..cf312ab4e7e8 100644
> --- a/include/media/videobuf2-v4l2.h
> +++ b/include/media/videobuf2-v4l2.h
> @@ -79,6 +79,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct
v4l2_create_buffers *create);
>    * vb2_prepare_buf() - Pass ownership of a buffer from userspace to the
kernel
>    *
>    * @q:         pointer to &struct vb2_queue with videobuf2 queue.
> + * @mdev:      pointer to &struct media_device, may be NULL.
>    * @b:         buffer structure passed from userspace to
>    *             &v4l2_ioctl_ops->vidioc_prepare_buf handler in driver
>    *
> @@ -90,15 +91,19 @@ int vb2_create_bufs(struct vb2_queue *q, struct
v4l2_create_buffers *create);
>    * #) verifies the passed buffer,
>    * #) calls &vb2_ops->buf_prepare callback in the driver (if provided),
>    *    in which driver-specific buffer initialization can be performed.
> + * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
> + *    then bind the prepared buffer to the request.
>    *
>    * The return values from this function are intended to be directly
returned
>    * from &v4l2_ioctl_ops->vidioc_prepare_buf handler in driver.
>    */
> -int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
> +int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
> +                   struct v4l2_buffer *b);

>   /**
>    * vb2_qbuf() - Queue a buffer from userspace
>    * @q:         pointer to &struct vb2_queue with videobuf2 queue.
> + * @mdev:      pointer to &struct media_device, may be NULL.
>    * @b:         buffer structure passed from userspace to
>    *             &v4l2_ioctl_ops->vidioc_qbuf handler in driver
>    *
> @@ -107,6 +112,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
v4l2_buffer *b);
>    * This function:
>    *
>    * #) verifies the passed buffer;
> + * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
> + *    then bind the buffer to the request.
>    * #) if necessary, calls &vb2_ops->buf_prepare callback in the driver
>    *    (if provided), in which driver-specific buffer initialization can
>    *    be performed;
> @@ -116,7 +123,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
v4l2_buffer *b);
>    * The return values from this function are intended to be directly
returned
>    * from &v4l2_ioctl_ops->vidioc_qbuf handler in driver.
>    */
> -int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b);
> +int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
> +            struct v4l2_buffer *b);

>   /**
>    * vb2_expbuf() - Export a buffer as a file descriptor
> --
> 2.16.3

Re: [RFCv11 PATCH 25/29] media: vim2m: add media device

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Alexandre Courbot <acourbot@chromium.org>

> Request API requires a media node. Add one to the vim2m driver so we can
> use requests with it.

> This probably needs a bit more work to correctly represent m2m
> hardware in the media topology.

> Signed-off-by: Alexandre Courbot <acourbot@chromium.org>

I don't remember writing this - actually IIRC you came with this patch
initially.

> ---
>   drivers/media/platform/vim2m.c | 43
+++++++++++++++++++++++++++++++++++++-----
>   1 file changed, 38 insertions(+), 5 deletions(-)

> diff --git a/drivers/media/platform/vim2m.c
b/drivers/media/platform/vim2m.c
> index 065483e62db4..ef970434af13 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -140,6 +140,10 @@ static struct vim2m_fmt *find_format(struct
v4l2_format *f)
>   struct vim2m_dev {
>          struct v4l2_device      v4l2_dev;
>          struct video_device     vfd;
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +       struct media_device     mdev;
> +       struct media_pad        pad[2];
> +#endif

>          atomic_t                num_inst;
>          struct mutex            dev_mutex;
> @@ -1000,11 +1004,6 @@ static int vim2m_probe(struct platform_device
*pdev)
>                  return -ENOMEM;

>          spin_lock_init(&dev->irqlock);
> -
> -       ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> -       if (ret)
> -               return ret;
> -
>          atomic_set(&dev->num_inst, 0);
>          mutex_init(&dev->dev_mutex);

> @@ -1013,6 +1012,22 @@ static int vim2m_probe(struct platform_device
*pdev)
>          vfd->lock = &dev->dev_mutex;
>          vfd->v4l2_dev = &dev->v4l2_dev;

> +#ifdef CONFIG_MEDIA_CONTROLLER
> +       dev->mdev.dev = &pdev->dev;
> +       strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
> +       media_device_init(&dev->mdev);
> +       dev->v4l2_dev.mdev = &dev->mdev;
> +       dev->pad[0].flags = MEDIA_PAD_FL_SINK;
> +       dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
> +       ret = media_entity_pads_init(&vfd->entity, 2, dev->pad);
> +       if (ret)
> +               return ret;
> +#endif
> +
> +       ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> +       if (ret)
> +               goto unreg_media;
> +
>          ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
>          if (ret) {
>                  v4l2_err(&dev->v4l2_dev, "Failed to register video
device\n");
> @@ -1034,6 +1049,13 @@ static int vim2m_probe(struct platform_device
*pdev)
>                  goto err_m2m;
>          }

> +#ifdef CONFIG_MEDIA_CONTROLLER
> +       /* Register the media device node */
> +       ret = media_device_register(&dev->mdev);
> +       if (ret)
> +               goto err_m2m;
> +#endif
> +
>          return 0;

>   err_m2m:
> @@ -1041,6 +1063,10 @@ static int vim2m_probe(struct platform_device
*pdev)
>          video_unregister_device(&dev->vfd);
>   unreg_dev:
>          v4l2_device_unregister(&dev->v4l2_dev);
> +unreg_media:
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +       media_device_unregister(&dev->mdev);
> +#endif

>          return ret;
>   }
> @@ -1050,6 +1076,13 @@ static int vim2m_remove(struct platform_device
*pdev)
>          struct vim2m_dev *dev = platform_get_drvdata(pdev);

>          v4l2_info(&dev->v4l2_dev, "Removing " MEM2MEM_NAME);
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER
> +       if (media_devnode_is_registered(dev->mdev.devnode))
> +               media_device_unregister(&dev->mdev);
> +       media_device_cleanup(&dev->mdev);
> +#endif
> +
>          v4l2_m2m_release(dev->m2m_dev);
>          del_timer_sync(&dev->timer);
>          video_unregister_device(&dev->vfd);
> --
> 2.16.3

Re: [RFCv11 PATCH 27/29] vim2m: support requests

[Alexandre Courbot]

On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> From: Hans Verkuil <hans.verkuil@cisco.com>

> Add support for requests to vim2m.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>   drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
>   1 file changed, 25 insertions(+)

> diff --git a/drivers/media/platform/vim2m.c
b/drivers/media/platform/vim2m.c
> index 9b18b32c255d..2dcf0ea85705 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -387,8 +387,26 @@ static void device_run(void *priv)
>          src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
>          dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);

> +       /* Apply request if needed */
> +       if (src_buf->vb2_buf.req_obj.req)
> +               v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
> +                                       &ctx->hdl);
> +       if (dst_buf->vb2_buf.req_obj.req &&
> +           dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
> +               v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
> +                                       &ctx->hdl);

This implies that we can have two different requests active at the same time
for the same device. Do we want to open that can of worms?

Valid scenarios for requests should be clearly defined. In the case of a
simple
buffer processor like vim2m, something like "request required for source
buffer,
optional and must be same as source if specified for destination", looks
simple
and sane.

> +
>          device_process(ctx, src_buf, dst_buf);

> +       /* Complete request if needed */
> +       if (src_buf->vb2_buf.req_obj.req)
> +               v4l2_ctrl_request_complete(src_buf->vb2_buf.req_obj.req,
> +                                       &ctx->hdl);
> +       if (dst_buf->vb2_buf.req_obj.req &&
> +           dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
> +               v4l2_ctrl_request_complete(dst_buf->vb2_buf.req_obj.req,
> +                                       &ctx->hdl);
> +
>          /* Run a timer, which simulates a hardware irq  */
>          schedule_irq(dev, ctx->transtime);
>   }
> @@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue *q)
>                          vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>                  if (vbuf == NULL)
>                          return;
> +               v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
> +                                          &ctx->hdl);
>                  spin_lock_irqsave(&ctx->dev->irqlock, flags);
>                  v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
>                  spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> @@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
>          .job_abort      = job_abort,
>   };

> +static const struct media_device_ops m2m_media_ops = {
> +       .req_queue = vb2_request_queue,
> +};
> +
>   static int vim2m_probe(struct platform_device *pdev)
>   {
>          struct vim2m_dev *dev;
> @@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device *pdev)
>          dev->mdev.dev = &pdev->dev;
>          strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
>          media_device_init(&dev->mdev);
> +       dev->mdev.ops = &m2m_media_ops;
>          dev->v4l2_dev.mdev = &dev->mdev;
>          dev->pad[0].flags = MEDIA_PAD_FL_SINK;
>          dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
> --
> 2.16.3

Re: [RFCv11 PATCH 00/29] Request API

[Hans Verkuil]

On 04/17/2018 06:33 AM, Alexandre Courbot wrote:
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Hi all,
> 
>> This is a cleaned up version of the v10 series (never posted to
>> the list since it was messy).
> 
> Hi Hans,
> 
> It took me a while to test and review this, but finally have been able to
> do it.
> 
> First the result of the test: I have tried porting my dummy vim2m test
> program
> (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a42 for
> reference),
> and am getting a hang when trying to queue the second OUTPUT buffer (right
> after
> queuing the first request). If I move the calls the VIDIOC_STREAMON after
> the
> requests are queued, the hang seems to happen at that moment. Probably a
> deadlock, haven't looked in detail yet.
> 
> I have a few other comments, will follow up per-patch.
> 

I had a similar/same (?) report about this from Paul:

https://www.mail-archive.com/linux-media@vger.kernel.org/msg129177.html

I hope to resume my Request API work on Thursday.

Regards,

	Hans

Re: [RFCv11 PATCH 00/29] Request API

[Alexandre Courbot]

On Tue, Apr 17, 2018 at 3:12 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:

> On 04/17/2018 06:33 AM, Alexandre Courbot wrote:
> > On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> >
> >> From: Hans Verkuil <hans.verkuil@cisco.com>
> >
> >> Hi all,
> >
> >> This is a cleaned up version of the v10 series (never posted to
> >> the list since it was messy).
> >
> > Hi Hans,
> >
> > It took me a while to test and review this, but finally have been able
to
> > do it.
> >
> > First the result of the test: I have tried porting my dummy vim2m test
> > program
> > (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a42 for
> > reference),
> > and am getting a hang when trying to queue the second OUTPUT buffer
(right
> > after
> > queuing the first request). If I move the calls the VIDIOC_STREAMON
after
> > the
> > requests are queued, the hang seems to happen at that moment. Probably a
> > deadlock, haven't looked in detail yet.
> >
> > I have a few other comments, will follow up per-patch.
> >

> I had a similar/same (?) report about this from Paul:

> https://www.mail-archive.com/linux-media@vger.kernel.org/msg129177.html

I saw this and tried to move the call to STREAMON to after the requests are
queued in my example program, but it then hanged there. So there is
probably something more intricate taking place.

Re: [RFCv11 PATCH 00/29] Request API

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2014 bytes --]

Hi,

On Tue, 2018-04-17 at 06:17 +0000, Alexandre Courbot wrote:
> On Tue, Apr 17, 2018 at 3:12 PM Hans Verkuil <hverkuil@xs4all.nl>
> wrote:
> 
> > On 04/17/2018 06:33 AM, Alexandre Courbot wrote:
> > > On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl>
> > > wrote:
> > > 
> > > > From: Hans Verkuil <hans.verkuil@cisco.com>
> > > > Hi all,
> > > > This is a cleaned up version of the v10 series (never posted to
> > > > the list since it was messy).
> > > 
> > > Hi Hans,
> > > 
> > > It took me a while to test and review this, but finally have been
> > > able
> 
> to
> > > do it.
> > > 
> > > First the result of the test: I have tried porting my dummy vim2m
> > > test
> > > program
> > > (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a42
> > > for
> > > reference),
> > > and am getting a hang when trying to queue the second OUTPUT
> > > buffer
> 
> (right
> > > after
> > > queuing the first request). If I move the calls the
> > > VIDIOC_STREAMON
> 
> after
> > > the
> > > requests are queued, the hang seems to happen at that moment.
> > > Probably a
> > > deadlock, haven't looked in detail yet.
> > > 
> > > I have a few other comments, will follow up per-patch.
> > > 
> > I had a similar/same (?) report about this from Paul:
> > https://www.mail-archive.com/linux-media@vger.kernel.org/msg129177.h
> > tml
> 
> I saw this and tried to move the call to STREAMON to after the
> requests are queued in my example program, but it then hanged there.
> So there is probably something more intricate taking place.

I figured out the issue (but forgot to report back to the list): Hans'
version of the request API doesn't set the POLLIN bit but POLLPRI
instead, so you need to select for expect_fds instead of read_fds in the
select call. That's pretty much all there is to it.

Hope this helps,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFCv11 PATCH 27/29] vim2m: support requests

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 3055 bytes --]

Hi,

On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> Add support for requests to vim2m.

Please find a nit below.

> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
>  1 file changed, 25 insertions(+)
> 
> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
> index 9b18b32c255d..2dcf0ea85705 100644
> --- a/drivers/media/platform/vim2m.c
> +++ b/drivers/media/platform/vim2m.c
> @@ -387,8 +387,26 @@ static void device_run(void *priv)
>  	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
>  	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
>  
> +	/* Apply request if needed */

This comment suggests that this is where request submission is handled.
I suggest making it clear that this the place where the *controls*
attached to the request are applied, instead.

> +	if (src_buf->vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +	if (dst_buf->vb2_buf.req_obj.req &&
> +	    dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +
>  	device_process(ctx, src_buf, dst_buf);
>  
> +	/* Complete request if needed */

Ditto here.

> +	if (src_buf->vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_complete(src_buf-
> >vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +	if (dst_buf->vb2_buf.req_obj.req &&
> +	    dst_buf->vb2_buf.req_obj.req != src_buf-
> >vb2_buf.req_obj.req)
> +		v4l2_ctrl_request_complete(dst_buf-
> >vb2_buf.req_obj.req,
> +					&ctx->hdl);
> +
>  	/* Run a timer, which simulates a hardware irq  */
>  	schedule_irq(dev, ctx->transtime);
>  }
> @@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue
> *q)
>  			vbuf = v4l2_m2m_dst_buf_remove(ctx-
> >fh.m2m_ctx);
>  		if (vbuf == NULL)
>  			return;
> +		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
> +					   &ctx->hdl);
>  		spin_lock_irqsave(&ctx->dev->irqlock, flags);
>  		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
>  		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
> @@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
>  	.job_abort	= job_abort,
>  };
>  
> +static const struct media_device_ops m2m_media_ops = {
> +	.req_queue = vb2_request_queue,
> +};
> +
>  static int vim2m_probe(struct platform_device *pdev)
>  {
>  	struct vim2m_dev *dev;
> @@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device
> *pdev)
>  	dev->mdev.dev = &pdev->dev;
>  	strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
>  	media_device_init(&dev->mdev);
> +	dev->mdev.ops = &m2m_media_ops;
>  	dev->v4l2_dev.mdev = &dev->mdev;
>  	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
>  	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFCv11 PATCH 00/29] Request API

[Alexandre Courbot]

On Tue, Apr 17, 2018 at 8:41 PM Paul Kocialkowski <
paul.kocialkowski@bootlin.com> wrote:

> Hi,

> On Tue, 2018-04-17 at 06:17 +0000, Alexandre Courbot wrote:
> > On Tue, Apr 17, 2018 at 3:12 PM Hans Verkuil <hverkuil@xs4all.nl>
> > wrote:
> >
> > > On 04/17/2018 06:33 AM, Alexandre Courbot wrote:
> > > > On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl>
> > > > wrote:
> > > >
> > > > > From: Hans Verkuil <hans.verkuil@cisco.com>
> > > > > Hi all,
> > > > > This is a cleaned up version of the v10 series (never posted to
> > > > > the list since it was messy).
> > > >
> > > > Hi Hans,
> > > >
> > > > It took me a while to test and review this, but finally have been
> > > > able
> >
> > to
> > > > do it.
> > > >
> > > > First the result of the test: I have tried porting my dummy vim2m
> > > > test
> > > > program
> > > > (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a42
> > > > for
> > > > reference),
> > > > and am getting a hang when trying to queue the second OUTPUT
> > > > buffer
> >
> > (right
> > > > after
> > > > queuing the first request). If I move the calls the
> > > > VIDIOC_STREAMON
> >
> > after
> > > > the
> > > > requests are queued, the hang seems to happen at that moment.
> > > > Probably a
> > > > deadlock, haven't looked in detail yet.
> > > >
> > > > I have a few other comments, will follow up per-patch.
> > > >
> > > I had a similar/same (?) report about this from Paul:
> > > https://www.mail-archive.com/linux-media@vger.kernel.org/msg129177.h
> > > tml
> >
> > I saw this and tried to move the call to STREAMON to after the
> > requests are queued in my example program, but it then hanged there.
> > So there is probably something more intricate taking place.

> I figured out the issue (but forgot to report back to the list): Hans'
> version of the request API doesn't set the POLLIN bit but POLLPRI
> instead, so you need to select for expect_fds instead of read_fds in the
> select call. That's pretty much all there is to it.

I am not using select() but poll() in my test program (see the gist link
above) and have set POLLPRI as the event to poll for. I may be missing
something but this looks correct to me?

Re: [RFCv11 PATCH 00/29] Request API

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4199 bytes --]

Hi,

On Wed, 2018-04-18 at 02:06 +0000, Alexandre Courbot wrote:
> On Tue, Apr 17, 2018 at 8:41 PM Paul Kocialkowski <
> paul.kocialkowski@bootlin.com> wrote:
> On Tue, 2018-04-17 at 06:17 +0000, Alexandre Courbot wrote:
> > > On Tue, Apr 17, 2018 at 3:12 PM Hans Verkuil <hverkuil@xs4all.nl>
> > > wrote:
> > > 
> > > > On 04/17/2018 06:33 AM, Alexandre Courbot wrote:
> > > > > On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.
> > > > > nl>
> > > > > wrote:
> > > > > 
> > > > > > From: Hans Verkuil <hans.verkuil@cisco.com>
> > > > > > Hi all,
> > > > > > This is a cleaned up version of the v10 series (never posted
> > > > > > to
> > > > > > the list since it was messy).
> > > > > 
> > > > > Hi Hans,
> > > > > 
> > > > > It took me a while to test and review this, but finally have
> > > > > been
> > > > > able
> > > 
> > > to
> > > > > do it.
> > > > > 
> > > > > First the result of the test: I have tried porting my dummy
> > > > > vim2m
> > > > > test
> > > > > program
> > > > > (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a
> > > > > 42
> > > > > for
> > > > > reference),
> > > > > and am getting a hang when trying to queue the second OUTPUT
> > > > > buffer
> > > 
> > > (right
> > > > > after
> > > > > queuing the first request). If I move the calls the
> > > > > VIDIOC_STREAMON
> > > 
> > > after
> > > > > the
> > > > > requests are queued, the hang seems to happen at that moment.
> > > > > Probably a
> > > > > deadlock, haven't looked in detail yet.
> > > > > 
> > > > > I have a few other comments, will follow up per-patch.
> > > > > 
> > > > 
> > > > I had a similar/same (?) report about this from Paul:
> > > > https://www.mail-archive.com/linux-media@vger.kernel.org/msg1291
> > > > 77.h
> > > > tml
> > > 
> > > I saw this and tried to move the call to STREAMON to after the
> > > requests are queued in my example program, but it then hanged
> > > there.
> > > So there is probably something more intricate taking place.
> > I figured out the issue (but forgot to report back to the list):
> > Hans'
> > version of the request API doesn't set the POLLIN bit but POLLPRI
> > instead, so you need to select for expect_fds instead of read_fds in
> > the
> > select call. That's pretty much all there is to it.
> 
> I am not using select() but poll() in my test program (see the gist
> link
> above) and have set POLLPRI as the event to poll for. I may be missing
> something but this looks correct to me?

You're right, I overlooked your email and assumed you were hitting the
same issue that I had at first.

Anyway, I also had a similar issue when calling the STREAMON ioctl
*before* enqueuing the request. What happens here is that
v4l2_m2m_streamon (called from the ioctl) will also try to schedule a
device run (v4l2_m2m_try_schedule). When the ioctl is called and the
request was not queued yet, the lack of buffers will delay the streamon
call. Then, when the request is later submitted, vb2_streamon is called
with the queue directly, and there is no m2m-specific provision there,
so no device run is scheduled and nothing happens. If the STREAMON ioctl
happens after queuing the request, then things should be fine (but
that's definitely not what we want userspace to be doing) since
the vb2_streamon call from the ioctl handler will take effect
and v4l2_m2m_try_schedule will be called.

The way that I have solved this with the Sunxi-Cedrus driver is to add a
req_complete callback function pointer (holding a call
to v4l2_m2m_try_schedule) in media_device_ops and call it (if available)
from media_request_ioctl_queue. I initially put this in req_queue
directly, but since it is wrapped by the request queue mutex lock and
provided that device_run needs to access the request queue, we need an
extra op called out of this lock, before completing the ioctl handler.

I will be sending v2 of my driver with preliminary patches to fix this
(perhaps I should also fix vim2m that way along the way).

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFCv11 PATCH 14/29] v4l2-ctrls: add core request support

[Hans Verkuil]

On 04/11/2018 11:43 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Integrate the request support. This adds the v4l2_ctrl_request_complete
>> and v4l2_ctrl_request_setup functions to complete a request and (as a
>> helper function) to apply a request to the hardware.
> 
> Please see my comments inline.
> 
> [snip]
>> +static int v4l2_ctrl_request_clone(struct v4l2_ctrl_handler *hdl,
>> +                                  const struct v4l2_ctrl_handler *from)
>> +{
>> +       struct v4l2_ctrl_ref *ref;
>> +       int err;
>> +
>> +       if (WARN_ON(!hdl || hdl == from))
>> +               return -EINVAL;
>> +
>> +       if (hdl->error)
>> +               return hdl->error;
>> +
>> +       WARN_ON(hdl->lock != &hdl->_lock);
>> +
>> +       mutex_lock(from->lock);
>> +       list_for_each_entry(ref, &from->ctrl_refs, node) {
>> +               struct v4l2_ctrl *ctrl = ref->ctrl;
>> +               struct v4l2_ctrl_ref *new_ref;
>> +
>> +               /* Skip refs inherited from other devices */
>> +               if (ref->from_other_dev)
>> +                       continue;
>> +               /* And buttons */
>> +               if (ctrl->type == V4L2_CTRL_TYPE_BUTTON)
>> +                       continue;
>> +               err = handler_new_ref(hdl, ctrl, &new_ref, false, true);
>> +               if (err) {
>> +                       printk("%s: handler_new_ref on control %x (%s)
> returned %d\n", __func__, ctrl->id, ctrl->name, err);
> 
> Perhaps pr_err()?
> 
>> +                       err = 0;
>> +                       continue;
> 
> Hmm, is it really fine to ignore the error and continue here?

It's a debugging left-over. It should just break and return the error.

> 
>> +               }
>> +               if (err)
>> +                       break;
>> +       }
>> +       mutex_unlock(from->lock);
>> +       return err;
>> +}
>> +
>> +static void v4l2_ctrl_request_queue(struct media_request_object *obj)
>> +{
>> +       struct v4l2_ctrl_handler *hdl =
>> +               container_of(obj, struct v4l2_ctrl_handler, req_obj);
>> +       struct v4l2_ctrl_handler *main_hdl = obj->priv;
>> +       struct v4l2_ctrl_handler *prev = NULL;
>> +       struct v4l2_ctrl_ref *ref_hdl, *ref_prev = NULL;
>> +
>> +       if (list_empty(&main_hdl->requests_queued))
>> +               goto queue;
>> +
>> +       prev = list_last_entry(&main_hdl->requests_queued,
>> +                              struct v4l2_ctrl_handler, requests_queued);
>> +       mutex_lock(prev->lock);
>> +       ref_prev = list_first_entry(&prev->ctrl_refs,
>> +                                   struct v4l2_ctrl_ref, node);
>> +       list_for_each_entry(ref_hdl, &hdl->ctrl_refs, node) {
>> +               if (ref_hdl->req)
>> +                       continue;
>> +               while (ref_prev->ctrl->id < ref_hdl->ctrl->id)
>> +                       ref_prev = list_next_entry(ref_prev, node);
> 
> Is this really safe? The only stop condition here is the control id.
> Perhaps the code below could be better?

The two ctrl_refs lists must contain the same controls, so it is safe.
However, I modified the code to check that ref_prev isn't the last
element in the list, just in case something gets messed up.

BTW, the variable names in this function are awful :-)

I've changed them so this code is hopefully easier to understand.

> 
> list_for_each_entry_from(ref_prev, &prev->ctrl_refs, node)
>          if (ref_prev->ctrl->id >= ref_hdl->ctrl->id)
>                  break;
> 
>> +               if (WARN_ON(ref_prev->ctrl->id != ref_hdl->ctrl->id))
>> +                       break;
>> +               ref_hdl->req = ref_prev->req;
>> +       }
>> +       mutex_unlock(prev->lock);
>> +queue:
>> +       list_add_tail(&hdl->requests_queued, &main_hdl->requests_queued);
>> +       hdl->request_is_queued = true;
>> +}
>> +
> 
> [snip]
>> +void v4l2_ctrl_request_complete(struct media_request *req,
>> +                               struct v4l2_ctrl_handler *main_hdl)
>> +{
>> +       struct media_request_object *obj;
>> +       struct v4l2_ctrl_handler *hdl;
>> +       struct v4l2_ctrl_ref *ref;
>> +
>> +       if (!req || !main_hdl)
>> +               return;
> 
> Can this happen normally? Perhaps WARN_ON() would make sense?

Yes, this can happen. See e.g. vim2m_stop_streaming(). This function is
called for all buffers, but not all buffers may have a request associated
with them. In that case req == NULL and this function should do nothing.

There are also cases (rare, but they exist) where a handler may be NULL
because the specific hardware version does not support the controls in that
handler. In that case the handler may never be created.

Basically this function means: if there is a request, and if there is a
handler, then complete the request.

> 
> [snip]
>> +void v4l2_ctrl_request_setup(struct media_request *req,
>> +                            struct v4l2_ctrl_handler *main_hdl)
>> +{
>> +       struct media_request_object *obj;
>> +       struct v4l2_ctrl_handler *hdl;
>> +       struct v4l2_ctrl_ref *ref;
>> +
>> +       if (!req || !main_hdl)
> 
> Can this happen normally? Perhaps WARN_ON() would make sense?

Same reason as above.

> 
>> +               return;
>> +
>> +       obj = media_request_object_find(req, &req_ops, main_hdl);
>> +       if (!obj)
>> +               return;
>> +       if (obj->completed) {
>> +               media_request_object_put(obj);
>> +               return;
>> +       }
>> +       hdl = container_of(obj, struct v4l2_ctrl_handler, req_obj);
>> +
>> +       mutex_lock(hdl->lock);
>> +
>> +       list_for_each_entry(ref, &hdl->ctrl_refs, node)
>> +               ref->done = false;
>> +
>> +       list_for_each_entry(ref, &hdl->ctrl_refs, node) {
>> +               struct v4l2_ctrl *ctrl = ref->ctrl;
>> +               struct v4l2_ctrl *master = ctrl->cluster[0];
>> +               bool have_new_data = false;
>> +               int i;
>> +
>> +               /* Skip if this control was already handled by a cluster.
> */
>> +               /* Skip button controls and read-only controls. */
>> +               if (ref->done || ctrl->type == V4L2_CTRL_TYPE_BUTTON ||
>> +                   (ctrl->flags & V4L2_CTRL_FLAG_READ_ONLY))
>> +                       continue;
>> +
>> +               v4l2_ctrl_lock(master);
>> +               for (i = 0; i < master->ncontrols; i++) {
>> +                       if (master->cluster[i]) {
>> +                               struct v4l2_ctrl_ref *r =
>> +                                       find_ref(hdl,
> master->cluster[i]->id);
>> +
>> +                               if (r->req && r == r->req) {
>> +                                       have_new_data = true;
>> +                                       break;
>> +                               }
>> +                       }
>> +               }
>> +               if (!have_new_data)
>> +                       continue;
> 
> No need to call v4l2_ctrl_unlock() here?
> 
> [snip]
>> @@ -263,6 +267,8 @@ struct v4l2_ctrl_ref {
>>          struct v4l2_ctrl *ctrl;
>>          struct v4l2_ctrl_helper *helper;
>>          bool from_other_dev;
>> +       bool done;
>> +       struct v4l2_ctrl_ref *req;
> 
> Perhaps it could make sense to call this ref_req, which would use the same
> convention as p_req below?

I'll have to think some more about this name. It is not descriptive enough.

Regards,

	Hans

> 
>>          union v4l2_ctrl_ptr p_req;
>>   };
> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Hans Verkuil]

On 04/11/2018 12:13 PM, Mauro Carvalho Chehab wrote:
> Em Tue, 10 Apr 2018 14:14:30 +0300
> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> 
>>>>  /**
>>>> @@ -88,6 +96,8 @@ struct media_device_ops {
>>>>   * @disable_source: Disable Source Handler function pointer
>>>>   *
>>>>   * @ops:	Operation handler callbacks
>>>> + * @req_lock:	Serialise access to requests
>>>> + * @req_queue_mutex: Serialise validating and queueing requests  
>>>
>>> IMHO, this would better describe it:
>>> 	Serialise validate and queue requests
>>>
>>> Yet, IMO, it doesn't let it clear when the spin lock should be
>>> used and when the mutex should be used.
>>>
>>> I mean, what of them protect what variable?  
>>
>> It might not be obvious, but the purpose of this mutex is to prevent
>> queueing multiple requests simultaneously in order to serialise access to
>> the top of the queue.
> 
> It is not obvious at all. The purpose of a lock is to prevent
> multiple acesses to the content of a data structure.
> 
> From what I see, here we have a single structure with two locks.
> To make it worse, you're introducing first the lock and then,
> on patch 04/29, the actual structs to be protected.
> 
> Well, a house with two doors is only closed if both doors are
> closed. The same happens with a data structure protected by
> two locks. It is only locked if both locks are locked.
> 
> So, every time I see two locks meant to protect the same struct, it
> sounds poorly designed or wrong. 
> 
> There's one exception, though: if you have an independent
> guest house and a main house, you could have two locks at the
> same place, one for the main house and another one for the
> guest house, but in this case, you should clearly tag with lock
> protects each house.
> 
> So, in this case, two new locks that are being proposed to be added
> at for struct media_device. So, I would be expecting a comment
> like:
> 
> 	@req_lock: serialize access to &struct media_request 

Urgh. This req_lock is a left-over from some older code. It's never
used and I've now deleted it. That explains the confusion. I thought you
were talking about the spinlock in struct media_request.

The req_queue_mutex is there to serialize queueing of requests, it
doesn't protect any data structures, it just ensures you can't queue
two requests in parallel for the same media device.

Regards,

	Hans


> 	@req_queue_mutex: serialize access to &struct media_request_object
> 
> Clearly stating what data structures each lock protects.
> 
> That was my concern when I pointed it. After looking at the entire
> patchset, what I saw was a non-consistent locking model.
> 
> It sounded to me that the original design to be:
> 
> 1) req_queue_mutex was designed to protect struct media_request
>    instances;
> 
> 2) req_lock was designed to protect just one field inside
>    struct media_request (the state field).
> 
> There is a big issue on that:
> 
> As state is part of media_request, assuming that the data struct
> won't disappear while in use (with is warranted by kref), before
> changing its value and touching other fields at req, the code should 
> be locking both req_queue_mutex and req_lock, but I didn't see that
> behavior on several places.
> 
> Also, I noticed several locking inconsistencies, as, on several places,
> the content of a media_request instance and/or its state was 
> accessed/altered without either locks.
> 
>> How about this instead:
>>
>> 	Serialise access to accessing device state on the tail of the
>> 	request queue.
> 
> It still doesn't mention what struct each of the new locks protect.
> IMHO, this patch should either be bound with patch 04/29 or come after
> that, and explicitly mention what data is protected by each lock.
> 
> Thanks,
> Mauro
> 

Re: [RFCv11 PATCH 17/29] vb2: store userspace data in vb2_v4l2_buffer

[Hans Verkuil]

On 04/09/2018 05:11 PM, Kieran Bingham wrote:
> Hi Hans,
> 
> Thank you for the patch series !
> 
> I'm looking forwards to finding some time to try out this work.
> 
> Just briefly scanning through the series, and I saw the minor issue below.
> 
> Regards
> 
> Kieran
> 
> 
> On 09/04/18 15:20, Hans Verkuil wrote:
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> The userspace-provided plane data needs to be stored in
>> vb2_v4l2_buffer. Currently this information is applied by
>> __fill_vb2_buffer() which is called by the core prepare_buf
>> and qbuf functions, but when using requests these functions
>> aren't called yet since the buffer won't be prepared until
>> the media request is actually queued.
>>
>> In the meantime this information has to be stored somewhere
>> and vb2_v4l2_buffer is a good place for it.
>>
>> The __fill_vb2_buffer callback now just copies the relevant
>> information from vb2_v4l2_buffer into the planes array.
>>
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/common/videobuf2/videobuf2-core.c |  25 +-
>>  drivers/media/common/videobuf2/videobuf2-v4l2.c | 324 +++++++++++++-----------
>>  drivers/media/dvb-core/dvb_vb2.c                |   3 +-
>>  include/media/videobuf2-core.h                  |   3 +-
>>  include/media/videobuf2-v4l2.h                  |   2 +
>>  5 files changed, 197 insertions(+), 160 deletions(-)
>>
>> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
>> index d3f7bb33a54d..3d436ccb61f8 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-core.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
>> @@ -968,9 +968,8 @@ static int __prepare_mmap(struct vb2_buffer *vb, const void *pb)
> 
> Now that pb is unused here, should it be removed from the function arguments ?

Correct. Dropped here and elsewhere.

Regards,

	Hans

Re: [RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Hans Verkuil]

On 04/10/2018 07:26 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Define the public request API.
> 
>> This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
>> and two ioctls that operate on a request in order to queue the
>> contents of the request to the driver and to re-initialize the
>> request.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   include/uapi/linux/media.h | 8 ++++++++
>>   1 file changed, 8 insertions(+)
> 
>> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
>> index c7e9a5cba24e..f8769e74f847 100644
>> --- a/include/uapi/linux/media.h
>> +++ b/include/uapi/linux/media.h
>> @@ -342,11 +342,19 @@ struct media_v2_topology {
> 
>>   /* ioctls */
> 
>> +struct __attribute__ ((packed)) media_request_alloc {
>> +       __s32 fd;
>> +};
>> +
>>   #define MEDIA_IOC_DEVICE_INFO  _IOWR('|', 0x00, struct media_device_info)
>>   #define MEDIA_IOC_ENUM_ENTITIES        _IOWR('|', 0x01, struct
> media_entity_desc)
>>   #define MEDIA_IOC_ENUM_LINKS   _IOWR('|', 0x02, struct media_links_enum)
>>   #define MEDIA_IOC_SETUP_LINK   _IOWR('|', 0x03, struct media_link_desc)
>>   #define MEDIA_IOC_G_TOPOLOGY   _IOWR('|', 0x04, struct media_v2_topology)
>> +#define MEDIA_IOC_REQUEST_ALLOC        _IOWR('|', 0x05, struct
> media_request_alloc)
>> +
>> +#define MEDIA_REQUEST_IOC_QUEUE                _IO('|',  0x80)
>> +#define MEDIA_REQUEST_IOC_REINIT       _IO('|',  0x81)
> 
> I wonder if it wouldn't make sense to add a comment here saying that these
> are called on request FD, as opposed to the others above, which are called
> on the media FD.

Added.

	Hans

> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Hans Verkuil]

On 04/10/2018 07:35 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> [snip]
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> new file mode 100644
>> index 000000000000..ead78613fdbe
>> --- /dev/null
>> +++ b/drivers/media/media-request.c
>> @@ -0,0 +1,23 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Media device request objects
>> + *
>> + * Copyright (C) 2018 Intel Corporation
>> + * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
> 
> I'm not sure about the origin of this line, but it's not a correct
> copyright for kernel code produced as a part of Chrome OS project. It would
> normally be something like
> 
> Copyright (C) 2018 Google, Inc.

Fixed.

> 
>> + *
>> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
>> + */
>> +
>> +#include <linux/anon_inodes.h>
>> +#include <linux/file.h>
>> +#include <linux/mm.h>
>> +#include <linux/string.h>
>> +
>> +#include <media/media-device.h>
>> +#include <media/media-request.h>
>> +
>> +int media_request_alloc(struct media_device *mdev,
>> +                       struct media_request_alloc *alloc)
>> +{
>> +       return -ENOMEM;
>> +}
>> diff --git a/include/media/media-device.h b/include/media/media-device.h
>> index bcc6ec434f1f..07e323c57202 100644
>> --- a/include/media/media-device.h
>> +++ b/include/media/media-device.h
>> @@ -19,6 +19,7 @@
>>   #ifndef _MEDIA_DEVICE_H
>>   #define _MEDIA_DEVICE_H
> 
>> +#include <linux/anon_inodes.h>
> 
> What is the need for anon_inodes in this header?
> 
>>   #include <linux/list.h>
>>   #include <linux/mutex.h>
> 
>> @@ -27,6 +28,7 @@
> 
>>   struct ida;
>>   struct device;
>> +struct media_device;
> 
>>   /**
>>    * struct media_entity_notify - Media Entity Notify
>> @@ -50,10 +52,16 @@ struct media_entity_notify {
>>    * struct media_device_ops - Media device operations
>>    * @link_notify: Link state change notification callback. This callback
> is
>>    *              called with the graph_mutex held.
>> + * @req_alloc: Allocate a request
>> + * @req_free: Free a request
>> + * @req_queue: Queue a request
>>    */
>>   struct media_device_ops {
>>          int (*link_notify)(struct media_link *link, u32 flags,
>>                             unsigned int notification);
>> +       struct media_request *(*req_alloc)(struct media_device *mdev);
>> +       void (*req_free)(struct media_request *req);
>> +       int (*req_queue)(struct media_request *req);
>>   };
> 
>>   /**
>> @@ -88,6 +96,8 @@ struct media_device_ops {
>>    * @disable_source: Disable Source Handler function pointer
>>    *
>>    * @ops:       Operation handler callbacks
>> + * @req_lock:  Serialise access to requests
>> + * @req_queue_mutex: Serialise validating and queueing requests
> 
> Let's bikeshed a bit! "access" sounds like a superset of "validating and
> queuing" to me. Perhaps it could make sense to be a bit more specific on
> what type of access the spinlock is used for?

req_lock is unused and is now deleted. It's a left-over from older code.

Regards,

	Hans

> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Hans Verkuil]

On 04/10/2018 10:21 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> [snip]
>> +static void media_request_clean(struct media_request *req)
>> +{
>> +       struct media_request_object *obj, *obj_safe;
>> +
>> +       WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
>> +
>> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
>> +               media_request_object_unbind(obj);
>> +               media_request_object_put(obj);
>> +       }
>> +
>> +       req->num_incomplete_objects = 0;
>> +       wake_up_interruptible(&req->poll_wait);
> 
> Do we intentionally wake up only one waiter here (as opposed to calling
> wake_up_interruptible_all())?

_all sounds reasonable. I wish there was better (any!) documentation for the wake_up
calls.

> 
>> +}
>> +
>> +static void media_request_release(struct kref *kref)
>> +{
>> +       struct media_request *req =
>> +               container_of(kref, struct media_request, kref);
>> +       struct media_device *mdev = req->mdev;
>> +       unsigned long flags;
>> +
>> +       dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       req->state = MEDIA_REQUEST_STATE_CLEANING;
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +
>> +       media_request_clean(req);
>> +
>> +       if (mdev->ops->req_free)
>> +               mdev->ops->req_free(req);
>> +       else
>> +               kfree(req);
>> +}
>> +
>> +void media_request_put(struct media_request *req)
>> +{
>> +       kref_put(&req->kref, media_request_release);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_put);
>> +
>> +void media_request_cancel(struct media_request *req)
>> +{
>> +       struct media_request_object *obj, *obj_safe;
>> +
>> +       if (req->state != MEDIA_REQUEST_STATE_QUEUED)
>> +               return;
> 
> I can see that media_request_release() guards the state change with
> req->lock. However we access the state here without holding the spinlock.
> 
> Also, should we perhaps have MEDIA_REQUEST_STATE_CANCELLED (or maybe just
> reset to MEDIA_REQUEST_STATE_IDLE?), so that we can guard against calling
> this multiple times?
> 
> Or perhaps we're expecting this to be called with req_queue_mutex held?

Hmm, I think that the core problem here is how to avoid a request being queued
while the queue is being canceled.

The best and probably only way to do that is to take the req_queue_mutex for
the streamon/off/_vb2_fop_release functions.

I've added locking for this to the v4l2 core and added a WARN_ON_ONCE check
to see if the req_queue_mutex is indeed locked when this function is called.

> 
>> +
>> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
>> +               if (obj->ops->cancel)
>> +                       obj->ops->cancel(obj);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_cancel);
> [snip]
>> +void media_request_object_init(struct media_request_object *obj)
>> +{
>> +       obj->ops = NULL;
>> +       obj->req = NULL;
>> +       obj->priv = NULL;
> 
> Perhaps it could make sense to pass ops and priv as arguments here? There
> is probably not much value in having a request object with both set to NULL.
> 
>> +       obj->completed = false;
>> +       INIT_LIST_HEAD(&obj->list);
>> +       kref_init(&obj->kref);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_init);
>> +
>> +void media_request_object_bind(struct media_request *req,
>> +                              const struct media_request_object_ops *ops,
>> +                              void *priv,
>> +                              struct media_request_object *obj)
>> +{
>> +       unsigned long flags;
>> +
>> +       if (WARN_ON(!ops->release || !ops->cancel))
>> +               return;
>> +
>> +       obj->req = req;
>> +       obj->ops = ops;
>> +       obj->priv = priv;
> 
> Ah, I see, they're being set here. I guess it's just the matter of how we
> define the semantics.
> 
> I wonder, though, what happens with a media_request_object struct that was
> allocated, initialized, but not bound to a request, because the client
> terminated. We would want to call media_request_object_put(obj), but
> obj->ops would be NULL.

You can only call media_request_object_put(obj) if obj->req != NULL.
I've added a WARN_ON to media_request_object_put() to check that.

> 
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
>> +               goto unlock;
>> +       list_add_tail(&obj->list, &req->objects);
>> +       req->num_incomplete_objects++;
>> +unlock:
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_bind);
>> +
>> +void media_request_object_unbind(struct media_request_object *obj)
>> +{
>> +       struct media_request *req = obj->req;
>> +       unsigned long flags;
>> +       bool completed = false;
>> +
>> +       if (!req)
>> +               return;
> 
> No need for any locking here?

No, but I did add a WARN_ON here. It really shouldn't be called if
req == NULL.

> 
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       list_del(&obj->list);
>> +       obj->req = NULL;
>> +
>> +       if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
>> +           req->state == MEDIA_REQUEST_STATE_CLEANING)
>> +               goto unlock;
>> +
>> +       if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
>> +               goto unlock;
>> +
>> +       if (WARN_ON(!req->num_incomplete_objects))
>> +               goto unlock;
>> +
>> +       req->num_incomplete_objects--;
>> +       if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
>> +           !req->num_incomplete_objects) {
>> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
>> +               completed = true;
>> +               wake_up_interruptible(&req->poll_wait);
> 
> Is this intentionally waking up only 1 waiter?
> 
>> +       }
>> +unlock:
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +       if (obj->ops->unbind)
>> +               obj->ops->unbind(obj);
>> +       if (completed)
>> +               media_request_put(req);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_unbind);
>> +
>> +void media_request_object_complete(struct media_request_object *obj)
>> +{
>> +       struct media_request *req = obj->req;
>> +       unsigned long flags;
>> +       bool completed = false;
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       if (obj->completed)
>> +               goto unlock;
>> +       obj->completed = true;
>> +       if (WARN_ON(!req->num_incomplete_objects) ||
>> +           WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
>> +               goto unlock;
>> +
>> +       if (!--req->num_incomplete_objects) {
>> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
>> +               wake_up_interruptible(&req->poll_wait);
> 
> Is this intentionally waking up only 1 waiter?

Changed them all to _all.

> 
>> +               completed = true;
>> +       }
>> +unlock:
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +       if (completed)
>> +               media_request_put(req);
>>   }
>> +EXPORT_SYMBOL_GPL(media_request_object_complete);
>> diff --git a/include/media/media-request.h b/include/media/media-request.h
>> index dae3eccd9aa7..082c3cae04ac 100644
>> --- a/include/media/media-request.h
>> +++ b/include/media/media-request.h
>> @@ -16,7 +16,163 @@
> 
>>   #include <media/media-device.h>
> 
>> +enum media_request_state {
>> +       MEDIA_REQUEST_STATE_IDLE,
>> +       MEDIA_REQUEST_STATE_QUEUEING,
>> +       MEDIA_REQUEST_STATE_QUEUED,
>> +       MEDIA_REQUEST_STATE_COMPLETE,
>> +       MEDIA_REQUEST_STATE_CLEANING,
>> +};
>> +
>> +struct media_request_object;
>> +
>> +/**
>> + * struct media_request - Media device request
>> + * @mdev: Media device this request belongs to
>> + * @kref: Reference count
>> + * @debug_prefix: Prefix for debug messages (process name:fd)
> 
> debug_str?

Hmm, I prefer debug_prefix.

> 
>> + * @state: The state of the request
>> + * @objects: List of @struct media_request_object request objects
>> + * @num_objects: The number objects in the request
>> + * @num_completed_objects: The number of completed objects in the request
> 
> num_incomplete_objects?
> 
>> + * @poll_wait: Wait queue for poll
>> + * @lock: Serializes access to this struct
> 
> Does it have to be acquired to access any field?

Used for state, objects, num_incomplete_objects, so pretty much all, yes.
The other fields are either atomic, have their own lock or do not change.

> 
>> + */
>> +struct media_request {
>> +       struct media_device *mdev;
>> +       struct kref kref;
>> +       char debug_str[TASK_COMM_LEN + 11];
>> +       enum media_request_state state;
>> +       struct list_head objects;
>> +       unsigned int num_incomplete_objects;
>> +       struct wait_queue_head poll_wait;
>> +       spinlock_t lock;
>> +};
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +
>> +static inline void media_request_get(struct media_request *req)
>> +{
>> +       kref_get(&req->kref);
>> +}
>> +
>> +void media_request_put(struct media_request *req);
>> +void media_request_cancel(struct media_request *req);
>> +
>>   int media_request_alloc(struct media_device *mdev,
>>                          struct media_request_alloc *alloc);
>> +#else
>> +static inline void media_request_get(struct media_request *req)
>> +{
>> +}
>> +
>> +static inline void media_request_put(struct media_request *req)
>> +{
>> +}
>> +
>> +static inline void media_request_cancel(struct media_request *req)
>> +{
>> +}
>> +
>> +#endif
>> +
> 
> Documentation. (Disclaimer from cover letter acknowledged, though. Just
> marking.)
> 
>> +struct media_request_object_ops {
>> +       int (*prepare)(struct media_request_object *object);
>> +       void (*unprepare)(struct media_request_object *object);
>> +       void (*queue)(struct media_request_object *object);
>> +       void (*unbind)(struct media_request_object *object);
>> +       void (*cancel)(struct media_request_object *object);
>> +       void (*release)(struct media_request_object *object);
>> +};
>> +
>> +/**
>> + * struct media_request_object - An opaque object that belongs to a media
>> + *                              request
>> + *
> 
> ops
> 
>> + * @priv: object's priv pointer
> 
> req
> 
>> + * @list: List entry of the object for @struct media_request
>> + * @kref: Reference count of the object, acquire before releasing
> req->lock
> 
> completed
> 
>> + *
>> + * An object related to the request. This struct is embedded in the
>> + * larger object data.
>> + */
>> +struct media_request_object {
>> +       const struct media_request_object_ops *ops;
>> +       void *priv;
>> +       struct media_request *req;
>> +       struct list_head list;
>> +       struct kref kref;
>> +       bool completed;
>> +};
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +static inline void media_request_object_get(struct media_request_object
> *obj)
>> +{
>> +       kref_get(&obj->kref);
>> +}
>> +
>> +/**
>> + * media_request_object_put - Put a media request object
>> + *
>> + * @obj: The object
>> + *
>> + * Put a media request object. Once all references are gone, the
>> + * object's memory is released.
>> + */
>> +void media_request_object_put(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_init - Initialise a media request object
>> + *
>> + * Initialise a media request object. The object will be released using
> the
>> + * release callback of the ops once it has no references (this function
>> + * initialises references to one).
>> + */
>> +void media_request_object_init(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_bind - Bind a media request object to a request
>> + */
>> +void media_request_object_bind(struct media_request *req,
>> +                              const struct media_request_object_ops *ops,
>> +                              void *priv,
>> +                              struct media_request_object *obj);
>> +
>> +void media_request_object_unbind(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_complete - Mark the media request object as
> complete
>> + */
>> +void media_request_object_complete(struct media_request_object *obj);
>> +#else
>> +static inline void media_request_object_get(struct media_request_object
> *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_put(struct media_request_object
> *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_init(struct media_request_object
> *obj)
>> +{
>> +       obj->ops = NULL;
>> +       obj->req = NULL;
> 
> Hmm. Do we need to do anything here if !defined(CONFIG_MEDIA_CONTROLLER)?

The object was likely zeroed when this function is called. But it doesn't hurt
to make sure.

Regards,

	Hans

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Hans Verkuil]

On 04/10/2018 10:59 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
>> ioctls.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/media-request.c | 80
> +++++++++++++++++++++++++++++++++++++++++--
>>   1 file changed, 78 insertions(+), 2 deletions(-)
> 
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> index dffc290e4ada..27739ff7cb09 100644
>> --- a/drivers/media/media-request.c
>> +++ b/drivers/media/media-request.c
>> @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file
> *filp,
>>          return 0;
>>   }
> 
>> +static long media_request_ioctl_queue(struct media_request *req)
>> +{
>> +       struct media_device *mdev = req->mdev;
>> +       unsigned long flags;
>> +       int ret = 0;
>> +
>> +       dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       if (req->state != MEDIA_REQUEST_STATE_IDLE) {
>> +               dev_dbg(mdev->dev,
>> +                       "request: unable to queue %s, request in state
> %s\n",
>> +                       req->debug_str,
> media_request_state_str(req->state));
>> +               spin_unlock_irqrestore(&req->lock, flags);
>> +               return -EINVAL;
> 
> nit: Perhaps -EBUSY? (vb2 returns -EINVAL, though, but IMHO it doesn't
> really represent the real error too closely.)

I agree, also in reinit.

> 
>> +       }
>> +       req->state = MEDIA_REQUEST_STATE_QUEUEING;
>> +
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +
>> +       /*
>> +        * Ensure the request that is validated will be the one that gets
> queued
>> +        * next by serialising the queueing process.
>> +        */
>> +       mutex_lock(&mdev->req_queue_mutex);
>> +
>> +       ret = mdev->ops->req_queue(req);
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       req->state = ret ? MEDIA_REQUEST_STATE_IDLE :
> MEDIA_REQUEST_STATE_QUEUED;
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +       mutex_unlock(&mdev->req_queue_mutex);
>> +
>> +       if (ret) {
>> +               dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
>> +                       req->debug_str, ret);
>> +       } else {
>> +               media_request_get(req);
> 
> I'm not convinced that this is the right place to take a reference. IMHO
> whoever saves a pointer to the request in its own internal data (the
> ->req_queue() callback?), should also grab a reference before doing so. Not
> a strong objection, though, if we clearly document this, so that whoever
> implements ->req_queue() callback can do the right thing.
No, it belongs here. The reason is that request_put is also called here when
the request is completed. And it makes no sense to move the put to drivers as
well since it is only called when the whole request is completed.

Regards,

	Hans

> 
>> +       }
>> +
>> +       return ret;
>> +}
>> +
>> +static long media_request_ioctl_reinit(struct media_request *req)
>> +{
>> +       struct media_device *mdev = req->mdev;
>> +       unsigned long flags;
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       if (req->state != MEDIA_REQUEST_STATE_IDLE &&
>> +           req->state != MEDIA_REQUEST_STATE_COMPLETE) {
>> +               dev_dbg(mdev->dev,
>> +                       "request: %s not in idle or complete state,
> cannot reinit\n",
>> +                       req->debug_str);
>> +               spin_unlock_irqrestore(&req->lock, flags);
>> +               return -EINVAL;
> 
> nit: Perhaps -EBUSY? (Again vb2 would return -EINVAL...)
> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 12/29] v4l2-ctrls: alloc memory for p_req

[Hans Verkuil]

On 04/10/2018 11:32 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> To store request data the handler_new_ref() allocates memory
>> for it if needed.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/v4l2-core/v4l2-ctrls.c | 20 ++++++++++++++++----
>>   1 file changed, 16 insertions(+), 4 deletions(-)
> 
>> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
>> index d09f49530d9e..3c1b00baa8d0 100644
>> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
>> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
>> @@ -1997,13 +1997,18 @@ EXPORT_SYMBOL(v4l2_ctrl_find);
>>   /* Allocate a new v4l2_ctrl_ref and hook it into the handler. */
>>   static int handler_new_ref(struct v4l2_ctrl_handler *hdl,
>>                             struct v4l2_ctrl *ctrl,
>> -                          bool from_other_dev)
>> +                          struct v4l2_ctrl_ref **ctrl_ref,
>> +                          bool from_other_dev, bool allocate_req)
>>   {
>>          struct v4l2_ctrl_ref *ref;
>>          struct v4l2_ctrl_ref *new_ref;
>>          u32 id = ctrl->id;
>>          u32 class_ctrl = V4L2_CTRL_ID2WHICH(id) | 1;
>>          int bucket = id % hdl->nr_of_buckets;   /* which bucket to use */
>> +       unsigned int sz_extra = 0;
>> +
>> +       if (ctrl_ref)
>> +               *ctrl_ref = NULL;
> 
>>          /*
>>           * Automatically add the control class if it is not yet present
> and
>> @@ -2017,11 +2022,16 @@ static int handler_new_ref(struct
> v4l2_ctrl_handler *hdl,
>>          if (hdl->error)
>>                  return hdl->error;
> 
>> -       new_ref = kzalloc(sizeof(*new_ref), GFP_KERNEL);
>> +       if (allocate_req)
>> +               sz_extra = ctrl->elems * ctrl->elem_size;
>> +       new_ref = kzalloc(sizeof(*new_ref) + sz_extra, GFP_KERNEL);
> 
> I think we might want to use kvzalloc() here to cover allocation of big
> elements and/or large arrays, without order>0 allocations killing the
> systems.

I don't want to touch that now. Perhaps something for the future.

> 
> I'm actually also wondering if it wouldn't be cleaner to just allocate the
> extra data separately, since we store a pointer in new_ref->p_req.p anyway.

It would give too much overhead since sz_extra is almost always 4. Controls that
need more space are very rare. Calling kzalloc again for just 4 bytes is overkill.

Regards,

	Hans

> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Hans Verkuil]

On 04/10/2018 01:00 PM, Sakari Ailus wrote:
> On Tue, Apr 10, 2018 at 06:38:56AM -0300, Mauro Carvalho Chehab wrote:
>> Em Mon,  9 Apr 2018 16:19:59 +0200
>> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
>>
>>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>>
>>> Define the public request API.
>>>
>>> This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
>>> and two ioctls that operate on a request in order to queue the
>>> contents of the request to the driver and to re-initialize the
>>> request.
>>>
>>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>>> ---
>>>  include/uapi/linux/media.h | 8 ++++++++
>>>  1 file changed, 8 insertions(+)
>>>
>>> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
>>> index c7e9a5cba24e..f8769e74f847 100644
>>> --- a/include/uapi/linux/media.h
>>> +++ b/include/uapi/linux/media.h
>>> @@ -342,11 +342,19 @@ struct media_v2_topology {
>>>  
>>>  /* ioctls */
>>>  
>>> +struct __attribute__ ((packed)) media_request_alloc {
>>> +	__s32 fd;
>>> +};
>>> +
>>>  #define MEDIA_IOC_DEVICE_INFO	_IOWR('|', 0x00, struct media_device_info)
>>>  #define MEDIA_IOC_ENUM_ENTITIES	_IOWR('|', 0x01, struct media_entity_desc)
>>>  #define MEDIA_IOC_ENUM_LINKS	_IOWR('|', 0x02, struct media_links_enum)
>>>  #define MEDIA_IOC_SETUP_LINK	_IOWR('|', 0x03, struct media_link_desc)
>>>  #define MEDIA_IOC_G_TOPOLOGY	_IOWR('|', 0x04, struct media_v2_topology)
>>> +#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, struct media_request_alloc)
>>> +
>>
>> Why use a struct here? Just declare it as:
>>
>> 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> 
> I'd say it's easier to extend it if it's a struct. All other IOCTLs also
> have a struct as an argument. As a struct member, the parameter (fd) also
> has a name; this is a plus.

While I do not have a very strong opinion on this, I do agree with Sakari here.

Regards,

	Hans

> 
>>
>>> +#define MEDIA_REQUEST_IOC_QUEUE		_IO('|',  0x80)
>>> +#define MEDIA_REQUEST_IOC_REINIT	_IO('|',  0x81)
>>>  
>>>  #if !defined(__KERNEL__) || defined(__NEED_MEDIA_LEGACY_API)
>>>  
>>
>> Thanks,
>> Mauro
> 

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Hans Verkuil]

On 04/10/2018 11:52 AM, Mauro Carvalho Chehab wrote:
> Em Mon,  9 Apr 2018 16:20:00 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> Add support for allocating a new request. This is only supported
>> if mdev->ops->req_queue is set, i.e. the driver indicates that it
>> supports queueing requests.
>>
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/Makefile        |  3 ++-
>>  drivers/media/media-device.c  | 14 ++++++++++++++
>>  drivers/media/media-request.c | 23 +++++++++++++++++++++++
>>  include/media/media-device.h  | 13 +++++++++++++
>>  include/media/media-request.h | 22 ++++++++++++++++++++++
>>  5 files changed, 74 insertions(+), 1 deletion(-)
>>  create mode 100644 drivers/media/media-request.c
>>  create mode 100644 include/media/media-request.h
>>
>> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
>> index 594b462ddf0e..985d35ec6b29 100644
>> --- a/drivers/media/Makefile
>> +++ b/drivers/media/Makefile
>> @@ -3,7 +3,8 @@
>>  # Makefile for the kernel multimedia device drivers.
>>  #
>>  
>> -media-objs	:= media-device.o media-devnode.o media-entity.o
>> +media-objs	:= media-device.o media-devnode.o media-entity.o \
>> +		   media-request.o
>>  
>>  #
>>  # I2C drivers should come before other drivers, otherwise they'll fail
>> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
>> index 35e81f7c0d2f..acb583c0eacd 100644
>> --- a/drivers/media/media-device.c
>> +++ b/drivers/media/media-device.c
>> @@ -32,6 +32,7 @@
>>  #include <media/media-device.h>
>>  #include <media/media-devnode.h>
>>  #include <media/media-entity.h>
>> +#include <media/media-request.h>
>>  
>>  #ifdef CONFIG_MEDIA_CONTROLLER
>>  
>> @@ -366,6 +367,15 @@ static long media_device_get_topology(struct media_device *mdev,
>>  	return ret;
>>  }
>>  
>> +static long media_device_request_alloc(struct media_device *mdev,
>> +				       struct media_request_alloc *alloc)
>> +{
>> +	if (!mdev->ops || !mdev->ops->req_queue)
>> +		return -ENOTTY;
>> +
>> +	return media_request_alloc(mdev, alloc);
>> +}
>> +
>>  static long copy_arg_from_user(void *karg, void __user *uarg, unsigned int cmd)
>>  {
>>  	/* All media IOCTLs are _IOWR() */
>> @@ -414,6 +424,7 @@ static const struct media_ioctl_info ioctl_info[] = {
>>  	MEDIA_IOC(ENUM_LINKS, media_device_enum_links, MEDIA_IOC_FL_GRAPH_MUTEX),
>>  	MEDIA_IOC(SETUP_LINK, media_device_setup_link, MEDIA_IOC_FL_GRAPH_MUTEX),
>>  	MEDIA_IOC(G_TOPOLOGY, media_device_get_topology, MEDIA_IOC_FL_GRAPH_MUTEX),
>> +	MEDIA_IOC(REQUEST_ALLOC, media_device_request_alloc, 0),
>>  };
>>  
>>  static long media_device_ioctl(struct file *filp, unsigned int cmd,
>> @@ -686,6 +697,9 @@ void media_device_init(struct media_device *mdev)
>>  	INIT_LIST_HEAD(&mdev->pads);
>>  	INIT_LIST_HEAD(&mdev->links);
>>  	INIT_LIST_HEAD(&mdev->entity_notify);
>> +
>> +	spin_lock_init(&mdev->req_lock);
>> +	mutex_init(&mdev->req_queue_mutex);
>>  	mutex_init(&mdev->graph_mutex);
>>  	ida_init(&mdev->entity_internal_idx);
>>  
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> new file mode 100644
>> index 000000000000..ead78613fdbe
>> --- /dev/null
>> +++ b/drivers/media/media-request.c
>> @@ -0,0 +1,23 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Media device request objects
>> + *
>> + * Copyright (C) 2018 Intel Corporation
>> + * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
>> + *
>> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
>> + */
>> +
>> +#include <linux/anon_inodes.h>
> 
> Not needed. You already included it at media_device.h.

Actually, it is needed here, but not in media_device.h

> 
>> +#include <linux/file.h>
>> +#include <linux/mm.h>
>> +#include <linux/string.h>
> 
> Why do you need so many includes for a stub function?

I'm going to merge patch 3 and 4. Splitting it up it too confusing for
reviewers, I've noticed. These headers are needed for patch 4.

As mentioned in an earlier reply: the req_lock spinlock is unused and is
now deleted. It was a left-over from older versions and that's what caused
the locking confusion.

Regards,

	Hans


> 
>> +
>> +#include <media/media-device.h>
>> +#include <media/media-request.h>
>> +
>> +int media_request_alloc(struct media_device *mdev,
>> +			struct media_request_alloc *alloc)
>> +{
>> +	return -ENOMEM;
>> +}
>> diff --git a/include/media/media-device.h b/include/media/media-device.h
>> index bcc6ec434f1f..07e323c57202 100644
>> --- a/include/media/media-device.h
>> +++ b/include/media/media-device.h
>> @@ -19,6 +19,7 @@
>>  #ifndef _MEDIA_DEVICE_H
>>  #define _MEDIA_DEVICE_H
>>  
>> +#include <linux/anon_inodes.h>
> 
> Why do you need it? I don't see anything below needing it.
> 
>>  #include <linux/list.h>
>>  #include <linux/mutex.h>
>>  
>> @@ -27,6 +28,7 @@
>>  
>>  struct ida;
>>  struct device;
>> +struct media_device;
>>  
>>  /**
>>   * struct media_entity_notify - Media Entity Notify
>> @@ -50,10 +52,16 @@ struct media_entity_notify {
>>   * struct media_device_ops - Media device operations
>>   * @link_notify: Link state change notification callback. This callback is
>>   *		 called with the graph_mutex held.
>> + * @req_alloc: Allocate a request
>> + * @req_free: Free a request
>> + * @req_queue: Queue a request
>>   */
>>  struct media_device_ops {
>>  	int (*link_notify)(struct media_link *link, u32 flags,
>>  			   unsigned int notification);
>> +	struct media_request *(*req_alloc)(struct media_device *mdev);
>> +	void (*req_free)(struct media_request *req);
>> +	int (*req_queue)(struct media_request *req);
>>  };
>>  
>>  /**
>> @@ -88,6 +96,8 @@ struct media_device_ops {
>>   * @disable_source: Disable Source Handler function pointer
>>   *
>>   * @ops:	Operation handler callbacks
>> + * @req_lock:	Serialise access to requests
>> + * @req_queue_mutex: Serialise validating and queueing requests
> 
> IMHO, this would better describe it:
> 	Serialise validate and queue requests
> 
> Yet, IMO, it doesn't let it clear when the spin lock should be
> used and when the mutex should be used.
> 
> I mean, what of them protect what variable?
> 
>>   *
>>   * This structure represents an abstract high-level media device. It allows easy
>>   * access to entities and provides basic media device-level support. The
>> @@ -158,6 +168,9 @@ struct media_device {
>>  	void (*disable_source)(struct media_entity *entity);
>>  
>>  	const struct media_device_ops *ops;
>> +
>> +	spinlock_t req_lock;
>> +	struct mutex req_queue_mutex;
>>  };
>>  
>>  /* We don't need to include pci.h or usb.h here */
>> diff --git a/include/media/media-request.h b/include/media/media-request.h
>> new file mode 100644
>> index 000000000000..dae3eccd9aa7
>> --- /dev/null
>> +++ b/include/media/media-request.h
>> @@ -0,0 +1,22 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Media device request objects
>> + *
>> + * Copyright (C) 2018 Intel Corporation
>> + *
>> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
>> + */
>> +
>> +#ifndef MEDIA_REQUEST_H
>> +#define MEDIA_REQUEST_H
>> +
>> +#include <linux/list.h>
>> +#include <linux/slab.h>
>> +#include <linux/spinlock.h>
> 
> Why those includes are needed?
> 
>> +
>> +#include <media/media-device.h>
>> +
>> +int media_request_alloc(struct media_device *mdev,
>> +			struct media_request_alloc *alloc);
>> +
>> +#endif
> 
> 
> 
> Thanks,
> Mauro
> 

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Hans Verkuil]

On 04/23/2018 01:24 PM, Hans Verkuil wrote:
>>> +/**
>>> + * struct media_request - Media device request
>>> + * @mdev: Media device this request belongs to
>>> + * @kref: Reference count
>>> + * @debug_prefix: Prefix for debug messages (process name:fd)
>>
>> debug_str?
> 
> Hmm, I prefer debug_prefix.

Apologies, I misunderstood this.

Indeed, it should be debug_str.

Regards,

	Hans

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Hans Verkuil]

On 04/10/2018 12:32 PM, Mauro Carvalho Chehab wrote:
> Em Mon,  9 Apr 2018 16:20:01 +0200
> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> Implement the core of the media request processing.
>>
>> Drivers can bind request objects to a request. These objects
>> can then be marked completed if the driver finished using them,
>> or just be unbound if the results do not need to be kept (e.g.
>> in the case of buffers).
>>
>> Once all objects that were added are either unbound or completed,
>> the request is marked 'complete' and a POLLPRI signal is sent
>> via poll.
>>
>> Both requests and request objects are refcounted.
>>
>> While a request is queued its refcount is incremented (since it
>> is in use by a driver). Once it is completed the refcount is
>> decremented. When the user closes the request file descriptor
>> the refcount is also decremented. Once it reaches 0 all request
>> objects in the request are unbound and put() and the request
>> itself is freed.
>>
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/media-request.c | 284 +++++++++++++++++++++++++++++++++++++++++-
>>  include/media/media-request.h | 156 +++++++++++++++++++++++
>>  2 files changed, 439 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> index ead78613fdbe..dffc290e4ada 100644
>> --- a/drivers/media/media-request.c
>> +++ b/drivers/media/media-request.c
>> @@ -16,8 +16,290 @@
>>  #include <media/media-device.h>
>>  #include <media/media-request.h>
>>  
>> +static const char * const request_state[] = {
>> +	"idle",
>> +	"queueing",
>> +	"queued",
>> +	"complete",
>> +	"cleaning",
>> +};
> 
> Please use c99 designated inits here, e. g.:
> 
> static const char * const request_state[] = {
> 	[MEDIA_REQUEST_STATE_IDLE] = "idle",
> 	...
> };

Done.

> 
>> +
>> +static const char *
>> +media_request_state_str(enum media_request_state state)
>> +{
>> +	if (WARN_ON(state >= ARRAY_SIZE(request_state)))
>> +		return "unknown";
>> +	return request_state[state];
>> +}
>> +
>> +static void media_request_clean(struct media_request *req)
>> +{
>> +	struct media_request_object *obj, *obj_safe;
>> +
>> +	WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
>> +
>> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
>> +		media_request_object_unbind(obj);
>> +		media_request_object_put(obj);
>> +	}
>> +
>> +	req->num_incomplete_objects = 0;
>> +	wake_up_interruptible(&req->poll_wait);
>> +}
>> +
>> +static void media_request_release(struct kref *kref)
>> +{
>> +	struct media_request *req =
>> +		container_of(kref, struct media_request, kref);
>> +	struct media_device *mdev = req->mdev;
>> +	unsigned long flags;
>> +
>> +	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
>> +
>> +	spin_lock_irqsave(&req->lock, flags);
>> +	req->state = MEDIA_REQUEST_STATE_CLEANING;
>> +	spin_unlock_irqrestore(&req->lock, flags);
>> +
>> +	media_request_clean(req);
>> +
>> +	if (mdev->ops->req_free)
>> +		mdev->ops->req_free(req);
>> +	else
>> +		kfree(req);
> 
> Without looking at req_free() implementation, I would actually prefer
> to always do a kfree(req) here. So, a req_free() function would only
> free "extra" allocations, and not the request itself. e. g.:
> 
> ...
> 	if (mdev->ops->req_free)
> 		mdev->ops->req_free(req);
> 
> 	kfree(req);
> }

That would be inconsistent with req_alloc: that allocates
the request and any other driver-specific data.

I prefer to keep it as-is.

Note: this might well change later once I have a better understanding of
how more complex drivers are going to use this.


> 
> 
>> +}
>> +
>> +void media_request_put(struct media_request *req)
>> +{
>> +	kref_put(&req->kref, media_request_release);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_put);
>> +
>> +void media_request_cancel(struct media_request *req)
>> +{
>> +	struct media_request_object *obj, *obj_safe;
>> +
>> +	if (req->state != MEDIA_REQUEST_STATE_QUEUED)
>> +		return;
>> +
>> +	list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
>> +		if (obj->ops->cancel)
>> +			obj->ops->cancel(obj);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_cancel);
>> +
>> +static int media_request_close(struct inode *inode, struct file *filp)
>> +{
>> +	struct media_request *req = filp->private_data;
>> +
>> +	media_request_put(req);
>> +	return 0;
>> +}
>> +
>> +static unsigned int media_request_poll(struct file *filp,
>> +				       struct poll_table_struct *wait)
>> +{
>> +	struct media_request *req = filp->private_data;
>> +	unsigned long flags;
>> +	enum media_request_state state;
>> +
>> +	if (!(poll_requested_events(wait) & POLLPRI))
>> +		return 0;
>> +
>> +	spin_lock_irqsave(&req->lock, flags);
>> +	state = req->state;
>> +	spin_unlock_irqrestore(&req->lock, flags);
> 
> IMO, it would be better to use an atomic var for state, having a
> lockless access to it.

In most cases I need to do more than just change the state. I don't see
enough benefits from using an atomic.

> 
>> +
>> +	if (state == MEDIA_REQUEST_STATE_COMPLETE)
>> +		return POLLPRI;
>> +	if (state == MEDIA_REQUEST_STATE_IDLE)
>> +		return POLLERR;
>> +
>> +	poll_wait(filp, &req->poll_wait, wait);
>> +	return 0;
>> +}
>> +
>> +static long media_request_ioctl(struct file *filp, unsigned int cmd,
>> +				unsigned long __arg)
>> +{
>> +	return -ENOIOCTLCMD;
>> +}
>> +
>> +static const struct file_operations request_fops = {
>> +	.owner = THIS_MODULE,
>> +	.poll = media_request_poll,
>> +	.unlocked_ioctl = media_request_ioctl,
>> +	.release = media_request_close,
>> +};
>> +
>>  int media_request_alloc(struct media_device *mdev,
>>  			struct media_request_alloc *alloc)
>>  {
>> -	return -ENOMEM;
>> +	struct media_request *req;
>> +	struct file *filp;
>> +	char comm[TASK_COMM_LEN];
>> +	int fd;
>> +	int ret;
>> +
>> +	fd = get_unused_fd_flags(O_CLOEXEC);
>> +	if (fd < 0)
>> +		return fd;
>> +
>> +	filp = anon_inode_getfile("request", &request_fops, NULL, O_CLOEXEC);
>> +	if (IS_ERR(filp)) {
>> +		ret = PTR_ERR(filp);
>> +		goto err_put_fd;
>> +	}
>> +
>> +	if (mdev->ops->req_alloc)
>> +		req = mdev->ops->req_alloc(mdev);
>> +	else
>> +		req = kzalloc(sizeof(*req), GFP_KERNEL);
>> +	if (!req) {
>> +		ret = -ENOMEM;
>> +		goto err_fput;
>> +	}
>> +
>> +	filp->private_data = req;
>> +	req->mdev = mdev;
>> +	req->state = MEDIA_REQUEST_STATE_IDLE;
>> +	req->num_incomplete_objects = 0;
>> +	kref_init(&req->kref);
>> +	INIT_LIST_HEAD(&req->objects);
>> +	spin_lock_init(&req->lock);
>> +	init_waitqueue_head(&req->poll_wait);
>> +
>> +	alloc->fd = fd;
> 
> Btw, this is a very good reason why you should define the ioctl to
> have an integer argument instead of a struct with a __s32 field
> on it, as per my comment to patch 02/29:
> 
> 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> 
> At 64 bit architectures, you're truncating the file descriptor!

What does this have to do with struct vs int?

In any case, fd's fit in 31 bits.

> 
>> +	get_task_comm(comm, current);
>> +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
>> +		 comm, fd);
> 
> Not sure if it is a good idea to store the task that allocated
> the request. While it makes sense for the dev_dbg() below, it
> may not make sense anymore on other dev_dbg() you would be
> using it.

This is actually copied from Laurent's code. I'm not sure either. I don't
have enough experience with this yet to tell whether or not this is useful.

> 
>> +	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
>> +
>> +	fd_install(fd, filp);
>> +
>> +	return 0;
>> +
>> +err_fput:
>> +	fput(filp);
>> +
>> +err_put_fd:
>> +	put_unused_fd(fd);
>> +
>> +	return ret;
>> +}
>> +
>> +static void media_request_object_release(struct kref *kref)
>> +{
>> +	struct media_request_object *obj =
>> +		container_of(kref, struct media_request_object, kref);
>> +	struct media_request *req = obj->req;
>> +
>> +	if (req)
>> +		media_request_object_unbind(obj);
>> +	obj->ops->release(obj);
>> +}
>> +
>> +void media_request_object_put(struct media_request_object *obj)
>> +{
>> +	kref_put(&obj->kref, media_request_object_release);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_put);
>> +
>> +void media_request_object_init(struct media_request_object *obj)
>> +{
>> +	obj->ops = NULL;
>> +	obj->req = NULL;
>> +	obj->priv = NULL;
>> +	obj->completed = false;
>> +	INIT_LIST_HEAD(&obj->list);
>> +	kref_init(&obj->kref);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_init);
>> +
>> +void media_request_object_bind(struct media_request *req,
>> +			       const struct media_request_object_ops *ops,
>> +			       void *priv,
>> +			       struct media_request_object *obj)
>> +{
>> +	unsigned long flags;
>> +
>> +	if (WARN_ON(!ops->release || !ops->cancel))
>> +		return;
>> +
>> +	obj->req = req;
>> +	obj->ops = ops;
>> +	obj->priv = priv;
>> +	spin_lock_irqsave(&req->lock, flags);
>> +	if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
>> +		goto unlock;
>> +	list_add_tail(&obj->list, &req->objects);
>> +	req->num_incomplete_objects++;
>> +unlock:
>> +	spin_unlock_irqrestore(&req->lock, flags);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_bind);
>> +
>> +void media_request_object_unbind(struct media_request_object *obj)
>> +{
>> +	struct media_request *req = obj->req;
>> +	unsigned long flags;
>> +	bool completed = false;
>> +
>> +	if (!req)
>> +		return;
>> +
>> +	spin_lock_irqsave(&req->lock, flags);
>> +	list_del(&obj->list);
>> +	obj->req = NULL;
>> +
>> +	if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
>> +	    req->state == MEDIA_REQUEST_STATE_CLEANING)
>> +		goto unlock;
>> +
>> +	if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
>> +		goto unlock;
>> +
>> +	if (WARN_ON(!req->num_incomplete_objects))
>> +		goto unlock;
>> +
>> +	req->num_incomplete_objects--;
>> +	if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
>> +	    !req->num_incomplete_objects) {
>> +		req->state = MEDIA_REQUEST_STATE_COMPLETE;
>> +		completed = true;
>> +		wake_up_interruptible(&req->poll_wait);
>> +	}
>> +unlock:
>> +	spin_unlock_irqrestore(&req->lock, flags);
>> +	if (obj->ops->unbind)
>> +		obj->ops->unbind(obj);
>> +	if (completed)
>> +		media_request_put(req);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_unbind);
>> +
>> +void media_request_object_complete(struct media_request_object *obj)
>> +{
>> +	struct media_request *req = obj->req;
>> +	unsigned long flags;
>> +	bool completed = false;
>> +
>> +	spin_lock_irqsave(&req->lock, flags);
>> +	if (obj->completed)
>> +		goto unlock;
>> +	obj->completed = true;
>> +	if (WARN_ON(!req->num_incomplete_objects) ||
>> +	    WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
>> +		goto unlock;
>> +
>> +	if (!--req->num_incomplete_objects) {
>> +		req->state = MEDIA_REQUEST_STATE_COMPLETE;
>> +		wake_up_interruptible(&req->poll_wait);
>> +		completed = true;
>> +	}
>> +unlock:
>> +	spin_unlock_irqrestore(&req->lock, flags);
>> +	if (completed)
>> +		media_request_put(req);
>>  }
>> +EXPORT_SYMBOL_GPL(media_request_object_complete);
>> diff --git a/include/media/media-request.h b/include/media/media-request.h
>> index dae3eccd9aa7..082c3cae04ac 100644
>> --- a/include/media/media-request.h
>> +++ b/include/media/media-request.h
>> @@ -16,7 +16,163 @@
>>  
>>  #include <media/media-device.h>
>>  
>> +enum media_request_state {
>> +	MEDIA_REQUEST_STATE_IDLE,
>> +	MEDIA_REQUEST_STATE_QUEUEING,
>> +	MEDIA_REQUEST_STATE_QUEUED,
>> +	MEDIA_REQUEST_STATE_COMPLETE,
>> +	MEDIA_REQUEST_STATE_CLEANING,
>> +};
>> +
>> +struct media_request_object;
>> +
>> +/**
>> + * struct media_request - Media device request
>> + * @mdev: Media device this request belongs to
>> + * @kref: Reference count
>> + * @debug_prefix: Prefix for debug messages (process name:fd)
>> + * @state: The state of the request
>> + * @objects: List of @struct media_request_object request objects
>> + * @num_objects: The number objects in the request
>> + * @num_completed_objects: The number of completed objects in the request
> 
> This doesn't match any var...

Yeah, should be num_incomplete_objects. I went back-and-forth a few times
and the doc is now out of sync...

> 
>> + * @poll_wait: Wait queue for poll
>> + * @lock: Serializes access to this struct
>> + */
>> +struct media_request {
>> +	struct media_device *mdev;
>> +	struct kref kref;
>> +	char debug_str[TASK_COMM_LEN + 11];
>> +	enum media_request_state state;
>> +	struct list_head objects;
>> +	unsigned int num_incomplete_objects;
> 
> ... as it seems that you decide to count incomplete objects, instead
> of complete ones.
> 
>> +	struct wait_queue_head poll_wait;
>> +	spinlock_t lock;
>> +};
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +
>> +static inline void media_request_get(struct media_request *req)
>> +{
>> +	kref_get(&req->kref);
>> +}
>> +
>> +void media_request_put(struct media_request *req);
>> +void media_request_cancel(struct media_request *req);
>> +
>>  int media_request_alloc(struct media_device *mdev,
>>  			struct media_request_alloc *alloc);
>> +#else
>> +static inline void media_request_get(struct media_request *req)
>> +{
>> +}
>> +
>> +static inline void media_request_put(struct media_request *req)
>> +{
>> +}
>> +
>> +static inline void media_request_cancel(struct media_request *req)
>> +{
>> +}
>> +
>> +#endif
>> +
>> +struct media_request_object_ops {
>> +	int (*prepare)(struct media_request_object *object);
>> +	void (*unprepare)(struct media_request_object *object);
>> +	void (*queue)(struct media_request_object *object);
>> +	void (*unbind)(struct media_request_object *object);
>> +	void (*cancel)(struct media_request_object *object);
>> +	void (*release)(struct media_request_object *object);
>> +};
> 
> -ENODESCRIPTION. Please describe the struct and its fields.

Known, I mentioned this in the cover letter.

> 
>> +
>> +/**
>> + * struct media_request_object - An opaque object that belongs to a media
>> + *				 request
>> + *
>> + * @priv: object's priv pointer
>> + * @list: List entry of the object for @struct media_request
>> + * @kref: Reference count of the object, acquire before releasing req->lock
> 
> Field descriptions missing.

Added.

> 
>> + *
>> + * An object related to the request. This struct is embedded in the
>> + * larger object data.
>> + */
>> +struct media_request_object {
>> +	const struct media_request_object_ops *ops;
>> +	void *priv;
>> +	struct media_request *req;
>> +	struct list_head list;
>> +	struct kref kref;
>> +	bool completed;
>> +};
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +static inline void media_request_object_get(struct media_request_object *obj)
>> +{
>> +	kref_get(&obj->kref);
>> +}
> 
> Why do you need a function? Just use kref_get() were needed, instead of
> aliasing it for no good reason.

Because that's what everyone does? That way you have nicely balanced
media_request_object_get/put functions. Easier to review.

> 
>> +
>> +/**
>> + * media_request_object_put - Put a media request object
>> + *
>> + * @obj: The object
>> + *
>> + * Put a media request object. Once all references are gone, the
>> + * object's memory is released.
>> + */
>> +void media_request_object_put(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_init - Initialise a media request object
>> + *
>> + * Initialise a media request object. The object will be released using the
>> + * release callback of the ops once it has no references (this function
>> + * initialises references to one).
>> + */
>> +void media_request_object_init(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_bind - Bind a media request object to a request
>> + */
>> +void media_request_object_bind(struct media_request *req,
>> +			       const struct media_request_object_ops *ops,
>> +			       void *priv,
>> +			       struct media_request_object *obj);
>> +
>> +void media_request_object_unbind(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_complete - Mark the media request object as complete
>> + */
>> +void media_request_object_complete(struct media_request_object *obj);
> 
> The kernel-doc tags there are wrong: They don't describe the function
> arguments.
> 
> Please validate it with sphinx to be sure that everything is ok.
> 
>> +#else
>> +static inline void media_request_object_get(struct media_request_object *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_put(struct media_request_object *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_init(struct media_request_object *obj)
>> +{
>> +	obj->ops = NULL;
>> +	obj->req = NULL;
>> +}
>> +
>> +static inline void media_request_object_bind(struct media_request *req,
>> +			       const struct media_request_object_ops *ops,
>> +			       void *priv,
>> +			       struct media_request_object *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_unbind(struct media_request_object *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_complete(struct media_request_object *obj)
>> +{
>> +}
>> +#endif
>>  
>>  #endif
> 
> 
> 
> Thanks,
> Mauro
> 

Regards,

	Hans

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Hans Verkuil]

On 04/12/2018 09:35 AM, Sakari Ailus wrote:
> Hi Tomasz,
> 
> On Tue, Apr 10, 2018 at 08:59:23AM +0000, Tomasz Figa wrote:
>> Hi Hans,
>>
>> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>>> Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
>>> ioctls.
>>
>>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>>> ---
>>>   drivers/media/media-request.c | 80
>> +++++++++++++++++++++++++++++++++++++++++--
>>>   1 file changed, 78 insertions(+), 2 deletions(-)
>>
>>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>>> index dffc290e4ada..27739ff7cb09 100644
>>> --- a/drivers/media/media-request.c
>>> +++ b/drivers/media/media-request.c
>>> @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file
>> *filp,
>>>          return 0;
>>>   }
>>
>>> +static long media_request_ioctl_queue(struct media_request *req)
>>> +{
>>> +       struct media_device *mdev = req->mdev;
>>> +       unsigned long flags;
>>> +       int ret = 0;
>>> +
>>> +       dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
>>> +
>>> +       spin_lock_irqsave(&req->lock, flags);
>>> +       if (req->state != MEDIA_REQUEST_STATE_IDLE) {
>>> +               dev_dbg(mdev->dev,
>>> +                       "request: unable to queue %s, request in state
>> %s\n",
>>> +                       req->debug_str,
>> media_request_state_str(req->state));
>>> +               spin_unlock_irqrestore(&req->lock, flags);
>>> +               return -EINVAL;
>>
>> nit: Perhaps -EBUSY? (vb2 returns -EINVAL, though, but IMHO it doesn't
>> really represent the real error too closely.)
>>
>>> +       }
>>> +       req->state = MEDIA_REQUEST_STATE_QUEUEING;
>>> +
>>> +       spin_unlock_irqrestore(&req->lock, flags);
>>> +
>>> +       /*
>>> +        * Ensure the request that is validated will be the one that gets
>> queued
>>> +        * next by serialising the queueing process.
>>> +        */
>>> +       mutex_lock(&mdev->req_queue_mutex);
>>> +
>>> +       ret = mdev->ops->req_queue(req);
>>> +       spin_lock_irqsave(&req->lock, flags);
>>> +       req->state = ret ? MEDIA_REQUEST_STATE_IDLE :
>> MEDIA_REQUEST_STATE_QUEUED;
>>> +       spin_unlock_irqrestore(&req->lock, flags);
>>> +       mutex_unlock(&mdev->req_queue_mutex);
>>> +
>>> +       if (ret) {
>>> +               dev_dbg(mdev->dev, "request: can't queue %s (%d)\n",
>>> +                       req->debug_str, ret);
>>> +       } else {
>>> +               media_request_get(req);
>>
>> I'm not convinced that this is the right place to take a reference. IMHO
>> whoever saves a pointer to the request in its own internal data (the
>> ->req_queue() callback?), should also grab a reference before doing so. Not
>> a strong objection, though, if we clearly document this, so that whoever
>> implements ->req_queue() callback can do the right thing.
> 
> The framework will also release that reference once the request is
> complete. In my opinion it's perfectly reasonable to do this in the
> framework; moving things to frameworks that do not need to be done in
> drivers generally reduces bugs and makes drivers more maintainable. Albeit
> this is a minor matter in this respect.
> 
> Hans: the reference must be taken before the request is queued: otherwise
> it is possible that the driver completes the request before the reference
> is taken here. That would mean the request might have been already released
> by the time of getting a refenrece to it.

Actually, it can't. As long as userspace has the request fd open there is always
at least one reference. And since we're in an ioctl (thus using the fd) it can't
be closed yet.

Regards,

	Hans

> 
>>
>>> +       }
>>> +
>>> +       return ret;
>>> +}
>>> +
>>> +static long media_request_ioctl_reinit(struct media_request *req)
>>> +{
>>> +       struct media_device *mdev = req->mdev;
>>> +       unsigned long flags;
>>> +
>>> +       spin_lock_irqsave(&req->lock, flags);
>>> +       if (req->state != MEDIA_REQUEST_STATE_IDLE &&
>>> +           req->state != MEDIA_REQUEST_STATE_COMPLETE) {
>>> +               dev_dbg(mdev->dev,
>>> +                       "request: %s not in idle or complete state,
>> cannot reinit\n",
>>> +                       req->debug_str);
>>> +               spin_unlock_irqrestore(&req->lock, flags);
>>> +               return -EINVAL;
>>
>> nit: Perhaps -EBUSY? (Again vb2 would return -EINVAL...)
> 
> I agree on both return values.
> 

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Hans Verkuil]

On 04/12/2018 09:11 AM, Sakari Ailus wrote:
> On Wed, Apr 11, 2018 at 01:13:44PM -0300, Mauro Carvalho Chehab wrote:
>> Em Wed, 11 Apr 2018 18:35:14 +0300
>> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
>>
>>> On Wed, Apr 11, 2018 at 12:17:27PM -0300, Mauro Carvalho Chehab wrote:
>>>> Em Wed, 11 Apr 2018 18:02:19 +0300
>>>> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
>>>>   
>>>>> On Wed, Apr 11, 2018 at 10:49:35AM -0300, Mauro Carvalho Chehab wrote:  
>>>>>> Em Wed, 11 Apr 2018 16:21:16 +0300
>>>>>> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
>>>>>>
>>>>>>     
>>>>>>>>>> Btw, this is a very good reason why you should define the ioctl to
>>>>>>>>>> have an integer argument instead of a struct with a __s32 field
>>>>>>>>>> on it, as per my comment to patch 02/29:
>>>>>>>>>>
>>>>>>>>>> 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
>>>>>>>>>>
>>>>>>>>>> At 64 bit architectures, you're truncating the file descriptor!        
>>>>>>>>>
>>>>>>>>> I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
>>>>>>>>> well.      
>>>>>>>>
>>>>>>>> Hmm.. you're right. I was thinking that it could be 64 bits on some
>>>>>>>> archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
>>>>>>>> according with:
>>>>>>>>
>>>>>>>> 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
>>>>>>>>
>>>>>>>> This is not the case on gcc.      
>>>>>>>
>>>>>>> Ok. The reasoning back then was that what "int" means varies across
>>>>>>> compilers and languages. And the intent was to codify this to __s32 which
>>>>>>> is what the kernel effectively uses.    
>>>>>>
>>>>>> ...
>>>>>>     
>>>>>>> The rest of the kernel uses int rather liberally in the uAPI so I'm not
>>>>>>> sure in the end whether something desirable was achieved. Perhaps it'd be
>>>>>>> good to go back to the original discussion to find out for sure.
>>>>>>>
>>>>>>> Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
>>>>>>> due to that difference.
>>>>>>>
>>>>>>> Well, I stop here for this begins to be off-topic. :-)    
>>>>>>
>>>>>> Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
>>>>>> about that :-)
>>>>>>     
>>>>>>>>>>> +	get_task_comm(comm, current);
>>>>>>>>>>> +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
>>>>>>>>>>> +		 comm, fd);        
>>>>>>>>>>
>>>>>>>>>> Not sure if it is a good idea to store the task that allocated
>>>>>>>>>> the request. While it makes sense for the dev_dbg() below, it
>>>>>>>>>> may not make sense anymore on other dev_dbg() you would be
>>>>>>>>>> using it.        
>>>>>>>>>
>>>>>>>>> The lifetime of the file handle roughly matches that of the request. It's
>>>>>>>>> for debug only anyway.
>>>>>>>>>
>>>>>>>>> Better proposals are always welcome of course. But I think we should have
>>>>>>>>> something here that helps debugging by meaningfully making the requests
>>>>>>>>> identifiable from logs.      
>>>>>>>>
>>>>>>>> What I meant to say is that one PID could be allocating the
>>>>>>>> request, while some other one could be actually doing Q/DQ_BUF.
>>>>>>>> On such scenario, the debug string could provide mislead prints.      
>>>>>>>
>>>>>>> Um, yes, indeed it would no longer match the process. But the request is
>>>>>>> still the same. That's actually a positive thing since it allows you to
>>>>>>> identify the request.
>>>>>>>
>>>>>>> With a global ID space this was trivial; you could just print the request
>>>>>>> ID and that was all that was ever needed. (I'm not proposing to consider
>>>>>>> that though.)
>>>>>>>     
>>>>>>
>>>>>> IMO, a global ID number would work better than get_task_comm().
>>>>>>
>>>>>> Just add a static int monotonic counter and use it for the debug purposes,
>>>>>> e. g.:
>>>>>>
>>>>>> {
>>>>>> 	static unsigned int req_count = 0;
>>>>>>
>>>>>> 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
>>>>>> 		req_count++, fd);    
>>>>>>
>>>>>> Ok, eventually, it will overflow, but, it will be unique within
>>>>>> a reasonable timeframe to be good enough for debugging purposes.    
>>>>>
>>>>> Yes, but you can't figure out which process allocated it anymore, making
>>>>> associating kernel debug logs with user space process logs harder.
>>>>>
>>>>> How about process id + file handle? That still doesn't tell which process
>>>>> operated on the request though, but I'm not sure whether that's really a
>>>>> crucial piece of information.  
>>>>
>>>> You don't need that. With dev_dbg() - and other *_dbg() macros - you can
>>>> enable process ID for all debug messages.  
>>>
>>> With this, the problem again is that it does not uniquely identify the
>>> request: the request is the same request independently of which process
>>> would operate on it. Or whether it is being processed in an interrupt
>>> context.
>>>
>>> AFAICT, the allocator PID (or process name) + file handle are both required
>>> to match a request between user and kernel space logs.
>>
>> Sorry, I was unable to understand what you're saying.
>>
>> If you set the debug string with:
>>
>> 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d", req_count++, fd);  
>>
>> With the remaining stuff at patch 04/29, e. g. those two printks:
>>
>> 	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
>> 	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
>>
>> And use "+pt" to enable those debug messages, for the request #1 with fd #45, 
>> created by PID 16613 you would have a log like:
>>
>> 	[  269.021116] [16613] request: allocated 1:45
>> 	[  269.024118] [16613] request: release 1:45
>>
>> (assuming that the same PID would create and release)
>>
>> The "1:45" is an unique global ID that would allow tracking it, even
>> if Q/DQ_BUF is done by some other PID.
>>
>> E. g. if a PID#16618 were responsible for Q/DQ_BUF, you would have
>> something like:
>>
>> 	[  269.021116] [16613] request: allocated 1:45
>> 	[  269.021117] [16618] request: Q_BUF 1:45
>> 	[  269.021118] [16618] request: DQ_BUF 1:45
>> 	[  269.024118] [16613] request: release 1:45
>>
>> (assuming that you would have a Q_BUF/DQ_BUF similar dev_dbg())
>>
>> That seems good enough to track it.
>>
>> Yet, in order to make easier to track, I would actually change the
>> dev_dbg() parameter order everywhere to something like:
>>
>> 	dev_dbg(mdev->dev, "request#%s: allocated\n", req->debug_str)
>> 	dev_dbg(mdev->dev, "request#%s: release\n", req->debug_str);
>>
>> In order to print something like:
>>
>> 	[  269.021116] [16613] request#1:45: allocated 
>> 	[  269.021117] [16618] request#1:45: Q_BUF
>> 	[  269.021118] [16618] request#1:45: DQ_BUF
>> 	[  269.024118] [16613] request#1:45: release
>>
>> Then, getting everything related to the first request would be as simple as:
>>
>> 	$ dmesg|grep request#1:
>>
>> That will provide the PID for both processes: the one that
>> created/released and the one that queued/dequeued.
> 
> Ah, right; yes, then you can. It's still a bit more complicated as you have
> one more piece of information to follow (the ID) vs. just PID and FD. For
> instance, you can't grep for requests created by a given process. Note that
> you can still print the PID of the process that operates on the request
> through dyndbg.
> 
> I'd like to hear what Hans thinks.
> 

Frankly, I have no opinion :-)

I'm OK with changing this to:

snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d", req_count++, fd);

if Sakari agrees with that.

Regards,

	Hans

Re: [RFCv11 PATCH 19/29] videobuf2-core: integrate with media requests

[Hans Verkuil]

On 04/12/2018 10:13 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Buffers can now be prepared or queued for a request.
> 
>> A buffer is unbound from the request at vb2_buffer_done time or
>> when the queue is cancelled.
> 
> Please see my comments inline.
> 
> [snip]
>> -int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void
> *pb)
>> +static int vb2_req_prepare(struct media_request_object *obj)
>>   {
>> -       struct vb2_buffer *vb;
>> +       struct vb2_buffer *vb = container_of(obj, struct vb2_buffer,
> req_obj);
>>          int ret;
> 
>> +       if (WARN_ON(vb->state != VB2_BUF_STATE_IN_REQUEST))
>> +               return -EINVAL;
>> +
>> +       ret = __buf_prepare(vb, NULL);
>> +       if (ret)
>> +               vb->state = VB2_BUF_STATE_IN_REQUEST;
> 
> Hmm, I suppose this is here because __buf_prepare() changes the state to
> VB2_BUF_STATE_DEQUEUED on error (other than q->error)? I guess it's
> harmless, but perhaps we could have a comment explaining this?

Yes, that's the reason. But it is really ugly, so I changed __buf_prepare
instead: it now remembers the original state and restores it on error.

Much cleaner.

Regards,

	Hans

> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 22/29] videobuf2-v4l2: add vb2_request_queue helper

[Hans Verkuil]

On 04/12/2018 10:29 AM, Tomasz Figa wrote:
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Generic helper function that checks if there are buffers in
>> the request and if so, prepares and queues all objects in the
>> request.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/common/videobuf2/videobuf2-v4l2.c | 39
> +++++++++++++++++++++++++
>>   include/media/videobuf2-v4l2.h                  |  3 ++
>>   2 files changed, 42 insertions(+)
> 
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index 73c1fd4da58a..3d0c74bb4220 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -1061,6 +1061,45 @@ void vb2_ops_wait_finish(struct vb2_queue *vq)
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_ops_wait_finish);
> 
>> +int vb2_request_queue(struct media_request *req)
>> +{
>> +       struct media_request_object *obj;
>> +       struct media_request_object *failed_obj = NULL;
>> +       int ret = 0;
>> +
>> +       if (!vb2_core_request_has_buffers(req))
>> +               return -ENOENT;
>> +
>> +       list_for_each_entry(obj, &req->objects, list) {
>> +               if (!obj->ops->prepare)
>> +                       continue;
>> +
>> +               ret = obj->ops->prepare(obj);
>> +
>> +               if (ret) {
>> +                       failed_obj = obj;
>> +                       break;
>> +               }
>> +       }
>> +
>> +       if (ret) {
>> +               list_for_each_entry(obj, &req->objects, list) {
>> +                       if (obj == failed_obj)
>> +                               break;
> 
> nit: If we use list_for_each_entry_continue_reverse() here, we wouldn't
> need failed_obj.

True. Done.

	Hans

> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 25/29] media: vim2m: add media device

[Hans Verkuil]

On 04/12/2018 10:54 AM, Tomasz Figa wrote:
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Alexandre Courbot <acourbot@chromium.org>
> 
>> Request API requires a media node. Add one to the vim2m driver so we can
>> use requests with it.
> 
>> This probably needs a bit more work to correctly represent m2m
>> hardware in the media topology.
> [snip]
>> @@ -1013,6 +1012,22 @@ static int vim2m_probe(struct platform_device
> *pdev)
>>          vfd->lock = &dev->dev_mutex;
>>          vfd->v4l2_dev = &dev->v4l2_dev;
> 
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +       dev->mdev.dev = &pdev->dev;
>> +       strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
>> +       media_device_init(&dev->mdev);
>> +       dev->v4l2_dev.mdev = &dev->mdev;
>> +       dev->pad[0].flags = MEDIA_PAD_FL_SINK;
>> +       dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
>> +       ret = media_entity_pads_init(&vfd->entity, 2, dev->pad);
>> +       if (ret)
>> +               return ret;
> 
> Hmm, what are these pads linked to?

Nothing. It's a quick hack, needs more work.

> 
> [snip]
>> @@ -1050,6 +1076,13 @@ static int vim2m_remove(struct platform_device
> *pdev)
>>          struct vim2m_dev *dev = platform_get_drvdata(pdev);
> 
>>          v4l2_info(&dev->v4l2_dev, "Removing " MEM2MEM_NAME);
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +       if (media_devnode_is_registered(dev->mdev.devnode))
> 
> Do we need to check this? Probe seems to fail if media device fails to
> register.

This was copy-and-paste from somewhere. I agree, this can be dropped.

Regards,

	Hans

> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 27/29] vim2m: support requests

[Hans Verkuil]

On 04/12/2018 11:15 AM, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Add support for requests to vim2m.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
>>   1 file changed, 25 insertions(+)
> 
>> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
>> index 9b18b32c255d..2dcf0ea85705 100644
>> --- a/drivers/media/platform/vim2m.c
>> +++ b/drivers/media/platform/vim2m.c
>> @@ -387,8 +387,26 @@ static void device_run(void *priv)
>>          src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
>>          dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
> 
>> +       /* Apply request if needed */
>> +       if (src_buf->vb2_buf.req_obj.req)
>> +               v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
>> +                                       &ctx->hdl);
>> +       if (dst_buf->vb2_buf.req_obj.req &&
>> +           dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
>> +               v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
>> +                                       &ctx->hdl);
> 
> I'm not sure I understand what's going on here. How is it possible that we
> have 2 different requests?

You can have one request with buffers for both queues, or two requests, one
for each queue. Or a request for just one of the queues and the other without
any requests.

So you can have 0, 1 or 2 requests associated with these two queues.

But you don't want to call v4l2_ctrl_request_setup twice if the same request
is associated with both queues. (Well, you can call it twice and the second
call would not do anything, but that's a waste of CPU cycles)

Regards,

	Hans

Re: [RFCv11 PATCH 02/29] uapi/linux/media.h: add request API

[Mauro Carvalho Chehab]

Em Mon, 23 Apr 2018 13:41:31 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> On 04/10/2018 01:00 PM, Sakari Ailus wrote:
> > On Tue, Apr 10, 2018 at 06:38:56AM -0300, Mauro Carvalho Chehab wrote:  
> >> Em Mon,  9 Apr 2018 16:19:59 +0200
> >> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> >>  
> >>> From: Hans Verkuil <hans.verkuil@cisco.com>
> >>>
> >>> Define the public request API.
> >>>
> >>> This adds the new MEDIA_IOC_REQUEST_ALLOC ioctl to allocate a request
> >>> and two ioctls that operate on a request in order to queue the
> >>> contents of the request to the driver and to re-initialize the
> >>> request.
> >>>
> >>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> >>> ---
> >>>  include/uapi/linux/media.h | 8 ++++++++
> >>>  1 file changed, 8 insertions(+)
> >>>
> >>> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> >>> index c7e9a5cba24e..f8769e74f847 100644
> >>> --- a/include/uapi/linux/media.h
> >>> +++ b/include/uapi/linux/media.h
> >>> @@ -342,11 +342,19 @@ struct media_v2_topology {
> >>>  
> >>>  /* ioctls */
> >>>  
> >>> +struct __attribute__ ((packed)) media_request_alloc {
> >>> +	__s32 fd;
> >>> +};
> >>> +
> >>>  #define MEDIA_IOC_DEVICE_INFO	_IOWR('|', 0x00, struct media_device_info)
> >>>  #define MEDIA_IOC_ENUM_ENTITIES	_IOWR('|', 0x01, struct media_entity_desc)
> >>>  #define MEDIA_IOC_ENUM_LINKS	_IOWR('|', 0x02, struct media_links_enum)
> >>>  #define MEDIA_IOC_SETUP_LINK	_IOWR('|', 0x03, struct media_link_desc)
> >>>  #define MEDIA_IOC_G_TOPOLOGY	_IOWR('|', 0x04, struct media_v2_topology)
> >>> +#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, struct media_request_alloc)
> >>> +  
> >>
> >> Why use a struct here? Just declare it as:
> >>
> >> 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)  
> > 
> > I'd say it's easier to extend it if it's a struct.

That's not true. Assuming that you declare it as:

 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)  

If you ever need a struct there, let's say:

struct media_request_alloc {
	int request;
	int new_field;
};

#define MEDIA_IOC_REQUEST_ALLOC_old _IOWR('|', 0x05, int)  
#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, struct media_request_alloc)

And add both MEDIA_IOC_REQUEST_ALLOC_old and MEDIA_IOC_REQUEST_ALLOC
to the ioctl handling logic, where the handler for
MEDIA_IOC_REQUEST_ALLOC_old being something like:

	switch (cmd) {
	...
	case MEDIA_IOC_REQUEST_ALLOC_old:
		struct media_request_alloc arg = {};

		arg.request = parg;

and calling the same routine as MEDIA_IOC_REQUEST_ALLOC would
call, passing &arg as its input.

One advantage of not passing an struct is that there's no
need to call copy_to_user(), making its handler faster.

> > All other IOCTLs also
> > have a struct as an argument. 

That is not true as well, if you consider V4L2 API (or the media
API as a hole). There are several ioctls there that takes just
an integer argument:

include/uapi/linux/videodev2.h:#define VIDIOC_OVERLAY		 _IOW('V', 14, int)
include/uapi/linux/videodev2.h:#define VIDIOC_STREAMON		 _IOW('V', 18, int)
include/uapi/linux/videodev2.h:#define VIDIOC_STREAMOFF	 _IOW('V', 19, int)
include/uapi/linux/videodev2.h:#define VIDIOC_G_INPUT		 _IOR('V', 38, int)
include/uapi/linux/videodev2.h:#define VIDIOC_S_INPUT		_IOWR('V', 39, int)
include/uapi/linux/videodev2.h:#define VIDIOC_G_OUTPUT		 _IOR('V', 46, int)
include/uapi/linux/videodev2.h:#define VIDIOC_S_OUTPUT		_IOWR('V', 47, int)
include/uapi/linux/dvb/frontend.h:#define FE_ENABLE_HIGH_LNB_VOLTAGE _IO('o', 68)  /* int */
include/uapi/linux/dvb/frontend.h:#define FE_SET_FRONTEND_TUNE_MODE  _IO('o', 81) /* unsigned int */
include/uapi/linux/dvb/frontend.h:#define FE_DISHNETWORK_SEND_LEGACY_CMD _IO('o', 80) /* unsigned int */
include/uapi/linux/lirc.h:#define LIRC_GET_FEATURES              _IOR('i', 0x00000000, __u32)
include/uapi/linux/lirc.h:#define LIRC_GET_SEND_MODE             _IOR('i', 0x00000001, __u32)
include/uapi/linux/lirc.h:#define LIRC_GET_REC_MODE              _IOR('i', 0x00000002, __u32)
include/uapi/linux/lirc.h:#define LIRC_GET_REC_RESOLUTION        _IOR('i', 0x00000007, __u32)
include/uapi/linux/lirc.h:#define LIRC_GET_MIN_TIMEOUT           _IOR('i', 0x00000008, __u32)
include/uapi/linux/lirc.h:#define LIRC_GET_MAX_TIMEOUT           _IOR('i', 0x00000009, __u32)
include/uapi/linux/lirc.h:#define LIRC_GET_LENGTH                _IOR('i', 0x0000000f, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_SEND_MODE             _IOW('i', 0x00000011, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_REC_MODE              _IOW('i', 0x00000012, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_SEND_CARRIER          _IOW('i', 0x00000013, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_REC_CARRIER           _IOW('i', 0x00000014, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_SEND_DUTY_CYCLE       _IOW('i', 0x00000015, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_TRANSMITTER_MASK      _IOW('i', 0x00000017, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_REC_TIMEOUT           _IOW('i', 0x00000018, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_REC_TIMEOUT_REPORTS   _IOW('i', 0x00000019, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_MEASURE_CARRIER_MODE _IOW('i', 0x0000001d, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_REC_CARRIER_RANGE     _IOW('i', 0x0000001f, __u32)
include/uapi/linux/lirc.h:#define LIRC_SET_WIDEBAND_RECEIVER     _IOW('i', 0x00000023, __u32)
include/uapi/linux/lirc.h:#define LIRC_GET_REC_TIMEOUT         _IOR('i', 0x00000024, __u32)
include/uapi/linux/dvb/frontend.h:#define FE_READ_BER		   _IOR('o', 70, __u32)
include/uapi/linux/dvb/frontend.h:#define FE_READ_SIGNAL_STRENGTH    _IOR('o', 71, __u16)
include/uapi/linux/dvb/frontend.h:#define FE_READ_SNR		   _IOR('o', 72, __u16)
include/uapi/linux/dvb/frontend.h:#define FE_READ_UNCORRECTED_BLOCKS _IOR('o', 73, __u32)

Btw, at the beginning, I found ugly the way lirc API were designed (there,
*all* ioctls pass/receive integers), but the more I looked into it and
started documenting it, the more I liked!

It is a way easier to understand what each ioctl does, as just one thing
is affected by each ioctl. Btw, sysfs follows the same logic: there, you
really need a very strong reason why not just pass a single value to each
sysfs node.

That makes the design better, and usually avoid the need of changes, as,
if you need something else, just add a new sysfs node, or a new ioctl
with would take just one integer.

> > As a struct member, the parameter (fd) also
> > has a name; this is a plus.  

Huh? Why is it a plus?

> 
> While I do not have a very strong opinion on this, I do agree with Sakari here.

The thing is: adding a struct there is overdesign. If are there any
real reason why to have it as a struct (e. g. if we know/foresee any other
parameter to be needed there), then 

> 
> Regards,
> 
> 	Hans
> 
> >   
> >>  
> >>> +#define MEDIA_REQUEST_IOC_QUEUE		_IO('|',  0x80)
> >>> +#define MEDIA_REQUEST_IOC_REINIT	_IO('|',  0x81)
> >>>  
> >>>  #if !defined(__KERNEL__) || defined(__NEED_MEDIA_LEGACY_API)
> >>>    
> >>
> >> Thanks,
> >> Mauro  
> >   
> 



Thanks,
Mauro

Re: [RFCv11 PATCH 03/29] media-request: allocate media requests

[Mauro Carvalho Chehab]

Em Mon, 23 Apr 2018 13:49:11 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> On 04/10/2018 11:52 AM, Mauro Carvalho Chehab wrote:
> > Em Mon,  9 Apr 2018 16:20:00 +0200
> > Hans Verkuil <hverkuil@xs4all.nl> escreveu:
> >   
> >> From: Hans Verkuil <hans.verkuil@cisco.com>
> >>
> >> Add support for allocating a new request. This is only supported
> >> if mdev->ops->req_queue is set, i.e. the driver indicates that it
> >> supports queueing requests.
> >>
> >> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
> >> ---
> >>  drivers/media/Makefile        |  3 ++-
> >>  drivers/media/media-device.c  | 14 ++++++++++++++
> >>  drivers/media/media-request.c | 23 +++++++++++++++++++++++
> >>  include/media/media-device.h  | 13 +++++++++++++
> >>  include/media/media-request.h | 22 ++++++++++++++++++++++
> >>  5 files changed, 74 insertions(+), 1 deletion(-)
> >>  create mode 100644 drivers/media/media-request.c
> >>  create mode 100644 include/media/media-request.h
> >>
> >> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> >> index 594b462ddf0e..985d35ec6b29 100644
> >> --- a/drivers/media/Makefile
> >> +++ b/drivers/media/Makefile
> >> @@ -3,7 +3,8 @@
> >>  # Makefile for the kernel multimedia device drivers.
> >>  #
> >>  
> >> -media-objs	:= media-device.o media-devnode.o media-entity.o
> >> +media-objs	:= media-device.o media-devnode.o media-entity.o \
> >> +		   media-request.o
> >>  
> >>  #
> >>  # I2C drivers should come before other drivers, otherwise they'll fail
> >> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> >> index 35e81f7c0d2f..acb583c0eacd 100644
> >> --- a/drivers/media/media-device.c
> >> +++ b/drivers/media/media-device.c
> >> @@ -32,6 +32,7 @@
> >>  #include <media/media-device.h>
> >>  #include <media/media-devnode.h>
> >>  #include <media/media-entity.h>
> >> +#include <media/media-request.h>
> >>  
> >>  #ifdef CONFIG_MEDIA_CONTROLLER
> >>  
> >> @@ -366,6 +367,15 @@ static long media_device_get_topology(struct media_device *mdev,
> >>  	return ret;
> >>  }
> >>  
> >> +static long media_device_request_alloc(struct media_device *mdev,
> >> +				       struct media_request_alloc *alloc)
> >> +{
> >> +	if (!mdev->ops || !mdev->ops->req_queue)
> >> +		return -ENOTTY;
> >> +
> >> +	return media_request_alloc(mdev, alloc);
> >> +}
> >> +
> >>  static long copy_arg_from_user(void *karg, void __user *uarg, unsigned int cmd)
> >>  {
> >>  	/* All media IOCTLs are _IOWR() */
> >> @@ -414,6 +424,7 @@ static const struct media_ioctl_info ioctl_info[] = {
> >>  	MEDIA_IOC(ENUM_LINKS, media_device_enum_links, MEDIA_IOC_FL_GRAPH_MUTEX),
> >>  	MEDIA_IOC(SETUP_LINK, media_device_setup_link, MEDIA_IOC_FL_GRAPH_MUTEX),
> >>  	MEDIA_IOC(G_TOPOLOGY, media_device_get_topology, MEDIA_IOC_FL_GRAPH_MUTEX),
> >> +	MEDIA_IOC(REQUEST_ALLOC, media_device_request_alloc, 0),
> >>  };
> >>  
> >>  static long media_device_ioctl(struct file *filp, unsigned int cmd,
> >> @@ -686,6 +697,9 @@ void media_device_init(struct media_device *mdev)
> >>  	INIT_LIST_HEAD(&mdev->pads);
> >>  	INIT_LIST_HEAD(&mdev->links);
> >>  	INIT_LIST_HEAD(&mdev->entity_notify);
> >> +
> >> +	spin_lock_init(&mdev->req_lock);
> >> +	mutex_init(&mdev->req_queue_mutex);
> >>  	mutex_init(&mdev->graph_mutex);
> >>  	ida_init(&mdev->entity_internal_idx);
> >>  
> >> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> >> new file mode 100644
> >> index 000000000000..ead78613fdbe
> >> --- /dev/null
> >> +++ b/drivers/media/media-request.c
> >> @@ -0,0 +1,23 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * Media device request objects
> >> + *
> >> + * Copyright (C) 2018 Intel Corporation
> >> + * Copyright (C) 2018, The Chromium OS Authors.  All rights reserved.
> >> + *
> >> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> >> + */
> >> +
> >> +#include <linux/anon_inodes.h>  
> > 
> > Not needed. You already included it at media_device.h.  
> 
> Actually, it is needed here, but not in media_device.h

After patch 4, yes it is needed, but not in this patch :-) 

> 
> >   
> >> +#include <linux/file.h>
> >> +#include <linux/mm.h>
> >> +#include <linux/string.h>  
> > 
> > Why do you need so many includes for a stub function?  
> 
> I'm going to merge patch 3 and 4. Splitting it up it too confusing for
> reviewers, I've noticed. These headers are needed for patch 4.

OK!
> 
> As mentioned in an earlier reply: the req_lock spinlock is unused and is
> now deleted. It was a left-over from older versions and that's what caused
> the locking confusion.

Ok. I'll review the locks again on your next patchset.

> 
> Regards,
> 
> 	Hans
> 
> 
> >   
> >> +
> >> +#include <media/media-device.h>
> >> +#include <media/media-request.h>
> >> +
> >> +int media_request_alloc(struct media_device *mdev,
> >> +			struct media_request_alloc *alloc)
> >> +{
> >> +	return -ENOMEM;
> >> +}
> >> diff --git a/include/media/media-device.h b/include/media/media-device.h
> >> index bcc6ec434f1f..07e323c57202 100644
> >> --- a/include/media/media-device.h
> >> +++ b/include/media/media-device.h
> >> @@ -19,6 +19,7 @@
> >>  #ifndef _MEDIA_DEVICE_H
> >>  #define _MEDIA_DEVICE_H
> >>  
> >> +#include <linux/anon_inodes.h>  
> > 
> > Why do you need it? I don't see anything below needing it.
> >   
> >>  #include <linux/list.h>
> >>  #include <linux/mutex.h>
> >>  
> >> @@ -27,6 +28,7 @@
> >>  
> >>  struct ida;
> >>  struct device;
> >> +struct media_device;
> >>  
> >>  /**
> >>   * struct media_entity_notify - Media Entity Notify
> >> @@ -50,10 +52,16 @@ struct media_entity_notify {
> >>   * struct media_device_ops - Media device operations
> >>   * @link_notify: Link state change notification callback. This callback is
> >>   *		 called with the graph_mutex held.
> >> + * @req_alloc: Allocate a request
> >> + * @req_free: Free a request
> >> + * @req_queue: Queue a request
> >>   */
> >>  struct media_device_ops {
> >>  	int (*link_notify)(struct media_link *link, u32 flags,
> >>  			   unsigned int notification);
> >> +	struct media_request *(*req_alloc)(struct media_device *mdev);
> >> +	void (*req_free)(struct media_request *req);
> >> +	int (*req_queue)(struct media_request *req);
> >>  };
> >>  
> >>  /**
> >> @@ -88,6 +96,8 @@ struct media_device_ops {
> >>   * @disable_source: Disable Source Handler function pointer
> >>   *
> >>   * @ops:	Operation handler callbacks
> >> + * @req_lock:	Serialise access to requests
> >> + * @req_queue_mutex: Serialise validating and queueing requests  
> > 
> > IMHO, this would better describe it:
> > 	Serialise validate and queue requests
> > 
> > Yet, IMO, it doesn't let it clear when the spin lock should be
> > used and when the mutex should be used.
> > 
> > I mean, what of them protect what variable?
> >   
> >>   *
> >>   * This structure represents an abstract high-level media device. It allows easy
> >>   * access to entities and provides basic media device-level support. The
> >> @@ -158,6 +168,9 @@ struct media_device {
> >>  	void (*disable_source)(struct media_entity *entity);
> >>  
> >>  	const struct media_device_ops *ops;
> >> +
> >> +	spinlock_t req_lock;
> >> +	struct mutex req_queue_mutex;
> >>  };
> >>  
> >>  /* We don't need to include pci.h or usb.h here */
> >> diff --git a/include/media/media-request.h b/include/media/media-request.h
> >> new file mode 100644
> >> index 000000000000..dae3eccd9aa7
> >> --- /dev/null
> >> +++ b/include/media/media-request.h
> >> @@ -0,0 +1,22 @@
> >> +// SPDX-License-Identifier: GPL-2.0
> >> +/*
> >> + * Media device request objects
> >> + *
> >> + * Copyright (C) 2018 Intel Corporation
> >> + *
> >> + * Author: Sakari Ailus <sakari.ailus@linux.intel.com>
> >> + */
> >> +
> >> +#ifndef MEDIA_REQUEST_H
> >> +#define MEDIA_REQUEST_H
> >> +
> >> +#include <linux/list.h>
> >> +#include <linux/slab.h>
> >> +#include <linux/spinlock.h>  
> > 
> > Why those includes are needed?
> >   
> >> +
> >> +#include <media/media-device.h>
> >> +
> >> +int media_request_alloc(struct media_device *mdev,
> >> +			struct media_request_alloc *alloc);
> >> +
> >> +#endif  
> > 
> > 
> > 
> > Thanks,
> > Mauro
> >   
> 



Thanks,
Mauro

Re: [RFCv11 PATCH 05/29] media-request: add request ioctls

[Hans Verkuil]

On 04/12/2018 12:40 PM, Sakari Ailus wrote:
> Hi Mauro,
> 
> On Tue, Apr 10, 2018 at 07:57:56AM -0300, Mauro Carvalho Chehab wrote:
>> Em Mon,  9 Apr 2018 16:20:02 +0200
>> Hans Verkuil <hverkuil@xs4all.nl> escreveu:
>>
>>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>>
>>> Implement the MEDIA_REQUEST_IOC_QUEUE and MEDIA_REQUEST_IOC_REINIT
>>> ioctls.
>>>
>>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>>> ---
>>>  drivers/media/media-request.c | 80 +++++++++++++++++++++++++++++++++++++++++--
>>>  1 file changed, 78 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>>> index dffc290e4ada..27739ff7cb09 100644
>>> --- a/drivers/media/media-request.c
>>> +++ b/drivers/media/media-request.c
>>> @@ -118,10 +118,86 @@ static unsigned int media_request_poll(struct file *filp,
>>>  	return 0;
>>>  }
>>>  
>>> +static long media_request_ioctl_queue(struct media_request *req)
>>> +{
>>> +	struct media_device *mdev = req->mdev;
>>> +	unsigned long flags;
>>> +	int ret = 0;
>>> +
>>> +	dev_dbg(mdev->dev, "request: queue %s\n", req->debug_str);
>>> +
>>> +	spin_lock_irqsave(&req->lock, flags);
>>> +	if (req->state != MEDIA_REQUEST_STATE_IDLE) {
>>> +		dev_dbg(mdev->dev,
>>> +			"request: unable to queue %s, request in state %s\n",
>>> +			req->debug_str, media_request_state_str(req->state));
>>> +		spin_unlock_irqrestore(&req->lock, flags);
>>> +		return -EINVAL;
>>> +	}
>>> +	req->state = MEDIA_REQUEST_STATE_QUEUEING;
>>> +
>>> +	spin_unlock_irqrestore(&req->lock, flags);
>>> +
>>> +	/*
>>> +	 * Ensure the request that is validated will be the one that gets queued
>>> +	 * next by serialising the queueing process.
>>> +	 */
>>> +	mutex_lock(&mdev->req_queue_mutex);
>>
>> The locking here seems really weird. IMHO, it should lock before
>> touching state, as otherwise race conditions may happen.
>>
>> As I suggested before, I would use an atomic type for state, and get rid
>> of the spin lock (as it seems that it is meant to be used just
>> for state).
> 
> The new request state depends on the old state; I don't think you can
> meaningfully do that using the atomic API.
> 
>>
>>> +
>>> +	ret = mdev->ops->req_queue(req);
>>> +	spin_lock_irqsave(&req->lock, flags);
>>> +	req->state = ret ? MEDIA_REQUEST_STATE_IDLE : MEDIA_REQUEST_STATE_QUEUED;
>>> +	spin_unlock_irqrestore(&req->lock, flags);
>>> +	mutex_unlock(&mdev->req_queue_mutex);
>>> +
>>
>> Here, you have both mutex and spin locked. This is a strong indication
>> that locks are not well designed, are you're using two different locks
>> to protect the same data.
> 
> Not the same, it's different data. What is no longer visible in this
> patchset is how request objects are referenced in a request. Effectively
> that part is missing altogether. It will be needed when adding support for
> objects that are not managed through the V4L2 framework such as pixel
> formats or selection rectangles.
> 
> You could move the serialisation of queueing requests to drivers altogether
> but I don't think that'd be a wise choice: the device state at request
> queue head will need to be maintained so that queued requests can be
> validated against it (right now validation is embedded in queueing).
> Failing validation will result into restoring the previous state.

req_queue_mutex doesn't protect data, it serializes the req_queue operation.

You don't want to allow req_queue being called for two different requests
at the same time: that would be a bit of a nightmare in the driver, and anyway,
requests really have to be queued one-by-one.

Actually, just serializing the req_queue ioctl is not enough, it also has to serialize
STREAMON/OFF and releasing (canceling) the vb2_queue. That's not in v11, but
will be in v12. When a queue is cancelled it needs to clean up any associated
requests, and you cannot call req_queue while that is in progress.

Regards,

	Hans

> 
> I had an implementation of that in my previous request set here:
> 
> <URL:https://www.spinics.net/lists/linux-media/msg130994.html>
> 
> We'll implement it but not yet: right now there's just a need for buffers
> and controls. Still, knowing where we're going I'd keep the mutex where it
> is.
> 
>>> +static long media_request_ioctl_reinit(struct media_request *req)
>>> +{
>>> +	struct media_device *mdev = req->mdev;
>>> +	unsigned long flags;
>>> +
>>> +	spin_lock_irqsave(&req->lock, flags);
>>> +	if (req->state != MEDIA_REQUEST_STATE_IDLE &&
>>> +	    req->state != MEDIA_REQUEST_STATE_COMPLETE) {
>>> +		dev_dbg(mdev->dev,
>>> +			"request: %s not in idle or complete state, cannot reinit\n",
>>> +			req->debug_str);
>>> +		spin_unlock_irqrestore(&req->lock, flags);
>>> +		return -EINVAL;
>>> +	}
>>> +	req->state = MEDIA_REQUEST_STATE_CLEANING;
>>> +	spin_unlock_irqrestore(&req->lock, flags);
>>> +
>>> +	media_request_clean(req);
>>> +
>>> +	spin_lock_irqsave(&req->lock, flags);
>>> +	req->state = MEDIA_REQUEST_STATE_IDLE;
>>> +	spin_unlock_irqrestore(&req->lock, flags);
>>
>> This code should be called with the mutex hold.
> 
> That's not necessary. A request which is being re-initialised was in an
> idle or complete state; nothing of that request is bound to the current
> device state in any way. Therefore the objects in that request can be
> simply thrown out.
> 
> The state change to CLEANING is there to prevent the request being e.g.
> queued during the cleaning.
> 

Re: [RFCv11 PATCH 06/29] media-request: add media_request_find

[Hans Verkuil]

On 04/12/2018 02:08 PM, Sakari Ailus wrote:
> Hi Hans,
> 
> On Mon, Apr 09, 2018 at 04:20:03PM +0200, Hans Verkuil wrote:
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> Add media_request_find() to find a request based on the file
>> descriptor.
>>
>> The caller has to call media_request_put() for the returned
>> request since this function increments the refcount.
>>
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/media-request.c | 47 +++++++++++++++++++++++++++++++++++++++++++
>>  include/media/media-request.h | 10 +++++++++
>>  2 files changed, 57 insertions(+)
>>
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> index 27739ff7cb09..02b620c81de5 100644
>> --- a/drivers/media/media-request.c
>> +++ b/drivers/media/media-request.c
>> @@ -207,6 +207,53 @@ static const struct file_operations request_fops = {
>>  	.release = media_request_close,
>>  };
>>  
>> +/**
>> + * media_request_find - Find a request based on the file descriptor
>> + * @mdev: The media device
>> + * @request: The request file handle
>> + *
>> + * Find and return the request associated with the given file descriptor, or
>> + * an error if no such request exists.
>> + *
>> + * When the function returns a request it increases its reference count. The
>> + * caller is responsible for releasing the reference by calling
>> + * media_request_put() on the request.
>> + */
>> +struct media_request *
>> +media_request_find(struct media_device *mdev, int request_fd)
>> +{
>> +	struct file *filp;
>> +	struct media_request *req;
>> +
>> +	if (!mdev || !mdev->ops || !mdev->ops->req_queue)
>> +		return ERR_PTR(-EPERM);
>> +
>> +	filp = fget(request_fd);
>> +	if (!filp)
>> +		return ERR_PTR(-ENOENT);
>> +
>> +	if (filp->f_op != &request_fops)
>> +		goto err_fput;
>> +	req = filp->private_data;
>> +	media_request_get(req);
> 
> As you're holding a reference to filp, you also indirectly have reference
> to ref --- you otherwise couldn't access file->private_data either.
> 
> Therefore you could move the check for the right mdev before getting a
> reference to req, thus simplifying error handling as a result.

True. Done.

Regards,

	Hans

> 
>> +
>> +	if (req->mdev != mdev)
>> +		goto err_kref_put;
>> +
>> +	fput(filp);
>> +
>> +	return req;
>> +
>> +err_kref_put:
>> +	media_request_put(req);
>> +
>> +err_fput:
>> +	fput(filp);
>> +
>> +	return ERR_PTR(-ENOENT);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_find);
>> +
>>  int media_request_alloc(struct media_device *mdev,
>>  			struct media_request_alloc *alloc)
>>  {
> 

Re: [RFCv11 PATCH 07/29] media-request: add media_request_object_find

[Hans Verkuil]

On 04/12/2018 02:23 PM, Sakari Ailus wrote:
> Hi Hans,
> 
> On Mon, Apr 09, 2018 at 04:20:04PM +0200, Hans Verkuil wrote:
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> Add media_request_object_find to find a request object inside a
>> request based on ops and/or priv values.
>>
>> Objects of the same type (vb2 buffer, control handler) will have
>> the same ops value. And objects that refer to the same 'parent'
>> object (e.g. the v4l2_ctrl_handler that has the current driver
>> state) will have the same priv value.
>>
>> The caller has to call media_request_object_put() for the returned
>> object since this function increments the refcount.
>>
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/media-request.c | 26 ++++++++++++++++++++++++++
>>  include/media/media-request.h | 25 +++++++++++++++++++++++++
>>  2 files changed, 51 insertions(+)
>>
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> index 02b620c81de5..415f7e31019d 100644
>> --- a/drivers/media/media-request.c
>> +++ b/drivers/media/media-request.c
>> @@ -322,6 +322,32 @@ static void media_request_object_release(struct kref *kref)
>>  	obj->ops->release(obj);
>>  }
>>  
>> +struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +			  const struct media_request_object_ops *ops,
>> +			  void *priv)
>> +{
>> +	struct media_request_object *obj;
>> +	struct media_request_object *found = NULL;
>> +	unsigned long flags;
>> +
>> +	if (!ops && !priv)
>> +		return NULL;
>> +
>> +	spin_lock_irqsave(&req->lock, flags);
>> +	list_for_each_entry(obj, &req->objects, list) {
>> +		if ((!ops || obj->ops == ops) &&
>> +		    (!priv || obj->priv == priv)) {
> 
> I think I may have given the similar comment on an earlier version but I
> don't remember a reply to that at least --- is there a case where the
> object wouldn't be uniquely identified by req together with priv?
> 
> It'd be still prudent to check that ops match, but failing that check, I'd
> add WARN_ON() because it's a sign something is wrong.

Currently both ops and priv are always non-NULL, so I've made that the
requirement and call WARN_ON if either is NULL.

Regards,

	Hans

> 
>> +			media_request_object_get(obj);
>> +			found = obj;
>> +			break;
>> +		}
>> +	}
>> +	spin_unlock_irqrestore(&req->lock, flags);
>> +	return found;
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_find);
>> +
>>  void media_request_object_put(struct media_request_object *obj)
>>  {
>>  	kref_put(&obj->kref, media_request_object_release);
>> diff --git a/include/media/media-request.h b/include/media/media-request.h
>> index 033697d493cd..ea990c8f76bc 100644
>> --- a/include/media/media-request.h
>> +++ b/include/media/media-request.h
>> @@ -130,6 +130,23 @@ static inline void media_request_object_get(struct media_request_object *obj)
>>   */
>>  void media_request_object_put(struct media_request_object *obj);
>>  
>> +/**
>> + * media_request_object_find - Find an object in a request
>> + *
>> + * @ops: Find an object with this ops value, may be NULL.
>> + * @priv: Find an object with this priv value, may be NULL.
>> + *
>> + * At least one of @ops and @priv must be non-NULL. If one of
>> + * these is NULL, then skip checking for that field.
>> + *
>> + * Returns NULL if not found or the object (the refcount is increased
>> + * in that case).
>> + */
>> +struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +			  const struct media_request_object_ops *ops,
>> +			  void *priv);
>> +
>>  /**
>>   * media_request_object_init - Initialise a media request object
>>   *
>> @@ -162,6 +179,14 @@ static inline void media_request_object_put(struct media_request_object *obj)
>>  {
>>  }
>>  
>> +static inline struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +			  const struct media_request_object_ops *ops,
>> +			  void *priv)
>> +{
>> +	return NULL;
>> +}
>> +
>>  static inline void media_request_object_init(struct media_request_object *obj)
>>  {
>>  	obj->ops = NULL;
> 

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Mauro Carvalho Chehab]

Em Mon, 23 Apr 2018 14:23:28 +0200
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> >> +	spin_lock_irqsave(&req->lock, flags);
> >> +	state = req->state;
> >> +	spin_unlock_irqrestore(&req->lock, flags);  
> > 
> > IMO, it would be better to use an atomic var for state, having a
> > lockless access to it.  
> 
> In most cases I need to do more than just change the state. I don't see
> enough benefits from using an atomic.

On several cases, it is doing this check without changing the state.
Also, the IRQ logic seems to require changing the status, and it can't
use a mutex_lock while there.

Anyway, I'll review the locking at the next version.

> >> +	get_task_comm(comm, current);
> >> +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> >> +		 comm, fd);  
> > 
> > Not sure if it is a good idea to store the task that allocated
> > the request. While it makes sense for the dev_dbg() below, it
> > may not make sense anymore on other dev_dbg() you would be
> > using it.  
> 
> This is actually copied from Laurent's code. I'm not sure either. I don't
> have enough experience with this yet to tell whether or not this is useful.

I remember a long time ago I considered using current task for
debugging. I ended by giving up, as separate tasks/threads could
be used internally by V4L2 apps. The same applies here. Also,
it seems overkill to me to always call get_task_comm() here, just
in case someone would ever enable a debug.

If the user wants the task, he can just enable it in realtime via
debugfs.

> >> +#ifdef CONFIG_MEDIA_CONTROLLER
> >> +static inline void media_request_object_get(struct media_request_object *obj)
> >> +{
> >> +	kref_get(&obj->kref);
> >> +}  
> > 
> > Why do you need a function? Just use kref_get() were needed, instead of
> > aliasing it for no good reason.  
> 
> Because that's what everyone does? That way you have nicely balanced
> media_request_object_get/put functions. Easier to review.

IMHO, having a kref_get()/kref_put() is a way easier to review, as
I know exactly where objects can be freed, but I can live with that.

Thanks,
Mauro

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Hans Verkuil]

On 04/17/2018 06:35 AM, Alexandre Courbot wrote:
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Implement the core of the media request processing.
> 
>> Drivers can bind request objects to a request. These objects
>> can then be marked completed if the driver finished using them,
>> or just be unbound if the results do not need to be kept (e.g.
>> in the case of buffers).
> 
>> Once all objects that were added are either unbound or completed,
>> the request is marked 'complete' and a POLLPRI signal is sent
>> via poll.
> 
>> Both requests and request objects are refcounted.
> 
>> While a request is queued its refcount is incremented (since it
>> is in use by a driver). Once it is completed the refcount is
>> decremented. When the user closes the request file descriptor
>> the refcount is also decremented. Once it reaches 0 all request
>> objects in the request are unbound and put() and the request
>> itself is freed.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/media-request.c | 284
> +++++++++++++++++++++++++++++++++++++++++-
>>   include/media/media-request.h | 156 +++++++++++++++++++++++
>>   2 files changed, 439 insertions(+), 1 deletion(-)
> 
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> index ead78613fdbe..dffc290e4ada 100644
>> --- a/drivers/media/media-request.c
>> +++ b/drivers/media/media-request.c
>> @@ -16,8 +16,290 @@
>>   #include <media/media-device.h>
>>   #include <media/media-request.h>
> 
>> +static const char * const request_state[] = {
>> +       "idle",
>> +       "queueing",
>> +       "queued",
>> +       "complete",
>> +       "cleaning",
>> +};
>> +
>> +static const char *
>> +media_request_state_str(enum media_request_state state)
>> +{
>> +       if (WARN_ON(state >= ARRAY_SIZE(request_state)))
>> +               return "unknown";
>> +       return request_state[state];
>> +}
>> +
>> +static void media_request_clean(struct media_request *req)
>> +{
>> +       struct media_request_object *obj, *obj_safe;
>> +
>> +       WARN_ON(req->state != MEDIA_REQUEST_STATE_CLEANING);
>> +
>> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list) {
>> +               media_request_object_unbind(obj);
>> +               media_request_object_put(obj);
>> +       }
>> +
>> +       req->num_incomplete_objects = 0;
>> +       wake_up_interruptible(&req->poll_wait);
>> +}
>> +
>> +static void media_request_release(struct kref *kref)
>> +{
>> +       struct media_request *req =
>> +               container_of(kref, struct media_request, kref);
>> +       struct media_device *mdev = req->mdev;
>> +       unsigned long flags;
>> +
>> +       dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       req->state = MEDIA_REQUEST_STATE_CLEANING;
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +
>> +       media_request_clean(req);
>> +
>> +       if (mdev->ops->req_free)
>> +               mdev->ops->req_free(req);
>> +       else
>> +               kfree(req);
> 
> Adding a third (different) opinion on this: if requests are to be embedded
> into
> other struct, then shouldn't we mandate an implementation for req_free
> anyway?
> Making it optional sounds error-prone to me.

I've added a 'if (WARN_ON(!req_alloc ^ !req_free)) return -ENOMEM;' check
in the alloc function: either both are set or both are NULL.

> 
>> +}
>> +
>> +void media_request_put(struct media_request *req)
>> +{
>> +       kref_put(&req->kref, media_request_release);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_put);
>> +
>> +void media_request_cancel(struct media_request *req)
>> +{
>> +       struct media_request_object *obj, *obj_safe;
>> +
>> +       if (req->state != MEDIA_REQUEST_STATE_QUEUED)
>> +               return;
>> +
>> +       list_for_each_entry_safe(obj, obj_safe, &req->objects, list)
>> +               if (obj->ops->cancel)
>> +                       obj->ops->cancel(obj);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_cancel);
>> +
>> +static int media_request_close(struct inode *inode, struct file *filp)
>> +{
>> +       struct media_request *req = filp->private_data;
>> +
>> +       media_request_put(req);
>> +       return 0;
>> +}
>> +
>> +static unsigned int media_request_poll(struct file *filp,
>> +                                      struct poll_table_struct *wait)
>> +{
>> +       struct media_request *req = filp->private_data;
>> +       unsigned long flags;
>> +       enum media_request_state state;
>> +
>> +       if (!(poll_requested_events(wait) & POLLPRI))
>> +               return 0;
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       state = req->state;
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +
>> +       if (state == MEDIA_REQUEST_STATE_COMPLETE)
>> +               return POLLPRI;
>> +       if (state == MEDIA_REQUEST_STATE_IDLE)
>> +               return POLLERR;
>> +
>> +       poll_wait(filp, &req->poll_wait, wait);
>> +       return 0;
>> +}
>> +
>> +static long media_request_ioctl(struct file *filp, unsigned int cmd,
>> +                               unsigned long __arg)
>> +{
>> +       return -ENOIOCTLCMD;
>> +}
>> +
>> +static const struct file_operations request_fops = {
>> +       .owner = THIS_MODULE,
>> +       .poll = media_request_poll,
>> +       .unlocked_ioctl = media_request_ioctl,
>> +       .release = media_request_close,
>> +};
>> +
>>   int media_request_alloc(struct media_device *mdev,
>>                          struct media_request_alloc *alloc)
>>   {
>> -       return -ENOMEM;
>> +       struct media_request *req;
>> +       struct file *filp;
>> +       char comm[TASK_COMM_LEN];
>> +       int fd;
>> +       int ret;
>> +
>> +       fd = get_unused_fd_flags(O_CLOEXEC);
>> +       if (fd < 0)
>> +               return fd;
>> +
>> +       filp = anon_inode_getfile("request", &request_fops, NULL,
> O_CLOEXEC);
>> +       if (IS_ERR(filp)) {
>> +               ret = PTR_ERR(filp);
>> +               goto err_put_fd;
>> +       }
>> +
>> +       if (mdev->ops->req_alloc)
>> +               req = mdev->ops->req_alloc(mdev);
>> +       else
>> +               req = kzalloc(sizeof(*req), GFP_KERNEL);
>> +       if (!req) {
>> +               ret = -ENOMEM;
>> +               goto err_fput;
>> +       }
>> +
>> +       filp->private_data = req;
>> +       req->mdev = mdev;
>> +       req->state = MEDIA_REQUEST_STATE_IDLE;
>> +       req->num_incomplete_objects = 0;
>> +       kref_init(&req->kref);
>> +       INIT_LIST_HEAD(&req->objects);
>> +       spin_lock_init(&req->lock);
>> +       init_waitqueue_head(&req->poll_wait);
>> +
>> +       alloc->fd = fd;
>> +
>> +       get_task_comm(comm, current);
>> +       snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
>> +                comm, fd);
>> +       dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
>> +
>> +       fd_install(fd, filp);
>> +
>> +       return 0;
>> +
>> +err_fput:
>> +       fput(filp);
>> +
>> +err_put_fd:
>> +       put_unused_fd(fd);
>> +
>> +       return ret;
>> +}
>> +
>> +static void media_request_object_release(struct kref *kref)
>> +{
>> +       struct media_request_object *obj =
>> +               container_of(kref, struct media_request_object, kref);
>> +       struct media_request *req = obj->req;
>> +
>> +       if (req)
>> +               media_request_object_unbind(obj);
>> +       obj->ops->release(obj);
>> +}
>> +
>> +void media_request_object_put(struct media_request_object *obj)
>> +{
>> +       kref_put(&obj->kref, media_request_object_release);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_put);
>> +
>> +void media_request_object_init(struct media_request_object *obj)
>> +{
>> +       obj->ops = NULL;
>> +       obj->req = NULL;
>> +       obj->priv = NULL;
>> +       obj->completed = false;
>> +       INIT_LIST_HEAD(&obj->list);
>> +       kref_init(&obj->kref);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_init);
>> +
>> +void media_request_object_bind(struct media_request *req,
>> +                              const struct media_request_object_ops *ops,
> 
> As suggested elsewhere, I think the ops would better be set at init() time.
> We
> probably don't want them to change during the object's lifetime. Probably
> the
> same for priv.

I'm undecided on this. The reason it's done in bind() is that that's when these
values become important since they are used to find the object in a request.

For now I prefer to pass this in bind().

> 
> Actually, looking closer at how these functions are used, I can always see
> media_request_object_init() being called in pair with
> media_request_object_bind(). Same thing with media_request_object_unbind()
> and
> media_request_object_put(). It looks like media_request_object_init() and
> media_request_object_bind() could be merged, and
> media_request_object_unbind()
> could include a call to media_request_object_put().

I think it is unexpected if unbind would call put(). bind/unbind do not change
the object's refcount, and I prefer to keep it like that. In fact, an object
can be put() and released while still bound to a request (the release will
automatically unbind the object from the request in that case).

> 
>> +                              void *priv,
>> +                              struct media_request_object *obj)
>> +{
>> +       unsigned long flags;
>> +
>> +       if (WARN_ON(!ops->release || !ops->cancel))
>> +               return;
>> +
>> +       obj->req = req;
>> +       obj->ops = ops;
>> +       obj->priv = priv;
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       if (WARN_ON(req->state != MEDIA_REQUEST_STATE_IDLE))
>> +               goto unlock;
>> +       list_add_tail(&obj->list, &req->objects);
>> +       req->num_incomplete_objects++;
>> +unlock:
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_bind);
>> +
>> +void media_request_object_unbind(struct media_request_object *obj)
>> +{
>> +       struct media_request *req = obj->req;
>> +       unsigned long flags;
>> +       bool completed = false;
>> +
>> +       if (!req)
>> +               return;
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       list_del(&obj->list);
>> +       obj->req = NULL;
>> +
>> +       if (req->state == MEDIA_REQUEST_STATE_COMPLETE ||
>> +           req->state == MEDIA_REQUEST_STATE_CLEANING)
>> +               goto unlock;
>> +
>> +       if (WARN_ON(req->state == MEDIA_REQUEST_STATE_QUEUEING))
>> +               goto unlock;
>> +
>> +       if (WARN_ON(!req->num_incomplete_objects))
>> +               goto unlock;
>> +
>> +       req->num_incomplete_objects--;
>> +       if (req->state == MEDIA_REQUEST_STATE_QUEUED &&
>> +           !req->num_incomplete_objects) {
>> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
>> +               completed = true;
>> +               wake_up_interruptible(&req->poll_wait);
>> +       }
>> +unlock:
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +       if (obj->ops->unbind)
>> +               obj->ops->unbind(obj);
>> +       if (completed)
>> +               media_request_put(req);
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_unbind);
>> +
>> +void media_request_object_complete(struct media_request_object *obj)
>> +{
>> +       struct media_request *req = obj->req;
>> +       unsigned long flags;
>> +       bool completed = false;
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       if (obj->completed)
>> +               goto unlock;
>> +       obj->completed = true;
>> +       if (WARN_ON(!req->num_incomplete_objects) ||
>> +           WARN_ON(req->state != MEDIA_REQUEST_STATE_QUEUED))
>> +               goto unlock;
>> +
>> +       if (!--req->num_incomplete_objects) {
>> +               req->state = MEDIA_REQUEST_STATE_COMPLETE;
>> +               wake_up_interruptible(&req->poll_wait);
>> +               completed = true;
>> +       }
>> +unlock:
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +       if (completed)
>> +               media_request_put(req);
>>   }
>> +EXPORT_SYMBOL_GPL(media_request_object_complete);
>> diff --git a/include/media/media-request.h b/include/media/media-request.h
>> index dae3eccd9aa7..082c3cae04ac 100644
>> --- a/include/media/media-request.h
>> +++ b/include/media/media-request.h
>> @@ -16,7 +16,163 @@
> 
>>   #include <media/media-device.h>
> 
>> +enum media_request_state {
>> +       MEDIA_REQUEST_STATE_IDLE,
>> +       MEDIA_REQUEST_STATE_QUEUEING,
>> +       MEDIA_REQUEST_STATE_QUEUED,
>> +       MEDIA_REQUEST_STATE_COMPLETE,
>> +       MEDIA_REQUEST_STATE_CLEANING,
>> +};
>> +
>> +struct media_request_object;
>> +
>> +/**
>> + * struct media_request - Media device request
>> + * @mdev: Media device this request belongs to
>> + * @kref: Reference count
>> + * @debug_prefix: Prefix for debug messages (process name:fd)
>> + * @state: The state of the request
>> + * @objects: List of @struct media_request_object request objects
>> + * @num_objects: The number objects in the request
>> + * @num_completed_objects: The number of completed objects in the request
>> + * @poll_wait: Wait queue for poll
>> + * @lock: Serializes access to this struct
>> + */
>> +struct media_request {
>> +       struct media_device *mdev;
>> +       struct kref kref;
> 
> I thought we wanted to use a struct file to manage the request reference
> count?
> That was even your idea IIRC.

It was my idea, but it didn't work :-)

I forgot the details, but trying to use struct file became very messy.

> 
>> +       char debug_str[TASK_COMM_LEN + 11];
>> +       enum media_request_state state;
> 
> If possible, using an atomic here would probably simplify locking
> considerably.
> Might require adding a few extra intermediate states though.

You still need the spinlock for more complicated processing. So you end
up with some functions that use the atomic access, and others that still
need to lock/unlock a spinlock. I find it more consistent to use the
same synchronization mechanism in all cases.

> 
>> +       struct list_head objects;
>> +       unsigned int num_incomplete_objects;
> 
> Same here. I did that in one of my previous versions (albeit with a bit
> mask)
> and it did help simplify locking.
> 
> By doing both, I think the only operation that needs lock protection would
> be
> manipulation of the objects list (and maybe serialization of ops on
> individual
> objects, which could be done either by the same request lock or individual
> objects locks).

I might change my opinion on this later, but for now I prefer to keep it as-is.

> 
>> +       struct wait_queue_head poll_wait;
>> +       spinlock_t lock;
>> +};
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +
>> +static inline void media_request_get(struct media_request *req)
>> +{
>> +       kref_get(&req->kref);
>> +}
> 
> I personally think it is a good idea to have this function - makes the
> intent
> clearer than a direct kref_get.
> 
>> +
>> +void media_request_put(struct media_request *req);
>> +void media_request_cancel(struct media_request *req);
>> +
>>   int media_request_alloc(struct media_device *mdev,
>>                          struct media_request_alloc *alloc);
>> +#else
>> +static inline void media_request_get(struct media_request *req)
>> +{
>> +}
>> +
>> +static inline void media_request_put(struct media_request *req)
>> +{
>> +}
>> +
>> +static inline void media_request_cancel(struct media_request *req)
>> +{
>> +}
>> +
>> +#endif
>> +
>> +struct media_request_object_ops {
>> +       int (*prepare)(struct media_request_object *object);
>> +       void (*unprepare)(struct media_request_object *object);
>> +       void (*queue)(struct media_request_object *object);
>> +       void (*unbind)(struct media_request_object *object);
>> +       void (*cancel)(struct media_request_object *object);
>> +       void (*release)(struct media_request_object *object);
>> +};
>> +
>> +/**
>> + * struct media_request_object - An opaque object that belongs to a media
>> + *                              request
>> + *
>> + * @priv: object's priv pointer
>> + * @list: List entry of the object for @struct media_request
>> + * @kref: Reference count of the object, acquire before releasing
> req->lock
>> + *
>> + * An object related to the request. This struct is embedded in the
>> + * larger object data.
>> + */
>> +struct media_request_object {
>> +       const struct media_request_object_ops *ops;
>> +       void *priv;
> 
> Is the priv member needed? Since it seems that we are embedding this struct
> into other structs and setting priv to these objects address, so we could
> just
> use container_of() here. Although this may still be required for
> media_request_object_find().

It's specifically for object_find(). In addition, priv does not point to
something in this object, so you can't use container_of.

I.e.: for buffer objects priv points to the vb2_queue and for control objects
priv points to the main control handler (that is the handler representing
the current hardware state).

> 
>> +       struct media_request *req;
>> +       struct list_head list;
>> +       struct kref kref;
>> +       bool completed;
>> +};
>> +
>> +#ifdef CONFIG_MEDIA_CONTROLLER
>> +static inline void media_request_object_get(struct media_request_object
> *obj)
>> +{
>> +       kref_get(&obj->kref);
>> +}
>> +
>> +/**
>> + * media_request_object_put - Put a media request object
>> + *
>> + * @obj: The object
>> + *
>> + * Put a media request object. Once all references are gone, the
>> + * object's memory is released.
>> + */
>> +void media_request_object_put(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_init - Initialise a media request object
>> + *
>> + * Initialise a media request object. The object will be released using
> the
>> + * release callback of the ops once it has no references (this function
>> + * initialises references to one).
>> + */
>> +void media_request_object_init(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_bind - Bind a media request object to a request
>> + */
>> +void media_request_object_bind(struct media_request *req,
>> +                              const struct media_request_object_ops *ops,
>> +                              void *priv,
>> +                              struct media_request_object *obj);
>> +
>> +void media_request_object_unbind(struct media_request_object *obj);
>> +
>> +/**
>> + * media_request_object_complete - Mark the media request object as
> complete
>> + */
>> +void media_request_object_complete(struct media_request_object *obj);
>> +#else
>> +static inline void media_request_object_get(struct media_request_object
> *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_put(struct media_request_object
> *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_init(struct media_request_object
> *obj)
>> +{
>> +       obj->ops = NULL;
>> +       obj->req = NULL;
>> +}
>> +
>> +static inline void media_request_object_bind(struct media_request *req,
>> +                              const struct media_request_object_ops *ops,
>> +                              void *priv,
>> +                              struct media_request_object *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_unbind(struct
> media_request_object *obj)
>> +{
>> +}
>> +
>> +static inline void media_request_object_complete(struct
> media_request_object *obj)
>> +{
>> +}
>> +#endif
> 
>>   #endif
>> --
>> 2.16.3

Regards,

	Hans

Re: [RFCv11 PATCH 07/29] media-request: add media_request_object_find

[Hans Verkuil]

On 04/17/2018 06:36 AM, Alexandre Courbot wrote:
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Add media_request_object_find to find a request object inside a
>> request based on ops and/or priv values.
> 
>> Objects of the same type (vb2 buffer, control handler) will have
>> the same ops value. And objects that refer to the same 'parent'
>> object (e.g. the v4l2_ctrl_handler that has the current driver
>> state) will have the same priv value.
> 
>> The caller has to call media_request_object_put() for the returned
>> object since this function increments the refcount.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/media-request.c | 26 ++++++++++++++++++++++++++
>>   include/media/media-request.h | 25 +++++++++++++++++++++++++
>>   2 files changed, 51 insertions(+)
> 
>> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
>> index 02b620c81de5..415f7e31019d 100644
>> --- a/drivers/media/media-request.c
>> +++ b/drivers/media/media-request.c
>> @@ -322,6 +322,32 @@ static void media_request_object_release(struct kref
> *kref)
>>          obj->ops->release(obj);
>>   }
> 
>> +struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +                         const struct media_request_object_ops *ops,
>> +                         void *priv)
>> +{
>> +       struct media_request_object *obj;
>> +       struct media_request_object *found = NULL;
>> +       unsigned long flags;
>> +
>> +       if (!ops && !priv)
>> +               return NULL;
>> +
>> +       spin_lock_irqsave(&req->lock, flags);
>> +       list_for_each_entry(obj, &req->objects, list) {
>> +               if ((!ops || obj->ops == ops) &&
>> +                   (!priv || obj->priv == priv)) {
>> +                       media_request_object_get(obj);
>> +                       found = obj;
>> +                       break;
>> +               }
>> +       }
>> +       spin_unlock_irqrestore(&req->lock, flags);
>> +       return found;
>> +}
>> +EXPORT_SYMBOL_GPL(media_request_object_find);
>> +
>>   void media_request_object_put(struct media_request_object *obj)
>>   {
>>          kref_put(&obj->kref, media_request_object_release);
>> diff --git a/include/media/media-request.h b/include/media/media-request.h
>> index 033697d493cd..ea990c8f76bc 100644
>> --- a/include/media/media-request.h
>> +++ b/include/media/media-request.h
>> @@ -130,6 +130,23 @@ static inline void media_request_object_get(struct
> media_request_object *obj)
>>    */
>>   void media_request_object_put(struct media_request_object *obj);
> 
>> +/**
>> + * media_request_object_find - Find an object in a request
>> + *
>> + * @ops: Find an object with this ops value, may be NULL.
>> + * @priv: Find an object with this priv value, may be NULL.
>> + *
>> + * At least one of @ops and @priv must be non-NULL. If one of
>> + * these is NULL, then skip checking for that field.
>> + *
>> + * Returns NULL if not found or the object (the refcount is increased
>> + * in that case).
>> + */
>> +struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +                         const struct media_request_object_ops *ops,
>> +                         void *priv);
> 
> Mm, that signature is weird. I don't yet know how this function is going to
> be
> called, but wouldn't priv be enough? If we look for ops, this means we are
> looking for the first object of a given class (IIUC the class/objects
> mechanism
> here), and I cannot see where we would want to do that.

This allows you to associate objects of different types with the same
priv pointer. E.g. right now only buffers objects are associated with
a vb2_queue. But what if you want to associate other objects with the vb2_queue
as well? That's what the ops is for.

This will almost certainly be needed for complex video pipelines.

If I am wrong, then this can be removed in the future.

Regards,

	Hans

> 
>> +
>>   /**
>>    * media_request_object_init - Initialise a media request object
>>    *
>> @@ -162,6 +179,14 @@ static inline void media_request_object_put(struct
> media_request_object *obj)
>>   {
>>   }
> 
>> +static inline struct media_request_object *
>> +media_request_object_find(struct media_request *req,
>> +                         const struct media_request_object_ops *ops,
>> +                         void *priv)
>> +{
>> +       return NULL;
>> +}
>> +
>>   static inline void media_request_object_init(struct media_request_object
> *obj)
>>   {
>>          obj->ops = NULL;
>> --
>> 2.16.3

Re: [RFCv11 PATCH 20/29] videobuf2-v4l2: integrate with media requests

[Hans Verkuil]

On 04/17/2018 06:36 AM, Alexandre Courbot wrote:
> On Mon, Apr 9, 2018 at 11:21 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> This implements the V4L2 part of the request support. The main
>> change is that vb2_qbuf and vb2_prepare_buf now have a new
>> media_device pointer. This required changes to several drivers
>> that did not use the vb2_ioctl_qbuf/prepare_buf helper functions.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/common/videobuf2/videobuf2-v4l2.c  | 84
> ++++++++++++++++++++----
>>   drivers/media/platform/omap3isp/ispvideo.c       |  2 +-
>>   drivers/media/platform/s3c-camif/camif-capture.c |  4 +-
>>   drivers/media/platform/s5p-mfc/s5p_mfc_dec.c     |  4 +-
>>   drivers/media/platform/s5p-mfc/s5p_mfc_enc.c     |  4 +-
>>   drivers/media/platform/soc_camera/soc_camera.c   |  4 +-
>>   drivers/media/usb/uvc/uvc_queue.c                |  5 +-
>>   drivers/media/usb/uvc/uvc_v4l2.c                 |  3 +-
>>   drivers/media/usb/uvc/uvcvideo.h                 |  1 +
>>   drivers/media/v4l2-core/v4l2-mem2mem.c           |  7 +-
>>   drivers/staging/media/davinci_vpfe/vpfe_video.c  |  3 +-
>>   drivers/staging/media/omap4iss/iss_video.c       |  3 +-
>>   drivers/usb/gadget/function/uvc_queue.c          |  2 +-
>>   include/media/videobuf2-v4l2.h                   | 12 +++-
>>   14 files changed, 106 insertions(+), 32 deletions(-)
> 
>> diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c
> b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> index b8d370b97cca..73c1fd4da58a 100644
>> --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
>> @@ -25,6 +25,7 @@
>>   #include <linux/kthread.h>
> 
>>   #include <media/v4l2-dev.h>
>> +#include <media/v4l2-device.h>
>>   #include <media/v4l2-fh.h>
>>   #include <media/v4l2-event.h>
>>   #include <media/v4l2-common.h>
>> @@ -40,10 +41,12 @@ module_param(debug, int, 0644);
>>                          pr_info("vb2-v4l2: %s: " fmt, __func__, ## arg); \
>>          } while (0)
> 
>> -/* Flags that are set by the vb2 core */
>> +/* Flags that are set by us */
>>   #define V4L2_BUFFER_MASK_FLAGS (V4L2_BUF_FLAG_MAPPED |
> V4L2_BUF_FLAG_QUEUED | \
>>                                   V4L2_BUF_FLAG_DONE | V4L2_BUF_FLAG_ERROR
> | \
>>                                   V4L2_BUF_FLAG_PREPARED | \
>> +                                V4L2_BUF_FLAG_IN_REQUEST | \
>> +                                V4L2_BUF_FLAG_REQUEST_FD | \
>>                                   V4L2_BUF_FLAG_TIMESTAMP_MASK)
>>   /* Output buffer flags that should be passed on to the driver */
>>   #define V4L2_BUFFER_OUT_FLAGS  (V4L2_BUF_FLAG_PFRAME |
> V4L2_BUF_FLAG_BFRAME | \
>> @@ -318,13 +321,17 @@ static int vb2_fill_vb2_v4l2_buffer(struct
> vb2_buffer *vb, struct v4l2_buffer *b
>>          return 0;
>>   }
> 
>> -static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct
> v4l2_buffer *b,
>> -                                   const char *opname)
>> +static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct
> media_device *mdev,
>> +                                   struct v4l2_buffer *b,
>> +                                   const char *opname,
>> +                                   struct media_request **p_req)
>>   {
>> +       struct media_request *req;
>>          struct vb2_v4l2_buffer *vbuf;
>>          struct vb2_buffer *vb;
>>          int ret;
> 
>> +       *p_req = NULL;
>>          if (b->type != q->type) {
>>                  dprintk(1, "%s: invalid buffer type\n", opname);
>>                  return -EINVAL;
>> @@ -354,7 +361,38 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue
> *q, struct v4l2_buffer *b,
> 
>>          /* Copy relevant information provided by the userspace */
>>          memset(vbuf->planes, 0, sizeof(vbuf->planes[0]) * vb->num_planes);
>> -       return vb2_fill_vb2_v4l2_buffer(vb, b);
>> +       ret = vb2_fill_vb2_v4l2_buffer(vb, b);
>> +       if (ret)
>> +               return ret;
>> +
>> +       if (!(b->flags & V4L2_BUF_FLAG_REQUEST_FD))
>> +               return 0;
>> +
>> +       if (vb->state != VB2_BUF_STATE_DEQUEUED) {
>> +               dprintk(1, "%s: buffer is not in dequeued state\n",
> opname);
>> +               return -EINVAL;
>> +       }
>> +
>> +       if (b->request_fd < 0) {
>> +               dprintk(1, "%s: request_fd < 0\n", opname);
>> +               return -EINVAL;
>> +       }
>> +
>> +       req = media_request_find(mdev, b->request_fd);
>> +       if (IS_ERR(req)) {
>> +               dprintk(1, "%s: invalid request_fd\n", opname);
>> +               return PTR_ERR(req);
>> +       }
>> +
>> +       if (req->state != MEDIA_REQUEST_STATE_IDLE) {
>> +               dprintk(1, "%s: request is not idle\n", opname);
>> +               media_request_put(req);
>> +               return -EBUSY;
>> +       }
>> +
>> +       *p_req = req;
>> +
>> +       return 0;
>>   }
> 
>>   /*
>> @@ -437,6 +475,9 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb,
> void *pb)
>>          case VB2_BUF_STATE_ACTIVE:
>>                  b->flags |= V4L2_BUF_FLAG_QUEUED;
>>                  break;
>> +       case VB2_BUF_STATE_IN_REQUEST:
>> +               b->flags |= V4L2_BUF_FLAG_IN_REQUEST;
>> +               break;
>>          case VB2_BUF_STATE_ERROR:
>>                  b->flags |= V4L2_BUF_FLAG_ERROR;
>>                  /* fall through */
>> @@ -455,6 +496,10 @@ static void __fill_v4l2_buffer(struct vb2_buffer
> *vb, void *pb)
> 
>>          if (vb2_buffer_in_use(q, vb))
>>                  b->flags |= V4L2_BUF_FLAG_MAPPED;
>> +       if (vb->req_obj.req) {
>> +               b->flags |= V4L2_BUF_FLAG_REQUEST_FD;
>> +               b->request_fd = -1;
>> +       }
> 
>>          if (!q->is_output &&
>>                  b->flags & V4L2_BUF_FLAG_DONE &&
>> @@ -533,8 +578,10 @@ int vb2_reqbufs(struct vb2_queue *q, struct
> v4l2_requestbuffers *req)
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_reqbufs);
> 
>> -int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b)
>> +int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> +                   struct v4l2_buffer *b)
>>   {
>> +       struct media_request *req;
>>          int ret;
> 
>>          if (vb2_fileio_is_active(q)) {
>> @@ -542,9 +589,13 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> v4l2_buffer *b)
>>                  return -EBUSY;
>>          }
> 
>> -       ret = vb2_queue_or_prepare_buf(q, b, "prepare_buf");
>> -
>> -       return ret ? ret : vb2_core_prepare_buf(q, b->index, b, NULL);
>> +       ret = vb2_queue_or_prepare_buf(q, mdev, b, "prepare_buf", &req);
>> +       if (ret)
>> +               return ret;
>> +       ret = vb2_core_prepare_buf(q, b->index, b, req);
>> +       if (req)
>> +               media_request_put(req);
>> +       return ret;
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_prepare_buf);
> 
>> @@ -602,8 +653,10 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> v4l2_create_buffers *create)
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_create_bufs);
> 
>> -int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b)
>> +int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
>> +            struct v4l2_buffer *b)
> 
> That extra mdev argument is request-specific and not very nice to have here.
> Isn't it possible instead to obtain the media_device by looking up through
> the
> queue, or at least to cache a pointer to the media_device inside vb2_queue
> when
> requests are in use? After all a queue can only be used with one
> media_device.

The media_device is only needed in videobuf2-v4l2.c. Adding it to vb2_queue
means that it is added to videobuf2-core.h, but not actually used in
videobuf2-core.c and I'm not keen on that.

Also, only qbuf and prepare_buf need the media_device, so keeping it local
to those functions seems reasonable.

Regards,

	Hans

> 
> Btw, that's also one of the reasons why I used to put request management
> data
> into its own structure instead of having it directly in media_device.
> Non-request users would wonder what is a media_device doing here. With a
> dedicated struct to manage requests, the intent is at least clear. Even if
> we
> mandate the use of media_device, it's never a bad idea to clearly sort your
> data members by purpose.
> 
>>   {
>> +       struct media_request *req;
>>          int ret;
> 
>>          if (vb2_fileio_is_active(q)) {
>> @@ -611,8 +664,13 @@ int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer
> *b)
>>                  return -EBUSY;
>>          }
> 
>> -       ret = vb2_queue_or_prepare_buf(q, b, "qbuf");
>> -       return ret ? ret : vb2_core_qbuf(q, b->index, b, NULL);
>> +       ret = vb2_queue_or_prepare_buf(q, mdev, b, "qbuf", &req);
>> +       if (ret)
>> +               return ret;
>> +       ret = vb2_core_qbuf(q, b->index, b, req);
>> +       if (req)
>> +               media_request_put(req);
>> +       return ret;
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_qbuf);
> 
>> @@ -802,7 +860,7 @@ int vb2_ioctl_prepare_buf(struct file *file, void
> *priv,
> 
>>          if (vb2_queue_is_busy(vdev, file))
>>                  return -EBUSY;
>> -       return vb2_prepare_buf(vdev->queue, p);
>> +       return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p);
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf);
> 
>> @@ -821,7 +879,7 @@ int vb2_ioctl_qbuf(struct file *file, void *priv,
> struct v4l2_buffer *p)
> 
>>          if (vb2_queue_is_busy(vdev, file))
>>                  return -EBUSY;
>> -       return vb2_qbuf(vdev->queue, p);
>> +       return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p);
>>   }
>>   EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf);
> 
>> diff --git a/drivers/media/platform/omap3isp/ispvideo.c
> b/drivers/media/platform/omap3isp/ispvideo.c
>> index bd564c2e767f..9fecbd8c6edd 100644
>> --- a/drivers/media/platform/omap3isp/ispvideo.c
>> +++ b/drivers/media/platform/omap3isp/ispvideo.c
>> @@ -940,7 +940,7 @@ isp_video_qbuf(struct file *file, void *fh, struct
> v4l2_buffer *b)
>>          int ret;
> 
>>          mutex_lock(&video->queue_lock);
>> -       ret = vb2_qbuf(&vfh->queue, b);
>> +       ret = vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
>>          mutex_unlock(&video->queue_lock);
> 
>>          return ret;
>> diff --git a/drivers/media/platform/s3c-camif/camif-capture.c
> b/drivers/media/platform/s3c-camif/camif-capture.c
>> index 9ab8e7ee2e1e..fafb6da3e804 100644
>> --- a/drivers/media/platform/s3c-camif/camif-capture.c
>> +++ b/drivers/media/platform/s3c-camif/camif-capture.c
>> @@ -941,7 +941,7 @@ static int s3c_camif_qbuf(struct file *file, void
> *priv,
>>          if (vp->owner && vp->owner != priv)
>>                  return -EBUSY;
> 
>> -       return vb2_qbuf(&vp->vb_queue, buf);
>> +       return vb2_qbuf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, buf);
>>   }
> 
>>   static int s3c_camif_dqbuf(struct file *file, void *priv,
>> @@ -979,7 +979,7 @@ static int s3c_camif_prepare_buf(struct file *file,
> void *priv,
>>                                   struct v4l2_buffer *b)
>>   {
>>          struct camif_vp *vp = video_drvdata(file);
>> -       return vb2_prepare_buf(&vp->vb_queue, b);
>> +       return vb2_prepare_buf(&vp->vb_queue, vp->vdev.v4l2_dev->mdev, b);
>>   }
> 
>>   static int s3c_camif_g_selection(struct file *file, void *priv,
>> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
> b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
>> index 5cf4d9921264..3d863e3f5798 100644
>> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
>> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c
>> @@ -632,9 +632,9 @@ static int vidioc_qbuf(struct file *file, void *priv,
> struct v4l2_buffer *buf)
>>                  return -EIO;
>>          }
>>          if (buf->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
>> -               return vb2_qbuf(&ctx->vq_src, buf);
>> +               return vb2_qbuf(&ctx->vq_src, NULL, buf);
>>          else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
>> -               return vb2_qbuf(&ctx->vq_dst, buf);
>> +               return vb2_qbuf(&ctx->vq_dst, NULL, buf);
>>          return -EINVAL;
>>   }
> 
>> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
> b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
>> index 5c0462ca9993..ed12d5d062e8 100644
>> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
>> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_enc.c
>> @@ -1621,9 +1621,9 @@ static int vidioc_qbuf(struct file *file, void
> *priv, struct v4l2_buffer *buf)
>>                          mfc_err("Call on QBUF after EOS command\n");
>>                          return -EIO;
>>                  }
>> -               return vb2_qbuf(&ctx->vq_src, buf);
>> +               return vb2_qbuf(&ctx->vq_src, NULL, buf);
>>          } else if (buf->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) {
>> -               return vb2_qbuf(&ctx->vq_dst, buf);
>> +               return vb2_qbuf(&ctx->vq_dst, NULL, buf);
>>          }
>>          return -EINVAL;
>>   }
>> diff --git a/drivers/media/platform/soc_camera/soc_camera.c
> b/drivers/media/platform/soc_camera/soc_camera.c
>> index e6787abc34ae..08adf9a79420 100644
>> --- a/drivers/media/platform/soc_camera/soc_camera.c
>> +++ b/drivers/media/platform/soc_camera/soc_camera.c
>> @@ -394,7 +394,7 @@ static int soc_camera_qbuf(struct file *file, void
> *priv,
>>          if (icd->streamer != file)
>>                  return -EBUSY;
> 
>> -       return vb2_qbuf(&icd->vb2_vidq, p);
>> +       return vb2_qbuf(&icd->vb2_vidq, NULL, p);
>>   }
> 
>>   static int soc_camera_dqbuf(struct file *file, void *priv,
>> @@ -430,7 +430,7 @@ static int soc_camera_prepare_buf(struct file *file,
> void *priv,
>>   {
>>          struct soc_camera_device *icd = file->private_data;
> 
>> -       return vb2_prepare_buf(&icd->vb2_vidq, b);
>> +       return vb2_prepare_buf(&icd->vb2_vidq, NULL, b);
>>   }
> 
>>   static int soc_camera_expbuf(struct file *file, void *priv,
>> diff --git a/drivers/media/usb/uvc/uvc_queue.c
> b/drivers/media/usb/uvc/uvc_queue.c
>> index fecccb5e7628..8964e16f2b22 100644
>> --- a/drivers/media/usb/uvc/uvc_queue.c
>> +++ b/drivers/media/usb/uvc/uvc_queue.c
>> @@ -300,12 +300,13 @@ int uvc_create_buffers(struct uvc_video_queue
> *queue,
>>          return ret;
>>   }
> 
>> -int uvc_queue_buffer(struct uvc_video_queue *queue, struct v4l2_buffer
> *buf)
>> +int uvc_queue_buffer(struct uvc_video_queue *queue,
>> +                    struct media_device *mdev, struct v4l2_buffer *buf)
>>   {
>>          int ret;
> 
>>          mutex_lock(&queue->mutex);
>> -       ret = vb2_qbuf(&queue->queue, buf);
>> +       ret = vb2_qbuf(&queue->queue, mdev, buf);
>>          mutex_unlock(&queue->mutex);
> 
>>          return ret;
>> diff --git a/drivers/media/usb/uvc/uvc_v4l2.c
> b/drivers/media/usb/uvc/uvc_v4l2.c
>> index bd32914259ae..3da5fdc002ac 100644
>> --- a/drivers/media/usb/uvc/uvc_v4l2.c
>> +++ b/drivers/media/usb/uvc/uvc_v4l2.c
>> @@ -751,7 +751,8 @@ static int uvc_ioctl_qbuf(struct file *file, void
> *fh, struct v4l2_buffer *buf)
>>          if (!uvc_has_privileges(handle))
>>                  return -EBUSY;
> 
>> -       return uvc_queue_buffer(&stream->queue, buf);
>> +       return uvc_queue_buffer(&stream->queue,
>> +                               stream->vdev.v4l2_dev->mdev, buf);
>>   }
> 
>>   static int uvc_ioctl_expbuf(struct file *file, void *fh,
>> diff --git a/drivers/media/usb/uvc/uvcvideo.h
> b/drivers/media/usb/uvc/uvcvideo.h
>> index be5cf179228b..bc9ed18f043c 100644
>> --- a/drivers/media/usb/uvc/uvcvideo.h
>> +++ b/drivers/media/usb/uvc/uvcvideo.h
>> @@ -680,6 +680,7 @@ int uvc_query_buffer(struct uvc_video_queue *queue,
>>   int uvc_create_buffers(struct uvc_video_queue *queue,
>>                         struct v4l2_create_buffers *v4l2_cb);
>>   int uvc_queue_buffer(struct uvc_video_queue *queue,
>> +                    struct media_device *mdev,
>>                       struct v4l2_buffer *v4l2_buf);
>>   int uvc_export_buffer(struct uvc_video_queue *queue,
>>                        struct v4l2_exportbuffer *exp);
>> diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
> b/drivers/media/v4l2-core/v4l2-mem2mem.c
>> index c4f963d96a79..438f1b869319 100644
>> --- a/drivers/media/v4l2-core/v4l2-mem2mem.c
>> +++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
>> @@ -20,6 +20,7 @@
>>   #include <media/videobuf2-v4l2.h>
>>   #include <media/v4l2-mem2mem.h>
>>   #include <media/v4l2-dev.h>
>> +#include <media/v4l2-device.h>
>>   #include <media/v4l2-fh.h>
>>   #include <media/v4l2-event.h>
> 
>> @@ -388,11 +389,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_querybuf);
>>   int v4l2_m2m_qbuf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>>                    struct v4l2_buffer *buf)
>>   {
>> +       struct video_device *vdev = video_devdata(file);
>>          struct vb2_queue *vq;
>>          int ret;
> 
>>          vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
>> -       ret = vb2_qbuf(vq, buf);
>> +       ret = vb2_qbuf(vq, vdev->v4l2_dev->mdev, buf);
>>          if (!ret)
>>                  v4l2_m2m_try_schedule(m2m_ctx);
> 
>> @@ -413,11 +415,12 @@ EXPORT_SYMBOL_GPL(v4l2_m2m_dqbuf);
>>   int v4l2_m2m_prepare_buf(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
>>                           struct v4l2_buffer *buf)
>>   {
>> +       struct video_device *vdev = video_devdata(file);
>>          struct vb2_queue *vq;
>>          int ret;
> 
>>          vq = v4l2_m2m_get_vq(m2m_ctx, buf->type);
>> -       ret = vb2_prepare_buf(vq, buf);
>> +       ret = vb2_prepare_buf(vq, vdev->v4l2_dev->mdev, buf);
>>          if (!ret)
>>                  v4l2_m2m_try_schedule(m2m_ctx);
> 
>> diff --git a/drivers/staging/media/davinci_vpfe/vpfe_video.c
> b/drivers/staging/media/davinci_vpfe/vpfe_video.c
>> index 588743a6fd8a..00bf28e830d4 100644
>> --- a/drivers/staging/media/davinci_vpfe/vpfe_video.c
>> +++ b/drivers/staging/media/davinci_vpfe/vpfe_video.c
>> @@ -1426,7 +1426,8 @@ static int vpfe_qbuf(struct file *file, void *priv,
>>                  return -EACCES;
>>          }
> 
>> -       return vb2_qbuf(&video->buffer_queue, p);
>> +       return vb2_qbuf(&video->buffer_queue,
>> +                       video->video_dev.v4l2_dev->mdev, p);
>>   }
> 
>>   /*
>> diff --git a/drivers/staging/media/omap4iss/iss_video.c
> b/drivers/staging/media/omap4iss/iss_video.c
>> index a3a83424a926..a35d1004b522 100644
>> --- a/drivers/staging/media/omap4iss/iss_video.c
>> +++ b/drivers/staging/media/omap4iss/iss_video.c
>> @@ -805,9 +805,10 @@ iss_video_querybuf(struct file *file, void *fh,
> struct v4l2_buffer *b)
>>   static int
>>   iss_video_qbuf(struct file *file, void *fh, struct v4l2_buffer *b)
>>   {
>> +       struct iss_video *video = video_drvdata(file);
>>          struct iss_video_fh *vfh = to_iss_video_fh(fh);
> 
>> -       return vb2_qbuf(&vfh->queue, b);
>> +       return vb2_qbuf(&vfh->queue, video->video.v4l2_dev->mdev, b);
>>   }
> 
>>   static int
>> diff --git a/drivers/usb/gadget/function/uvc_queue.c
> b/drivers/usb/gadget/function/uvc_queue.c
>> index 9e33d5206d54..f2497cb96abb 100644
>> --- a/drivers/usb/gadget/function/uvc_queue.c
>> +++ b/drivers/usb/gadget/function/uvc_queue.c
>> @@ -166,7 +166,7 @@ int uvcg_queue_buffer(struct uvc_video_queue *queue,
> struct v4l2_buffer *buf)
>>          unsigned long flags;
>>          int ret;
> 
>> -       ret = vb2_qbuf(&queue->queue, buf);
>> +       ret = vb2_qbuf(&queue->queue, NULL, buf);
>>          if (ret < 0)
>>                  return ret;
> 
>> diff --git a/include/media/videobuf2-v4l2.h
> b/include/media/videobuf2-v4l2.h
>> index 097bf3e6951d..cf312ab4e7e8 100644
>> --- a/include/media/videobuf2-v4l2.h
>> +++ b/include/media/videobuf2-v4l2.h
>> @@ -79,6 +79,7 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> v4l2_create_buffers *create);
>>    * vb2_prepare_buf() - Pass ownership of a buffer from userspace to the
> kernel
>>    *
>>    * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>> + * @mdev:      pointer to &struct media_device, may be NULL.
>>    * @b:         buffer structure passed from userspace to
>>    *             &v4l2_ioctl_ops->vidioc_prepare_buf handler in driver
>>    *
>> @@ -90,15 +91,19 @@ int vb2_create_bufs(struct vb2_queue *q, struct
> v4l2_create_buffers *create);
>>    * #) verifies the passed buffer,
>>    * #) calls &vb2_ops->buf_prepare callback in the driver (if provided),
>>    *    in which driver-specific buffer initialization can be performed.
>> + * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
>> + *    then bind the prepared buffer to the request.
>>    *
>>    * The return values from this function are intended to be directly
> returned
>>    * from &v4l2_ioctl_ops->vidioc_prepare_buf handler in driver.
>>    */
>> -int vb2_prepare_buf(struct vb2_queue *q, struct v4l2_buffer *b);
>> +int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
>> +                   struct v4l2_buffer *b);
> 
>>   /**
>>    * vb2_qbuf() - Queue a buffer from userspace
>>    * @q:         pointer to &struct vb2_queue with videobuf2 queue.
>> + * @mdev:      pointer to &struct media_device, may be NULL.
>>    * @b:         buffer structure passed from userspace to
>>    *             &v4l2_ioctl_ops->vidioc_qbuf handler in driver
>>    *
>> @@ -107,6 +112,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> v4l2_buffer *b);
>>    * This function:
>>    *
>>    * #) verifies the passed buffer;
>> + * #) if @b->request_fd is non-zero and @mdev->ops->req_queue is set,
>> + *    then bind the buffer to the request.
>>    * #) if necessary, calls &vb2_ops->buf_prepare callback in the driver
>>    *    (if provided), in which driver-specific buffer initialization can
>>    *    be performed;
>> @@ -116,7 +123,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct
> v4l2_buffer *b);
>>    * The return values from this function are intended to be directly
> returned
>>    * from &v4l2_ioctl_ops->vidioc_qbuf handler in driver.
>>    */
>> -int vb2_qbuf(struct vb2_queue *q, struct v4l2_buffer *b);
>> +int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
>> +            struct v4l2_buffer *b);
> 
>>   /**
>>    * vb2_expbuf() - Export a buffer as a file descriptor
>> --
>> 2.16.3

Re: [RFCv11 PATCH 27/29] vim2m: support requests

[Hans Verkuil]

On 04/17/2018 06:37 AM, Alexandre Courbot wrote:
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> Add support for requests to vim2m.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>   drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
>>   1 file changed, 25 insertions(+)
> 
>> diff --git a/drivers/media/platform/vim2m.c
> b/drivers/media/platform/vim2m.c
>> index 9b18b32c255d..2dcf0ea85705 100644
>> --- a/drivers/media/platform/vim2m.c
>> +++ b/drivers/media/platform/vim2m.c
>> @@ -387,8 +387,26 @@ static void device_run(void *priv)
>>          src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
>>          dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
> 
>> +       /* Apply request if needed */
>> +       if (src_buf->vb2_buf.req_obj.req)
>> +               v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
>> +                                       &ctx->hdl);
>> +       if (dst_buf->vb2_buf.req_obj.req &&
>> +           dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
>> +               v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
>> +                                       &ctx->hdl);
> 
> This implies that we can have two different requests active at the same time
> for the same device. Do we want to open that can of worms?
> 
> Valid scenarios for requests should be clearly defined. In the case of a
> simple
> buffer processor like vim2m, something like "request required for source
> buffer,
> optional and must be same as source if specified for destination", looks
> simple
> and sane.

Is it? It is sane for this specific use-case, but for other m2m devices you
might require the request for the sink buffer instead of the source buffer.
Or both. So for one driver you would have one set of rules, and for another
driver another set of rules.

That said, this probably requires some more discussion. I went with the generic
case here, and it was really very little code to make that happen. And it
avoids creating ad-hoc 'rules'.

Regards,

	Hans

> 
>> +
>>          device_process(ctx, src_buf, dst_buf);
> 
>> +       /* Complete request if needed */
>> +       if (src_buf->vb2_buf.req_obj.req)
>> +               v4l2_ctrl_request_complete(src_buf->vb2_buf.req_obj.req,
>> +                                       &ctx->hdl);
>> +       if (dst_buf->vb2_buf.req_obj.req &&
>> +           dst_buf->vb2_buf.req_obj.req != src_buf->vb2_buf.req_obj.req)
>> +               v4l2_ctrl_request_complete(dst_buf->vb2_buf.req_obj.req,
>> +                                       &ctx->hdl);
>> +
>>          /* Run a timer, which simulates a hardware irq  */
>>          schedule_irq(dev, ctx->transtime);
>>   }
>> @@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue *q)
>>                          vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
>>                  if (vbuf == NULL)
>>                          return;
>> +               v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
>> +                                          &ctx->hdl);
>>                  spin_lock_irqsave(&ctx->dev->irqlock, flags);
>>                  v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
>>                  spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
>> @@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
>>          .job_abort      = job_abort,
>>   };
> 
>> +static const struct media_device_ops m2m_media_ops = {
>> +       .req_queue = vb2_request_queue,
>> +};
>> +
>>   static int vim2m_probe(struct platform_device *pdev)
>>   {
>>          struct vim2m_dev *dev;
>> @@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device *pdev)
>>          dev->mdev.dev = &pdev->dev;
>>          strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
>>          media_device_init(&dev->mdev);
>> +       dev->mdev.ops = &m2m_media_ops;
>>          dev->v4l2_dev.mdev = &dev->mdev;
>>          dev->pad[0].flags = MEDIA_PAD_FL_SINK;
>>          dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
>> --
>> 2.16.3

Re: [RFCv11 PATCH 27/29] vim2m: support requests

[Hans Verkuil]

On 04/17/2018 01:42 PM, Paul Kocialkowski wrote:
> Hi,
> 
> On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> Add support for requests to vim2m.
> 
> Please find a nit below.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/platform/vim2m.c | 25 +++++++++++++++++++++++++
>>  1 file changed, 25 insertions(+)
>>
>> diff --git a/drivers/media/platform/vim2m.c
>> b/drivers/media/platform/vim2m.c
>> index 9b18b32c255d..2dcf0ea85705 100644
>> --- a/drivers/media/platform/vim2m.c
>> +++ b/drivers/media/platform/vim2m.c
>> @@ -387,8 +387,26 @@ static void device_run(void *priv)
>>  	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
>>  	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
>>  
>> +	/* Apply request if needed */
> 
> This comment suggests that this is where request submission is handled.
> I suggest making it clear that this the place where the *controls*
> attached to the request are applied, instead.

True. Fixed here and below.

Thanks!

	Hans

> 
>> +	if (src_buf->vb2_buf.req_obj.req)
>> +		v4l2_ctrl_request_setup(src_buf->vb2_buf.req_obj.req,
>> +					&ctx->hdl);
>> +	if (dst_buf->vb2_buf.req_obj.req &&
>> +	    dst_buf->vb2_buf.req_obj.req != src_buf-
>>> vb2_buf.req_obj.req)
>> +		v4l2_ctrl_request_setup(dst_buf->vb2_buf.req_obj.req,
>> +					&ctx->hdl);
>> +
>>  	device_process(ctx, src_buf, dst_buf);
>>  
>> +	/* Complete request if needed */
> 
> Ditto here.
> 
>> +	if (src_buf->vb2_buf.req_obj.req)
>> +		v4l2_ctrl_request_complete(src_buf-
>>> vb2_buf.req_obj.req,
>> +					&ctx->hdl);
>> +	if (dst_buf->vb2_buf.req_obj.req &&
>> +	    dst_buf->vb2_buf.req_obj.req != src_buf-
>>> vb2_buf.req_obj.req)
>> +		v4l2_ctrl_request_complete(dst_buf-
>>> vb2_buf.req_obj.req,
>> +					&ctx->hdl);
>> +
>>  	/* Run a timer, which simulates a hardware irq  */
>>  	schedule_irq(dev, ctx->transtime);
>>  }
>> @@ -823,6 +841,8 @@ static void vim2m_stop_streaming(struct vb2_queue
>> *q)
>>  			vbuf = v4l2_m2m_dst_buf_remove(ctx-
>>> fh.m2m_ctx);
>>  		if (vbuf == NULL)
>>  			return;
>> +		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
>> +					   &ctx->hdl);
>>  		spin_lock_irqsave(&ctx->dev->irqlock, flags);
>>  		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
>>  		spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
>> @@ -1003,6 +1023,10 @@ static const struct v4l2_m2m_ops m2m_ops = {
>>  	.job_abort	= job_abort,
>>  };
>>  
>> +static const struct media_device_ops m2m_media_ops = {
>> +	.req_queue = vb2_request_queue,
>> +};
>> +
>>  static int vim2m_probe(struct platform_device *pdev)
>>  {
>>  	struct vim2m_dev *dev;
>> @@ -1027,6 +1051,7 @@ static int vim2m_probe(struct platform_device
>> *pdev)
>>  	dev->mdev.dev = &pdev->dev;
>>  	strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
>>  	media_device_init(&dev->mdev);
>> +	dev->mdev.ops = &m2m_media_ops;
>>  	dev->v4l2_dev.mdev = &dev->mdev;
>>  	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
>>  	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;

Re: [RFCv11 PATCH 00/29] Request API

[Hans Verkuil]

Hi Alexandre,

On 04/19/2018 09:48 AM, Paul Kocialkowski wrote:
> Hi,
> 
> On Wed, 2018-04-18 at 02:06 +0000, Alexandre Courbot wrote:
>> On Tue, Apr 17, 2018 at 8:41 PM Paul Kocialkowski <
>> paul.kocialkowski@bootlin.com> wrote:
>> On Tue, 2018-04-17 at 06:17 +0000, Alexandre Courbot wrote:
>>>> On Tue, Apr 17, 2018 at 3:12 PM Hans Verkuil <hverkuil@xs4all.nl>
>>>> wrote:
>>>>
>>>>> On 04/17/2018 06:33 AM, Alexandre Courbot wrote:
>>>>>> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.
>>>>>> nl>
>>>>>> wrote:
>>>>>>
>>>>>>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>>>>>> Hi all,
>>>>>>> This is a cleaned up version of the v10 series (never posted
>>>>>>> to
>>>>>>> the list since it was messy).
>>>>>>
>>>>>> Hi Hans,
>>>>>>
>>>>>> It took me a while to test and review this, but finally have
>>>>>> been
>>>>>> able
>>>>
>>>> to
>>>>>> do it.
>>>>>>
>>>>>> First the result of the test: I have tried porting my dummy
>>>>>> vim2m
>>>>>> test
>>>>>> program
>>>>>> (https://gist.github.com/Gnurou/34c35f1f8e278dad454b51578d239a
>>>>>> 42
>>>>>> for
>>>>>> reference),
>>>>>> and am getting a hang when trying to queue the second OUTPUT
>>>>>> buffer
>>>>
>>>> (right
>>>>>> after
>>>>>> queuing the first request). If I move the calls the
>>>>>> VIDIOC_STREAMON
>>>>
>>>> after
>>>>>> the
>>>>>> requests are queued, the hang seems to happen at that moment.
>>>>>> Probably a
>>>>>> deadlock, haven't looked in detail yet.
>>>>>>
>>>>>> I have a few other comments, will follow up per-patch.
>>>>>>
>>>>>
>>>>> I had a similar/same (?) report about this from Paul:
>>>>> https://www.mail-archive.com/linux-media@vger.kernel.org/msg1291
>>>>> 77.h
>>>>> tml
>>>>
>>>> I saw this and tried to move the call to STREAMON to after the
>>>> requests are queued in my example program, but it then hanged
>>>> there.
>>>> So there is probably something more intricate taking place.
>>> I figured out the issue (but forgot to report back to the list):
>>> Hans'
>>> version of the request API doesn't set the POLLIN bit but POLLPRI
>>> instead, so you need to select for expect_fds instead of read_fds in
>>> the
>>> select call. That's pretty much all there is to it.
>>
>> I am not using select() but poll() in my test program (see the gist
>> link
>> above) and have set POLLPRI as the event to poll for. I may be missing
>> something but this looks correct to me?
> 
> You're right, I overlooked your email and assumed you were hitting the
> same issue that I had at first.
> 
> Anyway, I also had a similar issue when calling the STREAMON ioctl
> *before* enqueuing the request. What happens here is that
> v4l2_m2m_streamon (called from the ioctl) will also try to schedule a
> device run (v4l2_m2m_try_schedule). When the ioctl is called and the
> request was not queued yet, the lack of buffers will delay the streamon
> call. Then, when the request is later submitted, vb2_streamon is called
> with the queue directly, and there is no m2m-specific provision there,
> so no device run is scheduled and nothing happens. If the STREAMON ioctl
> happens after queuing the request, then things should be fine (but
> that's definitely not what we want userspace to be doing) since
> the vb2_streamon call from the ioctl handler will take effect
> and v4l2_m2m_try_schedule will be called.
> 
> The way that I have solved this with the Sunxi-Cedrus driver is to add a
> req_complete callback function pointer (holding a call
> to v4l2_m2m_try_schedule) in media_device_ops and call it (if available)
> from media_request_ioctl_queue. I initially put this in req_queue
> directly, but since it is wrapped by the request queue mutex lock and
> provided that device_run needs to access the request queue, we need an
> extra op called out of this lock, before completing the ioctl handler.
> 
> I will be sending v2 of my driver with preliminary patches to fix this
> (perhaps I should also fix vim2m that way along the way).

Can you verify that this indeed solves your problem? Or is there still
something else going on?

Regards,

	Hans

Re: [RFCv11 PATCH 26/29] vim2m: use workqueue

[Hans Verkuil]

On 11/04/18 16:06, Paul Kocialkowski wrote:
> Hi,
> 
> On Mon, 2018-04-09 at 16:20 +0200, Hans Verkuil wrote:
>> From: Hans Verkuil <hans.verkuil@cisco.com>
>>
>> v4l2_ctrl uses mutexes, so we can't setup a ctrl_handler in
>> interrupt context. Switch to a workqueue instead.
> 
> See one comment below.
> 
>> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
>> ---
>>  drivers/media/platform/vim2m.c | 15 +++++++++++++--
>>  1 file changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/media/platform/vim2m.c
>> b/drivers/media/platform/vim2m.c
>> index ef970434af13..9b18b32c255d 100644
>> --- a/drivers/media/platform/vim2m.c
>> +++ b/drivers/media/platform/vim2m.c
>> @@ -150,6 +150,7 @@ struct vim2m_dev {
>>  	spinlock_t		irqlock;
>>  
>>  	struct timer_list	timer;
>> +	struct work_struct	work_run;
> 
> Wouldn't it make more sense to move this to vim2m_ctx instead (since
> this is heavily m2m-specific)?

The work is triggered by a timer which is m2m_dev specific. So it makes
no sense to move this to the per-filehandle vim2m_ctx IMHO.

Regards,

	Hans

> 
>>  	struct v4l2_m2m_dev	*m2m_dev;
>>  };
>> @@ -392,9 +393,10 @@ static void device_run(void *priv)
>>  	schedule_irq(dev, ctx->transtime);
>>  }
>>  
>> -static void device_isr(struct timer_list *t)
>> +static void device_work(struct work_struct *w)
>>  {
>> -	struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t,
>> timer);
>> +	struct vim2m_dev *vim2m_dev =
>> +		container_of(w, struct vim2m_dev, work_run);
>>  	struct vim2m_ctx *curr_ctx;
>>  	struct vb2_v4l2_buffer *src_vb, *dst_vb;
>>  	unsigned long flags;
>> @@ -426,6 +428,13 @@ static void device_isr(struct timer_list *t)
>>  	}
>>  }
>>  
>> +static void device_isr(struct timer_list *t)
>> +{
>> +	struct vim2m_dev *vim2m_dev = from_timer(vim2m_dev, t,
>> timer);
>> +
>> +	schedule_work(&vim2m_dev->work_run);
>> +}
>> +
>>  /*
>>   * video ioctls
>>   */
>> @@ -806,6 +815,7 @@ static void vim2m_stop_streaming(struct vb2_queue
>> *q)
>>  	struct vb2_v4l2_buffer *vbuf;
>>  	unsigned long flags;
>>  
>> +	flush_scheduled_work();
>>  	for (;;) {
>>  		if (V4L2_TYPE_IS_OUTPUT(q->type))
>>  			vbuf = v4l2_m2m_src_buf_remove(ctx-
>>> fh.m2m_ctx);
>> @@ -1011,6 +1021,7 @@ static int vim2m_probe(struct platform_device
>> *pdev)
>>  	vfd = &dev->vfd;
>>  	vfd->lock = &dev->dev_mutex;
>>  	vfd->v4l2_dev = &dev->v4l2_dev;
>> +	INIT_WORK(&dev->work_run, device_work);
>>  
>>  #ifdef CONFIG_MEDIA_CONTROLLER
>>  	dev->mdev.dev = &pdev->dev;

Re: [RFCv11 PATCH 26/29] vim2m: use workqueue

[Hans Verkuil]

On 12/04/18 11:02, Tomasz Figa wrote:
> Hi Hans,
> 
> On Mon, Apr 9, 2018 at 11:20 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> 
>> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
>> v4l2_ctrl uses mutexes, so we can't setup a ctrl_handler in
>> interrupt context. Switch to a workqueue instead.
> 
> Could it make more sense to just replace the old (non-hr) timer used in
> this driver with delayed work?

I agree. I'll change that once v12 is posted to the mailing list.

Regards,

	Hans

> 
> Best regards,
> Tomasz
> 

Re: [RFCv11 PATCH 04/29] media-request: core request support

[Sakari Ailus]

Hi Hans,

On Mon, Apr 23, 2018 at 02:43:36PM +0200, Hans Verkuil wrote:
> On 04/12/2018 09:11 AM, Sakari Ailus wrote:
> > On Wed, Apr 11, 2018 at 01:13:44PM -0300, Mauro Carvalho Chehab wrote:
> >> Em Wed, 11 Apr 2018 18:35:14 +0300
> >> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> >>
> >>> On Wed, Apr 11, 2018 at 12:17:27PM -0300, Mauro Carvalho Chehab wrote:
> >>>> Em Wed, 11 Apr 2018 18:02:19 +0300
> >>>> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> >>>>   
> >>>>> On Wed, Apr 11, 2018 at 10:49:35AM -0300, Mauro Carvalho Chehab wrote:  
> >>>>>> Em Wed, 11 Apr 2018 16:21:16 +0300
> >>>>>> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> >>>>>>
> >>>>>>     
> >>>>>>>>>> Btw, this is a very good reason why you should define the ioctl to
> >>>>>>>>>> have an integer argument instead of a struct with a __s32 field
> >>>>>>>>>> on it, as per my comment to patch 02/29:
> >>>>>>>>>>
> >>>>>>>>>> 	#define MEDIA_IOC_REQUEST_ALLOC	_IOWR('|', 0x05, int)
> >>>>>>>>>>
> >>>>>>>>>> At 64 bit architectures, you're truncating the file descriptor!        
> >>>>>>>>>
> >>>>>>>>> I'm not quite sure what do you mean. int is 32 bits on 64-bit systems as
> >>>>>>>>> well.      
> >>>>>>>>
> >>>>>>>> Hmm.. you're right. I was thinking that it could be 64 bits on some
> >>>>>>>> archs like sparc64 (Tru64 C compiler declares it with 64 bits), but,
> >>>>>>>> according with:
> >>>>>>>>
> >>>>>>>> 	https://www.gnu.org/software/gnu-c-manual/gnu-c-manual.html
> >>>>>>>>
> >>>>>>>> This is not the case on gcc.      
> >>>>>>>
> >>>>>>> Ok. The reasoning back then was that what "int" means varies across
> >>>>>>> compilers and languages. And the intent was to codify this to __s32 which
> >>>>>>> is what the kernel effectively uses.    
> >>>>>>
> >>>>>> ...
> >>>>>>     
> >>>>>>> The rest of the kernel uses int rather liberally in the uAPI so I'm not
> >>>>>>> sure in the end whether something desirable was achieved. Perhaps it'd be
> >>>>>>> good to go back to the original discussion to find out for sure.
> >>>>>>>
> >>>>>>> Still binaries compiled with Tru64 C compiler wouldn't work on Linux anyway
> >>>>>>> due to that difference.
> >>>>>>>
> >>>>>>> Well, I stop here for this begins to be off-topic. :-)    
> >>>>>>
> >>>>>> Yes. Let's keep it as s32 as originally proposed. Just ignore my comments
> >>>>>> about that :-)
> >>>>>>     
> >>>>>>>>>>> +	get_task_comm(comm, current);
> >>>>>>>>>>> +	snprintf(req->debug_str, sizeof(req->debug_str), "%s:%d",
> >>>>>>>>>>> +		 comm, fd);        
> >>>>>>>>>>
> >>>>>>>>>> Not sure if it is a good idea to store the task that allocated
> >>>>>>>>>> the request. While it makes sense for the dev_dbg() below, it
> >>>>>>>>>> may not make sense anymore on other dev_dbg() you would be
> >>>>>>>>>> using it.        
> >>>>>>>>>
> >>>>>>>>> The lifetime of the file handle roughly matches that of the request. It's
> >>>>>>>>> for debug only anyway.
> >>>>>>>>>
> >>>>>>>>> Better proposals are always welcome of course. But I think we should have
> >>>>>>>>> something here that helps debugging by meaningfully making the requests
> >>>>>>>>> identifiable from logs.      
> >>>>>>>>
> >>>>>>>> What I meant to say is that one PID could be allocating the
> >>>>>>>> request, while some other one could be actually doing Q/DQ_BUF.
> >>>>>>>> On such scenario, the debug string could provide mislead prints.      
> >>>>>>>
> >>>>>>> Um, yes, indeed it would no longer match the process. But the request is
> >>>>>>> still the same. That's actually a positive thing since it allows you to
> >>>>>>> identify the request.
> >>>>>>>
> >>>>>>> With a global ID space this was trivial; you could just print the request
> >>>>>>> ID and that was all that was ever needed. (I'm not proposing to consider
> >>>>>>> that though.)
> >>>>>>>     
> >>>>>>
> >>>>>> IMO, a global ID number would work better than get_task_comm().
> >>>>>>
> >>>>>> Just add a static int monotonic counter and use it for the debug purposes,
> >>>>>> e. g.:
> >>>>>>
> >>>>>> {
> >>>>>> 	static unsigned int req_count = 0;
> >>>>>>
> >>>>>> 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d",
> >>>>>> 		req_count++, fd);    
> >>>>>>
> >>>>>> Ok, eventually, it will overflow, but, it will be unique within
> >>>>>> a reasonable timeframe to be good enough for debugging purposes.    
> >>>>>
> >>>>> Yes, but you can't figure out which process allocated it anymore, making
> >>>>> associating kernel debug logs with user space process logs harder.
> >>>>>
> >>>>> How about process id + file handle? That still doesn't tell which process
> >>>>> operated on the request though, but I'm not sure whether that's really a
> >>>>> crucial piece of information.  
> >>>>
> >>>> You don't need that. With dev_dbg() - and other *_dbg() macros - you can
> >>>> enable process ID for all debug messages.  
> >>>
> >>> With this, the problem again is that it does not uniquely identify the
> >>> request: the request is the same request independently of which process
> >>> would operate on it. Or whether it is being processed in an interrupt
> >>> context.
> >>>
> >>> AFAICT, the allocator PID (or process name) + file handle are both required
> >>> to match a request between user and kernel space logs.
> >>
> >> Sorry, I was unable to understand what you're saying.
> >>
> >> If you set the debug string with:
> >>
> >> 	snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d", req_count++, fd);  
> >>
> >> With the remaining stuff at patch 04/29, e. g. those two printks:
> >>
> >> 	dev_dbg(mdev->dev, "request: allocated %s\n", req->debug_str);
> >> 	dev_dbg(mdev->dev, "request: release %s\n", req->debug_str);
> >>
> >> And use "+pt" to enable those debug messages, for the request #1 with fd #45, 
> >> created by PID 16613 you would have a log like:
> >>
> >> 	[  269.021116] [16613] request: allocated 1:45
> >> 	[  269.024118] [16613] request: release 1:45
> >>
> >> (assuming that the same PID would create and release)
> >>
> >> The "1:45" is an unique global ID that would allow tracking it, even
> >> if Q/DQ_BUF is done by some other PID.
> >>
> >> E. g. if a PID#16618 were responsible for Q/DQ_BUF, you would have
> >> something like:
> >>
> >> 	[  269.021116] [16613] request: allocated 1:45
> >> 	[  269.021117] [16618] request: Q_BUF 1:45
> >> 	[  269.021118] [16618] request: DQ_BUF 1:45
> >> 	[  269.024118] [16613] request: release 1:45
> >>
> >> (assuming that you would have a Q_BUF/DQ_BUF similar dev_dbg())
> >>
> >> That seems good enough to track it.
> >>
> >> Yet, in order to make easier to track, I would actually change the
> >> dev_dbg() parameter order everywhere to something like:
> >>
> >> 	dev_dbg(mdev->dev, "request#%s: allocated\n", req->debug_str)
> >> 	dev_dbg(mdev->dev, "request#%s: release\n", req->debug_str);
> >>
> >> In order to print something like:
> >>
> >> 	[  269.021116] [16613] request#1:45: allocated 
> >> 	[  269.021117] [16618] request#1:45: Q_BUF
> >> 	[  269.021118] [16618] request#1:45: DQ_BUF
> >> 	[  269.024118] [16613] request#1:45: release
> >>
> >> Then, getting everything related to the first request would be as simple as:
> >>
> >> 	$ dmesg|grep request#1:
> >>
> >> That will provide the PID for both processes: the one that
> >> created/released and the one that queued/dequeued.
> > 
> > Ah, right; yes, then you can. It's still a bit more complicated as you have
> > one more piece of information to follow (the ID) vs. just PID and FD. For
> > instance, you can't grep for requests created by a given process. Note that
> > you can still print the PID of the process that operates on the request
> > through dyndbg.
> > 
> > I'd like to hear what Hans thinks.
> > 
> 
> Frankly, I have no opinion :-)
> 
> I'm OK with changing this to:
> 
> snprintf(req->debug_str, sizeof(req->debug_str), "%u:%d", req_count++, fd);
> 
> if Sakari agrees with that.

My preference is keeping debugging info as readable for humans as possible,
i.e. keeping the process name (or ID) of the creator of the request in the
messages --- this allows easy grepping of the logs for particular requests.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi