[PATCHv4 00/10] As was discussed here (among other places):

[PATCHv4 00/10] As was discussed here (among other places):

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

https://lkml.org/lkml/2018/10/19/440

using capture queue buffer indices to refer to reference frames is
not a good idea. A better idea is to use a 'tag' where the
application can assign a u32 tag to an output buffer, which is then 
copied to the capture buffer(s) derived from the output buffer.

It has been suggested that the timestamp can be used for this. But
there are a number of reasons why this is a bad idea:

1) the struct timeval is converted to a u64 in vb2. So there can be
   all sorts of unexpected conversion issues. In particular, the
   output of ns_to_timeval(timeval_to_ns(tv)) does not necessarily
   match the input.

2) it gets worse with the y2038 code where userspace either deals
   with a 32 bit tv_sec value or a 64 bit value.

In other words, using timestamp for this is not a good idea.

This implementation adds a new tag field in a union with the reserved2
field. The interpretation of that union depends on the flags field, so
it still can be used for other things as well. In addition, in the previous
patches the tag was in a union with the timecode field (again determined
by the flags field), so if we need to cram additional information in this
struct we can always put it in a union with the timecode field as well.
It worked for the tag, it should work for other things.

But we really need to start looking at a struct v4l2_ext_buffer.

The first three patches add core tag support, the next two patches document
the tag support, then a new helper function is added to v4l2-mem2mem.c
to easily copy data from a source to a destination buffer that drivers
can use.

Next a new supports_tags vb2_queue flag is added to indicate that
the driver supports tags. Ideally this should not be necessary, but
that would require that all m2m drivers are converted to using the
new helper function introduced in the previous patch. That takes more
time then I have now.

Finally the vim2m, vicodec and cedrus drivers are converted to support
tags.

I also removed the 'pad' fields from the mpeg2 control structs (it
should never been added in the first place) and aligned the structs
to a u32 boundary.

Note that this might change further (Paul suggested using bitfields).

Also note that the cedrus code doesn't set the sequence counter, that's
something that should still be added before this driver can be moved
out of staging.

Note: if no buffer is found for a certain tag, then the dma address
is just set to 0. That happened before as well with invalid buffer
indices. This should be checked in the driver!

Regards,

        Hans

Changes since v3:

- use reserved2 for the tag
- split the documentation in two: one documenting the tag, one
  cleaning up the timecode documentation.

Changes since v2:

- rebased
- added Reviewed-by tags
- fixed a few remaining references in the documentation to the old
  v4l2_buffer_tag struct that was used in early versions of this
  series.

Changes since v1:

- changed to a u32 tag. Using a 64 bit tag was overly complicated due
  to the bad layout of the v4l2_buffer struct, and there is no real
  need for it by applications.

Main changes since the RFC:

- Added new buffer capability flag
- Added m2m helper to copy data between buffers
- Added documentation
- Added tag logging in v4l2-ioctl.c


Hans Verkuil (10):
  videodev2.h: add tag support
  vb2: add tag support
  v4l2-ioctl.c: log v4l2_buffer tag
  buffer.rst: document the new buffer tag feature.
  buffer.rst: clean up timecode documentation
  v4l2-mem2mem: add v4l2_m2m_buf_copy_data helper function
  vb2: add new supports_tags queue flag
  vim2m: add tag support
  vicodec: add tag support
  cedrus: add tag support

 Documentation/media/uapi/v4l/buffer.rst       | 28 +++++++++----
 .../media/uapi/v4l/vidioc-reqbufs.rst         |  4 ++
 .../media/common/videobuf2/videobuf2-v4l2.c   | 41 ++++++++++++++++---
 drivers/media/platform/vicodec/vicodec-core.c | 14 ++-----
 drivers/media/platform/vim2m.c                | 14 ++-----
 drivers/media/v4l2-core/v4l2-ctrls.c          |  9 ----
 drivers/media/v4l2-core/v4l2-ioctl.c          |  9 ++--
 drivers/media/v4l2-core/v4l2-mem2mem.c        | 23 +++++++++++
 drivers/staging/media/sunxi/cedrus/cedrus.h   |  9 ++--
 .../staging/media/sunxi/cedrus/cedrus_dec.c   |  2 +
 .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 21 ++++------
 .../staging/media/sunxi/cedrus/cedrus_video.c |  2 +
 include/media/v4l2-mem2mem.h                  | 21 ++++++++++
 include/media/videobuf2-core.h                |  2 +
 include/media/videobuf2-v4l2.h                | 21 +++++++++-
 include/uapi/linux/v4l2-controls.h            | 14 +++----
 include/uapi/linux/videodev2.h                |  9 +++-
 17 files changed, 168 insertions(+), 75 deletions(-)

-- 
2.19.1

[PATCHv4 01/10] videodev2.h: add tag support

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Add support for 'tags' to struct v4l2_buffer. These can be used
by m2m devices so userspace can set a tag for an output buffer and
this value will then be copied to the capture buffer(s).

This tag can be used to refer to capture buffers, something that
is needed by stateless HW codecs.

The new V4L2_BUF_CAP_SUPPORTS_TAGS capability indicates whether
or not tags are supported.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 include/uapi/linux/videodev2.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 2db1635de956..9095d7abe10d 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -881,6 +881,7 @@ struct v4l2_requestbuffers {
 #define V4L2_BUF_CAP_SUPPORTS_DMABUF	(1 << 2)
 #define V4L2_BUF_CAP_SUPPORTS_REQUESTS	(1 << 3)
 #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
+#define V4L2_BUF_CAP_SUPPORTS_TAGS	(1 << 5)
 
 /**
  * struct v4l2_plane - plane info for multi-planar buffers
@@ -940,6 +941,7 @@ struct v4l2_plane {
  * @length:	size in bytes of the buffer (NOT its payload) for single-plane
  *		buffers (when type != *_MPLANE); number of elements in the
  *		planes array for multi-plane buffers
+ * @tag:	buffer tag
  * @request_fd: fd of the request that this buffer should use
  *
  * Contains data exchanged by application and driver using one of the Streaming
@@ -964,7 +966,10 @@ struct v4l2_buffer {
 		__s32		fd;
 	} m;
 	__u32			length;
-	__u32			reserved2;
+	union {
+		__u32		reserved2;
+		__u32		tag;
+	};
 	union {
 		__s32		request_fd;
 		__u32		reserved;
@@ -990,6 +995,8 @@ struct v4l2_buffer {
 #define V4L2_BUF_FLAG_IN_REQUEST		0x00000080
 /* timecode field is valid */
 #define V4L2_BUF_FLAG_TIMECODE			0x00000100
+/* tag field is valid */
+#define V4L2_BUF_FLAG_TAG			0x00000200
 /* Buffer is prepared for queuing */
 #define V4L2_BUF_FLAG_PREPARED			0x00000400
 /* Cache handling flags */
-- 
2.19.1

[PATCHv4 02/10] vb2: add tag support

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Add support for tags to vb2. Besides just storing and setting
the tag this patch also adds the vb2_find_tag() function that
can be used to find a buffer with the given tag.

This function will only look at DEQUEUED and DONE buffers, i.e.
buffers that are already processed.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 .../media/common/videobuf2/videobuf2-v4l2.c   | 39 ++++++++++++++++---
 include/media/videobuf2-v4l2.h                | 21 +++++++++-
 2 files changed, 53 insertions(+), 7 deletions(-)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 1244c246d0c4..e0e31e1c67c9 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -50,7 +50,8 @@ module_param(debug, int, 0644);
 				 V4L2_BUF_FLAG_TIMESTAMP_MASK)
 /* Output buffer flags that should be passed on to the driver */
 #define V4L2_BUFFER_OUT_FLAGS	(V4L2_BUF_FLAG_PFRAME | V4L2_BUF_FLAG_BFRAME | \
-				 V4L2_BUF_FLAG_KEYFRAME | V4L2_BUF_FLAG_TIMECODE)
+				 V4L2_BUF_FLAG_KEYFRAME | \
+				 V4L2_BUF_FLAG_TIMECODE | V4L2_BUF_FLAG_TAG)
 
 /*
  * __verify_planes_array() - verify that the planes array passed in struct
@@ -144,7 +145,10 @@ static void __copy_timestamp(struct vb2_buffer *vb, const void *pb)
 		 */
 		if (q->copy_timestamp)
 			vb->timestamp = timeval_to_ns(&b->timestamp);
-		vbuf->flags |= b->flags & V4L2_BUF_FLAG_TIMECODE;
+		vbuf->flags |= b->flags &
+			(V4L2_BUF_FLAG_TIMECODE | V4L2_BUF_FLAG_TAG);
+		if (b->flags & V4L2_BUF_FLAG_TAG)
+			vbuf->tag = b->tag;
 		if (b->flags & V4L2_BUF_FLAG_TIMECODE)
 			vbuf->timecode = b->timecode;
 	}
@@ -194,6 +198,7 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
 	}
 	vbuf->sequence = 0;
 	vbuf->request_fd = -1;
+	vbuf->tag = 0;
 
 	if (V4L2_TYPE_IS_MULTIPLANAR(b->type)) {
 		switch (b->memory) {
@@ -314,12 +319,12 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
 
 	if (V4L2_TYPE_IS_OUTPUT(b->type)) {
 		/*
-		 * For output buffers mask out the timecode flag:
-		 * this will be handled later in vb2_qbuf().
+		 * For output buffers mask out the timecode and tag flags:
+		 * these will be handled later in vb2_qbuf().
 		 * The 'field' is valid metadata for this output buffer
 		 * and so that needs to be copied here.
 		 */
-		vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
+		vbuf->flags &= ~(V4L2_BUF_FLAG_TIMECODE | V4L2_BUF_FLAG_TAG);
 		vbuf->field = b->field;
 	} else {
 		/* Zero any output buffer flags as this is a capture buffer */
@@ -460,7 +465,10 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
 	b->flags = vbuf->flags;
 	b->field = vbuf->field;
 	b->timestamp = ns_to_timeval(vb->timestamp);
-	b->timecode = vbuf->timecode;
+	if (b->flags & V4L2_BUF_FLAG_TAG)
+		b->tag = vbuf->tag;
+	if (b->flags & V4L2_BUF_FLAG_TIMECODE)
+		b->timecode = vbuf->timecode;
 	b->sequence = vbuf->sequence;
 	b->reserved2 = 0;
 	b->request_fd = 0;
@@ -586,6 +594,25 @@ static const struct vb2_buf_ops v4l2_buf_ops = {
 	.copy_timestamp		= __copy_timestamp,
 };
 
+int vb2_find_tag(const struct vb2_queue *q, u32 tag,
+		 unsigned int start_idx)
+{
+	unsigned int i;
+
+	for (i = start_idx; i < q->num_buffers; i++) {
+		struct vb2_buffer *vb = q->bufs[i];
+		struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+		if ((vb->state == VB2_BUF_STATE_DEQUEUED ||
+		     vb->state == VB2_BUF_STATE_DONE) &&
+		    (vbuf->flags & V4L2_BUF_FLAG_TAG) &&
+		    vbuf->tag == tag)
+			return i;
+	}
+	return -1;
+}
+EXPORT_SYMBOL_GPL(vb2_find_tag);
+
 /*
  * vb2_querybuf() - query video buffer information
  * @q:		videobuf queue
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 727855463838..c2a541af6b2c 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -31,8 +31,9 @@
  * @field:	field order of the image in the buffer, as defined by
  *		&enum v4l2_field.
  * @timecode:	frame timecode.
+ * @tag:	user specified buffer tag value.
  * @sequence:	sequence count of this frame.
- * @request_fd:	the request_fd associated with this buffer
+ * @request_fd:	the request_fd associated with this buffer.
  * @planes:	plane information (userptr/fd, length, bytesused, data_offset).
  *
  * Should contain enough information to be able to cover all the fields
@@ -44,6 +45,7 @@ struct vb2_v4l2_buffer {
 	__u32			flags;
 	__u32			field;
 	struct v4l2_timecode	timecode;
+	__u32			tag;
 	__u32			sequence;
 	__s32			request_fd;
 	struct vb2_plane	planes[VB2_MAX_PLANES];
@@ -55,6 +57,23 @@ struct vb2_v4l2_buffer {
 #define to_vb2_v4l2_buffer(vb) \
 	container_of(vb, struct vb2_v4l2_buffer, vb2_buf)
 
+/**
+ * vb2_find_tag() - Find buffer with given tag in the queue
+ *
+ * @q:		pointer to &struct vb2_queue with videobuf2 queue.
+ * @tag:	the tag to find. Only buffers in state DEQUEUED or DONE
+ *		are considered.
+ * @start_idx:	the start index (usually 0) in the buffer array to start
+ *		searching from. Note that there may be multiple buffers
+ *		with the same tag value, so you can restart the search
+ *		by setting @start_idx to the previously found index + 1.
+ *
+ * Returns the buffer index of the buffer with the given @tag, or
+ * -1 if no buffer with @tag was found.
+ */
+int vb2_find_tag(const struct vb2_queue *q, u32 tag,
+		 unsigned int start_idx);
+
 int vb2_querybuf(struct vb2_queue *q, struct v4l2_buffer *b);
 
 /**
-- 
2.19.1

[PATCHv4 03/10] v4l2-ioctl.c: log v4l2_buffer tag

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

When debugging is on, log the new tag field of struct v4l2_buffer
as well.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b9616b1f227b..07c6c939a23c 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -498,9 +498,12 @@ static void v4l_print_buffer(const void *arg, bool write_only)
 			p->bytesused, p->m.userptr, p->length);
 	}
 
-	printk(KERN_DEBUG "timecode=%02d:%02d:%02d type=%d, flags=0x%08x, frames=%d, userbits=0x%08x\n",
-			tc->hours, tc->minutes, tc->seconds,
-			tc->type, tc->flags, tc->frames, *(__u32 *)tc->userbits);
+	if (p->flags & V4L2_BUF_FLAG_TAG)
+		printk(KERN_DEBUG "tag=%x\n", p->tag);
+	if (p->flags & V4L2_BUF_FLAG_TIMECODE)
+		printk(KERN_DEBUG "timecode=%02d:%02d:%02d type=%d, flags=0x%08x, frames=%d, userbits=0x%08x\n",
+		       tc->hours, tc->minutes, tc->seconds,
+		       tc->type, tc->flags, tc->frames, *(__u32 *)tc->userbits);
 }
 
 static void v4l_print_exportbuffer(const void *arg, bool write_only)
-- 
2.19.1

[PATCHv4 04/10] buffer.rst: document the new buffer tag feature.

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Document V4L2_BUF_FLAG_TAG.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 Documentation/media/uapi/v4l/buffer.rst         | 17 ++++++++++++++---
 Documentation/media/uapi/v4l/vidioc-reqbufs.rst |  4 ++++
 2 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index 2e266d32470a..f83ee00cb30b 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -301,10 +301,13 @@ struct v4l2_buffer
 	elements in the ``planes`` array. The driver will fill in the
 	actual number of valid elements in that array.
     * - __u32
-      - ``reserved2``
+      - ``tag``
       -
-      - A place holder for future extensions. Drivers and applications
-	must set this to 0.
+      - When the ``V4L2_BUF_FLAG_TAG`` flag is set in ``flags``, this
+	field contains a user-specified tag value.
+
+	It is used by stateless codecs where this tag can be used to
+	refer to buffers that contain reference frames.
     * - __u32
       - ``request_fd``
       -
@@ -567,6 +570,14 @@ Buffer Flags
 	when the ``VIDIOC_DQBUF`` ioctl is called. Applications can set
 	this bit and the corresponding ``timecode`` structure when
 	``type`` refers to an output stream.
+    * .. _`V4L2-BUF-FLAG-TAG`:
+
+      - ``V4L2_BUF_FLAG_TAG``
+      - 0x00000200
+      - The ``tag`` field is valid. Applications can set
+	this bit and the corresponding ``tag`` field. If tags are
+	supported then the ``V4L2_BUF_CAP_SUPPORTS_TAGS`` capability
+	is also set.
     * .. _`V4L2-BUF-FLAG-PREPARED`:
 
       - ``V4L2_BUF_FLAG_PREPARED``
diff --git a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
index e62a15782790..38a7d0aee483 100644
--- a/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
+++ b/Documentation/media/uapi/v4l/vidioc-reqbufs.rst
@@ -118,6 +118,7 @@ aborting or finishing any DMA in progress, an implicit
 .. _V4L2-BUF-CAP-SUPPORTS-DMABUF:
 .. _V4L2-BUF-CAP-SUPPORTS-REQUESTS:
 .. _V4L2-BUF-CAP-SUPPORTS-ORPHANED-BUFS:
+.. _V4L2-BUF-CAP-SUPPORTS-TAGS:
 
 .. cssclass:: longtable
 
@@ -143,6 +144,9 @@ aborting or finishing any DMA in progress, an implicit
       - The kernel allows calling :ref:`VIDIOC_REQBUFS` while buffers are still
         mapped or exported via DMABUF. These orphaned buffers will be freed
         when they are unmapped or when the exported DMABUF fds are closed.
+    * - ``V4L2_BUF_CAP_SUPPORTS_TAGS``
+      - 0x00000020
+      - This buffer type supports ``V4L2_BUF_FLAG_TAG``.
 
 Return Value
 ============
-- 
2.19.1

[PATCHv4 05/10] buffer.rst: clean up timecode documentation

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

V4L2_BUF_FLAG_TIMECODE is not video capture specific, so drop that
part.

The 'Timecodes' section was a bit messy, so that's cleaned up.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 Documentation/media/uapi/v4l/buffer.rst | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/Documentation/media/uapi/v4l/buffer.rst b/Documentation/media/uapi/v4l/buffer.rst
index f83ee00cb30b..359b131a212d 100644
--- a/Documentation/media/uapi/v4l/buffer.rst
+++ b/Documentation/media/uapi/v4l/buffer.rst
@@ -223,8 +223,7 @@ struct v4l2_buffer
     * - struct :c:type:`v4l2_timecode`
       - ``timecode``
       -
-      - When ``type`` is ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` and the
-	``V4L2_BUF_FLAG_TIMECODE`` flag is set in ``flags``, this
+      - When the ``V4L2_BUF_FLAG_TIMECODE`` flag is set in ``flags``, this
 	structure contains a frame timecode. In
 	:c:type:`V4L2_FIELD_ALTERNATE <v4l2_field>` mode the top and
 	bottom field contain the same timecode. Timecodes are intended to
@@ -715,10 +714,10 @@ enum v4l2_memory
 Timecodes
 =========
 
-The struct :c:type:`v4l2_timecode` structure is designed to hold a
-:ref:`smpte12m` or similar timecode. (struct
-struct :c:type:`timeval` timestamps are stored in struct
-:c:type:`v4l2_buffer` field ``timestamp``.)
+The :c:type:`v4l2_buffer_timecode` structure is designed to hold a
+:ref:`smpte12m` or similar timecode.
+(struct :c:type:`timeval` timestamps are stored in the struct
+:c:type:`v4l2_buffer` ``timestamp`` field.)
 
 
 .. c:type:: v4l2_timecode
-- 
2.19.1

[PATCHv4 06/10] v4l2-mem2mem: add v4l2_m2m_buf_copy_data helper function

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Memory-to-memory devices should copy various parts of
struct v4l2_buffer from the output buffer to the capture buffer.

Add a helper function that does that to simplify the driver code.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-mem2mem.c | 23 +++++++++++++++++++++++
 include/media/v4l2-mem2mem.h           | 21 +++++++++++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c b/drivers/media/v4l2-core/v4l2-mem2mem.c
index 5bbdec55b7d7..a9cb1ac33dc0 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -975,6 +975,29 @@ void v4l2_m2m_buf_queue(struct v4l2_m2m_ctx *m2m_ctx,
 }
 EXPORT_SYMBOL_GPL(v4l2_m2m_buf_queue);
 
+void v4l2_m2m_buf_copy_data(const struct vb2_v4l2_buffer *out_vb,
+			    struct vb2_v4l2_buffer *cap_vb,
+			    bool copy_frame_flags)
+{
+	u32 mask = V4L2_BUF_FLAG_TIMECODE | V4L2_BUF_FLAG_TAG |
+		   V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
+
+	if (copy_frame_flags)
+		mask |= V4L2_BUF_FLAG_KEYFRAME | V4L2_BUF_FLAG_PFRAME |
+			V4L2_BUF_FLAG_BFRAME;
+
+	cap_vb->vb2_buf.timestamp = out_vb->vb2_buf.timestamp;
+
+	if (out_vb->flags & V4L2_BUF_FLAG_TAG)
+		cap_vb->tag = out_vb->tag;
+	if (out_vb->flags & V4L2_BUF_FLAG_TIMECODE)
+		cap_vb->timecode = out_vb->timecode;
+	cap_vb->field = out_vb->field;
+	cap_vb->flags &= ~mask;
+	cap_vb->flags |= out_vb->flags & mask;
+}
+EXPORT_SYMBOL_GPL(v4l2_m2m_buf_copy_data);
+
 void v4l2_m2m_request_queue(struct media_request *req)
 {
 	struct media_request_object *obj, *obj_safe;
diff --git a/include/media/v4l2-mem2mem.h b/include/media/v4l2-mem2mem.h
index 5467264771ec..bb4feb6969d2 100644
--- a/include/media/v4l2-mem2mem.h
+++ b/include/media/v4l2-mem2mem.h
@@ -622,6 +622,27 @@ v4l2_m2m_dst_buf_remove_by_idx(struct v4l2_m2m_ctx *m2m_ctx, unsigned int idx)
 	return v4l2_m2m_buf_remove_by_idx(&m2m_ctx->cap_q_ctx, idx);
 }
 
+/**
+ * v4l2_m2m_buf_copy_data() - copy buffer data from the output buffer to the
+ * capture buffer
+ *
+ * @out_vb: the output buffer that is the source of the data.
+ * @cap_vb: the capture buffer that will receive the data.
+ * @copy_frame_flags: copy the KEY/B/PFRAME flags as well.
+ *
+ * This helper function copies the timestamp, timecode (if the TIMECODE
+ * buffer flag was set), tag (if the TAG buffer flag was set), field
+ * and the TIMECODE, TAG, KEYFRAME, BFRAME, PFRAME and TSTAMP_SRC_MASK
+ * flags from @out_vb to @cap_vb.
+ *
+ * If @copy_frame_flags is false, then the KEYFRAME, BFRAME and PFRAME
+ * flags are not copied. This is typically needed for encoders that
+ * set this bits explicitly.
+ */
+void v4l2_m2m_buf_copy_data(const struct vb2_v4l2_buffer *out_vb,
+			    struct vb2_v4l2_buffer *cap_vb,
+			    bool copy_frame_flags);
+
 /* v4l2 request helper */
 
 void v4l2_m2m_request_queue(struct media_request *req);
-- 
2.19.1

[PATCHv4 07/10] vb2: add new supports_tags queue flag

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Add new flag to indicate that buffer tags are supported.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/common/videobuf2/videobuf2-v4l2.c | 2 ++
 include/media/videobuf2-core.h                  | 2 ++
 2 files changed, 4 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index e0e31e1c67c9..5aa5b1ea90a8 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -659,6 +659,8 @@ static void fill_buf_caps(struct vb2_queue *q, u32 *caps)
 		*caps |= V4L2_BUF_CAP_SUPPORTS_DMABUF;
 	if (q->supports_requests)
 		*caps |= V4L2_BUF_CAP_SUPPORTS_REQUESTS;
+	if (q->supports_tags)
+		*caps |= V4L2_BUF_CAP_SUPPORTS_TAGS;
 }
 
 int vb2_reqbufs(struct vb2_queue *q, struct v4l2_requestbuffers *req)
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index e86981d615ae..81f2dbfd0094 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -473,6 +473,7 @@ struct vb2_buf_ops {
  *              has not been called. This is a vb1 idiom that has been adopted
  *              also by vb2.
  * @supports_requests: this queue supports the Request API.
+ * @supports_tags: this queue supports tags in struct v4l2_buffer.
  * @uses_qbuf:	qbuf was used directly for this queue. Set to 1 the first
  *		time this is called. Set to 0 when the queue is canceled.
  *		If this is 1, then you cannot queue buffers from a request.
@@ -547,6 +548,7 @@ struct vb2_queue {
 	unsigned			allow_zero_bytesused:1;
 	unsigned		   quirk_poll_must_check_waiting_for_buffers:1;
 	unsigned			supports_requests:1;
+	unsigned			supports_tags:1;
 	unsigned			uses_qbuf:1;
 	unsigned			uses_requests:1;
 
-- 
2.19.1

[PATCHv4 08/10] vim2m: add tag support

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Copy tags in vim2m.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/platform/vim2m.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index d01821a6906a..be328483a53a 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -241,17 +241,7 @@ static int device_process(struct vim2m_ctx *ctx,
 	out_vb->sequence =
 		get_q_data(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE)->sequence++;
 	in_vb->sequence = q_data->sequence++;
-	out_vb->vb2_buf.timestamp = in_vb->vb2_buf.timestamp;
-
-	if (in_vb->flags & V4L2_BUF_FLAG_TIMECODE)
-		out_vb->timecode = in_vb->timecode;
-	out_vb->field = in_vb->field;
-	out_vb->flags = in_vb->flags &
-		(V4L2_BUF_FLAG_TIMECODE |
-		 V4L2_BUF_FLAG_KEYFRAME |
-		 V4L2_BUF_FLAG_PFRAME |
-		 V4L2_BUF_FLAG_BFRAME |
-		 V4L2_BUF_FLAG_TSTAMP_SRC_MASK);
+	v4l2_m2m_buf_copy_data(out_vb, in_vb, true);
 
 	switch (ctx->mode) {
 	case MEM2MEM_HFLIP | MEM2MEM_VFLIP:
@@ -855,6 +845,7 @@ static int queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *ds
 	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	src_vq->lock = &ctx->dev->dev_mutex;
 	src_vq->supports_requests = true;
+	src_vq->supports_tags = true;
 
 	ret = vb2_queue_init(src_vq);
 	if (ret)
@@ -868,6 +859,7 @@ static int queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *ds
 	dst_vq->mem_ops = &vb2_vmalloc_memops;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	dst_vq->lock = &ctx->dev->dev_mutex;
+	dst_vq->supports_tags = true;
 
 	return vb2_queue_init(dst_vq);
 }
-- 
2.19.1

[PATCHv4 09/10] vicodec: add tag support

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Copy tags in vicodec.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/platform/vicodec/vicodec-core.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)

diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index b7bdfe97215b..4d39ea033653 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -190,18 +190,8 @@ static int device_process(struct vicodec_ctx *ctx,
 	}
 
 	out_vb->sequence = q_cap->sequence++;
-	out_vb->vb2_buf.timestamp = in_vb->vb2_buf.timestamp;
-
-	if (in_vb->flags & V4L2_BUF_FLAG_TIMECODE)
-		out_vb->timecode = in_vb->timecode;
-	out_vb->field = in_vb->field;
 	out_vb->flags &= ~V4L2_BUF_FLAG_LAST;
-	out_vb->flags |= in_vb->flags &
-		(V4L2_BUF_FLAG_TIMECODE |
-		 V4L2_BUF_FLAG_KEYFRAME |
-		 V4L2_BUF_FLAG_PFRAME |
-		 V4L2_BUF_FLAG_BFRAME |
-		 V4L2_BUF_FLAG_TSTAMP_SRC_MASK);
+	v4l2_m2m_buf_copy_data(in_vb, out_vb, !ctx->is_enc);
 
 	return 0;
 }
@@ -1083,6 +1073,7 @@ static int queue_init(void *priv, struct vb2_queue *src_vq,
 	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	src_vq->lock = ctx->is_enc ? &ctx->dev->enc_mutex :
 		&ctx->dev->dec_mutex;
+	src_vq->supports_tags = true;
 
 	ret = vb2_queue_init(src_vq);
 	if (ret)
@@ -1098,6 +1089,7 @@ static int queue_init(void *priv, struct vb2_queue *src_vq,
 	dst_vq->mem_ops = &vb2_vmalloc_memops;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	dst_vq->lock = src_vq->lock;
+	dst_vq->supports_tags = true;
 
 	return vb2_queue_init(dst_vq);
 }
-- 
2.19.1

[PATCHv4 10/10] cedrus: add tag support

[hverkuil-cisco]

From: Hans Verkuil <hverkuil-cisco@xs4all.nl>

Replace old reference frame indices by new tag method.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c          |  9 --------
 drivers/staging/media/sunxi/cedrus/cedrus.h   |  9 +++++---
 .../staging/media/sunxi/cedrus/cedrus_dec.c   |  2 ++
 .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 21 ++++++++-----------
 .../staging/media/sunxi/cedrus/cedrus_video.c |  2 ++
 include/uapi/linux/v4l2-controls.h            | 14 +++++--------
 6 files changed, 24 insertions(+), 33 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index 129a986fa7e1..e859496e4e95 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -1661,15 +1661,6 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
 			return -EINVAL;
 		}
 
-		if (p_mpeg2_slice_params->backward_ref_index >= VIDEO_MAX_FRAME ||
-		    p_mpeg2_slice_params->forward_ref_index >= VIDEO_MAX_FRAME)
-			return -EINVAL;
-
-		if (p_mpeg2_slice_params->pad ||
-		    p_mpeg2_slice_params->picture.pad ||
-		    p_mpeg2_slice_params->sequence.pad)
-			return -EINVAL;
-
 		return 0;
 
 	case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION:
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h
index 3f61248c57ac..781676b55a1b 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.h
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.h
@@ -142,11 +142,14 @@ static inline dma_addr_t cedrus_buf_addr(struct vb2_buffer *buf,
 }
 
 static inline dma_addr_t cedrus_dst_buf_addr(struct cedrus_ctx *ctx,
-					     unsigned int index,
-					     unsigned int plane)
+					     int index, unsigned int plane)
 {
-	struct vb2_buffer *buf = ctx->dst_bufs[index];
+	struct vb2_buffer *buf;
 
+	if (index < 0)
+		return 0;
+
+	buf = ctx->dst_bufs[index];
 	return buf ? cedrus_buf_addr(buf, &ctx->dst_fmt, plane) : 0;
 }
 
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
index e40180a33951..0cfd6036d0cd 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
@@ -53,6 +53,8 @@ void cedrus_device_run(void *priv)
 		break;
 	}
 
+	v4l2_m2m_buf_copy_data(run.src, run.dst, true);
+
 	dev->dec_ops[ctx->current_codec]->setup(ctx, &run);
 
 	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
index 9abd39cae38c..fdde9a099153 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_mpeg2.c
@@ -82,7 +82,10 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx, struct cedrus_run *run)
 	dma_addr_t fwd_luma_addr, fwd_chroma_addr;
 	dma_addr_t bwd_luma_addr, bwd_chroma_addr;
 	struct cedrus_dev *dev = ctx->dev;
+	struct vb2_queue *cap_q = &ctx->fh.m2m_ctx->cap_q_ctx.q;
 	const u8 *matrix;
+	int forward_idx;
+	int backward_idx;
 	unsigned int i;
 	u32 reg;
 
@@ -156,23 +159,17 @@ static void cedrus_mpeg2_setup(struct cedrus_ctx *ctx, struct cedrus_run *run)
 	cedrus_write(dev, VE_DEC_MPEG_PICBOUNDSIZE, reg);
 
 	/* Forward and backward prediction reference buffers. */
+	forward_idx = vb2_find_tag(cap_q, slice_params->forward_ref_tag, 0);
 
-	fwd_luma_addr = cedrus_dst_buf_addr(ctx,
-					    slice_params->forward_ref_index,
-					    0);
-	fwd_chroma_addr = cedrus_dst_buf_addr(ctx,
-					      slice_params->forward_ref_index,
-					      1);
+	fwd_luma_addr = cedrus_dst_buf_addr(ctx, forward_idx, 0);
+	fwd_chroma_addr = cedrus_dst_buf_addr(ctx, forward_idx, 1);
 
 	cedrus_write(dev, VE_DEC_MPEG_FWD_REF_LUMA_ADDR, fwd_luma_addr);
 	cedrus_write(dev, VE_DEC_MPEG_FWD_REF_CHROMA_ADDR, fwd_chroma_addr);
 
-	bwd_luma_addr = cedrus_dst_buf_addr(ctx,
-					    slice_params->backward_ref_index,
-					    0);
-	bwd_chroma_addr = cedrus_dst_buf_addr(ctx,
-					      slice_params->backward_ref_index,
-					      1);
+	backward_idx = vb2_find_tag(cap_q, slice_params->backward_ref_tag, 0);
+	bwd_luma_addr = cedrus_dst_buf_addr(ctx, backward_idx, 0);
+	bwd_chroma_addr = cedrus_dst_buf_addr(ctx, backward_idx, 1);
 
 	cedrus_write(dev, VE_DEC_MPEG_BWD_REF_LUMA_ADDR, bwd_luma_addr);
 	cedrus_write(dev, VE_DEC_MPEG_BWD_REF_CHROMA_ADDR, bwd_chroma_addr);
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_video.c b/drivers/staging/media/sunxi/cedrus/cedrus_video.c
index 5c5fce678b93..293df48326cc 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_video.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_video.c
@@ -522,6 +522,7 @@ int cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
 	src_vq->lock = &ctx->dev->dev_mutex;
 	src_vq->dev = ctx->dev->dev;
 	src_vq->supports_requests = true;
+	src_vq->supports_tags = true;
 
 	ret = vb2_queue_init(src_vq);
 	if (ret)
@@ -537,6 +538,7 @@ int cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
 	dst_vq->lock = &ctx->dev->dev_mutex;
 	dst_vq->dev = ctx->dev->dev;
+	dst_vq->supports_tags = true;
 
 	return vb2_queue_init(dst_vq);
 }
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 998983a6e6b7..45a55bb27e5a 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -1109,10 +1109,9 @@ struct v4l2_mpeg2_sequence {
 	__u32	vbv_buffer_size;
 
 	/* ISO/IEC 13818-2, ITU-T Rec. H.262: Sequence extension */
-	__u8	profile_and_level_indication;
+	__u16	profile_and_level_indication;
 	__u8	progressive_sequence;
 	__u8	chroma_format;
-	__u8	pad;
 };
 
 struct v4l2_mpeg2_picture {
@@ -1130,23 +1129,20 @@ struct v4l2_mpeg2_picture {
 	__u8	intra_vlc_format;
 	__u8	alternate_scan;
 	__u8	repeat_first_field;
-	__u8	progressive_frame;
-	__u8	pad;
+	__u16	progressive_frame;
 };
 
 struct v4l2_ctrl_mpeg2_slice_params {
 	__u32	bit_size;
 	__u32	data_bit_offset;
+	__u32	backward_ref_tag;
+	__u32	forward_ref_tag;
 
 	struct v4l2_mpeg2_sequence sequence;
 	struct v4l2_mpeg2_picture picture;
 
 	/* ISO/IEC 13818-2, ITU-T Rec. H.262: Slice */
-	__u8	quantiser_scale_code;
-
-	__u8	backward_ref_index;
-	__u8	forward_ref_index;
-	__u8	pad;
+	__u32	quantiser_scale_code;
 };
 
 struct v4l2_ctrl_mpeg2_quantization {
-- 
2.19.1

[PATCHv4 11/10] extended-controls.rst: update the mpeg2 compound controls

[Hans Verkuil]

The layout of the compound controls has changed to fix
32/64 bit alignment issues and the use of tags instead of
buffer indices to refer to buffers. Note that these controls
are only used by the cedrus staging driver.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
 .../media/uapi/v4l/extended-controls.rst      | 24 ++++++++++---------
 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst
index 65a1d873196b..b9e3af29a704 100644
--- a/Documentation/media/uapi/v4l/extended-controls.rst
+++ b/Documentation/media/uapi/v4l/extended-controls.rst
@@ -1528,17 +1528,19 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
       - ``picture``
       - Structure with MPEG-2 picture metadata, merging relevant fields from
 	the picture header and picture coding extension parts of the bitstream.
-    * - __u8
+    * - __u32
+      - ``backward_ref_tag``
+      - Tag for the V4L2 buffer to use as backward reference, used with
+	B-coded and P-coded frames. The tag refers to the ``tag`` field in
+	struct :c:type:`v4l2_buffer`.
+    * - __u32
+      - ``forward_ref_tag``
+      - Tag for the V4L2 buffer to use as forward reference, used with
+	B-coded frames. The tag refers to the ``tag`` field in
+	struct :c:type:`v4l2_buffer`.
+    * - __u32
       - ``quantiser_scale_code``
       - Code used to determine the quantization scale to use for the IDCT.
-    * - __u8
-      - ``backward_ref_index``
-      - Index for the V4L2 buffer to use as backward reference, used with
-	B-coded and P-coded frames.
-    * - __u8
-      - ``forward_ref_index``
-      - Index for the V4L2 buffer to use as forward reference, used with
-	B-coded frames.

 .. c:type:: v4l2_mpeg2_sequence

@@ -1559,7 +1561,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
       - ``vbv_buffer_size``
       - Used to calculate the required size of the video buffering verifier,
 	defined (in bits) as: 16 * 1024 * vbv_buffer_size.
-    * - __u8
+    * - __u16
       - ``profile_and_level_indication``
       - The current profile and level indication as extracted from the
 	bitstream.
@@ -1617,7 +1619,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
     * - __u8
       - ``repeat_first_field``
       - This flag affects the decoding process of progressive frames.
-    * - __u8
+    * - __u16
       - ``progressive_frame``
       - Indicates whether the current frame is progressive.

-- 
2.19.1

Re: [PATCHv4 00/10] vb2/cedrus: add tag support

[Hans Verkuil]

(fix copy and paste error in the cover letter)

As was discussed here (among other places):

https://lkml.org/lkml/2018/10/19/440

using capture queue buffer indices to refer to reference frames is
not a good idea. A better idea is to use a 'tag' where the
application can assign a u32 tag to an output buffer, which is then
copied to the capture buffer(s) derived from the output buffer.

It has been suggested that the timestamp can be used for this. But
there are a number of reasons why this is a bad idea:

1) the struct timeval is converted to a u64 in vb2. So there can be
   all sorts of unexpected conversion issues. In particular, the
   output of ns_to_timeval(timeval_to_ns(tv)) does not necessarily
   match the input.

2) it gets worse with the y2038 code where userspace either deals
   with a 32 bit tv_sec value or a 64 bit value.

In other words, using timestamp for this is not a good idea.

This implementation adds a new tag field in a union with the reserved2
field. The interpretation of that union depends on the flags field, so
it still can be used for other things as well. In addition, in the previous
patches the tag was in a union with the timecode field (again determined
by the flags field), so if we need to cram additional information in this
struct we can always put it in a union with the timecode field as well.
It worked for the tag, it should work for other things.

But we really need to start looking at a struct v4l2_ext_buffer.

The first three patches add core tag support, the next two patches document
the tag support, then a new helper function is added to v4l2-mem2mem.c
to easily copy data from a source to a destination buffer that drivers
can use.

Next a new supports_tags vb2_queue flag is added to indicate that
the driver supports tags. Ideally this should not be necessary, but
that would require that all m2m drivers are converted to using the
new helper function introduced in the previous patch. That takes more
time then I have now.

Finally the vim2m, vicodec and cedrus drivers are converted to support
tags.

I also removed the 'pad' fields from the mpeg2 control structs (it
should never been added in the first place) and aligned the structs
to a u32 boundary.

Note that this might change further (Paul suggested using bitfields).

Also note that the cedrus code doesn't set the sequence counter, that's
something that should still be added before this driver can be moved
out of staging.

Note: if no buffer is found for a certain tag, then the dma address
is just set to 0. That happened before as well with invalid buffer
indices. This should be checked in the driver!

Regards,

        Hans

Changes since v3:

- use reserved2 for the tag
- split the documentation in two: one documenting the tag, one
  cleaning up the timecode documentation.

Changes since v2:

- rebased
- added Reviewed-by tags
- fixed a few remaining references in the documentation to the old
  v4l2_buffer_tag struct that was used in early versions of this
  series.

Changes since v1:

- changed to a u32 tag. Using a 64 bit tag was overly complicated due
  to the bad layout of the v4l2_buffer struct, and there is no real
  need for it by applications.

Main changes since the RFC:

- Added new buffer capability flag
- Added m2m helper to copy data between buffers
- Added documentation
- Added tag logging in v4l2-ioctl.c


Hans Verkuil (10):
  videodev2.h: add tag support
  vb2: add tag support
  v4l2-ioctl.c: log v4l2_buffer tag
  buffer.rst: document the new buffer tag feature.
  buffer.rst: clean up timecode documentation
  v4l2-mem2mem: add v4l2_m2m_buf_copy_data helper function
  vb2: add new supports_tags queue flag
  vim2m: add tag support
  vicodec: add tag support
  cedrus: add tag support

 Documentation/media/uapi/v4l/buffer.rst       | 28 +++++++++----
 .../media/uapi/v4l/vidioc-reqbufs.rst         |  4 ++
 .../media/common/videobuf2/videobuf2-v4l2.c   | 41 ++++++++++++++++---
 drivers/media/platform/vicodec/vicodec-core.c | 14 ++-----
 drivers/media/platform/vim2m.c                | 14 ++-----
 drivers/media/v4l2-core/v4l2-ctrls.c          |  9 ----
 drivers/media/v4l2-core/v4l2-ioctl.c          |  9 ++--
 drivers/media/v4l2-core/v4l2-mem2mem.c        | 23 +++++++++++
 drivers/staging/media/sunxi/cedrus/cedrus.h   |  9 ++--
 .../staging/media/sunxi/cedrus/cedrus_dec.c   |  2 +
 .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 21 ++++------
 .../staging/media/sunxi/cedrus/cedrus_video.c |  2 +
 include/media/v4l2-mem2mem.h                  | 21 ++++++++++
 include/media/videobuf2-core.h                |  2 +
 include/media/videobuf2-v4l2.h                | 21 +++++++++-
 include/uapi/linux/v4l2-controls.h            | 14 +++----
 include/uapi/linux/videodev2.h                |  9 +++-
 17 files changed, 168 insertions(+), 75 deletions(-)

-- 
2.19.1

Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Hans Verkuil]

Mauro raised a number of objections on irc regarding tags:

https://linuxtv.org/irc/irclogger_log/media-maint?date=2018-12-06,Thu

I would like to setup an irc meeting to discuss this and come to a
conclusion, since we need to decide this soon since this is critical
for stateless codec support.

Unfortunately timezone-wise this is a bit of a nightmare. I think
that at least Mauro, myself and Tomasz Figa should be there, so UTC-2,
UTC+1 and UTC+9 (if I got that right).

I propose 9 AM UTC which I think will work for everyone except Nicolas.
Any day next week works for me, and (for now) as well for Mauro. Let's pick
Monday to start with, and if you want to join in, then let me know. If that
day doesn't work for you, let me know what other days next week do work for
you.

Regards,

	Hans

On 12/05/18 11:20, hverkuil-cisco@xs4all.nl wrote:
> From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> 
> Add support for 'tags' to struct v4l2_buffer. These can be used
> by m2m devices so userspace can set a tag for an output buffer and
> this value will then be copied to the capture buffer(s).
> 
> This tag can be used to refer to capture buffers, something that
> is needed by stateless HW codecs.
> 
> The new V4L2_BUF_CAP_SUPPORTS_TAGS capability indicates whether
> or not tags are supported.
> 
> Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
> ---
>  include/uapi/linux/videodev2.h | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 2db1635de956..9095d7abe10d 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -881,6 +881,7 @@ struct v4l2_requestbuffers {
>  #define V4L2_BUF_CAP_SUPPORTS_DMABUF	(1 << 2)
>  #define V4L2_BUF_CAP_SUPPORTS_REQUESTS	(1 << 3)
>  #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
> +#define V4L2_BUF_CAP_SUPPORTS_TAGS	(1 << 5)
>  
>  /**
>   * struct v4l2_plane - plane info for multi-planar buffers
> @@ -940,6 +941,7 @@ struct v4l2_plane {
>   * @length:	size in bytes of the buffer (NOT its payload) for single-plane
>   *		buffers (when type != *_MPLANE); number of elements in the
>   *		planes array for multi-plane buffers
> + * @tag:	buffer tag
>   * @request_fd: fd of the request that this buffer should use
>   *
>   * Contains data exchanged by application and driver using one of the Streaming
> @@ -964,7 +966,10 @@ struct v4l2_buffer {
>  		__s32		fd;
>  	} m;
>  	__u32			length;
> -	__u32			reserved2;
> +	union {
> +		__u32		reserved2;
> +		__u32		tag;
> +	};
>  	union {
>  		__s32		request_fd;
>  		__u32		reserved;
> @@ -990,6 +995,8 @@ struct v4l2_buffer {
>  #define V4L2_BUF_FLAG_IN_REQUEST		0x00000080
>  /* timecode field is valid */
>  #define V4L2_BUF_FLAG_TIMECODE			0x00000100
> +/* tag field is valid */
> +#define V4L2_BUF_FLAG_TAG			0x00000200
>  /* Buffer is prepared for queuing */
>  #define V4L2_BUF_FLAG_PREPARED			0x00000400
>  /* Cache handling flags */
> 

Re: Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Alexandre Courbot]

Hi Hans,

On Fri, Dec 7, 2018 at 12:08 AM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:
>
> Mauro raised a number of objections on irc regarding tags:
>
> https://linuxtv.org/irc/irclogger_log/media-maint?date=2018-12-06,Thu
>
> I would like to setup an irc meeting to discuss this and come to a
> conclusion, since we need to decide this soon since this is critical
> for stateless codec support.
>
> Unfortunately timezone-wise this is a bit of a nightmare. I think
> that at least Mauro, myself and Tomasz Figa should be there, so UTC-2,
> UTC+1 and UTC+9 (if I got that right).
>
> I propose 9 AM UTC which I think will work for everyone except Nicolas.
> Any day next week works for me, and (for now) as well for Mauro. Let's pick
> Monday to start with, and if you want to join in, then let me know. If that
> day doesn't work for you, let me know what other days next week do work for
> you.

Monday (or any other day next week) 9AM UTC should work for me!

Thanks,
Alex.

Re: Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Tomasz Figa]

Hi Hans,

On Fri, Dec 7, 2018 at 12:08 AM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:
>
> Mauro raised a number of objections on irc regarding tags:
>
> https://linuxtv.org/irc/irclogger_log/media-maint?date=2018-12-06,Thu
>
> I would like to setup an irc meeting to discuss this and come to a
> conclusion, since we need to decide this soon since this is critical
> for stateless codec support.
>
> Unfortunately timezone-wise this is a bit of a nightmare. I think
> that at least Mauro, myself and Tomasz Figa should be there, so UTC-2,
> UTC+1 and UTC+9 (if I got that right).
>
> I propose 9 AM UTC which I think will work for everyone except Nicolas.
> Any day next week works for me, and (for now) as well for Mauro. Let's pick
> Monday to start with, and if you want to join in, then let me know. If that
> day doesn't work for you, let me know what other days next week do work for
> you.

9am UTC (which should be 6pm JST)  works for me on any day this week.

Best regards,
Tomasz

>
> Regards,
>
>         Hans
>
> On 12/05/18 11:20, hverkuil-cisco@xs4all.nl wrote:
> > From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> >
> > Add support for 'tags' to struct v4l2_buffer. These can be used
> > by m2m devices so userspace can set a tag for an output buffer and
> > this value will then be copied to the capture buffer(s).
> >
> > This tag can be used to refer to capture buffers, something that
> > is needed by stateless HW codecs.
> >
> > The new V4L2_BUF_CAP_SUPPORTS_TAGS capability indicates whether
> > or not tags are supported.
> >
> > Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> > Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
> > ---
> >  include/uapi/linux/videodev2.h | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 2db1635de956..9095d7abe10d 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -881,6 +881,7 @@ struct v4l2_requestbuffers {
> >  #define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2)
> >  #define V4L2_BUF_CAP_SUPPORTS_REQUESTS       (1 << 3)
> >  #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
> > +#define V4L2_BUF_CAP_SUPPORTS_TAGS   (1 << 5)
> >
> >  /**
> >   * struct v4l2_plane - plane info for multi-planar buffers
> > @@ -940,6 +941,7 @@ struct v4l2_plane {
> >   * @length:  size in bytes of the buffer (NOT its payload) for single-plane
> >   *           buffers (when type != *_MPLANE); number of elements in the
> >   *           planes array for multi-plane buffers
> > + * @tag:     buffer tag
> >   * @request_fd: fd of the request that this buffer should use
> >   *
> >   * Contains data exchanged by application and driver using one of the Streaming
> > @@ -964,7 +966,10 @@ struct v4l2_buffer {
> >               __s32           fd;
> >       } m;
> >       __u32                   length;
> > -     __u32                   reserved2;
> > +     union {
> > +             __u32           reserved2;
> > +             __u32           tag;
> > +     };
> >       union {
> >               __s32           request_fd;
> >               __u32           reserved;
> > @@ -990,6 +995,8 @@ struct v4l2_buffer {
> >  #define V4L2_BUF_FLAG_IN_REQUEST             0x00000080
> >  /* timecode field is valid */
> >  #define V4L2_BUF_FLAG_TIMECODE                       0x00000100
> > +/* tag field is valid */
> > +#define V4L2_BUF_FLAG_TAG                    0x00000200
> >  /* Buffer is prepared for queuing */
> >  #define V4L2_BUF_FLAG_PREPARED                       0x00000400
> >  /* Cache handling flags */
> >
>

Re: Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Tomasz Figa]

Hi Mauro,

On Mon, Dec 10, 2018 at 1:31 PM Mauro Carvalho Chehab
<mchehab@kernel.org> wrote:
>
> In time: please reply to mchehab@kernel.org.
>
>
>
> Em 10 de dezembro de 2018 02:28:21 BRST, Mauro Carvalho Chehab <mchehab@osg.samsung.com> escreveu:
>>
>> Let's do it on Wed.
>>
>> I'm very busy on Monday and Tuesday.

Do you mean Wednesday in your time zone? If so, that would be Thursday
for Europe and Asia.

Regardless of that, it should work for me.

Best regards,
Tomasz

>>
>> Regards,
>> Mauro
>>
>> Em 10 de dezembro de 2018 01:18:38 BRST, Tomasz Figa <tfiga@chromium.org> escreveu:
>>>
>>> Hi Hans,
>>>
>>> On Fri, Dec 7, 2018 at 12:08 AM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:
>>>>
>>>>
>>>>  Mauro raised a number of objections on irc regarding tags:
>>>>
>>>>  https://linuxtv.org/irc/irclogger_log/media-maint?date=2018-12-06,Thu
>>>>
>>>>  I would like to setup an irc meeting to discuss this and come to a
>>>>  conclusion, since we need to decide this soon since this is critical
>>>>  for stateless codec support.
>>>>
>>>>  Unfortunately timezone-wise this is a bit of a nightmare. I think
>>>>  that at least Mauro, myself and Tomasz Figa should be there, so UTC-2,
>>>>  UTC+1 and UTC+9 (if I got that right).
>>>>
>>>>  I propose 9 AM UTC which I think will work for everyone except Nicolas.
>>>>  Any day next week works for me, and (for now) as well for Mauro. Let's pick
>>>>  Monday to start with, and if you want to join in, then let me know. If that
>>>>  day doesn't work for you, let me know what other days next week do work for
>>>>  you.
>>>
>>>
>>> 9am UTC (which should be 6pm JST)  works for me on any day this week.
>>>
>>> Best regards,
>>> Tomasz
>>>
>>>>
>>>>  Regards,
>>>>
>>>>          Hans
>>>>
>>>>  On 12/05/18 11:20, hverkuil-cisco@xs4all.nl wrote:
>>>>>
>>>>>  From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
>>>>>
>>>>>  Add support for 'tags' to struct v4l2_buffer. These can be used
>>>>>  by m2m devices so userspace can set a tag for an output buffer and
>>>>>  this value will then be copied to the capture buffer(s).
>>>>>
>>>>>  This tag can be used to refer to capture buffers, something that
>>>>>  is needed by stateless HW codecs.
>>>>>
>>>>>  The new V4L2_BUF_CAP_SUPPORTS_TAGS capability indicates whether
>>>>>  or not tags are supported.
>>>>>
>>>>>  Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
>>>>>  Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
>>>>>  Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
>>>>> ________________________________
>>>>>   include/uapi/linux/videodev2.h | 9 ++++++++-
>>>>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>>>>
>>>>>  diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>>  index 2db1635de956..9095d7abe10d 100644
>>>>>  --- a/include/uapi/linux/videodev2.h
>>>>>  +++ b/include/uapi/linux/videodev2.h
>>>>>  @@ -881,6 +881,7 @@ struct v4l2_requestbuffers {
>>>>>   #define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2)
>>>>>   #define V4L2_BUF_CAP_SUPPORTS_REQUESTS       (1 << 3)
>>>>>   #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
>>>>>  +#define V4L2_BUF_CAP_SUPPORTS_TAGS   (1 << 5)
>>>>>
>>>>>   /**
>>>>>    * struct v4l2_plane - plane info for multi-planar buffers
>>>>>  @@ -940,6 +941,7 @@ struct v4l2_plane {
>>>>>    * @length:  size in bytes of the buffer (NOT its payload) for single-plane
>>>>>    *           buffers (when type != *_MPLANE); number of elements in the
>>>>>    *           planes array for multi-plane buffers
>>>>>  + * @tag:     buffer tag
>>>>>    * @request_fd: fd of the request that this buffer should use
>>>>>    *
>>>>>    * Contains data exchanged by application and driver using one of the Streaming
>>>>>  @@ -964,7 +966,10 @@ struct v4l2_buffer {
>>>>>                __s32           fd;
>>>>>        } m;
>>>>>        __u32                   length;
>>>>>  -     __u32                   reserved2;
>>>>>  +     union {
>>>>>  +             __u32           reserved2;
>>>>>  +             __u32           tag;
>>>>>  +     };
>>>>>        union {
>>>>>                __s32           request_fd;
>>>>>                __u32           reserved;
>>>>>  @@ -990,6 +995,8 @@ struct v4l2_buffer {
>>>>>   #define V4L2_BUF_FLAG_IN_REQUEST             0x00000080
>>>>>   /* timecode field is valid */
>>>>>   #define V4L2_BUF_FLAG_TIMECODE                       0x00000100
>>>>>  +/* tag field is valid */
>>>>>  +#define V4L2_BUF_FLAG_TAG                    0x00000200
>>>>>   /* Buffer is prepared for queuing */
>>>>>   #define V4L2_BUF_FLAG_PREPARED                       0x00000400
>>>>>   /* Cache handling flags */
>>>>>
>>>>
>
> --
>
> Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Mauro Carvalho Chehab]

Hi Tomasz,

I mean Wed morning on my TZ, with would be Wed late afternoon on your TZ :-)

As Hans pointed, we are at UTC-2 (Brazil), UTC+1 (Europe CET)
and UTC+9 (Japan). Hans is proposing to have it 9am UTC.

E, g:

	https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018&month=12&day=12&hour=9&min=0&sec=0&p1=45&p2=101&p3=248

Regards,
Mauro



Em Mon, 10 Dec 2018 15:11:26 +0900
Tomasz Figa <tfiga@chromium.org> escreveu:

> Hi Mauro,
> 
> On Mon, Dec 10, 2018 at 1:31 PM Mauro Carvalho Chehab
> <mchehab@kernel.org> wrote:
> >
> > In time: please reply to mchehab@kernel.org.
> >
> >
> >
> > Em 10 de dezembro de 2018 02:28:21 BRST, Mauro Carvalho Chehab <mchehab@osg.samsung.com> escreveu:  
> >>
> >> Let's do it on Wed.
> >>
> >> I'm very busy on Monday and Tuesday.  
> 
> Do you mean Wednesday in your time zone? If so, that would be Thursday
> for Europe and Asia.
> 
> Regardless of that, it should work for me.
> 
> Best regards,
> Tomasz
> 
> >>
> >> Regards,
> >> Mauro
> >>
> >> Em 10 de dezembro de 2018 01:18:38 BRST, Tomasz Figa <tfiga@chromium.org> escreveu:  
> >>>
> >>> Hi Hans,
> >>>
> >>> On Fri, Dec 7, 2018 at 12:08 AM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:  
> >>>>
> >>>>
> >>>>  Mauro raised a number of objections on irc regarding tags:
> >>>>
> >>>>  https://linuxtv.org/irc/irclogger_log/media-maint?date=2018-12-06,Thu
> >>>>
> >>>>  I would like to setup an irc meeting to discuss this and come to a
> >>>>  conclusion, since we need to decide this soon since this is critical
> >>>>  for stateless codec support.
> >>>>
> >>>>  Unfortunately timezone-wise this is a bit of a nightmare. I think
> >>>>  that at least Mauro, myself and Tomasz Figa should be there, so UTC-2,
> >>>>  UTC+1 and UTC+9 (if I got that right).
> >>>>
> >>>>  I propose 9 AM UTC which I think will work for everyone except Nicolas.
> >>>>  Any day next week works for me, and (for now) as well for Mauro. Let's pick
> >>>>  Monday to start with, and if you want to join in, then let me know. If that
> >>>>  day doesn't work for you, let me know what other days next week do work for
> >>>>  you.  
> >>>
> >>>
> >>> 9am UTC (which should be 6pm JST)  works for me on any day this week.
> >>>
> >>> Best regards,
> >>> Tomasz
> >>>  
> >>>>
> >>>>  Regards,
> >>>>
> >>>>          Hans
> >>>>
> >>>>  On 12/05/18 11:20, hverkuil-cisco@xs4all.nl wrote:  
> >>>>>
> >>>>>  From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> >>>>>
> >>>>>  Add support for 'tags' to struct v4l2_buffer. These can be used
> >>>>>  by m2m devices so userspace can set a tag for an output buffer and
> >>>>>  this value will then be copied to the capture buffer(s).
> >>>>>
> >>>>>  This tag can be used to refer to capture buffers, something that
> >>>>>  is needed by stateless HW codecs.
> >>>>>
> >>>>>  The new V4L2_BUF_CAP_SUPPORTS_TAGS capability indicates whether
> >>>>>  or not tags are supported.
> >>>>>
> >>>>>  Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> >>>>>  Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> >>>>>  Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
> >>>>> ________________________________
> >>>>>   include/uapi/linux/videodev2.h | 9 ++++++++-
> >>>>>   1 file changed, 8 insertions(+), 1 deletion(-)
> >>>>>
> >>>>>  diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >>>>>  index 2db1635de956..9095d7abe10d 100644
> >>>>>  --- a/include/uapi/linux/videodev2.h
> >>>>>  +++ b/include/uapi/linux/videodev2.h
> >>>>>  @@ -881,6 +881,7 @@ struct v4l2_requestbuffers {
> >>>>>   #define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2)
> >>>>>   #define V4L2_BUF_CAP_SUPPORTS_REQUESTS       (1 << 3)
> >>>>>   #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
> >>>>>  +#define V4L2_BUF_CAP_SUPPORTS_TAGS   (1 << 5)
> >>>>>
> >>>>>   /**
> >>>>>    * struct v4l2_plane - plane info for multi-planar buffers
> >>>>>  @@ -940,6 +941,7 @@ struct v4l2_plane {
> >>>>>    * @length:  size in bytes of the buffer (NOT its payload) for single-plane
> >>>>>    *           buffers (when type != *_MPLANE); number of elements in the
> >>>>>    *           planes array for multi-plane buffers
> >>>>>  + * @tag:     buffer tag
> >>>>>    * @request_fd: fd of the request that this buffer should use
> >>>>>    *
> >>>>>    * Contains data exchanged by application and driver using one of the Streaming
> >>>>>  @@ -964,7 +966,10 @@ struct v4l2_buffer {
> >>>>>                __s32           fd;
> >>>>>        } m;
> >>>>>        __u32                   length;
> >>>>>  -     __u32                   reserved2;
> >>>>>  +     union {
> >>>>>  +             __u32           reserved2;
> >>>>>  +             __u32           tag;
> >>>>>  +     };
> >>>>>        union {
> >>>>>                __s32           request_fd;
> >>>>>                __u32           reserved;
> >>>>>  @@ -990,6 +995,8 @@ struct v4l2_buffer {
> >>>>>   #define V4L2_BUF_FLAG_IN_REQUEST             0x00000080
> >>>>>   /* timecode field is valid */
> >>>>>   #define V4L2_BUF_FLAG_TIMECODE                       0x00000100
> >>>>>  +/* tag field is valid */
> >>>>>  +#define V4L2_BUF_FLAG_TAG                    0x00000200
> >>>>>   /* Buffer is prepared for queuing */
> >>>>>   #define V4L2_BUF_FLAG_PREPARED                       0x00000400
> >>>>>   /* Cache handling flags */
> >>>>>  
> >>>>  
> >
> > --
> >
> > Sent from my Android device with K-9 Mail. Please excuse my brevity.  



Thanks,
Mauro

Re: Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Hans Verkuil]

Wed works for me.

Regards,

	Hans

On 12/10/18 5:28 AM, Mauro Carvalho Chehab wrote:
> Let's do it on Wed.
> 
> I'm very busy on Monday and Tuesday.
> 
> Regards,
> Mauro
> 
> Em 10 de dezembro de 2018 01:18:38 BRST, Tomasz Figa <tfiga@chromium.org> escreveu:
> 
>     Hi Hans,
> 
>     On Fri, Dec 7, 2018 at 12:08 AM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:
> 
> 
>         Mauro raised a number of objections on irc regarding tags:
> 
>         https://linuxtv.org/irc/irclogger_log/media-maint?date=2018-12-06,Thu
> 
>         I would like to setup an irc meeting to discuss this and come to a
>         conclusion, since we need to decide this soon since this is critical
>         for stateless codec support.
> 
>         Unfortunately timezone-wise this is a bit of a nightmare. I think
>         that at least Mauro, myself and Tomasz Figa should be there, so UTC-2,
>         UTC+1 and UTC+9 (if I got that right).
> 
>         I propose 9 AM UTC which I think will work for everyone except Nicolas.
>         Any day next week works for me, and (for now) as well for Mauro. Let's pick
>         Monday to start with, and if you want to join in, then let me know. If that
>         day doesn't work for you, let me know what other days next week do work for
>         you.
> 
> 
>     9am UTC (which should be 6pm JST)  works for me on any day this week.
> 
>     Best regards,
>     Tomasz
> 
> 
>         Regards,
> 
>         Hans
> 
>         On 12/05/18 11:20, hverkuil-cisco@xs4all.nl wrote:
> 
>             From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> 
>             Add support for 'tags' to struct v4l2_buffer. These can be used
>             by m2m devices so userspace can set a tag for an output buffer and
>             this value will then be copied to the capture buffer(s).
> 
>             This tag can be used to refer to capture buffers, something that
>             is needed by stateless HW codecs.
> 
>             The new V4L2_BUF_CAP_SUPPORTS_TAGS capability indicates whether
>             or not tags are supported.
> 
>             Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
>             Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
>             Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
>             --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>             include/uapi/linux/videodev2.h | 9 ++++++++-
>             1 file changed, 8 insertions(+), 1 deletion(-)
> 
>             diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>             index 2db1635de956..9095d7abe10d 100644
>             --- a/include/uapi/linux/videodev2.h
>             +++ b/include/uapi/linux/videodev2.h
>             @@ -881,6 +881,7 @@ struct v4l2_requestbuffers {
>             #define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2)
>             #define V4L2_BUF_CAP_SUPPORTS_REQUESTS (1 << 3)
>             #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
>             +#define V4L2_BUF_CAP_SUPPORTS_TAGS (1 << 5)
> 
>             /**
>             * struct v4l2_plane - plane info for multi-planar buffers
>             @@ -940,6 +941,7 @@ struct v4l2_plane {
>             * @length: size in bytes of the buffer (NOT its payload) for single-plane
>             * buffers (when type != *_MPLANE); number of elements in the
>             * planes array for multi-plane buffers
>             + * @tag: buffer tag
>             * @request_fd: fd of the request that this buffer should use
>             *
>             * Contains data exchanged by application and driver using one of the Streaming
>             @@ -964,7 +966,10 @@ struct v4l2_buffer {
>             __s32 fd;
>             } m;
>             __u32 length;
>             - __u32 reserved2;
>             + union {
>             + __u32 reserved2;
>             + __u32 tag;
>             + };
>             union {
>             __s32 request_fd;
>             __u32 reserved;
>             @@ -990,6 +995,8 @@ struct v4l2_buffer {
>             #define V4L2_BUF_FLAG_IN_REQUEST 0x00000080
>             /* timecode field is valid */
>             #define V4L2_BUF_FLAG_TIMECODE 0x00000100
>             +/* tag field is valid */
>             +#define V4L2_BUF_FLAG_TAG 0x00000200
>             /* Buffer is prepared for queuing */
>             #define V4L2_BUF_FLAG_PREPARED 0x00000400
>             /* Cache handling flags */
> 
> 
> 
> -- 
> Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Tomasz Figa]

On Mon, Dec 10, 2018 at 4:57 PM Mauro Carvalho Chehab
<mchehab@kernel.org> wrote:
>
> Hi Tomasz,
>
> I mean Wed morning on my TZ, with would be Wed late afternoon on your TZ :-)
>
> As Hans pointed, we are at UTC-2 (Brazil), UTC+1 (Europe CET)
> and UTC+9 (Japan). Hans is proposing to have it 9am UTC.
>
> E, g:
>
>         https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018&month=12&day=12&hour=9&min=0&sec=0&p1=45&p2=101&p3=248
>

Okay, sorry, I got the time zones mixed up. Nevertheless, it works for me. :)

> Regards,
> Mauro
>
>
>
> Em Mon, 10 Dec 2018 15:11:26 +0900
> Tomasz Figa <tfiga@chromium.org> escreveu:
>
> > Hi Mauro,
> >
> > On Mon, Dec 10, 2018 at 1:31 PM Mauro Carvalho Chehab
> > <mchehab@kernel.org> wrote:
> > >
> > > In time: please reply to mchehab@kernel.org.
> > >
> > >
> > >
> > > Em 10 de dezembro de 2018 02:28:21 BRST, Mauro Carvalho Chehab <mchehab@osg.samsung.com> escreveu:
> > >>
> > >> Let's do it on Wed.
> > >>
> > >> I'm very busy on Monday and Tuesday.
> >
> > Do you mean Wednesday in your time zone? If so, that would be Thursday
> > for Europe and Asia.
> >
> > Regardless of that, it should work for me.
> >
> > Best regards,
> > Tomasz
> >
> > >>
> > >> Regards,
> > >> Mauro
> > >>
> > >> Em 10 de dezembro de 2018 01:18:38 BRST, Tomasz Figa <tfiga@chromium.org> escreveu:
> > >>>
> > >>> Hi Hans,
> > >>>
> > >>> On Fri, Dec 7, 2018 at 12:08 AM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:
> > >>>>
> > >>>>
> > >>>>  Mauro raised a number of objections on irc regarding tags:
> > >>>>
> > >>>>  https://linuxtv.org/irc/irclogger_log/media-maint?date=2018-12-06,Thu
> > >>>>
> > >>>>  I would like to setup an irc meeting to discuss this and come to a
> > >>>>  conclusion, since we need to decide this soon since this is critical
> > >>>>  for stateless codec support.
> > >>>>
> > >>>>  Unfortunately timezone-wise this is a bit of a nightmare. I think
> > >>>>  that at least Mauro, myself and Tomasz Figa should be there, so UTC-2,
> > >>>>  UTC+1 and UTC+9 (if I got that right).
> > >>>>
> > >>>>  I propose 9 AM UTC which I think will work for everyone except Nicolas.
> > >>>>  Any day next week works for me, and (for now) as well for Mauro. Let's pick
> > >>>>  Monday to start with, and if you want to join in, then let me know. If that
> > >>>>  day doesn't work for you, let me know what other days next week do work for
> > >>>>  you.
> > >>>
> > >>>
> > >>> 9am UTC (which should be 6pm JST)  works for me on any day this week.
> > >>>
> > >>> Best regards,
> > >>> Tomasz
> > >>>
> > >>>>
> > >>>>  Regards,
> > >>>>
> > >>>>          Hans
> > >>>>
> > >>>>  On 12/05/18 11:20, hverkuil-cisco@xs4all.nl wrote:
> > >>>>>
> > >>>>>  From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> > >>>>>
> > >>>>>  Add support for 'tags' to struct v4l2_buffer. These can be used
> > >>>>>  by m2m devices so userspace can set a tag for an output buffer and
> > >>>>>  this value will then be copied to the capture buffer(s).
> > >>>>>
> > >>>>>  This tag can be used to refer to capture buffers, something that
> > >>>>>  is needed by stateless HW codecs.
> > >>>>>
> > >>>>>  The new V4L2_BUF_CAP_SUPPORTS_TAGS capability indicates whether
> > >>>>>  or not tags are supported.
> > >>>>>
> > >>>>>  Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
> > >>>>>  Reviewed-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > >>>>>  Reviewed-by: Alexandre Courbot <acourbot@chromium.org>
> > >>>>> ________________________________
> > >>>>>   include/uapi/linux/videodev2.h | 9 ++++++++-
> > >>>>>   1 file changed, 8 insertions(+), 1 deletion(-)
> > >>>>>
> > >>>>>  diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > >>>>>  index 2db1635de956..9095d7abe10d 100644
> > >>>>>  --- a/include/uapi/linux/videodev2.h
> > >>>>>  +++ b/include/uapi/linux/videodev2.h
> > >>>>>  @@ -881,6 +881,7 @@ struct v4l2_requestbuffers {
> > >>>>>   #define V4L2_BUF_CAP_SUPPORTS_DMABUF (1 << 2)
> > >>>>>   #define V4L2_BUF_CAP_SUPPORTS_REQUESTS       (1 << 3)
> > >>>>>   #define V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS (1 << 4)
> > >>>>>  +#define V4L2_BUF_CAP_SUPPORTS_TAGS   (1 << 5)
> > >>>>>
> > >>>>>   /**
> > >>>>>    * struct v4l2_plane - plane info for multi-planar buffers
> > >>>>>  @@ -940,6 +941,7 @@ struct v4l2_plane {
> > >>>>>    * @length:  size in bytes of the buffer (NOT its payload) for single-plane
> > >>>>>    *           buffers (when type != *_MPLANE); number of elements in the
> > >>>>>    *           planes array for multi-plane buffers
> > >>>>>  + * @tag:     buffer tag
> > >>>>>    * @request_fd: fd of the request that this buffer should use
> > >>>>>    *
> > >>>>>    * Contains data exchanged by application and driver using one of the Streaming
> > >>>>>  @@ -964,7 +966,10 @@ struct v4l2_buffer {
> > >>>>>                __s32           fd;
> > >>>>>        } m;
> > >>>>>        __u32                   length;
> > >>>>>  -     __u32                   reserved2;
> > >>>>>  +     union {
> > >>>>>  +             __u32           reserved2;
> > >>>>>  +             __u32           tag;
> > >>>>>  +     };
> > >>>>>        union {
> > >>>>>                __s32           request_fd;
> > >>>>>                __u32           reserved;
> > >>>>>  @@ -990,6 +995,8 @@ struct v4l2_buffer {
> > >>>>>   #define V4L2_BUF_FLAG_IN_REQUEST             0x00000080
> > >>>>>   /* timecode field is valid */
> > >>>>>   #define V4L2_BUF_FLAG_TIMECODE                       0x00000100
> > >>>>>  +/* tag field is valid */
> > >>>>>  +#define V4L2_BUF_FLAG_TAG                    0x00000200
> > >>>>>   /* Buffer is prepared for queuing */
> > >>>>>   #define V4L2_BUF_FLAG_PREPARED                       0x00000400
> > >>>>>   /* Cache handling flags */
> > >>>>>
> > >>>>
> > >
> > > --
> > >
> > > Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
>
>
> Thanks,
> Mauro

Re: Invite for IRC meeting: Re: [PATCHv4 01/10] videodev2.h: add tag support

[Sakari Ailus]

On Mon, Dec 10, 2018 at 08:59:36AM +0100, Hans Verkuil wrote:
> Wed works for me.

Same here.

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

Re: [PATCHv4 00/10] As was discussed here (among other places):

[Randy Li]

On 12/5/18 6:20 PM, hverkuil-cisco@xs4all.nl wrote:
> From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
>
> https://lkml.org/lkml/2018/10/19/440
>
> using capture queue buffer indices to refer to reference frames is
> not a good idea. A better idea is to use a 'tag' where the
> application can assign a u32 tag to an output buffer, which is then
> copied to the capture buffer(s) derived from the output buffer.

I agree with the buffer tag. But coping from the OUPUT side to CAPTURE 
side is a bad idea.

I think we need a method to refer any buffers when they are allocated.

When I push a slice with its parameters into the driver, its previous 
picture in decoded order may not ready yet, using the buffer index, the 
driver

is still able to generate the registers table for it. Although you may 
though it just an additional buffer assignment work before wrote it into 
the device,

a few times seeking a buffer in a list. But there is a mode in new 
generation Rockchip device, called the link mode, you can put a 
registers into a memory, device would process that register link. You 
can't interrupt it. That is pretty useful for those codec converting.


Besides, I found it is little hard to refer a buffer with different 
offsets at the same time, it would be used for multiple slices and 
multiple CTU or filed picture which are not usual case nowadays.

> It has been suggested that the timestamp can be used for this. But
> there are a number of reasons why this is a bad idea:
I wonder why using a timestamp in the decoder or encoder, if the stream 
doesn't have a timestamp, we have to generate one.
>
> 1) the struct timeval is converted to a u64 in vb2. So there can be
>     all sorts of unexpected conversion issues. In particular, the
>     output of ns_to_timeval(timeval_to_ns(tv)) does not necessarily
>     match the input.
>
> 2) it gets worse with the y2038 code where userspace either deals
>     with a 32 bit tv_sec value or a 64 bit value.
>
> In other words, using timestamp for this is not a good idea.
>
> This implementation adds a new tag field in a union with the reserved2
> field. The interpretation of that union depends on the flags field, so
> it still can be used for other things as well. In addition, in the previous
> patches the tag was in a union with the timecode field (again determined
> by the flags field), so if we need to cram additional information in this
> struct we can always put it in a union with the timecode field as well.
> It worked for the tag, it should work for other things.
>
> But we really need to start looking at a struct v4l2_ext_buffer.
>
> The first three patches add core tag support, the next two patches document
> the tag support, then a new helper function is added to v4l2-mem2mem.c
> to easily copy data from a source to a destination buffer that drivers
> can use.
>
> Next a new supports_tags vb2_queue flag is added to indicate that
> the driver supports tags. Ideally this should not be necessary, but
> that would require that all m2m drivers are converted to using the
> new helper function introduced in the previous patch. That takes more
> time then I have now.
>
> Finally the vim2m, vicodec and cedrus drivers are converted to support
> tags.
>
> I also removed the 'pad' fields from the mpeg2 control structs (it
> should never been added in the first place) and aligned the structs
> to a u32 boundary.
>
> Note that this might change further (Paul suggested using bitfields).
>
> Also note that the cedrus code doesn't set the sequence counter, that's
> something that should still be added before this driver can be moved
> out of staging.
>
> Note: if no buffer is found for a certain tag, then the dma address
> is just set to 0. That happened before as well with invalid buffer
> indices. This should be checked in the driver!
>
> Regards,
>
>          Hans
>
> Changes since v3:
>
> - use reserved2 for the tag
> - split the documentation in two: one documenting the tag, one
>    cleaning up the timecode documentation.
>
> Changes since v2:
>
> - rebased
> - added Reviewed-by tags
> - fixed a few remaining references in the documentation to the old
>    v4l2_buffer_tag struct that was used in early versions of this
>    series.
>
> Changes since v1:
>
> - changed to a u32 tag. Using a 64 bit tag was overly complicated due
>    to the bad layout of the v4l2_buffer struct, and there is no real
>    need for it by applications.
>
> Main changes since the RFC:
>
> - Added new buffer capability flag
> - Added m2m helper to copy data between buffers
> - Added documentation
> - Added tag logging in v4l2-ioctl.c
>
>
> Hans Verkuil (10):
>    videodev2.h: add tag support
>    vb2: add tag support
>    v4l2-ioctl.c: log v4l2_buffer tag
>    buffer.rst: document the new buffer tag feature.
>    buffer.rst: clean up timecode documentation
>    v4l2-mem2mem: add v4l2_m2m_buf_copy_data helper function
>    vb2: add new supports_tags queue flag
>    vim2m: add tag support
>    vicodec: add tag support
>    cedrus: add tag support
>
>   Documentation/media/uapi/v4l/buffer.rst       | 28 +++++++++----
>   .../media/uapi/v4l/vidioc-reqbufs.rst         |  4 ++
>   .../media/common/videobuf2/videobuf2-v4l2.c   | 41 ++++++++++++++++---
>   drivers/media/platform/vicodec/vicodec-core.c | 14 ++-----
>   drivers/media/platform/vim2m.c                | 14 ++-----
>   drivers/media/v4l2-core/v4l2-ctrls.c          |  9 ----
>   drivers/media/v4l2-core/v4l2-ioctl.c          |  9 ++--
>   drivers/media/v4l2-core/v4l2-mem2mem.c        | 23 +++++++++++
>   drivers/staging/media/sunxi/cedrus/cedrus.h   |  9 ++--
>   .../staging/media/sunxi/cedrus/cedrus_dec.c   |  2 +
>   .../staging/media/sunxi/cedrus/cedrus_mpeg2.c | 21 ++++------
>   .../staging/media/sunxi/cedrus/cedrus_video.c |  2 +
>   include/media/v4l2-mem2mem.h                  | 21 ++++++++++
>   include/media/videobuf2-core.h                |  2 +
>   include/media/videobuf2-v4l2.h                | 21 +++++++++-
>   include/uapi/linux/v4l2-controls.h            | 14 +++----
>   include/uapi/linux/videodev2.h                |  9 +++-
>   17 files changed, 168 insertions(+), 75 deletions(-)
>