* [RFC v2 0/8] Move HEVC stateless controls out of staging
@ 2022-02-15 11:00 Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 1/8] videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY Benjamin Gaignard
` (10 more replies)
0 siblings, 11 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:00 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc, Benjamin Gaignard
This series aims to make HEVC uapi stable and usable for hardware
decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
and 2 out of the tree drivers (rkvdec and RPI).
The 3 first patches are from Hans to implement v4l2 dynamic control
feature which is need by patch 7 for V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET
definition.
Patch 4 move the existing uapi to stable, including definitions renaming
and CID number change to fit with v4l2 naming.
Patches 5 and 7 add fields needed for rkvdec and RPI decoders.
Patches 6 is cleaning up the uapi of useless field.
Patches 8 change one field description and name to define offset by
bytes rather than by bits
Benjamin
Benjamin Gaignard (5):
media: uapi: Move HEVC stateless controls out of staging
media: uapi: Add fields needed for RKVDEC driver
media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
media: uapi: Change data_bit_offset definition
Hans Verkuil (3):
videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
v4l2-ctrls: add support for dynamically allocated arrays.
vivid: add dynamic array test control
.../userspace-api/media/drivers/hantro.rst | 5 -
.../media/v4l/ext-ctrls-codec.rst | 58 ++--
.../media/v4l/vidioc-queryctrl.rst | 8 +
.../media/test-drivers/vivid/vivid-ctrls.c | 15 ++
drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++--
drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 ++++++++++---
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
drivers/staging/media/hantro/hantro_drv.c | 27 +-
drivers/staging/media/hantro/hantro_hevc.c | 8 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
.../staging/media/sunxi/cedrus/cedrus_h265.c | 13 +-
include/linux/hantro-media.h | 17 ++
include/media/hevc-ctrls.h | 250 ------------------
include/media/v4l2-ctrls.h | 48 +++-
include/uapi/linux/v4l2-controls.h | 224 ++++++++++++++++
include/uapi/linux/videodev2.h | 8 +
19 files changed, 640 insertions(+), 408 deletions(-)
create mode 100644 include/linux/hantro-media.h
delete mode 100644 include/media/hevc-ctrls.h
--
2.32.0
^ permalink raw reply [flat|nested] 35+ messages in thread
* [RFC v2 1/8] videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
@ 2022-02-15 11:00 ` Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 2/8] v4l2-ctrls: add support for dynamically allocated arrays Benjamin Gaignard
` (9 subsequent siblings)
10 siblings, 0 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:00 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add a new flag that indicates that this control is a dynamically sized
array. Also document this flag.
Currently dynamically sized arrays are limited to one dimensional arrays,
but that might change in the future if there is a need for it.
The initial use-case of dynamic arrays are stateless codecs. A frame
can be divided in many slices, so you want to provide an array containing
slice information for each slice. Typically the number of slices is small,
but the standard allow for hundreds or thousands of slices. Dynamic arrays
are a good solution since sizing the array for the worst case would waste
substantial amounts of memory.
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
.../userspace-api/media/v4l/vidioc-queryctrl.rst | 8 ++++++++
include/uapi/linux/videodev2.h | 1 +
2 files changed, 9 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
index 88f630252d98..a20dfa2a933b 100644
--- a/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
+++ b/Documentation/userspace-api/media/v4l/vidioc-queryctrl.rst
@@ -625,6 +625,14 @@ See also the examples in :ref:`control`.
``V4L2_CTRL_FLAG_GRABBED`` flag when buffers are allocated or
streaming is in progress since most drivers do not support changing
the format in that case.
+ * - ``V4L2_CTRL_FLAG_DYNAMIC_ARRAY``
+ - 0x0800
+ - This control is a dynamically sized 1-dimensional array. It
+ behaves the same as a regular array, except that the number
+ of elements as reported by the ``elems`` field is between 1 and
+ ``dims[0]``. So setting the control with a differently sized
+ array will change the ``elems`` field when the control is
+ queried afterwards.
Return Value
============
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index df8b9c486ba1..e27c8eae78c9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1884,6 +1884,7 @@ struct v4l2_querymenu {
#define V4L2_CTRL_FLAG_HAS_PAYLOAD 0x0100
#define V4L2_CTRL_FLAG_EXECUTE_ON_WRITE 0x0200
#define V4L2_CTRL_FLAG_MODIFY_LAYOUT 0x0400
+#define V4L2_CTRL_FLAG_DYNAMIC_ARRAY 0x0800
/* Query flags, to be ORed with the control ID */
#define V4L2_CTRL_FLAG_NEXT_CTRL 0x80000000
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [RFC v2 2/8] v4l2-ctrls: add support for dynamically allocated arrays.
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 1/8] videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY Benjamin Gaignard
@ 2022-02-15 11:00 ` Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 3/8] vivid: add dynamic array test control Benjamin Gaignard
` (8 subsequent siblings)
10 siblings, 0 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:00 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Implement support for dynamically allocated arrays.
Most of the changes concern keeping track of the number of elements
of the array and the number of elements allocated for the array and
reallocating memory if needed.
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++++---
drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 +++++++++++++++----
drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
include/media/v4l2-ctrls.h | 42 ++++-
5 files changed, 272 insertions(+), 71 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-api.c b/drivers/media/v4l2-core/v4l2-ctrls-api.c
index db9baa0bd05f..50d012ba3c02 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-api.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-api.c
@@ -97,29 +97,47 @@ static int def_to_user(struct v4l2_ext_control *c, struct v4l2_ctrl *ctrl)
return ptr_to_user(c, ctrl, ctrl->p_new);
}
-/* Helper function: copy the caller-provider value to the given control value */
-static int user_to_ptr(struct v4l2_ext_control *c,
- struct v4l2_ctrl *ctrl,
- union v4l2_ctrl_ptr ptr)
+/* Helper function: copy the caller-provider value as the new control value */
+static int user_to_new(struct v4l2_ext_control *c, struct v4l2_ctrl *ctrl)
{
int ret;
u32 size;
- ctrl->is_new = 1;
+ ctrl->is_new = 0;
+ if (ctrl->is_dyn_array &&
+ c->size > ctrl->p_dyn_alloc_elems * ctrl->elem_size) {
+ void *old = ctrl->p_dyn;
+ void *tmp = kvzalloc(2 * c->size, GFP_KERNEL);
+
+ if (!tmp)
+ return -ENOMEM;
+ memcpy(tmp, ctrl->p_new.p, ctrl->elems * ctrl->elem_size);
+ memcpy(tmp + c->size, ctrl->p_cur.p, ctrl->elems * ctrl->elem_size);
+ ctrl->p_new.p = tmp;
+ ctrl->p_cur.p = tmp + c->size;
+ ctrl->p_dyn = tmp;
+ ctrl->p_dyn_alloc_elems = c->size / ctrl->elem_size;
+ kvfree(old);
+ }
+
if (ctrl->is_ptr && !ctrl->is_string) {
+ unsigned int elems = c->size / ctrl->elem_size;
unsigned int idx;
- ret = copy_from_user(ptr.p, c->ptr, c->size) ? -EFAULT : 0;
- if (ret || !ctrl->is_array)
- return ret;
- for (idx = c->size / ctrl->elem_size; idx < ctrl->elems; idx++)
- ctrl->type_ops->init(ctrl, idx, ptr);
+ if (copy_from_user(ctrl->p_new.p, c->ptr, c->size))
+ return -EFAULT;
+ ctrl->is_new = 1;
+ if (ctrl->is_dyn_array)
+ ctrl->new_elems = elems;
+ else if (ctrl->is_array)
+ for (idx = elems; idx < ctrl->elems; idx++)
+ ctrl->type_ops->init(ctrl, idx, ctrl->p_new);
return 0;
}
switch (ctrl->type) {
case V4L2_CTRL_TYPE_INTEGER64:
- *ptr.p_s64 = c->value64;
+ *ctrl->p_new.p_s64 = c->value64;
break;
case V4L2_CTRL_TYPE_STRING:
size = c->size;
@@ -127,32 +145,27 @@ static int user_to_ptr(struct v4l2_ext_control *c,
return -ERANGE;
if (size > ctrl->maximum + 1)
size = ctrl->maximum + 1;
- ret = copy_from_user(ptr.p_char, c->string, size) ? -EFAULT : 0;
+ ret = copy_from_user(ctrl->p_new.p_char, c->string, size) ? -EFAULT : 0;
if (!ret) {
- char last = ptr.p_char[size - 1];
+ char last = ctrl->p_new.p_char[size - 1];
- ptr.p_char[size - 1] = 0;
+ ctrl->p_new.p_char[size - 1] = 0;
/*
* If the string was longer than ctrl->maximum,
* then return an error.
*/
- if (strlen(ptr.p_char) == ctrl->maximum && last)
+ if (strlen(ctrl->p_new.p_char) == ctrl->maximum && last)
return -ERANGE;
}
return ret;
default:
- *ptr.p_s32 = c->value;
+ *ctrl->p_new.p_s32 = c->value;
break;
}
+ ctrl->is_new = 1;
return 0;
}
-/* Helper function: copy the caller-provider value as the new control value */
-static int user_to_new(struct v4l2_ext_control *c, struct v4l2_ctrl *ctrl)
-{
- return user_to_ptr(c, ctrl, ctrl->p_new);
-}
-
/*
* VIDIOC_G/TRY/S_EXT_CTRLS implementation
*/
@@ -254,7 +267,31 @@ static int prepare_ext_ctrls(struct v4l2_ctrl_handler *hdl,
have_clusters = true;
if (ctrl->cluster[0] != ctrl)
ref = find_ref_lock(hdl, ctrl->cluster[0]->id);
- if (ctrl->is_ptr && !ctrl->is_string) {
+ if (ctrl->is_dyn_array) {
+ unsigned int max_size = ctrl->dims[0] * ctrl->elem_size;
+ unsigned int tot_size = ctrl->elem_size;
+
+ if (cs->which == V4L2_CTRL_WHICH_REQUEST_VAL)
+ tot_size *= ref->p_req_elems;
+ else
+ tot_size *= ctrl->elems;
+
+ c->size = ctrl->elem_size * (c->size / ctrl->elem_size);
+ if (get) {
+ if (c->size < tot_size) {
+ c->size = tot_size;
+ return -ENOSPC;
+ }
+ c->size = tot_size;
+ } else {
+ if (c->size > max_size) {
+ c->size = max_size;
+ return -ENOSPC;
+ }
+ if (!c->size)
+ return -EFAULT;
+ }
+ } else if (ctrl->is_ptr && !ctrl->is_string) {
unsigned int tot_size = ctrl->elems * ctrl->elem_size;
if (c->size < tot_size) {
@@ -346,7 +383,7 @@ static int class_check(struct v4l2_ctrl_handler *hdl, u32 which)
*
* Note that v4l2_g_ext_ctrls_common() with 'which' set to
* V4L2_CTRL_WHICH_REQUEST_VAL is only called if the request was
- * completed, and in that case valid_p_req is true for all controls.
+ * completed, and in that case p_req_valid is true for all controls.
*/
int v4l2_g_ext_ctrls_common(struct v4l2_ctrl_handler *hdl,
struct v4l2_ext_controls *cs,
@@ -430,7 +467,9 @@ int v4l2_g_ext_ctrls_common(struct v4l2_ctrl_handler *hdl,
if (is_default)
ret = def_to_user(cs->controls + idx, ref->ctrl);
- else if (is_request && ref->valid_p_req)
+ else if (is_request && ref->p_req_dyn_enomem)
+ ret = -ENOMEM;
+ else if (is_request && ref->p_req_valid)
ret = req_to_user(cs->controls + idx, ref);
else if (is_volatile)
ret = new_to_user(cs->controls + idx, ref->ctrl);
@@ -457,6 +496,17 @@ int v4l2_g_ext_ctrls(struct v4l2_ctrl_handler *hdl, struct video_device *vdev,
}
EXPORT_SYMBOL(v4l2_g_ext_ctrls);
+/* Validate a new control */
+static int validate_new(const struct v4l2_ctrl *ctrl, union v4l2_ctrl_ptr p_new)
+{
+ unsigned int idx;
+ int err = 0;
+
+ for (idx = 0; !err && idx < ctrl->new_elems; idx++)
+ err = ctrl->type_ops->validate(ctrl, idx, p_new);
+ return err;
+}
+
/* Validate controls. */
static int validate_ctrls(struct v4l2_ext_controls *cs,
struct v4l2_ctrl_helper *helpers,
@@ -872,6 +922,9 @@ int __v4l2_ctrl_s_ctrl_compound(struct v4l2_ctrl *ctrl,
/* It's a driver bug if this happens. */
if (WARN_ON(ctrl->type != type))
return -EINVAL;
+ /* Setting dynamic arrays is not (yet?) supported. */
+ if (WARN_ON(ctrl->is_dyn_array))
+ return -EINVAL;
memcpy(ctrl->p_new.p, p, ctrl->elems * ctrl->elem_size);
return set_ctrl(NULL, ctrl, 0);
}
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-core.c b/drivers/media/v4l2-core/v4l2-ctrls-core.c
index 85c2d3f39d96..2d24cb8d3926 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-core.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-core.c
@@ -973,11 +973,12 @@ EXPORT_SYMBOL(v4l2_ctrl_notify);
/* Copy the one value to another. */
static void ptr_to_ptr(struct v4l2_ctrl *ctrl,
- union v4l2_ctrl_ptr from, union v4l2_ctrl_ptr to)
+ union v4l2_ctrl_ptr from, union v4l2_ctrl_ptr to,
+ unsigned int elems)
{
if (ctrl == NULL)
return;
- memcpy(to.p, from.p_const, ctrl->elems * ctrl->elem_size);
+ memcpy(to.p, from.p_const, elems * ctrl->elem_size);
}
/* Copy the new value to the current value. */
@@ -990,8 +991,11 @@ void new_to_cur(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl, u32 ch_flags)
/* has_changed is set by cluster_changed */
changed = ctrl->has_changed;
- if (changed)
- ptr_to_ptr(ctrl, ctrl->p_new, ctrl->p_cur);
+ if (changed) {
+ if (ctrl->is_dyn_array)
+ ctrl->elems = ctrl->new_elems;
+ ptr_to_ptr(ctrl, ctrl->p_new, ctrl->p_cur, ctrl->elems);
+ }
if (ch_flags & V4L2_EVENT_CTRL_CH_FLAGS) {
/* Note: CH_FLAGS is only set for auto clusters. */
@@ -1021,36 +1025,122 @@ void cur_to_new(struct v4l2_ctrl *ctrl)
{
if (ctrl == NULL)
return;
- ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new);
+ if (ctrl->is_dyn_array)
+ ctrl->new_elems = ctrl->elems;
+ ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new, ctrl->new_elems);
+}
+
+static bool req_alloc_dyn_array(struct v4l2_ctrl_ref *ref, u32 elems)
+{
+ void *tmp;
+
+ if (elems < ref->p_req_dyn_alloc_elems)
+ return true;
+
+ tmp = kvmalloc(elems * ref->ctrl->elem_size, GFP_KERNEL);
+
+ if (!tmp) {
+ ref->p_req_dyn_enomem = true;
+ return false;
+ }
+ ref->p_req_dyn_enomem = false;
+ kvfree(ref->p_req.p);
+ ref->p_req.p = tmp;
+ ref->p_req_dyn_alloc_elems = elems;
+ return true;
}
/* Copy the new value to the request value */
void new_to_req(struct v4l2_ctrl_ref *ref)
{
+ struct v4l2_ctrl *ctrl;
+
if (!ref)
return;
- ptr_to_ptr(ref->ctrl, ref->ctrl->p_new, ref->p_req);
- ref->valid_p_req = true;
+
+ ctrl = ref->ctrl;
+ if (ctrl->is_dyn_array && !req_alloc_dyn_array(ref, ctrl->new_elems))
+ return;
+
+ ref->p_req_elems = ctrl->new_elems;
+ ptr_to_ptr(ctrl, ctrl->p_new, ref->p_req, ref->p_req_elems);
+ ref->p_req_valid = true;
}
/* Copy the current value to the request value */
void cur_to_req(struct v4l2_ctrl_ref *ref)
{
+ struct v4l2_ctrl *ctrl;
+
if (!ref)
return;
- ptr_to_ptr(ref->ctrl, ref->ctrl->p_cur, ref->p_req);
- ref->valid_p_req = true;
+
+ ctrl = ref->ctrl;
+ if (ctrl->is_dyn_array && !req_alloc_dyn_array(ref, ctrl->elems))
+ return;
+
+ ref->p_req_elems = ctrl->elems;
+ ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req, ctrl->elems);
+ ref->p_req_valid = true;
}
/* Copy the request value to the new value */
-void req_to_new(struct v4l2_ctrl_ref *ref)
+int req_to_new(struct v4l2_ctrl_ref *ref)
{
+ struct v4l2_ctrl *ctrl;
+
if (!ref)
- return;
- if (ref->valid_p_req)
- ptr_to_ptr(ref->ctrl, ref->p_req, ref->ctrl->p_new);
- else
- ptr_to_ptr(ref->ctrl, ref->ctrl->p_cur, ref->ctrl->p_new);
+ return 0;
+
+ ctrl = ref->ctrl;
+
+ /*
+ * This control was never set in the request, so just use the current
+ * value.
+ */
+ if (!ref->p_req_valid) {
+ if (ctrl->is_dyn_array)
+ ctrl->new_elems = ctrl->elems;
+ ptr_to_ptr(ctrl, ctrl->p_cur, ctrl->p_new, ctrl->new_elems);
+ return 0;
+ }
+
+ /* Not a dynamic array, so just copy the request value */
+ if (!ctrl->is_dyn_array) {
+ ptr_to_ptr(ctrl, ref->p_req, ctrl->p_new, ctrl->new_elems);
+ return 0;
+ }
+
+ /* Sanity check, should never happen */
+ if (WARN_ON(!ref->p_req_dyn_alloc_elems))
+ return -ENOMEM;
+
+ /*
+ * Check if the number of elements in the request is more than the
+ * elements in ctrl->p_dyn. If so, attempt to realloc ctrl->p_dyn.
+ * Note that p_dyn is allocated with twice the number of elements
+ * in the dynamic array since it has to store both the current and
+ * new value of such a control.
+ */
+ if (ref->p_req_elems > ctrl->p_dyn_alloc_elems) {
+ unsigned int sz = ref->p_req_elems * ctrl->elem_size;
+ void *old = ctrl->p_dyn;
+ void *tmp = kvzalloc(2 * sz, GFP_KERNEL);
+
+ if (!tmp)
+ return -ENOMEM;
+ memcpy(tmp, ctrl->p_new.p, ctrl->elems * ctrl->elem_size);
+ memcpy(tmp + sz, ctrl->p_cur.p, ctrl->elems * ctrl->elem_size);
+ ctrl->p_new.p = tmp;
+ ctrl->p_cur.p = tmp + sz;
+ ctrl->p_dyn = tmp;
+ ctrl->p_dyn_alloc_elems = ref->p_req_elems;
+ kvfree(old);
+ }
+
+ ctrl->new_elems = ref->p_req_elems;
+ ptr_to_ptr(ctrl, ref->p_req, ctrl->p_new, ctrl->new_elems);
+ return 0;
}
/* Control range checking */
@@ -1092,17 +1182,6 @@ int check_range(enum v4l2_ctrl_type type,
}
}
-/* Validate a new control */
-int validate_new(const struct v4l2_ctrl *ctrl, union v4l2_ctrl_ptr p_new)
-{
- unsigned idx;
- int err = 0;
-
- for (idx = 0; !err && idx < ctrl->elems; idx++)
- err = ctrl->type_ops->validate(ctrl, idx, p_new);
- return err;
-}
-
/* Set the handler's error code if it wasn't set earlier already */
static inline int handler_set_err(struct v4l2_ctrl_handler *hdl, int err)
{
@@ -1147,6 +1226,8 @@ void v4l2_ctrl_handler_free(struct v4l2_ctrl_handler *hdl)
/* Free all nodes */
list_for_each_entry_safe(ref, next_ref, &hdl->ctrl_refs, node) {
list_del(&ref->node);
+ if (ref->p_req_dyn_alloc_elems)
+ kvfree(ref->p_req.p);
kfree(ref);
}
/* Free all controls owned by the handler */
@@ -1154,6 +1235,7 @@ void v4l2_ctrl_handler_free(struct v4l2_ctrl_handler *hdl)
list_del(&ctrl->node);
list_for_each_entry_safe(sev, next_sev, &ctrl->ev_subs, node)
list_del(&sev->node);
+ kvfree(ctrl->p_dyn);
kvfree(ctrl);
}
kvfree(hdl->buckets);
@@ -1269,7 +1351,7 @@ int handler_new_ref(struct v4l2_ctrl_handler *hdl,
if (hdl->error)
return hdl->error;
- if (allocate_req)
+ if (allocate_req && !ctrl->is_dyn_array)
size_extra_req = ctrl->elems * ctrl->elem_size;
new_ref = kzalloc(sizeof(*new_ref) + size_extra_req, GFP_KERNEL);
if (!new_ref)
@@ -1443,7 +1525,6 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
elem_size = sizeof(s32);
break;
}
- tot_ctrl_size = elem_size * elems;
/* Sanity checks */
if (id == 0 || name == NULL || !elem_size ||
@@ -1464,17 +1545,33 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
handler_set_err(hdl, -EINVAL);
return NULL;
}
+ if (flags & V4L2_CTRL_FLAG_DYNAMIC_ARRAY) {
+ /*
+ * For now only support this for one-dimensional arrays only.
+ *
+ * This can be relaxed in the future, but this will
+ * require more effort.
+ */
+ if (nr_of_dims != 1) {
+ handler_set_err(hdl, -EINVAL);
+ return NULL;
+ }
+ /* Start with just 1 element */
+ elems = 1;
+ }
+ tot_ctrl_size = elem_size * elems;
sz_extra = 0;
if (type == V4L2_CTRL_TYPE_BUTTON)
flags |= V4L2_CTRL_FLAG_WRITE_ONLY |
V4L2_CTRL_FLAG_EXECUTE_ON_WRITE;
else if (type == V4L2_CTRL_TYPE_CTRL_CLASS)
flags |= V4L2_CTRL_FLAG_READ_ONLY;
- else if (type == V4L2_CTRL_TYPE_INTEGER64 ||
- type == V4L2_CTRL_TYPE_STRING ||
- type >= V4L2_CTRL_COMPOUND_TYPES ||
- is_array)
+ else if (!(flags & V4L2_CTRL_FLAG_DYNAMIC_ARRAY) &&
+ (type == V4L2_CTRL_TYPE_INTEGER64 ||
+ type == V4L2_CTRL_TYPE_STRING ||
+ type >= V4L2_CTRL_COMPOUND_TYPES ||
+ is_array))
sz_extra += 2 * tot_ctrl_size;
if (type >= V4L2_CTRL_COMPOUND_TYPES && p_def.p_const)
@@ -1503,7 +1600,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
ctrl->is_ptr = is_array || type >= V4L2_CTRL_COMPOUND_TYPES || ctrl->is_string;
ctrl->is_int = !ctrl->is_ptr && type != V4L2_CTRL_TYPE_INTEGER64;
ctrl->is_array = is_array;
+ ctrl->is_dyn_array = !!(flags & V4L2_CTRL_FLAG_DYNAMIC_ARRAY);
ctrl->elems = elems;
+ ctrl->new_elems = elems;
ctrl->nr_of_dims = nr_of_dims;
if (nr_of_dims)
memcpy(ctrl->dims, dims, nr_of_dims * sizeof(dims[0]));
@@ -1516,6 +1615,16 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
ctrl->cur.val = ctrl->val = def;
data = &ctrl[1];
+ if (ctrl->is_dyn_array) {
+ ctrl->p_dyn_alloc_elems = elems;
+ ctrl->p_dyn = kvzalloc(2 * elems * elem_size, GFP_KERNEL);
+ if (!ctrl->p_dyn) {
+ kvfree(ctrl);
+ return NULL;
+ }
+ data = ctrl->p_dyn;
+ }
+
if (!ctrl->is_int) {
ctrl->p_new.p = data;
ctrl->p_cur.p = data + tot_ctrl_size;
@@ -1525,7 +1634,10 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
}
if (type >= V4L2_CTRL_COMPOUND_TYPES && p_def.p_const) {
- ctrl->p_def.p = ctrl->p_cur.p + tot_ctrl_size;
+ if (ctrl->is_dyn_array)
+ ctrl->p_def.p = &ctrl[1];
+ else
+ ctrl->p_def.p = ctrl->p_cur.p + tot_ctrl_size;
memcpy(ctrl->p_def.p, p_def.p_const, elem_size);
}
@@ -1535,6 +1647,7 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
}
if (handler_new_ref(hdl, ctrl, NULL, false, false)) {
+ kvfree(ctrl->p_dyn);
kvfree(ctrl);
return NULL;
}
@@ -1872,6 +1985,9 @@ static int cluster_changed(struct v4l2_ctrl *master)
continue;
}
+ if (ctrl->elems != ctrl->new_elems)
+ ctrl_changed = true;
+
for (idx = 0; !ctrl_changed && idx < ctrl->elems; idx++)
ctrl_changed = !ctrl->type_ops->equal(ctrl, idx,
ctrl->p_cur, ctrl->p_new);
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-priv.h b/drivers/media/v4l2-core/v4l2-ctrls-priv.h
index d4bf2c716f97..aba6176fab6c 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-priv.h
+++ b/drivers/media/v4l2-core/v4l2-ctrls-priv.h
@@ -57,10 +57,9 @@ void cur_to_new(struct v4l2_ctrl *ctrl);
void cur_to_req(struct v4l2_ctrl_ref *ref);
void new_to_cur(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl, u32 ch_flags);
void new_to_req(struct v4l2_ctrl_ref *ref);
-void req_to_new(struct v4l2_ctrl_ref *ref);
+int req_to_new(struct v4l2_ctrl_ref *ref);
void send_initial_event(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl);
void send_event(struct v4l2_fh *fh, struct v4l2_ctrl *ctrl, u32 changes);
-int validate_new(const struct v4l2_ctrl *ctrl, union v4l2_ctrl_ptr p_new);
int handler_new_ref(struct v4l2_ctrl_handler *hdl,
struct v4l2_ctrl *ctrl,
struct v4l2_ctrl_ref **ctrl_ref,
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-request.c b/drivers/media/v4l2-core/v4l2-ctrls-request.c
index 7d098f287fd9..c637049d7a2b 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-request.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-request.c
@@ -143,7 +143,7 @@ v4l2_ctrl_request_hdl_ctrl_find(struct v4l2_ctrl_handler *hdl, u32 id)
{
struct v4l2_ctrl_ref *ref = find_ref_lock(hdl, id);
- return (ref && ref->valid_p_req) ? ref->ctrl : NULL;
+ return (ref && ref->p_req_valid) ? ref->ctrl : NULL;
}
EXPORT_SYMBOL_GPL(v4l2_ctrl_request_hdl_ctrl_find);
@@ -373,7 +373,7 @@ void v4l2_ctrl_request_complete(struct media_request *req,
v4l2_ctrl_unlock(master);
continue;
}
- if (ref->valid_p_req)
+ if (ref->p_req_valid)
continue;
/* Copy the current control value into the request */
@@ -442,7 +442,7 @@ int v4l2_ctrl_request_setup(struct media_request *req,
struct v4l2_ctrl_ref *r =
find_ref(hdl, master->cluster[i]->id);
- if (r->valid_p_req) {
+ if (r->p_req_valid) {
have_new_data = true;
break;
}
@@ -458,7 +458,11 @@ int v4l2_ctrl_request_setup(struct media_request *req,
struct v4l2_ctrl_ref *r =
find_ref(hdl, master->cluster[i]->id);
- req_to_new(r);
+ ret = req_to_new(r);
+ if (ret) {
+ v4l2_ctrl_unlock(master);
+ goto error;
+ }
master->cluster[i]->is_new = 1;
r->req_done = true;
}
@@ -490,6 +494,7 @@ int v4l2_ctrl_request_setup(struct media_request *req,
break;
}
+error:
media_request_object_put(obj);
return ret;
}
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index b3ce438f1329..f4105de8a8d2 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -185,6 +185,8 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv);
* and/or has type %V4L2_CTRL_TYPE_STRING. In other words, &struct
* v4l2_ext_control uses field p to point to the data.
* @is_array: If set, then this control contains an N-dimensional array.
+ * @is_dyn_array: If set, then this control contains a dynamically sized 1-dimensional array.
+ * If this is set, then @is_array is also set.
* @has_volatiles: If set, then one or more members of the cluster are volatile.
* Drivers should never touch this flag.
* @call_notify: If set, then call the handler's notify function whenever the
@@ -205,6 +207,9 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv);
* @step: The control's step value for non-menu controls.
* @elems: The number of elements in the N-dimensional array.
* @elem_size: The size in bytes of the control.
+ * @new_elems: The number of elements in p_new. This is the same as @elems,
+ * except for dynamic arrays. In that case it is in the range of
+ * 1 to @p_dyn_alloc_elems.
* @dims: The size of each dimension.
* @nr_of_dims:The number of dimensions in @dims.
* @menu_skip_mask: The control's skip mask for menu controls. This makes it
@@ -223,15 +228,21 @@ typedef void (*v4l2_ctrl_notify_fnc)(struct v4l2_ctrl *ctrl, void *priv);
* :math:`ceil(\frac{maximum - minimum}{step}) + 1`.
* Used only if the @type is %V4L2_CTRL_TYPE_INTEGER_MENU.
* @flags: The control's flags.
- * @cur: Structure to store the current value.
- * @cur.val: The control's current value, if the @type is represented via
- * a u32 integer (see &enum v4l2_ctrl_type).
- * @val: The control's new s32 value.
* @priv: The control's private pointer. For use by the driver. It is
* untouched by the control framework. Note that this pointer is
* not freed when the control is deleted. Should this be needed
* then a new internal bitfield can be added to tell the framework
* to free this pointer.
+ * @p_dyn: Pointer to the dynamically allocated array. Only valid if
+ * @is_dyn_array is true.
+ * @p_dyn_alloc_elems: The number of elements in the dynamically allocated
+ * array for both the cur and new values. So @p_dyn is actually
+ * sized for 2 * @p_dyn_alloc_elems * @elem_size. Only valid if
+ * @is_dyn_array is true.
+ * @cur: Structure to store the current value.
+ * @cur.val: The control's current value, if the @type is represented via
+ * a u32 integer (see &enum v4l2_ctrl_type).
+ * @val: The control's new s32 value.
* @p_def: The control's default value represented via a union which
* provides a standard way of accessing control types
* through a pointer (for compound controls only).
@@ -260,6 +271,7 @@ struct v4l2_ctrl {
unsigned int is_string:1;
unsigned int is_ptr:1;
unsigned int is_array:1;
+ unsigned int is_dyn_array:1;
unsigned int has_volatiles:1;
unsigned int call_notify:1;
unsigned int manual_mode_value:8;
@@ -272,6 +284,7 @@ struct v4l2_ctrl {
s64 minimum, maximum, default_value;
u32 elems;
u32 elem_size;
+ u32 new_elems;
u32 dims[V4L2_CTRL_MAX_DIMS];
u32 nr_of_dims;
union {
@@ -284,6 +297,8 @@ struct v4l2_ctrl {
};
unsigned long flags;
void *priv;
+ void *p_dyn;
+ u32 p_dyn_alloc_elems;
s32 val;
struct {
s32 val;
@@ -309,12 +324,22 @@ struct v4l2_ctrl {
* the control has been applied. This prevents applying controls
* from a cluster with multiple controls twice (when the first
* control of a cluster is applied, they all are).
- * @valid_p_req: If set, then p_req contains the control value for the request.
+ * @p_req_valid: If set, then p_req contains the control value for the request.
+ * @p_req_dyn_enomem: If set, then p_req is invalid since allocating space for
+ * a dynamic array failed. Attempting to read this value shall
+ * result in ENOMEM. Only valid if ctrl->is_dyn_array is true.
+ * @p_req_dyn_alloc_elems: The number of elements allocated for the dynamic
+ * array. Only valid if @p_req_valid and ctrl->is_dyn_array are
+ * true.
+ * @p_req_elems: The number of elements in @p_req. This is the same as
+ * ctrl->elems, except for dynamic arrays. In that case it is in
+ * the range of 1 to @p_req_dyn_alloc_elems. Only valid if
+ * @p_req_valid is true.
* @p_req: If the control handler containing this control reference
* is bound to a media request, then this points to the
* value of the control that must be applied when the request
* is executed, or to the value of the control at the time
- * that the request was completed. If @valid_p_req is false,
+ * that the request was completed. If @p_req_valid is false,
* then this control was never set for this request and the
* control will not be updated when this request is applied.
*
@@ -329,7 +354,10 @@ struct v4l2_ctrl_ref {
struct v4l2_ctrl_helper *helper;
bool from_other_dev;
bool req_done;
- bool valid_p_req;
+ bool p_req_valid;
+ bool p_req_dyn_enomem;
+ u32 p_req_dyn_alloc_elems;
+ u32 p_req_elems;
union v4l2_ctrl_ptr p_req;
};
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [RFC v2 3/8] vivid: add dynamic array test control
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 1/8] videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 2/8] v4l2-ctrls: add support for dynamically allocated arrays Benjamin Gaignard
@ 2022-02-15 11:00 ` Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging Benjamin Gaignard
` (7 subsequent siblings)
10 siblings, 0 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:00 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
From: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Add a dynamic array test control to help test support for this
feature.
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
---
drivers/media/test-drivers/vivid/vivid-ctrls.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-ctrls.c b/drivers/media/test-drivers/vivid/vivid-ctrls.c
index e7516dc1227b..7267892dc18a 100644
--- a/drivers/media/test-drivers/vivid/vivid-ctrls.c
+++ b/drivers/media/test-drivers/vivid/vivid-ctrls.c
@@ -34,6 +34,7 @@
#define VIVID_CID_U8_4D_ARRAY (VIVID_CID_CUSTOM_BASE + 10)
#define VIVID_CID_AREA (VIVID_CID_CUSTOM_BASE + 11)
#define VIVID_CID_RO_INTEGER (VIVID_CID_CUSTOM_BASE + 12)
+#define VIVID_CID_U32_DYN_ARRAY (VIVID_CID_CUSTOM_BASE + 13)
#define VIVID_CID_VIVID_BASE (0x00f00000 | 0xf000)
#define VIVID_CID_VIVID_CLASS (0x00f00000 | 1)
@@ -189,6 +190,19 @@ static const struct v4l2_ctrl_config vivid_ctrl_u32_array = {
.dims = { 1 },
};
+static const struct v4l2_ctrl_config vivid_ctrl_u32_dyn_array = {
+ .ops = &vivid_user_gen_ctrl_ops,
+ .id = VIVID_CID_U32_DYN_ARRAY,
+ .name = "U32 Dynamic Array",
+ .type = V4L2_CTRL_TYPE_U32,
+ .flags = V4L2_CTRL_FLAG_DYNAMIC_ARRAY,
+ .def = 50,
+ .min = 10,
+ .max = 90,
+ .step = 1,
+ .dims = { 100 },
+};
+
static const struct v4l2_ctrl_config vivid_ctrl_u16_matrix = {
.ops = &vivid_user_gen_ctrl_ops,
.id = VIVID_CID_U16_MATRIX,
@@ -1612,6 +1626,7 @@ int vivid_create_controls(struct vivid_dev *dev, bool show_ccs_cap,
dev->ro_int32 = v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_ro_int32, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_area, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u32_array, NULL);
+ v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u32_dyn_array, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u16_matrix, NULL);
v4l2_ctrl_new_custom(hdl_user_gen, &vivid_ctrl_u8_4d_array, NULL);
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (2 preceding siblings ...)
2022-02-15 11:00 ` [RFC v2 3/8] vivid: add dynamic array test control Benjamin Gaignard
@ 2022-02-15 11:00 ` Benjamin Gaignard
2022-02-15 18:02 ` Sebastian Fricke
2022-02-18 11:50 ` Hans Verkuil
2022-02-15 11:01 ` [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver Benjamin Gaignard
` (6 subsequent siblings)
10 siblings, 2 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:00 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc, Benjamin Gaignard
The HEVC stateless 'uAPI' was staging and marked explicitly in the
V4L2 specification that it will change and is unstable.
Note that these control IDs were never exported as a public API,
they were only defined in kernel-local headers (hevc-ctrls.h).
While moving the controls out of staging they are renamed and
control IDs get new numbers.
Drivers (Hantro, Cedrus) and Documentation are updated accordingly.
Hantro dedicated control is moving to hantro-media.h
Since hevc-ctrls.h content has been dispatched in others file, remove it.
fluster tests results on IMX8MQ is 77/147 for HEVC codec.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2 :
- Do not include rkvdec needed fields in this patch
- Remove 'non-public' notice in hantro.rst
- Take care of cache-line boundary
.../userspace-api/media/drivers/hantro.rst | 5 -
.../media/v4l/ext-ctrls-codec.rst | 26 +-
drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
drivers/staging/media/hantro/hantro_drv.c | 27 +-
drivers/staging/media/hantro/hantro_hevc.c | 8 +-
drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
.../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
include/linux/hantro-media.h | 17 ++
include/media/hevc-ctrls.h | 250 ------------------
include/media/v4l2-ctrls.h | 6 -
include/uapi/linux/v4l2-controls.h | 219 +++++++++++++++
include/uapi/linux/videodev2.h | 7 +
12 files changed, 307 insertions(+), 324 deletions(-)
create mode 100644 include/linux/hantro-media.h
delete mode 100644 include/media/hevc-ctrls.h
diff --git a/Documentation/userspace-api/media/drivers/hantro.rst b/Documentation/userspace-api/media/drivers/hantro.rst
index cd9754b4e005..78dcd2a44a03 100644
--- a/Documentation/userspace-api/media/drivers/hantro.rst
+++ b/Documentation/userspace-api/media/drivers/hantro.rst
@@ -12,8 +12,3 @@ The Hantro video decoder driver implements the following driver-specific control
to before syntax element "slice_temporal_mvp_enabled_flag".
If IDR, the skipped bits are just "pic_output_flag"
(separate_colour_plane_flag is not supported).
-
-.. note::
-
- This control is not yet part of the public kernel API and
- it is expected to change.
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 4cd7c541fc30..4f3b3ba8319f 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2639,7 +2639,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
.. _v4l2-mpeg-hevc:
-``V4L2_CID_MPEG_VIDEO_HEVC_SPS (struct)``
+``V4L2_CID_STATELESS_HEVC_SPS (struct)``
Specifies the Sequence Parameter Set fields (as extracted from the
bitstream) for the associated HEVC slice data.
These bitstream parameters are defined according to :ref:`hevc`.
@@ -2782,7 +2782,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_PPS (struct)``
+``V4L2_CID_STATELESS_HEVC_PPS (struct)``
Specifies the Picture Parameter Set fields (as extracted from the
bitstream) for the associated HEVC slice data.
These bitstream parameters are defined according to :ref:`hevc`.
@@ -2932,7 +2932,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
+``V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (struct)``
Specifies various slice-specific parameters, especially from the NAL unit
header, general slice segment header and weighted prediction parameter
parts of the bitstream.
@@ -3088,7 +3088,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
+``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
Specifies the HEVC scaling matrix parameters used for the scaling process
for transform coefficients.
These matrix and parameters are defined according to :ref:`hevc`.
@@ -3238,7 +3238,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (enum)``
+``V4L2_CID_STATELESS_HEVC_DECODE_MODE (enum)``
Specifies the decoding mode to use. Currently exposes slice-based and
frame-based decoding but new modes might be added later on.
This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
@@ -3253,7 +3253,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
This menu control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_mpeg_video_hevc_decode_mode
+.. c:type:: v4l2_stateless_hevc_decode_mode
.. raw:: latex
@@ -3266,11 +3266,11 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
- * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED``
+ * - ``V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED``
- 0
- Decoding is done at the slice granularity.
The OUTPUT buffer must contain a single slice.
- * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED``
+ * - ``V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED``
- 1
- Decoding is done at the frame granularity.
The OUTPUT buffer must contain all slices needed to decode the
@@ -3280,7 +3280,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
-``V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (enum)``
+``V4L2_CID_STATELESS_HEVC_START_CODE (enum)``
Specifies the HEVC slice start code expected for each slice.
This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
pixel format. Applications that support V4L2_PIX_FMT_HEVC_SLICE
@@ -3294,7 +3294,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
This menu control is not yet part of the public kernel API and
it is expected to change.
-.. c:type:: v4l2_mpeg_video_hevc_start_code
+.. c:type:: v4l2_stateless_hevc_start_code
.. tabularcolumns:: |p{9.2cm}|p{0.6cm}|p{7.5cm}|
@@ -3303,13 +3303,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
- * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE``
+ * - ``V4L2_STATELESS_HEVC_START_CODE_NONE``
- 0
- Selecting this value specifies that HEVC slices are passed
to the driver without any start code. The bitstream data should be
according to :ref:`hevc` 7.3.1.1 General NAL unit syntax, hence
contains emulation prevention bytes when required.
- * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B``
+ * - ``V4L2_STATELESS_HEVC_START_CODE_ANNEX_B``
- 1
- Selecting this value specifies that HEVC slices are expected
to be prefixed by Annex B start codes. According to :ref:`hevc`
@@ -3342,7 +3342,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
This provides a bitmask which consists of bits [0, LTR_COUNT-1].
This is applicable to the H264 and HEVC encoders.
-``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
+``V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (struct)``
Specifies various decode parameters, especially the references picture order
count (POC) for all the lists (short, long, before, current, after) and the
number of entries for each of them.
diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
index 54ca4e6b820b..4b68cbe23309 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
@@ -699,9 +699,9 @@ const char * const *v4l2_ctrl_get_menu(u32 id)
return hevc_tier;
case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
return hevc_loop_filter_mode;
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
return hevc_decode_mode;
- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
+ case V4L2_CID_STATELESS_HEVC_START_CODE:
return hevc_start_code;
case V4L2_CID_CAMERA_ORIENTATION:
return camera_orientation;
@@ -995,13 +995,6 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD: return "HEVC Size of Length Field";
case V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES: return "Reference Frames for a P-Frame";
case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR: return "Prepend SPS and PPS to IDR";
- case V4L2_CID_MPEG_VIDEO_HEVC_SPS: return "HEVC Sequence Parameter Set";
- case V4L2_CID_MPEG_VIDEO_HEVC_PPS: return "HEVC Picture Parameter Set";
- case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
- case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC Decode Mode";
- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC Start Code";
/* CAMERA controls */
/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1180,6 +1173,13 @@ const char *v4l2_ctrl_get_name(u32 id)
case V4L2_CID_STATELESS_MPEG2_QUANTISATION: return "MPEG-2 Quantisation Matrices";
case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR: return "VP9 Probabilities Updates";
case V4L2_CID_STATELESS_VP9_FRAME: return "VP9 Frame Decode Parameters";
+ case V4L2_CID_STATELESS_HEVC_SPS: return "HEVC Sequence Parameter Set";
+ case V4L2_CID_STATELESS_HEVC_PPS: return "HEVC Picture Parameter Set";
+ case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
+ case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
+ case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE: return "HEVC Decode Mode";
+ case V4L2_CID_STATELESS_HEVC_START_CODE: return "HEVC Start Code";
/* Colorimetry controls */
/* Keep the order of the 'case's the same as in v4l2-controls.h! */
@@ -1355,8 +1355,8 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD:
case V4L2_CID_MPEG_VIDEO_HEVC_TIER:
case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
+ case V4L2_CID_STATELESS_HEVC_START_CODE:
case V4L2_CID_STATELESS_H264_DECODE_MODE:
case V4L2_CID_STATELESS_H264_START_CODE:
case V4L2_CID_CAMERA_ORIENTATION:
@@ -1493,19 +1493,19 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
case V4L2_CID_STATELESS_VP8_FRAME:
*type = V4L2_CTRL_TYPE_VP8_FRAME;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
+ case V4L2_CID_STATELESS_HEVC_SPS:
*type = V4L2_CTRL_TYPE_HEVC_SPS;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
+ case V4L2_CID_STATELESS_HEVC_PPS:
*type = V4L2_CTRL_TYPE_HEVC_PPS;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
+ case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
+ case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX:
*type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
break;
- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
+ case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS:
*type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
break;
case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
index dc768884cb79..783a92a38c8e 100644
--- a/drivers/staging/media/hantro/hantro_drv.c
+++ b/drivers/staging/media/hantro/hantro_drv.c
@@ -11,6 +11,7 @@
*/
#include <linux/clk.h>
+#include <linux/hantro-media.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/platform_device.h>
@@ -255,7 +256,7 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
if (sps->bit_depth_luma_minus8 != 0)
/* Only 8-bit is supported */
return -EINVAL;
- } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
+ } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
@@ -428,18 +429,18 @@ static const struct hantro_ctrl controls[] = {
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
- .min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
- .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
- .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
+ .min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+ .max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+ .def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
- .min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
- .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
- .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
+ .id = V4L2_CID_STATELESS_HEVC_START_CODE,
+ .min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+ .max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+ .def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
},
}, {
.codec = HANTRO_HEVC_DECODER,
@@ -459,23 +460,23 @@ static const struct hantro_ctrl controls[] = {
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
+ .id = V4L2_CID_STATELESS_HEVC_SPS,
.ops = &hantro_ctrl_ops,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
+ .id = V4L2_CID_STATELESS_HEVC_PPS,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
},
}, {
.codec = HANTRO_HEVC_DECODER,
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
+ .id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
},
}, {
.codec = HANTRO_HEVC_DECODER,
diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
index b49a41d7ae91..b6ec86d03d91 100644
--- a/drivers/staging/media/hantro/hantro_hevc.c
+++ b/drivers/staging/media/hantro/hantro_hevc.c
@@ -201,22 +201,22 @@ int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx)
hantro_start_prepare_run(ctx);
ctrls->decode_params =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
if (WARN_ON(!ctrls->decode_params))
return -EINVAL;
ctrls->scaling =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
if (WARN_ON(!ctrls->scaling))
return -EINVAL;
ctrls->sps =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SPS);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_SPS);
if (WARN_ON(!ctrls->sps))
return -EINVAL;
ctrls->pps =
- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_PPS);
+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_PPS);
if (WARN_ON(!ctrls->pps))
return -EINVAL;
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
index 4a4b714b0f26..e0428163f82c 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
@@ -42,7 +42,7 @@ static int cedrus_try_ctrl(struct v4l2_ctrl *ctrl)
if (sps->bit_depth_luma_minus8 != 0)
/* Only 8-bit is supported */
return -EINVAL;
- } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
+ } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
struct cedrus_ctx *ctx = container_of(ctrl->handler, struct cedrus_ctx, hdl);
@@ -164,42 +164,42 @@ static const struct cedrus_control cedrus_controls[] = {
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
+ .id = V4L2_CID_STATELESS_HEVC_SPS,
.ops = &cedrus_ctrl_ops,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
+ .id = V4L2_CID_STATELESS_HEVC_PPS,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS,
+ .id = V4L2_CID_STATELESS_HEVC_SLICE_PARAMS,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
+ .id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
- .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
- .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
+ .max = V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
+ .def = V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
},
.codec = CEDRUS_CODEC_H265,
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
- .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
- .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
+ .id = V4L2_CID_STATELESS_HEVC_START_CODE,
+ .max = V4L2_STATELESS_HEVC_START_CODE_NONE,
+ .def = V4L2_STATELESS_HEVC_START_CODE_NONE,
},
.codec = CEDRUS_CODEC_H265,
},
@@ -211,7 +211,7 @@ static const struct cedrus_control cedrus_controls[] = {
},
{
.cfg = {
- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
+ .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
},
.codec = CEDRUS_CODEC_H265,
},
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
index a16c1422558f..f6be4ae72ee2 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
@@ -65,15 +65,15 @@ void cedrus_device_run(void *priv)
case V4L2_PIX_FMT_HEVC_SLICE:
run.h265.sps = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_SPS);
+ V4L2_CID_STATELESS_HEVC_SPS);
run.h265.pps = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_PPS);
+ V4L2_CID_STATELESS_HEVC_PPS);
run.h265.slice_params = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS);
+ V4L2_CID_STATELESS_HEVC_SLICE_PARAMS);
run.h265.decode_params = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
+ V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
run.h265.scaling_matrix = cedrus_find_control_data(ctx,
- V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
+ V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
break;
case V4L2_PIX_FMT_VP8_FRAME:
diff --git a/include/linux/hantro-media.h b/include/linux/hantro-media.h
new file mode 100644
index 000000000000..db2791ab7c3b
--- /dev/null
+++ b/include/linux/hantro-media.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef __LINUX_HANTRO_MEDIA_H__
+#define __LINUX_HANTRO_MEDIA_H__
+
+/*
+ * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
+ * the number of data (in bits) to skip in the
+ * slice segment header.
+ * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
+ * to before syntax element "slice_temporal_mvp_enabled_flag".
+ * If IDR, the skipped bits are just "pic_output_flag"
+ * (separate_colour_plane_flag is not supported).
+ */
+#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_USER_HANTRO_BASE + 0)
+
+#endif
diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
deleted file mode 100644
index 01ccda48d8c5..000000000000
--- a/include/media/hevc-ctrls.h
+++ /dev/null
@@ -1,250 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-/*
- * These are the HEVC state controls for use with stateless HEVC
- * codec drivers.
- *
- * It turns out that these structs are not stable yet and will undergo
- * more changes. So keep them private until they are stable and ready to
- * become part of the official public API.
- */
-
-#ifndef _HEVC_CTRLS_H_
-#define _HEVC_CTRLS_H_
-
-#include <linux/videodev2.h>
-
-/* The pixel format isn't stable at the moment and will likely be renamed. */
-#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
-
-#define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008)
-#define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009)
-#define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010)
-#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE + 1011)
-#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012)
-#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015)
-#define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
-
-/* enum v4l2_ctrl_type type values */
-#define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
-#define V4L2_CTRL_TYPE_HEVC_PPS 0x0121
-#define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122
-#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
-#define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
-
-enum v4l2_mpeg_video_hevc_decode_mode {
- V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
- V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
-};
-
-enum v4l2_mpeg_video_hevc_start_code {
- V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
- V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
-};
-
-#define V4L2_HEVC_SLICE_TYPE_B 0
-#define V4L2_HEVC_SLICE_TYPE_P 1
-#define V4L2_HEVC_SLICE_TYPE_I 2
-
-#define V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE (1ULL << 0)
-#define V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED (1ULL << 1)
-#define V4L2_HEVC_SPS_FLAG_AMP_ENABLED (1ULL << 2)
-#define V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET (1ULL << 3)
-#define V4L2_HEVC_SPS_FLAG_PCM_ENABLED (1ULL << 4)
-#define V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED (1ULL << 5)
-#define V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT (1ULL << 6)
-#define V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED (1ULL << 7)
-#define V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED (1ULL << 8)
-
-/* The controls are not stable at the moment and will likely be reworked. */
-struct v4l2_ctrl_hevc_sps {
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
- __u16 pic_width_in_luma_samples;
- __u16 pic_height_in_luma_samples;
- __u8 bit_depth_luma_minus8;
- __u8 bit_depth_chroma_minus8;
- __u8 log2_max_pic_order_cnt_lsb_minus4;
- __u8 sps_max_dec_pic_buffering_minus1;
- __u8 sps_max_num_reorder_pics;
- __u8 sps_max_latency_increase_plus1;
- __u8 log2_min_luma_coding_block_size_minus3;
- __u8 log2_diff_max_min_luma_coding_block_size;
- __u8 log2_min_luma_transform_block_size_minus2;
- __u8 log2_diff_max_min_luma_transform_block_size;
- __u8 max_transform_hierarchy_depth_inter;
- __u8 max_transform_hierarchy_depth_intra;
- __u8 pcm_sample_bit_depth_luma_minus1;
- __u8 pcm_sample_bit_depth_chroma_minus1;
- __u8 log2_min_pcm_luma_coding_block_size_minus3;
- __u8 log2_diff_max_min_pcm_luma_coding_block_size;
- __u8 num_short_term_ref_pic_sets;
- __u8 num_long_term_ref_pics_sps;
- __u8 chroma_format_idc;
- __u8 sps_max_sub_layers_minus1;
-
- __u64 flags;
-};
-
-#define V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED (1ULL << 0)
-#define V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT (1ULL << 1)
-#define V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED (1ULL << 2)
-#define V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT (1ULL << 3)
-#define V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED (1ULL << 4)
-#define V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED (1ULL << 5)
-#define V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED (1ULL << 6)
-#define V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT (1ULL << 7)
-#define V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED (1ULL << 8)
-#define V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED (1ULL << 9)
-#define V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED (1ULL << 10)
-#define V4L2_HEVC_PPS_FLAG_TILES_ENABLED (1ULL << 11)
-#define V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED (1ULL << 12)
-#define V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED (1ULL << 13)
-#define V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 14)
-#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED (1ULL << 15)
-#define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER (1ULL << 16)
-#define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT (1ULL << 17)
-#define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL << 18)
-#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
-#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
-
-struct v4l2_ctrl_hevc_pps {
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
- __u8 num_extra_slice_header_bits;
- __u8 num_ref_idx_l0_default_active_minus1;
- __u8 num_ref_idx_l1_default_active_minus1;
- __s8 init_qp_minus26;
- __u8 diff_cu_qp_delta_depth;
- __s8 pps_cb_qp_offset;
- __s8 pps_cr_qp_offset;
- __u8 num_tile_columns_minus1;
- __u8 num_tile_rows_minus1;
- __u8 column_width_minus1[20];
- __u8 row_height_minus1[22];
- __s8 pps_beta_offset_div2;
- __s8 pps_tc_offset_div2;
- __u8 log2_parallel_merge_level_minus2;
-
- __u8 padding[4];
- __u64 flags;
-};
-
-#define V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE 0x01
-
-#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX 16
-
-struct v4l2_hevc_dpb_entry {
- __u64 timestamp;
- __u8 flags;
- __u8 field_pic;
- __u16 pic_order_cnt[2];
- __u8 padding[2];
-};
-
-struct v4l2_hevc_pred_weight_table {
- __s8 delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __s8 luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __s8 delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
- __s8 chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
-
- __s8 delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __s8 luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __s8 delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
- __s8 chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
-
- __u8 padding[6];
-
- __u8 luma_log2_weight_denom;
- __s8 delta_chroma_log2_weight_denom;
-};
-
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_LUMA (1ULL << 0)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_CHROMA (1ULL << 1)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED (1ULL << 2)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_MVD_L1_ZERO (1ULL << 3)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_CABAC_INIT (1ULL << 4)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_COLLOCATED_FROM_L0 (1ULL << 5)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_USE_INTEGER_MV (1ULL << 6)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED (1ULL << 7)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 8)
-#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
-
-struct v4l2_ctrl_hevc_slice_params {
- __u32 bit_size;
- __u32 data_bit_offset;
-
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
- __u8 nal_unit_type;
- __u8 nuh_temporal_id_plus1;
-
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
- __u8 slice_type;
- __u8 colour_plane_id;
- __u16 slice_pic_order_cnt;
- __u8 num_ref_idx_l0_active_minus1;
- __u8 num_ref_idx_l1_active_minus1;
- __u8 collocated_ref_idx;
- __u8 five_minus_max_num_merge_cand;
- __s8 slice_qp_delta;
- __s8 slice_cb_qp_offset;
- __s8 slice_cr_qp_offset;
- __s8 slice_act_y_qp_offset;
- __s8 slice_act_cb_qp_offset;
- __s8 slice_act_cr_qp_offset;
- __s8 slice_beta_offset_div2;
- __s8 slice_tc_offset_div2;
-
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
- __u8 pic_struct;
-
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
- __u32 slice_segment_addr;
- __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
-
- __u8 padding;
-
- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
- struct v4l2_hevc_pred_weight_table pred_weight_table;
-
- __u64 flags;
-};
-
-#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC 0x1
-#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC 0x2
-#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR 0x4
-
-struct v4l2_ctrl_hevc_decode_params {
- __s32 pic_order_cnt_val;
- __u8 num_active_dpb_entries;
- struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __u8 num_poc_st_curr_before;
- __u8 num_poc_st_curr_after;
- __u8 num_poc_lt_curr;
- __u8 poc_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __u8 poc_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __u8 poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
- __u64 flags;
-};
-
-struct v4l2_ctrl_hevc_scaling_matrix {
- __u8 scaling_list_4x4[6][16];
- __u8 scaling_list_8x8[6][64];
- __u8 scaling_list_16x16[6][64];
- __u8 scaling_list_32x32[2][64];
- __u8 scaling_list_dc_coef_16x16[6];
- __u8 scaling_list_dc_coef_32x32[2];
-};
-
-/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */
-#define V4L2_CID_CODEC_HANTRO_BASE (V4L2_CTRL_CLASS_CODEC | 0x1200)
-/*
- * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
- * the number of data (in bits) to skip in the
- * slice segment header.
- * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
- * to before syntax element "slice_temporal_mvp_enabled_flag".
- * If IDR, the skipped bits are just "pic_output_flag"
- * (separate_colour_plane_flag is not supported).
- */
-#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_CODEC_HANTRO_BASE + 0)
-
-#endif
diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
index f4105de8a8d2..00828a4f9404 100644
--- a/include/media/v4l2-ctrls.h
+++ b/include/media/v4l2-ctrls.h
@@ -13,12 +13,6 @@
#include <linux/videodev2.h>
#include <media/media-request.h>
-/*
- * Include the stateless codec compound control definitions.
- * This will move to the public headers once this API is fully stable.
- */
-#include <media/hevc-ctrls.h>
-
/* forward references */
struct file;
struct poll_table_struct;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index c8e0f84d204d..0e0ec2c61b80 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -218,6 +218,11 @@ enum v4l2_colorfx {
* We reserve 16 controls for this driver.
*/
#define V4L2_CID_USER_ALLEGRO_BASE (V4L2_CID_USER_BASE + 0x1170)
+/*
+ * The base for Hantro driver controls.
+ * We reserve 128 controls for this driver.
+ */
+#define V4L2_CID_USER_HANTRO_BASE (V4L2_CID_USER_BASE + 0x1180)
/* MPEG-class control IDs */
/* The MPEG controls are applicable to all codec controls
@@ -2302,6 +2307,220 @@ struct v4l2_ctrl_vp9_compressed_hdr {
struct v4l2_vp9_mv_probs mv;
};
+#define V4L2_CID_STATELESS_HEVC_SPS (V4L2_CID_CODEC_STATELESS_BASE + 400)
+#define V4L2_CID_STATELESS_HEVC_PPS (V4L2_CID_CODEC_STATELESS_BASE + 401)
+#define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 402)
+#define V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_STATELESS_BASE + 403)
+#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404)
+#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405)
+#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406)
+
+enum v4l2_stateless_hevc_decode_mode {
+ V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
+ V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
+};
+
+enum v4l2_stateless_hevc_start_code {
+ V4L2_STATELESS_HEVC_START_CODE_NONE,
+ V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
+};
+
+#define V4L2_HEVC_SLICE_TYPE_B 0
+#define V4L2_HEVC_SLICE_TYPE_P 1
+#define V4L2_HEVC_SLICE_TYPE_I 2
+
+#define V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE (1ULL << 0)
+#define V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED (1ULL << 1)
+#define V4L2_HEVC_SPS_FLAG_AMP_ENABLED (1ULL << 2)
+#define V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET (1ULL << 3)
+#define V4L2_HEVC_SPS_FLAG_PCM_ENABLED (1ULL << 4)
+#define V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED (1ULL << 5)
+#define V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT (1ULL << 6)
+#define V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED (1ULL << 7)
+#define V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED (1ULL << 8)
+
+struct v4l2_ctrl_hevc_sps {
+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
+ __u16 pic_width_in_luma_samples;
+ __u16 pic_height_in_luma_samples;
+ __u8 bit_depth_luma_minus8;
+ __u8 bit_depth_chroma_minus8;
+ __u8 log2_max_pic_order_cnt_lsb_minus4;
+ __u8 sps_max_dec_pic_buffering_minus1;
+ __u8 sps_max_num_reorder_pics;
+ __u8 sps_max_latency_increase_plus1;
+ __u8 log2_min_luma_coding_block_size_minus3;
+ __u8 log2_diff_max_min_luma_coding_block_size;
+ __u8 log2_min_luma_transform_block_size_minus2;
+ __u8 log2_diff_max_min_luma_transform_block_size;
+ __u8 max_transform_hierarchy_depth_inter;
+ __u8 max_transform_hierarchy_depth_intra;
+ __u8 pcm_sample_bit_depth_luma_minus1;
+ __u8 pcm_sample_bit_depth_chroma_minus1;
+ __u8 log2_min_pcm_luma_coding_block_size_minus3;
+ __u8 log2_diff_max_min_pcm_luma_coding_block_size;
+ __u8 num_short_term_ref_pic_sets;
+ __u8 num_long_term_ref_pics_sps;
+ __u8 chroma_format_idc;
+ __u8 sps_max_sub_layers_minus1;
+
+ __u8 padding[6];
+ __u64 flags;
+ __u8 reserved[24];
+};
+
+#define V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED (1ULL << 0)
+#define V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT (1ULL << 1)
+#define V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED (1ULL << 2)
+#define V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT (1ULL << 3)
+#define V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED (1ULL << 4)
+#define V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED (1ULL << 5)
+#define V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED (1ULL << 6)
+#define V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT (1ULL << 7)
+#define V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED (1ULL << 8)
+#define V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED (1ULL << 9)
+#define V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED (1ULL << 10)
+#define V4L2_HEVC_PPS_FLAG_TILES_ENABLED (1ULL << 11)
+#define V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED (1ULL << 12)
+#define V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED (1ULL << 13)
+#define V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 14)
+#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED (1ULL << 15)
+#define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER (1ULL << 16)
+#define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT (1ULL << 17)
+#define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL << 18)
+#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
+#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
+
+struct v4l2_ctrl_hevc_pps {
+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
+ __u8 num_extra_slice_header_bits;
+ __u8 num_ref_idx_l0_default_active_minus1;
+ __u8 num_ref_idx_l1_default_active_minus1;
+ __s8 init_qp_minus26;
+ __u8 diff_cu_qp_delta_depth;
+ __s8 pps_cb_qp_offset;
+ __s8 pps_cr_qp_offset;
+ __u8 num_tile_columns_minus1;
+ __u8 num_tile_rows_minus1;
+ __u8 column_width_minus1[20];
+ __u8 row_height_minus1[22];
+ __s8 pps_beta_offset_div2;
+ __s8 pps_tc_offset_div2;
+ __u8 log2_parallel_merge_level_minus2;
+ __u8 padding[9];
+
+ __u64 flags;
+ __u8 reserved[56];
+};
+
+#define V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE 0x01
+
+#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX 16
+
+struct v4l2_hevc_dpb_entry {
+ __u64 timestamp;
+ __u8 flags;
+ __u8 field_pic;
+ __u16 pic_order_cnt[2];
+ __u8 padding[2];
+};
+
+struct v4l2_hevc_pred_weight_table {
+ __s8 delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __s8 luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __s8 delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+ __s8 chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+
+ __s8 delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __s8 luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __s8 delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+ __s8 chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
+
+ __u8 padding[6];
+
+ __u8 luma_log2_weight_denom;
+ __s8 delta_chroma_log2_weight_denom;
+};
+
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_LUMA (1ULL << 0)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_CHROMA (1ULL << 1)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED (1ULL << 2)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_MVD_L1_ZERO (1ULL << 3)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_CABAC_INIT (1ULL << 4)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_COLLOCATED_FROM_L0 (1ULL << 5)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_USE_INTEGER_MV (1ULL << 6)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED (1ULL << 7)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 8)
+#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
+
+struct v4l2_ctrl_hevc_slice_params {
+ __u32 bit_size;
+ __u32 data_bit_offset;
+
+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
+ __u8 nal_unit_type;
+ __u8 nuh_temporal_id_plus1;
+
+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
+ __u8 slice_type;
+ __u8 colour_plane_id;
+ __u16 slice_pic_order_cnt;
+ __u8 num_ref_idx_l0_active_minus1;
+ __u8 num_ref_idx_l1_active_minus1;
+ __u8 collocated_ref_idx;
+ __u8 five_minus_max_num_merge_cand;
+ __s8 slice_qp_delta;
+ __s8 slice_cb_qp_offset;
+ __s8 slice_cr_qp_offset;
+ __s8 slice_act_y_qp_offset;
+ __s8 slice_act_cb_qp_offset;
+ __s8 slice_act_cr_qp_offset;
+ __s8 slice_beta_offset_div2;
+ __s8 slice_tc_offset_div2;
+
+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
+ __u8 pic_struct;
+
+ __u8 reserved;
+
+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
+ __u32 slice_segment_addr;
+ __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+
+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
+ struct v4l2_hevc_pred_weight_table pred_weight_table;
+
+ __u64 flags;
+ __u8 padding[48];
+};
+
+#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC 0x1
+#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC 0x2
+#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR 0x4
+
+struct v4l2_ctrl_hevc_decode_params {
+ __s32 pic_order_cnt_val;
+ __u8 num_active_dpb_entries;
+ __u8 num_poc_st_curr_before;
+ __u8 num_poc_st_curr_after;
+ __u8 num_poc_lt_curr;
+ __u8 poc_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __u8 poc_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __u8 poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __u64 flags;
+};
+
+struct v4l2_ctrl_hevc_scaling_matrix {
+ __u8 scaling_list_4x4[6][16];
+ __u8 scaling_list_8x8[6][64];
+ __u8 scaling_list_16x16[6][64];
+ __u8 scaling_list_32x32[2][64];
+ __u8 scaling_list_dc_coef_16x16[6];
+ __u8 scaling_list_dc_coef_32x32[2];
+};
+
/* MPEG-compression definitions kept for backwards compatibility */
#ifndef __KERNEL__
#define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index e27c8eae78c9..9af1c2460148 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -708,6 +708,7 @@ struct v4l2_pix_format {
#define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh Hadamard Transform (vicodec) */
#define V4L2_PIX_FMT_FWHT_STATELESS v4l2_fourcc('S', 'F', 'W', 'H') /* Stateless FWHT (vicodec) */
#define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
+#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
/* Vendor-specific formats */
#define V4L2_PIX_FMT_CPIA1 v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
@@ -1829,6 +1830,12 @@ enum v4l2_ctrl_type {
V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR = 0x0260,
V4L2_CTRL_TYPE_VP9_FRAME = 0x0261,
+
+ V4L2_CTRL_TYPE_HEVC_SPS = 0x0270,
+ V4L2_CTRL_TYPE_HEVC_PPS = 0x0271,
+ V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
+ V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
+ V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
};
/* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (3 preceding siblings ...)
2022-02-15 11:00 ` [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging Benjamin Gaignard
@ 2022-02-15 11:01 ` Benjamin Gaignard
2022-02-18 11:32 ` Hans Verkuil
2022-02-15 11:01 ` [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params Benjamin Gaignard
` (5 subsequent siblings)
10 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:01 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc, Benjamin Gaignard
RKVDEC driver requires additional fields to perform HEVC decoding.
Even if the driver isn't mainlined yet WIP patches could be find here:
https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
This patch only include the change in HEVC uAPI.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 16 ++++++++++++++++
include/uapi/linux/v4l2-controls.h | 5 +++++
2 files changed, 21 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 4f3b3ba8319f..3296ac3b9fca 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2661,6 +2661,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
+ * - __u8
+ - ``video_parameter_set_id``
+ - Identifies the VPS for reference by other syntax elements.
+ * - __u8
+ - ``seq_parameter_set_id``
+ - Provides an identifier for the SPS for reference by other syntax
+ elements.
* - __u16
- ``pic_width_in_luma_samples``
-
@@ -2800,6 +2807,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
+ * - __u8
+ - ``pic_parameter_set_id``
+ - Identifies the PPS for reference by other syntax elements.
* - __u8
- ``num_extra_slice_header_bits``
-
@@ -3026,6 +3036,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
* - __u8
- ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
- The list of L1 reference elements as indices in the DPB.
+ * - __u16
+ - ``short_term_ref_pic_set_size``
+ -
+ * - __u16
+ - ``long_term_ref_pic_set_size``
+ -
* - __u8
- ``padding``
- Applications and drivers must set this to zero.
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 0e0ec2c61b80..b1a3dc05f02f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -2341,6 +2341,8 @@ enum v4l2_stateless_hevc_start_code {
struct v4l2_ctrl_hevc_sps {
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
+ __u8 video_parameter_set_id;
+ __u8 seq_parameter_set_id;
__u16 pic_width_in_luma_samples;
__u16 pic_height_in_luma_samples;
__u8 bit_depth_luma_minus8;
@@ -2393,6 +2395,7 @@ struct v4l2_ctrl_hevc_sps {
struct v4l2_ctrl_hevc_pps {
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
+ __u8 pic_parameter_set_id;
__u8 num_extra_slice_header_bits;
__u8 num_ref_idx_l0_default_active_minus1;
__u8 num_ref_idx_l1_default_active_minus1;
@@ -2487,6 +2490,8 @@ struct v4l2_ctrl_hevc_slice_params {
__u32 slice_segment_addr;
__u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
__u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
+ __u16 short_term_ref_pic_set_size;
+ __u16 long_term_ref_pic_set_size;
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
struct v4l2_hevc_pred_weight_table pred_weight_table;
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (4 preceding siblings ...)
2022-02-15 11:01 ` [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver Benjamin Gaignard
@ 2022-02-15 11:01 ` Benjamin Gaignard
2022-02-15 14:17 ` John Cox
2022-02-15 11:01 ` [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control Benjamin Gaignard
` (4 subsequent siblings)
10 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:01 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc, Benjamin Gaignard
The bit size of the slice could be deduced from the buffer payload
so remove bit_size field to avoid duplicated the information.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 ++++-------
include/uapi/linux/v4l2-controls.h | 3 +--
3 files changed, 5 insertions(+), 12 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 3296ac3b9fca..c3ae97657fa7 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:stub-columns: 0
:widths: 1 1 2
- * - __u32
- - ``bit_size``
- - Size (in bits) of the current slice data.
* - __u32
- ``data_bit_offset``
- Offset (in bits) to the video data in the current slice data.
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
index 8ab2d9c6f048..db8c7475eeb8 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
@@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
const struct v4l2_hevc_pred_weight_table *pred_weight_table;
unsigned int width_in_ctb_luma, ctb_size_luma;
unsigned int log2_max_luma_coding_block_size;
+ size_t slice_bytes;
dma_addr_t src_buf_addr;
- dma_addr_t src_buf_end_addr;
u32 chroma_log2_weight_denom;
u32 output_pic_list_index;
u32 pic_order_cnt[2];
@@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
- reg = slice_params->bit_size;
- cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
+ slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
+ cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
/* Source beginning and end addresses. */
@@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
- src_buf_end_addr = src_buf_addr +
- DIV_ROUND_UP(slice_params->bit_size, 8);
-
- reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
+ reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
/* Coding tree block address */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index b1a3dc05f02f..27f5d272dc43 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
struct v4l2_ctrl_hevc_slice_params {
- __u32 bit_size;
__u32 data_bit_offset;
/* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
@@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
/* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
__u8 pic_struct;
- __u8 reserved;
+ __u8 reserved[5];
/* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
__u32 slice_segment_addr;
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (5 preceding siblings ...)
2022-02-15 11:01 ` [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params Benjamin Gaignard
@ 2022-02-15 11:01 ` Benjamin Gaignard
2022-02-16 10:54 ` John Cox
2022-02-15 11:01 ` [RFC v2 8/8] media: uapi: Change data_bit_offset definition Benjamin Gaignard
` (3 subsequent siblings)
10 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:01 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc, Benjamin Gaignard
The number of 'entry point offset' could be very variable.
Rather than use a large static array define a v4l2 dynamic array
of integer control.
The number of entry point offsets is reported by the elems field.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
version 2:
- Define a new CID for entry point offsets parameters
.../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
include/uapi/linux/v4l2-controls.h | 1 +
2 files changed, 10 insertions(+)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index c3ae97657fa7..679595e94643 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -3101,6 +3101,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
\normalsize
+``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
+ Specifies the i-th entry point offset in bytes and is represented by
+ offset_len_minus1 plus 1 bits.
+ This control is a dynamically sized array. The number of entry point
+ offsets is reported by the ``elems`` field.
+ This bitstream parameter is defined according to :ref:`hevc`.
+ They are described in section 7.4.7.1 "General slice segment header
+ semantics" of the specification.
+
``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
Specifies the HEVC scaling matrix parameters used for the scaling process
for transform coefficients.
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 27f5d272dc43..700e6cce958c 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -2314,6 +2314,7 @@ struct v4l2_ctrl_vp9_compressed_hdr {
#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404)
#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405)
#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406)
+#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_STATELESS_BASE + 407)
enum v4l2_stateless_hevc_decode_mode {
V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [RFC v2 8/8] media: uapi: Change data_bit_offset definition
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (6 preceding siblings ...)
2022-02-15 11:01 ` [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control Benjamin Gaignard
@ 2022-02-15 11:01 ` Benjamin Gaignard
2022-02-15 13:01 ` [RFC v2 0/8] Move HEVC stateless controls out of staging Sebastian Fricke
` (2 subsequent siblings)
10 siblings, 0 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 11:01 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc, Benjamin Gaignard
'F.7.3.6.1 General slice segment header syntax' section of HEVC
specification describes that a slice header always end byte aligned,
therefore we only need to provide the data offset in byte.
Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
---
Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst | 4 ++--
drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 2 +-
include/uapi/linux/v4l2-controls.h | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
index 679595e94643..d5bdc2359b88 100644
--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
@@ -2966,8 +2966,8 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
:widths: 1 1 2
* - __u32
- - ``data_bit_offset``
- - Offset (in bits) to the video data in the current slice data.
+ - ``data_byte_offset``
+ - Offset (in byte) to the video data in the current slice data.
* - __u8
- ``nal_unit_type``
-
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
index db8c7475eeb8..81c4362ddbf4 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
@@ -402,7 +402,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
/* Initialize bitstream access. */
cedrus_write(dev, VE_DEC_H265_TRIGGER, VE_DEC_H265_TRIGGER_INIT_SWDEC);
- cedrus_h265_skip_bits(dev, slice_params->data_bit_offset);
+ cedrus_h265_skip_bits(dev, slice_params->data_byte_offset * 8);
/* Bitstream parameters. */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 700e6cce958c..6968b5888607 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -2458,7 +2458,7 @@ struct v4l2_hevc_pred_weight_table {
#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
struct v4l2_ctrl_hevc_slice_params {
- __u32 data_bit_offset;
+ __u32 data_byte_offset;
/* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
__u8 nal_unit_type;
--
2.32.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [RFC v2 0/8] Move HEVC stateless controls out of staging
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (7 preceding siblings ...)
2022-02-15 11:01 ` [RFC v2 8/8] media: uapi: Change data_bit_offset definition Benjamin Gaignard
@ 2022-02-15 13:01 ` Sebastian Fricke
2022-02-15 13:24 ` Benjamin Gaignard
2022-02-15 16:14 ` Jernej Škrabec
2022-02-18 9:59 ` Hans Verkuil
10 siblings, 1 reply; 35+ messages in thread
From: Sebastian Fricke @ 2022-02-15 13:01 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Hey Benjamin,
On 15.02.2022 12:00, Benjamin Gaignard wrote:
>This series aims to make HEVC uapi stable and usable for hardware
>decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>and 2 out of the tree drivers (rkvdec and RPI).
Why is rkvdec out-of-tree? It is in the staging directory just like
hantro and cedrus? Am I missing something here?
Greetings,
Sebastian
>
>The 3 first patches are from Hans to implement v4l2 dynamic control
>feature which is need by patch 7 for V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET
>definition.
>
>Patch 4 move the existing uapi to stable, including definitions renaming
>and CID number change to fit with v4l2 naming.
>
>Patches 5 and 7 add fields needed for rkvdec and RPI decoders.
>
>Patches 6 is cleaning up the uapi of useless field.
>Patches 8 change one field description and name to define offset by
>bytes rather than by bits
>
>Benjamin
>
>Benjamin Gaignard (5):
> media: uapi: Move HEVC stateless controls out of staging
> media: uapi: Add fields needed for RKVDEC driver
> media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> media: uapi: Change data_bit_offset definition
>
>Hans Verkuil (3):
> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> v4l2-ctrls: add support for dynamically allocated arrays.
> vivid: add dynamic array test control
>
> .../userspace-api/media/drivers/hantro.rst | 5 -
> .../media/v4l/ext-ctrls-codec.rst | 58 ++--
> .../media/v4l/vidioc-queryctrl.rst | 8 +
> .../media/test-drivers/vivid/vivid-ctrls.c | 15 ++
> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++--
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 ++++++++++---
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> .../staging/media/sunxi/cedrus/cedrus_h265.c | 13 +-
> include/linux/hantro-media.h | 17 ++
> include/media/hevc-ctrls.h | 250 ------------------
> include/media/v4l2-ctrls.h | 48 +++-
> include/uapi/linux/v4l2-controls.h | 224 ++++++++++++++++
> include/uapi/linux/videodev2.h | 8 +
> 19 files changed, 640 insertions(+), 408 deletions(-)
> create mode 100644 include/linux/hantro-media.h
> delete mode 100644 include/media/hevc-ctrls.h
>
>--
>2.32.0
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 0/8] Move HEVC stateless controls out of staging
2022-02-15 13:01 ` [RFC v2 0/8] Move HEVC stateless controls out of staging Sebastian Fricke
@ 2022-02-15 13:24 ` Benjamin Gaignard
0 siblings, 0 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 13:24 UTC (permalink / raw)
To: Sebastian Fricke
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Le 15/02/2022 à 14:01, Sebastian Fricke a écrit :
> Hey Benjamin,
>
> On 15.02.2022 12:00, Benjamin Gaignard wrote:
>> This series aims to make HEVC uapi stable and usable for hardware
>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>> and 2 out of the tree drivers (rkvdec and RPI).
>
> Why is rkvdec out-of-tree? It is in the staging directory just like
> hantro and cedrus? Am I missing something here?
I have made a shortcut here: HEVC part of rkvdec isn't in staging directory.
I hope that this series will enough to add it after.
Regards,
Benjamin
>
> Greetings,
> Sebastian
>
>>
>> The 3 first patches are from Hans to implement v4l2 dynamic control
>> feature which is need by patch 7 for
>> V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET
>> definition.
>>
>> Patch 4 move the existing uapi to stable, including definitions renaming
>> and CID number change to fit with v4l2 naming.
>>
>> Patches 5 and 7 add fields needed for rkvdec and RPI decoders.
>>
>> Patches 6 is cleaning up the uapi of useless field.
>> Patches 8 change one field description and name to define offset by
>> bytes rather than by bits
>>
>> Benjamin
>>
>> Benjamin Gaignard (5):
>> media: uapi: Move HEVC stateless controls out of staging
>> media: uapi: Add fields needed for RKVDEC driver
>> media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>> media: uapi: Change data_bit_offset definition
>>
>> Hans Verkuil (3):
>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>> v4l2-ctrls: add support for dynamically allocated arrays.
>> vivid: add dynamic array test control
>>
>> .../userspace-api/media/drivers/hantro.rst | 5 -
>> .../media/v4l/ext-ctrls-codec.rst | 58 ++--
>> .../media/v4l/vidioc-queryctrl.rst | 8 +
>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 ++
>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++--
>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 ++++++++++---
>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 13 +-
>> include/linux/hantro-media.h | 17 ++
>> include/media/hevc-ctrls.h | 250 ------------------
>> include/media/v4l2-ctrls.h | 48 +++-
>> include/uapi/linux/v4l2-controls.h | 224 ++++++++++++++++
>> include/uapi/linux/videodev2.h | 8 +
>> 19 files changed, 640 insertions(+), 408 deletions(-)
>> create mode 100644 include/linux/hantro-media.h
>> delete mode 100644 include/media/hevc-ctrls.h
>>
>> --
>> 2.32.0
>>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 11:01 ` [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params Benjamin Gaignard
@ 2022-02-15 14:17 ` John Cox
2022-02-15 14:35 ` Benjamin Gaignard
0 siblings, 1 reply; 35+ messages in thread
From: John Cox @ 2022-02-15 14:17 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, Benjamin Gaignard
Hi
>The bit size of the slice could be deduced from the buffer payload
>so remove bit_size field to avoid duplicated the information.
I think this is a bad idea. In the future we are (I hope) going to want
to have an array (variable) of slice headers all referring to the same
bit buffer. When we do that we will need this field.
>Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>---
> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
> drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 ++++-------
> include/uapi/linux/v4l2-controls.h | 3 +--
> 3 files changed, 5 insertions(+), 12 deletions(-)
>
>diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>index 3296ac3b9fca..c3ae97657fa7 100644
>--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>@@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
>- * - __u32
>- - ``bit_size``
>- - Size (in bits) of the current slice data.
> * - __u32
> - ``data_bit_offset``
> - Offset (in bits) to the video data in the current slice data.
>diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>index 8ab2d9c6f048..db8c7475eeb8 100644
>--- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>+++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>@@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
> const struct v4l2_hevc_pred_weight_table *pred_weight_table;
> unsigned int width_in_ctb_luma, ctb_size_luma;
> unsigned int log2_max_luma_coding_block_size;
>+ size_t slice_bytes;
> dma_addr_t src_buf_addr;
>- dma_addr_t src_buf_end_addr;
> u32 chroma_log2_weight_denom;
> u32 output_pic_list_index;
> u32 pic_order_cnt[2];
>@@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>
> cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
>
>- reg = slice_params->bit_size;
>- cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
>+ slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
>+ cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
I think one of these must be wrong. bit_size is in bits,
vb2_get_plane_payload is in bytes?
Regards
John Cox
> /* Source beginning and end addresses. */
>
>@@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>
> cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
>
>- src_buf_end_addr = src_buf_addr +
>- DIV_ROUND_UP(slice_params->bit_size, 8);
>-
>- reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
>+ reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
> cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
>
> /* Coding tree block address */
>diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>index b1a3dc05f02f..27f5d272dc43 100644
>--- a/include/uapi/linux/v4l2-controls.h
>+++ b/include/uapi/linux/v4l2-controls.h
>@@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
> #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
>
> struct v4l2_ctrl_hevc_slice_params {
>- __u32 bit_size;
> __u32 data_bit_offset;
>
> /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>@@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
> __u8 pic_struct;
>
>- __u8 reserved;
>+ __u8 reserved[5];
>
> /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
> __u32 slice_segment_addr;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 14:17 ` John Cox
@ 2022-02-15 14:35 ` Benjamin Gaignard
2022-02-15 14:50 ` John Cox
0 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 14:35 UTC (permalink / raw)
To: John Cox
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
Le 15/02/2022 à 15:17, John Cox a écrit :
> Hi
>
>> The bit size of the slice could be deduced from the buffer payload
>> so remove bit_size field to avoid duplicated the information.
> I think this is a bad idea. In the future we are (I hope) going to want
> to have an array (variable) of slice headers all referring to the same
> bit buffer. When we do that we will need this field.
I wonder if that could be considering like another decode mode and so
use an other control ?
>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
>> drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 ++++-------
>> include/uapi/linux/v4l2-controls.h | 3 +--
>> 3 files changed, 5 insertions(+), 12 deletions(-)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index 3296ac3b9fca..c3ae97657fa7 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> :stub-columns: 0
>> :widths: 1 1 2
>>
>> - * - __u32
>> - - ``bit_size``
>> - - Size (in bits) of the current slice data.
>> * - __u32
>> - ``data_bit_offset``
>> - Offset (in bits) to the video data in the current slice data.
>> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> index 8ab2d9c6f048..db8c7475eeb8 100644
>> --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>> const struct v4l2_hevc_pred_weight_table *pred_weight_table;
>> unsigned int width_in_ctb_luma, ctb_size_luma;
>> unsigned int log2_max_luma_coding_block_size;
>> + size_t slice_bytes;
>> dma_addr_t src_buf_addr;
>> - dma_addr_t src_buf_end_addr;
>> u32 chroma_log2_weight_denom;
>> u32 output_pic_list_index;
>> u32 pic_order_cnt[2];
>> @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>
>> cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
>>
>> - reg = slice_params->bit_size;
>> - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
>> + slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
>> + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
> I think one of these must be wrong. bit_size is in bits,
> vb2_get_plane_payload is in bytes?
You are right it should be vb2_get_plane_payload() * 8 to get the size in bits.
I will change that in v3.
>
> Regards
>
> John Cox
>
>> /* Source beginning and end addresses. */
>>
>> @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>
>> cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
>>
>> - src_buf_end_addr = src_buf_addr +
>> - DIV_ROUND_UP(slice_params->bit_size, 8);
>> -
>> - reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
>> + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
>> cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
>>
>> /* Coding tree block address */
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index b1a3dc05f02f..27f5d272dc43 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
>> #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
>>
>> struct v4l2_ctrl_hevc_slice_params {
>> - __u32 bit_size;
>> __u32 data_bit_offset;
>>
>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>> @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
>> __u8 pic_struct;
>>
>> - __u8 reserved;
>> + __u8 reserved[5];
>>
>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>> __u32 slice_segment_addr;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 14:35 ` Benjamin Gaignard
@ 2022-02-15 14:50 ` John Cox
2022-02-15 14:57 ` Benjamin Gaignard
2022-02-15 15:28 ` Nicolas Dufresne
0 siblings, 2 replies; 35+ messages in thread
From: John Cox @ 2022-02-15 14:50 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
>
>Le 15/02/2022 à 15:17, John Cox a écrit :
>> Hi
>>
>>> The bit size of the slice could be deduced from the buffer payload
>>> so remove bit_size field to avoid duplicated the information.
>> I think this is a bad idea. In the future we are (I hope) going to want
>> to have an array (variable) of slice headers all referring to the same
>> bit buffer. When we do that we will need this field.
>
>I wonder if that could be considering like another decode mode and so
>use an other control ?
I, personally, would be in favour of making the slice header control a
variable array just as it is. If userland can't cope with multiple
entries then just send them one at a time and the code looks exactly
like it does at the moment and if the driver can't then set max array
entries to 1.
Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
can say with experience that the code and effort overhead is very low.
Either way having a multiple slice header control in the UAPI is
important for efficiency.
Regards
John Cox
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
>>> drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 ++++-------
>>> include/uapi/linux/v4l2-controls.h | 3 +--
>>> 3 files changed, 5 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> index 3296ac3b9fca..c3ae97657fa7 100644
>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> @@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>> :stub-columns: 0
>>> :widths: 1 1 2
>>>
>>> - * - __u32
>>> - - ``bit_size``
>>> - - Size (in bits) of the current slice data.
>>> * - __u32
>>> - ``data_bit_offset``
>>> - Offset (in bits) to the video data in the current slice data.
>>> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>>> index 8ab2d9c6f048..db8c7475eeb8 100644
>>> --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>>> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>>> @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>> const struct v4l2_hevc_pred_weight_table *pred_weight_table;
>>> unsigned int width_in_ctb_luma, ctb_size_luma;
>>> unsigned int log2_max_luma_coding_block_size;
>>> + size_t slice_bytes;
>>> dma_addr_t src_buf_addr;
>>> - dma_addr_t src_buf_end_addr;
>>> u32 chroma_log2_weight_denom;
>>> u32 output_pic_list_index;
>>> u32 pic_order_cnt[2];
>>> @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>>
>>> cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
>>>
>>> - reg = slice_params->bit_size;
>>> - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
>>> + slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
>>> + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
>> I think one of these must be wrong. bit_size is in bits,
>> vb2_get_plane_payload is in bytes?
>
>You are right it should be vb2_get_plane_payload() * 8 to get the size in bits.
>
>I will change that in v3.
>
>>
>> Regards
>>
>> John Cox
>>
>>> /* Source beginning and end addresses. */
>>>
>>> @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>>
>>> cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
>>>
>>> - src_buf_end_addr = src_buf_addr +
>>> - DIV_ROUND_UP(slice_params->bit_size, 8);
>>> -
>>> - reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
>>> + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
>>> cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
>>>
>>> /* Coding tree block address */
>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>> index b1a3dc05f02f..27f5d272dc43 100644
>>> --- a/include/uapi/linux/v4l2-controls.h
>>> +++ b/include/uapi/linux/v4l2-controls.h
>>> @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
>>> #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
>>>
>>> struct v4l2_ctrl_hevc_slice_params {
>>> - __u32 bit_size;
>>> __u32 data_bit_offset;
>>>
>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>>> @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
>>> __u8 pic_struct;
>>>
>>> - __u8 reserved;
>>> + __u8 reserved[5];
>>>
>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>>> __u32 slice_segment_addr;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 14:50 ` John Cox
@ 2022-02-15 14:57 ` Benjamin Gaignard
2022-02-15 15:28 ` Nicolas Dufresne
1 sibling, 0 replies; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-15 14:57 UTC (permalink / raw)
To: John Cox
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
Le 15/02/2022 à 15:50, John Cox a écrit :
> On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
>
>> Le 15/02/2022 à 15:17, John Cox a écrit :
>>> Hi
>>>
>>>> The bit size of the slice could be deduced from the buffer payload
>>>> so remove bit_size field to avoid duplicated the information.
>>> I think this is a bad idea. In the future we are (I hope) going to want
>>> to have an array (variable) of slice headers all referring to the same
>>> bit buffer. When we do that we will need this field.
>> I wonder if that could be considering like another decode mode and so
>> use an other control ?
> I, personally, would be in favour of making the slice header control a
> variable array just as it is. If userland can't cope with multiple
> entries then just send them one at a time and the code looks exactly
> like it does at the moment and if the driver can't then set max array
> entries to 1.
>
> Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
> can say with experience that the code and effort overhead is very low.
>
> Either way having a multiple slice header control in the UAPI is
> important for efficiency.
Removing this field was one the Jernej's remarks.
If he agree with you it won't be a big effort for me to remove this patch
from v3.
Regards,
Benjamin
>
> Regards
>
> John Cox
>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
>>>> drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 ++++-------
>>>> include/uapi/linux/v4l2-controls.h | 3 +--
>>>> 3 files changed, 5 insertions(+), 12 deletions(-)
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> index 3296ac3b9fca..c3ae97657fa7 100644
>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> @@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>> :stub-columns: 0
>>>> :widths: 1 1 2
>>>>
>>>> - * - __u32
>>>> - - ``bit_size``
>>>> - - Size (in bits) of the current slice data.
>>>> * - __u32
>>>> - ``data_bit_offset``
>>>> - Offset (in bits) to the video data in the current slice data.
>>>> diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>>>> index 8ab2d9c6f048..db8c7475eeb8 100644
>>>> --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>>>> +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>>>> @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>>> const struct v4l2_hevc_pred_weight_table *pred_weight_table;
>>>> unsigned int width_in_ctb_luma, ctb_size_luma;
>>>> unsigned int log2_max_luma_coding_block_size;
>>>> + size_t slice_bytes;
>>>> dma_addr_t src_buf_addr;
>>>> - dma_addr_t src_buf_end_addr;
>>>> u32 chroma_log2_weight_denom;
>>>> u32 output_pic_list_index;
>>>> u32 pic_order_cnt[2];
>>>> @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>>>
>>>> cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
>>>>
>>>> - reg = slice_params->bit_size;
>>>> - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
>>>> + slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
>>>> + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
>>> I think one of these must be wrong. bit_size is in bits,
>>> vb2_get_plane_payload is in bytes?
>> You are right it should be vb2_get_plane_payload() * 8 to get the size in bits.
>>
>> I will change that in v3.
>>
>>> Regards
>>>
>>> John Cox
>>>
>>>> /* Source beginning and end addresses. */
>>>>
>>>> @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>>>>
>>>> cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
>>>>
>>>> - src_buf_end_addr = src_buf_addr +
>>>> - DIV_ROUND_UP(slice_params->bit_size, 8);
>>>> -
>>>> - reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
>>>> + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
>>>> cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
>>>>
>>>> /* Coding tree block address */
>>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>>> index b1a3dc05f02f..27f5d272dc43 100644
>>>> --- a/include/uapi/linux/v4l2-controls.h
>>>> +++ b/include/uapi/linux/v4l2-controls.h
>>>> @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
>>>> #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
>>>>
>>>> struct v4l2_ctrl_hevc_slice_params {
>>>> - __u32 bit_size;
>>>> __u32 data_bit_offset;
>>>>
>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>>>> @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
>>>> __u8 pic_struct;
>>>>
>>>> - __u8 reserved;
>>>> + __u8 reserved[5];
>>>>
>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>>>> __u32 slice_segment_addr;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 14:50 ` John Cox
2022-02-15 14:57 ` Benjamin Gaignard
@ 2022-02-15 15:28 ` Nicolas Dufresne
2022-02-15 16:00 ` John Cox
1 sibling, 1 reply; 35+ messages in thread
From: Nicolas Dufresne @ 2022-02-15 15:28 UTC (permalink / raw)
To: John Cox, Benjamin Gaignard
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, linux-media,
linux-kernel, linux-staging, linux-arm-kernel, linux-sunxi,
kernel, knaerzche
Le mardi 15 février 2022 à 14:50 +0000, John Cox a écrit :
> On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
>
> >
> > Le 15/02/2022 à 15:17, John Cox a écrit :
> > > Hi
> > >
> > > > The bit size of the slice could be deduced from the buffer payload
> > > > so remove bit_size field to avoid duplicated the information.
> > > I think this is a bad idea. In the future we are (I hope) going to want
> > > to have an array (variable) of slice headers all referring to the same
> > > bit buffer. When we do that we will need this field.
> >
> > I wonder if that could be considering like another decode mode and so
> > use an other control ?
>
> I, personally, would be in favour of making the slice header control a
> variable array just as it is. If userland can't cope with multiple
> entries then just send them one at a time and the code looks exactly
> like it does at the moment and if the driver can't then set max array
> entries to 1.
>
> Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
> can say with experience that the code and effort overhead is very low.
>
> Either way having a multiple slice header control in the UAPI is
> important for efficiency.
Just to clarify the idea, we would have a single slice controls, always dynamic:
1. For sliced based decoder
The dynamic array slice control is implemented by the driver and its size must
be 1.
2. For frame based decoder that don't care for slices
The dynamic array slice controls is not implement. Userland detects that at
runtime, similar to the VP9 compressed headers.
3. For frame based decoders that needs slices (or driver that supports offset
and can gain performance with such mode)
The dynamic array slice controls is implemented, and should contain all the
slices found in the OUTPUT buffer.
So the reason for this bit_size (not sure why its bits though, perhaps someone
can educate me ?) Would be to let the driver offset inside the the single
OUTPUT/bitstream buffer in case this is not automatically found by the driver
(or that no start-code is needed). Is that last bit correct ? If so, should we
change it to an offset rather then a size ? Shall we allow using offesets inside
larger buffer (e.g. to avoid some memory copies) for the Sliced Base cases ?
>
> Regards
>
> John Cox
>
> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> > > > ---
> > > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
> > > > drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 ++++-------
> > > > include/uapi/linux/v4l2-controls.h | 3 +--
> > > > 3 files changed, 5 insertions(+), 12 deletions(-)
> > > >
> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > index 3296ac3b9fca..c3ae97657fa7 100644
> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> > > > @@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> > > > :stub-columns: 0
> > > > :widths: 1 1 2
> > > >
> > > > - * - __u32
> > > > - - ``bit_size``
> > > > - - Size (in bits) of the current slice data.
> > > > * - __u32
> > > > - ``data_bit_offset``
> > > > - Offset (in bits) to the video data in the current slice data.
> > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> > > > index 8ab2d9c6f048..db8c7475eeb8 100644
> > > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> > > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> > > > @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
> > > > const struct v4l2_hevc_pred_weight_table *pred_weight_table;
> > > > unsigned int width_in_ctb_luma, ctb_size_luma;
> > > > unsigned int log2_max_luma_coding_block_size;
> > > > + size_t slice_bytes;
> > > > dma_addr_t src_buf_addr;
> > > > - dma_addr_t src_buf_end_addr;
> > > > u32 chroma_log2_weight_denom;
> > > > u32 output_pic_list_index;
> > > > u32 pic_order_cnt[2];
> > > > @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
> > > >
> > > > cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
> > > >
> > > > - reg = slice_params->bit_size;
> > > > - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
> > > > + slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
> > > > + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
> > > I think one of these must be wrong. bit_size is in bits,
> > > vb2_get_plane_payload is in bytes?
> >
> > You are right it should be vb2_get_plane_payload() * 8 to get the size in bits.
> >
> > I will change that in v3.
> >
> > >
> > > Regards
> > >
> > > John Cox
> > >
> > > > /* Source beginning and end addresses. */
> > > >
> > > > @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
> > > >
> > > > cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
> > > >
> > > > - src_buf_end_addr = src_buf_addr +
> > > > - DIV_ROUND_UP(slice_params->bit_size, 8);
> > > > -
> > > > - reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
> > > > + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
> > > > cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
> > > >
> > > > /* Coding tree block address */
> > > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> > > > index b1a3dc05f02f..27f5d272dc43 100644
> > > > --- a/include/uapi/linux/v4l2-controls.h
> > > > +++ b/include/uapi/linux/v4l2-controls.h
> > > > @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
> > > > #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
> > > >
> > > > struct v4l2_ctrl_hevc_slice_params {
> > > > - __u32 bit_size;
> > > > __u32 data_bit_offset;
> > > >
> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
> > > > @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
> > > > __u8 pic_struct;
> > > >
> > > > - __u8 reserved;
> > > > + __u8 reserved[5];
> > > >
> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
> > > > __u32 slice_segment_addr;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 15:28 ` Nicolas Dufresne
@ 2022-02-15 16:00 ` John Cox
2022-02-15 16:11 ` Jernej Škrabec
0 siblings, 1 reply; 35+ messages in thread
From: John Cox @ 2022-02-15 16:00 UTC (permalink / raw)
To: Nicolas Dufresne
Cc: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, hverkuil-cisco, jonas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
On Tue, 15 Feb 2022 10:28:55 -0500, you wrote:
>Le mardi 15 février 2022 à 14:50 +0000, John Cox a écrit :
>> On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
>>
>> >
>> > Le 15/02/2022 à 15:17, John Cox a écrit :
>> > > Hi
>> > >
>> > > > The bit size of the slice could be deduced from the buffer payload
>> > > > so remove bit_size field to avoid duplicated the information.
>> > > I think this is a bad idea. In the future we are (I hope) going to want
>> > > to have an array (variable) of slice headers all referring to the same
>> > > bit buffer. When we do that we will need this field.
>> >
>> > I wonder if that could be considering like another decode mode and so
>> > use an other control ?
>>
>> I, personally, would be in favour of making the slice header control a
>> variable array just as it is. If userland can't cope with multiple
>> entries then just send them one at a time and the code looks exactly
>> like it does at the moment and if the driver can't then set max array
>> entries to 1.
>>
>> Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
>> can say with experience that the code and effort overhead is very low.
>>
>> Either way having a multiple slice header control in the UAPI is
>> important for efficiency.
>
>Just to clarify the idea, we would have a single slice controls, always dynamic:
>
>1. For sliced based decoder
>
>The dynamic array slice control is implemented by the driver and its size must
>be 1.
Yes
>2. For frame based decoder that don't care for slices
>
>The dynamic array slice controls is not implement. Userland detects that at
>runtime, similar to the VP9 compressed headers.
If the driver parses all the slice header then that seems plausible
>3. For frame based decoders that needs slices (or driver that supports offset
>and can gain performance with such mode)
>
>The dynamic array slice controls is implemented, and should contain all the
>slices found in the OUTPUT buffer.
>
>So the reason for this bit_size (not sure why its bits though, perhaps someone
>can educate me ?)
RPi doesn't need bits and would be happy with bytes however
slice_segment data isn't byte aligned at the end so its possible that
there might be decoders out there that want an accurate length for that.
> Would be to let the driver offset inside the the single
>OUTPUT/bitstream buffer in case this is not automatically found by the driver
>(or that no start-code is needed). Is that last bit correct ? If so, should we
>change it to an offset rather then a size ? Shall we allow using offesets inside
>larger buffer (e.g. to avoid some memory copies) for the Sliced Base cases ?
I use (in the current structure) data_bit_offset to find the start of
each slice's slice_segment_data within the OUTPUT buffer and bit_size to
find the end. RPi doesn't / can't parse the slice_header and so wants
all of that. Decoders that do parse the header might plausably want
header offsets too and it would facilitate zero copy of the bit buffer.
>> Regards
>>
>> John Cox
>>
>> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> > > > ---
>> > > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
>> > > > drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 ++++-------
>> > > > include/uapi/linux/v4l2-controls.h | 3 +--
>> > > > 3 files changed, 5 insertions(+), 12 deletions(-)
>> > > >
>> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > > index 3296ac3b9fca..c3ae97657fa7 100644
>> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> > > > @@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> > > > :stub-columns: 0
>> > > > :widths: 1 1 2
>> > > >
>> > > > - * - __u32
>> > > > - - ``bit_size``
>> > > > - - Size (in bits) of the current slice data.
>> > > > * - __u32
>> > > > - ``data_bit_offset``
>> > > > - Offset (in bits) to the video data in the current slice data.
>> > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> > > > index 8ab2d9c6f048..db8c7475eeb8 100644
>> > > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> > > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> > > > @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>> > > > const struct v4l2_hevc_pred_weight_table *pred_weight_table;
>> > > > unsigned int width_in_ctb_luma, ctb_size_luma;
>> > > > unsigned int log2_max_luma_coding_block_size;
>> > > > + size_t slice_bytes;
>> > > > dma_addr_t src_buf_addr;
>> > > > - dma_addr_t src_buf_end_addr;
>> > > > u32 chroma_log2_weight_denom;
>> > > > u32 output_pic_list_index;
>> > > > u32 pic_order_cnt[2];
>> > > > @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>> > > >
>> > > > cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
>> > > >
>> > > > - reg = slice_params->bit_size;
>> > > > - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
>> > > > + slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
>> > > > + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
>> > > I think one of these must be wrong. bit_size is in bits,
>> > > vb2_get_plane_payload is in bytes?
>> >
>> > You are right it should be vb2_get_plane_payload() * 8 to get the size in bits.
>> >
>> > I will change that in v3.
>> >
>> > >
>> > > Regards
>> > >
>> > > John Cox
>> > >
>> > > > /* Source beginning and end addresses. */
>> > > >
>> > > > @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct cedrus_ctx *ctx,
>> > > >
>> > > > cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
>> > > >
>> > > > - src_buf_end_addr = src_buf_addr +
>> > > > - DIV_ROUND_UP(slice_params->bit_size, 8);
>> > > > -
>> > > > - reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
>> > > > + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
>> > > > cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
>> > > >
>> > > > /* Coding tree block address */
>> > > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> > > > index b1a3dc05f02f..27f5d272dc43 100644
>> > > > --- a/include/uapi/linux/v4l2-controls.h
>> > > > +++ b/include/uapi/linux/v4l2-controls.h
>> > > > @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
>> > > > #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
>> > > >
>> > > > struct v4l2_ctrl_hevc_slice_params {
>> > > > - __u32 bit_size;
>> > > > __u32 data_bit_offset;
>> > > >
>> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>> > > > @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
>> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
>> > > > __u8 pic_struct;
>> > > >
>> > > > - __u8 reserved;
>> > > > + __u8 reserved[5];
>> > > >
>> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>> > > > __u32 slice_segment_addr;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 16:00 ` John Cox
@ 2022-02-15 16:11 ` Jernej Škrabec
2022-02-15 16:31 ` John Cox
0 siblings, 1 reply; 35+ messages in thread
From: Jernej Škrabec @ 2022-02-15 16:11 UTC (permalink / raw)
To: Nicolas Dufresne, John Cox
Cc: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, hverkuil-cisco, jonas, linux-media,
linux-kernel, linux-staging, linux-arm-kernel, linux-sunxi,
kernel, knaerzche
Dne torek, 15. februar 2022 ob 17:00:33 CET je John Cox napisal(a):
> On Tue, 15 Feb 2022 10:28:55 -0500, you wrote:
>
> >Le mardi 15 février 2022 à 14:50 +0000, John Cox a écrit :
> >> On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
> >>
> >> >
> >> > Le 15/02/2022 à 15:17, John Cox a écrit :
> >> > > Hi
> >> > >
> >> > > > The bit size of the slice could be deduced from the buffer payload
> >> > > > so remove bit_size field to avoid duplicated the information.
> >> > > I think this is a bad idea. In the future we are (I hope) going to
want
> >> > > to have an array (variable) of slice headers all referring to the
same
> >> > > bit buffer. When we do that we will need this field.
> >> >
> >> > I wonder if that could be considering like another decode mode and so
> >> > use an other control ?
> >>
> >> I, personally, would be in favour of making the slice header control a
> >> variable array just as it is. If userland can't cope with multiple
> >> entries then just send them one at a time and the code looks exactly
> >> like it does at the moment and if the driver can't then set max array
> >> entries to 1.
> >>
> >> Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
> >> can say with experience that the code and effort overhead is very low.
> >>
> >> Either way having a multiple slice header control in the UAPI is
> >> important for efficiency.
> >
> >Just to clarify the idea, we would have a single slice controls, always
dynamic:
> >
> >1. For sliced based decoder
> >
> >The dynamic array slice control is implemented by the driver and its size
must
> >be 1.
>
> Yes
>
> >2. For frame based decoder that don't care for slices
> >
> >The dynamic array slice controls is not implement. Userland detects that at
> >runtime, similar to the VP9 compressed headers.
>
> If the driver parses all the slice header then that seems plausible
>
> >3. For frame based decoders that needs slices (or driver that supports
offset
> >and can gain performance with such mode)
> >
> >The dynamic array slice controls is implemented, and should contain all the
> >slices found in the OUTPUT buffer.
> >
> >So the reason for this bit_size (not sure why its bits though, perhaps
someone
> >can educate me ?)
>
> RPi doesn't need bits and would be happy with bytes however
> slice_segment data isn't byte aligned at the end so its possible that
> there might be decoders out there that want an accurate length for that.
There are two fields, please don't mix them up:
__u32 bit_size;
__u32 data_bit_offset; (changed to data_byte_offset in this series)
data_bit_offset/data_byte_offset is useful, while bit_size is IMO not. If you
have multiple slices in array, you only need to know start of the slice data
and that offset is always offset from start of the buffer (absolute, it's not
relative to previous slice data).
Best regards,
Jernej
>
> > Would be to let the driver offset inside the the single
> >OUTPUT/bitstream buffer in case this is not automatically found by the
driver
> >(or that no start-code is needed). Is that last bit correct ? If so, should
we
> >change it to an offset rather then a size ? Shall we allow using offesets
inside
> >larger buffer (e.g. to avoid some memory copies) for the Sliced Base cases ?
>
> I use (in the current structure) data_bit_offset to find the start of
> each slice's slice_segment_data within the OUTPUT buffer and bit_size to
> find the end. RPi doesn't / can't parse the slice_header and so wants
> all of that. Decoders that do parse the header might plausably want
> header offsets too and it would facilitate zero copy of the bit buffer.
>
>
> >> Regards
> >>
> >> John Cox
> >>
> >> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> >> > > > ---
> >> > > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
> >> > > > drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 +++
+-------
> >> > > > include/uapi/linux/v4l2-controls.h | 3 +--
> >> > > > 3 files changed, 5 insertions(+), 12 deletions(-)
> >> > > >
> >> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-
codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> >> > > > index 3296ac3b9fca..c3ae97657fa7 100644
> >> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> >> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> >> > > > @@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field
-
> >> > > > :stub-columns: 0
> >> > > > :widths: 1 1 2
> >> > > >
> >> > > > - * - __u32
> >> > > > - - ``bit_size``
> >> > > > - - Size (in bits) of the current slice data.
> >> > > > * - __u32
> >> > > > - ``data_bit_offset``
> >> > > > - Offset (in bits) to the video data in the current slice
data.
> >> > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/
drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> >> > > > index 8ab2d9c6f048..db8c7475eeb8 100644
> >> > > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> >> > > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> >> > > > @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx
*ctx,
> >> > > > const struct v4l2_hevc_pred_weight_table *pred_weight_table;
> >> > > > unsigned int width_in_ctb_luma, ctb_size_luma;
> >> > > > unsigned int log2_max_luma_coding_block_size;
> >> > > > + size_t slice_bytes;
> >> > > > dma_addr_t src_buf_addr;
> >> > > > - dma_addr_t src_buf_end_addr;
> >> > > > u32 chroma_log2_weight_denom;
> >> > > > u32 output_pic_list_index;
> >> > > > u32 pic_order_cnt[2];
> >> > > > @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx
*ctx,
> >> > > >
> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
> >> > > >
> >> > > > - reg = slice_params->bit_size;
> >> > > > - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
> >> > > > + slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
> >> > > > + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
> >> > > I think one of these must be wrong. bit_size is in bits,
> >> > > vb2_get_plane_payload is in bytes?
> >> >
> >> > You are right it should be vb2_get_plane_payload() * 8 to get the size
in bits.
> >> >
> >> > I will change that in v3.
> >> >
> >> > >
> >> > > Regards
> >> > >
> >> > > John Cox
> >> > >
> >> > > > /* Source beginning and end addresses. */
> >> > > >
> >> > > > @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct
cedrus_ctx *ctx,
> >> > > >
> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
> >> > > >
> >> > > > - src_buf_end_addr = src_buf_addr +
> >> > > > - DIV_ROUND_UP(slice_params->bit_size,
8);
> >> > > > -
> >> > > > - reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
> >> > > > + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
> >> > > >
> >> > > > /* Coding tree block address */
> >> > > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/
linux/v4l2-controls.h
> >> > > > index b1a3dc05f02f..27f5d272dc43 100644
> >> > > > --- a/include/uapi/linux/v4l2-controls.h
> >> > > > +++ b/include/uapi/linux/v4l2-controls.h
> >> > > > @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
> >> > > > #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT
(1ULL << 9)
> >> > > >
> >> > > > struct v4l2_ctrl_hevc_slice_params {
> >> > > > - __u32 bit_size;
> >> > > > __u32 data_bit_offset;
> >> > > >
> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
> >> > > > @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message
*/
> >> > > > __u8 pic_struct;
> >> > > >
> >> > > > - __u8 reserved;
> >> > > > + __u8 reserved[5];
> >> > > >
> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment
header */
> >> > > > __u32 slice_segment_addr;
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 0/8] Move HEVC stateless controls out of staging
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (8 preceding siblings ...)
2022-02-15 13:01 ` [RFC v2 0/8] Move HEVC stateless controls out of staging Sebastian Fricke
@ 2022-02-15 16:14 ` Jernej Škrabec
2022-02-18 9:59 ` Hans Verkuil
10 siblings, 0 replies; 35+ messages in thread
From: Jernej Škrabec @ 2022-02-15 16:14 UTC (permalink / raw)
To: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, hverkuil-cisco, jonas, nicolas, Benjamin Gaignard
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc, Benjamin Gaignard
Hi Benjamin!
Dne torek, 15. februar 2022 ob 12:00:55 CET je Benjamin Gaignard napisal(a):
> This series aims to make HEVC uapi stable and usable for hardware
> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
> and 2 out of the tree drivers (rkvdec and RPI).
>
> The 3 first patches are from Hans to implement v4l2 dynamic control
> feature which is need by patch 7 for
V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET
> definition.
Great, I'll test this soon on Cedrus, but I can't promise it will be this
week.
>
> Patch 4 move the existing uapi to stable, including definitions renaming
> and CID number change to fit with v4l2 naming.
This should be last patch in series, otherwise you're changing stable uAPI,
even if only few commits later.
Best regards,
Jernej
>
> Patches 5 and 7 add fields needed for rkvdec and RPI decoders.
>
> Patches 6 is cleaning up the uapi of useless field.
> Patches 8 change one field description and name to define offset by
> bytes rather than by bits
>
> Benjamin
>
> Benjamin Gaignard (5):
> media: uapi: Move HEVC stateless controls out of staging
> media: uapi: Add fields needed for RKVDEC driver
> media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> media: uapi: Change data_bit_offset definition
>
> Hans Verkuil (3):
> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> v4l2-ctrls: add support for dynamically allocated arrays.
> vivid: add dynamic array test control
>
> .../userspace-api/media/drivers/hantro.rst | 5 -
> .../media/v4l/ext-ctrls-codec.rst | 58 ++--
> .../media/v4l/vidioc-queryctrl.rst | 8 +
> .../media/test-drivers/vivid/vivid-ctrls.c | 15 ++
> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++--
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 ++++++++++---
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> .../staging/media/sunxi/cedrus/cedrus_h265.c | 13 +-
> include/linux/hantro-media.h | 17 ++
> include/media/hevc-ctrls.h | 250 ------------------
> include/media/v4l2-ctrls.h | 48 +++-
> include/uapi/linux/v4l2-controls.h | 224 ++++++++++++++++
> include/uapi/linux/videodev2.h | 8 +
> 19 files changed, 640 insertions(+), 408 deletions(-)
> create mode 100644 include/linux/hantro-media.h
> delete mode 100644 include/media/hevc-ctrls.h
>
> --
> 2.32.0
>
>
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 16:11 ` Jernej Škrabec
@ 2022-02-15 16:31 ` John Cox
2022-02-15 20:27 ` Jernej Škrabec
0 siblings, 1 reply; 35+ messages in thread
From: John Cox @ 2022-02-15 16:31 UTC (permalink / raw)
To: Jernej Škrabec
Cc: Nicolas Dufresne, Benjamin Gaignard, mchehab, ezequiel, p.zabel,
gregkh, mripard, paul.kocialkowski, wens, hverkuil-cisco, jonas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
On Tue, 15 Feb 2022 17:11:12 +0100, you wrote:
>Dne torek, 15. februar 2022 ob 17:00:33 CET je John Cox napisal(a):
>> On Tue, 15 Feb 2022 10:28:55 -0500, you wrote:
>>
>> >Le mardi 15 février 2022 à 14:50 +0000, John Cox a écrit :
>> >> On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
>> >>
>> >> >
>> >> > Le 15/02/2022 à 15:17, John Cox a écrit :
>> >> > > Hi
>> >> > >
>> >> > > > The bit size of the slice could be deduced from the buffer payload
>> >> > > > so remove bit_size field to avoid duplicated the information.
>> >> > > I think this is a bad idea. In the future we are (I hope) going to
>want
>> >> > > to have an array (variable) of slice headers all referring to the
>same
>> >> > > bit buffer. When we do that we will need this field.
>> >> >
>> >> > I wonder if that could be considering like another decode mode and so
>> >> > use an other control ?
>> >>
>> >> I, personally, would be in favour of making the slice header control a
>> >> variable array just as it is. If userland can't cope with multiple
>> >> entries then just send them one at a time and the code looks exactly
>> >> like it does at the moment and if the driver can't then set max array
>> >> entries to 1.
>> >>
>> >> Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
>> >> can say with experience that the code and effort overhead is very low.
>> >>
>> >> Either way having a multiple slice header control in the UAPI is
>> >> important for efficiency.
>> >
>> >Just to clarify the idea, we would have a single slice controls, always
>dynamic:
>> >
>> >1. For sliced based decoder
>> >
>> >The dynamic array slice control is implemented by the driver and its size
>must
>> >be 1.
>>
>> Yes
>>
>> >2. For frame based decoder that don't care for slices
>> >
>> >The dynamic array slice controls is not implement. Userland detects that at
>> >runtime, similar to the VP9 compressed headers.
>>
>> If the driver parses all the slice header then that seems plausible
>>
>> >3. For frame based decoders that needs slices (or driver that supports
>offset
>> >and can gain performance with such mode)
>> >
>> >The dynamic array slice controls is implemented, and should contain all the
>> >slices found in the OUTPUT buffer.
>> >
>> >So the reason for this bit_size (not sure why its bits though, perhaps
>someone
>> >can educate me ?)
>>
>> RPi doesn't need bits and would be happy with bytes however
>> slice_segment data isn't byte aligned at the end so its possible that
>> there might be decoders out there that want an accurate length for that.
>
>There are two fields, please don't mix them up:
>
>__u32 bit_size;
>__u32 data_bit_offset; (changed to data_byte_offset in this series)
>
>data_bit_offset/data_byte_offset is useful, while bit_size is IMO not. If you
>have multiple slices in array, you only need to know start of the slice data
>and that offset is always offset from start of the buffer (absolute, it's not
>relative to previous slice data).
No... or at least I think not. RPi needs the start and end of the
slice_segment_data elements of each slice. If slices are arranged in the
buffer with slice_segment_headers attached then I don't see how I get to
know that. Also if the OUTPUT buffer is just a bit of bitstream, which
might well be very convienient for some userspace, then it is legitimate
to have SEIs between slice headers so you can't even guarantee that your
coded slice segments are contiguous.
Regards
JC
>Best regards,
>Jernej
>
>>
>> > Would be to let the driver offset inside the the single
>> >OUTPUT/bitstream buffer in case this is not automatically found by the
>driver
>> >(or that no start-code is needed). Is that last bit correct ? If so, should
>we
>> >change it to an offset rather then a size ? Shall we allow using offesets
>inside
>> >larger buffer (e.g. to avoid some memory copies) for the Sliced Base cases ?
>>
>> I use (in the current structure) data_bit_offset to find the start of
>> each slice's slice_segment_data within the OUTPUT buffer and bit_size to
>> find the end. RPi doesn't / can't parse the slice_header and so wants
>> all of that. Decoders that do parse the header might plausably want
>> header offsets too and it would facilitate zero copy of the bit buffer.
>>
>>
>> >> Regards
>> >>
>> >> John Cox
>> >>
>> >> > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> >> > > > ---
>> >> > > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
>> >> > > > drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 +++
>+-------
>> >> > > > include/uapi/linux/v4l2-controls.h | 3 +--
>> >> > > > 3 files changed, 5 insertions(+), 12 deletions(-)
>> >> > > >
>> >> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-
>codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> >> > > > index 3296ac3b9fca..c3ae97657fa7 100644
>> >> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> >> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> >> > > > @@ -2965,9 +2965,6 @@ enum v4l2_mpeg_video_hevc_size_of_length_field
>-
>> >> > > > :stub-columns: 0
>> >> > > > :widths: 1 1 2
>> >> > > >
>> >> > > > - * - __u32
>> >> > > > - - ``bit_size``
>> >> > > > - - Size (in bits) of the current slice data.
>> >> > > > * - __u32
>> >> > > > - ``data_bit_offset``
>> >> > > > - Offset (in bits) to the video data in the current slice
>data.
>> >> > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/
>drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> >> > > > index 8ab2d9c6f048..db8c7475eeb8 100644
>> >> > > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> >> > > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> >> > > > @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct cedrus_ctx
>*ctx,
>> >> > > > const struct v4l2_hevc_pred_weight_table *pred_weight_table;
>> >> > > > unsigned int width_in_ctb_luma, ctb_size_luma;
>> >> > > > unsigned int log2_max_luma_coding_block_size;
>> >> > > > + size_t slice_bytes;
>> >> > > > dma_addr_t src_buf_addr;
>> >> > > > - dma_addr_t src_buf_end_addr;
>> >> > > > u32 chroma_log2_weight_denom;
>> >> > > > u32 output_pic_list_index;
>> >> > > > u32 pic_order_cnt[2];
>> >> > > > @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct cedrus_ctx
>*ctx,
>> >> > > >
>> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
>> >> > > >
>> >> > > > - reg = slice_params->bit_size;
>> >> > > > - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
>> >> > > > + slice_bytes = vb2_get_plane_payload(&run->src->vb2_buf, 0);
>> >> > > > + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
>> >> > > I think one of these must be wrong. bit_size is in bits,
>> >> > > vb2_get_plane_payload is in bytes?
>> >> >
>> >> > You are right it should be vb2_get_plane_payload() * 8 to get the size
>in bits.
>> >> >
>> >> > I will change that in v3.
>> >> >
>> >> > >
>> >> > > Regards
>> >> > >
>> >> > > John Cox
>> >> > >
>> >> > > > /* Source beginning and end addresses. */
>> >> > > >
>> >> > > > @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct
>cedrus_ctx *ctx,
>> >> > > >
>> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
>> >> > > >
>> >> > > > - src_buf_end_addr = src_buf_addr +
>> >> > > > - DIV_ROUND_UP(slice_params->bit_size,
>8);
>> >> > > > -
>> >> > > > - reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
>> >> > > > + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr + slice_bytes);
>> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
>> >> > > >
>> >> > > > /* Coding tree block address */
>> >> > > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/
>linux/v4l2-controls.h
>> >> > > > index b1a3dc05f02f..27f5d272dc43 100644
>> >> > > > --- a/include/uapi/linux/v4l2-controls.h
>> >> > > > +++ b/include/uapi/linux/v4l2-controls.h
>> >> > > > @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
>> >> > > > #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT
>(1ULL << 9)
>> >> > > >
>> >> > > > struct v4l2_ctrl_hevc_slice_params {
>> >> > > > - __u32 bit_size;
>> >> > > > __u32 data_bit_offset;
>> >> > > >
>> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>> >> > > > @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
>> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message
>*/
>> >> > > > __u8 pic_struct;
>> >> > > >
>> >> > > > - __u8 reserved;
>> >> > > > + __u8 reserved[5];
>> >> > > >
>> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment
>header */
>> >> > > > __u32 slice_segment_addr;
>>
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging
2022-02-15 11:00 ` [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging Benjamin Gaignard
@ 2022-02-15 18:02 ` Sebastian Fricke
2022-02-18 11:50 ` Hans Verkuil
1 sibling, 0 replies; 35+ messages in thread
From: Sebastian Fricke @ 2022-02-15 18:02 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Hey Benjamin,
On 15.02.2022 12:00, Benjamin Gaignard wrote:
>The HEVC stateless 'uAPI' was staging and marked explicitly in the
s/was staging/was in staging/
>V4L2 specification that it will change and is unstable.
>
>Note that these control IDs were never exported as a public API,
>they were only defined in kernel-local headers (hevc-ctrls.h).
>
>While moving the controls out of staging they are renamed and
>control IDs get new numbers.
>Drivers (Hantro, Cedrus) and Documentation are updated accordingly.
>
>Hantro dedicated control is moving to hantro-media.h
s/Hantro dedicated control is moving/Hantro dedicated controls are moved/
>Since hevc-ctrls.h content has been dispatched in others file, remove it.
s/in others file/in another file/
or
s/in others file/in other files/
Greetings,
Sebastian
>
>fluster tests results on IMX8MQ is 77/147 for HEVC codec.
>
>Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>---
>version 2 :
>- Do not include rkvdec needed fields in this patch
>- Remove 'non-public' notice in hantro.rst
>- Take care of cache-line boundary
>
> .../userspace-api/media/drivers/hantro.rst | 5 -
> .../media/v4l/ext-ctrls-codec.rst | 26 +-
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> include/linux/hantro-media.h | 17 ++
> include/media/hevc-ctrls.h | 250 ------------------
> include/media/v4l2-ctrls.h | 6 -
> include/uapi/linux/v4l2-controls.h | 219 +++++++++++++++
> include/uapi/linux/videodev2.h | 7 +
> 12 files changed, 307 insertions(+), 324 deletions(-)
> create mode 100644 include/linux/hantro-media.h
> delete mode 100644 include/media/hevc-ctrls.h
>
>diff --git a/Documentation/userspace-api/media/drivers/hantro.rst b/Documentation/userspace-api/media/drivers/hantro.rst
>index cd9754b4e005..78dcd2a44a03 100644
>--- a/Documentation/userspace-api/media/drivers/hantro.rst
>+++ b/Documentation/userspace-api/media/drivers/hantro.rst
>@@ -12,8 +12,3 @@ The Hantro video decoder driver implements the following driver-specific control
> to before syntax element "slice_temporal_mvp_enabled_flag".
> If IDR, the skipped bits are just "pic_output_flag"
> (separate_colour_plane_flag is not supported).
>-
>-.. note::
>-
>- This control is not yet part of the public kernel API and
>- it is expected to change.
>diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>index 4cd7c541fc30..4f3b3ba8319f 100644
>--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>@@ -2639,7 +2639,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> .. _v4l2-mpeg-hevc:
>
>-``V4L2_CID_MPEG_VIDEO_HEVC_SPS (struct)``
>+``V4L2_CID_STATELESS_HEVC_SPS (struct)``
> Specifies the Sequence Parameter Set fields (as extracted from the
> bitstream) for the associated HEVC slice data.
> These bitstream parameters are defined according to :ref:`hevc`.
>@@ -2782,7 +2782,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
>-``V4L2_CID_MPEG_VIDEO_HEVC_PPS (struct)``
>+``V4L2_CID_STATELESS_HEVC_PPS (struct)``
> Specifies the Picture Parameter Set fields (as extracted from the
> bitstream) for the associated HEVC slice data.
> These bitstream parameters are defined according to :ref:`hevc`.
>@@ -2932,7 +2932,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
>-``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
>+``V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (struct)``
> Specifies various slice-specific parameters, especially from the NAL unit
> header, general slice segment header and weighted prediction parameter
> parts of the bitstream.
>@@ -3088,7 +3088,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
>-``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
>+``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
> Specifies the HEVC scaling matrix parameters used for the scaling process
> for transform coefficients.
> These matrix and parameters are defined according to :ref:`hevc`.
>@@ -3238,7 +3238,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
>-``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (enum)``
>+``V4L2_CID_STATELESS_HEVC_DECODE_MODE (enum)``
> Specifies the decoding mode to use. Currently exposes slice-based and
> frame-based decoding but new modes might be added later on.
> This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
>@@ -3253,7 +3253,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This menu control is not yet part of the public kernel API and
> it is expected to change.
>
>-.. c:type:: v4l2_mpeg_video_hevc_decode_mode
>+.. c:type:: v4l2_stateless_hevc_decode_mode
>
> .. raw:: latex
>
>@@ -3266,11 +3266,11 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
>- * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED``
>+ * - ``V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED``
> - 0
> - Decoding is done at the slice granularity.
> The OUTPUT buffer must contain a single slice.
>- * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED``
>+ * - ``V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED``
> - 1
> - Decoding is done at the frame granularity.
> The OUTPUT buffer must contain all slices needed to decode the
>@@ -3280,7 +3280,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
>-``V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (enum)``
>+``V4L2_CID_STATELESS_HEVC_START_CODE (enum)``
> Specifies the HEVC slice start code expected for each slice.
> This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
> pixel format. Applications that support V4L2_PIX_FMT_HEVC_SLICE
>@@ -3294,7 +3294,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This menu control is not yet part of the public kernel API and
> it is expected to change.
>
>-.. c:type:: v4l2_mpeg_video_hevc_start_code
>+.. c:type:: v4l2_stateless_hevc_start_code
>
> .. tabularcolumns:: |p{9.2cm}|p{0.6cm}|p{7.5cm}|
>
>@@ -3303,13 +3303,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
>- * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE``
>+ * - ``V4L2_STATELESS_HEVC_START_CODE_NONE``
> - 0
> - Selecting this value specifies that HEVC slices are passed
> to the driver without any start code. The bitstream data should be
> according to :ref:`hevc` 7.3.1.1 General NAL unit syntax, hence
> contains emulation prevention bytes when required.
>- * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B``
>+ * - ``V4L2_STATELESS_HEVC_START_CODE_ANNEX_B``
> - 1
> - Selecting this value specifies that HEVC slices are expected
> to be prefixed by Annex B start codes. According to :ref:`hevc`
>@@ -3342,7 +3342,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This provides a bitmask which consists of bits [0, LTR_COUNT-1].
> This is applicable to the H264 and HEVC encoders.
>
>-``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
>+``V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (struct)``
> Specifies various decode parameters, especially the references picture order
> count (POC) for all the lists (short, long, before, current, after) and the
> number of entries for each of them.
>diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>index 54ca4e6b820b..4b68cbe23309 100644
>--- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>+++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
>@@ -699,9 +699,9 @@ const char * const *v4l2_ctrl_get_menu(u32 id)
> return hevc_tier;
> case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
> return hevc_loop_filter_mode;
>- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
>+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
> return hevc_decode_mode;
>- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
>+ case V4L2_CID_STATELESS_HEVC_START_CODE:
> return hevc_start_code;
> case V4L2_CID_CAMERA_ORIENTATION:
> return camera_orientation;
>@@ -995,13 +995,6 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD: return "HEVC Size of Length Field";
> case V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES: return "Reference Frames for a P-Frame";
> case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR: return "Prepend SPS and PPS to IDR";
>- case V4L2_CID_MPEG_VIDEO_HEVC_SPS: return "HEVC Sequence Parameter Set";
>- case V4L2_CID_MPEG_VIDEO_HEVC_PPS: return "HEVC Picture Parameter Set";
>- case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
>- case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
>- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
>- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC Decode Mode";
>- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC Start Code";
>
> /* CAMERA controls */
> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
>@@ -1180,6 +1173,13 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_STATELESS_MPEG2_QUANTISATION: return "MPEG-2 Quantisation Matrices";
> case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR: return "VP9 Probabilities Updates";
> case V4L2_CID_STATELESS_VP9_FRAME: return "VP9 Frame Decode Parameters";
>+ case V4L2_CID_STATELESS_HEVC_SPS: return "HEVC Sequence Parameter Set";
>+ case V4L2_CID_STATELESS_HEVC_PPS: return "HEVC Picture Parameter Set";
>+ case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
>+ case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
>+ case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
>+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE: return "HEVC Decode Mode";
>+ case V4L2_CID_STATELESS_HEVC_START_CODE: return "HEVC Start Code";
>
> /* Colorimetry controls */
> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
>@@ -1355,8 +1355,8 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD:
> case V4L2_CID_MPEG_VIDEO_HEVC_TIER:
> case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
>- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
>- case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
>+ case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
>+ case V4L2_CID_STATELESS_HEVC_START_CODE:
> case V4L2_CID_STATELESS_H264_DECODE_MODE:
> case V4L2_CID_STATELESS_H264_START_CODE:
> case V4L2_CID_CAMERA_ORIENTATION:
>@@ -1493,19 +1493,19 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_STATELESS_VP8_FRAME:
> *type = V4L2_CTRL_TYPE_VP8_FRAME;
> break;
>- case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
>+ case V4L2_CID_STATELESS_HEVC_SPS:
> *type = V4L2_CTRL_TYPE_HEVC_SPS;
> break;
>- case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
>+ case V4L2_CID_STATELESS_HEVC_PPS:
> *type = V4L2_CTRL_TYPE_HEVC_PPS;
> break;
>- case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
>+ case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
> break;
>- case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
>+ case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX:
> *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
> break;
>- case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
>+ case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
> break;
> case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
>diff --git a/drivers/staging/media/hantro/hantro_drv.c b/drivers/staging/media/hantro/hantro_drv.c
>index dc768884cb79..783a92a38c8e 100644
>--- a/drivers/staging/media/hantro/hantro_drv.c
>+++ b/drivers/staging/media/hantro/hantro_drv.c
>@@ -11,6 +11,7 @@
> */
>
> #include <linux/clk.h>
>+#include <linux/hantro-media.h>
> #include <linux/module.h>
> #include <linux/of.h>
> #include <linux/platform_device.h>
>@@ -255,7 +256,7 @@ static int hantro_try_ctrl(struct v4l2_ctrl *ctrl)
> if (sps->bit_depth_luma_minus8 != 0)
> /* Only 8-bit is supported */
> return -EINVAL;
>- } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
>+ } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
> const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
>
> if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8)
>@@ -428,18 +429,18 @@ static const struct hantro_ctrl controls[] = {
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
>- .min = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
>- .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
>- .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
>+ .id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
>+ .min = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
>+ .max = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
>+ .def = V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
>- .min = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
>- .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
>- .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
>+ .id = V4L2_CID_STATELESS_HEVC_START_CODE,
>+ .min = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
>+ .max = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
>+ .def = V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
>@@ -459,23 +460,23 @@ static const struct hantro_ctrl controls[] = {
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
>+ .id = V4L2_CID_STATELESS_HEVC_SPS,
> .ops = &hantro_ctrl_ops,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
>+ .id = V4L2_CID_STATELESS_HEVC_PPS,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
>+ .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
>+ .id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
> },
> }, {
> .codec = HANTRO_HEVC_DECODER,
>diff --git a/drivers/staging/media/hantro/hantro_hevc.c b/drivers/staging/media/hantro/hantro_hevc.c
>index b49a41d7ae91..b6ec86d03d91 100644
>--- a/drivers/staging/media/hantro/hantro_hevc.c
>+++ b/drivers/staging/media/hantro/hantro_hevc.c
>@@ -201,22 +201,22 @@ int hantro_hevc_dec_prepare_run(struct hantro_ctx *ctx)
> hantro_start_prepare_run(ctx);
>
> ctrls->decode_params =
>- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
>+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
> if (WARN_ON(!ctrls->decode_params))
> return -EINVAL;
>
> ctrls->scaling =
>- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
>+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
> if (WARN_ON(!ctrls->scaling))
> return -EINVAL;
>
> ctrls->sps =
>- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_SPS);
>+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_SPS);
> if (WARN_ON(!ctrls->sps))
> return -EINVAL;
>
> ctrls->pps =
>- hantro_get_ctrl(ctx, V4L2_CID_MPEG_VIDEO_HEVC_PPS);
>+ hantro_get_ctrl(ctx, V4L2_CID_STATELESS_HEVC_PPS);
> if (WARN_ON(!ctrls->pps))
> return -EINVAL;
>
>diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c
>index 4a4b714b0f26..e0428163f82c 100644
>--- a/drivers/staging/media/sunxi/cedrus/cedrus.c
>+++ b/drivers/staging/media/sunxi/cedrus/cedrus.c
>@@ -42,7 +42,7 @@ static int cedrus_try_ctrl(struct v4l2_ctrl *ctrl)
> if (sps->bit_depth_luma_minus8 != 0)
> /* Only 8-bit is supported */
> return -EINVAL;
>- } else if (ctrl->id == V4L2_CID_MPEG_VIDEO_HEVC_SPS) {
>+ } else if (ctrl->id == V4L2_CID_STATELESS_HEVC_SPS) {
> const struct v4l2_ctrl_hevc_sps *sps = ctrl->p_new.p_hevc_sps;
> struct cedrus_ctx *ctx = container_of(ctrl->handler, struct cedrus_ctx, hdl);
>
>@@ -164,42 +164,42 @@ static const struct cedrus_control cedrus_controls[] = {
> },
> {
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_SPS,
>+ .id = V4L2_CID_STATELESS_HEVC_SPS,
> .ops = &cedrus_ctrl_ops,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_PPS,
>+ .id = V4L2_CID_STATELESS_HEVC_PPS,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS,
>+ .id = V4L2_CID_STATELESS_HEVC_SLICE_PARAMS,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX,
>+ .id = V4L2_CID_STATELESS_HEVC_SCALING_MATRIX,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE,
>- .max = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
>- .def = V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
>+ .id = V4L2_CID_STATELESS_HEVC_DECODE_MODE,
>+ .max = V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
>+ .def = V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
> },
> .codec = CEDRUS_CODEC_H265,
> },
> {
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_START_CODE,
>- .max = V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
>- .def = V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
>+ .id = V4L2_CID_STATELESS_HEVC_START_CODE,
>+ .max = V4L2_STATELESS_HEVC_START_CODE_NONE,
>+ .def = V4L2_STATELESS_HEVC_START_CODE_NONE,
> },
> .codec = CEDRUS_CODEC_H265,
> },
>@@ -211,7 +211,7 @@ static const struct cedrus_control cedrus_controls[] = {
> },
> {
> .cfg = {
>- .id = V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS,
>+ .id = V4L2_CID_STATELESS_HEVC_DECODE_PARAMS,
> },
> .codec = CEDRUS_CODEC_H265,
> },
>diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
>index a16c1422558f..f6be4ae72ee2 100644
>--- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
>+++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c
>@@ -65,15 +65,15 @@ void cedrus_device_run(void *priv)
>
> case V4L2_PIX_FMT_HEVC_SLICE:
> run.h265.sps = cedrus_find_control_data(ctx,
>- V4L2_CID_MPEG_VIDEO_HEVC_SPS);
>+ V4L2_CID_STATELESS_HEVC_SPS);
> run.h265.pps = cedrus_find_control_data(ctx,
>- V4L2_CID_MPEG_VIDEO_HEVC_PPS);
>+ V4L2_CID_STATELESS_HEVC_PPS);
> run.h265.slice_params = cedrus_find_control_data(ctx,
>- V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS);
>+ V4L2_CID_STATELESS_HEVC_SLICE_PARAMS);
> run.h265.decode_params = cedrus_find_control_data(ctx,
>- V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS);
>+ V4L2_CID_STATELESS_HEVC_DECODE_PARAMS);
> run.h265.scaling_matrix = cedrus_find_control_data(ctx,
>- V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX);
>+ V4L2_CID_STATELESS_HEVC_SCALING_MATRIX);
> break;
>
> case V4L2_PIX_FMT_VP8_FRAME:
>diff --git a/include/linux/hantro-media.h b/include/linux/hantro-media.h
>new file mode 100644
>index 000000000000..db2791ab7c3b
>--- /dev/null
>+++ b/include/linux/hantro-media.h
>@@ -0,0 +1,17 @@
>+/* SPDX-License-Identifier: GPL-2.0-or-later */
>+
>+#ifndef __LINUX_HANTRO_MEDIA_H__
>+#define __LINUX_HANTRO_MEDIA_H__
>+
>+/*
>+ * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
>+ * the number of data (in bits) to skip in the
>+ * slice segment header.
>+ * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
>+ * to before syntax element "slice_temporal_mvp_enabled_flag".
>+ * If IDR, the skipped bits are just "pic_output_flag"
>+ * (separate_colour_plane_flag is not supported).
>+ */
>+#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_USER_HANTRO_BASE + 0)
>+
>+#endif
>diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
>deleted file mode 100644
>index 01ccda48d8c5..000000000000
>--- a/include/media/hevc-ctrls.h
>+++ /dev/null
>@@ -1,250 +0,0 @@
>-/* SPDX-License-Identifier: GPL-2.0 */
>-/*
>- * These are the HEVC state controls for use with stateless HEVC
>- * codec drivers.
>- *
>- * It turns out that these structs are not stable yet and will undergo
>- * more changes. So keep them private until they are stable and ready to
>- * become part of the official public API.
>- */
>-
>-#ifndef _HEVC_CTRLS_H_
>-#define _HEVC_CTRLS_H_
>-
>-#include <linux/videodev2.h>
>-
>-/* The pixel format isn't stable at the moment and will likely be renamed. */
>-#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
>-
>-#define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008)
>-#define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009)
>-#define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010)
>-#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE + 1011)
>-#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012)
>-#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015)
>-#define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
>-
>-/* enum v4l2_ctrl_type type values */
>-#define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
>-#define V4L2_CTRL_TYPE_HEVC_PPS 0x0121
>-#define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122
>-#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
>-#define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
>-
>-enum v4l2_mpeg_video_hevc_decode_mode {
>- V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
>- V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
>-};
>-
>-enum v4l2_mpeg_video_hevc_start_code {
>- V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
>- V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
>-};
>-
>-#define V4L2_HEVC_SLICE_TYPE_B 0
>-#define V4L2_HEVC_SLICE_TYPE_P 1
>-#define V4L2_HEVC_SLICE_TYPE_I 2
>-
>-#define V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE (1ULL << 0)
>-#define V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED (1ULL << 1)
>-#define V4L2_HEVC_SPS_FLAG_AMP_ENABLED (1ULL << 2)
>-#define V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET (1ULL << 3)
>-#define V4L2_HEVC_SPS_FLAG_PCM_ENABLED (1ULL << 4)
>-#define V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED (1ULL << 5)
>-#define V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT (1ULL << 6)
>-#define V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED (1ULL << 7)
>-#define V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED (1ULL << 8)
>-
>-/* The controls are not stable at the moment and will likely be reworked. */
>-struct v4l2_ctrl_hevc_sps {
>- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
>- __u16 pic_width_in_luma_samples;
>- __u16 pic_height_in_luma_samples;
>- __u8 bit_depth_luma_minus8;
>- __u8 bit_depth_chroma_minus8;
>- __u8 log2_max_pic_order_cnt_lsb_minus4;
>- __u8 sps_max_dec_pic_buffering_minus1;
>- __u8 sps_max_num_reorder_pics;
>- __u8 sps_max_latency_increase_plus1;
>- __u8 log2_min_luma_coding_block_size_minus3;
>- __u8 log2_diff_max_min_luma_coding_block_size;
>- __u8 log2_min_luma_transform_block_size_minus2;
>- __u8 log2_diff_max_min_luma_transform_block_size;
>- __u8 max_transform_hierarchy_depth_inter;
>- __u8 max_transform_hierarchy_depth_intra;
>- __u8 pcm_sample_bit_depth_luma_minus1;
>- __u8 pcm_sample_bit_depth_chroma_minus1;
>- __u8 log2_min_pcm_luma_coding_block_size_minus3;
>- __u8 log2_diff_max_min_pcm_luma_coding_block_size;
>- __u8 num_short_term_ref_pic_sets;
>- __u8 num_long_term_ref_pics_sps;
>- __u8 chroma_format_idc;
>- __u8 sps_max_sub_layers_minus1;
>-
>- __u64 flags;
>-};
>-
>-#define V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED (1ULL << 0)
>-#define V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT (1ULL << 1)
>-#define V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED (1ULL << 2)
>-#define V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT (1ULL << 3)
>-#define V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED (1ULL << 4)
>-#define V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED (1ULL << 5)
>-#define V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED (1ULL << 6)
>-#define V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT (1ULL << 7)
>-#define V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED (1ULL << 8)
>-#define V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED (1ULL << 9)
>-#define V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED (1ULL << 10)
>-#define V4L2_HEVC_PPS_FLAG_TILES_ENABLED (1ULL << 11)
>-#define V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED (1ULL << 12)
>-#define V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED (1ULL << 13)
>-#define V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 14)
>-#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED (1ULL << 15)
>-#define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER (1ULL << 16)
>-#define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT (1ULL << 17)
>-#define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL << 18)
>-#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
>-#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
>-
>-struct v4l2_ctrl_hevc_pps {
>- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
>- __u8 num_extra_slice_header_bits;
>- __u8 num_ref_idx_l0_default_active_minus1;
>- __u8 num_ref_idx_l1_default_active_minus1;
>- __s8 init_qp_minus26;
>- __u8 diff_cu_qp_delta_depth;
>- __s8 pps_cb_qp_offset;
>- __s8 pps_cr_qp_offset;
>- __u8 num_tile_columns_minus1;
>- __u8 num_tile_rows_minus1;
>- __u8 column_width_minus1[20];
>- __u8 row_height_minus1[22];
>- __s8 pps_beta_offset_div2;
>- __s8 pps_tc_offset_div2;
>- __u8 log2_parallel_merge_level_minus2;
>-
>- __u8 padding[4];
>- __u64 flags;
>-};
>-
>-#define V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE 0x01
>-
>-#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX 16
>-
>-struct v4l2_hevc_dpb_entry {
>- __u64 timestamp;
>- __u8 flags;
>- __u8 field_pic;
>- __u16 pic_order_cnt[2];
>- __u8 padding[2];
>-};
>-
>-struct v4l2_hevc_pred_weight_table {
>- __s8 delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __s8 luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __s8 delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>- __s8 chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>-
>- __s8 delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __s8 luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __s8 delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>- __s8 chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>-
>- __u8 padding[6];
>-
>- __u8 luma_log2_weight_denom;
>- __s8 delta_chroma_log2_weight_denom;
>-};
>-
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_LUMA (1ULL << 0)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_CHROMA (1ULL << 1)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED (1ULL << 2)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_MVD_L1_ZERO (1ULL << 3)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_CABAC_INIT (1ULL << 4)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_COLLOCATED_FROM_L0 (1ULL << 5)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_USE_INTEGER_MV (1ULL << 6)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED (1ULL << 7)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 8)
>-#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
>-
>-struct v4l2_ctrl_hevc_slice_params {
>- __u32 bit_size;
>- __u32 data_bit_offset;
>-
>- /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>- __u8 nal_unit_type;
>- __u8 nuh_temporal_id_plus1;
>-
>- /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>- __u8 slice_type;
>- __u8 colour_plane_id;
>- __u16 slice_pic_order_cnt;
>- __u8 num_ref_idx_l0_active_minus1;
>- __u8 num_ref_idx_l1_active_minus1;
>- __u8 collocated_ref_idx;
>- __u8 five_minus_max_num_merge_cand;
>- __s8 slice_qp_delta;
>- __s8 slice_cb_qp_offset;
>- __s8 slice_cr_qp_offset;
>- __s8 slice_act_y_qp_offset;
>- __s8 slice_act_cb_qp_offset;
>- __s8 slice_act_cr_qp_offset;
>- __s8 slice_beta_offset_div2;
>- __s8 slice_tc_offset_div2;
>-
>- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
>- __u8 pic_struct;
>-
>- /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>- __u32 slice_segment_addr;
>- __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>-
>- __u8 padding;
>-
>- /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
>- struct v4l2_hevc_pred_weight_table pred_weight_table;
>-
>- __u64 flags;
>-};
>-
>-#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC 0x1
>-#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC 0x2
>-#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR 0x4
>-
>-struct v4l2_ctrl_hevc_decode_params {
>- __s32 pic_order_cnt_val;
>- __u8 num_active_dpb_entries;
>- struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __u8 num_poc_st_curr_before;
>- __u8 num_poc_st_curr_after;
>- __u8 num_poc_lt_curr;
>- __u8 poc_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __u8 poc_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __u8 poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>- __u64 flags;
>-};
>-
>-struct v4l2_ctrl_hevc_scaling_matrix {
>- __u8 scaling_list_4x4[6][16];
>- __u8 scaling_list_8x8[6][64];
>- __u8 scaling_list_16x16[6][64];
>- __u8 scaling_list_32x32[2][64];
>- __u8 scaling_list_dc_coef_16x16[6];
>- __u8 scaling_list_dc_coef_32x32[2];
>-};
>-
>-/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */
>-#define V4L2_CID_CODEC_HANTRO_BASE (V4L2_CTRL_CLASS_CODEC | 0x1200)
>-/*
>- * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
>- * the number of data (in bits) to skip in the
>- * slice segment header.
>- * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
>- * to before syntax element "slice_temporal_mvp_enabled_flag".
>- * If IDR, the skipped bits are just "pic_output_flag"
>- * (separate_colour_plane_flag is not supported).
>- */
>-#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_CODEC_HANTRO_BASE + 0)
>-
>-#endif
>diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
>index f4105de8a8d2..00828a4f9404 100644
>--- a/include/media/v4l2-ctrls.h
>+++ b/include/media/v4l2-ctrls.h
>@@ -13,12 +13,6 @@
> #include <linux/videodev2.h>
> #include <media/media-request.h>
>
>-/*
>- * Include the stateless codec compound control definitions.
>- * This will move to the public headers once this API is fully stable.
>- */
>-#include <media/hevc-ctrls.h>
>-
> /* forward references */
> struct file;
> struct poll_table_struct;
>diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>index c8e0f84d204d..0e0ec2c61b80 100644
>--- a/include/uapi/linux/v4l2-controls.h
>+++ b/include/uapi/linux/v4l2-controls.h
>@@ -218,6 +218,11 @@ enum v4l2_colorfx {
> * We reserve 16 controls for this driver.
> */
> #define V4L2_CID_USER_ALLEGRO_BASE (V4L2_CID_USER_BASE + 0x1170)
>+/*
>+ * The base for Hantro driver controls.
>+ * We reserve 128 controls for this driver.
>+ */
>+#define V4L2_CID_USER_HANTRO_BASE (V4L2_CID_USER_BASE + 0x1180)
>
> /* MPEG-class control IDs */
> /* The MPEG controls are applicable to all codec controls
>@@ -2302,6 +2307,220 @@ struct v4l2_ctrl_vp9_compressed_hdr {
> struct v4l2_vp9_mv_probs mv;
> };
>
>+#define V4L2_CID_STATELESS_HEVC_SPS (V4L2_CID_CODEC_STATELESS_BASE + 400)
>+#define V4L2_CID_STATELESS_HEVC_PPS (V4L2_CID_CODEC_STATELESS_BASE + 401)
>+#define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 402)
>+#define V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_STATELESS_BASE + 403)
>+#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404)
>+#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405)
>+#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406)
>+
>+enum v4l2_stateless_hevc_decode_mode {
>+ V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
>+ V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
>+};
>+
>+enum v4l2_stateless_hevc_start_code {
>+ V4L2_STATELESS_HEVC_START_CODE_NONE,
>+ V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
>+};
>+
>+#define V4L2_HEVC_SLICE_TYPE_B 0
>+#define V4L2_HEVC_SLICE_TYPE_P 1
>+#define V4L2_HEVC_SLICE_TYPE_I 2
>+
>+#define V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE (1ULL << 0)
>+#define V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED (1ULL << 1)
>+#define V4L2_HEVC_SPS_FLAG_AMP_ENABLED (1ULL << 2)
>+#define V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET (1ULL << 3)
>+#define V4L2_HEVC_SPS_FLAG_PCM_ENABLED (1ULL << 4)
>+#define V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED (1ULL << 5)
>+#define V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT (1ULL << 6)
>+#define V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED (1ULL << 7)
>+#define V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED (1ULL << 8)
>+
>+struct v4l2_ctrl_hevc_sps {
>+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
>+ __u16 pic_width_in_luma_samples;
>+ __u16 pic_height_in_luma_samples;
>+ __u8 bit_depth_luma_minus8;
>+ __u8 bit_depth_chroma_minus8;
>+ __u8 log2_max_pic_order_cnt_lsb_minus4;
>+ __u8 sps_max_dec_pic_buffering_minus1;
>+ __u8 sps_max_num_reorder_pics;
>+ __u8 sps_max_latency_increase_plus1;
>+ __u8 log2_min_luma_coding_block_size_minus3;
>+ __u8 log2_diff_max_min_luma_coding_block_size;
>+ __u8 log2_min_luma_transform_block_size_minus2;
>+ __u8 log2_diff_max_min_luma_transform_block_size;
>+ __u8 max_transform_hierarchy_depth_inter;
>+ __u8 max_transform_hierarchy_depth_intra;
>+ __u8 pcm_sample_bit_depth_luma_minus1;
>+ __u8 pcm_sample_bit_depth_chroma_minus1;
>+ __u8 log2_min_pcm_luma_coding_block_size_minus3;
>+ __u8 log2_diff_max_min_pcm_luma_coding_block_size;
>+ __u8 num_short_term_ref_pic_sets;
>+ __u8 num_long_term_ref_pics_sps;
>+ __u8 chroma_format_idc;
>+ __u8 sps_max_sub_layers_minus1;
>+
>+ __u8 padding[6];
>+ __u64 flags;
>+ __u8 reserved[24];
>+};
>+
>+#define V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED (1ULL << 0)
>+#define V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT (1ULL << 1)
>+#define V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED (1ULL << 2)
>+#define V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT (1ULL << 3)
>+#define V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED (1ULL << 4)
>+#define V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED (1ULL << 5)
>+#define V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED (1ULL << 6)
>+#define V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT (1ULL << 7)
>+#define V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED (1ULL << 8)
>+#define V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED (1ULL << 9)
>+#define V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED (1ULL << 10)
>+#define V4L2_HEVC_PPS_FLAG_TILES_ENABLED (1ULL << 11)
>+#define V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED (1ULL << 12)
>+#define V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED (1ULL << 13)
>+#define V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 14)
>+#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED (1ULL << 15)
>+#define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER (1ULL << 16)
>+#define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT (1ULL << 17)
>+#define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL << 18)
>+#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
>+#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
>+
>+struct v4l2_ctrl_hevc_pps {
>+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
>+ __u8 num_extra_slice_header_bits;
>+ __u8 num_ref_idx_l0_default_active_minus1;
>+ __u8 num_ref_idx_l1_default_active_minus1;
>+ __s8 init_qp_minus26;
>+ __u8 diff_cu_qp_delta_depth;
>+ __s8 pps_cb_qp_offset;
>+ __s8 pps_cr_qp_offset;
>+ __u8 num_tile_columns_minus1;
>+ __u8 num_tile_rows_minus1;
>+ __u8 column_width_minus1[20];
>+ __u8 row_height_minus1[22];
>+ __s8 pps_beta_offset_div2;
>+ __s8 pps_tc_offset_div2;
>+ __u8 log2_parallel_merge_level_minus2;
>+ __u8 padding[9];
>+
>+ __u64 flags;
>+ __u8 reserved[56];
>+};
>+
>+#define V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE 0x01
>+
>+#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX 16
>+
>+struct v4l2_hevc_dpb_entry {
>+ __u64 timestamp;
>+ __u8 flags;
>+ __u8 field_pic;
>+ __u16 pic_order_cnt[2];
>+ __u8 padding[2];
>+};
>+
>+struct v4l2_hevc_pred_weight_table {
>+ __s8 delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __s8 luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __s8 delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>+ __s8 chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>+
>+ __s8 delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __s8 luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __s8 delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>+ __s8 chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
>+
>+ __u8 padding[6];
>+
>+ __u8 luma_log2_weight_denom;
>+ __s8 delta_chroma_log2_weight_denom;
>+};
>+
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_LUMA (1ULL << 0)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_CHROMA (1ULL << 1)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED (1ULL << 2)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_MVD_L1_ZERO (1ULL << 3)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_CABAC_INIT (1ULL << 4)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_COLLOCATED_FROM_L0 (1ULL << 5)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_USE_INTEGER_MV (1ULL << 6)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED (1ULL << 7)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 8)
>+#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
>+
>+struct v4l2_ctrl_hevc_slice_params {
>+ __u32 bit_size;
>+ __u32 data_bit_offset;
>+
>+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
>+ __u8 nal_unit_type;
>+ __u8 nuh_temporal_id_plus1;
>+
>+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>+ __u8 slice_type;
>+ __u8 colour_plane_id;
>+ __u16 slice_pic_order_cnt;
>+ __u8 num_ref_idx_l0_active_minus1;
>+ __u8 num_ref_idx_l1_active_minus1;
>+ __u8 collocated_ref_idx;
>+ __u8 five_minus_max_num_merge_cand;
>+ __s8 slice_qp_delta;
>+ __s8 slice_cb_qp_offset;
>+ __s8 slice_cr_qp_offset;
>+ __s8 slice_act_y_qp_offset;
>+ __s8 slice_act_cb_qp_offset;
>+ __s8 slice_act_cr_qp_offset;
>+ __s8 slice_beta_offset_div2;
>+ __s8 slice_tc_offset_div2;
>+
>+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
>+ __u8 pic_struct;
>+
>+ __u8 reserved;
>+
>+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
>+ __u32 slice_segment_addr;
>+ __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+
>+ /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
>+ struct v4l2_hevc_pred_weight_table pred_weight_table;
>+
>+ __u64 flags;
>+ __u8 padding[48];
>+};
>+
>+#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC 0x1
>+#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC 0x2
>+#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR 0x4
>+
>+struct v4l2_ctrl_hevc_decode_params {
>+ __s32 pic_order_cnt_val;
>+ __u8 num_active_dpb_entries;
>+ __u8 num_poc_st_curr_before;
>+ __u8 num_poc_st_curr_after;
>+ __u8 num_poc_lt_curr;
>+ __u8 poc_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __u8 poc_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __u8 poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>+ __u64 flags;
>+};
>+
>+struct v4l2_ctrl_hevc_scaling_matrix {
>+ __u8 scaling_list_4x4[6][16];
>+ __u8 scaling_list_8x8[6][64];
>+ __u8 scaling_list_16x16[6][64];
>+ __u8 scaling_list_32x32[2][64];
>+ __u8 scaling_list_dc_coef_16x16[6];
>+ __u8 scaling_list_dc_coef_32x32[2];
>+};
>+
> /* MPEG-compression definitions kept for backwards compatibility */
> #ifndef __KERNEL__
> #define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC
>diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>index e27c8eae78c9..9af1c2460148 100644
>--- a/include/uapi/linux/videodev2.h
>+++ b/include/uapi/linux/videodev2.h
>@@ -708,6 +708,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh Hadamard Transform (vicodec) */
> #define V4L2_PIX_FMT_FWHT_STATELESS v4l2_fourcc('S', 'F', 'W', 'H') /* Stateless FWHT (vicodec) */
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
>+#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
>
> /* Vendor-specific formats */
> #define V4L2_PIX_FMT_CPIA1 v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
>@@ -1829,6 +1830,12 @@ enum v4l2_ctrl_type {
>
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR = 0x0260,
> V4L2_CTRL_TYPE_VP9_FRAME = 0x0261,
>+
>+ V4L2_CTRL_TYPE_HEVC_SPS = 0x0270,
>+ V4L2_CTRL_TYPE_HEVC_PPS = 0x0271,
>+ V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
>+ V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
>+ V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
> };
>
> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
>--
>2.32.0
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 16:31 ` John Cox
@ 2022-02-15 20:27 ` Jernej Škrabec
2022-02-16 10:38 ` John Cox
0 siblings, 1 reply; 35+ messages in thread
From: Jernej Škrabec @ 2022-02-15 20:27 UTC (permalink / raw)
To: John Cox
Cc: Nicolas Dufresne, Benjamin Gaignard, mchehab, ezequiel, p.zabel,
gregkh, mripard, paul.kocialkowski, wens, hverkuil-cisco, jonas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
Dne torek, 15. februar 2022 ob 17:31:28 CET je John Cox napisal(a):
> On Tue, 15 Feb 2022 17:11:12 +0100, you wrote:
>
> >Dne torek, 15. februar 2022 ob 17:00:33 CET je John Cox napisal(a):
> >> On Tue, 15 Feb 2022 10:28:55 -0500, you wrote:
> >>
> >> >Le mardi 15 février 2022 à 14:50 +0000, John Cox a écrit :
> >> >> On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
> >> >>
> >> >> >
> >> >> > Le 15/02/2022 à 15:17, John Cox a écrit :
> >> >> > > Hi
> >> >> > >
> >> >> > > > The bit size of the slice could be deduced from the buffer
payload
> >> >> > > > so remove bit_size field to avoid duplicated the information.
> >> >> > > I think this is a bad idea. In the future we are (I hope) going to
> >want
> >> >> > > to have an array (variable) of slice headers all referring to the
> >same
> >> >> > > bit buffer. When we do that we will need this field.
> >> >> >
> >> >> > I wonder if that could be considering like another decode mode and
so
> >> >> > use an other control ?
> >> >>
> >> >> I, personally, would be in favour of making the slice header control a
> >> >> variable array just as it is. If userland can't cope with multiple
> >> >> entries then just send them one at a time and the code looks exactly
> >> >> like it does at the moment and if the driver can't then set max array
> >> >> entries to 1.
> >> >>
> >> >> Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
> >> >> can say with experience that the code and effort overhead is very low.
> >> >>
> >> >> Either way having a multiple slice header control in the UAPI is
> >> >> important for efficiency.
> >> >
> >> >Just to clarify the idea, we would have a single slice controls, always
> >dynamic:
> >> >
> >> >1. For sliced based decoder
> >> >
> >> >The dynamic array slice control is implemented by the driver and its
size
> >must
> >> >be 1.
> >>
> >> Yes
> >>
> >> >2. For frame based decoder that don't care for slices
> >> >
> >> >The dynamic array slice controls is not implement. Userland detects that
at
> >> >runtime, similar to the VP9 compressed headers.
> >>
> >> If the driver parses all the slice header then that seems plausible
> >>
> >> >3. For frame based decoders that needs slices (or driver that supports
> >offset
> >> >and can gain performance with such mode)
> >> >
> >> >The dynamic array slice controls is implemented, and should contain all
the
> >> >slices found in the OUTPUT buffer.
> >> >
> >> >So the reason for this bit_size (not sure why its bits though, perhaps
> >someone
> >> >can educate me ?)
> >>
> >> RPi doesn't need bits and would be happy with bytes however
> >> slice_segment data isn't byte aligned at the end so its possible that
> >> there might be decoders out there that want an accurate length for that.
> >
> >There are two fields, please don't mix them up:
> >
> >__u32 bit_size;
> >__u32 data_bit_offset; (changed to data_byte_offset in this series)
> >
> >data_bit_offset/data_byte_offset is useful, while bit_size is IMO not. If you
> >have multiple slices in array, you only need to know start of the slice
data
> >and that offset is always offset from start of the buffer (absolute, it's not
> >relative to previous slice data).
>
> No... or at least I think not. RPi needs the start and end of the
> slice_segment_data elements of each slice.
It would be good to know if size needs to be exact or can overshoot, like
using end of buffer for that.
Cedrus also wants to know slice data size, but it turns out that bigger than
necessary size doesn't pose any problems. If that's not the case, then
bit_size needs stay in for sure.
Best regards,
Jernej
> If slices are arranged in the
> buffer with slice_segment_headers attached then I don't see how I get to
> know that. Also if the OUTPUT buffer is just a bit of bitstream, which
> might well be very convienient for some userspace, then it is legitimate
> to have SEIs between slice headers so you can't even guarantee that your
> coded slice segments are contiguous.
>
> Regards
>
> JC
>
> >Best regards,
> >Jernej
> >
> >>
> >> > Would be to let the driver offset inside the the single
> >> >OUTPUT/bitstream buffer in case this is not automatically found by the
> >driver
> >> >(or that no start-code is needed). Is that last bit correct ? If so,
should
> >we
> >> >change it to an offset rather then a size ? Shall we allow using offesets
> >inside
> >> >larger buffer (e.g. to avoid some memory copies) for the Sliced Base
cases ?
> >>
> >> I use (in the current structure) data_bit_offset to find the start of
> >> each slice's slice_segment_data within the OUTPUT buffer and bit_size to
> >> find the end. RPi doesn't / can't parse the slice_header and so wants
> >> all of that. Decoders that do parse the header might plausably want
> >> header offsets too and it would facilitate zero copy of the bit buffer.
> >>
> >>
> >> >> Regards
> >> >>
> >> >> John Cox
> >> >>
> >> >> > > > Signed-off-by: Benjamin Gaignard
<benjamin.gaignard@collabora.com>
> >> >> > > > ---
> >> >> > > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
> >> >> > > > drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 +++
> >+-------
> >> >> > > > include/uapi/linux/v4l2-controls.h | 3 +--
> >> >> > > > 3 files changed, 5 insertions(+), 12 deletions(-)
> >> >> > > >
> >> >> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-
> >codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> >> >> > > > index 3296ac3b9fca..c3ae97657fa7 100644
> >> >> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> >> >> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> >> >> > > > @@ -2965,9 +2965,6 @@ enum
v4l2_mpeg_video_hevc_size_of_length_field
> >-
> >> >> > > > :stub-columns: 0
> >> >> > > > :widths: 1 1 2
> >> >> > > >
> >> >> > > > - * - __u32
> >> >> > > > - - ``bit_size``
> >> >> > > > - - Size (in bits) of the current slice data.
> >> >> > > > * - __u32
> >> >> > > > - ``data_bit_offset``
> >> >> > > > - Offset (in bits) to the video data in the current slice
> >data.
> >> >> > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/
> >drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> >> >> > > > index 8ab2d9c6f048..db8c7475eeb8 100644
> >> >> > > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> >> >> > > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
> >> >> > > > @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct
cedrus_ctx
> >*ctx,
> >> >> > > > const struct v4l2_hevc_pred_weight_table
*pred_weight_table;
> >> >> > > > unsigned int width_in_ctb_luma, ctb_size_luma;
> >> >> > > > unsigned int log2_max_luma_coding_block_size;
> >> >> > > > + size_t slice_bytes;
> >> >> > > > dma_addr_t src_buf_addr;
> >> >> > > > - dma_addr_t src_buf_end_addr;
> >> >> > > > u32 chroma_log2_weight_denom;
> >> >> > > > u32 output_pic_list_index;
> >> >> > > > u32 pic_order_cnt[2];
> >> >> > > > @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct
cedrus_ctx
> >*ctx,
> >> >> > > >
> >> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
> >> >> > > >
> >> >> > > > - reg = slice_params->bit_size;
> >> >> > > > - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
> >> >> > > > + slice_bytes = vb2_get_plane_payload(&run->src-
>vb2_buf, 0);
> >> >> > > > + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
> >> >> > > I think one of these must be wrong. bit_size is in bits,
> >> >> > > vb2_get_plane_payload is in bytes?
> >> >> >
> >> >> > You are right it should be vb2_get_plane_payload() * 8 to get the
size
> >in bits.
> >> >> >
> >> >> > I will change that in v3.
> >> >> >
> >> >> > >
> >> >> > > Regards
> >> >> > >
> >> >> > > John Cox
> >> >> > >
> >> >> > > > /* Source beginning and end addresses. */
> >> >> > > >
> >> >> > > > @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct
> >cedrus_ctx *ctx,
> >> >> > > >
> >> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
> >> >> > > >
> >> >> > > > - src_buf_end_addr = src_buf_addr +
> >> >> > > > - DIV_ROUND_UP(slice_params-
>bit_size,
> >8);
> >> >> > > > -
> >> >> > > > - reg =
VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
> >> >> > > > + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr +
slice_bytes);
> >> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
> >> >> > > >
> >> >> > > > /* Coding tree block address */
> >> >> > > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/
> >linux/v4l2-controls.h
> >> >> > > > index b1a3dc05f02f..27f5d272dc43 100644
> >> >> > > > --- a/include/uapi/linux/v4l2-controls.h
> >> >> > > > +++ b/include/uapi/linux/v4l2-controls.h
> >> >> > > > @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
> >> >> > > > #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT
> >(1ULL << 9)
> >> >> > > >
> >> >> > > > struct v4l2_ctrl_hevc_slice_params {
> >> >> > > > - __u32 bit_size;
> >> >> > > > __u32 data_bit_offset;
> >> >> > > >
> >> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header
*/
> >> >> > > > @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
> >> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing
SEI message
> >*/
> >> >> > > > __u8 pic_struct;
> >> >> > > >
> >> >> > > > - __u8 reserved;
> >> >> > > > + __u8 reserved[5];
> >> >> > > >
> >> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice
segment
> >header */
> >> >> > > > __u32 slice_segment_addr;
> >>
> >
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
2022-02-15 20:27 ` Jernej Škrabec
@ 2022-02-16 10:38 ` John Cox
0 siblings, 0 replies; 35+ messages in thread
From: John Cox @ 2022-02-16 10:38 UTC (permalink / raw)
To: Jernej Škrabec
Cc: Nicolas Dufresne, Benjamin Gaignard, mchehab, ezequiel, p.zabel,
gregkh, mripard, paul.kocialkowski, wens, hverkuil-cisco, jonas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
On Tue, 15 Feb 2022 21:27:30 +0100, you wrote:
>Dne torek, 15. februar 2022 ob 17:31:28 CET je John Cox napisal(a):
>> On Tue, 15 Feb 2022 17:11:12 +0100, you wrote:
>>
>> >Dne torek, 15. februar 2022 ob 17:00:33 CET je John Cox napisal(a):
>> >> On Tue, 15 Feb 2022 10:28:55 -0500, you wrote:
>> >>
>> >> >Le mardi 15 février 2022 à 14:50 +0000, John Cox a écrit :
>> >> >> On Tue, 15 Feb 2022 15:35:12 +0100, you wrote:
>> >> >>
>> >> >> >
>> >> >> > Le 15/02/2022 à 15:17, John Cox a écrit :
>> >> >> > > Hi
>> >> >> > >
>> >> >> > > > The bit size of the slice could be deduced from the buffer
>payload
>> >> >> > > > so remove bit_size field to avoid duplicated the information.
>> >> >> > > I think this is a bad idea. In the future we are (I hope) going to
>> >want
>> >> >> > > to have an array (variable) of slice headers all referring to the
>> >same
>> >> >> > > bit buffer. When we do that we will need this field.
>> >> >> >
>> >> >> > I wonder if that could be considering like another decode mode and
>so
>> >> >> > use an other control ?
>> >> >>
>> >> >> I, personally, would be in favour of making the slice header control a
>> >> >> variable array just as it is. If userland can't cope with multiple
>> >> >> entries then just send them one at a time and the code looks exactly
>> >> >> like it does at the moment and if the driver can't then set max array
>> >> >> entries to 1.
>> >> >>
>> >> >> Having implemented this in rpi port of ffmpeg and the RPi V4L2 driver I
>> >> >> can say with experience that the code and effort overhead is very low.
>> >> >>
>> >> >> Either way having a multiple slice header control in the UAPI is
>> >> >> important for efficiency.
>> >> >
>> >> >Just to clarify the idea, we would have a single slice controls, always
>> >dynamic:
>> >> >
>> >> >1. For sliced based decoder
>> >> >
>> >> >The dynamic array slice control is implemented by the driver and its
>size
>> >must
>> >> >be 1.
>> >>
>> >> Yes
>> >>
>> >> >2. For frame based decoder that don't care for slices
>> >> >
>> >> >The dynamic array slice controls is not implement. Userland detects that
>at
>> >> >runtime, similar to the VP9 compressed headers.
>> >>
>> >> If the driver parses all the slice header then that seems plausible
>> >>
>> >> >3. For frame based decoders that needs slices (or driver that supports
>> >offset
>> >> >and can gain performance with such mode)
>> >> >
>> >> >The dynamic array slice controls is implemented, and should contain all
>the
>> >> >slices found in the OUTPUT buffer.
>> >> >
>> >> >So the reason for this bit_size (not sure why its bits though, perhaps
>> >someone
>> >> >can educate me ?)
>> >>
>> >> RPi doesn't need bits and would be happy with bytes however
>> >> slice_segment data isn't byte aligned at the end so its possible that
>> >> there might be decoders out there that want an accurate length for that.
>> >
>> >There are two fields, please don't mix them up:
>> >
>> >__u32 bit_size;
>> >__u32 data_bit_offset; (changed to data_byte_offset in this series)
>> >
>> >data_bit_offset/data_byte_offset is useful, while bit_size is IMO not. If you
>> >have multiple slices in array, you only need to know start of the slice
>data
>> >and that offset is always offset from start of the buffer (absolute, it's not
>> >relative to previous slice data).
>>
>> No... or at least I think not. RPi needs the start and end of the
>> slice_segment_data elements of each slice.
>
>It would be good to know if size needs to be exact or can overshoot, like
>using end of buffer for that.
>
>Cedrus also wants to know slice data size, but it turns out that bigger than
>necessary size doesn't pose any problems. If that's not the case, then
>bit_size needs stay in for sure.
RPi needs the exact size (bytes will do - but I don't see the harm in
specifying it in bits in case some future h/w wants the extra precision
as long as we nail down exactly which bit we mean)
Regards
JC
>Best regards,
>Jernej
>
>> If slices are arranged in the
>> buffer with slice_segment_headers attached then I don't see how I get to
>> know that. Also if the OUTPUT buffer is just a bit of bitstream, which
>> might well be very convienient for some userspace, then it is legitimate
>> to have SEIs between slice headers so you can't even guarantee that your
>> coded slice segments are contiguous.
>>
>> Regards
>>
>> JC
>>
>> >Best regards,
>> >Jernej
>> >
>> >>
>> >> > Would be to let the driver offset inside the the single
>> >> >OUTPUT/bitstream buffer in case this is not automatically found by the
>> >driver
>> >> >(or that no start-code is needed). Is that last bit correct ? If so,
>should
>> >we
>> >> >change it to an offset rather then a size ? Shall we allow using offesets
>> >inside
>> >> >larger buffer (e.g. to avoid some memory copies) for the Sliced Base
>cases ?
>> >>
>> >> I use (in the current structure) data_bit_offset to find the start of
>> >> each slice's slice_segment_data within the OUTPUT buffer and bit_size to
>> >> find the end. RPi doesn't / can't parse the slice_header and so wants
>> >> all of that. Decoders that do parse the header might plausably want
>> >> header offsets too and it would facilitate zero copy of the bit buffer.
>> >>
>> >>
>> >> >> Regards
>> >> >>
>> >> >> John Cox
>> >> >>
>> >> >> > > > Signed-off-by: Benjamin Gaignard
><benjamin.gaignard@collabora.com>
>> >> >> > > > ---
>> >> >> > > > .../userspace-api/media/v4l/ext-ctrls-codec.rst | 3 ---
>> >> >> > > > drivers/staging/media/sunxi/cedrus/cedrus_h265.c | 11 +++
>> >+-------
>> >> >> > > > include/uapi/linux/v4l2-controls.h | 3 +--
>> >> >> > > > 3 files changed, 5 insertions(+), 12 deletions(-)
>> >> >> > > >
>> >> >> > > > diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-
>> >codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> >> >> > > > index 3296ac3b9fca..c3ae97657fa7 100644
>> >> >> > > > --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> >> >> > > > +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> >> >> > > > @@ -2965,9 +2965,6 @@ enum
>v4l2_mpeg_video_hevc_size_of_length_field
>> >-
>> >> >> > > > :stub-columns: 0
>> >> >> > > > :widths: 1 1 2
>> >> >> > > >
>> >> >> > > > - * - __u32
>> >> >> > > > - - ``bit_size``
>> >> >> > > > - - Size (in bits) of the current slice data.
>> >> >> > > > * - __u32
>> >> >> > > > - ``data_bit_offset``
>> >> >> > > > - Offset (in bits) to the video data in the current slice
>> >data.
>> >> >> > > > diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c b/
>> >drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> >> >> > > > index 8ab2d9c6f048..db8c7475eeb8 100644
>> >> >> > > > --- a/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> >> >> > > > +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h265.c
>> >> >> > > > @@ -312,8 +312,8 @@ static void cedrus_h265_setup(struct
>cedrus_ctx
>> >*ctx,
>> >> >> > > > const struct v4l2_hevc_pred_weight_table
>*pred_weight_table;
>> >> >> > > > unsigned int width_in_ctb_luma, ctb_size_luma;
>> >> >> > > > unsigned int log2_max_luma_coding_block_size;
>> >> >> > > > + size_t slice_bytes;
>> >> >> > > > dma_addr_t src_buf_addr;
>> >> >> > > > - dma_addr_t src_buf_end_addr;
>> >> >> > > > u32 chroma_log2_weight_denom;
>> >> >> > > > u32 output_pic_list_index;
>> >> >> > > > u32 pic_order_cnt[2];
>> >> >> > > > @@ -370,8 +370,8 @@ static void cedrus_h265_setup(struct
>cedrus_ctx
>> >*ctx,
>> >> >> > > >
>> >> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_OFFSET, 0);
>> >> >> > > >
>> >> >> > > > - reg = slice_params->bit_size;
>> >> >> > > > - cedrus_write(dev, VE_DEC_H265_BITS_LEN, reg);
>> >> >> > > > + slice_bytes = vb2_get_plane_payload(&run->src-
>>vb2_buf, 0);
>> >> >> > > > + cedrus_write(dev, VE_DEC_H265_BITS_LEN, slice_bytes);
>> >> >> > > I think one of these must be wrong. bit_size is in bits,
>> >> >> > > vb2_get_plane_payload is in bytes?
>> >> >> >
>> >> >> > You are right it should be vb2_get_plane_payload() * 8 to get the
>size
>> >in bits.
>> >> >> >
>> >> >> > I will change that in v3.
>> >> >> >
>> >> >> > >
>> >> >> > > Regards
>> >> >> > >
>> >> >> > > John Cox
>> >> >> > >
>> >> >> > > > /* Source beginning and end addresses. */
>> >> >> > > >
>> >> >> > > > @@ -384,10 +384,7 @@ static void cedrus_h265_setup(struct
>> >cedrus_ctx *ctx,
>> >> >> > > >
>> >> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_ADDR, reg);
>> >> >> > > >
>> >> >> > > > - src_buf_end_addr = src_buf_addr +
>> >> >> > > > - DIV_ROUND_UP(slice_params-
>>bit_size,
>> >8);
>> >> >> > > > -
>> >> >> > > > - reg =
>VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_end_addr);
>> >> >> > > > + reg = VE_DEC_H265_BITS_END_ADDR_BASE(src_buf_addr +
>slice_bytes);
>> >> >> > > > cedrus_write(dev, VE_DEC_H265_BITS_END_ADDR, reg);
>> >> >> > > >
>> >> >> > > > /* Coding tree block address */
>> >> >> > > > diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/
>> >linux/v4l2-controls.h
>> >> >> > > > index b1a3dc05f02f..27f5d272dc43 100644
>> >> >> > > > --- a/include/uapi/linux/v4l2-controls.h
>> >> >> > > > +++ b/include/uapi/linux/v4l2-controls.h
>> >> >> > > > @@ -2457,7 +2457,6 @@ struct v4l2_hevc_pred_weight_table {
>> >> >> > > > #define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT
>> >(1ULL << 9)
>> >> >> > > >
>> >> >> > > > struct v4l2_ctrl_hevc_slice_params {
>> >> >> > > > - __u32 bit_size;
>> >> >> > > > __u32 data_bit_offset;
>> >> >> > > >
>> >> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header
>*/
>> >> >> > > > @@ -2484,7 +2483,7 @@ struct v4l2_ctrl_hevc_slice_params {
>> >> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing
>SEI message
>> >*/
>> >> >> > > > __u8 pic_struct;
>> >> >> > > >
>> >> >> > > > - __u8 reserved;
>> >> >> > > > + __u8 reserved[5];
>> >> >> > > >
>> >> >> > > > /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice
>segment
>> >header */
>> >> >> > > > __u32 slice_segment_addr;
>> >>
>> >
>>
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
2022-02-15 11:01 ` [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control Benjamin Gaignard
@ 2022-02-16 10:54 ` John Cox
2022-02-16 17:19 ` Benjamin Gaignard
0 siblings, 1 reply; 35+ messages in thread
From: John Cox @ 2022-02-16 10:54 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, Benjamin Gaignard
On Tue, 15 Feb 2022 12:01:02 +0100, you wrote:
>The number of 'entry point offset' could be very variable.
>Rather than use a large static array define a v4l2 dynamic array
>of integer control.
>The number of entry point offsets is reported by the elems field.
If we are going to support multiple slice headers in a single control
then we are going to need to know how to split these entries between
them as I assume that the entry point offset entries for all slice
headers would just be catenated here. You could add extra elements here
or generate a completely new control but I think it would be easier to
reinstate the num_entry_point_offsets field in the slice_params
structure (its a field that userpace has in its hand anyway so its no
effort for it to fill in). num_entry_point_offsets is a field that RPi
wants (without needing the entry points themselves) so that would help
me.
Regards
John Cox
>Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>---
>version 2:
>- Define a new CID for entry point offsets parameters
> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
> include/uapi/linux/v4l2-controls.h | 1 +
> 2 files changed, 10 insertions(+)
>
>diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>index c3ae97657fa7..679595e94643 100644
>--- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>+++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>@@ -3101,6 +3101,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
>+``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
>+ Specifies the i-th entry point offset in bytes and is represented by
>+ offset_len_minus1 plus 1 bits.
>+ This control is a dynamically sized array. The number of entry point
>+ offsets is reported by the ``elems`` field.
>+ This bitstream parameter is defined according to :ref:`hevc`.
>+ They are described in section 7.4.7.1 "General slice segment header
>+ semantics" of the specification.
>+
> ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
> Specifies the HEVC scaling matrix parameters used for the scaling process
> for transform coefficients.
>diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>index 27f5d272dc43..700e6cce958c 100644
>--- a/include/uapi/linux/v4l2-controls.h
>+++ b/include/uapi/linux/v4l2-controls.h
>@@ -2314,6 +2314,7 @@ struct v4l2_ctrl_vp9_compressed_hdr {
> #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404)
> #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405)
> #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406)
>+#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_STATELESS_BASE + 407)
>
> enum v4l2_stateless_hevc_decode_mode {
> V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
2022-02-16 10:54 ` John Cox
@ 2022-02-16 17:19 ` Benjamin Gaignard
2022-02-16 17:35 ` John Cox
0 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-16 17:19 UTC (permalink / raw)
To: John Cox
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
Le 16/02/2022 à 11:54, John Cox a écrit :
> On Tue, 15 Feb 2022 12:01:02 +0100, you wrote:
>
>> The number of 'entry point offset' could be very variable.
>> Rather than use a large static array define a v4l2 dynamic array
>> of integer control.
>> The number of entry point offsets is reported by the elems field.
> If we are going to support multiple slice headers in a single control
> then we are going to need to know how to split these entries between
> them as I assume that the entry point offset entries for all slice
> headers would just be catenated here. You could add extra elements here
> or generate a completely new control but I think it would be easier to
> reinstate the num_entry_point_offsets field in the slice_params
> structure (its a field that userpace has in its hand anyway so its no
> effort for it to fill in). num_entry_point_offsets is a field that RPi
> wants (without needing the entry points themselves) so that would help
> me.
Would that means num_entry_point_offsets field in slice parameter control
+ one control for the entry points themselves ?
or the both on slice parameter control (like now) ?
Regards
Benjamin
>
> Regards
>
> John Cox
>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> version 2:
>> - Define a new CID for entry point offsets parameters
>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
>> include/uapi/linux/v4l2-controls.h | 1 +
>> 2 files changed, 10 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index c3ae97657fa7..679595e94643 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -3101,6 +3101,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>
>> \normalsize
>>
>> +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
>> + Specifies the i-th entry point offset in bytes and is represented by
>> + offset_len_minus1 plus 1 bits.
>> + This control is a dynamically sized array. The number of entry point
>> + offsets is reported by the ``elems`` field.
>> + This bitstream parameter is defined according to :ref:`hevc`.
>> + They are described in section 7.4.7.1 "General slice segment header
>> + semantics" of the specification.
>> +
>> ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
>> Specifies the HEVC scaling matrix parameters used for the scaling process
>> for transform coefficients.
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index 27f5d272dc43..700e6cce958c 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -2314,6 +2314,7 @@ struct v4l2_ctrl_vp9_compressed_hdr {
>> #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404)
>> #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405)
>> #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406)
>> +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_STATELESS_BASE + 407)
>>
>> enum v4l2_stateless_hevc_decode_mode {
>> V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
2022-02-16 17:19 ` Benjamin Gaignard
@ 2022-02-16 17:35 ` John Cox
0 siblings, 0 replies; 35+ messages in thread
From: John Cox @ 2022-02-16 17:35 UTC (permalink / raw)
To: Benjamin Gaignard
Cc: mchehab, ezequiel, p.zabel, gregkh, mripard, paul.kocialkowski,
wens, jernej.skrabec, hverkuil-cisco, jonas, nicolas,
linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche
On Wed, 16 Feb 2022 18:19:10 +0100, you wrote:
>
>Le 16/02/2022 à 11:54, John Cox a écrit :
>> On Tue, 15 Feb 2022 12:01:02 +0100, you wrote:
>>
>>> The number of 'entry point offset' could be very variable.
>>> Rather than use a large static array define a v4l2 dynamic array
>>> of integer control.
>>> The number of entry point offsets is reported by the elems field.
>> If we are going to support multiple slice headers in a single control
>> then we are going to need to know how to split these entries between
>> them as I assume that the entry point offset entries for all slice
>> headers would just be catenated here. You could add extra elements here
>> or generate a completely new control but I think it would be easier to
>> reinstate the num_entry_point_offsets field in the slice_params
>> structure (its a field that userpace has in its hand anyway so its no
>> effort for it to fill in). num_entry_point_offsets is a field that RPi
>> wants (without needing the entry points themselves) so that would help
>> me.
>
>Would that means num_entry_point_offsets field in slice parameter control
>+ one control for the entry points themselves ?
>or the both on slice parameter control (like now) ?
I meant num_entry_point_offsets field in each slice parameter control +
one control for the entry points themselves. Given that you have a
varible number of entry points associated with each slice and you have a
variable number of slices this seems the easiest answer to me both to
construct in userspace and to use in the driver.
Regards
JC
>Regards
>Benjamin
>
>>
>> Regards
>>
>> John Cox
>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> version 2:
>>> - Define a new CID for entry point offsets parameters
>>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 +++++++++
>>> include/uapi/linux/v4l2-controls.h | 1 +
>>> 2 files changed, 10 insertions(+)
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> index c3ae97657fa7..679595e94643 100644
>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> @@ -3101,6 +3101,15 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>
>>> \normalsize
>>>
>>> +``V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS (integer)``
>>> + Specifies the i-th entry point offset in bytes and is represented by
>>> + offset_len_minus1 plus 1 bits.
>>> + This control is a dynamically sized array. The number of entry point
>>> + offsets is reported by the ``elems`` field.
>>> + This bitstream parameter is defined according to :ref:`hevc`.
>>> + They are described in section 7.4.7.1 "General slice segment header
>>> + semantics" of the specification.
>>> +
>>> ``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
>>> Specifies the HEVC scaling matrix parameters used for the scaling process
>>> for transform coefficients.
>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>> index 27f5d272dc43..700e6cce958c 100644
>>> --- a/include/uapi/linux/v4l2-controls.h
>>> +++ b/include/uapi/linux/v4l2-controls.h
>>> @@ -2314,6 +2314,7 @@ struct v4l2_ctrl_vp9_compressed_hdr {
>>> #define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404)
>>> #define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405)
>>> #define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406)
>>> +#define V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET (V4L2_CID_CODEC_STATELESS_BASE + 407)
>>>
>>> enum v4l2_stateless_hevc_decode_mode {
>>> V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 0/8] Move HEVC stateless controls out of staging
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
` (9 preceding siblings ...)
2022-02-15 16:14 ` Jernej Škrabec
@ 2022-02-18 9:59 ` Hans Verkuil
2022-02-18 10:54 ` Benjamin Gaignard
10 siblings, 1 reply; 35+ messages in thread
From: Hans Verkuil @ 2022-02-18 9:59 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Hi Benjamin,
On 15/02/2022 12:00, Benjamin Gaignard wrote:
> This series aims to make HEVC uapi stable and usable for hardware
> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
> and 2 out of the tree drivers (rkvdec and RPI).
>
> The 3 first patches are from Hans to implement v4l2 dynamic control
> feature which is need by patch 7 for V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET
> definition.
>
> Patch 4 move the existing uapi to stable, including definitions renaming
> and CID number change to fit with v4l2 naming.
>
> Patches 5 and 7 add fields needed for rkvdec and RPI decoders.
>
> Patches 6 is cleaning up the uapi of useless field.
> Patches 8 change one field description and name to define offset by
> bytes rather than by bits
I have some housekeeping questions:
I have several older HEVC-related patches from you, which of those are still
valid?
"[v4,0/9] Additional features for Hantro HEVC":
https://patchwork.linuxtv.org/project/linux-media/cover/20210625141143.577998-1-benjamin.gaignard@collabora.com/
"media: hevc: fix pictures lists type":
https://patchwork.linuxtv.org/project/linux-media/patch/20210823082949.237716-1-benjamin.gaignard@collabora.com/
"media: hantro: Trace hevc hw cycles performance register":
https://patchwork.linuxtv.org/project/linux-media/patch/20210823135606.633052-1-benjamin.gaignard@collabora.com/
"media: hantro: Add support of compressed reference buffers"
https://patchwork.linuxtv.org/project/linux-media/patch/20210823162916.824336-1-benjamin.gaignard@collabora.com/
Before moving the HEVC API out of staging I would prefer to have any remaining
issues fixed. So one series that adds any remaining fixes (i.e. from the older patches
mentioned above), and a second series on top that moves it out of staging.
That way I can mark the older patches as Superseded, and we have a
fresh series that we can discuss.
Regards,
Hans
>
> Benjamin
>
> Benjamin Gaignard (5):
> media: uapi: Move HEVC stateless controls out of staging
> media: uapi: Add fields needed for RKVDEC driver
> media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
> media: uapi: Change data_bit_offset definition
>
> Hans Verkuil (3):
> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
> v4l2-ctrls: add support for dynamically allocated arrays.
> vivid: add dynamic array test control
>
> .../userspace-api/media/drivers/hantro.rst | 5 -
> .../media/v4l/ext-ctrls-codec.rst | 58 ++--
> .../media/v4l/vidioc-queryctrl.rst | 8 +
> .../media/test-drivers/vivid/vivid-ctrls.c | 15 ++
> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++--
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 ++++++++++---
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> .../staging/media/sunxi/cedrus/cedrus_h265.c | 13 +-
> include/linux/hantro-media.h | 17 ++
> include/media/hevc-ctrls.h | 250 ------------------
> include/media/v4l2-ctrls.h | 48 +++-
> include/uapi/linux/v4l2-controls.h | 224 ++++++++++++++++
> include/uapi/linux/videodev2.h | 8 +
> 19 files changed, 640 insertions(+), 408 deletions(-)
> create mode 100644 include/linux/hantro-media.h
> delete mode 100644 include/media/hevc-ctrls.h
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 0/8] Move HEVC stateless controls out of staging
2022-02-18 9:59 ` Hans Verkuil
@ 2022-02-18 10:54 ` Benjamin Gaignard
2022-02-18 11:05 ` Hans Verkuil
0 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-18 10:54 UTC (permalink / raw)
To: Hans Verkuil, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Le 18/02/2022 à 10:59, Hans Verkuil a écrit :
> Hi Benjamin,
>
> On 15/02/2022 12:00, Benjamin Gaignard wrote:
>> This series aims to make HEVC uapi stable and usable for hardware
>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>> and 2 out of the tree drivers (rkvdec and RPI).
>>
>> The 3 first patches are from Hans to implement v4l2 dynamic control
>> feature which is need by patch 7 for V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET
>> definition.
>>
>> Patch 4 move the existing uapi to stable, including definitions renaming
>> and CID number change to fit with v4l2 naming.
>>
>> Patches 5 and 7 add fields needed for rkvdec and RPI decoders.
>>
>> Patches 6 is cleaning up the uapi of useless field.
>> Patches 8 change one field description and name to define offset by
>> bytes rather than by bits
> I have some housekeeping questions:
>
> I have several older HEVC-related patches from you, which of those are still
> valid?
>
> "[v4,0/9] Additional features for Hantro HEVC":
> https://patchwork.linuxtv.org/project/linux-media/cover/20210625141143.577998-1-benjamin.gaignard@collabora.com/
>
> "media: hevc: fix pictures lists type":
> https://patchwork.linuxtv.org/project/linux-media/patch/20210823082949.237716-1-benjamin.gaignard@collabora.com/
>
> "media: hantro: Trace hevc hw cycles performance register":
> https://patchwork.linuxtv.org/project/linux-media/patch/20210823135606.633052-1-benjamin.gaignard@collabora.com/
>
> "media: hantro: Add support of compressed reference buffers"
> https://patchwork.linuxtv.org/project/linux-media/patch/20210823162916.824336-1-benjamin.gaignard@collabora.com/
>
> Before moving the HEVC API out of staging I would prefer to have any remaining
> issues fixed. So one series that adds any remaining fixes (i.e. from the older patches
> mentioned above), and a second series on top that moves it out of staging.
Hi Hans,
They are all related to Hantro diver and, after post-proc patches, the internal driver design
has change so they won't apply anymore. You can discard them.
HEVC UAPI migration isn't not impacted by these patches.
My plan is a finish the HEVC UAPI first, send patches to add features (compression, 10bits, scaling)
to Hantro driver and finally move it out of staging too.
If possible I would like to not make HEVC UAPI topic depends of Hantro features.
Does that makes sense for you ?
Regards,
Benjamin
>
> That way I can mark the older patches as Superseded, and we have a
> fresh series that we can discuss.
>
> Regards,
>
> Hans
>
>> Benjamin
>>
>> Benjamin Gaignard (5):
>> media: uapi: Move HEVC stateless controls out of staging
>> media: uapi: Add fields needed for RKVDEC driver
>> media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>> media: uapi: Change data_bit_offset definition
>>
>> Hans Verkuil (3):
>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>> v4l2-ctrls: add support for dynamically allocated arrays.
>> vivid: add dynamic array test control
>>
>> .../userspace-api/media/drivers/hantro.rst | 5 -
>> .../media/v4l/ext-ctrls-codec.rst | 58 ++--
>> .../media/v4l/vidioc-queryctrl.rst | 8 +
>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 ++
>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++--
>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 ++++++++++---
>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 13 +-
>> include/linux/hantro-media.h | 17 ++
>> include/media/hevc-ctrls.h | 250 ------------------
>> include/media/v4l2-ctrls.h | 48 +++-
>> include/uapi/linux/v4l2-controls.h | 224 ++++++++++++++++
>> include/uapi/linux/videodev2.h | 8 +
>> 19 files changed, 640 insertions(+), 408 deletions(-)
>> create mode 100644 include/linux/hantro-media.h
>> delete mode 100644 include/media/hevc-ctrls.h
>>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 0/8] Move HEVC stateless controls out of staging
2022-02-18 10:54 ` Benjamin Gaignard
@ 2022-02-18 11:05 ` Hans Verkuil
0 siblings, 0 replies; 35+ messages in thread
From: Hans Verkuil @ 2022-02-18 11:05 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
On 18/02/2022 11:54, Benjamin Gaignard wrote:
>
> Le 18/02/2022 à 10:59, Hans Verkuil a écrit :
>> Hi Benjamin,
>>
>> On 15/02/2022 12:00, Benjamin Gaignard wrote:
>>> This series aims to make HEVC uapi stable and usable for hardware
>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro)
>>> and 2 out of the tree drivers (rkvdec and RPI).
>>>
>>> The 3 first patches are from Hans to implement v4l2 dynamic control
>>> feature which is need by patch 7 for V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET
>>> definition.
>>>
>>> Patch 4 move the existing uapi to stable, including definitions renaming
>>> and CID number change to fit with v4l2 naming.
>>>
>>> Patches 5 and 7 add fields needed for rkvdec and RPI decoders.
>>>
>>> Patches 6 is cleaning up the uapi of useless field.
>>> Patches 8 change one field description and name to define offset by
>>> bytes rather than by bits
>> I have some housekeeping questions:
>>
>> I have several older HEVC-related patches from you, which of those are still
>> valid?
>>
>> "[v4,0/9] Additional features for Hantro HEVC":
>> https://patchwork.linuxtv.org/project/linux-media/cover/20210625141143.577998-1-benjamin.gaignard@collabora.com/
>>
>> "media: hevc: fix pictures lists type":
>> https://patchwork.linuxtv.org/project/linux-media/patch/20210823082949.237716-1-benjamin.gaignard@collabora.com/
>>
>> "media: hantro: Trace hevc hw cycles performance register":
>> https://patchwork.linuxtv.org/project/linux-media/patch/20210823135606.633052-1-benjamin.gaignard@collabora.com/
>>
>> "media: hantro: Add support of compressed reference buffers"
>> https://patchwork.linuxtv.org/project/linux-media/patch/20210823162916.824336-1-benjamin.gaignard@collabora.com/
>>
>> Before moving the HEVC API out of staging I would prefer to have any remaining
>> issues fixed. So one series that adds any remaining fixes (i.e. from the older patches
>> mentioned above), and a second series on top that moves it out of staging.
>
> Hi Hans,
>
> They are all related to Hantro diver and, after post-proc patches, the internal driver design
> has change so they won't apply anymore. You can discard them.
OK, I'll mark them as Obsoleted.
> HEVC UAPI migration isn't not impacted by these patches.
> My plan is a finish the HEVC UAPI first, send patches to add features (compression, 10bits, scaling)
> to Hantro driver and finally move it out of staging too.
> If possible I would like to not make HEVC UAPI topic depends of Hantro features.
>
> Does that makes sense for you ?
Well, only if you are absolutely certain that those features won't impact the uAPI.
The easiest way to do that is to implement them and show that it doesn't :-)
That's really my main concern: that those additions require uAPI changes.
Looking over those patches it appears that this is the only one touching on
the uAPI:
"media: hevc: fix pictures lists type":
https://patchwork.linuxtv.org/project/linux-media/patch/20210823082949.237716-1-benjamin.gaignard@collabora.com/
I haven't read all the comments in details, but it is something that needs to
be clarified.
Regards,
Hans
>
> Regards,
> Benjamin
>
>>
>> That way I can mark the older patches as Superseded, and we have a
>> fresh series that we can discuss.
>>
>> Regards,
>>
>> Hans
>>
>>> Benjamin
>>>
>>> Benjamin Gaignard (5):
>>> media: uapi: Move HEVC stateless controls out of staging
>>> media: uapi: Add fields needed for RKVDEC driver
>>> media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params
>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control
>>> media: uapi: Change data_bit_offset definition
>>>
>>> Hans Verkuil (3):
>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY
>>> v4l2-ctrls: add support for dynamically allocated arrays.
>>> vivid: add dynamic array test control
>>>
>>> .../userspace-api/media/drivers/hantro.rst | 5 -
>>> .../media/v4l/ext-ctrls-codec.rst | 58 ++--
>>> .../media/v4l/vidioc-queryctrl.rst | 8 +
>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 ++
>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++++++--
>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 182 ++++++++++---
>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +-
>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +-
>>> drivers/staging/media/hantro/hantro_drv.c | 27 +-
>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
>>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 13 +-
>>> include/linux/hantro-media.h | 17 ++
>>> include/media/hevc-ctrls.h | 250 ------------------
>>> include/media/v4l2-ctrls.h | 48 +++-
>>> include/uapi/linux/v4l2-controls.h | 224 ++++++++++++++++
>>> include/uapi/linux/videodev2.h | 8 +
>>> 19 files changed, 640 insertions(+), 408 deletions(-)
>>> create mode 100644 include/linux/hantro-media.h
>>> delete mode 100644 include/media/hevc-ctrls.h
>>>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver
2022-02-15 11:01 ` [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver Benjamin Gaignard
@ 2022-02-18 11:32 ` Hans Verkuil
2022-02-18 12:19 ` Benjamin Gaignard
0 siblings, 1 reply; 35+ messages in thread
From: Hans Verkuil @ 2022-02-18 11:32 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
On 15/02/2022 12:01, Benjamin Gaignard wrote:
> RKVDEC driver requires additional fields to perform HEVC decoding.
> Even if the driver isn't mainlined yet WIP patches could be find here:
> https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
>
> This patch only include the change in HEVC uAPI.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 16 ++++++++++++++++
> include/uapi/linux/v4l2-controls.h | 5 +++++
> 2 files changed, 21 insertions(+)
>
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 4f3b3ba8319f..3296ac3b9fca 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -2661,6 +2661,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
> + * - __u8
> + - ``video_parameter_set_id``
> + - Identifies the VPS for reference by other syntax elements.
> + * - __u8
> + - ``seq_parameter_set_id``
> + - Provides an identifier for the SPS for reference by other syntax
> + elements.
> * - __u16
> - ``pic_width_in_luma_samples``
> -
> @@ -2800,6 +2807,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
> + * - __u8
> + - ``pic_parameter_set_id``
> + - Identifies the PPS for reference by other syntax elements.
> * - __u8
> - ``num_extra_slice_header_bits``
> -
> @@ -3026,6 +3036,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> * - __u8
> - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
> - The list of L1 reference elements as indices in the DPB.
> + * - __u16
> + - ``short_term_ref_pic_set_size``
> + -
> + * - __u16
> + - ``long_term_ref_pic_set_size``
> + -
> * - __u8
> - ``padding``
> - Applications and drivers must set this to zero.
Just to confirm: these additional fields are all from the H.265 spec, right?
They are not rkvdec specific.
Regards,
Hans
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 0e0ec2c61b80..b1a3dc05f02f 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -2341,6 +2341,8 @@ enum v4l2_stateless_hevc_start_code {
>
> struct v4l2_ctrl_hevc_sps {
> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
> + __u8 video_parameter_set_id;
> + __u8 seq_parameter_set_id;
> __u16 pic_width_in_luma_samples;
> __u16 pic_height_in_luma_samples;
> __u8 bit_depth_luma_minus8;
> @@ -2393,6 +2395,7 @@ struct v4l2_ctrl_hevc_sps {
>
> struct v4l2_ctrl_hevc_pps {
> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
> + __u8 pic_parameter_set_id;
> __u8 num_extra_slice_header_bits;
> __u8 num_ref_idx_l0_default_active_minus1;
> __u8 num_ref_idx_l1_default_active_minus1;
> @@ -2487,6 +2490,8 @@ struct v4l2_ctrl_hevc_slice_params {
> __u32 slice_segment_addr;
> __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __u16 short_term_ref_pic_set_size;
> + __u16 long_term_ref_pic_set_size;
>
> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
> struct v4l2_hevc_pred_weight_table pred_weight_table;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging
2022-02-15 11:00 ` [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging Benjamin Gaignard
2022-02-15 18:02 ` Sebastian Fricke
@ 2022-02-18 11:50 ` Hans Verkuil
1 sibling, 0 replies; 35+ messages in thread
From: Hans Verkuil @ 2022-02-18 11:50 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Hi Benjamin,
Some comments:
First of all, take a look at the series that moved VP8 out of staging:
https://patchwork.linuxtv.org/project/linux-media/cover/20210304140755.85581-1-ezequiel@collabora.com/
It's a good reference of what is needed and I'll refer to this in some of
the comments below.
On 15/02/2022 12:00, Benjamin Gaignard wrote:
> The HEVC stateless 'uAPI' was staging and marked explicitly in the
> V4L2 specification that it will change and is unstable.
>
> Note that these control IDs were never exported as a public API,
> they were only defined in kernel-local headers (hevc-ctrls.h).
>
> While moving the controls out of staging they are renamed and
> control IDs get new numbers.
> Drivers (Hantro, Cedrus) and Documentation are updated accordingly.
>
> Hantro dedicated control is moving to hantro-media.h
> Since hevc-ctrls.h content has been dispatched in others file, remove it.
>
> fluster tests results on IMX8MQ is 77/147 for HEVC codec.
>
> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
> ---
> version 2 :
> - Do not include rkvdec needed fields in this patch
> - Remove 'non-public' notice in hantro.rst
> - Take care of cache-line boundary
>
> .../userspace-api/media/drivers/hantro.rst | 5 -
> .../media/v4l/ext-ctrls-codec.rst | 26 +-
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 32 +--
> drivers/staging/media/hantro/hantro_drv.c | 27 +-
> drivers/staging/media/hantro/hantro_hevc.c | 8 +-
> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +-
> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +-
> include/linux/hantro-media.h | 17 ++
> include/media/hevc-ctrls.h | 250 ------------------
> include/media/v4l2-ctrls.h | 6 -
> include/uapi/linux/v4l2-controls.h | 219 +++++++++++++++
> include/uapi/linux/videodev2.h | 7 +
> 12 files changed, 307 insertions(+), 324 deletions(-)
> create mode 100644 include/linux/hantro-media.h
> delete mode 100644 include/media/hevc-ctrls.h
>
> diff --git a/Documentation/userspace-api/media/drivers/hantro.rst b/Documentation/userspace-api/media/drivers/hantro.rst
> index cd9754b4e005..78dcd2a44a03 100644
> --- a/Documentation/userspace-api/media/drivers/hantro.rst
> +++ b/Documentation/userspace-api/media/drivers/hantro.rst
> @@ -12,8 +12,3 @@ The Hantro video decoder driver implements the following driver-specific control
> to before syntax element "slice_temporal_mvp_enabled_flag".
> If IDR, the skipped bits are just "pic_output_flag"
> (separate_colour_plane_flag is not supported).
> -
> -.. note::
> -
> - This control is not yet part of the public kernel API and
> - it is expected to change.
> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> index 4cd7c541fc30..4f3b3ba8319f 100644
> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
> @@ -2639,7 +2639,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> .. _v4l2-mpeg-hevc:
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_SPS (struct)``
> +``V4L2_CID_STATELESS_HEVC_SPS (struct)``
> Specifies the Sequence Parameter Set fields (as extracted from the
> bitstream) for the associated HEVC slice data.
> These bitstream parameters are defined according to :ref:`hevc`.
The stateless controls must all be moved to ext-ctrls-codec-stateless.rst.
See VP8 patch 8/8. It should also be done in a separate patch.
> @@ -2782,7 +2782,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_PPS (struct)``
> +``V4L2_CID_STATELESS_HEVC_PPS (struct)``
> Specifies the Picture Parameter Set fields (as extracted from the
> bitstream) for the associated HEVC slice data.
> These bitstream parameters are defined according to :ref:`hevc`.
> @@ -2932,7 +2932,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (struct)``
> +``V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (struct)``
> Specifies various slice-specific parameters, especially from the NAL unit
> header, general slice segment header and weighted prediction parameter
> parts of the bitstream.
> @@ -3088,7 +3088,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (struct)``
> +``V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (struct)``
> Specifies the HEVC scaling matrix parameters used for the scaling process
> for transform coefficients.
> These matrix and parameters are defined according to :ref:`hevc`.
> @@ -3238,7 +3238,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (enum)``
> +``V4L2_CID_STATELESS_HEVC_DECODE_MODE (enum)``
> Specifies the decoding mode to use. Currently exposes slice-based and
> frame-based decoding but new modes might be added later on.
> This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
> @@ -3253,7 +3253,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This menu control is not yet part of the public kernel API and
> it is expected to change.
>
> -.. c:type:: v4l2_mpeg_video_hevc_decode_mode
> +.. c:type:: v4l2_stateless_hevc_decode_mode
>
> .. raw:: latex
>
> @@ -3266,11 +3266,11 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
> - * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED``
> + * - ``V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED``
> - 0
> - Decoding is done at the slice granularity.
> The OUTPUT buffer must contain a single slice.
> - * - ``V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED``
> + * - ``V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED``
> - 1
> - Decoding is done at the frame granularity.
> The OUTPUT buffer must contain all slices needed to decode the
> @@ -3280,7 +3280,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>
> \normalsize
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (enum)``
> +``V4L2_CID_STATELESS_HEVC_START_CODE (enum)``
> Specifies the HEVC slice start code expected for each slice.
> This control is used as a modifier for V4L2_PIX_FMT_HEVC_SLICE
> pixel format. Applications that support V4L2_PIX_FMT_HEVC_SLICE
> @@ -3294,7 +3294,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This menu control is not yet part of the public kernel API and
> it is expected to change.
>
> -.. c:type:: v4l2_mpeg_video_hevc_start_code
> +.. c:type:: v4l2_stateless_hevc_start_code
>
> .. tabularcolumns:: |p{9.2cm}|p{0.6cm}|p{7.5cm}|
>
> @@ -3303,13 +3303,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> :stub-columns: 0
> :widths: 1 1 2
>
> - * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE``
> + * - ``V4L2_STATELESS_HEVC_START_CODE_NONE``
> - 0
> - Selecting this value specifies that HEVC slices are passed
> to the driver without any start code. The bitstream data should be
> according to :ref:`hevc` 7.3.1.1 General NAL unit syntax, hence
> contains emulation prevention bytes when required.
> - * - ``V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B``
> + * - ``V4L2_STATELESS_HEVC_START_CODE_ANNEX_B``
> - 1
> - Selecting this value specifies that HEVC slices are expected
> to be prefixed by Annex B start codes. According to :ref:`hevc`
> @@ -3342,7 +3342,7 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
> This provides a bitmask which consists of bits [0, LTR_COUNT-1].
> This is applicable to the H264 and HEVC encoders.
>
> -``V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (struct)``
> +``V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (struct)``
> Specifies various decode parameters, especially the references picture order
> count (POC) for all the lists (short, long, before, current, after) and the
> number of entries for each of them.
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls-defs.c b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> index 54ca4e6b820b..4b68cbe23309 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls-defs.c
> @@ -699,9 +699,9 @@ const char * const *v4l2_ctrl_get_menu(u32 id)
> return hevc_tier;
> case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
> return hevc_loop_filter_mode;
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
> + case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
> return hevc_decode_mode;
> - case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
> + case V4L2_CID_STATELESS_HEVC_START_CODE:
> return hevc_start_code;
> case V4L2_CID_CAMERA_ORIENTATION:
> return camera_orientation;
> @@ -995,13 +995,6 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD: return "HEVC Size of Length Field";
> case V4L2_CID_MPEG_VIDEO_REF_NUMBER_FOR_PFRAMES: return "Reference Frames for a P-Frame";
> case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR: return "Prepend SPS and PPS to IDR";
> - case V4L2_CID_MPEG_VIDEO_HEVC_SPS: return "HEVC Sequence Parameter Set";
> - case V4L2_CID_MPEG_VIDEO_HEVC_PPS: return "HEVC Picture Parameter Set";
> - case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
> - case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC Decode Mode";
> - case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC Start Code";
>
> /* CAMERA controls */
> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1180,6 +1173,13 @@ const char *v4l2_ctrl_get_name(u32 id)
> case V4L2_CID_STATELESS_MPEG2_QUANTISATION: return "MPEG-2 Quantisation Matrices";
> case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR: return "VP9 Probabilities Updates";
> case V4L2_CID_STATELESS_VP9_FRAME: return "VP9 Frame Decode Parameters";
> + case V4L2_CID_STATELESS_HEVC_SPS: return "HEVC Sequence Parameter Set";
> + case V4L2_CID_STATELESS_HEVC_PPS: return "HEVC Picture Parameter Set";
> + case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS: return "HEVC Slice Parameters";
> + case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX: return "HEVC Scaling Matrix";
> + case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS: return "HEVC Decode Parameters";
> + case V4L2_CID_STATELESS_HEVC_DECODE_MODE: return "HEVC Decode Mode";
> + case V4L2_CID_STATELESS_HEVC_START_CODE: return "HEVC Start Code";
>
> /* Colorimetry controls */
> /* Keep the order of the 'case's the same as in v4l2-controls.h! */
> @@ -1355,8 +1355,8 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_MPEG_VIDEO_HEVC_SIZE_OF_LENGTH_FIELD:
> case V4L2_CID_MPEG_VIDEO_HEVC_TIER:
> case V4L2_CID_MPEG_VIDEO_HEVC_LOOP_FILTER_MODE:
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE:
> - case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE:
> + case V4L2_CID_STATELESS_HEVC_DECODE_MODE:
> + case V4L2_CID_STATELESS_HEVC_START_CODE:
> case V4L2_CID_STATELESS_H264_DECODE_MODE:
> case V4L2_CID_STATELESS_H264_START_CODE:
> case V4L2_CID_CAMERA_ORIENTATION:
> @@ -1493,19 +1493,19 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
> case V4L2_CID_STATELESS_VP8_FRAME:
> *type = V4L2_CTRL_TYPE_VP8_FRAME;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_SPS:
> + case V4L2_CID_STATELESS_HEVC_SPS:
> *type = V4L2_CTRL_TYPE_HEVC_SPS;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_PPS:
> + case V4L2_CID_STATELESS_HEVC_PPS:
> *type = V4L2_CTRL_TYPE_HEVC_PPS;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS:
> + case V4L2_CID_STATELESS_HEVC_SLICE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX:
> + case V4L2_CID_STATELESS_HEVC_SCALING_MATRIX:
> *type = V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX;
> break;
> - case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS:
> + case V4L2_CID_STATELESS_HEVC_DECODE_PARAMS:
> *type = V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS;
> break;
> case V4L2_CID_STATELESS_VP9_COMPRESSED_HDR:
I'm missing code in drivers/media/v4l2-core/v4l2-ctrls-core.c, specifically
in std_log() to log the compound types (conform VP8 patch 6/8).
<snip>
> diff --git a/include/linux/hantro-media.h b/include/linux/hantro-media.h
> new file mode 100644
> index 000000000000..db2791ab7c3b
> --- /dev/null
> +++ b/include/linux/hantro-media.h
Shouldn't this be in include/uapi?
> @@ -0,0 +1,17 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +
> +#ifndef __LINUX_HANTRO_MEDIA_H__
> +#define __LINUX_HANTRO_MEDIA_H__
> +
> +/*
> + * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
> + * the number of data (in bits) to skip in the
> + * slice segment header.
> + * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
> + * to before syntax element "slice_temporal_mvp_enabled_flag".
> + * If IDR, the skipped bits are just "pic_output_flag"
> + * (separate_colour_plane_flag is not supported).
> + */
> +#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_USER_HANTRO_BASE + 0)
> +
> +#endif
I any case, this should be in a separate patch as well.
> diff --git a/include/media/hevc-ctrls.h b/include/media/hevc-ctrls.h
> deleted file mode 100644
> index 01ccda48d8c5..000000000000
> --- a/include/media/hevc-ctrls.h
> +++ /dev/null
> @@ -1,250 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -/*
> - * These are the HEVC state controls for use with stateless HEVC
> - * codec drivers.
> - *
> - * It turns out that these structs are not stable yet and will undergo
> - * more changes. So keep them private until they are stable and ready to
> - * become part of the official public API.
> - */
> -
> -#ifndef _HEVC_CTRLS_H_
> -#define _HEVC_CTRLS_H_
> -
> -#include <linux/videodev2.h>
> -
> -/* The pixel format isn't stable at the moment and will likely be renamed. */
> -#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
The move of the pixel format to uAPI should be done in a separate patch.
See VP8 series patch 4/8.
> -
> -#define V4L2_CID_MPEG_VIDEO_HEVC_SPS (V4L2_CID_CODEC_BASE + 1008)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_PPS (V4L2_CID_CODEC_BASE + 1009)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_BASE + 1010)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_BASE + 1011)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_BASE + 1012)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE (V4L2_CID_CODEC_BASE + 1015)
> -#define V4L2_CID_MPEG_VIDEO_HEVC_START_CODE (V4L2_CID_CODEC_BASE + 1016)
> -
> -/* enum v4l2_ctrl_type type values */
> -#define V4L2_CTRL_TYPE_HEVC_SPS 0x0120
> -#define V4L2_CTRL_TYPE_HEVC_PPS 0x0121
> -#define V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS 0x0122
> -#define V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX 0x0123
> -#define V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS 0x0124
Moving the control types to uAPI is also a separate patch (as per
5/8 of the VP8 series).
> -
> -enum v4l2_mpeg_video_hevc_decode_mode {
> - V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_SLICE_BASED,
> - V4L2_MPEG_VIDEO_HEVC_DECODE_MODE_FRAME_BASED,
> -};
> -
> -enum v4l2_mpeg_video_hevc_start_code {
> - V4L2_MPEG_VIDEO_HEVC_START_CODE_NONE,
> - V4L2_MPEG_VIDEO_HEVC_START_CODE_ANNEX_B,
> -};
> -
> -#define V4L2_HEVC_SLICE_TYPE_B 0
> -#define V4L2_HEVC_SLICE_TYPE_P 1
> -#define V4L2_HEVC_SLICE_TYPE_I 2
> -
> -#define V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE (1ULL << 0)
> -#define V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED (1ULL << 1)
> -#define V4L2_HEVC_SPS_FLAG_AMP_ENABLED (1ULL << 2)
> -#define V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET (1ULL << 3)
> -#define V4L2_HEVC_SPS_FLAG_PCM_ENABLED (1ULL << 4)
> -#define V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED (1ULL << 5)
> -#define V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT (1ULL << 6)
> -#define V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED (1ULL << 7)
> -#define V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED (1ULL << 8)
> -
> -/* The controls are not stable at the moment and will likely be reworked. */
> -struct v4l2_ctrl_hevc_sps {
> - /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
> - __u16 pic_width_in_luma_samples;
> - __u16 pic_height_in_luma_samples;
> - __u8 bit_depth_luma_minus8;
> - __u8 bit_depth_chroma_minus8;
> - __u8 log2_max_pic_order_cnt_lsb_minus4;
> - __u8 sps_max_dec_pic_buffering_minus1;
> - __u8 sps_max_num_reorder_pics;
> - __u8 sps_max_latency_increase_plus1;
> - __u8 log2_min_luma_coding_block_size_minus3;
> - __u8 log2_diff_max_min_luma_coding_block_size;
> - __u8 log2_min_luma_transform_block_size_minus2;
> - __u8 log2_diff_max_min_luma_transform_block_size;
> - __u8 max_transform_hierarchy_depth_inter;
> - __u8 max_transform_hierarchy_depth_intra;
> - __u8 pcm_sample_bit_depth_luma_minus1;
> - __u8 pcm_sample_bit_depth_chroma_minus1;
> - __u8 log2_min_pcm_luma_coding_block_size_minus3;
> - __u8 log2_diff_max_min_pcm_luma_coding_block_size;
> - __u8 num_short_term_ref_pic_sets;
> - __u8 num_long_term_ref_pics_sps;
> - __u8 chroma_format_idc;
> - __u8 sps_max_sub_layers_minus1;
> -
> - __u64 flags;
> -};
> -
> -#define V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED (1ULL << 0)
> -#define V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT (1ULL << 1)
> -#define V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED (1ULL << 2)
> -#define V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT (1ULL << 3)
> -#define V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED (1ULL << 4)
> -#define V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED (1ULL << 5)
> -#define V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED (1ULL << 6)
> -#define V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT (1ULL << 7)
> -#define V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED (1ULL << 8)
> -#define V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED (1ULL << 9)
> -#define V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED (1ULL << 10)
> -#define V4L2_HEVC_PPS_FLAG_TILES_ENABLED (1ULL << 11)
> -#define V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED (1ULL << 12)
> -#define V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED (1ULL << 13)
> -#define V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 14)
> -#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED (1ULL << 15)
> -#define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER (1ULL << 16)
> -#define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT (1ULL << 17)
> -#define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL << 18)
> -#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
> -#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
> -
> -struct v4l2_ctrl_hevc_pps {
> - /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
> - __u8 num_extra_slice_header_bits;
> - __u8 num_ref_idx_l0_default_active_minus1;
> - __u8 num_ref_idx_l1_default_active_minus1;
> - __s8 init_qp_minus26;
> - __u8 diff_cu_qp_delta_depth;
> - __s8 pps_cb_qp_offset;
> - __s8 pps_cr_qp_offset;
> - __u8 num_tile_columns_minus1;
> - __u8 num_tile_rows_minus1;
> - __u8 column_width_minus1[20];
> - __u8 row_height_minus1[22];
> - __s8 pps_beta_offset_div2;
> - __s8 pps_tc_offset_div2;
> - __u8 log2_parallel_merge_level_minus2;
> -
> - __u8 padding[4];
> - __u64 flags;
> -};
> -
> -#define V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE 0x01
> -
> -#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX 16
> -
> -struct v4l2_hevc_dpb_entry {
> - __u64 timestamp;
> - __u8 flags;
> - __u8 field_pic;
> - __u16 pic_order_cnt[2];
> - __u8 padding[2];
> -};
> -
> -struct v4l2_hevc_pred_weight_table {
> - __s8 delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __s8 luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __s8 delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> - __s8 chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> -
> - __s8 delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __s8 luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __s8 delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> - __s8 chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> -
> - __u8 padding[6];
> -
> - __u8 luma_log2_weight_denom;
> - __s8 delta_chroma_log2_weight_denom;
> -};
> -
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_LUMA (1ULL << 0)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_CHROMA (1ULL << 1)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED (1ULL << 2)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_MVD_L1_ZERO (1ULL << 3)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_CABAC_INIT (1ULL << 4)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_COLLOCATED_FROM_L0 (1ULL << 5)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_USE_INTEGER_MV (1ULL << 6)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED (1ULL << 7)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 8)
> -#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
> -
> -struct v4l2_ctrl_hevc_slice_params {
> - __u32 bit_size;
> - __u32 data_bit_offset;
> -
> - /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
> - __u8 nal_unit_type;
> - __u8 nuh_temporal_id_plus1;
> -
> - /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
> - __u8 slice_type;
> - __u8 colour_plane_id;
> - __u16 slice_pic_order_cnt;
> - __u8 num_ref_idx_l0_active_minus1;
> - __u8 num_ref_idx_l1_active_minus1;
> - __u8 collocated_ref_idx;
> - __u8 five_minus_max_num_merge_cand;
> - __s8 slice_qp_delta;
> - __s8 slice_cb_qp_offset;
> - __s8 slice_cr_qp_offset;
> - __s8 slice_act_y_qp_offset;
> - __s8 slice_act_cb_qp_offset;
> - __s8 slice_act_cr_qp_offset;
> - __s8 slice_beta_offset_div2;
> - __s8 slice_tc_offset_div2;
> -
> - /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
> - __u8 pic_struct;
> -
> - /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
> - __u32 slice_segment_addr;
> - __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> -
> - __u8 padding;
> -
> - /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
> - struct v4l2_hevc_pred_weight_table pred_weight_table;
> -
> - __u64 flags;
> -};
> -
> -#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC 0x1
> -#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC 0x2
> -#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR 0x4
> -
> -struct v4l2_ctrl_hevc_decode_params {
> - __s32 pic_order_cnt_val;
> - __u8 num_active_dpb_entries;
> - struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __u8 num_poc_st_curr_before;
> - __u8 num_poc_st_curr_after;
> - __u8 num_poc_lt_curr;
> - __u8 poc_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __u8 poc_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __u8 poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> - __u64 flags;
> -};
> -
> -struct v4l2_ctrl_hevc_scaling_matrix {
> - __u8 scaling_list_4x4[6][16];
> - __u8 scaling_list_8x8[6][64];
> - __u8 scaling_list_16x16[6][64];
> - __u8 scaling_list_32x32[2][64];
> - __u8 scaling_list_dc_coef_16x16[6];
> - __u8 scaling_list_dc_coef_32x32[2];
> -};
> -
> -/* MPEG-class control IDs specific to the Hantro driver as defined by V4L2 */
> -#define V4L2_CID_CODEC_HANTRO_BASE (V4L2_CTRL_CLASS_CODEC | 0x1200)
> -/*
> - * V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP -
> - * the number of data (in bits) to skip in the
> - * slice segment header.
> - * If non-IDR, the bits to be skipped go from syntax element "pic_output_flag"
> - * to before syntax element "slice_temporal_mvp_enabled_flag".
> - * If IDR, the skipped bits are just "pic_output_flag"
> - * (separate_colour_plane_flag is not supported).
> - */
> -#define V4L2_CID_HANTRO_HEVC_SLICE_HEADER_SKIP (V4L2_CID_CODEC_HANTRO_BASE + 0)
> -
> -#endif
> diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h
> index f4105de8a8d2..00828a4f9404 100644
> --- a/include/media/v4l2-ctrls.h
> +++ b/include/media/v4l2-ctrls.h
> @@ -13,12 +13,6 @@
> #include <linux/videodev2.h>
> #include <media/media-request.h>
>
> -/*
> - * Include the stateless codec compound control definitions.
> - * This will move to the public headers once this API is fully stable.
> - */
> -#include <media/hevc-ctrls.h>
> -
> /* forward references */
> struct file;
> struct poll_table_struct;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index c8e0f84d204d..0e0ec2c61b80 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -218,6 +218,11 @@ enum v4l2_colorfx {
> * We reserve 16 controls for this driver.
> */
> #define V4L2_CID_USER_ALLEGRO_BASE (V4L2_CID_USER_BASE + 0x1170)
> +/*
> + * The base for Hantro driver controls.
> + * We reserve 128 controls for this driver.
> + */
> +#define V4L2_CID_USER_HANTRO_BASE (V4L2_CID_USER_BASE + 0x1180)
>
> /* MPEG-class control IDs */
> /* The MPEG controls are applicable to all codec controls
> @@ -2302,6 +2307,220 @@ struct v4l2_ctrl_vp9_compressed_hdr {
> struct v4l2_vp9_mv_probs mv;
> };
>
> +#define V4L2_CID_STATELESS_HEVC_SPS (V4L2_CID_CODEC_STATELESS_BASE + 400)
> +#define V4L2_CID_STATELESS_HEVC_PPS (V4L2_CID_CODEC_STATELESS_BASE + 401)
> +#define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 402)
> +#define V4L2_CID_STATELESS_HEVC_SCALING_MATRIX (V4L2_CID_CODEC_STATELESS_BASE + 403)
> +#define V4L2_CID_STATELESS_HEVC_DECODE_PARAMS (V4L2_CID_CODEC_STATELESS_BASE + 404)
> +#define V4L2_CID_STATELESS_HEVC_DECODE_MODE (V4L2_CID_CODEC_STATELESS_BASE + 405)
> +#define V4L2_CID_STATELESS_HEVC_START_CODE (V4L2_CID_CODEC_STATELESS_BASE + 406)
> +
> +enum v4l2_stateless_hevc_decode_mode {
> + V4L2_STATELESS_HEVC_DECODE_MODE_SLICE_BASED,
> + V4L2_STATELESS_HEVC_DECODE_MODE_FRAME_BASED,
> +};
> +
> +enum v4l2_stateless_hevc_start_code {
> + V4L2_STATELESS_HEVC_START_CODE_NONE,
> + V4L2_STATELESS_HEVC_START_CODE_ANNEX_B,
> +};
> +
> +#define V4L2_HEVC_SLICE_TYPE_B 0
> +#define V4L2_HEVC_SLICE_TYPE_P 1
> +#define V4L2_HEVC_SLICE_TYPE_I 2
> +
> +#define V4L2_HEVC_SPS_FLAG_SEPARATE_COLOUR_PLANE (1ULL << 0)
> +#define V4L2_HEVC_SPS_FLAG_SCALING_LIST_ENABLED (1ULL << 1)
> +#define V4L2_HEVC_SPS_FLAG_AMP_ENABLED (1ULL << 2)
> +#define V4L2_HEVC_SPS_FLAG_SAMPLE_ADAPTIVE_OFFSET (1ULL << 3)
> +#define V4L2_HEVC_SPS_FLAG_PCM_ENABLED (1ULL << 4)
> +#define V4L2_HEVC_SPS_FLAG_PCM_LOOP_FILTER_DISABLED (1ULL << 5)
> +#define V4L2_HEVC_SPS_FLAG_LONG_TERM_REF_PICS_PRESENT (1ULL << 6)
> +#define V4L2_HEVC_SPS_FLAG_SPS_TEMPORAL_MVP_ENABLED (1ULL << 7)
> +#define V4L2_HEVC_SPS_FLAG_STRONG_INTRA_SMOOTHING_ENABLED (1ULL << 8)
> +
> +struct v4l2_ctrl_hevc_sps {
> + /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
> + __u16 pic_width_in_luma_samples;
> + __u16 pic_height_in_luma_samples;
> + __u8 bit_depth_luma_minus8;
> + __u8 bit_depth_chroma_minus8;
> + __u8 log2_max_pic_order_cnt_lsb_minus4;
> + __u8 sps_max_dec_pic_buffering_minus1;
> + __u8 sps_max_num_reorder_pics;
> + __u8 sps_max_latency_increase_plus1;
> + __u8 log2_min_luma_coding_block_size_minus3;
> + __u8 log2_diff_max_min_luma_coding_block_size;
> + __u8 log2_min_luma_transform_block_size_minus2;
> + __u8 log2_diff_max_min_luma_transform_block_size;
> + __u8 max_transform_hierarchy_depth_inter;
> + __u8 max_transform_hierarchy_depth_intra;
> + __u8 pcm_sample_bit_depth_luma_minus1;
> + __u8 pcm_sample_bit_depth_chroma_minus1;
> + __u8 log2_min_pcm_luma_coding_block_size_minus3;
> + __u8 log2_diff_max_min_pcm_luma_coding_block_size;
> + __u8 num_short_term_ref_pic_sets;
> + __u8 num_long_term_ref_pics_sps;
> + __u8 chroma_format_idc;
> + __u8 sps_max_sub_layers_minus1;
> +
> + __u8 padding[6];
> + __u64 flags;
> + __u8 reserved[24];
> +};
> +
> +#define V4L2_HEVC_PPS_FLAG_DEPENDENT_SLICE_SEGMENT_ENABLED (1ULL << 0)
> +#define V4L2_HEVC_PPS_FLAG_OUTPUT_FLAG_PRESENT (1ULL << 1)
> +#define V4L2_HEVC_PPS_FLAG_SIGN_DATA_HIDING_ENABLED (1ULL << 2)
> +#define V4L2_HEVC_PPS_FLAG_CABAC_INIT_PRESENT (1ULL << 3)
> +#define V4L2_HEVC_PPS_FLAG_CONSTRAINED_INTRA_PRED (1ULL << 4)
> +#define V4L2_HEVC_PPS_FLAG_TRANSFORM_SKIP_ENABLED (1ULL << 5)
> +#define V4L2_HEVC_PPS_FLAG_CU_QP_DELTA_ENABLED (1ULL << 6)
> +#define V4L2_HEVC_PPS_FLAG_PPS_SLICE_CHROMA_QP_OFFSETS_PRESENT (1ULL << 7)
> +#define V4L2_HEVC_PPS_FLAG_WEIGHTED_PRED (1ULL << 8)
> +#define V4L2_HEVC_PPS_FLAG_WEIGHTED_BIPRED (1ULL << 9)
> +#define V4L2_HEVC_PPS_FLAG_TRANSQUANT_BYPASS_ENABLED (1ULL << 10)
> +#define V4L2_HEVC_PPS_FLAG_TILES_ENABLED (1ULL << 11)
> +#define V4L2_HEVC_PPS_FLAG_ENTROPY_CODING_SYNC_ENABLED (1ULL << 12)
> +#define V4L2_HEVC_PPS_FLAG_LOOP_FILTER_ACROSS_TILES_ENABLED (1ULL << 13)
> +#define V4L2_HEVC_PPS_FLAG_PPS_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 14)
> +#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_OVERRIDE_ENABLED (1ULL << 15)
> +#define V4L2_HEVC_PPS_FLAG_PPS_DISABLE_DEBLOCKING_FILTER (1ULL << 16)
> +#define V4L2_HEVC_PPS_FLAG_LISTS_MODIFICATION_PRESENT (1ULL << 17)
> +#define V4L2_HEVC_PPS_FLAG_SLICE_SEGMENT_HEADER_EXTENSION_PRESENT (1ULL << 18)
> +#define V4L2_HEVC_PPS_FLAG_DEBLOCKING_FILTER_CONTROL_PRESENT (1ULL << 19)
> +#define V4L2_HEVC_PPS_FLAG_UNIFORM_SPACING (1ULL << 20)
> +
> +struct v4l2_ctrl_hevc_pps {
> + /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
> + __u8 num_extra_slice_header_bits;
> + __u8 num_ref_idx_l0_default_active_minus1;
> + __u8 num_ref_idx_l1_default_active_minus1;
> + __s8 init_qp_minus26;
> + __u8 diff_cu_qp_delta_depth;
> + __s8 pps_cb_qp_offset;
> + __s8 pps_cr_qp_offset;
> + __u8 num_tile_columns_minus1;
> + __u8 num_tile_rows_minus1;
> + __u8 column_width_minus1[20];
> + __u8 row_height_minus1[22];
> + __s8 pps_beta_offset_div2;
> + __s8 pps_tc_offset_div2;
> + __u8 log2_parallel_merge_level_minus2;
> + __u8 padding[9];
> +
> + __u64 flags;
> + __u8 reserved[56];
> +};
> +
> +#define V4L2_HEVC_DPB_ENTRY_LONG_TERM_REFERENCE 0x01
> +
> +#define V4L2_HEVC_DPB_ENTRIES_NUM_MAX 16
> +
> +struct v4l2_hevc_dpb_entry {
> + __u64 timestamp;
> + __u8 flags;
> + __u8 field_pic;
> + __u16 pic_order_cnt[2];
> + __u8 padding[2];
> +};
> +
> +struct v4l2_hevc_pred_weight_table {
> + __s8 delta_luma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __s8 luma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __s8 delta_chroma_weight_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> + __s8 chroma_offset_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> +
> + __s8 delta_luma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __s8 luma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __s8 delta_chroma_weight_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> + __s8 chroma_offset_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX][2];
> +
> + __u8 padding[6];
> +
> + __u8 luma_log2_weight_denom;
> + __s8 delta_chroma_log2_weight_denom;
> +};
> +
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_LUMA (1ULL << 0)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_SAO_CHROMA (1ULL << 1)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_TEMPORAL_MVP_ENABLED (1ULL << 2)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_MVD_L1_ZERO (1ULL << 3)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_CABAC_INIT (1ULL << 4)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_COLLOCATED_FROM_L0 (1ULL << 5)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_USE_INTEGER_MV (1ULL << 6)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_DEBLOCKING_FILTER_DISABLED (1ULL << 7)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_SLICE_LOOP_FILTER_ACROSS_SLICES_ENABLED (1ULL << 8)
> +#define V4L2_HEVC_SLICE_PARAMS_FLAG_DEPENDENT_SLICE_SEGMENT (1ULL << 9)
> +
> +struct v4l2_ctrl_hevc_slice_params {
> + __u32 bit_size;
> + __u32 data_bit_offset;
> +
> + /* ISO/IEC 23008-2, ITU-T Rec. H.265: NAL unit header */
> + __u8 nal_unit_type;
> + __u8 nuh_temporal_id_plus1;
> +
> + /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
> + __u8 slice_type;
> + __u8 colour_plane_id;
> + __u16 slice_pic_order_cnt;
> + __u8 num_ref_idx_l0_active_minus1;
> + __u8 num_ref_idx_l1_active_minus1;
> + __u8 collocated_ref_idx;
> + __u8 five_minus_max_num_merge_cand;
> + __s8 slice_qp_delta;
> + __s8 slice_cb_qp_offset;
> + __s8 slice_cr_qp_offset;
> + __s8 slice_act_y_qp_offset;
> + __s8 slice_act_cb_qp_offset;
> + __s8 slice_act_cr_qp_offset;
> + __s8 slice_beta_offset_div2;
> + __s8 slice_tc_offset_div2;
> +
> + /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture timing SEI message */
> + __u8 pic_struct;
> +
> + __u8 reserved;
> +
> + /* ISO/IEC 23008-2, ITU-T Rec. H.265: General slice segment header */
> + __u32 slice_segment_addr;
> + __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> +
> + /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
> + struct v4l2_hevc_pred_weight_table pred_weight_table;
> +
> + __u64 flags;
> + __u8 padding[48];
> +};
> +
> +#define V4L2_HEVC_DECODE_PARAM_FLAG_IRAP_PIC 0x1
> +#define V4L2_HEVC_DECODE_PARAM_FLAG_IDR_PIC 0x2
> +#define V4L2_HEVC_DECODE_PARAM_FLAG_NO_OUTPUT_OF_PRIOR 0x4
> +
> +struct v4l2_ctrl_hevc_decode_params {
> + __s32 pic_order_cnt_val;
> + __u8 num_active_dpb_entries;
> + __u8 num_poc_st_curr_before;
> + __u8 num_poc_st_curr_after;
> + __u8 num_poc_lt_curr;
> + __u8 poc_st_curr_before[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __u8 poc_st_curr_after[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __u8 poc_lt_curr[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + struct v4l2_hevc_dpb_entry dpb[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
> + __u64 flags;
> +};
> +
> +struct v4l2_ctrl_hevc_scaling_matrix {
> + __u8 scaling_list_4x4[6][16];
> + __u8 scaling_list_8x8[6][64];
> + __u8 scaling_list_16x16[6][64];
> + __u8 scaling_list_32x32[2][64];
> + __u8 scaling_list_dc_coef_16x16[6];
> + __u8 scaling_list_dc_coef_32x32[2];
> +};
> +
> /* MPEG-compression definitions kept for backwards compatibility */
> #ifndef __KERNEL__
> #define V4L2_CTRL_CLASS_MPEG V4L2_CTRL_CLASS_CODEC
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index e27c8eae78c9..9af1c2460148 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -708,6 +708,7 @@ struct v4l2_pix_format {
> #define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh Hadamard Transform (vicodec) */
> #define V4L2_PIX_FMT_FWHT_STATELESS v4l2_fourcc('S', 'F', 'W', 'H') /* Stateless FWHT (vicodec) */
> #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
> +#define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
>
> /* Vendor-specific formats */
> #define V4L2_PIX_FMT_CPIA1 v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1829,6 +1830,12 @@ enum v4l2_ctrl_type {
>
> V4L2_CTRL_TYPE_VP9_COMPRESSED_HDR = 0x0260,
> V4L2_CTRL_TYPE_VP9_FRAME = 0x0261,
> +
> + V4L2_CTRL_TYPE_HEVC_SPS = 0x0270,
> + V4L2_CTRL_TYPE_HEVC_PPS = 0x0271,
> + V4L2_CTRL_TYPE_HEVC_SLICE_PARAMS = 0x0272,
> + V4L2_CTRL_TYPE_HEVC_SCALING_MATRIX = 0x0273,
> + V4L2_CTRL_TYPE_HEVC_DECODE_PARAMS = 0x0274,
> };
>
> /* Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
Regards,
Hans
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver
2022-02-18 11:32 ` Hans Verkuil
@ 2022-02-18 12:19 ` Benjamin Gaignard
2022-02-18 12:22 ` Hans Verkuil
0 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-18 12:19 UTC (permalink / raw)
To: Hans Verkuil, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Le 18/02/2022 à 12:32, Hans Verkuil a écrit :
> On 15/02/2022 12:01, Benjamin Gaignard wrote:
>> RKVDEC driver requires additional fields to perform HEVC decoding.
>> Even if the driver isn't mainlined yet WIP patches could be find here:
>> https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
>>
>> This patch only include the change in HEVC uAPI.
>>
>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>> ---
>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 16 ++++++++++++++++
>> include/uapi/linux/v4l2-controls.h | 5 +++++
>> 2 files changed, 21 insertions(+)
>>
>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> index 4f3b3ba8319f..3296ac3b9fca 100644
>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>> @@ -2661,6 +2661,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> :stub-columns: 0
>> :widths: 1 1 2
>>
>> + * - __u8
>> + - ``video_parameter_set_id``
>> + - Identifies the VPS for reference by other syntax elements.
>> + * - __u8
>> + - ``seq_parameter_set_id``
>> + - Provides an identifier for the SPS for reference by other syntax
>> + elements.
>> * - __u16
>> - ``pic_width_in_luma_samples``
>> -
>> @@ -2800,6 +2807,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> :stub-columns: 0
>> :widths: 1 1 2
>>
>> + * - __u8
>> + - ``pic_parameter_set_id``
>> + - Identifies the PPS for reference by other syntax elements.
>> * - __u8
>> - ``num_extra_slice_header_bits``
>> -
>> @@ -3026,6 +3036,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>> * - __u8
>> - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>> - The list of L1 reference elements as indices in the DPB.
>> + * - __u16
>> + - ``short_term_ref_pic_set_size``
>> + -
>> + * - __u16
>> + - ``long_term_ref_pic_set_size``
>> + -
>> * - __u8
>> - ``padding``
>> - Applications and drivers must set this to zero.
> Just to confirm: these additional fields are all from the H.265 spec, right?
> They are not rkvdec specific.
They are in H.265 spec section "7.4.3.2.2 Sequence parameter set range extension semantics":
- num_short_term_ref_pic_sets specifies the number of st_ref_pic_set( ) syntax structures included in the SPS. The value
of num_short_term_ref_pic_sets shall be in the range of 0 to 64, inclusive.
- num_long_term_ref_pics_sps specifies the number of candidate long-term reference pictures that are specified in the
SPS. The value of num_long_term_ref_pics_sps shall be in the range of 0 to 32, inclusive.
I mention rkvdec because that it is the only driver to use they (as far I knows)
Regards,
Benjamin
>
> Regards,
>
> Hans
>
>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>> index 0e0ec2c61b80..b1a3dc05f02f 100644
>> --- a/include/uapi/linux/v4l2-controls.h
>> +++ b/include/uapi/linux/v4l2-controls.h
>> @@ -2341,6 +2341,8 @@ enum v4l2_stateless_hevc_start_code {
>>
>> struct v4l2_ctrl_hevc_sps {
>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
>> + __u8 video_parameter_set_id;
>> + __u8 seq_parameter_set_id;
>> __u16 pic_width_in_luma_samples;
>> __u16 pic_height_in_luma_samples;
>> __u8 bit_depth_luma_minus8;
>> @@ -2393,6 +2395,7 @@ struct v4l2_ctrl_hevc_sps {
>>
>> struct v4l2_ctrl_hevc_pps {
>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
>> + __u8 pic_parameter_set_id;
>> __u8 num_extra_slice_header_bits;
>> __u8 num_ref_idx_l0_default_active_minus1;
>> __u8 num_ref_idx_l1_default_active_minus1;
>> @@ -2487,6 +2490,8 @@ struct v4l2_ctrl_hevc_slice_params {
>> __u32 slice_segment_addr;
>> __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>> __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>> + __u16 short_term_ref_pic_set_size;
>> + __u16 long_term_ref_pic_set_size;
>>
>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
>> struct v4l2_hevc_pred_weight_table pred_weight_table;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver
2022-02-18 12:19 ` Benjamin Gaignard
@ 2022-02-18 12:22 ` Hans Verkuil
2022-02-18 12:30 ` Benjamin Gaignard
0 siblings, 1 reply; 35+ messages in thread
From: Hans Verkuil @ 2022-02-18 12:22 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
On 18/02/2022 13:19, Benjamin Gaignard wrote:
>
> Le 18/02/2022 à 12:32, Hans Verkuil a écrit :
>> On 15/02/2022 12:01, Benjamin Gaignard wrote:
>>> RKVDEC driver requires additional fields to perform HEVC decoding.
>>> Even if the driver isn't mainlined yet WIP patches could be find here:
>>> https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
>>>
>>> This patch only include the change in HEVC uAPI.
>>>
>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>> ---
>>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 16 ++++++++++++++++
>>> include/uapi/linux/v4l2-controls.h | 5 +++++
>>> 2 files changed, 21 insertions(+)
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> index 4f3b3ba8319f..3296ac3b9fca 100644
>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>> @@ -2661,6 +2661,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>> :stub-columns: 0
>>> :widths: 1 1 2
>>> + * - __u8
>>> + - ``video_parameter_set_id``
>>> + - Identifies the VPS for reference by other syntax elements.
>>> + * - __u8
>>> + - ``seq_parameter_set_id``
>>> + - Provides an identifier for the SPS for reference by other syntax
>>> + elements.
>>> * - __u16
>>> - ``pic_width_in_luma_samples``
>>> -
>>> @@ -2800,6 +2807,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>> :stub-columns: 0
>>> :widths: 1 1 2
>>> + * - __u8
>>> + - ``pic_parameter_set_id``
>>> + - Identifies the PPS for reference by other syntax elements.
>>> * - __u8
>>> - ``num_extra_slice_header_bits``
>>> -
>>> @@ -3026,6 +3036,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>> * - __u8
>>> - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>>> - The list of L1 reference elements as indices in the DPB.
>>> + * - __u16
>>> + - ``short_term_ref_pic_set_size``
>>> + -
>>> + * - __u16
>>> + - ``long_term_ref_pic_set_size``
>>> + -
>>> * - __u8
>>> - ``padding``
>>> - Applications and drivers must set this to zero.
>> Just to confirm: these additional fields are all from the H.265 spec, right?
>> They are not rkvdec specific.
>
> They are in H.265 spec section "7.4.3.2.2 Sequence parameter set range extension semantics":
> - num_short_term_ref_pic_sets specifies the number of st_ref_pic_set( ) syntax structures included in the SPS. The value
> of num_short_term_ref_pic_sets shall be in the range of 0 to 64, inclusive.
>
> - num_long_term_ref_pics_sps specifies the number of candidate long-term reference pictures that are specified in the
> SPS. The value of num_long_term_ref_pics_sps shall be in the range of 0 to 32, inclusive.
And what about video/seq/pic_parameter_set_id?
Regards,
Hans
>
> I mention rkvdec because that it is the only driver to use they (as far I knows)
>
> Regards,
> Benjamin
>
>>
>> Regards,
>>
>> Hans
>>
>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>> index 0e0ec2c61b80..b1a3dc05f02f 100644
>>> --- a/include/uapi/linux/v4l2-controls.h
>>> +++ b/include/uapi/linux/v4l2-controls.h
>>> @@ -2341,6 +2341,8 @@ enum v4l2_stateless_hevc_start_code {
>>> struct v4l2_ctrl_hevc_sps {
>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
>>> + __u8 video_parameter_set_id;
>>> + __u8 seq_parameter_set_id;
>>> __u16 pic_width_in_luma_samples;
>>> __u16 pic_height_in_luma_samples;
>>> __u8 bit_depth_luma_minus8;
>>> @@ -2393,6 +2395,7 @@ struct v4l2_ctrl_hevc_sps {
>>> struct v4l2_ctrl_hevc_pps {
>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
>>> + __u8 pic_parameter_set_id;
>>> __u8 num_extra_slice_header_bits;
>>> __u8 num_ref_idx_l0_default_active_minus1;
>>> __u8 num_ref_idx_l1_default_active_minus1;
>>> @@ -2487,6 +2490,8 @@ struct v4l2_ctrl_hevc_slice_params {
>>> __u32 slice_segment_addr;
>>> __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>> __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>> + __u16 short_term_ref_pic_set_size;
>>> + __u16 long_term_ref_pic_set_size;
>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
>>> struct v4l2_hevc_pred_weight_table pred_weight_table;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver
2022-02-18 12:22 ` Hans Verkuil
@ 2022-02-18 12:30 ` Benjamin Gaignard
2022-02-18 12:32 ` Hans Verkuil
0 siblings, 1 reply; 35+ messages in thread
From: Benjamin Gaignard @ 2022-02-18 12:30 UTC (permalink / raw)
To: Hans Verkuil, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
Le 18/02/2022 à 13:22, Hans Verkuil a écrit :
> On 18/02/2022 13:19, Benjamin Gaignard wrote:
>> Le 18/02/2022 à 12:32, Hans Verkuil a écrit :
>>> On 15/02/2022 12:01, Benjamin Gaignard wrote:
>>>> RKVDEC driver requires additional fields to perform HEVC decoding.
>>>> Even if the driver isn't mainlined yet WIP patches could be find here:
>>>> https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
>>>>
>>>> This patch only include the change in HEVC uAPI.
>>>>
>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>> ---
>>>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 16 ++++++++++++++++
>>>> include/uapi/linux/v4l2-controls.h | 5 +++++
>>>> 2 files changed, 21 insertions(+)
>>>>
>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> index 4f3b3ba8319f..3296ac3b9fca 100644
>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>> @@ -2661,6 +2661,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>> :stub-columns: 0
>>>> :widths: 1 1 2
>>>> + * - __u8
>>>> + - ``video_parameter_set_id``
>>>> + - Identifies the VPS for reference by other syntax elements.
>>>> + * - __u8
>>>> + - ``seq_parameter_set_id``
>>>> + - Provides an identifier for the SPS for reference by other syntax
>>>> + elements.
>>>> * - __u16
>>>> - ``pic_width_in_luma_samples``
>>>> -
>>>> @@ -2800,6 +2807,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>> :stub-columns: 0
>>>> :widths: 1 1 2
>>>> + * - __u8
>>>> + - ``pic_parameter_set_id``
>>>> + - Identifies the PPS for reference by other syntax elements.
>>>> * - __u8
>>>> - ``num_extra_slice_header_bits``
>>>> -
>>>> @@ -3026,6 +3036,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>> * - __u8
>>>> - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>>>> - The list of L1 reference elements as indices in the DPB.
>>>> + * - __u16
>>>> + - ``short_term_ref_pic_set_size``
>>>> + -
>>>> + * - __u16
>>>> + - ``long_term_ref_pic_set_size``
>>>> + -
>>>> * - __u8
>>>> - ``padding``
>>>> - Applications and drivers must set this to zero.
>>> Just to confirm: these additional fields are all from the H.265 spec, right?
>>> They are not rkvdec specific.
>> They are in H.265 spec section "7.4.3.2.2 Sequence parameter set range extension semantics":
>> - num_short_term_ref_pic_sets specifies the number of st_ref_pic_set( ) syntax structures included in the SPS. The value
>> of num_short_term_ref_pic_sets shall be in the range of 0 to 64, inclusive.
>>
>> - num_long_term_ref_pics_sps specifies the number of candidate long-term reference pictures that are specified in the
>> SPS. The value of num_long_term_ref_pics_sps shall be in the range of 0 to 32, inclusive.
> And what about video/seq/pic_parameter_set_id?
It is the same they come from section "7.4.3.2.1 General sequence parameter set RBSP semantics":
- sps_video_parameter_set_id specifies the value of the vps_video_parameter_set_id of the active VPS.
- sps_seq_parameter_set_id provides an identifier for the SPS for reference by other syntax elements.
The value of
sps_seq_parameter_set_id shall be in the range of 0 to 15, inclusive.
Regards,
Benjamin
>
> Regards,
>
> Hans
>
>> I mention rkvdec because that it is the only driver to use they (as far I knows)
>>
>> Regards,
>> Benjamin
>>
>>> Regards,
>>>
>>> Hans
>>>
>>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>>> index 0e0ec2c61b80..b1a3dc05f02f 100644
>>>> --- a/include/uapi/linux/v4l2-controls.h
>>>> +++ b/include/uapi/linux/v4l2-controls.h
>>>> @@ -2341,6 +2341,8 @@ enum v4l2_stateless_hevc_start_code {
>>>> struct v4l2_ctrl_hevc_sps {
>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
>>>> + __u8 video_parameter_set_id;
>>>> + __u8 seq_parameter_set_id;
>>>> __u16 pic_width_in_luma_samples;
>>>> __u16 pic_height_in_luma_samples;
>>>> __u8 bit_depth_luma_minus8;
>>>> @@ -2393,6 +2395,7 @@ struct v4l2_ctrl_hevc_sps {
>>>> struct v4l2_ctrl_hevc_pps {
>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
>>>> + __u8 pic_parameter_set_id;
>>>> __u8 num_extra_slice_header_bits;
>>>> __u8 num_ref_idx_l0_default_active_minus1;
>>>> __u8 num_ref_idx_l1_default_active_minus1;
>>>> @@ -2487,6 +2490,8 @@ struct v4l2_ctrl_hevc_slice_params {
>>>> __u32 slice_segment_addr;
>>>> __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>> __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>> + __u16 short_term_ref_pic_set_size;
>>>> + __u16 long_term_ref_pic_set_size;
>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
>>>> struct v4l2_hevc_pred_weight_table pred_weight_table;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver
2022-02-18 12:30 ` Benjamin Gaignard
@ 2022-02-18 12:32 ` Hans Verkuil
0 siblings, 0 replies; 35+ messages in thread
From: Hans Verkuil @ 2022-02-18 12:32 UTC (permalink / raw)
To: Benjamin Gaignard, mchehab, ezequiel, p.zabel, gregkh, mripard,
paul.kocialkowski, wens, jernej.skrabec, jonas, nicolas
Cc: linux-media, linux-kernel, linux-staging, linux-arm-kernel,
linux-sunxi, kernel, knaerzche, jc
On 18/02/2022 13:30, Benjamin Gaignard wrote:
>
> Le 18/02/2022 à 13:22, Hans Verkuil a écrit :
>> On 18/02/2022 13:19, Benjamin Gaignard wrote:
>>> Le 18/02/2022 à 12:32, Hans Verkuil a écrit :
>>>> On 15/02/2022 12:01, Benjamin Gaignard wrote:
>>>>> RKVDEC driver requires additional fields to perform HEVC decoding.
>>>>> Even if the driver isn't mainlined yet WIP patches could be find here:
>>>>> https://github.com/LibreELEC/LibreELEC.tv/blob/master/projects/Rockchip/patches/linux/default/linux-2000-v4l2-wip-rkvdec-hevc.patch
>>>>>
>>>>> This patch only include the change in HEVC uAPI.
>>>>>
>>>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com>
>>>>> ---
>>>>> .../userspace-api/media/v4l/ext-ctrls-codec.rst | 16 ++++++++++++++++
>>>>> include/uapi/linux/v4l2-controls.h | 5 +++++
>>>>> 2 files changed, 21 insertions(+)
>>>>>
>>>>> diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>>> index 4f3b3ba8319f..3296ac3b9fca 100644
>>>>> --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>>> +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst
>>>>> @@ -2661,6 +2661,13 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>>> :stub-columns: 0
>>>>> :widths: 1 1 2
>>>>> + * - __u8
>>>>> + - ``video_parameter_set_id``
>>>>> + - Identifies the VPS for reference by other syntax elements.
>>>>> + * - __u8
>>>>> + - ``seq_parameter_set_id``
>>>>> + - Provides an identifier for the SPS for reference by other syntax
>>>>> + elements.
>>>>> * - __u16
>>>>> - ``pic_width_in_luma_samples``
>>>>> -
>>>>> @@ -2800,6 +2807,9 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>>> :stub-columns: 0
>>>>> :widths: 1 1 2
>>>>> + * - __u8
>>>>> + - ``pic_parameter_set_id``
>>>>> + - Identifies the PPS for reference by other syntax elements.
>>>>> * - __u8
>>>>> - ``num_extra_slice_header_bits``
>>>>> -
>>>>> @@ -3026,6 +3036,12 @@ enum v4l2_mpeg_video_hevc_size_of_length_field -
>>>>> * - __u8
>>>>> - ``ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX]``
>>>>> - The list of L1 reference elements as indices in the DPB.
>>>>> + * - __u16
>>>>> + - ``short_term_ref_pic_set_size``
>>>>> + -
>>>>> + * - __u16
>>>>> + - ``long_term_ref_pic_set_size``
>>>>> + -
>>>>> * - __u8
>>>>> - ``padding``
>>>>> - Applications and drivers must set this to zero.
>>>> Just to confirm: these additional fields are all from the H.265 spec, right?
>>>> They are not rkvdec specific.
>>> They are in H.265 spec section "7.4.3.2.2 Sequence parameter set range extension semantics":
>>> - num_short_term_ref_pic_sets specifies the number of st_ref_pic_set( ) syntax structures included in the SPS. The value
>>> of num_short_term_ref_pic_sets shall be in the range of 0 to 64, inclusive.
>>>
>>> - num_long_term_ref_pics_sps specifies the number of candidate long-term reference pictures that are specified in the
>>> SPS. The value of num_long_term_ref_pics_sps shall be in the range of 0 to 32, inclusive.
>> And what about video/seq/pic_parameter_set_id?
>
> It is the same they come from section "7.4.3.2.1 General sequence parameter set RBSP semantics":
> - sps_video_parameter_set_id specifies the value of the vps_video_parameter_set_id of the active VPS.
> - sps_seq_parameter_set_id provides an identifier for the SPS for reference by other syntax elements.
> The value of
> sps_seq_parameter_set_id shall be in the range of 0 to 15, inclusive.
Then I'm satisfied :-)
Thanks!
Hans
>
> Regards,
> Benjamin
>
>>
>> Regards,
>>
>> Hans
>>
>>> I mention rkvdec because that it is the only driver to use they (as far I knows)
>>>
>>> Regards,
>>> Benjamin
>>>
>>>> Regards,
>>>>
>>>> Hans
>>>>
>>>>> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
>>>>> index 0e0ec2c61b80..b1a3dc05f02f 100644
>>>>> --- a/include/uapi/linux/v4l2-controls.h
>>>>> +++ b/include/uapi/linux/v4l2-controls.h
>>>>> @@ -2341,6 +2341,8 @@ enum v4l2_stateless_hevc_start_code {
>>>>> struct v4l2_ctrl_hevc_sps {
>>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Sequence parameter set */
>>>>> + __u8 video_parameter_set_id;
>>>>> + __u8 seq_parameter_set_id;
>>>>> __u16 pic_width_in_luma_samples;
>>>>> __u16 pic_height_in_luma_samples;
>>>>> __u8 bit_depth_luma_minus8;
>>>>> @@ -2393,6 +2395,7 @@ struct v4l2_ctrl_hevc_sps {
>>>>> struct v4l2_ctrl_hevc_pps {
>>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Picture parameter set */
>>>>> + __u8 pic_parameter_set_id;
>>>>> __u8 num_extra_slice_header_bits;
>>>>> __u8 num_ref_idx_l0_default_active_minus1;
>>>>> __u8 num_ref_idx_l1_default_active_minus1;
>>>>> @@ -2487,6 +2490,8 @@ struct v4l2_ctrl_hevc_slice_params {
>>>>> __u32 slice_segment_addr;
>>>>> __u8 ref_idx_l0[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>>> __u8 ref_idx_l1[V4L2_HEVC_DPB_ENTRIES_NUM_MAX];
>>>>> + __u16 short_term_ref_pic_set_size;
>>>>> + __u16 long_term_ref_pic_set_size;
>>>>> /* ISO/IEC 23008-2, ITU-T Rec. H.265: Weighted prediction parameter */
>>>>> struct v4l2_hevc_pred_weight_table pred_weight_table;
^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2022-02-18 12:32 UTC | newest]
Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-15 11:00 [RFC v2 0/8] Move HEVC stateless controls out of staging Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 1/8] videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 2/8] v4l2-ctrls: add support for dynamically allocated arrays Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 3/8] vivid: add dynamic array test control Benjamin Gaignard
2022-02-15 11:00 ` [RFC v2 4/8] media: uapi: Move HEVC stateless controls out of staging Benjamin Gaignard
2022-02-15 18:02 ` Sebastian Fricke
2022-02-18 11:50 ` Hans Verkuil
2022-02-15 11:01 ` [RFC v2 5/8] media: uapi: Add fields needed for RKVDEC driver Benjamin Gaignard
2022-02-18 11:32 ` Hans Verkuil
2022-02-18 12:19 ` Benjamin Gaignard
2022-02-18 12:22 ` Hans Verkuil
2022-02-18 12:30 ` Benjamin Gaignard
2022-02-18 12:32 ` Hans Verkuil
2022-02-15 11:01 ` [RFC v2 6/8] media: uapi: Remove bit_size field from v4l2_ctrl_hevc_slice_params Benjamin Gaignard
2022-02-15 14:17 ` John Cox
2022-02-15 14:35 ` Benjamin Gaignard
2022-02-15 14:50 ` John Cox
2022-02-15 14:57 ` Benjamin Gaignard
2022-02-15 15:28 ` Nicolas Dufresne
2022-02-15 16:00 ` John Cox
2022-02-15 16:11 ` Jernej Škrabec
2022-02-15 16:31 ` John Cox
2022-02-15 20:27 ` Jernej Škrabec
2022-02-16 10:38 ` John Cox
2022-02-15 11:01 ` [RFC v2 7/8] media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSET control Benjamin Gaignard
2022-02-16 10:54 ` John Cox
2022-02-16 17:19 ` Benjamin Gaignard
2022-02-16 17:35 ` John Cox
2022-02-15 11:01 ` [RFC v2 8/8] media: uapi: Change data_bit_offset definition Benjamin Gaignard
2022-02-15 13:01 ` [RFC v2 0/8] Move HEVC stateless controls out of staging Sebastian Fricke
2022-02-15 13:24 ` Benjamin Gaignard
2022-02-15 16:14 ` Jernej Škrabec
2022-02-18 9:59 ` Hans Verkuil
2022-02-18 10:54 ` Benjamin Gaignard
2022-02-18 11:05 ` Hans Verkuil
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).