* [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
@ 2023-02-06 4:33 ` ayaka
[not found] ` <1e3a583f-f9ef-d2d3-6571-ac3b79d800ce@synaptics.com>
2023-07-13 10:26 ` Tomasz Figa
2023-02-06 4:33 ` [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format ayaka
` (7 subsequent siblings)
8 siblings, 2 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Randy Li, Helen Koike
From: Randy Li <ayaka@soulik.info>
This is part of the multiplanar and singleplanar unification process.
v4l2_ext_pix_format is supposed to work for both cases.
We also add the concept of modifiers already employed in DRM to expose
HW-specific formats (like tiled or compressed formats) and allow
exchanging this information with the DRM subsystem in a consistent way.
Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
v4l2_ext_pix_format, other types will be rejected if you use the
{G,S,TRY}_EXT_PIX_FMT ioctls.
New hooks have been added to v4l2_ioctl_ops to support those new ioctls
in drivers, but, in the meantime, the core takes care of converting
{S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
still work if the userspace app/lib uses the new ioctls.
The conversion is also done the other around to allow userspace
apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
_ext_ hooks.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Signed-off-by: Randy Li <ayaka@soulik.info>
---
Changes in v7:
- use the resevered fields in v4l2_fmtdesc to store the modifier for format
enumerate.
- Dropping the old ioctl() calling new interfaces adapter.
- Refresh the patch
Changes in v6:
The main change here was fixing the conversion, so planes reflects color planes,
and to implement this properly I made major refactors compared to the previous
version.
- struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz)
- refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans)
- reorder colorimetry fields in struct v4l2_ext_pix_format (Hans)
- do not set Ext ioctls as valid for vid_out_overlay (Tomasz)
- refactor conversion functions, so planes are color planes (Tomasz)
- Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz)
- Use "ef" for extended formats in the framework for consistency (Tomasz)
- Handle xfer_func field in conversions (Tomasz)
- Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz)
- Refactor format functions to use v4l_fmt_ioctl_via_ext()
- Several fixes/refactoring/changes
- Remove EXT API for touch devices
Changes in v5:
- change sizes and reorder fields to avoid holes in the struct and make
it the same for 32 and 64 bits
- removed __attribute__ ((packed)) from uapi structs
- Fix doc warning from make htmldocs
- Updated commit message with EXT_PIX prefix for the ioctls.
Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Add reserved fields
- Removed num_planes from struct v4l2_ext_pix_format
- Removed flag field from struct v4l2_ext_pix_format, since the only
defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
where we can use modifiers, or add it back later through the reserved
bits.
- In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
!= MOD_INVALID
- Fix type assignment in v4l_g_fmt_ext_pix()
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- Move the modifier in v4l2_ext_format (was formerly placed in
v4l2_ext_plane)
- Fix a few bugs in the converters and add a strict parameter to
allow conversion of uninitialized/mis-initialized objects
---
drivers/media/v4l2-core/v4l2-dev.c | 13 +
drivers/media/v4l2-core/v4l2-ioctl.c | 357 ++++++++++++++++++++++++++-
include/media/v4l2-ioctl.h | 28 +++
include/uapi/linux/videodev2.h | 46 +++-
4 files changed, 438 insertions(+), 6 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index 397d553177fa..cfe90bfd47f1 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -615,6 +615,10 @@ static void determine_valid_ioctls(struct video_device *vdev)
ops->vidioc_g_fmt_vid_out_mplane ||
ops->vidioc_g_fmt_vid_out_overlay)))
__set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
+ if ((is_rx && ops->vidioc_g_ext_pix_fmt_vid_cap) ||
+ (is_tx && ops->vidioc_g_ext_pix_fmt_vid_out)) {
+ __set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls);
+ }
if ((is_rx && (ops->vidioc_s_fmt_vid_cap ||
ops->vidioc_s_fmt_vid_cap_mplane ||
ops->vidioc_s_fmt_vid_overlay)) ||
@@ -622,6 +626,11 @@ static void determine_valid_ioctls(struct video_device *vdev)
ops->vidioc_s_fmt_vid_out_mplane ||
ops->vidioc_s_fmt_vid_out_overlay)))
__set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls);
+
+ if ((is_rx && ops->vidioc_s_ext_pix_fmt_vid_cap) ||
+ (is_tx && ops->vidioc_s_ext_pix_fmt_vid_out)) {
+ __set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls);
+ }
if ((is_rx && (ops->vidioc_try_fmt_vid_cap ||
ops->vidioc_try_fmt_vid_cap_mplane ||
ops->vidioc_try_fmt_vid_overlay)) ||
@@ -629,6 +638,10 @@ static void determine_valid_ioctls(struct video_device *vdev)
ops->vidioc_try_fmt_vid_out_mplane ||
ops->vidioc_try_fmt_vid_out_overlay)))
__set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls);
+ if ((is_rx && ops->vidioc_try_ext_pix_fmt_vid_cap) ||
+ (is_tx && ops->vidioc_try_ext_pix_fmt_vid_out)) {
+ __set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls);
+ }
SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay);
SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf);
SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf);
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 87f163a89c80..52b77a968bb3 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -18,8 +18,10 @@
#include <linux/v4l2-subdev.h>
#include <linux/videodev2.h>
+#include <drm/drm_fourcc.h>
#include <media/media-device.h> /* for media_set_bus_info() */
+
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
#include <media/v4l2-ctrls.h>
@@ -34,6 +36,11 @@
#define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls)
+#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \
+ (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \
+ V4L2_CAP_VIDEO_OUTPUT_MPLANE | \
+ V4L2_CAP_VIDEO_M2M_MPLANE))
+
struct std_descr {
v4l2_std_id std;
const char *descr;
@@ -353,6 +360,27 @@ static void v4l_print_format(const void *arg, bool write_only)
}
}
+static void v4l_print_ext_pix_format(const void *arg, bool write_only)
+{
+ const struct v4l2_ext_pix_format *ef = arg;
+ unsigned int i;
+
+ pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n",
+ prt_names(ef->type, v4l2_type_names),
+ ef->width, ef->height,
+ (ef->pixelformat & 0xff),
+ (ef->pixelformat >> 8) & 0xff,
+ (ef->pixelformat >> 16) & 0xff,
+ (ef->pixelformat >> 24) & 0xff,
+ ef->modifier, prt_names(ef->field, v4l2_field_names),
+ ef->colorspace, ef->ycbcr_enc,
+ ef->quantization, ef->xfer_func);
+ for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
+ pr_debug("plane %u: bytesperline=%u sizeimage=%u\n",
+ i, ef->plane_fmt[i].bytesperline,
+ ef->plane_fmt[i].sizeimage);
+}
+
static void v4l_print_framebuffer(const void *arg, bool write_only)
{
const struct v4l2_framebuffer *p = arg;
@@ -940,7 +968,9 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
switch (type) {
case V4L2_BUF_TYPE_VIDEO_CAPTURE:
if ((is_vid || is_tch) && is_rx &&
- (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane))
+ (ops->vidioc_g_fmt_vid_cap ||
+ ops->vidioc_g_ext_pix_fmt_vid_cap ||
+ ops->vidioc_g_fmt_vid_cap_mplane))
return 0;
break;
case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
@@ -953,7 +983,9 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT:
if (is_vid && is_tx &&
- (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane))
+ (ops->vidioc_g_fmt_vid_out ||
+ ops->vidioc_g_ext_pix_fmt_vid_out ||
+ ops->vidioc_g_fmt_vid_out_mplane))
return 0;
break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
@@ -1082,6 +1114,203 @@ static void v4l_sanitize_format(struct v4l2_format *fmt)
}
}
+static void
+v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef,
+ struct v4l2_pix_format *pix)
+{
+ unsigned int i;
+
+ pix->width = ef->width;
+ pix->height = ef->height;
+ pix->field = ef->field;
+ pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC;
+ pix->colorspace = ef->colorspace;
+ pix->ycbcr_enc = ef->ycbcr_enc;
+ pix->priv = V4L2_PIX_FMT_PRIV_MAGIC;
+ pix->quantization = ef->quantization;
+ pix->pixelformat = ef->pixelformat;
+ pix->bytesperline = ef->plane_fmt[0].bytesperline;
+ pix->sizeimage = ef->plane_fmt[0].sizeimage;
+ for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
+ pix->sizeimage += ef->plane_fmt[i].sizeimage;
+}
+
+static void
+v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef,
+ struct v4l2_pix_format_mplane *pix_mp)
+{
+ const struct v4l2_format_info *info =
+ v4l2_format_info(ef->pixelformat);
+ unsigned int i;
+
+ pix_mp->width = ef->width;
+ pix_mp->height = ef->height;
+ pix_mp->field = ef->field;
+ pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC;
+ pix_mp->colorspace = ef->colorspace;
+ pix_mp->ycbcr_enc = ef->ycbcr_enc;
+ pix_mp->quantization = ef->quantization;
+ pix_mp->pixelformat = ef->pixelformat;
+
+ /* This is true when converting to non-M-variant */
+ if (info && info->mem_planes == 1) {
+ pix_mp->plane_fmt[0] = ef->plane_fmt[0];
+ pix_mp->num_planes = 1;
+ for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
+ pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage;
+
+ return;
+ }
+
+ for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
+ pix_mp->plane_fmt[i] = ef->plane_fmt[i];
+ pix_mp->num_planes = i;
+}
+
+/*
+ * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format
+ *
+ * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted.
+ * @f: A pointer to struct v4l2_format to be filled.
+ * @is_mplane: Bool indicating if multiplanar API should be used in @f.
+ *
+ * If pixelformat should be converted to M-variant, change ef->pixelformat
+ * to the M-variant before calling this function.
+ */
+static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef,
+ struct v4l2_format *f, bool is_mplane)
+{
+ memset(f, 0, sizeof(*f));
+
+ if (ef->modifier != DRM_FORMAT_MOD_LINEAR &&
+ ef->modifier != DRM_FORMAT_MOD_INVALID)
+ pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n",
+ ef->modifier);
+
+ if (!is_mplane) {
+ f->type = ef->type;
+ v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix);
+ return;
+ }
+
+ if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ else
+ f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+
+ v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp);
+}
+
+static void
+v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix,
+ struct v4l2_ext_pix_format *ef)
+{
+ const struct v4l2_format_info *info =
+ v4l2_format_info(pix->pixelformat);
+ unsigned int i;
+
+ ef->width = pix->width;
+ ef->height = pix->height;
+ ef->field = pix->field;
+ ef->colorspace = pix->colorspace;
+ ef->ycbcr_enc = pix->ycbcr_enc;
+ ef->quantization = pix->quantization;
+ ef->xfer_func = pix->xfer_func;
+ if (pix->flags)
+ pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags);
+
+ /* We assume M-variants won't be used in this function */
+ ef->pixelformat = pix->pixelformat;
+
+ ef->plane_fmt[0].bytesperline = pix->bytesperline;
+ ef->plane_fmt[0].sizeimage = pix->sizeimage;
+
+ if (!info)
+ return;
+
+ for (i = 1; i < info->comp_planes; i++) {
+ ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv;
+ ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline *
+ ef->height / info->vdiv;
+ ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage;
+ }
+}
+
+static void
+v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp,
+ struct v4l2_ext_pix_format *ef)
+{
+ const struct v4l2_format_info *info =
+ v4l2_format_info(pix_mp->pixelformat);
+ unsigned int i;
+
+ ef->width = pix_mp->width;
+ ef->height = pix_mp->height;
+ ef->field = pix_mp->field;
+ ef->colorspace = pix_mp->colorspace;
+ ef->ycbcr_enc = pix_mp->ycbcr_enc;
+ ef->quantization = pix_mp->quantization;
+ ef->xfer_func = pix_mp->xfer_func;
+ if (pix_mp->flags)
+ pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags);
+
+ if (!info)
+ return;
+
+ ef->pixelformat = pix_mp->pixelformat;
+
+ if (info->comp_planes == info->mem_planes) {
+ for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++)
+ ef->plane_fmt[i] = pix_mp->plane_fmt[i];
+
+ return;
+ }
+
+ /* case where mem_planes is 1 and comp_planes > 1 */
+ ef->plane_fmt[0] = pix_mp->plane_fmt[0];
+ for (i = 1; i < info->comp_planes; i++) {
+ ef->plane_fmt[i].bytesperline =
+ pix_mp->plane_fmt[0].bytesperline / info->hdiv;
+ ef->plane_fmt[i].sizeimage =
+ ef->plane_fmt[i].bytesperline * ef->height / info->vdiv;
+ ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage;
+ }
+}
+
+/*
+ * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format
+ *
+ * @f: A pointer to struct v4l2_format to be converted.
+ * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled.
+ *
+ * This method normalize the pixelformat to non-M variant.
+ */
+static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f,
+ struct v4l2_ext_pix_format *ef)
+{
+ memset(ef, 0, sizeof(*ef));
+
+ switch (f->type) {
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ ef->type = f->type;
+ v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef);
+ break;
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+ ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef);
+ break;
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+ ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
+ v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef);
+ break;
+ default:
+ WARN("Converting to Ext Pix Format with wrong buffer type %s\n",
+ prt_names(f->type, v4l2_type_names));
+ break;
+ }
+}
+
static int v4l_querycap(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -1654,7 +1883,9 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
v4l_pix_format_touch(&p->fmt.pix);
return ret;
case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
- return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg);
+ if (ops->vidioc_g_fmt_vid_cap_mplane)
+ return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg);
+ break;
case V4L2_BUF_TYPE_VIDEO_OVERLAY:
return ops->vidioc_g_fmt_vid_overlay(file, fh, arg);
case V4L2_BUF_TYPE_VBI_CAPTURE:
@@ -1662,7 +1893,8 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE:
return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg);
case V4L2_BUF_TYPE_VIDEO_OUTPUT:
- if (unlikely(!ops->vidioc_g_fmt_vid_out))
+ if (unlikely(!ops->vidioc_g_fmt_vid_out &&
+ !ops->vidioc_g_ext_pix_fmt_vid_out))
break;
p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC;
ret = ops->vidioc_g_fmt_vid_out(file, fh, arg);
@@ -1670,7 +1902,9 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC;
return ret;
case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
- return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg);
+ if (ops->vidioc_g_fmt_vid_out_mplane)
+ return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg);
+ break;
case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY:
return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg);
case V4L2_BUF_TYPE_VBI_OUTPUT:
@@ -1689,6 +1923,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
return -EINVAL;
}
+static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct v4l2_ext_pix_format *ef = arg;
+ struct v4l2_format f = {
+ .type = ef->type,
+ };
+ int ret = check_fmt(file, ef->type);
+
+ if (ret)
+ return ret;
+
+ memset(ef, 0, sizeof(*ef));
+ ef->type = f.type;
+
+ switch (f.type) {
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ if (ops->vidioc_g_ext_pix_fmt_vid_cap)
+ return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef);
+ break;
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ if (ops->vidioc_g_ext_pix_fmt_vid_out)
+ return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ret = v4l_g_fmt(ops, file, fh, &f);
+ if (ret)
+ return ret;
+
+ v4l2_format_to_ext_pix_format(&f, ef);
+ return 0;
+}
+
static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -1794,6 +2064,42 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops,
return -EINVAL;
}
+static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct video_device *vfd = video_devdata(file);
+ struct v4l2_ext_pix_format *ef = arg;
+ struct v4l2_format f;
+ int ret = check_fmt(file, ef->type);
+
+ if (ret)
+ return ret;
+
+ memset(ef->reserved, 0, sizeof(ef->reserved));
+
+ switch (ef->type) {
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ if (ops->vidioc_s_ext_pix_fmt_vid_cap)
+ return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef);
+ break;
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ if (ops->vidioc_s_ext_pix_fmt_vid_out)
+ return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd));
+
+ ret = v4l_s_fmt(ops, file, fh, &f);
+ if (ret)
+ return ret;
+
+ v4l2_format_to_ext_pix_format(&f, ef);
+ return 0;
+}
+
static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -1896,6 +2202,44 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops,
return -EINVAL;
}
+static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct video_device *vfd = video_devdata(file);
+ struct v4l2_ext_pix_format *ef = arg;
+ struct v4l2_format f;
+ int ret = check_fmt(file, ef->type);
+
+ if (ret)
+ return ret;
+
+ memset(ef->reserved, 0, sizeof(ef->reserved));
+
+ switch (ef->type) {
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ if (ops->vidioc_try_ext_pix_fmt_vid_cap)
+ return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh,
+ ef);
+ break;
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ if (ops->vidioc_try_ext_pix_fmt_vid_out)
+ return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh,
+ ef);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd));
+
+ ret = v4l_try_fmt(ops, file, fh, &f);
+ if (ret)
+ return ret;
+
+ v4l2_format_to_ext_pix_format(&f, ef);
+ return 0;
+}
+
static int v4l_streamon(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -2902,6 +3246,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
+ IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0),
+ IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO),
+ IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0),
};
#define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index edb733f21604..c44708dc9355 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -48,11 +48,17 @@ struct v4l2_fh;
* @vidioc_g_fmt_vid_cap: pointer to the function that implements
* :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture
* in single plane mode
+ * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements
+ * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
+ * capture
* @vidioc_g_fmt_vid_overlay: pointer to the function that implements
* :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay
* @vidioc_g_fmt_vid_out: pointer to the function that implements
* :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out
* in single plane mode
+ * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements
+ * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
+ * out
* @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements
* :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output
* @vidioc_g_fmt_vbi_cap: pointer to the function that implements
@@ -82,11 +88,16 @@ struct v4l2_fh;
* @vidioc_s_fmt_vid_cap: pointer to the function that implements
* :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture
* in single plane mode
+ * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements
+ * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
+ * capture
* @vidioc_s_fmt_vid_overlay: pointer to the function that implements
* :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay
* @vidioc_s_fmt_vid_out: pointer to the function that implements
* :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out
* in single plane mode
+ * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements
+ * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out
* @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements
* :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output
* @vidioc_s_fmt_vbi_cap: pointer to the function that implements
@@ -116,11 +127,16 @@ struct v4l2_fh;
* @vidioc_try_fmt_vid_cap: pointer to the function that implements
* :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture
* in single plane mode
+ * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements
+ * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for
+ video capture
* @vidioc_try_fmt_vid_overlay: pointer to the function that implements
* :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay
* @vidioc_try_fmt_vid_out: pointer to the function that implements
* :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out
* in single plane mode
+ * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements
+ * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out
* @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements
* :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay
* output
@@ -319,10 +335,14 @@ struct v4l2_ioctl_ops {
/* VIDIOC_G_FMT handlers */
int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh,
struct v4l2_format *f);
+ int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
+ struct v4l2_ext_pix_format *ef);
int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh,
struct v4l2_format *f);
int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh,
struct v4l2_format *f);
+ int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh,
+ struct v4l2_ext_pix_format *ef);
int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh,
struct v4l2_format *f);
int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh,
@@ -349,10 +369,14 @@ struct v4l2_ioctl_ops {
/* VIDIOC_S_FMT handlers */
int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh,
struct v4l2_format *f);
+ int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
+ struct v4l2_ext_pix_format *ef);
int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh,
struct v4l2_format *f);
int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh,
struct v4l2_format *f);
+ int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh,
+ struct v4l2_ext_pix_format *ef);
int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh,
struct v4l2_format *f);
int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh,
@@ -379,10 +403,14 @@ struct v4l2_ioctl_ops {
/* VIDIOC_TRY_FMT handlers */
int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh,
struct v4l2_format *f);
+ int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
+ struct v4l2_ext_pix_format *ef);
int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh,
struct v4l2_format *f);
int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh,
struct v4l2_format *f);
+ int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh,
+ struct v4l2_ext_pix_format *ef);
int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh,
struct v4l2_format *f);
int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh,
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 17a9b975177a..74a8dd7f7637 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -840,7 +840,8 @@ struct v4l2_fmtdesc {
__u8 description[32]; /* Description string */
__u32 pixelformat; /* Format fourcc */
__u32 mbus_code; /* Media bus code */
- __u32 reserved[3];
+ __u64 modifier; /* Format modifier */
+ __u32 reserved;
};
#define V4L2_FMT_FLAG_COMPRESSED 0x0001
@@ -2417,6 +2418,45 @@ struct v4l2_format {
} fmt;
};
+/**
+ * struct v4l2_ext_pix_format - extended multiplanar format definition
+ * @type: enum v4l2_buf_type; type of the data stream
+ * @width: image width in pixels
+ * @height: image height in pixels
+ * @pixelformat: little endian four character code (fourcc)
+ * @modifier: modifier applied to the format (used for tiled formats
+ * and other kind of HW-specific formats, like compressed
+ * formats) as defined in drm_fourcc.h
+ * @field: enum v4l2_field; field order (for interlaced video)
+ * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
+ * @plane_fmt: per-plane information
+ * @flags: format flags (V4L2_PIX_FMT_FLAG_*)
+ * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
+ * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding
+ * @quantization: enum v4l2_quantization, colorspace quantization
+ * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
+ * @reserved: drivers and applications must zero this array
+ */
+struct v4l2_ext_pix_format {
+ __u32 type;
+ __u32 width;
+ __u32 height;
+ __u32 pixelformat;
+ __u64 modifier;
+ __u32 field;
+ __u32 colorspace;
+
+ struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES];
+ __u8 flags;
+ union {
+ __u8 ycbcr_enc;
+ __u8 hsv_enc;
+ };
+ __u8 quantization;
+ __u8 xfer_func;
+ __u32 reserved[10];
+} __attribute__ ((packed));
+
/* Stream type-dependent parameters
*/
struct v4l2_streamparm {
@@ -2690,6 +2730,10 @@ struct v4l2_create_buffers {
#define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
+#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format)
+#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format)
+#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format)
+
/* Reminder: when adding new ioctls please add support for them to
drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread[parent not found: <1e3a583f-f9ef-d2d3-6571-ac3b79d800ce@synaptics.com>]
* Re: [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
[not found] ` <1e3a583f-f9ef-d2d3-6571-ac3b79d800ce@synaptics.com>
@ 2023-06-28 16:27 ` Nicolas Dufresne
2023-06-29 3:43 ` Hsia-Jun Li
0 siblings, 1 reply; 21+ messages in thread
From: Nicolas Dufresne @ 2023-06-28 16:27 UTC (permalink / raw)
To: Hsia-Jun Li, ayaka
Cc: linux-media, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, sakari.ailus,
stanimir.varbanov, tfiga, Helen Koike
Hi,
please avoid HTML replies on the mailing list.
Le mardi 27 juin 2023 à 14:42 +0800, Hsia-Jun Li a écrit :
> > +/**
> > + * struct v4l2_ext_pix_format - extended multiplanar format definition
> > + * @type: enum v4l2_buf_type; type of the data stream
> > + * @width: image width in pixels
> > + * @height: image height in pixels
> > + * @pixelformat: little endian four character code (fourcc)
> > + * @modifier: modifier applied to the format (used for tiled formats
> > + * and other kind of HW-specific formats, like compressed
> > + * formats) as defined in drm_fourcc.h
> > + * @field: enum v4l2_field; field order (for interlaced video)
> > + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
> > + * @plane_fmt: per-plane information
> > + * @flags: format flags (V4L2_PIX_FMT_FLAG_*)
> > + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
> > + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding
> > + * @quantization: enum v4l2_quantization, colorspace quantization
> > + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
> > + * @reserved: drivers and applications must zero this array
> > + */
> > +struct v4l2_ext_pix_format {
> > + __u32 type;
> > + __u32 width;
> > + __u32 height;
> > + __u32 pixelformat;
> > + __u64 modifier;
> > + __u32 field;
> > + __u32 colorspace;
> > +
> > + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES];
> > + __u8 flags;
> > + union {
> > + __u8 ycbcr_enc;
> > + __u8 hsv_enc;
> > + };
> > + __u8 quantization;
> > + __u8 xfer_func;
> >
> >
> >
> >
> >
>
> I heard that a suggestion that we could remove colorimetry fields
> here.
> Although those are useless for codec M2M drivers if no pixel
> formats translation invoked.
> Even HDMI(DRM) cares about colorspace.
> For example if a downsink(TV) shows RGB formats, with an YUV
> input frame buffer, colorimetry would be important or the wrong
> EOTF would be used. If YUV is MPEG range(linear EOTF) while a
> non-linear EOFT (full range) is used, you would found the black is
> not black enough while the white looks a gray. Also color bias
> would happen.
> This problem may not happen to a ultra high resolution TV while
> only YUV type color formats are supported due to HDMI bandwidth
> limitation.
> The problem I want to raise is the time cost for enumeration.
> Each pixel format with a colorimetry setting would invoke a
> ioctl(). For the application likes Gstreamer would enum all the
> possible colorimetries.
> It would be better we could have something like DRM blob id that
> application could copy the data from a non-DMA buffer from the
> kernel.
This is a good topic. Colorimetry could indeed be moved away from the format,
considering they cannot be enumerated. It remains that this information needs to
be passed around, and the format of a blob in media space is not has restricted
as with display HW. I think keeping an "exploded version" of the colorimetry
remains needed.
Remember though that for stateful decoder, were the information could be stored
in the bitstream, the decoder is responsible for returning that information.
Currently its passed through the G_FMT call, it would need to be replaced with a
control, similar to the HDR10 static metadata. If the colorimetry is no longer
static in the future, and may change while streaming, one option would be RO
request. This was foreseen for HDR10+ and Dolby Vision metadata notably, though
other options exists.
There exist known decoders that can do YUV to RGB conversion using an inline
post procesor (VC8000D and newer is an example), and for these to return correct
colors, the colorimetry information needs to be passed. So its not strictly
useless.
In short, if we drop colorimetry from format, we also need to go ahead and
design a replacement for it, that allow for the application to detect changes.
regards,
Nicolas
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
2023-06-28 16:27 ` Nicolas Dufresne
@ 2023-06-29 3:43 ` Hsia-Jun Li
0 siblings, 0 replies; 21+ messages in thread
From: Hsia-Jun Li @ 2023-06-29 3:43 UTC (permalink / raw)
To: Nicolas Dufresne
Cc: linux-media, ayaka, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, sakari.ailus,
stanimir.varbanov, tfiga, Helen Koike
On 6/29/23 00:27, Nicolas Dufresne wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> Hi,
>
> please avoid HTML replies on the mailing list.
I need to blame outlook.
> Le mardi 27 juin 2023 à 14:42 +0800, Hsia-Jun Li a écrit :
>>> +/**
>>> + * struct v4l2_ext_pix_format - extended multiplanar format definition
>>> + * @type: enum v4l2_buf_type; type of the data stream
>>> + * @width: image width in pixels
>>> + * @height: image height in pixels
>>> + * @pixelformat: little endian four character code (fourcc)
>>> + * @modifier: modifier applied to the format (used for tiled formats
>>> + * and other kind of HW-specific formats, like compressed
>>> + * formats) as defined in drm_fourcc.h
>>> + * @field: enum v4l2_field; field order (for interlaced video)
>>> + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
>>> + * @plane_fmt: per-plane information
>>> + * @flags: format flags (V4L2_PIX_FMT_FLAG_*)
>>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
>>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding
>>> + * @quantization: enum v4l2_quantization, colorspace quantization
>>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
>>> + * @reserved: drivers and applications must zero this array
>>> + */
>>> +struct v4l2_ext_pix_format {
>>> + __u32 type;
>>> + __u32 width;
>>> + __u32 height;
>>> + __u32 pixelformat;
>>> + __u64 modifier;
>>> + __u32 field;
>>> + __u32 colorspace;
>>> +
>>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES];
>>> + __u8 flags;
>>> + union {
>>> + __u8 ycbcr_enc;
>>> + __u8 hsv_enc;
>>> + };
>>> + __u8 quantization;
>>> + __u8 xfer_func;
>>>
>>>
>>>
>>>
>>>
>> I heard that a suggestion that we could remove colorimetry fields
>> here.
>> Although those are useless for codec M2M drivers if no pixel
>> formats translation invoked.
>> Even HDMI(DRM) cares about colorspace.
>> For example if a downsink(TV) shows RGB formats, with an YUV
>> input frame buffer, colorimetry would be important or the wrong
>> EOTF would be used. If YUV is MPEG range(linear EOTF) while a
>> non-linear EOFT (full range) is used, you would found the black is
>> not black enough while the white looks a gray. Also color bias
>> would happen.
>> This problem may not happen to a ultra high resolution TV while
>> only YUV type color formats are supported due to HDMI bandwidth
>> limitation.
>> The problem I want to raise is the time cost for enumeration.
>> Each pixel format with a colorimetry setting would invoke a
>> ioctl(). For the application likes Gstreamer would enum all the
>> possible colorimetries.
>> It would be better we could have something like DRM blob id that
>> application could copy the data from a non-DMA buffer from the
>> kernel.
> This is a good topic. Colorimetry could indeed be moved away from the format,
> considering they cannot be enumerated. It remains that this information needs to
> be passed around, and the format of a blob in media space is not has restricted
> as with display HW. I think keeping an "exploded version" of the colorimetry
> remains needed.
>
> Remember though that for stateful decoder, were the information could be stored
> in the bitstream, the decoder is responsible for returning that information.
> Currently its passed through the G_FMT call, it would need to be replaced with a
> control, similar to the HDR10 static metadata. If the colorimetry is no longer
> static in the future, and may change while streaming, one option would be RO
> request. This was foreseen for HDR10+ and Dolby Vision metadata notably, though
> other options exists.
That is why I want to promote importing a whole buffer(framebuffer id)
from drm.
We can't pass such DMA metadate with the main data(graphics here) in a
queue.
>
> There exist known decoders that can do YUV to RGB conversion using an inline
> post procesor (VC8000D and newer is an example), and for these to return correct
> colors, the colorimetry information needs to be passed. So its not strictly
> useless.
>
> In short, if we drop colorimetry from format, we also need to go ahead and
I am worrying when people setup up a UVC device(usb camera), they may
ignore this step.
Camera could be in a BT.601 colorspace than BT.709 which has a slight
difference. And BT.601 has a full range variant.
> design a replacement for it, that allow for the application to detect changes.
>
> regards,
> Nicolas
>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
2023-02-06 4:33 ` [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more) ayaka
[not found] ` <1e3a583f-f9ef-d2d3-6571-ac3b79d800ce@synaptics.com>
@ 2023-07-13 10:26 ` Tomasz Figa
2023-07-21 6:03 ` Hsia-Jun Li
1 sibling, 1 reply; 21+ messages in thread
From: Tomasz Figa @ 2023-07-13 10:26 UTC (permalink / raw)
To: ayaka
Cc: linux-media, randy.li, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, nicolas, sakari.ailus,
stanimir.varbanov, Helen Koike
On Mon, Feb 06, 2023 at 12:33:00PM +0800, ayaka wrote:
> From: Randy Li <ayaka@soulik.info>
>
> This is part of the multiplanar and singleplanar unification process.
> v4l2_ext_pix_format is supposed to work for both cases.
>
> We also add the concept of modifiers already employed in DRM to expose
> HW-specific formats (like tiled or compressed formats) and allow
> exchanging this information with the DRM subsystem in a consistent way.
>
> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
> v4l2_ext_pix_format, other types will be rejected if you use the
> {G,S,TRY}_EXT_PIX_FMT ioctls.
>
> New hooks have been added to v4l2_ioctl_ops to support those new ioctls
> in drivers, but, in the meantime, the core takes care of converting
> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
> still work if the userspace app/lib uses the new ioctls.
>
> The conversion is also done the other around to allow userspace
> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
> _ext_ hooks.
>
> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> Signed-off-by: Helen Koike <helen.koike@collabora.com>
> Signed-off-by: Randy Li <ayaka@soulik.info>
> ---
>
> Changes in v7:
> - use the resevered fields in v4l2_fmtdesc to store the modifier for format
> enumerate.
> - Dropping the old ioctl() calling new interfaces adapter.
Why?
We need backwards compatibility with the legacy API, but requiring
drivers to implement both will add even more boiler plate code to the
drivers and potential for each driver to have a buggy implementation in
its own way. (From my experience, every driver already handles the
TRY_FMT/S_FMT operations in its own slightly buggy way.)
What we probably don't need is core code for making old drivers
automatically support the new API (i.e. the other way around). We can
have the drivers modified to the new API whenever one needs to use the
new features with the old hardware. (And possibly other modernization of
the driver code could happen at the same time.)
> - Refresh the patch
> Changes in v6:
> The main change here was fixing the conversion, so planes reflects color planes,
> and to implement this properly I made major refactors compared to the previous
> version.
> - struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz)
> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans)
> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans)
> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz)
> - refactor conversion functions, so planes are color planes (Tomasz)
> - Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz)
> - Use "ef" for extended formats in the framework for consistency (Tomasz)
> - Handle xfer_func field in conversions (Tomasz)
> - Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz)
> - Refactor format functions to use v4l_fmt_ioctl_via_ext()
> - Several fixes/refactoring/changes
> - Remove EXT API for touch devices
>
> Changes in v5:
> - change sizes and reorder fields to avoid holes in the struct and make
> it the same for 32 and 64 bits
> - removed __attribute__ ((packed)) from uapi structs
> - Fix doc warning from make htmldocs
> - Updated commit message with EXT_PIX prefix for the ioctls.
>
> Changes in v4:
> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
> - Add reserved fields
> - Removed num_planes from struct v4l2_ext_pix_format
> - Removed flag field from struct v4l2_ext_pix_format, since the only
> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
> where we can use modifiers, or add it back later through the reserved
> bits.
> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
> != MOD_INVALID
> - Fix type assignment in v4l_g_fmt_ext_pix()
> - Rebased on top of media/master (post 5.8-rc1)
>
> Changes in v3:
> - Rebased on top of media/master (post 5.4-rc1)
>
> Changes in v2:
> - Move the modifier in v4l2_ext_format (was formerly placed in
> v4l2_ext_plane)
> - Fix a few bugs in the converters and add a strict parameter to
> allow conversion of uninitialized/mis-initialized objects
> ---
> drivers/media/v4l2-core/v4l2-dev.c | 13 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 357 ++++++++++++++++++++++++++-
> include/media/v4l2-ioctl.h | 28 +++
> include/uapi/linux/videodev2.h | 46 +++-
> 4 files changed, 438 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
> index 397d553177fa..cfe90bfd47f1 100644
> --- a/drivers/media/v4l2-core/v4l2-dev.c
> +++ b/drivers/media/v4l2-core/v4l2-dev.c
> @@ -615,6 +615,10 @@ static void determine_valid_ioctls(struct video_device *vdev)
> ops->vidioc_g_fmt_vid_out_mplane ||
> ops->vidioc_g_fmt_vid_out_overlay)))
> __set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
> + if ((is_rx && ops->vidioc_g_ext_pix_fmt_vid_cap) ||
> + (is_tx && ops->vidioc_g_ext_pix_fmt_vid_out)) {
> + __set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls);
> + }
> if ((is_rx && (ops->vidioc_s_fmt_vid_cap ||
> ops->vidioc_s_fmt_vid_cap_mplane ||
> ops->vidioc_s_fmt_vid_overlay)) ||
> @@ -622,6 +626,11 @@ static void determine_valid_ioctls(struct video_device *vdev)
> ops->vidioc_s_fmt_vid_out_mplane ||
> ops->vidioc_s_fmt_vid_out_overlay)))
> __set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls);
> +
> + if ((is_rx && ops->vidioc_s_ext_pix_fmt_vid_cap) ||
> + (is_tx && ops->vidioc_s_ext_pix_fmt_vid_out)) {
> + __set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls);
> + }
> if ((is_rx && (ops->vidioc_try_fmt_vid_cap ||
> ops->vidioc_try_fmt_vid_cap_mplane ||
> ops->vidioc_try_fmt_vid_overlay)) ||
> @@ -629,6 +638,10 @@ static void determine_valid_ioctls(struct video_device *vdev)
> ops->vidioc_try_fmt_vid_out_mplane ||
> ops->vidioc_try_fmt_vid_out_overlay)))
> __set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls);
> + if ((is_rx && ops->vidioc_try_ext_pix_fmt_vid_cap) ||
> + (is_tx && ops->vidioc_try_ext_pix_fmt_vid_out)) {
> + __set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls);
> + }
> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay);
> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf);
> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 87f163a89c80..52b77a968bb3 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -18,8 +18,10 @@
>
> #include <linux/v4l2-subdev.h>
> #include <linux/videodev2.h>
> +#include <drm/drm_fourcc.h>
>
> #include <media/media-device.h> /* for media_set_bus_info() */
> +
> #include <media/v4l2-common.h>
> #include <media/v4l2-ioctl.h>
> #include <media/v4l2-ctrls.h>
> @@ -34,6 +36,11 @@
>
> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls)
>
> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \
> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \
> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \
> + V4L2_CAP_VIDEO_M2M_MPLANE))
> +
> struct std_descr {
> v4l2_std_id std;
> const char *descr;
> @@ -353,6 +360,27 @@ static void v4l_print_format(const void *arg, bool write_only)
> }
> }
>
> +static void v4l_print_ext_pix_format(const void *arg, bool write_only)
> +{
> + const struct v4l2_ext_pix_format *ef = arg;
> + unsigned int i;
> +
> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n",
> + prt_names(ef->type, v4l2_type_names),
> + ef->width, ef->height,
> + (ef->pixelformat & 0xff),
> + (ef->pixelformat >> 8) & 0xff,
> + (ef->pixelformat >> 16) & 0xff,
> + (ef->pixelformat >> 24) & 0xff,
> + ef->modifier, prt_names(ef->field, v4l2_field_names),
> + ef->colorspace, ef->ycbcr_enc,
> + ef->quantization, ef->xfer_func);
> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n",
> + i, ef->plane_fmt[i].bytesperline,
> + ef->plane_fmt[i].sizeimage);
> +}
> +
> static void v4l_print_framebuffer(const void *arg, bool write_only)
> {
> const struct v4l2_framebuffer *p = arg;
> @@ -940,7 +968,9 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
> switch (type) {
> case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> if ((is_vid || is_tch) && is_rx &&
> - (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane))
> + (ops->vidioc_g_fmt_vid_cap ||
> + ops->vidioc_g_ext_pix_fmt_vid_cap ||
> + ops->vidioc_g_fmt_vid_cap_mplane))
> return 0;
> break;
> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> @@ -953,7 +983,9 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
> break;
> case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> if (is_vid && is_tx &&
> - (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane))
> + (ops->vidioc_g_fmt_vid_out ||
> + ops->vidioc_g_ext_pix_fmt_vid_out ||
> + ops->vidioc_g_fmt_vid_out_mplane))
> return 0;
> break;
> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> @@ -1082,6 +1114,203 @@ static void v4l_sanitize_format(struct v4l2_format *fmt)
> }
> }
>
> +static void
> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef,
> + struct v4l2_pix_format *pix)
> +{
> + unsigned int i;
> +
> + pix->width = ef->width;
> + pix->height = ef->height;
> + pix->field = ef->field;
> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC;
> + pix->colorspace = ef->colorspace;
> + pix->ycbcr_enc = ef->ycbcr_enc;
> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC;
> + pix->quantization = ef->quantization;
> + pix->pixelformat = ef->pixelformat;
> + pix->bytesperline = ef->plane_fmt[0].bytesperline;
> + pix->sizeimage = ef->plane_fmt[0].sizeimage;
> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
> + pix->sizeimage += ef->plane_fmt[i].sizeimage;
> +}
> +
> +static void
> +v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef,
> + struct v4l2_pix_format_mplane *pix_mp)
> +{
> + const struct v4l2_format_info *info =
> + v4l2_format_info(ef->pixelformat);
> + unsigned int i;
> +
> + pix_mp->width = ef->width;
> + pix_mp->height = ef->height;
> + pix_mp->field = ef->field;
> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC;
> + pix_mp->colorspace = ef->colorspace;
> + pix_mp->ycbcr_enc = ef->ycbcr_enc;
> + pix_mp->quantization = ef->quantization;
> + pix_mp->pixelformat = ef->pixelformat;
> +
> + /* This is true when converting to non-M-variant */
> + if (info && info->mem_planes == 1) {
> + pix_mp->plane_fmt[0] = ef->plane_fmt[0];
> + pix_mp->num_planes = 1;
> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
> + pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage;
> +
> + return;
> + }
> +
> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
> + pix_mp->plane_fmt[i] = ef->plane_fmt[i];
> + pix_mp->num_planes = i;
> +}
> +
> +/*
> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format
> + *
> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted.
> + * @f: A pointer to struct v4l2_format to be filled.
> + * @is_mplane: Bool indicating if multiplanar API should be used in @f.
> + *
> + * If pixelformat should be converted to M-variant, change ef->pixelformat
> + * to the M-variant before calling this function.
> + */
> +static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef,
> + struct v4l2_format *f, bool is_mplane)
> +{
> + memset(f, 0, sizeof(*f));
> +
> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR &&
> + ef->modifier != DRM_FORMAT_MOD_INVALID)
> + pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n",
> + ef->modifier);
> +
> + if (!is_mplane) {
> + f->type = ef->type;
> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix);
> + return;
> + }
> +
> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> + else
> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
> +
> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp);
> +}
> +
> +static void
> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix,
> + struct v4l2_ext_pix_format *ef)
> +{
> + const struct v4l2_format_info *info =
> + v4l2_format_info(pix->pixelformat);
> + unsigned int i;
> +
> + ef->width = pix->width;
> + ef->height = pix->height;
> + ef->field = pix->field;
> + ef->colorspace = pix->colorspace;
> + ef->ycbcr_enc = pix->ycbcr_enc;
> + ef->quantization = pix->quantization;
> + ef->xfer_func = pix->xfer_func;
> + if (pix->flags)
> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags);
> +
> + /* We assume M-variants won't be used in this function */
> + ef->pixelformat = pix->pixelformat;
> +
> + ef->plane_fmt[0].bytesperline = pix->bytesperline;
> + ef->plane_fmt[0].sizeimage = pix->sizeimage;
> +
> + if (!info)
> + return;
Should we print a warning in case of !info?
> +
> + for (i = 1; i < info->comp_planes; i++) {
> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv;
> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline *
> + ef->height / info->vdiv;
> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage;
> + }
> +}
> +
> +static void
> +v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp,
> + struct v4l2_ext_pix_format *ef)
> +{
> + const struct v4l2_format_info *info =
> + v4l2_format_info(pix_mp->pixelformat);
> + unsigned int i;
> +
> + ef->width = pix_mp->width;
> + ef->height = pix_mp->height;
> + ef->field = pix_mp->field;
> + ef->colorspace = pix_mp->colorspace;
> + ef->ycbcr_enc = pix_mp->ycbcr_enc;
> + ef->quantization = pix_mp->quantization;
> + ef->xfer_func = pix_mp->xfer_func;
> + if (pix_mp->flags)
> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags);
> +
> + if (!info)
> + return;
If we return here, the returned ef will not make any sense, because it
would have uninitialized plane_fmt[]. Should we print a warning here too?
> +
> + ef->pixelformat = pix_mp->pixelformat;
> +
> + if (info->comp_planes == info->mem_planes) {
> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++)
> + ef->plane_fmt[i] = pix_mp->plane_fmt[i];
> +
> + return;
> + }
> +
> + /* case where mem_planes is 1 and comp_planes > 1 */
> + ef->plane_fmt[0] = pix_mp->plane_fmt[0];
> + for (i = 1; i < info->comp_planes; i++) {
> + ef->plane_fmt[i].bytesperline =
> + pix_mp->plane_fmt[0].bytesperline / info->hdiv;
> + ef->plane_fmt[i].sizeimage =
> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv;
> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage;
> + }
> +}
> +
> +/*
> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format
> + *
> + * @f: A pointer to struct v4l2_format to be converted.
> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled.
> + *
> + * This method normalize the pixelformat to non-M variant.
> + */
> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f,
> + struct v4l2_ext_pix_format *ef)
> +{
> + memset(ef, 0, sizeof(*ef));
> +
> + switch (f->type) {
> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> + ef->type = f->type;
> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef);
> + break;
> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef);
> + break;
> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef);
> + break;
> + default:
> + WARN("Converting to Ext Pix Format with wrong buffer type %s\n",
> + prt_names(f->type, v4l2_type_names));
> + break;
> + }
> +}
> +
> static int v4l_querycap(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg)
> {
> @@ -1654,7 +1883,9 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
> v4l_pix_format_touch(&p->fmt.pix);
> return ret;
> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg);
> + if (ops->vidioc_g_fmt_vid_cap_mplane)
> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg);
What is this change needed for?
> + break;
> case V4L2_BUF_TYPE_VIDEO_OVERLAY:
> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg);
> case V4L2_BUF_TYPE_VBI_CAPTURE:
> @@ -1662,7 +1893,8 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE:
> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg);
> case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> - if (unlikely(!ops->vidioc_g_fmt_vid_out))
> + if (unlikely(!ops->vidioc_g_fmt_vid_out &&
> + !ops->vidioc_g_ext_pix_fmt_vid_out))
Ditto.
> break;
> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC;
> ret = ops->vidioc_g_fmt_vid_out(file, fh, arg);
> @@ -1670,7 +1902,9 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC;
> return ret;
> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg);
> + if (ops->vidioc_g_fmt_vid_out_mplane)
> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg);
> + break;
Ditto.
> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY:
> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg);
> case V4L2_BUF_TYPE_VBI_OUTPUT:
> @@ -1689,6 +1923,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
> return -EINVAL;
> }
>
> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,
> + struct file *file, void *fh, void *arg)
> +{
> + struct v4l2_ext_pix_format *ef = arg;
> + struct v4l2_format f = {
> + .type = ef->type,
> + };
> + int ret = check_fmt(file, ef->type);
> +
> + if (ret)
> + return ret;
> +
> + memset(ef, 0, sizeof(*ef));
> + ef->type = f.type;
> +
> + switch (f.type) {
> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> + if (ops->vidioc_g_ext_pix_fmt_vid_cap)
> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef);
> + break;
> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> + if (ops->vidioc_g_ext_pix_fmt_vid_out)
> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef);
> + break;
> + default:
> + return -EINVAL;
> + }
> +
Given that determine_valid_ioctls() will only set the valid ioctl bit if
the right operation is provided by the driver, when would the code below
be run?
> + ret = v4l_g_fmt(ops, file, fh, &f);
> + if (ret)
> + return ret;
> +
> + v4l2_format_to_ext_pix_format(&f, ef);
> + return 0;
> +}
> +
> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg)
> {
> @@ -1794,6 +2064,42 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops,
> return -EINVAL;
> }
>
> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,()
> + struct file *file, void *fh, void *arg)
> +{
> + struct video_device *vfd = video_devdata(file);
> + struct v4l2_ext_pix_format *ef = arg;
> + struct v4l2_format f;
> + int ret = check_fmt(file, ef->type);
> +
> + if (ret)
> + return ret;
> +
> + memset(ef->reserved, 0, sizeof(ef->reserved));
> +
> + switch (ef->type) {
> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> + if (ops->vidioc_s_ext_pix_fmt_vid_cap)
> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef);
> + break;
> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> + if (ops->vidioc_s_ext_pix_fmt_vid_out)
> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef);
> + break;
> + default:
> + return -EINVAL;
> + }
> +
Ditto.
> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd));
> +
> + ret = v4l_s_fmt(ops, file, fh, &f);
> + if (ret)
> + return ret;
> +
> + v4l2_format_to_ext_pix_format(&f, ef);
> + return 0;
> +}
> +
> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg)
> {
> @@ -1896,6 +2202,44 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops,
> return -EINVAL;
> }
>
> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,
> + struct file *file, void *fh, void *arg)
> +{
> + struct video_device *vfd = video_devdata(file);
> + struct v4l2_ext_pix_format *ef = arg;
> + struct v4l2_format f;
> + int ret = check_fmt(file, ef->type);
> +
> + if (ret)
> + return ret;
> +
> + memset(ef->reserved, 0, sizeof(ef->reserved));
> +
> + switch (ef->type) {
> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> + if (ops->vidioc_try_ext_pix_fmt_vid_cap)
> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh,
> + ef);
> + break;
> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> + if (ops->vidioc_try_ext_pix_fmt_vid_out)
> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh,
> + ef);
> + break;
> + default:
> + return -EINVAL;
> + }
Ditto.
> +
> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd));
> +
> + ret = v4l_try_fmt(ops, file, fh, &f);
> + if (ret)
> + return ret;
> +
> + v4l2_format_to_ext_pix_format(&f, ef);
> + return 0;
> +}
> +
> static int v4l_streamon(const struct v4l2_ioctl_ops *ops,
> struct file *file, void *fh, void *arg)
> {
> @@ -2902,6 +3246,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0),
> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO),
> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0),
> };
> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>
> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
> index edb733f21604..c44708dc9355 100644
> --- a/include/media/v4l2-ioctl.h
> +++ b/include/media/v4l2-ioctl.h
> @@ -48,11 +48,17 @@ struct v4l2_fh;
> * @vidioc_g_fmt_vid_cap: pointer to the function that implements
> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture
> * in single plane mode
> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements
> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
> + * capture
> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements
> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay
> * @vidioc_g_fmt_vid_out: pointer to the function that implements
> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out
> * in single plane mode
> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements
> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
> + * out
> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements
> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output
> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements
> @@ -82,11 +88,16 @@ struct v4l2_fh;
> * @vidioc_s_fmt_vid_cap: pointer to the function that implements
> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture
> * in single plane mode
> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements
> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
> + * capture
> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements
> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay
> * @vidioc_s_fmt_vid_out: pointer to the function that implements
> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out
> * in single plane mode
> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements
> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out
> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements
> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output
> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements
> @@ -116,11 +127,16 @@ struct v4l2_fh;
> * @vidioc_try_fmt_vid_cap: pointer to the function that implements
> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture
> * in single plane mode
> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements
> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for
> + video capture
> * @vidioc_try_fmt_vid_overlay: pointer to the function that implements
> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay
> * @vidioc_try_fmt_vid_out: pointer to the function that implements
> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out
> * in single plane mode
> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements
> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out
> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements
> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay
> * output
> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops {
> /* VIDIOC_G_FMT handlers */
> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh,
> struct v4l2_format *f);
> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
> + struct v4l2_ext_pix_format *ef);
> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh,
> struct v4l2_format *f);
> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh,
> struct v4l2_format *f);
> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh,
> + struct v4l2_ext_pix_format *ef);
> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh,
> struct v4l2_format *f);
> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh,
> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops {
> /* VIDIOC_S_FMT handlers */
> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh,
> struct v4l2_format *f);
> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
> + struct v4l2_ext_pix_format *ef);
> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh,
> struct v4l2_format *f);
> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh,
> struct v4l2_format *f);
> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh,
> + struct v4l2_ext_pix_format *ef);
> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh,
> struct v4l2_format *f);
> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh,
> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops {
> /* VIDIOC_TRY_FMT handlers */
> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh,
> struct v4l2_format *f);
> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
> + struct v4l2_ext_pix_format *ef);
> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh,
> struct v4l2_format *f);
> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh,
> struct v4l2_format *f);
> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh,
> + struct v4l2_ext_pix_format *ef);
> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh,
> struct v4l2_format *f);
> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh,
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 17a9b975177a..74a8dd7f7637 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -840,7 +840,8 @@ struct v4l2_fmtdesc {
> __u8 description[32]; /* Description string */
> __u32 pixelformat; /* Format fourcc */
> __u32 mbus_code; /* Media bus code */
> - __u32 reserved[3];
> + __u64 modifier; /* Format modifier */
How would that work? Would the driver expose multiple entries for the
same pixelformat, just with different modifier?
> + __u32 reserved;
> };
>
> #define V4L2_FMT_FLAG_COMPRESSED 0x0001
> @@ -2417,6 +2418,45 @@ struct v4l2_format {
> } fmt;
> };
>
> +/**
> + * struct v4l2_ext_pix_format - extended multiplanar format definition
> + * @type: enum v4l2_buf_type; type of the data stream
> + * @width: image width in pixels
> + * @height: image height in pixels
> + * @pixelformat: little endian four character code (fourcc)
> + * @modifier: modifier applied to the format (used for tiled formats
> + * and other kind of HW-specific formats, like compressed
> + * formats) as defined in drm_fourcc.h
> + * @field: enum v4l2_field; field order (for interlaced video)
> + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
> + * @plane_fmt: per-plane information
> + * @flags: format flags (V4L2_PIX_FMT_FLAG_*)
> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding
> + * @quantization: enum v4l2_quantization, colorspace quantization
> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
> + * @reserved: drivers and applications must zero this array
> + */
> +struct v4l2_ext_pix_format {
> + __u32 type;
> + __u32 width;
> + __u32 height;
> + __u32 pixelformat;
> + __u64 modifier;
> + __u32 field;
> + __u32 colorspace;
> +
> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES];
> + __u8 flags;
> + union {
nit: Spurious space before "union".
Best regards,
Tomasz
> + __u8 ycbcr_enc;
> + __u8 hsv_enc;
> + };
> + __u8 quantization;
> + __u8 xfer_func;
> + __u32 reserved[10];
> +} __attribute__ ((packed));
> +
> /* Stream type-dependent parameters
> */
> struct v4l2_streamparm {
> @@ -2690,6 +2730,10 @@ struct v4l2_create_buffers {
>
> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>
> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format)
> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format)
> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format)
> +
> /* Reminder: when adding new ioctls please add support for them to
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more)
2023-07-13 10:26 ` Tomasz Figa
@ 2023-07-21 6:03 ` Hsia-Jun Li
0 siblings, 0 replies; 21+ messages in thread
From: Hsia-Jun Li @ 2023-07-21 6:03 UTC (permalink / raw)
To: Tomasz Figa
Cc: linux-media, ayaka, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, nicolas, sakari.ailus,
stanimir.varbanov, Helen Koike
On 7/13/23 18:26, Tomasz Figa wrote:
> CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
> On Mon, Feb 06, 2023 at 12:33:00PM +0800, ayaka wrote:
>> From: Randy Li <ayaka@soulik.info>
>>
>> This is part of the multiplanar and singleplanar unification process.
>> v4l2_ext_pix_format is supposed to work for both cases.
>>
>> We also add the concept of modifiers already employed in DRM to expose
>> HW-specific formats (like tiled or compressed formats) and allow
>> exchanging this information with the DRM subsystem in a consistent way.
>>
>> Note that only V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] are accepted in
>> v4l2_ext_pix_format, other types will be rejected if you use the
>> {G,S,TRY}_EXT_PIX_FMT ioctls.
>>
>> New hooks have been added to v4l2_ioctl_ops to support those new ioctls
>> in drivers, but, in the meantime, the core takes care of converting
>> {S,G,TRY}_EXT_PIX_FMT requests into {S,G,TRY}_FMT so that old drivers can
>> still work if the userspace app/lib uses the new ioctls.
>>
>> The conversion is also done the other around to allow userspace
>> apps/libs using {S,G,TRY}_FMT to work with drivers implementing the
>> _ext_ hooks.
>>
>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
>> Signed-off-by: Helen Koike <helen.koike@collabora.com>
>> Signed-off-by: Randy Li <ayaka@soulik.info>
>> ---
>>
>> Changes in v7:
>> - use the resevered fields in v4l2_fmtdesc to store the modifier for format
>> enumerate.
>> - Dropping the old ioctl() calling new interfaces adapter.
>
> Why?
>
> We need backwards compatibility with the legacy API, but requiring
> drivers to implement both will add even more boiler plate code to the
> drivers and potential for each driver to have a buggy implementation in
> its own way. (From my experience, every driver already handles the
> TRY_FMT/S_FMT operations in its own slightly buggy way.)
Then we would continues on dirtying the fourcc namespace. I think the
whole point of this extend API is to trying not squeeze those tiling
formats into a small 32bits namespace.
>
> What we probably don't need is core code for making old drivers
> automatically support the new API (i.e. the other way around).
Make this wrapper layer in kernel to save application from keepping the
legacy code. I really want people dropping a v4l2 fourcc pixel format,
using drm pixel formats instead.
The rest fixup would be considered in the new version of this series.
We can
> have the drivers modified to the new API whenever one needs to use the
> new features with the old hardware. (And possibly other modernization of
> the driver code could happen at the same time.)
>
>> - Refresh the patch
>> Changes in v6:
>> The main change here was fixing the conversion, so planes reflects color planes,
>> and to implement this properly I made major refactors compared to the previous
>> version.
>> - struct v4l2_plane_ext_pix_format removed, using struct v4l2_plane_pix_format instead (Tomasz)
>> - refer to drm_fourcc.h in struct v4l2_ext_pix_format docs (Hans)
>> - reorder colorimetry fields in struct v4l2_ext_pix_format (Hans)
>> - do not set Ext ioctls as valid for vid_out_overlay (Tomasz)
>> - refactor conversion functions, so planes are color planes (Tomasz)
>> - Don't explicitly check for e->modifier != 0 in v4l2_ext_pix_format_to_format() (Tomasz)
>> - Use "ef" for extended formats in the framework for consistency (Tomasz)
>> - Handle xfer_func field in conversions (Tomasz)
>> - Zero reserved fields in v4l_s_ext_pix_fmt() and v4l_try_ext_pix_fmt() (Tomasz)
>> - Refactor format functions to use v4l_fmt_ioctl_via_ext()
>> - Several fixes/refactoring/changes
>> - Remove EXT API for touch devices
>>
>> Changes in v5:
>> - change sizes and reorder fields to avoid holes in the struct and make
>> it the same for 32 and 64 bits
>> - removed __attribute__ ((packed)) from uapi structs
>> - Fix doc warning from make htmldocs
>> - Updated commit message with EXT_PIX prefix for the ioctls.
>>
>> Changes in v4:
>> - Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
>> making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
>> - Add reserved fields
>> - Removed num_planes from struct v4l2_ext_pix_format
>> - Removed flag field from struct v4l2_ext_pix_format, since the only
>> defined value is V4L2_PIX_FMT_FLAG_PREMUL_ALPHA only used by vsp1,
>> where we can use modifiers, or add it back later through the reserved
>> bits.
>> - In v4l2_ext_format_to_format(), check if modifier is != MOD_LINEAR &&
>> != MOD_INVALID
>> - Fix type assignment in v4l_g_fmt_ext_pix()
>> - Rebased on top of media/master (post 5.8-rc1)
>>
>> Changes in v3:
>> - Rebased on top of media/master (post 5.4-rc1)
>>
>> Changes in v2:
>> - Move the modifier in v4l2_ext_format (was formerly placed in
>> v4l2_ext_plane)
>> - Fix a few bugs in the converters and add a strict parameter to
>> allow conversion of uninitialized/mis-initialized objects
>> ---
>> drivers/media/v4l2-core/v4l2-dev.c | 13 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 357 ++++++++++++++++++++++++++-
>> include/media/v4l2-ioctl.h | 28 +++
>> include/uapi/linux/videodev2.h | 46 +++-
>> 4 files changed, 438 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
>> index 397d553177fa..cfe90bfd47f1 100644
>> --- a/drivers/media/v4l2-core/v4l2-dev.c
>> +++ b/drivers/media/v4l2-core/v4l2-dev.c
>> @@ -615,6 +615,10 @@ static void determine_valid_ioctls(struct video_device *vdev)
>> ops->vidioc_g_fmt_vid_out_mplane ||
>> ops->vidioc_g_fmt_vid_out_overlay)))
>> __set_bit(_IOC_NR(VIDIOC_G_FMT), valid_ioctls);
>> + if ((is_rx && ops->vidioc_g_ext_pix_fmt_vid_cap) ||
>> + (is_tx && ops->vidioc_g_ext_pix_fmt_vid_out)) {
>> + __set_bit(_IOC_NR(VIDIOC_G_EXT_PIX_FMT), valid_ioctls);
>> + }
>> if ((is_rx && (ops->vidioc_s_fmt_vid_cap ||
>> ops->vidioc_s_fmt_vid_cap_mplane ||
>> ops->vidioc_s_fmt_vid_overlay)) ||
>> @@ -622,6 +626,11 @@ static void determine_valid_ioctls(struct video_device *vdev)
>> ops->vidioc_s_fmt_vid_out_mplane ||
>> ops->vidioc_s_fmt_vid_out_overlay)))
>> __set_bit(_IOC_NR(VIDIOC_S_FMT), valid_ioctls);
>> +
>> + if ((is_rx && ops->vidioc_s_ext_pix_fmt_vid_cap) ||
>> + (is_tx && ops->vidioc_s_ext_pix_fmt_vid_out)) {
>> + __set_bit(_IOC_NR(VIDIOC_S_EXT_PIX_FMT), valid_ioctls);
>> + }
>> if ((is_rx && (ops->vidioc_try_fmt_vid_cap ||
>> ops->vidioc_try_fmt_vid_cap_mplane ||
>> ops->vidioc_try_fmt_vid_overlay)) ||
>> @@ -629,6 +638,10 @@ static void determine_valid_ioctls(struct video_device *vdev)
>> ops->vidioc_try_fmt_vid_out_mplane ||
>> ops->vidioc_try_fmt_vid_out_overlay)))
>> __set_bit(_IOC_NR(VIDIOC_TRY_FMT), valid_ioctls);
>> + if ((is_rx && ops->vidioc_try_ext_pix_fmt_vid_cap) ||
>> + (is_tx && ops->vidioc_try_ext_pix_fmt_vid_out)) {
>> + __set_bit(_IOC_NR(VIDIOC_TRY_EXT_PIX_FMT), valid_ioctls);
>> + }
>> SET_VALID_IOCTL(ops, VIDIOC_OVERLAY, vidioc_overlay);
>> SET_VALID_IOCTL(ops, VIDIOC_G_FBUF, vidioc_g_fbuf);
>> SET_VALID_IOCTL(ops, VIDIOC_S_FBUF, vidioc_s_fbuf);
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index 87f163a89c80..52b77a968bb3 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -18,8 +18,10 @@
>>
>> #include <linux/v4l2-subdev.h>
>> #include <linux/videodev2.h>
>> +#include <drm/drm_fourcc.h>
>>
>> #include <media/media-device.h> /* for media_set_bus_info() */
>> +
>> #include <media/v4l2-common.h>
>> #include <media/v4l2-ioctl.h>
>> #include <media/v4l2-ctrls.h>
>> @@ -34,6 +36,11 @@
>>
>> #define is_valid_ioctl(vfd, cmd) test_bit(_IOC_NR(cmd), (vfd)->valid_ioctls)
>>
>> +#define V4L2_IS_CAP_MULTIPLANAR(vdev) (vdev->device_caps & \
>> + (V4L2_CAP_VIDEO_CAPTURE_MPLANE | \
>> + V4L2_CAP_VIDEO_OUTPUT_MPLANE | \
>> + V4L2_CAP_VIDEO_M2M_MPLANE))
>> +
>> struct std_descr {
>> v4l2_std_id std;
>> const char *descr;
>> @@ -353,6 +360,27 @@ static void v4l_print_format(const void *arg, bool write_only)
>> }
>> }
>>
>> +static void v4l_print_ext_pix_format(const void *arg, bool write_only)
>> +{
>> + const struct v4l2_ext_pix_format *ef = arg;
>> + unsigned int i;
>> +
>> + pr_cont("type=%s, width=%u, height=%u, format=%c%c%c%c, modifier %llx, field=%s, colorspace=%d, ycbcr_enc=%u, quantization=%u, xfer_func=%u\n",
>> + prt_names(ef->type, v4l2_type_names),
>> + ef->width, ef->height,
>> + (ef->pixelformat & 0xff),
>> + (ef->pixelformat >> 8) & 0xff,
>> + (ef->pixelformat >> 16) & 0xff,
>> + (ef->pixelformat >> 24) & 0xff,
>> + ef->modifier, prt_names(ef->field, v4l2_field_names),
>> + ef->colorspace, ef->ycbcr_enc,
>> + ef->quantization, ef->xfer_func);
>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
>> + pr_debug("plane %u: bytesperline=%u sizeimage=%u\n",
>> + i, ef->plane_fmt[i].bytesperline,
>> + ef->plane_fmt[i].sizeimage);
>> +}
>> +
>> static void v4l_print_framebuffer(const void *arg, bool write_only)
>> {
>> const struct v4l2_framebuffer *p = arg;
>> @@ -940,7 +968,9 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
>> switch (type) {
>> case V4L2_BUF_TYPE_VIDEO_CAPTURE:
>> if ((is_vid || is_tch) && is_rx &&
>> - (ops->vidioc_g_fmt_vid_cap || ops->vidioc_g_fmt_vid_cap_mplane))
>> + (ops->vidioc_g_fmt_vid_cap ||
>> + ops->vidioc_g_ext_pix_fmt_vid_cap ||
>> + ops->vidioc_g_fmt_vid_cap_mplane))
>> return 0;
>> break;
>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
>> @@ -953,7 +983,9 @@ static int check_fmt(struct file *file, enum v4l2_buf_type type)
>> break;
>> case V4L2_BUF_TYPE_VIDEO_OUTPUT:
>> if (is_vid && is_tx &&
>> - (ops->vidioc_g_fmt_vid_out || ops->vidioc_g_fmt_vid_out_mplane))
>> + (ops->vidioc_g_fmt_vid_out ||
>> + ops->vidioc_g_ext_pix_fmt_vid_out ||
>> + ops->vidioc_g_fmt_vid_out_mplane))
>> return 0;
>> break;
>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>> @@ -1082,6 +1114,203 @@ static void v4l_sanitize_format(struct v4l2_format *fmt)
>> }
>> }
>>
>> +static void
>> +v4l2_ext_pix_format_to_pix_format(const struct v4l2_ext_pix_format *ef,
>> + struct v4l2_pix_format *pix)
>> +{
>> + unsigned int i;
>> +
>> + pix->width = ef->width;
>> + pix->height = ef->height;
>> + pix->field = ef->field;
>> + pix->flags = V4L2_PIX_FMT_FLAG_SET_CSC;
>> + pix->colorspace = ef->colorspace;
>> + pix->ycbcr_enc = ef->ycbcr_enc;
>> + pix->priv = V4L2_PIX_FMT_PRIV_MAGIC;
>> + pix->quantization = ef->quantization;
>> + pix->pixelformat = ef->pixelformat;
>> + pix->bytesperline = ef->plane_fmt[0].bytesperline;
>> + pix->sizeimage = ef->plane_fmt[0].sizeimage;
>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
>> + pix->sizeimage += ef->plane_fmt[i].sizeimage;
>> +}
>> +
>> +static void
>> +v4l2_ext_pix_format_to_pix_mp_format(const struct v4l2_ext_pix_format *ef,
>> + struct v4l2_pix_format_mplane *pix_mp)
>> +{
>> + const struct v4l2_format_info *info =
>> + v4l2_format_info(ef->pixelformat);
>> + unsigned int i;
>> +
>> + pix_mp->width = ef->width;
>> + pix_mp->height = ef->height;
>> + pix_mp->field = ef->field;
>> + pix_mp->flags = V4L2_PIX_FMT_FLAG_SET_CSC;
>> + pix_mp->colorspace = ef->colorspace;
>> + pix_mp->ycbcr_enc = ef->ycbcr_enc;
>> + pix_mp->quantization = ef->quantization;
>> + pix_mp->pixelformat = ef->pixelformat;
>> +
>> + /* This is true when converting to non-M-variant */
>> + if (info && info->mem_planes == 1) {
>> + pix_mp->plane_fmt[0] = ef->plane_fmt[0];
>> + pix_mp->num_planes = 1;
>> + for (i = 1; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
>> + pix_mp->plane_fmt[0].sizeimage += ef->plane_fmt[i].sizeimage;
>> +
>> + return;
>> + }
>> +
>> + for (i = 0; i < VIDEO_MAX_PLANES && ef->plane_fmt[i].sizeimage; i++)
>> + pix_mp->plane_fmt[i] = ef->plane_fmt[i];
>> + pix_mp->num_planes = i;
>> +}
>> +
>> +/*
>> + * v4l2_ext_pix_format_to_format - convert to v4l2_ext_pix_format to v4l2_format
>> + *
>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be converted.
>> + * @f: A pointer to struct v4l2_format to be filled.
>> + * @is_mplane: Bool indicating if multiplanar API should be used in @f.
>> + *
>> + * If pixelformat should be converted to M-variant, change ef->pixelformat
>> + * to the M-variant before calling this function.
>> + */
>> +static void v4l2_ext_pix_format_to_format(const struct v4l2_ext_pix_format *ef,
>> + struct v4l2_format *f, bool is_mplane)
>> +{
>> + memset(f, 0, sizeof(*f));
>> +
>> + if (ef->modifier != DRM_FORMAT_MOD_LINEAR &&
>> + ef->modifier != DRM_FORMAT_MOD_INVALID)
>> + pr_warn("Modifiers are not supported in v4l2_format, ignoring %llx\n",
>> + ef->modifier);
>> +
>> + if (!is_mplane) {
>> + f->type = ef->type;
>> + v4l2_ext_pix_format_to_pix_format(ef, &f->fmt.pix);
>> + return;
>> + }
>> +
>> + if (ef->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
>> + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
>> + else
>> + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
>> +
>> + v4l2_ext_pix_format_to_pix_mp_format(ef, &f->fmt.pix_mp);
>> +}
>> +
>> +static void
>> +v4l2_pix_format_to_ext_pix_format(const struct v4l2_pix_format *pix,
>> + struct v4l2_ext_pix_format *ef)
>> +{
>> + const struct v4l2_format_info *info =
>> + v4l2_format_info(pix->pixelformat);
>> + unsigned int i;
>> +
>> + ef->width = pix->width;
>> + ef->height = pix->height;
>> + ef->field = pix->field;
>> + ef->colorspace = pix->colorspace;
>> + ef->ycbcr_enc = pix->ycbcr_enc;
>> + ef->quantization = pix->quantization;
>> + ef->xfer_func = pix->xfer_func;
>> + if (pix->flags)
>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix->flags);
>> +
>> + /* We assume M-variants won't be used in this function */
>> + ef->pixelformat = pix->pixelformat;
>> +
>> + ef->plane_fmt[0].bytesperline = pix->bytesperline;
>> + ef->plane_fmt[0].sizeimage = pix->sizeimage;
>> +
>> + if (!info)
>> + return;
>
> Should we print a warning in case of !info?
>
>> +
>> + for (i = 1; i < info->comp_planes; i++) {
>> + ef->plane_fmt[i].bytesperline = pix->bytesperline / info->hdiv;
>> + ef->plane_fmt[i].sizeimage = ef->plane_fmt[i].bytesperline *
>> + ef->height / info->vdiv;
>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage;
>> + }
>> +}
>> +
>> +static void
>> +v4l2_pix_mp_format_to_ext_pix_format(const struct v4l2_pix_format_mplane *pix_mp,
>> + struct v4l2_ext_pix_format *ef)
>> +{
>> + const struct v4l2_format_info *info =
>> + v4l2_format_info(pix_mp->pixelformat);
>> + unsigned int i;
>> +
>> + ef->width = pix_mp->width;
>> + ef->height = pix_mp->height;
>> + ef->field = pix_mp->field;
>> + ef->colorspace = pix_mp->colorspace;
>> + ef->ycbcr_enc = pix_mp->ycbcr_enc;
>> + ef->quantization = pix_mp->quantization;
>> + ef->xfer_func = pix_mp->xfer_func;
>> + if (pix_mp->flags)
>> + pr_warn("Ignoring pixelformat flags 0x%x\n", pix_mp->flags);
>> +
>> + if (!info)
>> + return;
>
> If we return here, the returned ef will not make any sense, because it
> would have uninitialized plane_fmt[]. Should we print a warning here too?
>
>> +
>> + ef->pixelformat = pix_mp->pixelformat;
>> +
>> + if (info->comp_planes == info->mem_planes) {
>> + for (i = 0; i < pix_mp->num_planes && i < VIDEO_MAX_PLANES; i++)
>> + ef->plane_fmt[i] = pix_mp->plane_fmt[i];
>> +
>> + return;
>> + }
>> +
>> + /* case where mem_planes is 1 and comp_planes > 1 */
>> + ef->plane_fmt[0] = pix_mp->plane_fmt[0];
>> + for (i = 1; i < info->comp_planes; i++) {
>> + ef->plane_fmt[i].bytesperline =
>> + pix_mp->plane_fmt[0].bytesperline / info->hdiv;
>> + ef->plane_fmt[i].sizeimage =
>> + ef->plane_fmt[i].bytesperline * ef->height / info->vdiv;
>> + ef->plane_fmt[0].sizeimage -= ef->plane_fmt[i].sizeimage;
>> + }
>> +}
>> +
>> +/*
>> + * v4l2_format_to_ext_pix_format - convert to v4l2_format to v4l2_ext_pix_format
>> + *
>> + * @f: A pointer to struct v4l2_format to be converted.
>> + * @ef: A pointer to struct struct v4l2_ext_pix_format to be filled.
>> + *
>> + * This method normalize the pixelformat to non-M variant.
>> + */
>> +static void v4l2_format_to_ext_pix_format(const struct v4l2_format *f,
>> + struct v4l2_ext_pix_format *ef)
>> +{
>> + memset(ef, 0, sizeof(*ef));
>> +
>> + switch (f->type) {
>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
>> + ef->type = f->type;
>> + v4l2_pix_format_to_ext_pix_format(&f->fmt.pix, ef);
>> + break;
>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
>> + ef->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef);
>> + break;
>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>> + ef->type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
>> + v4l2_pix_mp_format_to_ext_pix_format(&f->fmt.pix_mp, ef);
>> + break;
>> + default:
>> + WARN("Converting to Ext Pix Format with wrong buffer type %s\n",
>> + prt_names(f->type, v4l2_type_names));
>> + break;
>> + }
>> +}
>> +
>> static int v4l_querycap(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg)
>> {
>> @@ -1654,7 +1883,9 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
>> v4l_pix_format_touch(&p->fmt.pix);
>> return ret;
>> case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
>> - return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg);
>> + if (ops->vidioc_g_fmt_vid_cap_mplane)
>> + return ops->vidioc_g_fmt_vid_cap_mplane(file, fh, arg);
>
> What is this change needed for?
>
>> + break;
>> case V4L2_BUF_TYPE_VIDEO_OVERLAY:
>> return ops->vidioc_g_fmt_vid_overlay(file, fh, arg);
>> case V4L2_BUF_TYPE_VBI_CAPTURE:
>> @@ -1662,7 +1893,8 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
>> case V4L2_BUF_TYPE_SLICED_VBI_CAPTURE:
>> return ops->vidioc_g_fmt_sliced_vbi_cap(file, fh, arg);
>> case V4L2_BUF_TYPE_VIDEO_OUTPUT:
>> - if (unlikely(!ops->vidioc_g_fmt_vid_out))
>> + if (unlikely(!ops->vidioc_g_fmt_vid_out &&
>> + !ops->vidioc_g_ext_pix_fmt_vid_out))
>
> Ditto.
>
>> break;
>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC;
>> ret = ops->vidioc_g_fmt_vid_out(file, fh, arg);
>> @@ -1670,7 +1902,9 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
>> p->fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC;
>> return ret;
>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
>> - return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg);
>> + if (ops->vidioc_g_fmt_vid_out_mplane)
>> + return ops->vidioc_g_fmt_vid_out_mplane(file, fh, arg);
>> + break;
>
> Ditto.
>
>> case V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY:
>> return ops->vidioc_g_fmt_vid_out_overlay(file, fh, arg);
>> case V4L2_BUF_TYPE_VBI_OUTPUT:
>> @@ -1689,6 +1923,42 @@ static int v4l_g_fmt(const struct v4l2_ioctl_ops *ops,
>> return -EINVAL;
>> }
>>
>> +static int v4l_g_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg)
>> +{
>> + struct v4l2_ext_pix_format *ef = arg;
>> + struct v4l2_format f = {
>> + .type = ef->type,
>> + };
>> + int ret = check_fmt(file, ef->type);
>> +
>> + if (ret)
>> + return ret;
>> +
>> + memset(ef, 0, sizeof(*ef));
>> + ef->type = f.type;
>> +
>> + switch (f.type) {
>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
>> + if (ops->vidioc_g_ext_pix_fmt_vid_cap)
>> + return ops->vidioc_g_ext_pix_fmt_vid_cap(file, fh, ef);
>> + break;
>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
>> + if (ops->vidioc_g_ext_pix_fmt_vid_out)
>> + return ops->vidioc_g_ext_pix_fmt_vid_out(file, fh, ef);
>> + break;
>> + default:
>> + return -EINVAL;
>> + }
>> +
>
> Given that determine_valid_ioctls() will only set the valid ioctl bit if
> the right operation is provided by the driver, when would the code below
> be run?
>
>> + ret = v4l_g_fmt(ops, file, fh, &f);
>> + if (ret)
>> + return ret;
>> +
>> + v4l2_format_to_ext_pix_format(&f, ef);
>> + return 0;
>> +}
>> +
>> static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg)
>> {
>> @@ -1794,6 +2064,42 @@ static int v4l_s_fmt(const struct v4l2_ioctl_ops *ops,
>> return -EINVAL;
>> }
>>
>> +static int v4l_s_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,()
>> + struct file *file, void *fh, void *arg)
>> +{
>> + struct video_device *vfd = video_devdata(file);
>> + struct v4l2_ext_pix_format *ef = arg;
>> + struct v4l2_format f;
>> + int ret = check_fmt(file, ef->type);
>> +
>> + if (ret)
>> + return ret;
>> +
>> + memset(ef->reserved, 0, sizeof(ef->reserved));
>> +
>> + switch (ef->type) {
>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
>> + if (ops->vidioc_s_ext_pix_fmt_vid_cap)
>> + return ops->vidioc_s_ext_pix_fmt_vid_cap(file, fh, ef);
>> + break;
>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
>> + if (ops->vidioc_s_ext_pix_fmt_vid_out)
>> + return ops->vidioc_s_ext_pix_fmt_vid_out(file, fh, ef);
>> + break;
>> + default:
>> + return -EINVAL;
>> + }
>> +
>
> Ditto.
>
>> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd));
>> +
>> + ret = v4l_s_fmt(ops, file, fh, &f);
>> + if (ret)
>> + return ret;
>> +
>> + v4l2_format_to_ext_pix_format(&f, ef);
>> + return 0;
>> +}
>> +
>> static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg)
>> {
>> @@ -1896,6 +2202,44 @@ static int v4l_try_fmt(const struct v4l2_ioctl_ops *ops,
>> return -EINVAL;
>> }
>>
>> +static int v4l_try_ext_pix_fmt(const struct v4l2_ioctl_ops *ops,
>> + struct file *file, void *fh, void *arg)
>> +{
>> + struct video_device *vfd = video_devdata(file);
>> + struct v4l2_ext_pix_format *ef = arg;
>> + struct v4l2_format f;
>> + int ret = check_fmt(file, ef->type);
>> +
>> + if (ret)
>> + return ret;
>> +
>> + memset(ef->reserved, 0, sizeof(ef->reserved));
>> +
>> + switch (ef->type) {
>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
>> + if (ops->vidioc_try_ext_pix_fmt_vid_cap)
>> + return ops->vidioc_try_ext_pix_fmt_vid_cap(file, fh,
>> + ef);
>> + break;
>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
>> + if (ops->vidioc_try_ext_pix_fmt_vid_out)
>> + return ops->vidioc_try_ext_pix_fmt_vid_out(file, fh,
>> + ef);
>> + break;
>> + default:
>> + return -EINVAL;
>> + }
>
> Ditto.
>
>> +
>> + v4l2_ext_pix_format_to_format(ef, &f, V4L2_IS_CAP_MULTIPLANAR(vfd));
>> +
>> + ret = v4l_try_fmt(ops, file, fh, &f);
>> + if (ret)
>> + return ret;
>> +
>> + v4l2_format_to_ext_pix_format(&f, ef);
>> + return 0;
>> +}
>> +
>> static int v4l_streamon(const struct v4l2_ioctl_ops *ops,
>> struct file *file, void *fh, void *arg)
>> {
>> @@ -2902,6 +3246,9 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
>> IOCTL_INFO(VIDIOC_ENUM_FREQ_BANDS, v4l_enum_freq_bands, v4l_print_freq_band, 0),
>> IOCTL_INFO(VIDIOC_DBG_G_CHIP_INFO, v4l_dbg_g_chip_info, v4l_print_dbg_chip_info, INFO_FL_CLEAR(v4l2_dbg_chip_info, match)),
>> IOCTL_INFO(VIDIOC_QUERY_EXT_CTRL, v4l_query_ext_ctrl, v4l_print_query_ext_ctrl, INFO_FL_CTRL | INFO_FL_CLEAR(v4l2_query_ext_ctrl, id)),
>> + IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0),
>> + IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO),
>> + IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0),
>> };
>> #define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
>>
>> diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
>> index edb733f21604..c44708dc9355 100644
>> --- a/include/media/v4l2-ioctl.h
>> +++ b/include/media/v4l2-ioctl.h
>> @@ -48,11 +48,17 @@ struct v4l2_fh;
>> * @vidioc_g_fmt_vid_cap: pointer to the function that implements
>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video capture
>> * in single plane mode
>> + * @vidioc_g_ext_pix_fmt_vid_cap: pointer to the function that implements
>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
>> + * capture
>> * @vidioc_g_fmt_vid_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay
>> * @vidioc_g_fmt_vid_out: pointer to the function that implements
>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video out
>> * in single plane mode
>> + * @vidioc_g_ext_pix_fmt_vid_out: pointer to the function that implements
>> + * :ref:`VIDIOC_G_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
>> + * out
>> * @vidioc_g_fmt_vid_out_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_G_FMT <vidioc_g_fmt>` ioctl logic for video overlay output
>> * @vidioc_g_fmt_vbi_cap: pointer to the function that implements
>> @@ -82,11 +88,16 @@ struct v4l2_fh;
>> * @vidioc_s_fmt_vid_cap: pointer to the function that implements
>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video capture
>> * in single plane mode
>> + * @vidioc_s_ext_pix_fmt_vid_cap: pointer to the function that implements
>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for video
>> + * capture
>> * @vidioc_s_fmt_vid_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay
>> * @vidioc_s_fmt_vid_out: pointer to the function that implements
>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video out
>> * in single plane mode
>> + * @vidioc_s_ext_pix_fmt_vid_out: pointer to the function that implements
>> + * :ref:`VIDIOC_S_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out
>> * @vidioc_s_fmt_vid_out_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_S_FMT <vidioc_g_fmt>` ioctl logic for video overlay output
>> * @vidioc_s_fmt_vbi_cap: pointer to the function that implements
>> @@ -116,11 +127,16 @@ struct v4l2_fh;
>> * @vidioc_try_fmt_vid_cap: pointer to the function that implements
>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video capture
>> * in single plane mode
>> + * @vidioc_try_ext_pix_fmt_vid_cap: pointer to the function that implements
>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_ext_pix_fmt>` ioctl logic for
>> + video capture
>> * @vidioc_try_fmt_vid_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay
>> * @vidioc_try_fmt_vid_out: pointer to the function that implements
>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video out
>> * in single plane mode
>> + * @vidioc_try_ext_pix_fmt_vid_out: pointer to the function that implements
>> + * :ref:`VIDIOC_TRY_EXT_PIX_FMT <vidioc_g_fmt>` ioctl logic for video out
>> * @vidioc_try_fmt_vid_out_overlay: pointer to the function that implements
>> * :ref:`VIDIOC_TRY_FMT <vidioc_g_fmt>` ioctl logic for video overlay
>> * output
>> @@ -319,10 +335,14 @@ struct v4l2_ioctl_ops {
>> /* VIDIOC_G_FMT handlers */
>> int (*vidioc_g_fmt_vid_cap)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> + int (*vidioc_g_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
>> + struct v4l2_ext_pix_format *ef);
>> int (*vidioc_g_fmt_vid_overlay)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> int (*vidioc_g_fmt_vid_out)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> + int (*vidioc_g_ext_pix_fmt_vid_out)(struct file *file, void *fh,
>> + struct v4l2_ext_pix_format *ef);
>> int (*vidioc_g_fmt_vid_out_overlay)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> int (*vidioc_g_fmt_vbi_cap)(struct file *file, void *fh,
>> @@ -349,10 +369,14 @@ struct v4l2_ioctl_ops {
>> /* VIDIOC_S_FMT handlers */
>> int (*vidioc_s_fmt_vid_cap)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> + int (*vidioc_s_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
>> + struct v4l2_ext_pix_format *ef);
>> int (*vidioc_s_fmt_vid_overlay)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> int (*vidioc_s_fmt_vid_out)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> + int (*vidioc_s_ext_pix_fmt_vid_out)(struct file *file, void *fh,
>> + struct v4l2_ext_pix_format *ef);
>> int (*vidioc_s_fmt_vid_out_overlay)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> int (*vidioc_s_fmt_vbi_cap)(struct file *file, void *fh,
>> @@ -379,10 +403,14 @@ struct v4l2_ioctl_ops {
>> /* VIDIOC_TRY_FMT handlers */
>> int (*vidioc_try_fmt_vid_cap)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> + int (*vidioc_try_ext_pix_fmt_vid_cap)(struct file *file, void *fh,
>> + struct v4l2_ext_pix_format *ef);
>> int (*vidioc_try_fmt_vid_overlay)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> int (*vidioc_try_fmt_vid_out)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> + int (*vidioc_try_ext_pix_fmt_vid_out)(struct file *file, void *fh,
>> + struct v4l2_ext_pix_format *ef);
>> int (*vidioc_try_fmt_vid_out_overlay)(struct file *file, void *fh,
>> struct v4l2_format *f);
>> int (*vidioc_try_fmt_vbi_cap)(struct file *file, void *fh,
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 17a9b975177a..74a8dd7f7637 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -840,7 +840,8 @@ struct v4l2_fmtdesc {
>> __u8 description[32]; /* Description string */
>> __u32 pixelformat; /* Format fourcc */
>> __u32 mbus_code; /* Media bus code */
>> - __u32 reserved[3];
>> + __u64 modifier; /* Format modifier */
>
> How would that work? Would the driver expose multiple entries for the
> same pixelformat, just with different modifier?
>
>> + __u32 reserved;
>> };
>>
>> #define V4L2_FMT_FLAG_COMPRESSED 0x0001
>> @@ -2417,6 +2418,45 @@ struct v4l2_format {
>> } fmt;
>> };
>>
>> +/**
>> + * struct v4l2_ext_pix_format - extended multiplanar format definition
>> + * @type: enum v4l2_buf_type; type of the data stream
>> + * @width: image width in pixels
>> + * @height: image height in pixels
>> + * @pixelformat: little endian four character code (fourcc)
>> + * @modifier: modifier applied to the format (used for tiled formats
>> + * and other kind of HW-specific formats, like compressed
>> + * formats) as defined in drm_fourcc.h
>> + * @field: enum v4l2_field; field order (for interlaced video)
>> + * @colorspace: enum v4l2_colorspace; supplemental to pixelformat
>> + * @plane_fmt: per-plane information
>> + * @flags: format flags (V4L2_PIX_FMT_FLAG_*)
>> + * @ycbcr_enc: enum v4l2_ycbcr_encoding, Y'CbCr encoding
>> + * @hsv_enc: enum v4l2_hsv_encoding, HSV encoding
>> + * @quantization: enum v4l2_quantization, colorspace quantization
>> + * @xfer_func: enum v4l2_xfer_func, colorspace transfer function
>> + * @reserved: drivers and applications must zero this array
>> + */
>> +struct v4l2_ext_pix_format {
>> + __u32 type;
>> + __u32 width;
>> + __u32 height;
>> + __u32 pixelformat;
>> + __u64 modifier;
>> + __u32 field;
>> + __u32 colorspace;
>> +
>> + struct v4l2_plane_pix_format plane_fmt[VIDEO_MAX_PLANES];
>> + __u8 flags;
>> + union {
>
> nit: Spurious space before "union".
>
> Best regards,
> Tomasz
>
>> + __u8 ycbcr_enc;
>> + __u8 hsv_enc;
>> + };
>> + __u8 quantization;
>> + __u8 xfer_func;
>> + __u32 reserved[10];
>> +} __attribute__ ((packed));
>> +
>> /* Stream type-dependent parameters
>> */
>> struct v4l2_streamparm {
>> @@ -2690,6 +2730,10 @@ struct v4l2_create_buffers {
>>
>> #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl)
>>
>> +#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format)
>> +#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format)
>> +#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format)
>> +
>> /* Reminder: when adding new ioctls please add support for them to
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
>>
>> --
>> 2.17.1
>>
--
Hsia-Jun(Randy) Li
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
2023-02-06 4:33 ` [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more) ayaka
@ 2023-02-06 4:33 ` ayaka
2023-07-13 10:39 ` Tomasz Figa
2023-02-06 4:33 ` [PATCH v7 3/9] media: v4l2: Add extended buffer (de)queue operations for video types ayaka
` (6 subsequent siblings)
8 siblings, 1 reply; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike
From: Helen Koike <helen.koike@collabora.com>
Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
framework to use the ext hooks when the classic Api is used from
userspace in Vid devices, and to keep touch devices with classic hook.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v7:
- Force the userspace using the new APIs to operate non-touch drivers.
- Refresh the patch
Changes in v6:
- Update with new format and buffer structs
- duplicate v4l2_ioctl_ops for touch devices.
Changes in v4:
- Update with new format and buffer structs
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
---
drivers/media/test-drivers/vivid/vivid-core.c | 104 ++-------
.../media/test-drivers/vivid/vivid-vid-cap.c | 202 ++++++------------
.../media/test-drivers/vivid/vivid-vid-cap.h | 15 +-
.../media/test-drivers/vivid/vivid-vid-out.c | 197 ++++++-----------
.../media/test-drivers/vivid/vivid-vid-out.h | 15 +-
5 files changed, 169 insertions(+), 364 deletions(-)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.c b/drivers/media/test-drivers/vivid/vivid-core.c
index f28440e6c9f8..d4ed55aef1bb 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.c
+++ b/drivers/media/test-drivers/vivid/vivid-core.c
@@ -508,76 +508,6 @@ static int vivid_s_input(struct file *file, void *priv, unsigned int i)
return vidioc_s_input(file, priv, i);
}
-static int vivid_enum_fmt_cap(struct file *file, void *priv,
- struct v4l2_fmtdesc *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_enum_fmt_tch(file, priv, f);
- return vivid_enum_fmt_vid(file, priv, f);
-}
-
-static int vivid_g_fmt_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch(file, priv, f);
- return vidioc_g_fmt_vid_cap(file, priv, f);
-}
-
-static int vivid_try_fmt_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch(file, priv, f);
- return vidioc_try_fmt_vid_cap(file, priv, f);
-}
-
-static int vivid_s_fmt_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch(file, priv, f);
- return vidioc_s_fmt_vid_cap(file, priv, f);
-}
-
-static int vivid_g_fmt_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch_mplane(file, priv, f);
- return vidioc_g_fmt_vid_cap_mplane(file, priv, f);
-}
-
-static int vivid_try_fmt_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch_mplane(file, priv, f);
- return vidioc_try_fmt_vid_cap_mplane(file, priv, f);
-}
-
-static int vivid_s_fmt_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct video_device *vdev = video_devdata(file);
-
- if (vdev->vfl_type == VFL_TYPE_TOUCH)
- return vivid_g_fmt_tch_mplane(file, priv, f);
- return vidioc_s_fmt_vid_cap_mplane(file, priv, f);
-}
-
static bool vivid_is_in_use(bool valid, struct video_device *vdev)
{
unsigned long flags;
@@ -730,21 +660,25 @@ static int vidioc_create_bufs(struct file *file, void *priv,
static const struct v4l2_ioctl_ops vivid_ioctl_ops = {
.vidioc_querycap = vidioc_querycap,
- .vidioc_enum_fmt_vid_cap = vivid_enum_fmt_cap,
- .vidioc_g_fmt_vid_cap = vivid_g_fmt_cap,
- .vidioc_try_fmt_vid_cap = vivid_try_fmt_cap,
- .vidioc_s_fmt_vid_cap = vivid_s_fmt_cap,
- .vidioc_g_fmt_vid_cap_mplane = vivid_g_fmt_cap_mplane,
- .vidioc_try_fmt_vid_cap_mplane = vivid_try_fmt_cap_mplane,
- .vidioc_s_fmt_vid_cap_mplane = vivid_s_fmt_cap_mplane,
-
- .vidioc_enum_fmt_vid_out = vivid_enum_fmt_vid,
- .vidioc_g_fmt_vid_out = vidioc_g_fmt_vid_out,
- .vidioc_try_fmt_vid_out = vidioc_try_fmt_vid_out,
- .vidioc_s_fmt_vid_out = vidioc_s_fmt_vid_out,
- .vidioc_g_fmt_vid_out_mplane = vidioc_g_fmt_vid_out_mplane,
- .vidioc_try_fmt_vid_out_mplane = vidioc_try_fmt_vid_out_mplane,
- .vidioc_s_fmt_vid_out_mplane = vidioc_s_fmt_vid_out_mplane,
+ /* touch device would still use old ioctl() */
+ .vidioc_enum_fmt_vid_cap = vivid_enum_fmt_tch,
+ .vidioc_g_fmt_vid_cap = vivid_g_fmt_tch,
+ .vidioc_try_fmt_vid_cap = vivid_g_fmt_tch,
+ .vidioc_s_fmt_vid_cap = vivid_g_fmt_tch,
+ .vidioc_g_fmt_vid_cap_mplane = vivid_g_fmt_tch_mplane,
+ .vidioc_try_fmt_vid_cap_mplane = vivid_g_fmt_tch_mplane,
+ .vidioc_s_fmt_vid_cap_mplane = vivid_g_fmt_tch_mplane,
+
+ /* force the userspace using new ioctl() */
+ .vidioc_enum_fmt_vid_cap = vivid_enum_fmt_vid,
+ .vidioc_g_ext_pix_fmt_vid_cap = vivid_g_fmt_vid_cap,
+ .vidioc_try_ext_pix_fmt_vid_cap = vivid_try_fmt_vid_cap,
+ .vidioc_s_ext_pix_fmt_vid_cap = vivid_s_fmt_vid_cap,
+
+ .vidioc_enum_fmt_vid_out = vivid_enum_fmt_vid,
+ .vidioc_g_ext_pix_fmt_vid_out = vivid_g_fmt_vid_out,
+ .vidioc_try_ext_pix_fmt_vid_out = vivid_try_fmt_vid_out,
+ .vidioc_s_ext_pix_fmt_vid_out = vivid_s_fmt_vid_out,
.vidioc_g_selection = vidioc_g_selection,
.vidioc_s_selection = vidioc_s_selection,
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-cap.c b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
index c0999581c599..3e3a94a2e3d6 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-cap.c
+++ b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
@@ -14,6 +14,7 @@
#include <media/v4l2-common.h>
#include <media/v4l2-event.h>
#include <media/v4l2-dv-timings.h>
+#include <media/v4l2-ioctl.h>
#include <media/v4l2-rect.h>
#include "vivid-core.h"
@@ -539,61 +540,59 @@ static unsigned vivid_quantization_cap(struct vivid_dev *dev)
}
int vivid_g_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
struct vivid_dev *dev = video_drvdata(file);
- struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
unsigned p;
- mp->width = dev->fmt_cap_rect.width;
- mp->height = dev->fmt_cap_rect.height;
- mp->field = dev->field_cap;
- mp->pixelformat = dev->fmt_cap->fourcc;
- mp->colorspace = vivid_colorspace_cap(dev);
- mp->xfer_func = vivid_xfer_func_cap(dev);
+ f->width = dev->fmt_cap_rect.width;
+ f->height = dev->fmt_cap_rect.height;
+ f->field = dev->field_cap;
+ f->pixelformat = dev->fmt_cap->fourcc;
+ f->colorspace = vivid_colorspace_cap(dev);
+ f->xfer_func = vivid_xfer_func_cap(dev);
if (dev->fmt_cap->color_enc == TGP_COLOR_ENC_HSV)
- mp->hsv_enc = vivid_hsv_enc_cap(dev);
+ f->hsv_enc = vivid_hsv_enc_cap(dev);
else
- mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
- mp->quantization = vivid_quantization_cap(dev);
- mp->num_planes = dev->fmt_cap->buffers;
- for (p = 0; p < mp->num_planes; p++) {
- mp->plane_fmt[p].bytesperline = tpg_g_bytesperline(&dev->tpg, p);
- mp->plane_fmt[p].sizeimage =
- (tpg_g_line_width(&dev->tpg, p) * mp->height) /
+ f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
+ f->quantization = vivid_quantization_cap(dev);
+ memset(f->plane_fmt, 0, sizeof(f->plane_fmt));
+ for (p = 0; p < dev->fmt_cap->buffers; p++) {
+ f->plane_fmt[p].bytesperline = tpg_g_bytesperline(&dev->tpg, p);
+ f->plane_fmt[p].sizeimage =
+ (tpg_g_line_width(&dev->tpg, p) * f->height) /
dev->fmt_cap->vdownsampling[p] +
dev->fmt_cap->data_offset[p];
}
+
return 0;
}
int vivid_try_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
- struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
- struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
struct vivid_dev *dev = video_drvdata(file);
+ struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
const struct vivid_fmt *fmt;
unsigned bytesperline, max_bpl;
unsigned factor = 1;
unsigned w, h;
unsigned p;
- bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
- fmt = vivid_get_format(dev, mp->pixelformat);
+ fmt = vivid_get_format(dev, f->pixelformat);
if (!fmt) {
dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
- mp->pixelformat);
- mp->pixelformat = V4L2_PIX_FMT_YUYV;
- fmt = vivid_get_format(dev, mp->pixelformat);
+ f->pixelformat);
+ f->pixelformat = V4L2_PIX_FMT_YUYV;
+ fmt = vivid_get_format(dev, f->pixelformat);
}
- mp->field = vivid_field_cap(dev, mp->field);
+ f->field = vivid_field_cap(dev, f->field);
if (vivid_is_webcam(dev)) {
const struct v4l2_frmsize_discrete *sz =
v4l2_find_nearest_size(webcam_sizes,
VIVID_WEBCAM_SIZES, width,
- height, mp->width, mp->height);
+ height, f->width, f->height);
w = sz->width;
h = sz->height;
@@ -604,14 +603,14 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
w = dev->src_rect.width;
h = dev->src_rect.height;
}
- if (V4L2_FIELD_HAS_T_OR_B(mp->field))
+ if (V4L2_FIELD_HAS_T_OR_B(f->field))
factor = 2;
if (vivid_is_webcam(dev) ||
(!dev->has_scaler_cap && !dev->has_crop_cap && !dev->has_compose_cap)) {
- mp->width = w;
- mp->height = h / factor;
+ f->width = w;
+ f->height = h / factor;
} else {
- struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
+ struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
v4l2_rect_set_min_size(&r, &vivid_min_rect);
v4l2_rect_set_max_size(&r, &vivid_max_rect);
@@ -624,16 +623,15 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
} else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
v4l2_rect_set_min_size(&r, &dev->src_rect);
}
- mp->width = r.width;
- mp->height = r.height / factor;
+ f->width = r.width;
+ f->height = r.height / factor;
}
/* This driver supports custom bytesperline values */
- mp->num_planes = fmt->buffers;
for (p = 0; p < fmt->buffers; p++) {
/* Calculate the minimum supported bytesperline value */
- bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
+ bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
/* Calculate the maximum supported bytesperline value */
max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
@@ -642,48 +640,49 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
if (pfmt[p].bytesperline < bytesperline)
pfmt[p].bytesperline = bytesperline;
- pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
+ pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
fmt->vdownsampling[p] + fmt->data_offset[p];
-
- memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
}
+
+ if (p < VIDEO_MAX_PLANES)
+ pfmt[p].sizeimage = 0;
+
for (p = fmt->buffers; p < fmt->planes; p++)
- pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
+ pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
(fmt->bit_depth[p] / fmt->vdownsampling[p])) /
(fmt->bit_depth[0] / fmt->vdownsampling[0]);
- if (!user_set_csc || !v4l2_is_colorspace_valid(mp->colorspace))
- mp->colorspace = vivid_colorspace_cap(dev);
+ if (!v4l2_is_colorspace_valid(f->colorspace))
+ f->colorspace = vivid_colorspace_cap(dev);
- if (!user_set_csc || !v4l2_is_xfer_func_valid(mp->xfer_func))
- mp->xfer_func = vivid_xfer_func_cap(dev);
+ if (!v4l2_is_xfer_func_valid(f->xfer_func))
+ f->xfer_func = vivid_xfer_func_cap(dev);
if (fmt->color_enc == TGP_COLOR_ENC_HSV) {
- if (!user_set_csc || !v4l2_is_hsv_enc_valid(mp->hsv_enc))
- mp->hsv_enc = vivid_hsv_enc_cap(dev);
+ if (!v4l2_is_hsv_enc_valid(f->hsv_enc))
+ f->hsv_enc = vivid_hsv_enc_cap(dev);
} else if (fmt->color_enc == TGP_COLOR_ENC_YCBCR) {
- if (!user_set_csc || !v4l2_is_ycbcr_enc_valid(mp->ycbcr_enc))
- mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
+ if (!v4l2_is_ycbcr_enc_valid(f->ycbcr_enc))
+ f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
} else {
- mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
+ f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
}
if (fmt->color_enc == TGP_COLOR_ENC_YCBCR ||
fmt->color_enc == TGP_COLOR_ENC_RGB) {
- if (!user_set_csc || !v4l2_is_quant_valid(mp->quantization))
- mp->quantization = vivid_quantization_cap(dev);
+ if (!v4l2_is_quant_valid(f->quantization))
+ f->quantization = vivid_quantization_cap(dev);
} else {
- mp->quantization = vivid_quantization_cap(dev);
+ f->quantization = vivid_quantization_cap(dev);
}
- memset(mp->reserved, 0, sizeof(mp->reserved));
+ memset(f->reserved, 0, sizeof(f->reserved));
return 0;
}
int vivid_s_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
- struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
struct vivid_dev *dev = video_drvdata(file);
struct v4l2_rect *crop = &dev->crop_cap;
struct v4l2_rect *compose = &dev->compose_cap;
@@ -701,20 +700,21 @@ int vivid_s_fmt_vid_cap(struct file *file, void *priv,
return -EBUSY;
}
- if (dev->overlay_cap_owner && dev->fb_cap.fmt.pixelformat != mp->pixelformat) {
+ if (dev->overlay_cap_owner &&
+ dev->fb_cap.fmt.pixelformat != f->pixelformat) {
dprintk(dev, 1, "overlay is active, can't change pixelformat\n");
return -EBUSY;
}
- dev->fmt_cap = vivid_get_format(dev, mp->pixelformat);
- if (V4L2_FIELD_HAS_T_OR_B(mp->field))
+ dev->fmt_cap = vivid_get_format(dev, f->pixelformat);
+ if (V4L2_FIELD_HAS_T_OR_B(f->field))
factor = 2;
/* Note: the webcam input doesn't support scaling, cropping or composing */
if (!vivid_is_webcam(dev) &&
(dev->has_scaler_cap || dev->has_crop_cap || dev->has_compose_cap)) {
- struct v4l2_rect r = { 0, 0, mp->width, mp->height };
+ struct v4l2_rect r = { 0, 0, f->width, f->height };
if (dev->has_scaler_cap) {
if (dev->has_compose_cap)
@@ -775,107 +775,47 @@ int vivid_s_fmt_vid_cap(struct file *file, void *priv,
} else if (vivid_is_webcam(dev)) {
/* Guaranteed to be a match */
for (i = 0; i < ARRAY_SIZE(webcam_sizes); i++)
- if (webcam_sizes[i].width == mp->width &&
- webcam_sizes[i].height == mp->height)
+ if (webcam_sizes[i].width == f->width &&
+ webcam_sizes[i].height == f->height)
break;
dev->webcam_size_idx = i;
if (dev->webcam_ival_idx >= 2 * (VIVID_WEBCAM_SIZES - i))
dev->webcam_ival_idx = 2 * (VIVID_WEBCAM_SIZES - i) - 1;
vivid_update_format_cap(dev, false);
} else {
- struct v4l2_rect r = { 0, 0, mp->width, mp->height };
+ struct v4l2_rect r = { 0, 0, f->width, f->height };
v4l2_rect_set_size_to(compose, &r);
r.height *= factor;
v4l2_rect_set_size_to(crop, &r);
}
- dev->fmt_cap_rect.width = mp->width;
- dev->fmt_cap_rect.height = mp->height;
- tpg_s_buf_height(&dev->tpg, mp->height);
+ dev->fmt_cap_rect.width = f->width;
+ dev->fmt_cap_rect.height = f->height;
+ tpg_s_buf_height(&dev->tpg, f->height);
tpg_s_fourcc(&dev->tpg, dev->fmt_cap->fourcc);
for (p = 0; p < tpg_g_buffers(&dev->tpg); p++)
- tpg_s_bytesperline(&dev->tpg, p, mp->plane_fmt[p].bytesperline);
- dev->field_cap = mp->field;
+ tpg_s_bytesperline(&dev->tpg, p, f->plane_fmt[p].bytesperline);
+ dev->field_cap = f->field;
if (dev->field_cap == V4L2_FIELD_ALTERNATE)
tpg_s_field(&dev->tpg, V4L2_FIELD_TOP, true);
else
tpg_s_field(&dev->tpg, dev->field_cap, false);
tpg_s_crop_compose(&dev->tpg, &dev->crop_cap, &dev->compose_cap);
if (vivid_is_sdtv_cap(dev))
- dev->tv_field_cap = mp->field;
+ dev->tv_field_cap = f->field;
tpg_update_mv_step(&dev->tpg);
- dev->tpg.colorspace = mp->colorspace;
- dev->tpg.xfer_func = mp->xfer_func;
+ dev->tpg.colorspace = f->colorspace;
+ dev->tpg.xfer_func = f->xfer_func;
if (dev->fmt_cap->color_enc == TGP_COLOR_ENC_YCBCR)
- dev->tpg.ycbcr_enc = mp->ycbcr_enc;
+ dev->tpg.ycbcr_enc = f->ycbcr_enc;
else
- dev->tpg.hsv_enc = mp->hsv_enc;
- dev->tpg.quantization = mp->quantization;
+ dev->tpg.hsv_enc = f->hsv_enc;
+ dev->tpg.quantization = f->quantization;
return 0;
}
-int vidioc_g_fmt_vid_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (!dev->multiplanar)
- return -ENOTTY;
- return vivid_g_fmt_vid_cap(file, priv, f);
-}
-
-int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (!dev->multiplanar)
- return -ENOTTY;
- return vivid_try_fmt_vid_cap(file, priv, f);
-}
-
-int vidioc_s_fmt_vid_cap_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (!dev->multiplanar)
- return -ENOTTY;
- return vivid_s_fmt_vid_cap(file, priv, f);
-}
-
-int vidioc_g_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (dev->multiplanar)
- return -ENOTTY;
- return fmt_sp2mp_func(file, priv, f, vivid_g_fmt_vid_cap);
-}
-
-int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (dev->multiplanar)
- return -ENOTTY;
- return fmt_sp2mp_func(file, priv, f, vivid_try_fmt_vid_cap);
-}
-
-int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (dev->multiplanar)
- return -ENOTTY;
- return fmt_sp2mp_func(file, priv, f, vivid_s_fmt_vid_cap);
-}
-
int vivid_vid_cap_g_selection(struct file *file, void *priv,
struct v4l2_selection *sel)
{
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-cap.h b/drivers/media/test-drivers/vivid/vivid-vid-cap.h
index 1e422a59eeab..7c9fc5c787b5 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-cap.h
+++ b/drivers/media/test-drivers/vivid/vivid-vid-cap.h
@@ -17,15 +17,12 @@ extern const char * const vivid_ctrl_standard_strings[];
extern const struct vb2_ops vivid_vid_cap_qops;
-int vivid_g_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f);
-int vivid_try_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f);
-int vivid_s_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_g_fmt_vid_cap_mplane(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_try_fmt_vid_cap_mplane(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_s_fmt_vid_cap_mplane(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_g_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_try_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_s_fmt_vid_cap(struct file *file, void *priv, struct v4l2_format *f);
+int vivid_g_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_ext_pix_format *f);
+int vivid_try_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_ext_pix_format *f);
+int vivid_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_ext_pix_format *f);
int vivid_vid_cap_g_selection(struct file *file, void *priv, struct v4l2_selection *sel);
int vivid_vid_cap_s_selection(struct file *file, void *fh, struct v4l2_selection *s);
int vivid_vid_cap_g_pixelaspect(struct file *file, void *priv, int type, struct v4l2_fract *f);
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-out.c b/drivers/media/test-drivers/vivid/vivid-vid-out.c
index 9f731f085179..936ba93c00cc 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-out.c
+++ b/drivers/media/test-drivers/vivid/vivid-vid-out.c
@@ -13,6 +13,7 @@
#include <media/v4l2-common.h>
#include <media/v4l2-event.h>
#include <media/v4l2-dv-timings.h>
+#include <media/v4l2-ioctl.h>
#include <media/v4l2-rect.h>
#include "vivid-core.h"
@@ -315,59 +316,57 @@ static enum tpg_pixel_aspect vivid_get_pixel_aspect(const struct vivid_dev *dev)
}
int vivid_g_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
struct vivid_dev *dev = video_drvdata(file);
- struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
const struct vivid_fmt *fmt = dev->fmt_out;
unsigned p;
- mp->width = dev->fmt_out_rect.width;
- mp->height = dev->fmt_out_rect.height;
- mp->field = dev->field_out;
- mp->pixelformat = fmt->fourcc;
- mp->colorspace = dev->colorspace_out;
- mp->xfer_func = dev->xfer_func_out;
- mp->ycbcr_enc = dev->ycbcr_enc_out;
- mp->quantization = dev->quantization_out;
- mp->num_planes = fmt->buffers;
- for (p = 0; p < mp->num_planes; p++) {
- mp->plane_fmt[p].bytesperline = dev->bytesperline_out[p];
- mp->plane_fmt[p].sizeimage =
- mp->plane_fmt[p].bytesperline * mp->height +
+ f->width = dev->fmt_out_rect.width;
+ f->height = dev->fmt_out_rect.height;
+ f->field = dev->field_out;
+ f->pixelformat = fmt->fourcc;
+ f->colorspace = dev->colorspace_out;
+ f->xfer_func = dev->xfer_func_out;
+ f->ycbcr_enc = dev->ycbcr_enc_out;
+ f->quantization = dev->quantization_out;
+ memset(f->plane_fmt, 0, sizeof(f->plane_fmt));
+ for (p = 0; p < fmt->buffers; p++) {
+ f->plane_fmt[p].bytesperline = dev->bytesperline_out[p];
+ f->plane_fmt[p].sizeimage =
+ f->plane_fmt[p].bytesperline * f->height +
fmt->data_offset[p];
}
for (p = fmt->buffers; p < fmt->planes; p++) {
unsigned stride = dev->bytesperline_out[p];
- mp->plane_fmt[0].sizeimage +=
- (stride * mp->height) / fmt->vdownsampling[p];
+ f->plane_fmt[0].sizeimage +=
+ (stride * f->height) / fmt->vdownsampling[p];
}
return 0;
}
int vivid_try_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
struct vivid_dev *dev = video_drvdata(file);
+ struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
struct v4l2_bt_timings *bt = &dev->dv_timings_out.bt;
- struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
- struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
const struct vivid_fmt *fmt;
unsigned bytesperline, max_bpl;
unsigned factor = 1;
unsigned w, h;
unsigned p;
- fmt = vivid_get_format(dev, mp->pixelformat);
+ fmt = vivid_get_format(dev, f->pixelformat);
if (!fmt) {
dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
- mp->pixelformat);
- mp->pixelformat = V4L2_PIX_FMT_YUYV;
- fmt = vivid_get_format(dev, mp->pixelformat);
+ f->pixelformat);
+ f->pixelformat = V4L2_PIX_FMT_YUYV;
+ fmt = vivid_get_format(dev, f->pixelformat);
}
- mp->field = vivid_field_out(dev, mp->field);
+ f->field = vivid_field_out(dev, f->field);
if (vivid_is_svid_out(dev)) {
w = 720;
h = (dev->std_out & V4L2_STD_525_60) ? 480 : 576;
@@ -375,13 +374,13 @@ int vivid_try_fmt_vid_out(struct file *file, void *priv,
w = dev->sink_rect.width;
h = dev->sink_rect.height;
}
- if (V4L2_FIELD_HAS_T_OR_B(mp->field))
+ if (V4L2_FIELD_HAS_T_OR_B(f->field))
factor = 2;
if (!dev->has_scaler_out && !dev->has_crop_out && !dev->has_compose_out) {
- mp->width = w;
- mp->height = h / factor;
+ f->width = w;
+ f->height = h / factor;
} else {
- struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
+ struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
v4l2_rect_set_min_size(&r, &vivid_min_rect);
v4l2_rect_set_max_size(&r, &vivid_max_rect);
@@ -394,16 +393,15 @@ int vivid_try_fmt_vid_out(struct file *file, void *priv,
} else if (!dev->has_scaler_out && !dev->has_compose_out) {
v4l2_rect_set_min_size(&r, &dev->sink_rect);
}
- mp->width = r.width;
- mp->height = r.height / factor;
+ f->width = r.width;
+ f->height = r.height / factor;
}
/* This driver supports custom bytesperline values */
- mp->num_planes = fmt->buffers;
for (p = 0; p < fmt->buffers; p++) {
/* Calculate the minimum supported bytesperline value */
- bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
+ bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
/* Calculate the maximum supported bytesperline value */
max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
@@ -412,42 +410,41 @@ int vivid_try_fmt_vid_out(struct file *file, void *priv,
if (pfmt[p].bytesperline < bytesperline)
pfmt[p].bytesperline = bytesperline;
- pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
+ pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
fmt->vdownsampling[p] + fmt->data_offset[p];
- memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
}
+ if (p < VIDEO_MAX_PLANES)
+ pfmt[p].sizeimage = 0;
for (p = fmt->buffers; p < fmt->planes; p++)
- pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
+ pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
(fmt->bit_depth[p] / fmt->vdownsampling[p])) /
(fmt->bit_depth[0] / fmt->vdownsampling[0]);
- mp->xfer_func = V4L2_XFER_FUNC_DEFAULT;
- mp->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
- mp->quantization = V4L2_QUANTIZATION_DEFAULT;
+ f->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+ f->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+ f->quantization = V4L2_QUANTIZATION_DEFAULT;
if (vivid_is_svid_out(dev)) {
- mp->colorspace = V4L2_COLORSPACE_SMPTE170M;
+ f->colorspace = V4L2_COLORSPACE_SMPTE170M;
} else if (dev->dvi_d_out || !(bt->flags & V4L2_DV_FL_IS_CE_VIDEO)) {
- mp->colorspace = V4L2_COLORSPACE_SRGB;
+ f->colorspace = V4L2_COLORSPACE_SRGB;
if (dev->dvi_d_out)
- mp->quantization = V4L2_QUANTIZATION_LIM_RANGE;
+ f->quantization = V4L2_QUANTIZATION_LIM_RANGE;
} else if (bt->width == 720 && bt->height <= 576) {
- mp->colorspace = V4L2_COLORSPACE_SMPTE170M;
- } else if (mp->colorspace != V4L2_COLORSPACE_SMPTE170M &&
- mp->colorspace != V4L2_COLORSPACE_REC709 &&
- mp->colorspace != V4L2_COLORSPACE_OPRGB &&
- mp->colorspace != V4L2_COLORSPACE_BT2020 &&
- mp->colorspace != V4L2_COLORSPACE_SRGB) {
- mp->colorspace = V4L2_COLORSPACE_REC709;
+ f->colorspace = V4L2_COLORSPACE_SMPTE170M;
+ } else if (f->colorspace != V4L2_COLORSPACE_SMPTE170M &&
+ f->colorspace != V4L2_COLORSPACE_REC709 &&
+ f->colorspace != V4L2_COLORSPACE_OPRGB &&
+ f->colorspace != V4L2_COLORSPACE_BT2020 &&
+ f->colorspace != V4L2_COLORSPACE_SRGB) {
+ f->colorspace = V4L2_COLORSPACE_REC709;
}
- memset(mp->reserved, 0, sizeof(mp->reserved));
return 0;
}
int vivid_s_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
- struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
struct vivid_dev *dev = video_drvdata(file);
struct v4l2_rect *crop = &dev->crop_out;
struct v4l2_rect *compose = &dev->compose_out;
@@ -461,10 +458,10 @@ int vivid_s_fmt_vid_out(struct file *file, void *priv,
if (vb2_is_busy(q) &&
(vivid_is_svid_out(dev) ||
- mp->width != dev->fmt_out_rect.width ||
- mp->height != dev->fmt_out_rect.height ||
- mp->pixelformat != dev->fmt_out->fourcc ||
- mp->field != dev->field_out)) {
+ f->width != dev->fmt_out_rect.width ||
+ f->height != dev->fmt_out_rect.height ||
+ f->pixelformat != dev->fmt_out->fourcc ||
+ f->field != dev->field_out)) {
dprintk(dev, 1, "%s device busy\n", __func__);
return -EBUSY;
}
@@ -477,12 +474,12 @@ int vivid_s_fmt_vid_out(struct file *file, void *priv,
if (vb2_is_busy(q))
goto set_colorspace;
- dev->fmt_out = vivid_get_format(dev, mp->pixelformat);
- if (V4L2_FIELD_HAS_T_OR_B(mp->field))
+ dev->fmt_out = vivid_get_format(dev, f->pixelformat);
+ if (V4L2_FIELD_HAS_T_OR_B(f->field))
factor = 2;
if (dev->has_scaler_out || dev->has_crop_out || dev->has_compose_out) {
- struct v4l2_rect r = { 0, 0, mp->width, mp->height };
+ struct v4l2_rect r = { 0, 0, f->width, f->height };
if (dev->has_scaler_out) {
if (dev->has_crop_out)
@@ -541,30 +538,30 @@ int vivid_s_fmt_vid_out(struct file *file, void *priv,
crop->height /= factor;
}
} else {
- struct v4l2_rect r = { 0, 0, mp->width, mp->height };
+ struct v4l2_rect r = { 0, 0, f->width, f->height };
v4l2_rect_set_size_to(crop, &r);
r.height /= factor;
v4l2_rect_set_size_to(compose, &r);
}
- dev->fmt_out_rect.width = mp->width;
- dev->fmt_out_rect.height = mp->height;
- for (p = 0; p < mp->num_planes; p++)
- dev->bytesperline_out[p] = mp->plane_fmt[p].bytesperline;
+ dev->fmt_out_rect.width = f->width;
+ dev->fmt_out_rect.height = f->height;
+ for (p = 0; p < VIDEO_MAX_PLANES && f->plane_fmt[p].sizeimage; p++)
+ dev->bytesperline_out[p] = f->plane_fmt[p].bytesperline;
for (p = dev->fmt_out->buffers; p < dev->fmt_out->planes; p++)
dev->bytesperline_out[p] =
(dev->bytesperline_out[0] * dev->fmt_out->bit_depth[p]) /
dev->fmt_out->bit_depth[0];
- dev->field_out = mp->field;
+ dev->field_out = f->field;
if (vivid_is_svid_out(dev))
- dev->tv_field_out = mp->field;
+ dev->tv_field_out = f->field;
set_colorspace:
- dev->colorspace_out = mp->colorspace;
- dev->xfer_func_out = mp->xfer_func;
- dev->ycbcr_enc_out = mp->ycbcr_enc;
- dev->quantization_out = mp->quantization;
+ dev->colorspace_out = f->colorspace;
+ dev->xfer_func_out = f->xfer_func;
+ dev->ycbcr_enc_out = f->ycbcr_enc;
+ dev->quantization_out = f->quantization;
if (dev->loop_video) {
vivid_send_source_change(dev, SVID);
vivid_send_source_change(dev, HDMI);
@@ -572,66 +569,6 @@ int vivid_s_fmt_vid_out(struct file *file, void *priv,
return 0;
}
-int vidioc_g_fmt_vid_out_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (!dev->multiplanar)
- return -ENOTTY;
- return vivid_g_fmt_vid_out(file, priv, f);
-}
-
-int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (!dev->multiplanar)
- return -ENOTTY;
- return vivid_try_fmt_vid_out(file, priv, f);
-}
-
-int vidioc_s_fmt_vid_out_mplane(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (!dev->multiplanar)
- return -ENOTTY;
- return vivid_s_fmt_vid_out(file, priv, f);
-}
-
-int vidioc_g_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (dev->multiplanar)
- return -ENOTTY;
- return fmt_sp2mp_func(file, priv, f, vivid_g_fmt_vid_out);
-}
-
-int vidioc_try_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (dev->multiplanar)
- return -ENOTTY;
- return fmt_sp2mp_func(file, priv, f, vivid_try_fmt_vid_out);
-}
-
-int vidioc_s_fmt_vid_out(struct file *file, void *priv,
- struct v4l2_format *f)
-{
- struct vivid_dev *dev = video_drvdata(file);
-
- if (dev->multiplanar)
- return -ENOTTY;
- return fmt_sp2mp_func(file, priv, f, vivid_s_fmt_vid_out);
-}
-
int vivid_vid_out_g_selection(struct file *file, void *priv,
struct v4l2_selection *sel)
{
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-out.h b/drivers/media/test-drivers/vivid/vivid-vid-out.h
index 8d56314f4ea1..b84dc578af36 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-out.h
+++ b/drivers/media/test-drivers/vivid/vivid-vid-out.h
@@ -12,15 +12,12 @@ extern const struct vb2_ops vivid_vid_out_qops;
void vivid_update_format_out(struct vivid_dev *dev);
-int vivid_g_fmt_vid_out(struct file *file, void *priv, struct v4l2_format *f);
-int vivid_try_fmt_vid_out(struct file *file, void *priv, struct v4l2_format *f);
-int vivid_s_fmt_vid_out(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_g_fmt_vid_out_mplane(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_try_fmt_vid_out_mplane(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_s_fmt_vid_out_mplane(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_g_fmt_vid_out(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_try_fmt_vid_out(struct file *file, void *priv, struct v4l2_format *f);
-int vidioc_s_fmt_vid_out(struct file *file, void *priv, struct v4l2_format *f);
+int vivid_g_fmt_vid_out(struct file *file, void *priv,
+ struct v4l2_ext_pix_format *f);
+int vivid_try_fmt_vid_out(struct file *file, void *priv,
+ struct v4l2_ext_pix_format *f);
+int vivid_s_fmt_vid_out(struct file *file, void *priv,
+ struct v4l2_ext_pix_format *f);
int vivid_vid_out_g_selection(struct file *file, void *priv, struct v4l2_selection *sel);
int vivid_vid_out_s_selection(struct file *file, void *fh, struct v4l2_selection *s);
int vivid_vid_out_g_pixelaspect(struct file *file, void *priv, int type, struct v4l2_fract *f);
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread* Re: [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-02-06 4:33 ` [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format ayaka
@ 2023-07-13 10:39 ` Tomasz Figa
2023-07-17 16:00 ` Randy Li
0 siblings, 1 reply; 21+ messages in thread
From: Tomasz Figa @ 2023-07-13 10:39 UTC (permalink / raw)
To: ayaka
Cc: linux-media, randy.li, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, nicolas, sakari.ailus,
stanimir.varbanov, Helen Koike
On Mon, Feb 06, 2023 at 12:33:01PM +0800, ayaka wrote:
> From: Helen Koike <helen.koike@collabora.com>
>
> Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
>
> Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
> framework to use the ext hooks when the classic Api is used from
> userspace in Vid devices, and to keep touch devices with classic hook.
>
> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> Signed-off-by: Helen Koike <helen.koike@collabora.com>
> ---
> Changes in v7:
> - Force the userspace using the new APIs to operate non-touch drivers.
The primary objective of Linux development is not to break the
userspace. We can't just remove the old API, especially not from
existing drivers.
[snip]
> int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> - struct v4l2_format *f)
> + struct v4l2_ext_pix_format *f)
> {
> - struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
> - struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
> struct vivid_dev *dev = video_drvdata(file);
> + struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
> const struct vivid_fmt *fmt;
> unsigned bytesperline, max_bpl;
> unsigned factor = 1;
> unsigned w, h;
> unsigned p;
> - bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
Why is this condition being removed?
Best regards,
Tomasz
>
> - fmt = vivid_get_format(dev, mp->pixelformat);
> + fmt = vivid_get_format(dev, f->pixelformat);
> if (!fmt) {
> dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
> - mp->pixelformat);
> - mp->pixelformat = V4L2_PIX_FMT_YUYV;
> - fmt = vivid_get_format(dev, mp->pixelformat);
> + f->pixelformat);
> + f->pixelformat = V4L2_PIX_FMT_YUYV;
> + fmt = vivid_get_format(dev, f->pixelformat);
> }
>
> - mp->field = vivid_field_cap(dev, mp->field);
> + f->field = vivid_field_cap(dev, f->field);
> if (vivid_is_webcam(dev)) {
> const struct v4l2_frmsize_discrete *sz =
> v4l2_find_nearest_size(webcam_sizes,
> VIVID_WEBCAM_SIZES, width,
> - height, mp->width, mp->height);
> + height, f->width, f->height);
>
> w = sz->width;
> h = sz->height;
> @@ -604,14 +603,14 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> w = dev->src_rect.width;
> h = dev->src_rect.height;
> }
> - if (V4L2_FIELD_HAS_T_OR_B(mp->field))
> + if (V4L2_FIELD_HAS_T_OR_B(f->field))
> factor = 2;
> if (vivid_is_webcam(dev) ||
> (!dev->has_scaler_cap && !dev->has_crop_cap && !dev->has_compose_cap)) {
> - mp->width = w;
> - mp->height = h / factor;
> + f->width = w;
> + f->height = h / factor;
> } else {
> - struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
> + struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
>
> v4l2_rect_set_min_size(&r, &vivid_min_rect);
> v4l2_rect_set_max_size(&r, &vivid_max_rect);
> @@ -624,16 +623,15 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> } else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
> v4l2_rect_set_min_size(&r, &dev->src_rect);
> }
> - mp->width = r.width;
> - mp->height = r.height / factor;
> + f->width = r.width;
> + f->height = r.height / factor;
> }
>
> /* This driver supports custom bytesperline values */
>
> - mp->num_planes = fmt->buffers;
> for (p = 0; p < fmt->buffers; p++) {
> /* Calculate the minimum supported bytesperline value */
> - bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
> + bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
> /* Calculate the maximum supported bytesperline value */
> max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
>
> @@ -642,48 +640,49 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> if (pfmt[p].bytesperline < bytesperline)
> pfmt[p].bytesperline = bytesperline;
>
> - pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
> + pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
> fmt->vdownsampling[p] + fmt->data_offset[p];
> -
> - memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
> }
> +
> + if (p < VIDEO_MAX_PLANES)
> + pfmt[p].sizeimage = 0;
> +
> for (p = fmt->buffers; p < fmt->planes; p++)
> - pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
> + pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
> (fmt->bit_depth[p] / fmt->vdownsampling[p])) /
> (fmt->bit_depth[0] / fmt->vdownsampling[0]);
>
> - if (!user_set_csc || !v4l2_is_colorspace_valid(mp->colorspace))
> - mp->colorspace = vivid_colorspace_cap(dev);
> + if (!v4l2_is_colorspace_valid(f->colorspace))
> + f->colorspace = vivid_colorspace_cap(dev);
>
> - if (!user_set_csc || !v4l2_is_xfer_func_valid(mp->xfer_func))
> - mp->xfer_func = vivid_xfer_func_cap(dev);
> + if (!v4l2_is_xfer_func_valid(f->xfer_func))
> + f->xfer_func = vivid_xfer_func_cap(dev);
>
> if (fmt->color_enc == TGP_COLOR_ENC_HSV) {
> - if (!user_set_csc || !v4l2_is_hsv_enc_valid(mp->hsv_enc))
> - mp->hsv_enc = vivid_hsv_enc_cap(dev);
> + if (!v4l2_is_hsv_enc_valid(f->hsv_enc))
> + f->hsv_enc = vivid_hsv_enc_cap(dev);
> } else if (fmt->color_enc == TGP_COLOR_ENC_YCBCR) {
> - if (!user_set_csc || !v4l2_is_ycbcr_enc_valid(mp->ycbcr_enc))
> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> + if (!v4l2_is_ycbcr_enc_valid(f->ycbcr_enc))
> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> } else {
> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> }
>
> if (fmt->color_enc == TGP_COLOR_ENC_YCBCR ||
> fmt->color_enc == TGP_COLOR_ENC_RGB) {
> - if (!user_set_csc || !v4l2_is_quant_valid(mp->quantization))
> - mp->quantization = vivid_quantization_cap(dev);
> + if (!v4l2_is_quant_valid(f->quantization))
> + f->quantization = vivid_quantization_cap(dev);
> } else {
> - mp->quantization = vivid_quantization_cap(dev);
> + f->quantization = vivid_quantization_cap(dev);
> }
>
> - memset(mp->reserved, 0, sizeof(mp->reserved));
> + memset(f->reserved, 0, sizeof(f->reserved));
> return 0;
> }
[snip]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-07-13 10:39 ` Tomasz Figa
@ 2023-07-17 16:00 ` Randy Li
2023-07-20 20:09 ` Nicolas Dufresne
2023-07-28 7:18 ` Tomasz Figa
0 siblings, 2 replies; 21+ messages in thread
From: Randy Li @ 2023-07-17 16:00 UTC (permalink / raw)
To: Tomasz Figa
Cc: linux-media, randy.li, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, nicolas, sakari.ailus,
stanimir.varbanov, Helen Koike
On 2023/7/13 18:39, Tomasz Figa wrote:
> On Mon, Feb 06, 2023 at 12:33:01PM +0800, ayaka wrote:
>> From: Helen Koike <helen.koike@collabora.com>
>>
>> Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
>>
>> Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
>> framework to use the ext hooks when the classic Api is used from
>> userspace in Vid devices, and to keep touch devices with classic hook.
>>
>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
>> Signed-off-by: Helen Koike <helen.koike@collabora.com>
>> ---
>> Changes in v7:
>> - Force the userspace using the new APIs to operate non-touch drivers.
> The primary objective of Linux development is not to break the
> userspace. We can't just remove the old API, especially not from
> existing drivers.
Maybe I should create a new virtual driver here? It is impossible to
support the new fourcc modifier with the old APIs.
>
> [snip]
>> int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>> - struct v4l2_format *f)
>> + struct v4l2_ext_pix_format *f)
>> {
>> - struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
>> - struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
>> struct vivid_dev *dev = video_drvdata(file);
>> + struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
>> const struct vivid_fmt *fmt;
>> unsigned bytesperline, max_bpl;
>> unsigned factor = 1;
>> unsigned w, h;
>> unsigned p;
>> - bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
> Why is this condition being removed?
Because the v4l2_ext_pix has a struct for the colorspace?
Would you like the idea that driver exports a buffer contains all the
info for an enumeration ?
>
> Best regards,
> Tomasz
>
>>
>> - fmt = vivid_get_format(dev, mp->pixelformat);
>> + fmt = vivid_get_format(dev, f->pixelformat);
>> if (!fmt) {
>> dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
>> - mp->pixelformat);
>> - mp->pixelformat = V4L2_PIX_FMT_YUYV;
>> - fmt = vivid_get_format(dev, mp->pixelformat);
>> + f->pixelformat);
>> + f->pixelformat = V4L2_PIX_FMT_YUYV;
>> + fmt = vivid_get_format(dev, f->pixelformat);
>> }
>>
>> - mp->field = vivid_field_cap(dev, mp->field);
>> + f->field = vivid_field_cap(dev, f->field);
>> if (vivid_is_webcam(dev)) {
>> const struct v4l2_frmsize_discrete *sz =
>> v4l2_find_nearest_size(webcam_sizes,
>> VIVID_WEBCAM_SIZES, width,
>> - height, mp->width, mp->height);
>> + height, f->width, f->height);
>>
>> w = sz->width;
>> h = sz->height;
>> @@ -604,14 +603,14 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>> w = dev->src_rect.width;
>> h = dev->src_rect.height;
>> }
>> - if (V4L2_FIELD_HAS_T_OR_B(mp->field))
>> + if (V4L2_FIELD_HAS_T_OR_B(f->field))
>> factor = 2;
>> if (vivid_is_webcam(dev) ||
>> (!dev->has_scaler_cap && !dev->has_crop_cap && !dev->has_compose_cap)) {
>> - mp->width = w;
>> - mp->height = h / factor;
>> + f->width = w;
>> + f->height = h / factor;
>> } else {
>> - struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
>> + struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
>>
>> v4l2_rect_set_min_size(&r, &vivid_min_rect);
>> v4l2_rect_set_max_size(&r, &vivid_max_rect);
>> @@ -624,16 +623,15 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>> } else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
>> v4l2_rect_set_min_size(&r, &dev->src_rect);
>> }
>> - mp->width = r.width;
>> - mp->height = r.height / factor;
>> + f->width = r.width;
>> + f->height = r.height / factor;
>> }
>>
>> /* This driver supports custom bytesperline values */
>>
>> - mp->num_planes = fmt->buffers;
>> for (p = 0; p < fmt->buffers; p++) {
>> /* Calculate the minimum supported bytesperline value */
>> - bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
>> + bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
>> /* Calculate the maximum supported bytesperline value */
>> max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
>>
>> @@ -642,48 +640,49 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>> if (pfmt[p].bytesperline < bytesperline)
>> pfmt[p].bytesperline = bytesperline;
>>
>> - pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
>> + pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
>> fmt->vdownsampling[p] + fmt->data_offset[p];
>> -
>> - memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
>> }
>> +
>> + if (p < VIDEO_MAX_PLANES)
>> + pfmt[p].sizeimage = 0;
>> +
>> for (p = fmt->buffers; p < fmt->planes; p++)
>> - pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
>> + pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
>> (fmt->bit_depth[p] / fmt->vdownsampling[p])) /
>> (fmt->bit_depth[0] / fmt->vdownsampling[0]);
>>
>> - if (!user_set_csc || !v4l2_is_colorspace_valid(mp->colorspace))
>> - mp->colorspace = vivid_colorspace_cap(dev);
>> + if (!v4l2_is_colorspace_valid(f->colorspace))
>> + f->colorspace = vivid_colorspace_cap(dev);
>>
>> - if (!user_set_csc || !v4l2_is_xfer_func_valid(mp->xfer_func))
>> - mp->xfer_func = vivid_xfer_func_cap(dev);
>> + if (!v4l2_is_xfer_func_valid(f->xfer_func))
>> + f->xfer_func = vivid_xfer_func_cap(dev);
>>
>> if (fmt->color_enc == TGP_COLOR_ENC_HSV) {
>> - if (!user_set_csc || !v4l2_is_hsv_enc_valid(mp->hsv_enc))
>> - mp->hsv_enc = vivid_hsv_enc_cap(dev);
>> + if (!v4l2_is_hsv_enc_valid(f->hsv_enc))
>> + f->hsv_enc = vivid_hsv_enc_cap(dev);
>> } else if (fmt->color_enc == TGP_COLOR_ENC_YCBCR) {
>> - if (!user_set_csc || !v4l2_is_ycbcr_enc_valid(mp->ycbcr_enc))
>> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>> + if (!v4l2_is_ycbcr_enc_valid(f->ycbcr_enc))
>> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>> } else {
>> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>> }
>>
>> if (fmt->color_enc == TGP_COLOR_ENC_YCBCR ||
>> fmt->color_enc == TGP_COLOR_ENC_RGB) {
>> - if (!user_set_csc || !v4l2_is_quant_valid(mp->quantization))
>> - mp->quantization = vivid_quantization_cap(dev);
>> + if (!v4l2_is_quant_valid(f->quantization))
>> + f->quantization = vivid_quantization_cap(dev);
>> } else {
>> - mp->quantization = vivid_quantization_cap(dev);
>> + f->quantization = vivid_quantization_cap(dev);
>> }
>>
>> - memset(mp->reserved, 0, sizeof(mp->reserved));
>> + memset(f->reserved, 0, sizeof(f->reserved));
>> return 0;
>> }
> [snip]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-07-17 16:00 ` Randy Li
@ 2023-07-20 20:09 ` Nicolas Dufresne
2023-07-28 7:22 ` Tomasz Figa
2023-07-28 7:18 ` Tomasz Figa
1 sibling, 1 reply; 21+ messages in thread
From: Nicolas Dufresne @ 2023-07-20 20:09 UTC (permalink / raw)
To: Randy Li, Tomasz Figa
Cc: linux-media, randy.li, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, sakari.ailus,
stanimir.varbanov, Helen Koike
Le mardi 18 juillet 2023 à 00:00 +0800, Randy Li a écrit :
> On 2023/7/13 18:39, Tomasz Figa wrote:
> > On Mon, Feb 06, 2023 at 12:33:01PM +0800, ayaka wrote:
> > > From: Helen Koike <helen.koike@collabora.com>
> > >
> > > Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
> > >
> > > Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
> > > framework to use the ext hooks when the classic Api is used from
> > > userspace in Vid devices, and to keep touch devices with classic hook.
> > >
> > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > Signed-off-by: Helen Koike <helen.koike@collabora.com>
> > > ---
> > > Changes in v7:
> > > - Force the userspace using the new APIs to operate non-touch drivers.
> > The primary objective of Linux development is not to break the
> > userspace. We can't just remove the old API, especially not from
> > existing drivers.
> Maybe I should create a new virtual driver here? It is impossible to
> support the new fourcc modifier with the old APIs.
For MPLANE, where backward compatibility was built into libv4l2 LD_PRELOAD
wrapper, it simply failed the cases that could not be supported (non contiguous
planes).
regards,
Nicolas
> >
> > [snip]
> > > int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > - struct v4l2_format *f)
> > > + struct v4l2_ext_pix_format *f)
> > > {
> > > - struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
> > > - struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
> > > struct vivid_dev *dev = video_drvdata(file);
> > > + struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
> > > const struct vivid_fmt *fmt;
> > > unsigned bytesperline, max_bpl;
> > > unsigned factor = 1;
> > > unsigned w, h;
> > > unsigned p;
> > > - bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
> > Why is this condition being removed?
>
> Because the v4l2_ext_pix has a struct for the colorspace?
>
> Would you like the idea that driver exports a buffer contains all the
> info for an enumeration ?
>
> >
> > Best regards,
> > Tomasz
> >
> > >
> > > - fmt = vivid_get_format(dev, mp->pixelformat);
> > > + fmt = vivid_get_format(dev, f->pixelformat);
> > > if (!fmt) {
> > > dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
> > > - mp->pixelformat);
> > > - mp->pixelformat = V4L2_PIX_FMT_YUYV;
> > > - fmt = vivid_get_format(dev, mp->pixelformat);
> > > + f->pixelformat);
> > > + f->pixelformat = V4L2_PIX_FMT_YUYV;
> > > + fmt = vivid_get_format(dev, f->pixelformat);
> > > }
> > >
> > > - mp->field = vivid_field_cap(dev, mp->field);
> > > + f->field = vivid_field_cap(dev, f->field);
> > > if (vivid_is_webcam(dev)) {
> > > const struct v4l2_frmsize_discrete *sz =
> > > v4l2_find_nearest_size(webcam_sizes,
> > > VIVID_WEBCAM_SIZES, width,
> > > - height, mp->width, mp->height);
> > > + height, f->width, f->height);
> > >
> > > w = sz->width;
> > > h = sz->height;
> > > @@ -604,14 +603,14 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > w = dev->src_rect.width;
> > > h = dev->src_rect.height;
> > > }
> > > - if (V4L2_FIELD_HAS_T_OR_B(mp->field))
> > > + if (V4L2_FIELD_HAS_T_OR_B(f->field))
> > > factor = 2;
> > > if (vivid_is_webcam(dev) ||
> > > (!dev->has_scaler_cap && !dev->has_crop_cap && !dev->has_compose_cap)) {
> > > - mp->width = w;
> > > - mp->height = h / factor;
> > > + f->width = w;
> > > + f->height = h / factor;
> > > } else {
> > > - struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
> > > + struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
> > >
> > > v4l2_rect_set_min_size(&r, &vivid_min_rect);
> > > v4l2_rect_set_max_size(&r, &vivid_max_rect);
> > > @@ -624,16 +623,15 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > } else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
> > > v4l2_rect_set_min_size(&r, &dev->src_rect);
> > > }
> > > - mp->width = r.width;
> > > - mp->height = r.height / factor;
> > > + f->width = r.width;
> > > + f->height = r.height / factor;
> > > }
> > >
> > > /* This driver supports custom bytesperline values */
> > >
> > > - mp->num_planes = fmt->buffers;
> > > for (p = 0; p < fmt->buffers; p++) {
> > > /* Calculate the minimum supported bytesperline value */
> > > - bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
> > > + bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
> > > /* Calculate the maximum supported bytesperline value */
> > > max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
> > >
> > > @@ -642,48 +640,49 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > if (pfmt[p].bytesperline < bytesperline)
> > > pfmt[p].bytesperline = bytesperline;
> > >
> > > - pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
> > > + pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
> > > fmt->vdownsampling[p] + fmt->data_offset[p];
> > > -
> > > - memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
> > > }
> > > +
> > > + if (p < VIDEO_MAX_PLANES)
> > > + pfmt[p].sizeimage = 0;
> > > +
> > > for (p = fmt->buffers; p < fmt->planes; p++)
> > > - pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
> > > + pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
> > > (fmt->bit_depth[p] / fmt->vdownsampling[p])) /
> > > (fmt->bit_depth[0] / fmt->vdownsampling[0]);
> > >
> > > - if (!user_set_csc || !v4l2_is_colorspace_valid(mp->colorspace))
> > > - mp->colorspace = vivid_colorspace_cap(dev);
> > > + if (!v4l2_is_colorspace_valid(f->colorspace))
> > > + f->colorspace = vivid_colorspace_cap(dev);
> > >
> > > - if (!user_set_csc || !v4l2_is_xfer_func_valid(mp->xfer_func))
> > > - mp->xfer_func = vivid_xfer_func_cap(dev);
> > > + if (!v4l2_is_xfer_func_valid(f->xfer_func))
> > > + f->xfer_func = vivid_xfer_func_cap(dev);
> > >
> > > if (fmt->color_enc == TGP_COLOR_ENC_HSV) {
> > > - if (!user_set_csc || !v4l2_is_hsv_enc_valid(mp->hsv_enc))
> > > - mp->hsv_enc = vivid_hsv_enc_cap(dev);
> > > + if (!v4l2_is_hsv_enc_valid(f->hsv_enc))
> > > + f->hsv_enc = vivid_hsv_enc_cap(dev);
> > > } else if (fmt->color_enc == TGP_COLOR_ENC_YCBCR) {
> > > - if (!user_set_csc || !v4l2_is_ycbcr_enc_valid(mp->ycbcr_enc))
> > > - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > + if (!v4l2_is_ycbcr_enc_valid(f->ycbcr_enc))
> > > + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > } else {
> > > - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > }
> > >
> > > if (fmt->color_enc == TGP_COLOR_ENC_YCBCR ||
> > > fmt->color_enc == TGP_COLOR_ENC_RGB) {
> > > - if (!user_set_csc || !v4l2_is_quant_valid(mp->quantization))
> > > - mp->quantization = vivid_quantization_cap(dev);
> > > + if (!v4l2_is_quant_valid(f->quantization))
> > > + f->quantization = vivid_quantization_cap(dev);
> > > } else {
> > > - mp->quantization = vivid_quantization_cap(dev);
> > > + f->quantization = vivid_quantization_cap(dev);
> > > }
> > >
> > > - memset(mp->reserved, 0, sizeof(mp->reserved));
> > > + memset(f->reserved, 0, sizeof(f->reserved));
> > > return 0;
> > > }
> > [snip]
>
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-07-20 20:09 ` Nicolas Dufresne
@ 2023-07-28 7:22 ` Tomasz Figa
2023-07-28 18:40 ` Randy Li
2023-08-04 20:19 ` Nicolas Dufresne
0 siblings, 2 replies; 21+ messages in thread
From: Tomasz Figa @ 2023-07-28 7:22 UTC (permalink / raw)
To: Nicolas Dufresne
Cc: Randy Li, linux-media, randy.li, Brian.Starkey, boris.brezillon,
frkoenig, hans.verkuil, hiroh, hverkuil, kernel,
laurent.pinchart, linux-kernel, mchehab, narmstrong,
sakari.ailus, stanimir.varbanov, Helen Koike
On Fri, Jul 21, 2023 at 5:10 AM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>
> Le mardi 18 juillet 2023 à 00:00 +0800, Randy Li a écrit :
> > On 2023/7/13 18:39, Tomasz Figa wrote:
> > > On Mon, Feb 06, 2023 at 12:33:01PM +0800, ayaka wrote:
> > > > From: Helen Koike <helen.koike@collabora.com>
> > > >
> > > > Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
> > > >
> > > > Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
> > > > framework to use the ext hooks when the classic Api is used from
> > > > userspace in Vid devices, and to keep touch devices with classic hook.
> > > >
> > > > Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> > > > Signed-off-by: Helen Koike <helen.koike@collabora.com>
> > > > ---
> > > > Changes in v7:
> > > > - Force the userspace using the new APIs to operate non-touch drivers.
> > > The primary objective of Linux development is not to break the
> > > userspace. We can't just remove the old API, especially not from
> > > existing drivers.
> > Maybe I should create a new virtual driver here? It is impossible to
> > support the new fourcc modifier with the old APIs.
>
> For MPLANE, where backward compatibility was built into libv4l2 LD_PRELOAD
> wrapper,
Could you refresh my memory on what kind of backwards compatibility we
had in libv4l2? Was that to make it possible to use new MPLANE-only
drivers with old applications?
> it simply failed the cases that could not be supported (non contiguous
> planes).
>
> regards,
> Nicolas
>
> > >
> > > [snip]
> > > > int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > > - struct v4l2_format *f)
> > > > + struct v4l2_ext_pix_format *f)
> > > > {
> > > > - struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
> > > > - struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
> > > > struct vivid_dev *dev = video_drvdata(file);
> > > > + struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
> > > > const struct vivid_fmt *fmt;
> > > > unsigned bytesperline, max_bpl;
> > > > unsigned factor = 1;
> > > > unsigned w, h;
> > > > unsigned p;
> > > > - bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
> > > Why is this condition being removed?
> >
> > Because the v4l2_ext_pix has a struct for the colorspace?
> >
> > Would you like the idea that driver exports a buffer contains all the
> > info for an enumeration ?
> >
> > >
> > > Best regards,
> > > Tomasz
> > >
> > > >
> > > > - fmt = vivid_get_format(dev, mp->pixelformat);
> > > > + fmt = vivid_get_format(dev, f->pixelformat);
> > > > if (!fmt) {
> > > > dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
> > > > - mp->pixelformat);
> > > > - mp->pixelformat = V4L2_PIX_FMT_YUYV;
> > > > - fmt = vivid_get_format(dev, mp->pixelformat);
> > > > + f->pixelformat);
> > > > + f->pixelformat = V4L2_PIX_FMT_YUYV;
> > > > + fmt = vivid_get_format(dev, f->pixelformat);
> > > > }
> > > >
> > > > - mp->field = vivid_field_cap(dev, mp->field);
> > > > + f->field = vivid_field_cap(dev, f->field);
> > > > if (vivid_is_webcam(dev)) {
> > > > const struct v4l2_frmsize_discrete *sz =
> > > > v4l2_find_nearest_size(webcam_sizes,
> > > > VIVID_WEBCAM_SIZES, width,
> > > > - height, mp->width, mp->height);
> > > > + height, f->width, f->height);
> > > >
> > > > w = sz->width;
> > > > h = sz->height;
> > > > @@ -604,14 +603,14 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > > w = dev->src_rect.width;
> > > > h = dev->src_rect.height;
> > > > }
> > > > - if (V4L2_FIELD_HAS_T_OR_B(mp->field))
> > > > + if (V4L2_FIELD_HAS_T_OR_B(f->field))
> > > > factor = 2;
> > > > if (vivid_is_webcam(dev) ||
> > > > (!dev->has_scaler_cap && !dev->has_crop_cap && !dev->has_compose_cap)) {
> > > > - mp->width = w;
> > > > - mp->height = h / factor;
> > > > + f->width = w;
> > > > + f->height = h / factor;
> > > > } else {
> > > > - struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
> > > > + struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
> > > >
> > > > v4l2_rect_set_min_size(&r, &vivid_min_rect);
> > > > v4l2_rect_set_max_size(&r, &vivid_max_rect);
> > > > @@ -624,16 +623,15 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > > } else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
> > > > v4l2_rect_set_min_size(&r, &dev->src_rect);
> > > > }
> > > > - mp->width = r.width;
> > > > - mp->height = r.height / factor;
> > > > + f->width = r.width;
> > > > + f->height = r.height / factor;
> > > > }
> > > >
> > > > /* This driver supports custom bytesperline values */
> > > >
> > > > - mp->num_planes = fmt->buffers;
> > > > for (p = 0; p < fmt->buffers; p++) {
> > > > /* Calculate the minimum supported bytesperline value */
> > > > - bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
> > > > + bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
> > > > /* Calculate the maximum supported bytesperline value */
> > > > max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
> > > >
> > > > @@ -642,48 +640,49 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> > > > if (pfmt[p].bytesperline < bytesperline)
> > > > pfmt[p].bytesperline = bytesperline;
> > > >
> > > > - pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
> > > > + pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
> > > > fmt->vdownsampling[p] + fmt->data_offset[p];
> > > > -
> > > > - memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
> > > > }
> > > > +
> > > > + if (p < VIDEO_MAX_PLANES)
> > > > + pfmt[p].sizeimage = 0;
> > > > +
> > > > for (p = fmt->buffers; p < fmt->planes; p++)
> > > > - pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
> > > > + pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
> > > > (fmt->bit_depth[p] / fmt->vdownsampling[p])) /
> > > > (fmt->bit_depth[0] / fmt->vdownsampling[0]);
> > > >
> > > > - if (!user_set_csc || !v4l2_is_colorspace_valid(mp->colorspace))
> > > > - mp->colorspace = vivid_colorspace_cap(dev);
> > > > + if (!v4l2_is_colorspace_valid(f->colorspace))
> > > > + f->colorspace = vivid_colorspace_cap(dev);
> > > >
> > > > - if (!user_set_csc || !v4l2_is_xfer_func_valid(mp->xfer_func))
> > > > - mp->xfer_func = vivid_xfer_func_cap(dev);
> > > > + if (!v4l2_is_xfer_func_valid(f->xfer_func))
> > > > + f->xfer_func = vivid_xfer_func_cap(dev);
> > > >
> > > > if (fmt->color_enc == TGP_COLOR_ENC_HSV) {
> > > > - if (!user_set_csc || !v4l2_is_hsv_enc_valid(mp->hsv_enc))
> > > > - mp->hsv_enc = vivid_hsv_enc_cap(dev);
> > > > + if (!v4l2_is_hsv_enc_valid(f->hsv_enc))
> > > > + f->hsv_enc = vivid_hsv_enc_cap(dev);
> > > > } else if (fmt->color_enc == TGP_COLOR_ENC_YCBCR) {
> > > > - if (!user_set_csc || !v4l2_is_ycbcr_enc_valid(mp->ycbcr_enc))
> > > > - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > > + if (!v4l2_is_ycbcr_enc_valid(f->ycbcr_enc))
> > > > + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > > } else {
> > > > - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > > + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> > > > }
> > > >
> > > > if (fmt->color_enc == TGP_COLOR_ENC_YCBCR ||
> > > > fmt->color_enc == TGP_COLOR_ENC_RGB) {
> > > > - if (!user_set_csc || !v4l2_is_quant_valid(mp->quantization))
> > > > - mp->quantization = vivid_quantization_cap(dev);
> > > > + if (!v4l2_is_quant_valid(f->quantization))
> > > > + f->quantization = vivid_quantization_cap(dev);
> > > > } else {
> > > > - mp->quantization = vivid_quantization_cap(dev);
> > > > + f->quantization = vivid_quantization_cap(dev);
> > > > }
> > > >
> > > > - memset(mp->reserved, 0, sizeof(mp->reserved));
> > > > + memset(f->reserved, 0, sizeof(f->reserved));
> > > > return 0;
> > > > }
> > > [snip]
> >
>
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-07-28 7:22 ` Tomasz Figa
@ 2023-07-28 18:40 ` Randy Li
2023-08-04 20:19 ` Nicolas Dufresne
1 sibling, 0 replies; 21+ messages in thread
From: Randy Li @ 2023-07-28 18:40 UTC (permalink / raw)
To: Tomasz Figa
Cc: linux-media, Nicolas Dufresne, randy.li, Brian.Starkey,
boris.brezillon, frkoenig, hans.verkuil, hiroh, hverkuil, kernel,
laurent.pinchart, linux-kernel, mchehab, narmstrong,
sakari.ailus, stanimir.varbanov, Helen Koike
On 2023/7/28 15:22, Tomasz Figa wrote:
> On Fri, Jul 21, 2023 at 5:10 AM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>> Le mardi 18 juillet 2023 à 00:00 +0800, Randy Li a écrit :
>>> On 2023/7/13 18:39, Tomasz Figa wrote:
>>>> On Mon, Feb 06, 2023 at 12:33:01PM +0800, ayaka wrote:
>>>>> From: Helen Koike <helen.koike@collabora.com>
>>>>>
>>>>> Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
>>>>>
>>>>> Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
>>>>> framework to use the ext hooks when the classic Api is used from
>>>>> userspace in Vid devices, and to keep touch devices with classic hook.
>>>>>
>>>>> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
>>>>> Signed-off-by: Helen Koike <helen.koike@collabora.com>
>>>>> ---
>>>>> Changes in v7:
>>>>> - Force the userspace using the new APIs to operate non-touch drivers.
>>>> The primary objective of Linux development is not to break the
>>>> userspace. We can't just remove the old API, especially not from
>>>> existing drivers.
>>> Maybe I should create a new virtual driver here? It is impossible to
>>> support the new fourcc modifier with the old APIs.
>> For MPLANE, where backward compatibility was built into libv4l2 LD_PRELOAD
>> wrapper,
> Could you refresh my memory on what kind of backwards compatibility we
> had in libv4l2? Was that to make it possible to use new MPLANE-only
> drivers with old applications?
lib/libv4l-mplane/libv4l-mplane.c
I think application don't need to know the new MPLANE variant of pixel
formats or queue type.
>> it simply failed the cases that could not be supported (non contiguous
>> planes).
>>
>> regards,
>> Nicolas
>>
>>>> [snip]
>>>>> int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>>>>> - struct v4l2_format *f)
>>>>> + struct v4l2_ext_pix_format *f)
>>>>> {
>>>>> - struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
>>>>> - struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
>>>>> struct vivid_dev *dev = video_drvdata(file);
>>>>> + struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
>>>>> const struct vivid_fmt *fmt;
>>>>> unsigned bytesperline, max_bpl;
>>>>> unsigned factor = 1;
>>>>> unsigned w, h;
>>>>> unsigned p;
>>>>> - bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
>>>> Why is this condition being removed?
>>> Because the v4l2_ext_pix has a struct for the colorspace?
>>>
>>> Would you like the idea that driver exports a buffer contains all the
>>> info for an enumeration ?
>>>
>>>> Best regards,
>>>> Tomasz
>>>>
>>>>> - fmt = vivid_get_format(dev, mp->pixelformat);
>>>>> + fmt = vivid_get_format(dev, f->pixelformat);
>>>>> if (!fmt) {
>>>>> dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
>>>>> - mp->pixelformat);
>>>>> - mp->pixelformat = V4L2_PIX_FMT_YUYV;
>>>>> - fmt = vivid_get_format(dev, mp->pixelformat);
>>>>> + f->pixelformat);
>>>>> + f->pixelformat = V4L2_PIX_FMT_YUYV;
>>>>> + fmt = vivid_get_format(dev, f->pixelformat);
>>>>> }
>>>>>
>>>>> - mp->field = vivid_field_cap(dev, mp->field);
>>>>> + f->field = vivid_field_cap(dev, f->field);
>>>>> if (vivid_is_webcam(dev)) {
>>>>> const struct v4l2_frmsize_discrete *sz =
>>>>> v4l2_find_nearest_size(webcam_sizes,
>>>>> VIVID_WEBCAM_SIZES, width,
>>>>> - height, mp->width, mp->height);
>>>>> + height, f->width, f->height);
>>>>>
>>>>> w = sz->width;
>>>>> h = sz->height;
>>>>> @@ -604,14 +603,14 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>>>>> w = dev->src_rect.width;
>>>>> h = dev->src_rect.height;
>>>>> }
>>>>> - if (V4L2_FIELD_HAS_T_OR_B(mp->field))
>>>>> + if (V4L2_FIELD_HAS_T_OR_B(f->field))
>>>>> factor = 2;
>>>>> if (vivid_is_webcam(dev) ||
>>>>> (!dev->has_scaler_cap && !dev->has_crop_cap && !dev->has_compose_cap)) {
>>>>> - mp->width = w;
>>>>> - mp->height = h / factor;
>>>>> + f->width = w;
>>>>> + f->height = h / factor;
>>>>> } else {
>>>>> - struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
>>>>> + struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
>>>>>
>>>>> v4l2_rect_set_min_size(&r, &vivid_min_rect);
>>>>> v4l2_rect_set_max_size(&r, &vivid_max_rect);
>>>>> @@ -624,16 +623,15 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>>>>> } else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
>>>>> v4l2_rect_set_min_size(&r, &dev->src_rect);
>>>>> }
>>>>> - mp->width = r.width;
>>>>> - mp->height = r.height / factor;
>>>>> + f->width = r.width;
>>>>> + f->height = r.height / factor;
>>>>> }
>>>>>
>>>>> /* This driver supports custom bytesperline values */
>>>>>
>>>>> - mp->num_planes = fmt->buffers;
>>>>> for (p = 0; p < fmt->buffers; p++) {
>>>>> /* Calculate the minimum supported bytesperline value */
>>>>> - bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
>>>>> + bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
>>>>> /* Calculate the maximum supported bytesperline value */
>>>>> max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
>>>>>
>>>>> @@ -642,48 +640,49 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
>>>>> if (pfmt[p].bytesperline < bytesperline)
>>>>> pfmt[p].bytesperline = bytesperline;
>>>>>
>>>>> - pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
>>>>> + pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
>>>>> fmt->vdownsampling[p] + fmt->data_offset[p];
>>>>> -
>>>>> - memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
>>>>> }
>>>>> +
>>>>> + if (p < VIDEO_MAX_PLANES)
>>>>> + pfmt[p].sizeimage = 0;
>>>>> +
>>>>> for (p = fmt->buffers; p < fmt->planes; p++)
>>>>> - pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
>>>>> + pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
>>>>> (fmt->bit_depth[p] / fmt->vdownsampling[p])) /
>>>>> (fmt->bit_depth[0] / fmt->vdownsampling[0]);
>>>>>
>>>>> - if (!user_set_csc || !v4l2_is_colorspace_valid(mp->colorspace))
>>>>> - mp->colorspace = vivid_colorspace_cap(dev);
>>>>> + if (!v4l2_is_colorspace_valid(f->colorspace))
>>>>> + f->colorspace = vivid_colorspace_cap(dev);
>>>>>
>>>>> - if (!user_set_csc || !v4l2_is_xfer_func_valid(mp->xfer_func))
>>>>> - mp->xfer_func = vivid_xfer_func_cap(dev);
>>>>> + if (!v4l2_is_xfer_func_valid(f->xfer_func))
>>>>> + f->xfer_func = vivid_xfer_func_cap(dev);
>>>>>
>>>>> if (fmt->color_enc == TGP_COLOR_ENC_HSV) {
>>>>> - if (!user_set_csc || !v4l2_is_hsv_enc_valid(mp->hsv_enc))
>>>>> - mp->hsv_enc = vivid_hsv_enc_cap(dev);
>>>>> + if (!v4l2_is_hsv_enc_valid(f->hsv_enc))
>>>>> + f->hsv_enc = vivid_hsv_enc_cap(dev);
>>>>> } else if (fmt->color_enc == TGP_COLOR_ENC_YCBCR) {
>>>>> - if (!user_set_csc || !v4l2_is_ycbcr_enc_valid(mp->ycbcr_enc))
>>>>> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>>>>> + if (!v4l2_is_ycbcr_enc_valid(f->ycbcr_enc))
>>>>> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>>>>> } else {
>>>>> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>>>>> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
>>>>> }
>>>>>
>>>>> if (fmt->color_enc == TGP_COLOR_ENC_YCBCR ||
>>>>> fmt->color_enc == TGP_COLOR_ENC_RGB) {
>>>>> - if (!user_set_csc || !v4l2_is_quant_valid(mp->quantization))
>>>>> - mp->quantization = vivid_quantization_cap(dev);
>>>>> + if (!v4l2_is_quant_valid(f->quantization))
>>>>> + f->quantization = vivid_quantization_cap(dev);
>>>>> } else {
>>>>> - mp->quantization = vivid_quantization_cap(dev);
>>>>> + f->quantization = vivid_quantization_cap(dev);
>>>>> }
>>>>>
>>>>> - memset(mp->reserved, 0, sizeof(mp->reserved));
>>>>> + memset(f->reserved, 0, sizeof(f->reserved));
>>>>> return 0;
>>>>> }
>>>> [snip]
^ permalink raw reply [flat|nested] 21+ messages in thread* Re: [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-07-28 7:22 ` Tomasz Figa
2023-07-28 18:40 ` Randy Li
@ 2023-08-04 20:19 ` Nicolas Dufresne
1 sibling, 0 replies; 21+ messages in thread
From: Nicolas Dufresne @ 2023-08-04 20:19 UTC (permalink / raw)
To: Tomasz Figa
Cc: Randy Li, linux-media, randy.li, Brian.Starkey, boris.brezillon,
frkoenig, hans.verkuil, hiroh, hverkuil, kernel,
laurent.pinchart, linux-kernel, mchehab, narmstrong,
sakari.ailus, stanimir.varbanov, Helen Koike
Le vendredi 28 juillet 2023 à 16:22 +0900, Tomasz Figa a écrit :
> > For MPLANE, where backward compatibility was built into libv4l2 LD_PRELOAD
> > wrapper,
>
> Could you refresh my memory on what kind of backwards compatibility we
> had in libv4l2? Was that to make it possible to use new MPLANE-only
> drivers with old applications?
https://git.linuxtv.org/v4l-utils.git/tree/lib/libv4l-mplane
It does some "magic" trick due to some of the new fields not guarantied to be
zero, translate the type, the capabilities. Strangely it does not filter the
pixel formats. I suspect it assumes legacy application won't know about the new
one and will just ignore/not use them.
Nicolas
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format
2023-07-17 16:00 ` Randy Li
2023-07-20 20:09 ` Nicolas Dufresne
@ 2023-07-28 7:18 ` Tomasz Figa
1 sibling, 0 replies; 21+ messages in thread
From: Tomasz Figa @ 2023-07-28 7:18 UTC (permalink / raw)
To: Randy Li
Cc: linux-media, randy.li, Brian.Starkey, boris.brezillon, frkoenig,
hans.verkuil, hiroh, hverkuil, kernel, laurent.pinchart,
linux-kernel, mchehab, narmstrong, nicolas, sakari.ailus,
stanimir.varbanov, Helen Koike
On Tue, Jul 18, 2023 at 1:00 AM Randy Li <ayaka@soulik.info> wrote:
>
>
> On 2023/7/13 18:39, Tomasz Figa wrote:
> > On Mon, Feb 06, 2023 at 12:33:01PM +0800, ayaka wrote:
> >> From: Helen Koike <helen.koike@collabora.com>
> >>
> >> Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
> >>
> >> Duplicate v4l2_ioctl_ops for touch devices. This is done to force the
> >> framework to use the ext hooks when the classic Api is used from
> >> userspace in Vid devices, and to keep touch devices with classic hook.
> >>
> >> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
> >> Signed-off-by: Helen Koike <helen.koike@collabora.com>
> >> ---
> >> Changes in v7:
> >> - Force the userspace using the new APIs to operate non-touch drivers.
> > The primary objective of Linux development is not to break the
> > userspace. We can't just remove the old API, especially not from
> > existing drivers.
> Maybe I should create a new virtual driver here? It is impossible to
> support the new fourcc modifier with the old APIs.
We need to find a way to make an existing driver support both the old
and new API. Obviously any new functionality of the new API doesn't
have to be retrofitted to the old API.
> >
> > [snip]
> >> int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> >> - struct v4l2_format *f)
> >> + struct v4l2_ext_pix_format *f)
> >> {
> >> - struct v4l2_pix_format_mplane *mp = &f->fmt.pix_mp;
> >> - struct v4l2_plane_pix_format *pfmt = mp->plane_fmt;
> >> struct vivid_dev *dev = video_drvdata(file);
> >> + struct v4l2_plane_pix_format *pfmt = f->plane_fmt;
> >> const struct vivid_fmt *fmt;
> >> unsigned bytesperline, max_bpl;
> >> unsigned factor = 1;
> >> unsigned w, h;
> >> unsigned p;
> >> - bool user_set_csc = !!(mp->flags & V4L2_PIX_FMT_FLAG_SET_CSC);
> > Why is this condition being removed?
>
> Because the v4l2_ext_pix has a struct for the colorspace?
What do you mean? I see it has the same enum field for colorspace as
the original v4l2_pix_format_mplane.
The flag was needed for CAPTURE format to tell the driver whether it
should perform a conversion to the requested colorspace or just fill
in the color space as inferred from the current configuration (e.g.
OUTPUT format).
How was that addressed in the new API?
>
> Would you like the idea that driver exports a buffer contains all the
> info for an enumeration ?
>
> >
> > Best regards,
> > Tomasz
> >
> >>
> >> - fmt = vivid_get_format(dev, mp->pixelformat);
> >> + fmt = vivid_get_format(dev, f->pixelformat);
> >> if (!fmt) {
> >> dprintk(dev, 1, "Fourcc format (0x%08x) unknown.\n",
> >> - mp->pixelformat);
> >> - mp->pixelformat = V4L2_PIX_FMT_YUYV;
> >> - fmt = vivid_get_format(dev, mp->pixelformat);
> >> + f->pixelformat);
> >> + f->pixelformat = V4L2_PIX_FMT_YUYV;
> >> + fmt = vivid_get_format(dev, f->pixelformat);
> >> }
> >>
> >> - mp->field = vivid_field_cap(dev, mp->field);
> >> + f->field = vivid_field_cap(dev, f->field);
> >> if (vivid_is_webcam(dev)) {
> >> const struct v4l2_frmsize_discrete *sz =
> >> v4l2_find_nearest_size(webcam_sizes,
> >> VIVID_WEBCAM_SIZES, width,
> >> - height, mp->width, mp->height);
> >> + height, f->width, f->height);
> >>
> >> w = sz->width;
> >> h = sz->height;
> >> @@ -604,14 +603,14 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> >> w = dev->src_rect.width;
> >> h = dev->src_rect.height;
> >> }
> >> - if (V4L2_FIELD_HAS_T_OR_B(mp->field))
> >> + if (V4L2_FIELD_HAS_T_OR_B(f->field))
> >> factor = 2;
> >> if (vivid_is_webcam(dev) ||
> >> (!dev->has_scaler_cap && !dev->has_crop_cap && !dev->has_compose_cap)) {
> >> - mp->width = w;
> >> - mp->height = h / factor;
> >> + f->width = w;
> >> + f->height = h / factor;
> >> } else {
> >> - struct v4l2_rect r = { 0, 0, mp->width, mp->height * factor };
> >> + struct v4l2_rect r = { 0, 0, f->width, f->height * factor };
> >>
> >> v4l2_rect_set_min_size(&r, &vivid_min_rect);
> >> v4l2_rect_set_max_size(&r, &vivid_max_rect);
> >> @@ -624,16 +623,15 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> >> } else if (!dev->has_scaler_cap && !dev->has_crop_cap) {
> >> v4l2_rect_set_min_size(&r, &dev->src_rect);
> >> }
> >> - mp->width = r.width;
> >> - mp->height = r.height / factor;
> >> + f->width = r.width;
> >> + f->height = r.height / factor;
> >> }
> >>
> >> /* This driver supports custom bytesperline values */
> >>
> >> - mp->num_planes = fmt->buffers;
> >> for (p = 0; p < fmt->buffers; p++) {
> >> /* Calculate the minimum supported bytesperline value */
> >> - bytesperline = (mp->width * fmt->bit_depth[p]) >> 3;
> >> + bytesperline = (f->width * fmt->bit_depth[p]) >> 3;
> >> /* Calculate the maximum supported bytesperline value */
> >> max_bpl = (MAX_ZOOM * MAX_WIDTH * fmt->bit_depth[p]) >> 3;
> >>
> >> @@ -642,48 +640,49 @@ int vivid_try_fmt_vid_cap(struct file *file, void *priv,
> >> if (pfmt[p].bytesperline < bytesperline)
> >> pfmt[p].bytesperline = bytesperline;
> >>
> >> - pfmt[p].sizeimage = (pfmt[p].bytesperline * mp->height) /
> >> + pfmt[p].sizeimage = (pfmt[p].bytesperline * f->height) /
> >> fmt->vdownsampling[p] + fmt->data_offset[p];
> >> -
> >> - memset(pfmt[p].reserved, 0, sizeof(pfmt[p].reserved));
> >> }
> >> +
> >> + if (p < VIDEO_MAX_PLANES)
> >> + pfmt[p].sizeimage = 0;
> >> +
> >> for (p = fmt->buffers; p < fmt->planes; p++)
> >> - pfmt[0].sizeimage += (pfmt[0].bytesperline * mp->height *
> >> + pfmt[0].sizeimage += (pfmt[0].bytesperline * f->height *
> >> (fmt->bit_depth[p] / fmt->vdownsampling[p])) /
> >> (fmt->bit_depth[0] / fmt->vdownsampling[0]);
> >>
> >> - if (!user_set_csc || !v4l2_is_colorspace_valid(mp->colorspace))
> >> - mp->colorspace = vivid_colorspace_cap(dev);
> >> + if (!v4l2_is_colorspace_valid(f->colorspace))
> >> + f->colorspace = vivid_colorspace_cap(dev);
> >>
> >> - if (!user_set_csc || !v4l2_is_xfer_func_valid(mp->xfer_func))
> >> - mp->xfer_func = vivid_xfer_func_cap(dev);
> >> + if (!v4l2_is_xfer_func_valid(f->xfer_func))
> >> + f->xfer_func = vivid_xfer_func_cap(dev);
> >>
> >> if (fmt->color_enc == TGP_COLOR_ENC_HSV) {
> >> - if (!user_set_csc || !v4l2_is_hsv_enc_valid(mp->hsv_enc))
> >> - mp->hsv_enc = vivid_hsv_enc_cap(dev);
> >> + if (!v4l2_is_hsv_enc_valid(f->hsv_enc))
> >> + f->hsv_enc = vivid_hsv_enc_cap(dev);
> >> } else if (fmt->color_enc == TGP_COLOR_ENC_YCBCR) {
> >> - if (!user_set_csc || !v4l2_is_ycbcr_enc_valid(mp->ycbcr_enc))
> >> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> >> + if (!v4l2_is_ycbcr_enc_valid(f->ycbcr_enc))
> >> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> >> } else {
> >> - mp->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> >> + f->ycbcr_enc = vivid_ycbcr_enc_cap(dev);
> >> }
> >>
> >> if (fmt->color_enc == TGP_COLOR_ENC_YCBCR ||
> >> fmt->color_enc == TGP_COLOR_ENC_RGB) {
> >> - if (!user_set_csc || !v4l2_is_quant_valid(mp->quantization))
> >> - mp->quantization = vivid_quantization_cap(dev);
> >> + if (!v4l2_is_quant_valid(f->quantization))
> >> + f->quantization = vivid_quantization_cap(dev);
> >> } else {
> >> - mp->quantization = vivid_quantization_cap(dev);
> >> + f->quantization = vivid_quantization_cap(dev);
> >> }
> >>
> >> - memset(mp->reserved, 0, sizeof(mp->reserved));
> >> + memset(f->reserved, 0, sizeof(f->reserved));
> >> return 0;
> >> }
> > [snip]
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v7 3/9] media: v4l2: Add extended buffer (de)queue operations for video types
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
2023-02-06 4:33 ` [PATCH v7 1/9] media: v4l2: Extend pixel formats to unify single/multi-planar handling (and more) ayaka
2023-02-06 4:33 ` [PATCH v7 2/9] media: vivid: Convert to v4l2_ext_pix_format ayaka
@ 2023-02-06 4:33 ` ayaka
2023-02-06 4:33 ` [PATCH v7 4/9] media: videobuf2-v4l2: reorganize flags handling ayaka
` (5 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike
From: Helen Koike <helen.koike@collabora.com>
Those extended buffer ops have several purpose:
1/ Fix y2038 issues by converting the timestamp into an u64 counting
the number of ns elapsed since 1970
2/ Unify single/multiplanar handling
3/ Add a new start offset field to each v4l2 plane buffer info struct
to support the case where a single buffer object is storing all
planes data, each one being placed at a different offset
New hooks are created in v4l2_ioctl_ops so that drivers can start using
these new objects.
Note that the timecode field is gone, since there doesn't seem to be
in-kernel users. We can be added back in the reserved area if needed or
use the Request API to collect more metadata information from the
frame.
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v7:
Just refresh this.
Changes in v6:
This patch was completely refactored, and based on previous version from
Hans and Boris.
- Refactor conversions v4l2_buffer <-> v4l2_ext_buffer for (d)qbuf
- I removed EXT_CREATE_BUFS since it is basically only usefull to MMAP.
If this is going towards DMA-fd centric, then we can use the current
REQUESTBUF to switch to it, and we can think a better way to support
MMAP later if there are usecases.
I also moved memory field from v4l2_ext_plane to v4l2_ext_buffer,
since it is very unlikely to mix memory types, and REQUESTBUF can
switch the whole buffer object to a given type.
- I removed EXT_QUERYBUF, since it is only useful to MMAP, for the
same reason above.
- I removed EXT_PREPARE_BUF, since it is basically just an optimization,
we can add it later (my intention is to simplify this patchset).
- These ioctls are only valid for video types (and not for overlay,
vbi, touch, meta, etc).
- Refactor struct v4l2_ext_buffer and struct v4l2_ext_planes as
discussed with Tomasz:
- add bytesused back
- remove lenght field
- move memory field from planes to buffer object
- Fix order in documentation of struct v4l2_ext_buffer (Tomasz)
- Fix flags documentation of struct v4l2_ext_buffer, don't say when flags are ignored (Tomasz)
- v4l_print_ext_buffer(): print request_fd and offset/userptr (Tomasz)
Changes in v5:
- migrate memory from v4l2_ext_buffer to v4l2_ext_plane
- return mem_offset to struct v4l2_ext_plane
- change sizes and reorder fields to avoid holes in the struct and make
it the same for 32 and 64 bits
Changes in v4:
- Use v4l2_ext_pix_format directly in the ioctl, drop v4l2_ext_format,
making V4L2_BUF_TYPE_VIDEO_[OUTPUT,CAPTURE] the only valid types.
- Drop VIDIOC_EXT_EXPBUF, since the only difference from VIDIOC_EXPBUF
was that with VIDIOC_EXT_EXPBUF we could export multiple planes at once.
I think we can add this later, so I removed it from this RFC to simplify it.
- Remove num_planes field from struct v4l2_ext_buffer
- Add flags field to struct v4l2_ext_create_buffers
- Reformulate struct v4l2_ext_plane
- Fix some bugs caught by v4l2-compliance
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- Add reserved space to v4l2_ext_buffer so that new fields can be added
later on
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
drivers/media/v4l2-core/v4l2-dev.c | 4 +
drivers/media/v4l2-core/v4l2-ioctl.c | 184 +++++++++++++++++++++++++++
include/media/v4l2-ioctl.h | 8 ++
include/uapi/linux/videodev2.h | 55 ++++++++
4 files changed, 251 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c
index cfe90bfd47f1..f7a8e0146659 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -662,6 +662,10 @@ static void determine_valid_ioctls(struct video_device *vdev)
__set_bit(_IOC_NR(VIDIOC_S_CROP), valid_ioctls);
SET_VALID_IOCTL(ops, VIDIOC_G_SELECTION, vidioc_g_selection);
SET_VALID_IOCTL(ops, VIDIOC_S_SELECTION, vidioc_s_selection);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_QBUF, vidioc_ext_qbuf);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_QBUF, vidioc_qbuf);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_DQBUF, vidioc_ext_dqbuf);
+ SET_VALID_IOCTL(ops, VIDIOC_EXT_DQBUF, vidioc_dqbuf);
}
if (is_meta && is_rx) {
/* metadata capture specific ioctls */
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 52b77a968bb3..d6e860cb15bf 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -502,6 +502,24 @@ static void v4l_print_buffer(const void *arg, bool write_only)
tc->type, tc->flags, tc->frames, *(__u32 *)tc->userbits);
}
+static void v4l_print_ext_buffer(const void *arg, bool write_only)
+{
+ const struct v4l2_ext_buffer *e = arg;
+ unsigned int i;
+
+ pr_cont("%lld index=%d, type=%s, request_fd=%d, flags=0x%08llx, field=%s, sequence=%d, memory=%s\n",
+ e->timestamp, e->index, prt_names(e->type, v4l2_type_names),
+ e->request_fd, e->flags, prt_names(e->field, v4l2_field_names),
+ e->sequence, prt_names(e->memory, v4l2_memory_names));
+
+ for (i = 0; i < VIDEO_MAX_PLANES && e->planes[i].m.userptr; i++) {
+ const struct v4l2_ext_plane *plane = &e->planes[i];
+
+ pr_cont("plane %d: bytesused=%d, offset=0x%08x, userptr=0x%llx\n",
+ i, plane->bytesused, plane->offset, plane->m.userptr);
+ }
+}
+
static void v4l_print_exportbuffer(const void *arg, bool write_only)
{
const struct v4l2_exportbuffer *p = arg;
@@ -2456,6 +2474,130 @@ static int v4l_qbuf(const struct v4l2_ioctl_ops *ops,
return ret ? ret : ops->vidioc_qbuf(file, fh, p);
}
+static bool v4l2_ext_buffer_is_single_membuf(const struct v4l2_ext_buffer *eb)
+{
+ unsigned int i;
+
+ for (i = 1; i < VIDEO_MAX_PLANES && eb->planes[i].m.userptr; i++)
+ if (eb->planes[i].m.userptr != eb->planes[i - 1].m.userptr)
+ return false;
+ return true;
+}
+
+static int v4l2_fill_buffer_from_ext_buffer(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh,
+ const struct v4l2_ext_buffer *eb,
+ struct v4l2_buffer *b,
+ struct v4l2_plane *bplanes)
+{
+ const struct v4l2_ext_plane *eplanes = (struct v4l2_ext_plane *)&eb->planes;
+ struct video_device *vfd = video_devdata(file);
+ bool is_mplane = V4L2_IS_CAP_MULTIPLANAR(vfd);
+ unsigned int i;
+ u64 nsecs;
+ int ret;
+
+ b->index = eb->index;
+ if (is_mplane) {
+ b->m.planes = bplanes;
+ b->length = VIDEO_MAX_PLANES;
+ if (eb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ b->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ else if (eb->type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ b->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+ } else {
+ b->type = eb->type;
+ }
+
+ /* Fill the rest of the v4l2_buffer */
+ ret = v4l_querybuf(ops, file, fh, b);
+ if (ret)
+ return ret;
+
+ /* Fill other fields from v4l2_ext_buffer */
+ b->flags = eb->flags;
+ b->field = eb->field;
+ b->timestamp.tv_sec = div64_u64_rem(eb->timestamp, NSEC_PER_SEC, &nsecs);
+ b->timestamp.tv_usec = (u32)nsecs / NSEC_PER_USEC;
+ b->sequence = eb->sequence;
+
+ if (!is_mplane) {
+ for (i = 0; i < VIDEO_MAX_PLANES; i++) {
+ b->bytesused += eplanes[i].bytesused;
+ WARN_ON(eplanes[i].offset);
+ }
+
+ /* MMAP info was filled by querybuf */
+ if (b->memory == V4L2_MEMORY_MMAP)
+ return 0 ;
+
+ /*
+ * TODO: get the length of the buffer, for now, just
+ * set to max to avoid errors in checks.
+ */
+ b->length = U32_MAX;
+ b->m.userptr = eplanes[0].m.userptr;
+ return 0;
+ }
+
+ bplanes[0].bytesused = eplanes[0].bytesused + eplanes[0].offset;
+ bplanes[0].data_offset = eplanes[0].offset;
+ if (v4l2_ext_buffer_is_single_membuf(eb))
+ for (i = 1; i < VIDEO_MAX_PLANES && eplanes[i].bytesused; i++) {
+ bplanes[0].bytesused += eplanes[i].bytesused;
+ WARN_ON(eplanes[i].offset);
+ }
+ else
+ for (i = 1; i < VIDEO_MAX_PLANES && eplanes[i].bytesused; i++) {
+ bplanes[i].bytesused = eplanes[i].bytesused +
+ eplanes[i].offset;
+ bplanes[i].data_offset = eplanes[i].offset;
+ }
+
+ /* MMAP info was filled by querybuf */
+ if (b->memory == V4L2_MEMORY_MMAP)
+ return 0;
+
+ for (i = 0; i < VIDEO_MAX_PLANES && eplanes[i].m.userptr; i++) {
+ bplanes[i].m.userptr = eplanes[i].m.userptr;
+ /*
+ * TODO: get the length of the buffer, for now, just
+ * set to max to avoid errors in checks.
+ */
+ bplanes[i].length = U32_MAX;
+ }
+ return 0;
+}
+
+static int v4l_ext_qbuf(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct v4l2_plane planes[VIDEO_MAX_PLANES] = {0};
+ struct v4l2_ext_buffer *eb = arg;
+ struct v4l2_buffer b = {0};
+
+ int ret = check_fmt(file, eb->type);
+
+ if (!ret)
+ return ret;
+
+ if (ops->vidioc_ext_qbuf)
+ return ops->vidioc_ext_qbuf(file, fh, eb);
+
+ /* Fill other fields from v4l2_ext_buffer */
+ ret = v4l2_fill_buffer_from_ext_buffer(ops, file, fh, eb, &b, planes);
+ if (ret)
+ return ret;
+
+ ret = v4l_qbuf(ops, file, fh, &b);
+ if (ret)
+ return ret;
+
+ /* TODO: check if we need to fill other fields */
+ eb->flags = b.flags;
+ return 0;
+}
+
static int v4l_dqbuf(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -2465,6 +2607,46 @@ static int v4l_dqbuf(const struct v4l2_ioctl_ops *ops,
return ret ? ret : ops->vidioc_dqbuf(file, fh, p);
}
+static int v4l_ext_dqbuf(const struct v4l2_ioctl_ops *ops,
+ struct file *file, void *fh, void *arg)
+{
+ struct v4l2_plane planes[VIDEO_MAX_PLANES] = {0};
+ struct v4l2_ext_buffer *eb = arg;
+ struct v4l2_buffer b = {0};
+ unsigned int i;
+
+ int ret = check_fmt(file, eb->type);
+
+ if (!ret)
+ return ret;
+
+ if (ops->vidioc_ext_qbuf)
+ return ops->vidioc_ext_qbuf(file, fh, eb);
+
+ /* Fill other fields from v4l2_ext_buffer */
+ ret = v4l2_fill_buffer_from_ext_buffer(ops, file, fh, eb, &b, planes);
+ if (ret)
+ return ret;
+
+ ret = v4l_qbuf(ops, file, fh, &b);
+ if (ret)
+ return ret;
+
+ /* TODO: check if we need to fill other fields */
+ eb->flags = b.flags;
+
+ /*
+ * Set buffer pointers to zero. Usecase: DMA-fd might have being
+ * alread closed, so just request userspace to fill it again in queue
+ * time.
+ */
+ for (i = 0; i < VIDEO_MAX_PLANES && eb->planes[i].m.userptr; i++) {
+ eb->planes[i].m.userptr = 0;
+ }
+
+ return 0;
+}
+
static int v4l_create_bufs(const struct v4l2_ioctl_ops *ops,
struct file *file, void *fh, void *arg)
{
@@ -3249,6 +3431,8 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = {
IOCTL_INFO(VIDIOC_G_EXT_PIX_FMT, v4l_g_ext_pix_fmt, v4l_print_ext_pix_format, 0),
IOCTL_INFO(VIDIOC_S_EXT_PIX_FMT, v4l_s_ext_pix_fmt, v4l_print_ext_pix_format, INFO_FL_PRIO),
IOCTL_INFO(VIDIOC_TRY_EXT_PIX_FMT, v4l_try_ext_pix_fmt, v4l_print_ext_pix_format, 0),
+ IOCTL_INFO(VIDIOC_EXT_QBUF, v4l_ext_qbuf, v4l_print_ext_buffer, INFO_FL_QUEUE),
+ IOCTL_INFO(VIDIOC_EXT_DQBUF, v4l_ext_dqbuf, v4l_print_ext_buffer, INFO_FL_QUEUE),
};
#define V4L2_IOCTLS ARRAY_SIZE(v4l2_ioctls)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index c44708dc9355..1d0ed36e5e67 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -171,10 +171,14 @@ struct v4l2_fh;
* :ref:`VIDIOC_QUERYBUF <vidioc_querybuf>` ioctl
* @vidioc_qbuf: pointer to the function that implements
* :ref:`VIDIOC_QBUF <vidioc_qbuf>` ioctl
+ * @vidioc_ext_qbuf: pointer to the function that implements
+ * :ref:`VIDIOC_EXT_QBUF <vidioc_ext_qbuf>` ioctl
* @vidioc_expbuf: pointer to the function that implements
* :ref:`VIDIOC_EXPBUF <vidioc_expbuf>` ioctl
* @vidioc_dqbuf: pointer to the function that implements
* :ref:`VIDIOC_DQBUF <vidioc_qbuf>` ioctl
+ * @vidioc_ext_dqbuf: pointer to the function that implements
+ * :ref:`VIDIOC_EXT_DQBUF <vidioc_ext_qbuf>` ioctl
* @vidioc_create_bufs: pointer to the function that implements
* :ref:`VIDIOC_CREATE_BUFS <vidioc_create_bufs>` ioctl
* @vidioc_prepare_buf: pointer to the function that implements
@@ -441,10 +445,14 @@ struct v4l2_ioctl_ops {
struct v4l2_buffer *b);
int (*vidioc_qbuf)(struct file *file, void *fh,
struct v4l2_buffer *b);
+ int (*vidioc_ext_qbuf)(struct file *file, void *fh,
+ struct v4l2_ext_buffer *b);
int (*vidioc_expbuf)(struct file *file, void *fh,
struct v4l2_exportbuffer *e);
int (*vidioc_dqbuf)(struct file *file, void *fh,
struct v4l2_buffer *b);
+ int (*vidioc_ext_dqbuf)(struct file *file, void *fh,
+ struct v4l2_ext_buffer *b);
int (*vidioc_create_bufs)(struct file *file, void *fh,
struct v4l2_create_buffers *b);
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 74a8dd7f7637..690bd24b17a6 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1120,6 +1120,59 @@ struct v4l2_buffer {
};
};
+/**
+ * struct v4l2_ext_plane - extended plane buffer info
+ * @offset: offset in the memory buffer where the plane starts.
+ * @bytesused: number of bytes occupied by data in the plane (payload).
+ * @mmap_offset: If V4L2_MEMORY_MMAP is used, then it can be a "cookie"
+ * that should be passed to mmap() called on the video node.
+ * @userptr: when memory is V4L2_MEMORY_USERPTR, a userspace pointer pointing
+ * to this plane.
+ * @dmabuf_fd: when memory is V4L2_MEMORY_DMABUF, a userspace file descriptor
+ * associated with this plane.
+ * @reserved: extra space reserved for future fields, must be set to 0.
+ */
+struct v4l2_ext_plane {
+ __u32 offset;
+ __u32 bytesused;
+ union {
+ __u32 mmap_offset;
+ __u64 userptr;
+ __s32 dmabuf_fd;
+ } m;
+ __u32 reserved[6];
+};
+
+/**
+ * struct v4l2_ext_buffer - extended video buffer info
+ * @index: id number of the buffer
+ * @type: V4L2_BUF_TYPE_VIDEO_CAPTURE or V4L2_BUF_TYPE_VIDEO_OUTPUT
+ * @field: enum v4l2_field; field order of the image in the buffer
+ * @sequence: sequence count of this frame
+ * @flags: buffer informational flags
+ * @timestamp: frame timestamp
+ * @memory: enum v4l2_memory; the method, in which the actual video
+ * data is passed
+ * @request_fd: fd of the request that this buffer should use
+ * @planes: per-plane buffer information
+ * @reserved: extra space reserved for future fields, must be set to 0
+ *
+ * Contains data exchanged by application and driver using one of the Streaming
+ * I/O methods.
+ */
+struct v4l2_ext_buffer {
+ __u32 index;
+ __u32 type;
+ __u32 field;
+ __u32 sequence;
+ __u64 flags;
+ __u64 timestamp;
+ __u32 memory;
+ __s32 request_fd;
+ struct v4l2_ext_plane planes[VIDEO_MAX_PLANES];
+ __u32 reserved[10];
+};
+
#ifndef __KERNEL__
/**
* v4l2_timeval_to_ns - Convert timeval to nanoseconds
@@ -2733,6 +2786,8 @@ struct v4l2_create_buffers {
#define VIDIOC_G_EXT_PIX_FMT _IOWR('V', 104, struct v4l2_ext_pix_format)
#define VIDIOC_S_EXT_PIX_FMT _IOWR('V', 105, struct v4l2_ext_pix_format)
#define VIDIOC_TRY_EXT_PIX_FMT _IOWR('V', 106, struct v4l2_ext_pix_format)
+#define VIDIOC_EXT_QBUF _IOWR('V', 107, struct v4l2_ext_buffer)
+#define VIDIOC_EXT_DQBUF _IOWR('V', 118, struct v4l2_ext_buffer)
/* Reminder: when adding new ioctls please add support for them to
drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v7 4/9] media: videobuf2-v4l2: reorganize flags handling
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
` (2 preceding siblings ...)
2023-02-06 4:33 ` [PATCH v7 3/9] media: v4l2: Add extended buffer (de)queue operations for video types ayaka
@ 2023-02-06 4:33 ` ayaka
2023-02-06 4:33 ` [PATCH v7 5/9] media: videobuf2: Expose helpers for Ext qbuf/dqbuf ayaka
` (4 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike
From: Helen Koike <helen.koike@collabora.com>
Reorganize flags handling to be easily reuseble when Ext functions get
added.
No logic is changed, just moving around code.
- Two new functions:
v4l2_clear_buffer_flags()
vb2_fill_vb2_v4l2_buffer_flags()
- set_buffer_cache_hints() receives a pointer to flags instead of the
v4l2_buffer object, making it undependent of this struct.
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v7:
- Refresh and rebase
Changes in v6:
- New patch
---
.../media/common/videobuf2/videobuf2-v4l2.c | 176 ++++++++++--------
1 file changed, 97 insertions(+), 79 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index 1f5d235a8441..eb158de7d14a 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -173,6 +173,43 @@ static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
pr_warn("use the actual size instead.\n");
}
+static void
+vb2_fill_vb2_v4l2_buffer_flags(struct vb2_buffer *vb,
+ u32 type, u32 field, u32 flags)
+{
+ struct vb2_queue *q = vb->vb2_queue;
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+
+ /* Zero flags that we handle */
+ vbuf->flags = flags & ~V4L2_BUFFER_MASK_FLAGS;
+ if (!vb->vb2_queue->copy_timestamp || V4L2_TYPE_IS_CAPTURE(type)) {
+ /*
+ * Non-COPY timestamps and non-OUTPUT queues will get
+ * their timestamp and timestamp source flags from the
+ * queue.
+ */
+ vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
+ }
+
+ if (V4L2_TYPE_IS_OUTPUT(type)) {
+ /*
+ * For output buffers mask out the timecode flag:
+ * this will be handled later in vb2_qbuf().
+ * The 'field' is valid metadata for this output buffer
+ * and so that needs to be copied here.
+ */
+ vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
+ vbuf->field = field;
+ if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF))
+ vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF;
+ } else {
+ /* Zero any output buffer flags as this is a capture buffer */
+ vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
+ /* Zero last flag, this is a signal from driver to userspace */
+ vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
+ }
+}
+
static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b)
{
struct vb2_queue *q = vb->vb2_queue;
@@ -309,41 +346,14 @@ static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b
}
- /* Zero flags that we handle */
- vbuf->flags = b->flags & ~V4L2_BUFFER_MASK_FLAGS;
- if (!vb->vb2_queue->copy_timestamp || V4L2_TYPE_IS_CAPTURE(b->type)) {
- /*
- * Non-COPY timestamps and non-OUTPUT queues will get
- * their timestamp and timestamp source flags from the
- * queue.
- */
- vbuf->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
- }
-
- if (V4L2_TYPE_IS_OUTPUT(b->type)) {
- /*
- * For output buffers mask out the timecode flag:
- * this will be handled later in vb2_qbuf().
- * The 'field' is valid metadata for this output buffer
- * and so that needs to be copied here.
- */
- vbuf->flags &= ~V4L2_BUF_FLAG_TIMECODE;
- vbuf->field = b->field;
- if (!(q->subsystem_flags & VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF))
- vbuf->flags &= ~V4L2_BUF_FLAG_M2M_HOLD_CAPTURE_BUF;
- } else {
- /* Zero any output buffer flags as this is a capture buffer */
- vbuf->flags &= ~V4L2_BUFFER_OUT_FLAGS;
- /* Zero last flag, this is a signal from driver to userspace */
- vbuf->flags &= ~V4L2_BUF_FLAG_LAST;
- }
+ vb2_fill_vb2_v4l2_buffer_flags(vb, b->type, b->field, b->flags);
return 0;
}
static void set_buffer_cache_hints(struct vb2_queue *q,
struct vb2_buffer *vb,
- struct v4l2_buffer *b)
+ u32 *flags)
{
if (!vb2_queue_allows_cache_hints(q)) {
/*
@@ -351,15 +361,15 @@ static void set_buffer_cache_hints(struct vb2_queue *q,
* space hints. That's to indicate to userspace that these
* flags won't work.
*/
- b->flags &= ~V4L2_BUF_FLAG_NO_CACHE_INVALIDATE;
- b->flags &= ~V4L2_BUF_FLAG_NO_CACHE_CLEAN;
+ *flags &= ~V4L2_BUF_FLAG_NO_CACHE_INVALIDATE;
+ *flags &= ~V4L2_BUF_FLAG_NO_CACHE_CLEAN;
return;
}
- if (b->flags & V4L2_BUF_FLAG_NO_CACHE_INVALIDATE)
+ if (*flags & V4L2_BUF_FLAG_NO_CACHE_INVALIDATE)
vb->skip_cache_sync_on_finish = 1;
- if (b->flags & V4L2_BUF_FLAG_NO_CACHE_CLEAN)
+ if (*flags & V4L2_BUF_FLAG_NO_CACHE_CLEAN)
vb->skip_cache_sync_on_prepare = 1;
}
@@ -407,7 +417,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
}
if (!vb->prepared) {
- set_buffer_cache_hints(q, vb, b);
+ set_buffer_cache_hints(q, vb, &b->flags);
/* Copy relevant information provided by the userspace */
memset(vbuf->planes, 0,
sizeof(vbuf->planes[0]) * vb->num_planes);
@@ -485,6 +495,58 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
return 0;
}
+/*
+ * Clear any buffer state related flags.
+ */
+static void v4l2_clear_buffer_flags(struct vb2_buffer *vb, u32 *flags,
+ struct vb2_queue *q,
+ struct vb2_v4l2_buffer *vbuf,
+ u32* request_fd)
+{
+ *flags &= ~V4L2_BUFFER_MASK_FLAGS;
+ *flags |= q->timestamp_flags & V4L2_BUF_FLAG_TIMESTAMP_MASK;
+ if (!q->copy_timestamp) {
+ /*
+ * For non-COPY timestamps, drop timestamp source bits
+ * and obtain the timestamp source from the queue.
+ */
+ *flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
+ *flags |= q->timestamp_flags & V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
+ }
+
+ switch (vb->state) {
+ case VB2_BUF_STATE_QUEUED:
+ case VB2_BUF_STATE_ACTIVE:
+ *flags |= V4L2_BUF_FLAG_QUEUED;
+ break;
+ case VB2_BUF_STATE_IN_REQUEST:
+ *flags |= V4L2_BUF_FLAG_IN_REQUEST;
+ break;
+ case VB2_BUF_STATE_ERROR:
+ *flags |= V4L2_BUF_FLAG_ERROR;
+ fallthrough;
+ case VB2_BUF_STATE_DONE:
+ *flags |= V4L2_BUF_FLAG_DONE;
+ break;
+ case VB2_BUF_STATE_PREPARING:
+ case VB2_BUF_STATE_DEQUEUED:
+ /* nothing */
+ break;
+ }
+
+ if ((vb->state == VB2_BUF_STATE_DEQUEUED ||
+ vb->state == VB2_BUF_STATE_IN_REQUEST) &&
+ vb->synced && vb->prepared)
+ *flags |= V4L2_BUF_FLAG_PREPARED;
+
+ if (vb2_buffer_in_use(q, vb))
+ *flags |= V4L2_BUF_FLAG_MAPPED;
+ if (vbuf->request_fd >= 0) {
+ *flags |= V4L2_BUF_FLAG_REQUEST_FD;
+ *request_fd = vbuf->request_fd;
+ }
+}
+
/*
* __fill_v4l2_buffer() - fill in a struct v4l2_buffer with information to be
* returned to userspace
@@ -546,51 +608,7 @@ static void __fill_v4l2_buffer(struct vb2_buffer *vb, void *pb)
b->m.fd = vb->planes[0].m.fd;
}
- /*
- * Clear any buffer state related flags.
- */
- b->flags &= ~V4L2_BUFFER_MASK_FLAGS;
- b->flags |= q->timestamp_flags & V4L2_BUF_FLAG_TIMESTAMP_MASK;
- if (!q->copy_timestamp) {
- /*
- * For non-COPY timestamps, drop timestamp source bits
- * and obtain the timestamp source from the queue.
- */
- b->flags &= ~V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
- b->flags |= q->timestamp_flags & V4L2_BUF_FLAG_TSTAMP_SRC_MASK;
- }
-
- switch (vb->state) {
- case VB2_BUF_STATE_QUEUED:
- case VB2_BUF_STATE_ACTIVE:
- b->flags |= V4L2_BUF_FLAG_QUEUED;
- break;
- case VB2_BUF_STATE_IN_REQUEST:
- b->flags |= V4L2_BUF_FLAG_IN_REQUEST;
- break;
- case VB2_BUF_STATE_ERROR:
- b->flags |= V4L2_BUF_FLAG_ERROR;
- fallthrough;
- case VB2_BUF_STATE_DONE:
- b->flags |= V4L2_BUF_FLAG_DONE;
- break;
- case VB2_BUF_STATE_PREPARING:
- case VB2_BUF_STATE_DEQUEUED:
- /* nothing */
- break;
- }
-
- if ((vb->state == VB2_BUF_STATE_DEQUEUED ||
- vb->state == VB2_BUF_STATE_IN_REQUEST) &&
- vb->synced && vb->prepared)
- b->flags |= V4L2_BUF_FLAG_PREPARED;
-
- if (vb2_buffer_in_use(q, vb))
- b->flags |= V4L2_BUF_FLAG_MAPPED;
- if (vbuf->request_fd >= 0) {
- b->flags |= V4L2_BUF_FLAG_REQUEST_FD;
- b->request_fd = vbuf->request_fd;
- }
+ v4l2_clear_buffer_flags(vb, &b->flags, q, vbuf, &b->request_fd);
}
/*
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v7 5/9] media: videobuf2: Expose helpers for Ext qbuf/dqbuf
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
` (3 preceding siblings ...)
2023-02-06 4:33 ` [PATCH v7 4/9] media: videobuf2-v4l2: reorganize flags handling ayaka
@ 2023-02-06 4:33 ` ayaka
2023-02-06 4:33 ` [PATCH v7 6/9] media: vivid: use vb2_ioctls_ext_{d}qbuf hooks ayaka
` (3 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike
From: Helen Koike <helen.koike@collabora.com>
To overcome the limitations of Ext ioctls, that is being converted to
classic hooks, add helpers to allow applications support layouts such as
using the same buffer with planes in different offsets.
To use the new hooks, drivers should:
static const struct v4l2_ioctl_ops ioctl_ops = {
...
+ .vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
+ .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
...
}
+ vb2_set_pixelformat(dev->pixelformat)
The old hooks should be kept to keep the driver compatible with classic
Api.
Add mem_offset field to struct vb2_plane, to allow tracking where the plane
starts in a buffer, as defined from userspace.
When returning the buffer to userspace, this offset can be adjusted
depending on the data_offset returned from the driver.
Add pixelformat field to struct vb2_buffer, to allow vb2 to know how to
decompose the payload set with vb2_set_plane_payload() into color planes
when a single memory buffer is used.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v7:
Refresh
Changes in v6:
This patch is based on original "media: videobuf2: Expose helpers to implement the _ext_fmt and _ext_buf hooks"
- This patch was completly refactored
- Conversions from v4l2_buffer to v4l2_ext_buffer was removed from vb2.
Both v4l2_buffer and v4l2_ext_buffer need to be supported, since Ext is only valid
for video devices, v4l2_buffer needs to be supported for vbi, meta, and others.
- Zero v4l2_ext_buffer.planes.m field (Tomasz for DMA-fd)
Changes in v5:
- Update with new format and buffer structs
- Updated commit message with the uAPI prefix
Changes in v4:
- Update with new format and buffer structs
- Fix some bugs caught by v4l2-compliance
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
.../media/common/videobuf2/videobuf2-core.c | 46 ++-
.../media/common/videobuf2/videobuf2-v4l2.c | 325 +++++++++++++++++-
include/media/videobuf2-core.h | 33 +-
include/media/videobuf2-v4l2.h | 8 +-
4 files changed, 387 insertions(+), 25 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 53e495223ea0..f5bea3890807 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1653,8 +1653,10 @@ static int vb2_start_streaming(struct vb2_queue *q)
return ret;
}
-int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
- struct media_request *req)
+static int
+vb2_core_qbuf_common(struct vb2_queue *q, unsigned int index,
+ struct v4l2_buffer *pb, struct v4l2_ext_buffer *eb,
+ struct media_request *req)
{
struct vb2_buffer *vb;
enum vb2_buffer_state orig_state;
@@ -1727,6 +1729,9 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
if (pb) {
call_void_bufop(q, copy_timestamp, vb, pb);
call_void_bufop(q, fill_user_buffer, vb, pb);
+ } else if (eb) {
+ call_void_bufop(q, copy_timestamp_ext, vb, eb);
+ call_void_bufop(q, fill_user_ext_buffer, vb, eb);
}
dprintk(q, 2, "qbuf of buffer %d succeeded\n", vb->index);
@@ -1766,6 +1771,8 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
if (pb)
call_void_bufop(q, copy_timestamp, vb, pb);
+ else if (eb)
+ call_void_bufop(q, copy_timestamp_ext, vb, eb);
trace_vb2_qbuf(q, vb);
@@ -1779,6 +1786,8 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
/* Fill buffer information for the userspace */
if (pb)
call_void_bufop(q, fill_user_buffer, vb, pb);
+ else if (eb)
+ call_void_bufop(q, fill_user_ext_buffer, vb, eb);
/*
* If streamon has been called, and we haven't yet called
@@ -1805,8 +1814,21 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
dprintk(q, 2, "qbuf of buffer %d succeeded\n", vb->index);
return 0;
}
+
+int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct media_request *req)
+{
+ return vb2_core_qbuf_common(q, index, pb, NULL, req);
+}
EXPORT_SYMBOL_GPL(vb2_core_qbuf);
+int vb2_core_ext_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct media_request *req)
+{
+ return vb2_core_qbuf_common(q, index, NULL, pb, req);
+}
+EXPORT_SYMBOL_GPL(vb2_core_ext_qbuf);
+
/*
* __vb2_wait_for_done_vb() - wait for a buffer to become available
* for dequeuing
@@ -1956,8 +1978,10 @@ static void __vb2_dqbuf(struct vb2_buffer *vb)
call_void_bufop(q, init_buffer, vb);
}
-int vb2_core_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
- bool nonblocking)
+static int
+vb2_core_dqbuf_common(struct vb2_queue *q, unsigned int *pindex,
+ struct v4l2_buffer *pb, struct v4l2_ext_buffer *eb,
+ bool nonblocking)
{
struct vb2_buffer *vb = NULL;
int ret;
@@ -1988,6 +2012,8 @@ int vb2_core_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
/* Fill buffer information for the userspace */
if (pb)
call_void_bufop(q, fill_user_buffer, vb, pb);
+ else if (eb)
+ call_void_bufop(q, fill_user_ext_buffer, vb, eb);
/* Remove from vb2 queue */
list_del(&vb->queued_entry);
@@ -2010,10 +2036,22 @@ int vb2_core_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
vb->index, vb2_state_name(vb->state));
return 0;
+}
+int vb2_core_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
+ bool nonblocking)
+{
+ return vb2_core_dqbuf_common(q, pindex, pb, NULL, nonblocking);
}
EXPORT_SYMBOL_GPL(vb2_core_dqbuf);
+int vb2_core_ext_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
+ bool nonblocking)
+{
+ return vb2_core_dqbuf_common(q, pindex, NULL, pb, nonblocking);
+}
+EXPORT_SYMBOL_GPL(vb2_core_ext_dqbuf);
+
/*
* __vb2_queue_cancel() - cancel and stop (pause) streaming
*
diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c
index eb158de7d14a..01541140f902 100644
--- a/drivers/media/common/videobuf2/videobuf2-v4l2.c
+++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c
@@ -157,6 +157,23 @@ static void __copy_timestamp(struct vb2_buffer *vb, const void *pb)
}
};
+static void __copy_timestamp_ext(struct vb2_buffer *vb, const void *pb)
+{
+ const struct v4l2_ext_buffer *eb = pb;
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_queue *q = vb->vb2_queue;
+
+ if (q->is_output) {
+ /*
+ * For output buffers copy the timestamp if needed,
+ * and the timecode field and flag if needed.
+ */
+ if (q->copy_timestamp)
+ vb->timestamp = eb->timestamp;
+ vbuf->flags |= eb->flags & V4L2_BUF_FLAG_TIMECODE;
+ }
+};
+
static void vb2_warn_zero_bytesused(struct vb2_buffer *vb)
{
static bool check_once;
@@ -210,6 +227,68 @@ vb2_fill_vb2_v4l2_buffer_flags(struct vb2_buffer *vb,
}
}
+static int vb2_fill_vb2_v4l2_buffer_ext(struct vb2_buffer *vb,
+ const struct v4l2_ext_buffer *eb)
+{
+ struct vb2_queue *q = vb->vb2_queue;
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_plane *planes = vbuf->planes;
+ unsigned int i;
+
+ if (eb->field == V4L2_FIELD_ALTERNATE && q->is_output) {
+ /*
+ * If the format's field is ALTERNATE, then the buffer's field
+ * should be either TOP or BOTTOM, not ALTERNATE since that
+ * makes no sense. The driver has to know whether the
+ * buffer represents a top or a bottom field in order to
+ * program any DMA correctly. Using ALTERNATE is wrong, since
+ * that just says that it is either a top or a bottom field,
+ * but not which of the two it is.
+ */
+ dprintk(q, 1, "the field is incorrectly set to ALTERNATE for an output buffer\n");
+ return -EINVAL;
+ }
+ vbuf->sequence = 0;
+ vbuf->request_fd = -1;
+ vbuf->is_held = false;
+
+ for (i = 0; i < VIDEO_MAX_PLANES && eb->planes[i].m.userptr; i++) {
+ if (eb->memory == VB2_MEMORY_MMAP) {
+ planes[i].m.userptr = vb->planes[i].m.userptr;
+ planes[i].length = vb->planes[i].length;
+ planes[i].mem_offset = vb->planes[i].mem_offset;
+ } else {
+ planes[i].m.userptr = eb->planes[i].m.userptr;
+ /*
+ * TODO: get dmabuf/mmap length, assing to max only for
+ * userptr, that we assume userspace passed the right
+ * size (can we assume that?)
+ */
+ planes[i].length = UINT_MAX;
+ planes[i].mem_offset = eb->planes[i].offset;
+ }
+
+ planes[i].data_offset = 0;
+ if (V4L2_TYPE_IS_OUTPUT(eb->type)) {
+ if (eb->planes[i].bytesused == 0)
+ vb2_warn_zero_bytesused(vb);
+
+ if (vb->vb2_queue->allow_zero_bytesused)
+ planes[i].bytesused = eb->planes[i].bytesused;
+ else
+ planes[i].bytesused = eb->planes[i].bytesused ?
+ eb->planes[i].bytesused :
+ planes[i].length;
+ } else {
+ planes[i].bytesused = 0;
+ }
+ }
+
+ vb2_fill_vb2_v4l2_buffer_flags(vb, eb->type, eb->field, eb->flags);
+
+ return 0;
+}
+
static int vb2_fill_vb2_v4l2_buffer(struct vb2_buffer *vb, struct v4l2_buffer *b)
{
struct vb2_queue *q = vb->vb2_queue;
@@ -373,8 +452,35 @@ static void set_buffer_cache_hints(struct vb2_queue *q,
vb->skip_cache_sync_on_prepare = 1;
}
-static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
- struct v4l2_buffer *b, bool is_prepare,
+static enum v4l2_buf_type vb2_ext_to_queue_type(unsigned int type,
+ bool is_multiplanar)
+{
+ if (!is_multiplanar)
+ return type;
+
+ if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
+ return V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ else if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT)
+ return V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+
+ return type;
+}
+
+static enum v4l2_buf_type vb2_queue_to_ext_type(unsigned int type)
+{
+ if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+ return V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ else if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE)
+ return V4L2_BUF_TYPE_VIDEO_OUTPUT;
+
+ return type;
+}
+
+static int vb2_queue_or_prepare_buf(struct vb2_queue *q,
+ struct media_device *mdev,
+ struct v4l2_buffer *b,
+ struct v4l2_ext_buffer *eb,
+ bool is_prepare,
struct media_request **p_req)
{
const char *opname = is_prepare ? "prepare_buf" : "qbuf";
@@ -382,46 +488,68 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
struct vb2_v4l2_buffer *vbuf;
struct vb2_buffer *vb;
int ret;
+ u32 index, memory, type, request_fd;
+ u32 *flags;
+
+ /* TODO: see how to improve this */
+ if (eb) {
+ index = eb->index;
+ type = vb2_ext_to_queue_type(eb->type, q->is_multiplanar);
+ memory = eb->memory;
+ flags = (u32*)&eb->flags;
+ request_fd = eb->request_fd;
+ } else {
+ index = b->index;
+ type = b->type;
+ memory = b->memory;
+ flags = &b->flags;
+ request_fd = b->request_fd;
+ }
- if (b->type != q->type) {
+ if (type != q->type) {
dprintk(q, 1, "%s: invalid buffer type\n", opname);
return -EINVAL;
}
- if (b->index >= q->num_buffers) {
+ if (index >= q->num_buffers) {
dprintk(q, 1, "%s: buffer index out of range\n", opname);
return -EINVAL;
}
- if (q->bufs[b->index] == NULL) {
+ if (q->bufs[index] == NULL) {
/* Should never happen */
dprintk(q, 1, "%s: buffer is NULL\n", opname);
return -EINVAL;
}
- if (b->memory != q->memory) {
+ if (memory != q->memory) {
dprintk(q, 1, "%s: invalid memory type\n", opname);
return -EINVAL;
}
- vb = q->bufs[b->index];
+ vb = q->bufs[index];
vbuf = to_vb2_v4l2_buffer(vb);
- ret = __verify_planes_array(vb, b);
- if (ret)
- return ret;
+ if (b) {
+ ret = __verify_planes_array(vb, b);
+ if (ret)
+ return ret;
+ }
- if (!is_prepare && (b->flags & V4L2_BUF_FLAG_REQUEST_FD) &&
+ if (!is_prepare && (*flags & V4L2_BUF_FLAG_REQUEST_FD) &&
vb->state != VB2_BUF_STATE_DEQUEUED) {
dprintk(q, 1, "%s: buffer is not in dequeued state\n", opname);
return -EINVAL;
}
if (!vb->prepared) {
- set_buffer_cache_hints(q, vb, &b->flags);
+ set_buffer_cache_hints(q, vb, flags);
/* Copy relevant information provided by the userspace */
memset(vbuf->planes, 0,
sizeof(vbuf->planes[0]) * vb->num_planes);
- ret = vb2_fill_vb2_v4l2_buffer(vb, b);
+ if (b)
+ ret = vb2_fill_vb2_v4l2_buffer(vb, b);
+ else
+ ret = vb2_fill_vb2_v4l2_buffer_ext(vb, eb);
if (ret)
return ret;
}
@@ -429,7 +557,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
if (is_prepare)
return 0;
- if (!(b->flags & V4L2_BUF_FLAG_REQUEST_FD)) {
+ if (!(*flags & V4L2_BUF_FLAG_REQUEST_FD)) {
if (q->requires_requests) {
dprintk(q, 1, "%s: queue requires requests\n", opname);
return -EBADR;
@@ -472,7 +600,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
!q->ops->buf_out_validate))
return -EINVAL;
- req = media_request_get_by_fd(mdev, b->request_fd);
+ req = media_request_get_by_fd(mdev, request_fd);
if (IS_ERR(req)) {
dprintk(q, 1, "%s: invalid request_fd\n", opname);
return PTR_ERR(req);
@@ -490,7 +618,7 @@ static int vb2_queue_or_prepare_buf(struct vb2_queue *q, struct media_device *md
}
*p_req = req;
- vbuf->request_fd = b->request_fd;
+ vbuf->request_fd = request_fd;
return 0;
}
@@ -547,6 +675,91 @@ static void v4l2_clear_buffer_flags(struct vb2_buffer *vb, u32 *flags,
}
}
+/*
+ * v4l2_decompose_planes - decompose format into color components
+ *
+ * @eplanes: pointer to struct v4l2_ext_plane to fill.
+ * @sizeimage: size of the image when using a single plane.
+ * @pixelformat: forcc to consider when decomposing @sizeimage.
+ * @data_offset: offset of data in buffer
+ * @membuf: buffer to fill in eplanes[i].m
+ *
+ * Fill out @eplanes, dividing @sizeimage according to @pixelformat, where
+ * @sizeimage is the size of all planes in a contiguous manner.
+ */
+static int v4l2_decompose_planes(struct v4l2_ext_plane *eplanes,
+ unsigned int sizeimage,
+ const struct v4l2_format_info *info,
+ unsigned int data_offset, u64 membuf)
+{
+ unsigned int i, y_plane_size, chroma_plane_size;
+
+ /*
+ * Considering:
+ * sizeimage = Y_plane_size + (comp_planes - 1) * chroma_plane_size
+ * Y_plane_size = chroma_plane_size * (hdiv*vdiv)
+ * Then:
+ * chroma_plane_size = sizeimage / (hdiv*vdiv + comp_planes - 1)
+ */
+ chroma_plane_size = sizeimage /
+ ((info->hdiv * info->vdiv) + info->comp_planes - 1);
+ y_plane_size = chroma_plane_size * info->hdiv * info->vdiv;
+ eplanes[0].m.userptr = membuf;
+ eplanes[0].bytesused = y_plane_size;
+ eplanes[0].offset = data_offset;
+
+ for (i = 1; i < info->comp_planes; i++) {
+ eplanes[i].m.userptr = membuf;
+ eplanes[i] = eplanes[0];
+ eplanes[i].offset = data_offset + y_plane_size +
+ chroma_plane_size * (i - 1);
+ eplanes[i].bytesused = chroma_plane_size;
+ }
+ return 0;
+}
+
+/*
+ * __fill_v4l2_ext_buffer() - fill in a struct v4l2_ext_buffer with information to be
+ * returned to userspace
+ */
+static void __fill_v4l2_ext_buffer(struct vb2_buffer *vb, void *pb)
+{
+ const struct v4l2_format_info *pix_info =
+ v4l2_format_info(vb->pixelformat);
+ struct v4l2_ext_buffer *eb = pb;
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct vb2_queue *q = vb->vb2_queue;
+ unsigned int i;
+
+ memset(eb, 0, sizeof(*eb));
+
+ /* Copy back data such as timestamp, flags, etc. */
+ eb->index = vb->index;
+ eb->type = vb2_queue_to_ext_type(vb->type);
+ eb->flags = vbuf->flags;
+ eb->field = vbuf->field;
+ eb->timestamp = vb->timestamp;
+ eb->sequence = vbuf->sequence;
+ eb->request_fd = 0;
+ eb->memory = vb->memory;
+
+ if (vb->num_planes == 1 && pix_info)
+ v4l2_decompose_planes(eb->planes,
+ vb->planes[0].bytesused - vb->planes[0].data_offset,
+ pix_info, vb->planes[i].data_offset,
+ vb->planes[0].m.userptr);
+ else
+ for (i = 0; i < vb->num_planes; i++) {
+ eb->planes[i].m.userptr = vb->planes[i].m.userptr;
+ eb->planes[i].bytesused = vb->planes[i].bytesused -
+ vb->planes[i].data_offset;
+ eb->planes[i].offset = vb->planes[i].mem_offset +
+ vb->planes[i].data_offset;
+ }
+
+ v4l2_clear_buffer_flags(vb, (u32*)&eb->flags, q, vbuf, &eb->request_fd);
+}
+
/*
* __fill_v4l2_buffer() - fill in a struct v4l2_buffer with information to be
* returned to userspace
@@ -639,8 +852,10 @@ static const struct vb2_buf_ops v4l2_buf_ops = {
.verify_planes_array = __verify_planes_array_core,
.init_buffer = __init_vb2_v4l2_buffer,
.fill_user_buffer = __fill_v4l2_buffer,
+ .fill_user_ext_buffer = __fill_v4l2_ext_buffer,
.fill_vb2_buffer = __fill_vb2_buffer,
.copy_timestamp = __copy_timestamp,
+ .copy_timestamp_ext = __copy_timestamp_ext,
};
struct vb2_buffer *vb2_find_buffer(struct vb2_queue *q, u64 timestamp)
@@ -751,7 +966,7 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
if (b->flags & V4L2_BUF_FLAG_REQUEST_FD)
return -EINVAL;
- ret = vb2_queue_or_prepare_buf(q, mdev, b, true, NULL);
+ ret = vb2_queue_or_prepare_buf(q, mdev, b, NULL, true, NULL);
return ret ? ret : vb2_core_prepare_buf(q, b->index, b);
}
@@ -828,7 +1043,7 @@ int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
return -EBUSY;
}
- ret = vb2_queue_or_prepare_buf(q, mdev, b, false, &req);
+ ret = vb2_queue_or_prepare_buf(q, mdev, b, NULL, false, &req);
if (ret)
return ret;
ret = vb2_core_qbuf(q, b->index, b, req);
@@ -838,6 +1053,27 @@ int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
}
EXPORT_SYMBOL_GPL(vb2_qbuf);
+int vb2_ext_qbuf(struct vb2_queue *q, struct media_device *mdev,
+ struct v4l2_ext_buffer *eb)
+{
+ struct media_request *req = NULL;
+ int ret;
+
+ if (vb2_fileio_is_active(q)) {
+ dprintk(q, 1, "file io in progress\n");
+ return -EBUSY;
+ }
+
+ ret = vb2_queue_or_prepare_buf(q, mdev, NULL, eb, false, &req);
+ if (ret)
+ return ret;
+ ret = vb2_core_ext_qbuf(q, eb->index, eb, req);
+ if (req)
+ media_request_put(req);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(vb2_ext_qbuf);
+
int vb2_dqbuf(struct vb2_queue *q, struct v4l2_buffer *b, bool nonblocking)
{
int ret;
@@ -869,6 +1105,38 @@ int vb2_dqbuf(struct vb2_queue *q, struct v4l2_buffer *b, bool nonblocking)
}
EXPORT_SYMBOL_GPL(vb2_dqbuf);
+int
+vb2_ext_dqbuf(struct vb2_queue *q, struct v4l2_ext_buffer *eb, bool nonblocking)
+{
+ int ret;
+
+ if (vb2_fileio_is_active(q)) {
+ dprintk(q, 1, "file io in progress\n");
+ return -EBUSY;
+ }
+
+ if (vb2_ext_to_queue_type(eb->type, q->is_multiplanar) != q->type) {
+ dprintk(q, 1, "invalid buffer type\n");
+ return -EINVAL;
+ }
+
+ ret = vb2_core_ext_dqbuf(q, NULL, eb, nonblocking);
+
+ if (!q->is_output &&
+ eb->flags & V4L2_BUF_FLAG_DONE &&
+ eb->flags & V4L2_BUF_FLAG_LAST)
+ q->last_buffer_dequeued = true;
+
+ /*
+ * After calling the VIDIOC_DQBUF V4L2_BUF_FLAG_DONE must be
+ * cleared.
+ */
+ eb->flags &= ~V4L2_BUF_FLAG_DONE;
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(vb2_ext_dqbuf);
+
int vb2_streamon(struct vb2_queue *q, enum v4l2_buf_type type)
{
if (vb2_fileio_is_active(q)) {
@@ -1077,6 +1345,16 @@ int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p)
}
EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf);
+int vb2_ioctl_ext_qbuf(struct file *file, void *priv, struct v4l2_ext_buffer *p)
+{
+ struct video_device *vdev = video_devdata(file);
+
+ if (vb2_queue_is_busy(vdev->queue, file))
+ return -EBUSY;
+ return vb2_ext_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p);
+}
+EXPORT_SYMBOL_GPL(vb2_ioctl_ext_qbuf);
+
int vb2_ioctl_dqbuf(struct file *file, void *priv, struct v4l2_buffer *p)
{
struct video_device *vdev = video_devdata(file);
@@ -1087,6 +1365,17 @@ int vb2_ioctl_dqbuf(struct file *file, void *priv, struct v4l2_buffer *p)
}
EXPORT_SYMBOL_GPL(vb2_ioctl_dqbuf);
+int
+vb2_ioctl_ext_dqbuf(struct file *file, void *priv, struct v4l2_ext_buffer *p)
+{
+ struct video_device *vdev = video_devdata(file);
+
+ if (vb2_queue_is_busy(vdev->queue, file))
+ return -EBUSY;
+ return vb2_ext_dqbuf(vdev->queue, p, file->f_flags & O_NONBLOCK);
+}
+EXPORT_SYMBOL_GPL(vb2_ioctl_ext_dqbuf);
+
int vb2_ioctl_streamon(struct file *file, void *priv, enum v4l2_buf_type i)
{
struct video_device *vdev = video_devdata(file);
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 4b6a9d2ea372..27dd4279e68b 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -169,8 +169,12 @@ struct vb2_mem_ops {
* pointing to this plane.
* @m.fd: when memory is %VB2_MEMORY_DMABUF, a userspace file
* descriptor associated with this plane.
+ * @mem_offset: offset where the plane starts. Usually 0, unless the buffer
+ * is shared by all planes of a multi-planar format.
* @data_offset: offset in the plane to the start of data; usually 0,
- * unless there is a header in front of the data.
+ * unless there is a header in front of the data. In Ext API, this
+ * is used by the driver to update the offset value from
+ * v4l2_ext_buffer accordinly.
*
* Should contain enough information to be able to cover all the fields
* of &struct v4l2_plane at videodev2.h.
@@ -187,6 +191,7 @@ struct vb2_plane {
unsigned long userptr;
int fd;
} m;
+ unsigned int mem_offset;
unsigned int data_offset;
};
@@ -239,12 +244,14 @@ struct vb2_queue;
* @index: id number of the buffer.
* @type: buffer type.
* @memory: the method, in which the actual data is passed.
- * @num_planes: number of planes in the buffer
+ * @num_planes: number of memory planes in the buffer
* on an internal driver queue.
* @timestamp: frame timestamp in ns.
* @request: the request this buffer is associated with.
* @req_obj: used to bind this buffer to a request. This
* request object has a refcount.
+ * @pixelformat pixelformat so vb2 can calculate the number of color
+ * planes.
*/
struct vb2_buffer {
struct vb2_queue *vb2_queue;
@@ -255,6 +262,7 @@ struct vb2_buffer {
u64 timestamp;
struct media_request *request;
struct media_request_object req_obj;
+ u32 pixelformat;
/* private: internal use only
*
@@ -463,18 +471,24 @@ struct vb2_ops {
* For V4L2 this is a &struct vb2_v4l2_buffer.
* @fill_user_buffer: given a &vb2_buffer fill in the userspace structure.
* For V4L2 this is a &struct v4l2_buffer.
+ * @fill_user_ext_buffer:given a &vb2_buffer fill in the userspace structure.
+ * For V4L2 this is a &struct v4l2_ext_buffer.
* @fill_vb2_buffer: given a userspace structure, fill in the &vb2_buffer.
* If the userspace structure is invalid, then this op
* will return an error.
* @copy_timestamp: copy the timestamp from a userspace structure to
* the &struct vb2_buffer.
+ * @copy_timestamp_ext: copy the timestamp from a userspace structure to
+ * the &struct vb2_buffer.
*/
struct vb2_buf_ops {
int (*verify_planes_array)(struct vb2_buffer *vb, const void *pb);
void (*init_buffer)(struct vb2_buffer *vb);
void (*fill_user_buffer)(struct vb2_buffer *vb, void *pb);
+ void (*fill_user_ext_buffer)(struct vb2_buffer *vb, void *pb);
int (*fill_vb2_buffer)(struct vb2_buffer *vb, struct vb2_plane *planes);
void (*copy_timestamp)(struct vb2_buffer *vb, const void *pb);
+ void (*copy_timestamp_ext)(struct vb2_buffer *vb, const void *eb);
};
/**
@@ -869,6 +883,8 @@ int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb);
*/
int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
struct media_request *req);
+int vb2_core_ext_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
+ struct media_request *req);
/**
* vb2_core_dqbuf() - Dequeue a buffer to the userspace
@@ -895,6 +911,8 @@ int vb2_core_qbuf(struct vb2_queue *q, unsigned int index, void *pb,
*/
int vb2_core_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
bool nonblocking);
+int vb2_core_ext_dqbuf(struct vb2_queue *q, unsigned int *pindex, void *pb,
+ bool nonblocking);
/**
* vb2_core_streamon() - Implements VB2 stream ON logic
@@ -1180,6 +1198,17 @@ static inline void vb2_set_plane_payload(struct vb2_buffer *vb,
}
}
+/**
+ * vb2_set_pixelformat() - set pixelformat for the buffer
+ * @vb: pointer to &struct vb2_buffer.
+ * @pixelformat: pixelformat to associate with the buffer.
+ */
+static inline void vb2_set_pixelformat(struct vb2_buffer *vb,
+ u32 pixelformat)
+{
+ vb->pixelformat = pixelformat;
+}
+
/**
* vb2_get_plane_payload() - get bytesused for the plane plane_no
* @vb: pointer to &struct vb2_buffer to which the plane in
diff --git a/include/media/videobuf2-v4l2.h b/include/media/videobuf2-v4l2.h
index 5a845887850b..fe07f4378400 100644
--- a/include/media/videobuf2-v4l2.h
+++ b/include/media/videobuf2-v4l2.h
@@ -39,7 +39,7 @@ struct video_device;
* @planes: plane information (userptr/fd, length, bytesused, data_offset).
*
* Should contain enough information to be able to cover all the fields
- * of &struct v4l2_buffer at ``videodev2.h``.
+ * of &struct v4l2_buffer and &struct v4l2_ext_buffer at ``videodev2.h``.
*/
struct vb2_v4l2_buffer {
struct vb2_buffer vb2_buf;
@@ -144,6 +144,8 @@ int vb2_prepare_buf(struct vb2_queue *q, struct media_device *mdev,
*/
int vb2_qbuf(struct vb2_queue *q, struct media_device *mdev,
struct v4l2_buffer *b);
+int vb2_ext_qbuf(struct vb2_queue *q, struct media_device *mdev,
+ struct v4l2_ext_buffer *b);
/**
* vb2_expbuf() - Export a buffer as a file descriptor
@@ -181,6 +183,8 @@ int vb2_expbuf(struct vb2_queue *q, struct v4l2_exportbuffer *eb);
* from &v4l2_ioctl_ops->vidioc_dqbuf handler in driver.
*/
int vb2_dqbuf(struct vb2_queue *q, struct v4l2_buffer *b, bool nonblocking);
+int vb2_ext_dqbuf(struct vb2_queue *q, struct v4l2_ext_buffer *b,
+ bool nonblocking);
/**
* vb2_streamon - start streaming
@@ -329,7 +333,9 @@ int vb2_ioctl_prepare_buf(struct file *file, void *priv,
struct v4l2_buffer *p);
int vb2_ioctl_querybuf(struct file *file, void *priv, struct v4l2_buffer *p);
int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p);
+int vb2_ioctl_ext_qbuf(struct file *file, void *priv, struct v4l2_ext_buffer *p);
int vb2_ioctl_dqbuf(struct file *file, void *priv, struct v4l2_buffer *p);
+int vb2_ioctl_ext_dqbuf(struct file *file, void *priv, struct v4l2_ext_buffer *p);
int vb2_ioctl_streamon(struct file *file, void *priv, enum v4l2_buf_type i);
int vb2_ioctl_streamoff(struct file *file, void *priv, enum v4l2_buf_type i);
int vb2_ioctl_expbuf(struct file *file, void *priv,
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v7 6/9] media: vivid: use vb2_ioctls_ext_{d}qbuf hooks
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
` (4 preceding siblings ...)
2023-02-06 4:33 ` [PATCH v7 5/9] media: videobuf2: Expose helpers for Ext qbuf/dqbuf ayaka
@ 2023-02-06 4:33 ` ayaka
2023-02-06 4:33 ` [PATCH v7 7/9] media: vimc: " ayaka
` (2 subsequent siblings)
8 siblings, 0 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike
From: Helen Koike <helen.koike@collabora.com>
Add vb2 ext hooks and call vb2_set_pixelformat().
This allows more flexibility with buffer handling.
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v6:
- New patch to exemplify how drivers would easily support features from Ext Buf
---
drivers/media/test-drivers/vivid/vivid-core.c | 2 ++
drivers/media/test-drivers/vivid/vivid-vid-cap.c | 1 +
drivers/media/test-drivers/vivid/vivid-vid-out.c | 1 +
3 files changed, 4 insertions(+)
diff --git a/drivers/media/test-drivers/vivid/vivid-core.c b/drivers/media/test-drivers/vivid/vivid-core.c
index d4ed55aef1bb..f033c9c4e9aa 100644
--- a/drivers/media/test-drivers/vivid/vivid-core.c
+++ b/drivers/media/test-drivers/vivid/vivid-core.c
@@ -728,6 +728,8 @@ static const struct v4l2_ioctl_ops vivid_ioctl_ops = {
.vidioc_querybuf = vb2_ioctl_querybuf,
.vidioc_qbuf = vb2_ioctl_qbuf,
.vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
+ .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
.vidioc_expbuf = vb2_ioctl_expbuf,
.vidioc_streamon = vb2_ioctl_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-cap.c b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
index 3e3a94a2e3d6..bbe0fc2a3627 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-cap.c
+++ b/drivers/media/test-drivers/vivid/vivid-vid-cap.c
@@ -170,6 +170,7 @@ static int vid_cap_buf_prepare(struct vb2_buffer *vb)
}
vb2_set_plane_payload(vb, p, size);
+ vb2_set_pixelformat(vb, dev->fmt_cap->fourcc);
vb->planes[p].data_offset = dev->fmt_cap->data_offset[p];
}
diff --git a/drivers/media/test-drivers/vivid/vivid-vid-out.c b/drivers/media/test-drivers/vivid/vivid-vid-out.c
index 936ba93c00cc..5f224f13aa99 100644
--- a/drivers/media/test-drivers/vivid/vivid-vid-out.c
+++ b/drivers/media/test-drivers/vivid/vivid-vid-out.c
@@ -137,6 +137,7 @@ static int vid_out_buf_prepare(struct vb2_buffer *vb)
return -EINVAL;
}
}
+ vb2_set_pixelformat(vb, dev->fmt_out->fourcc);
return 0;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v7 7/9] media: vimc: use vb2_ioctls_ext_{d}qbuf hooks
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
` (5 preceding siblings ...)
2023-02-06 4:33 ` [PATCH v7 6/9] media: vivid: use vb2_ioctls_ext_{d}qbuf hooks ayaka
@ 2023-02-06 4:33 ` ayaka
2023-02-06 4:33 ` [PATCH v7 8/9] media: mediabus: Add helpers to convert a ext_pix format to/from a mbus_fmt ayaka
2023-02-06 4:33 ` [PATCH v7 9/9] media: vimc: Convert to v4l2_ext_pix_format ayaka
8 siblings, 0 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike
From: Helen Koike <helen.koike@collabora.com>
Add vb2 ext hooks and call vb2_set_pixelformat().
This allows more flexibility with buffer handling.
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v7:
- Refresh and rebase
Changes in v6:
- New patch to exemplify how drivers would easily support features from Ext Buf
---
drivers/media/test-drivers/vimc/vimc-capture.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c b/drivers/media/test-drivers/vimc/vimc-capture.c
index aa944270e716..246d90d1f5ae 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -218,6 +218,8 @@ static const struct v4l2_ioctl_ops vimc_capture_ioctl_ops = {
.vidioc_querybuf = vb2_ioctl_querybuf,
.vidioc_qbuf = vb2_ioctl_qbuf,
.vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_ext_qbuf = vb2_ioctl_ext_qbuf,
+ .vidioc_ext_dqbuf = vb2_ioctl_ext_dqbuf,
.vidioc_expbuf = vb2_ioctl_expbuf,
.vidioc_streamon = vb2_ioctl_streamon,
.vidioc_streamoff = vb2_ioctl_streamoff,
@@ -389,6 +391,8 @@ static void *vimc_capture_process_frame(struct vimc_ent_device *ved,
/* Set it as ready */
vb2_set_plane_payload(&vimc_buf->vb2.vb2_buf, 0,
vcapture->format.sizeimage);
+ vb2_set_pixelformat(&vimc_buf->vb2.vb2_buf,
+ vcapture->format.pixelformat);
vb2_buffer_done(&vimc_buf->vb2.vb2_buf, VB2_BUF_STATE_DONE);
return NULL;
}
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v7 8/9] media: mediabus: Add helpers to convert a ext_pix format to/from a mbus_fmt
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
` (6 preceding siblings ...)
2023-02-06 4:33 ` [PATCH v7 7/9] media: vimc: " ayaka
@ 2023-02-06 4:33 ` ayaka
2023-02-06 4:33 ` [PATCH v7 9/9] media: vimc: Convert to v4l2_ext_pix_format ayaka
8 siblings, 0 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike
From: Helen Koike <helen.koike@collabora.com>
Just a new version of v4l2_fill_mbus_format() and v4l2_fill_ext_pix_format()
to deal with the new v4l2_ext_pix_format struct.
This is needed to convert the VIMC driver to the EXT_FMT/EXT_BUF iocts.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
---
Changes in v6:
- Rename v4l2_fill_ext_pix_format() to v4l2_fill_ext_pix_format_from_mbus() (Tomasz)
Changes in v4:
- Add helper v4l2_fill_ext_pix_format()
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
include/media/v4l2-mediabus.h | 42 +++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)
diff --git a/include/media/v4l2-mediabus.h b/include/media/v4l2-mediabus.h
index 5bce6e423e94..3959d0abde03 100644
--- a/include/media/v4l2-mediabus.h
+++ b/include/media/v4l2-mediabus.h
@@ -252,4 +252,46 @@ v4l2_fill_mbus_format_mplane(struct v4l2_mbus_framefmt *mbus_fmt,
mbus_fmt->xfer_func = pix_mp_fmt->xfer_func;
}
+/**
+ * v4l2_fill_ext_pix_format_from_mbus - Ancillary routine that fills a &struct
+ * v4l2_ext_pix_format fields from a &struct v4l2_mbus_framefmt.
+ *
+ * @pix_fmt: pointer to &struct v4l2_ext_pix_format to be filled
+ * @mbus_fmt: pointer to &struct v4l2_mbus_framefmt to be used as model
+ */
+static inline void
+v4l2_fill_ext_pix_format_from_mbus(struct v4l2_ext_pix_format *pix_fmt,
+ const struct v4l2_mbus_framefmt *mbus_fmt)
+{
+ pix_fmt->width = mbus_fmt->width;
+ pix_fmt->height = mbus_fmt->height;
+ pix_fmt->field = mbus_fmt->field;
+ pix_fmt->colorspace = mbus_fmt->colorspace;
+ pix_fmt->ycbcr_enc = mbus_fmt->ycbcr_enc;
+ pix_fmt->quantization = mbus_fmt->quantization;
+ pix_fmt->xfer_func = mbus_fmt->xfer_func;
+}
+
+/**
+ * v4l2_fill_mbus_format_ext - Ancillary routine that fills a &struct
+ * v4l2_mbus_framefmt from a &struct v4l2_ext_pix_format.
+ *
+ * @mbus_fmt: pointer to &struct v4l2_mbus_framefmt to be filled
+ * @pix_fmt: pointer to &struct v4l2_ext_pix_format to be used as model
+ * @code: data format code (from &enum v4l2_mbus_pixelcode)
+ */
+static inline void
+v4l2_fill_mbus_format_ext(struct v4l2_mbus_framefmt *mbus_fmt,
+ const struct v4l2_ext_pix_format *pix_fmt, u32 code)
+{
+ mbus_fmt->width = pix_fmt->width;
+ mbus_fmt->height = pix_fmt->height;
+ mbus_fmt->field = pix_fmt->field;
+ mbus_fmt->colorspace = pix_fmt->colorspace;
+ mbus_fmt->ycbcr_enc = pix_fmt->ycbcr_enc;
+ mbus_fmt->quantization = pix_fmt->quantization;
+ mbus_fmt->xfer_func = pix_fmt->xfer_func;
+ mbus_fmt->code = code;
+}
+
#endif
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread* [PATCH v7 9/9] media: vimc: Convert to v4l2_ext_pix_format
2023-02-06 4:32 [RFC PATCH v7 0/9] media: v4l2: Add extended fmt and buffer ayaka
` (7 preceding siblings ...)
2023-02-06 4:33 ` [PATCH v7 8/9] media: mediabus: Add helpers to convert a ext_pix format to/from a mbus_fmt ayaka
@ 2023-02-06 4:33 ` ayaka
8 siblings, 0 replies; 21+ messages in thread
From: ayaka @ 2023-02-06 4:33 UTC (permalink / raw)
To: linux-media
Cc: randy.li, Brian.Starkey, boris.brezillon, frkoenig, hans.verkuil,
hiroh, hverkuil, kernel, laurent.pinchart, linux-kernel, mchehab,
narmstrong, nicolas, sakari.ailus, stanimir.varbanov, tfiga,
Helen Koike, Randy Li
From: Helen Koike <helen.koike@collabora.com>
Simplify Multi/Single planer API handling by converting to v4l2_ext_pix_format.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Signed-off-by: Randy Li <ayaka@soulik.info>
---
Changes in v7:
- Refresh and rebase
Changes in v6:
- Update with new format and buffer structs
Changes in v4:
- Update with new format and buffer structs
- Rebased on top of media/master (post 5.8-rc1)
Changes in v3:
- Rebased on top of media/master (post 5.4-rc1)
Changes in v2:
- New patch
---
.../media/test-drivers/vimc/vimc-capture.c | 115 ++++++++++--------
drivers/media/test-drivers/vimc/vimc-common.c | 6 +-
drivers/media/test-drivers/vimc/vimc-common.h | 2 +-
3 files changed, 68 insertions(+), 55 deletions(-)
diff --git a/drivers/media/test-drivers/vimc/vimc-capture.c b/drivers/media/test-drivers/vimc/vimc-capture.c
index 246d90d1f5ae..5429398bea8d 100644
--- a/drivers/media/test-drivers/vimc/vimc-capture.c
+++ b/drivers/media/test-drivers/vimc/vimc-capture.c
@@ -16,7 +16,7 @@
struct vimc_capture_device {
struct vimc_ent_device ved;
struct video_device vdev;
- struct v4l2_pix_format format;
+ struct v4l2_ext_pix_format format;
struct vb2_queue queue;
struct list_head buf_list;
/*
@@ -33,7 +33,8 @@ struct vimc_capture_device {
struct media_pad pad;
};
-static const struct v4l2_pix_format fmt_default = {
+static const struct v4l2_ext_pix_format fmt_default = {
+ .type = V4L2_BUF_TYPE_VIDEO_CAPTURE,
.width = 640,
.height = 480,
.pixelformat = V4L2_PIX_FMT_RGB24,
@@ -53,7 +54,7 @@ struct vimc_capture_buffer {
};
static int vimc_capture_querycap(struct file *file, void *priv,
- struct v4l2_capability *cap)
+ struct v4l2_capability *cap)
{
strscpy(cap->driver, VIMC_PDEV_NAME, sizeof(cap->driver));
strscpy(cap->card, KBUILD_MODNAME, sizeof(cap->card));
@@ -64,28 +65,28 @@ static int vimc_capture_querycap(struct file *file, void *priv,
}
static void vimc_capture_get_format(struct vimc_ent_device *ved,
- struct v4l2_pix_format *fmt)
+ struct v4l2_ext_pix_format *fmt)
{
- struct vimc_capture_device *vcapture = container_of(ved, struct vimc_capture_device,
- ved);
+ struct vimc_capture_device *vcapture =
+ container_of(ved, struct vimc_capture_device,
+ ved);
*fmt = vcapture->format;
}
static int vimc_capture_g_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
struct vimc_capture_device *vcapture = video_drvdata(file);
- f->fmt.pix = vcapture->format;
+ *f = vcapture->format;
return 0;
}
static int vimc_capture_try_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *format)
{
- struct v4l2_pix_format *format = &f->fmt.pix;
const struct vimc_pix_map *vpix;
format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
@@ -100,8 +101,10 @@ static int vimc_capture_try_fmt_vid_cap(struct file *file, void *priv,
vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
}
/* TODO: Add support for custom bytesperline values */
- format->bytesperline = format->width * vpix->bpp;
- format->sizeimage = format->bytesperline * format->height;
+ memset(format->plane_fmt, 0, sizeof(format->plane_fmt));
+ format->plane_fmt[0].bytesperline = format->width * vpix->bpp;
+ format->plane_fmt[0].sizeimage = format->plane_fmt[0].bytesperline *
+ format->height;
if (format->field == V4L2_FIELD_ANY)
format->field = fmt_default.field;
@@ -115,7 +118,7 @@ static int vimc_capture_try_fmt_vid_cap(struct file *file, void *priv,
}
static int vimc_capture_s_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_format *f)
+ struct v4l2_ext_pix_format *f)
{
struct vimc_capture_device *vcapture = video_drvdata(file);
int ret;
@@ -137,18 +140,16 @@ static int vimc_capture_s_fmt_vid_cap(struct file *file, void *priv,
vcapture->format.quantization, vcapture->format.xfer_func,
vcapture->format.ycbcr_enc,
/* new */
- f->fmt.pix.width, f->fmt.pix.height,
- f->fmt.pix.pixelformat, f->fmt.pix.colorspace,
- f->fmt.pix.quantization, f->fmt.pix.xfer_func,
- f->fmt.pix.ycbcr_enc);
+ f->width, f->height, f->pixelformat, f->colorspace,
+ f->quantization, f->xfer_func, f->ycbcr_enc);
- vcapture->format = f->fmt.pix;
+ vcapture->format = *f;
return 0;
}
static int vimc_capture_enum_fmt_vid_cap(struct file *file, void *priv,
- struct v4l2_fmtdesc *f)
+ struct v4l2_fmtdesc *f)
{
const struct vimc_pix_map *vpix;
@@ -170,7 +171,7 @@ static int vimc_capture_enum_fmt_vid_cap(struct file *file, void *priv,
}
static int vimc_capture_enum_framesizes(struct file *file, void *fh,
- struct v4l2_frmsizeenum *fsize)
+ struct v4l2_frmsizeenum *fsize)
{
const struct vimc_pix_map *vpix;
@@ -206,9 +207,9 @@ static const struct v4l2_file_operations vimc_capture_fops = {
static const struct v4l2_ioctl_ops vimc_capture_ioctl_ops = {
.vidioc_querycap = vimc_capture_querycap,
- .vidioc_g_fmt_vid_cap = vimc_capture_g_fmt_vid_cap,
- .vidioc_s_fmt_vid_cap = vimc_capture_s_fmt_vid_cap,
- .vidioc_try_fmt_vid_cap = vimc_capture_try_fmt_vid_cap,
+ .vidioc_g_ext_pix_fmt_vid_cap = vimc_capture_g_fmt_vid_cap,
+ .vidioc_s_ext_pix_fmt_vid_cap = vimc_capture_s_fmt_vid_cap,
+ .vidioc_try_ext_pix_fmt_vid_cap = vimc_capture_try_fmt_vid_cap,
.vidioc_enum_fmt_vid_cap = vimc_capture_enum_fmt_vid_cap,
.vidioc_enum_framesizes = vimc_capture_enum_framesizes,
@@ -225,8 +226,9 @@ static const struct v4l2_ioctl_ops vimc_capture_ioctl_ops = {
.vidioc_streamoff = vb2_ioctl_streamoff,
};
-static void vimc_capture_return_all_buffers(struct vimc_capture_device *vcapture,
- enum vb2_buffer_state state)
+static void vimc_capture_return_all_buffers(struct vimc_capture_device
+ *vcapture,
+ enum vb2_buffer_state state)
{
struct vimc_capture_buffer *vbuf, *node;
@@ -240,7 +242,8 @@ static void vimc_capture_return_all_buffers(struct vimc_capture_device *vcapture
spin_unlock(&vcapture->qlock);
}
-static int vimc_capture_start_streaming(struct vb2_queue *vq, unsigned int count)
+static int vimc_capture_start_streaming(struct vb2_queue *vq,
+ unsigned int count)
{
struct vimc_capture_device *vcapture = vb2_get_drv_priv(vq);
int ret;
@@ -248,7 +251,9 @@ static int vimc_capture_start_streaming(struct vb2_queue *vq, unsigned int count
vcapture->sequence = 0;
/* Start the media pipeline */
- ret = video_device_pipeline_start(&vcapture->vdev, &vcapture->stream.pipe);
+ ret =
+ video_device_pipeline_start(&vcapture->vdev,
+ &vcapture->stream.pipe);
if (ret) {
vimc_capture_return_all_buffers(vcapture, VB2_BUF_STATE_QUEUED);
return ret;
@@ -283,27 +288,31 @@ static void vimc_capture_stop_streaming(struct vb2_queue *vq)
static void vimc_capture_buf_queue(struct vb2_buffer *vb2_buf)
{
- struct vimc_capture_device *vcapture = vb2_get_drv_priv(vb2_buf->vb2_queue);
+ struct vimc_capture_device *vcapture =
+ vb2_get_drv_priv(vb2_buf->vb2_queue);
struct vimc_capture_buffer *buf = container_of(vb2_buf,
- struct vimc_capture_buffer,
- vb2.vb2_buf);
+ struct
+ vimc_capture_buffer,
+ vb2.vb2_buf);
spin_lock(&vcapture->qlock);
list_add_tail(&buf->list, &vcapture->buf_list);
spin_unlock(&vcapture->qlock);
}
-static int vimc_capture_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers,
- unsigned int *nplanes, unsigned int sizes[],
- struct device *alloc_devs[])
+static int vimc_capture_queue_setup(struct vb2_queue *vq,
+ unsigned int *nbuffers,
+ unsigned int *nplanes, unsigned int sizes[],
+ struct device *alloc_devs[])
{
struct vimc_capture_device *vcapture = vb2_get_drv_priv(vq);
if (*nplanes)
- return sizes[0] < vcapture->format.sizeimage ? -EINVAL : 0;
+ return sizes[0] <
+ vcapture->format.plane_fmt[0].sizeimage ? -EINVAL : 0;
/* We don't support multiplanes for now */
*nplanes = 1;
- sizes[0] = vcapture->format.sizeimage;
+ sizes[0] = vcapture->format.plane_fmt[0].sizeimage;
return 0;
}
@@ -311,7 +320,7 @@ static int vimc_capture_queue_setup(struct vb2_queue *vq, unsigned int *nbuffers
static int vimc_capture_buffer_prepare(struct vb2_buffer *vb)
{
struct vimc_capture_device *vcapture = vb2_get_drv_priv(vb->vb2_queue);
- unsigned long size = vcapture->format.sizeimage;
+ unsigned long size = vcapture->format.plane_fmt[0].sizeimage;
if (vb2_plane_size(vb, 0) < size) {
dev_err(vcapture->ved.dev, "%s: buffer too small (%lu < %lu)\n",
@@ -342,7 +351,7 @@ static const struct media_entity_operations vimc_capture_mops = {
static void vimc_capture_release(struct vimc_ent_device *ved)
{
struct vimc_capture_device *vcapture =
- container_of(ved, struct vimc_capture_device, ved);
+ container_of(ved, struct vimc_capture_device, ved);
media_entity_cleanup(vcapture->ved.ent);
kfree(vcapture);
@@ -351,16 +360,17 @@ static void vimc_capture_release(struct vimc_ent_device *ved)
static void vimc_capture_unregister(struct vimc_ent_device *ved)
{
struct vimc_capture_device *vcapture =
- container_of(ved, struct vimc_capture_device, ved);
+ container_of(ved, struct vimc_capture_device, ved);
vb2_video_unregister_device(&vcapture->vdev);
}
static void *vimc_capture_process_frame(struct vimc_ent_device *ved,
- const void *frame)
+ const void *frame)
{
- struct vimc_capture_device *vcapture = container_of(ved, struct vimc_capture_device,
- ved);
+ struct vimc_capture_device *vcapture =
+ container_of(ved, struct vimc_capture_device,
+ ved);
struct vimc_capture_buffer *vimc_buf;
void *vbuf;
@@ -386,11 +396,11 @@ static void *vimc_capture_process_frame(struct vimc_ent_device *ved,
vbuf = vb2_plane_vaddr(&vimc_buf->vb2.vb2_buf, 0);
- memcpy(vbuf, frame, vcapture->format.sizeimage);
+ memcpy(vbuf, frame, vcapture->format.plane_fmt[0].sizeimage);
/* Set it as ready */
vb2_set_plane_payload(&vimc_buf->vb2.vb2_buf, 0,
- vcapture->format.sizeimage);
+ vcapture->format.plane_fmt[0].sizeimage);
vb2_set_pixelformat(&vimc_buf->vb2.vb2_buf,
vcapture->format.pixelformat);
vb2_buffer_done(&vimc_buf->vb2.vb2_buf, VB2_BUF_STATE_DONE);
@@ -398,7 +408,7 @@ static void *vimc_capture_process_frame(struct vimc_ent_device *ved,
}
static struct vimc_ent_device *vimc_capture_add(struct vimc_device *vimc,
- const char *vcfg_name)
+ const char *vcfg_name)
{
struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
const struct vimc_pix_map *vpix;
@@ -416,8 +426,7 @@ static struct vimc_ent_device *vimc_capture_add(struct vimc_device *vimc,
vcapture->vdev.entity.name = vcfg_name;
vcapture->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
vcapture->pad.flags = MEDIA_PAD_FL_SINK;
- ret = media_entity_pads_init(&vcapture->vdev.entity,
- 1, &vcapture->pad);
+ ret = media_entity_pads_init(&vcapture->vdev.entity, 1, &vcapture->pad);
if (ret)
goto err_free_vcapture;
@@ -434,7 +443,7 @@ static struct vimc_ent_device *vimc_capture_add(struct vimc_device *vimc,
q->buf_struct_size = sizeof(struct vimc_capture_buffer);
q->ops = &vimc_capture_qops;
q->mem_ops = vimc_allocator == VIMC_ALLOCATOR_DMA_CONTIG
- ? &vb2_dma_contig_memops : &vb2_vmalloc_memops;
+ ? &vb2_dma_contig_memops : &vb2_vmalloc_memops;
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
q->min_buffers_needed = 2;
q->lock = &vcapture->lock;
@@ -454,9 +463,13 @@ static struct vimc_ent_device *vimc_capture_add(struct vimc_device *vimc,
/* Set default frame format */
vcapture->format = fmt_default;
vpix = vimc_pix_map_by_pixelformat(vcapture->format.pixelformat);
- vcapture->format.bytesperline = vcapture->format.width * vpix->bpp;
- vcapture->format.sizeimage = vcapture->format.bytesperline *
- vcapture->format.height;
+ memset(vcapture->format.plane_fmt, 0,
+ sizeof(vcapture->format.plane_fmt));
+ vcapture->format.plane_fmt[0].bytesperline =
+ vcapture->format.width * vpix->bpp;
+ vcapture->format.plane_fmt[0].sizeimage =
+ vcapture->format.plane_fmt[0].bytesperline *
+ vcapture->format.height;
/* Fill the vimc_ent_device struct */
vcapture->ved.ent = &vcapture->vdev.entity;
@@ -467,7 +480,7 @@ static struct vimc_ent_device *vimc_capture_add(struct vimc_device *vimc,
/* Initialize the video_device struct */
vdev = &vcapture->vdev;
vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING
- | V4L2_CAP_IO_MC;
+ | V4L2_CAP_IO_MC;
vdev->entity.ops = &vimc_capture_mops;
vdev->release = video_device_release_empty;
vdev->fops = &vimc_capture_fops;
diff --git a/drivers/media/test-drivers/vimc/vimc-common.c b/drivers/media/test-drivers/vimc/vimc-common.c
index 7b27153c0728..8bbf80f2acb9 100644
--- a/drivers/media/test-drivers/vimc/vimc-common.c
+++ b/drivers/media/test-drivers/vimc/vimc-common.c
@@ -236,7 +236,7 @@ const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat)
}
static int vimc_get_pix_format(struct media_pad *pad,
- struct v4l2_pix_format *fmt)
+ struct v4l2_ext_pix_format *fmt)
{
if (is_media_entity_v4l2_subdev(pad->entity)) {
struct v4l2_subdev *sd =
@@ -252,7 +252,7 @@ static int vimc_get_pix_format(struct media_pad *pad,
if (ret)
return ret;
- v4l2_fill_pix_format(fmt, &sd_fmt.format);
+ v4l2_fill_ext_pix_format_from_mbus(fmt, &sd_fmt.format);
pix_map = vimc_pix_map_by_code(sd_fmt.format.code);
fmt->pixelformat = pix_map->pixelformat;
} else if (is_media_entity_v4l2_video_device(pad->entity)) {
@@ -274,7 +274,7 @@ static int vimc_get_pix_format(struct media_pad *pad,
int vimc_vdev_link_validate(struct media_link *link)
{
- struct v4l2_pix_format source_fmt, sink_fmt;
+ struct v4l2_ext_pix_format source_fmt, sink_fmt;
int ret;
ret = vimc_get_pix_format(link->source, &source_fmt);
diff --git a/drivers/media/test-drivers/vimc/vimc-common.h b/drivers/media/test-drivers/vimc/vimc-common.h
index 7641a101a728..f29429df10dd 100644
--- a/drivers/media/test-drivers/vimc/vimc-common.h
+++ b/drivers/media/test-drivers/vimc/vimc-common.h
@@ -111,7 +111,7 @@ struct vimc_ent_device {
void * (*process_frame)(struct vimc_ent_device *ved,
const void *frame);
void (*vdev_get_format)(struct vimc_ent_device *ved,
- struct v4l2_pix_format *fmt);
+ struct v4l2_ext_pix_format *fmt);
};
/**
--
2.17.1
^ permalink raw reply related [flat|nested] 21+ messages in thread