[PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

Hello,

The tvp5150 video decoder is usually used on a video pipeline with several
video processing integrated circuits. So the driver has to be migrated to
the new media device API to reflect this at the software level.

Also the tvp5150 is able to detect what is the video standard at which
the device is currently operating, so it makes sense to add video format
detection in the driver as well as.

This patch-set migrates the tvp5150 driver to the MCF and adds video format detection.

It is composed of the following patches:

[PATCH 1/3] [media] tvp5150: Add constants for PAL and NTSC video standards

[Javier Martinez Canillas]

Signed-off-by: Javier Martinez Canillas <martinez.javier@gmail.com>
---
 include/media/tvp5150.h |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/include/media/tvp5150.h b/include/media/tvp5150.h
index 72bd2a2..ccd4ed0 100644
--- a/include/media/tvp5150.h
+++ b/include/media/tvp5150.h
@@ -30,5 +30,11 @@
 #define TVP5150_NORMAL       0
 #define TVP5150_BLACK_SCREEN 1
 
+/* Number of pixels and number of lines per frame for different standards */
+#define NTSC_NUM_ACTIVE_PIXELS		(720)
+#define NTSC_NUM_ACTIVE_LINES		(480)
+#define PAL_NUM_ACTIVE_PIXELS		(720)
+#define PAL_NUM_ACTIVE_LINES		(576)
+
 #endif
 
-- 
1.7.4.1

[PATCH 2/3] [media] tvp5150: Add video format registers configuration values

[Javier Martinez Canillas]

The tvp5150 video decoder has two operation modes to configure the video
standard used.

If auto-switch mode is enable, the device can sense the signal and detect which
video standard the device is operating. Also the device can be forced to use a
user defined video standard.

Each operation mode uses a different register and the bitmask values to
represent each standard is different.

So we add video standard constants for both autoswitch and no-autoswitch mode.

Signed-off-by: Javier Martinez Canillas <martinez.javier@gmail.com>
---
 drivers/media/video/tvp5150_reg.h |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/drivers/media/video/tvp5150_reg.h b/drivers/media/video/tvp5150_reg.h
index 4240043..25a9949 100644
--- a/drivers/media/video/tvp5150_reg.h
+++ b/drivers/media/video/tvp5150_reg.h
@@ -45,7 +45,22 @@
 
 /* Reserved 1Fh-27h */
 
-#define TVP5150_VIDEO_STD           0x28 /* Video standard */
+#define VIDEO_STD_MASK			 (0x07 >> 1)
+#define TVP5150_VIDEO_STD                0x28 /* Video standard */
+#define VIDEO_STD_AUTO_SWITCH_BIT	 0x00
+#define VIDEO_STD_NTSC_MJ_BIT		 0x02
+#define VIDEO_STD_PAL_BDGHIN_BIT	 0x04
+#define VIDEO_STD_PAL_M_BIT		 0x06
+#define VIDEO_STD_PAL_COMBINATION_N_BIT	 0x08
+#define VIDEO_STD_NTSC_4_43_BIT		 0x0a
+#define VIDEO_STD_SECAM_BIT		 0x0c
+
+#define VIDEO_STD_NTSC_MJ_BIT_AS                 0x01
+#define VIDEO_STD_PAL_BDGHIN_BIT_AS              0x03
+#define VIDEO_STD_PAL_M_BIT_AS		         0x05
+#define VIDEO_STD_PAL_COMBINATION_N_BIT_AS	 0x07
+#define VIDEO_STD_NTSC_4_43_BIT_AS		 0x09
+#define VIDEO_STD_SECAM_BIT_AS		         0x0b
 
 /* Reserved 29h-2bh */
 
-- 
1.7.4.1

[PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

The tvp5150 video decoder is usually used on a video pipeline with several
video processing integrated circuits. So the driver has to be migrated to
the new media device API to reflect this at the software level.

Also the tvp5150 is able to detect what is the video standard at which
the device is currently operating, so it makes sense to add video format
detection in the driver.

Signed-off-by: Javier Martinez Canillas <martinez.javier@gmail.com>
---
 drivers/media/video/tvp5150.c |  400 +++++++++++++++++++++++++++++++++++++++--
 1 files changed, 389 insertions(+), 11 deletions(-)

diff --git a/drivers/media/video/tvp5150.c b/drivers/media/video/tvp5150.c
index e927d25..1c771f9 100644
--- a/drivers/media/video/tvp5150.c
+++ b/drivers/media/video/tvp5150.c
@@ -13,6 +13,7 @@
 #include <media/tvp5150.h>
 #include <media/v4l2-chip-ident.h>
 #include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
 
 #include "tvp5150_reg.h"
 
@@ -25,11 +26,79 @@ static int debug;
 module_param(debug, int, 0);
 MODULE_PARM_DESC(debug, "Debug level (0-2)");
 
+/* enum tvp515x_std - enum for supported standards */
+enum tvp515x_std {
+	STD_PAL_BDGHIN = 0,
+	STD_NTSC_MJ,
+	STD_INVALID
+};
+
+/**
+ * struct tvp515x_std_info - Structure to store standard informations
+ * @width: Line width in pixels
+ * @height:Number of active lines
+ * @video_std: Value to write in REG_VIDEO_STD register
+ * @standard: v4l2 standard structure information
+ */
+struct tvp515x_std_info {
+	u8 video_std;
+	struct v4l2_standard standard;
+	struct v4l2_mbus_framefmt format;
+};
+
+/**
+ * Supported standards -
+ *
+ * Currently supports two standards only, need to add support for rest of the
+ * modes, like SECAM, etc...
+ */
+static struct tvp515x_std_info tvp515x_std_list[] = {
+	/* Standard: STD_NTSC_MJ */
+	/* Standard: STD_PAL_BDGHIN */
+	[STD_PAL_BDGHIN] = {
+		.video_std = VIDEO_STD_PAL_BDGHIN_BIT,
+		.standard = {
+			.index = 1,
+			.id = V4L2_STD_PAL,
+			.name = "PAL",
+			.frameperiod = {1, 25},
+			.framelines = 625
+		},
+		.format = {
+			.width = PAL_NUM_ACTIVE_PIXELS,
+			.height = PAL_NUM_ACTIVE_LINES,
+			.code = V4L2_MBUS_FMT_UYVY8_2X8,
+			.field = V4L2_FIELD_INTERLACED,
+			.colorspace = V4L2_COLORSPACE_SMPTE170M,
+		},
+	},
+	[STD_NTSC_MJ] = {
+		.video_std = VIDEO_STD_NTSC_MJ_BIT,
+		.standard = {
+			.index = 0,
+			.id = V4L2_STD_NTSC,
+			.name = "NTSC",
+			.frameperiod = {1001, 30000},
+			.framelines = 525
+		},
+		.format = {
+			.width = NTSC_NUM_ACTIVE_PIXELS,
+			.height = NTSC_NUM_ACTIVE_LINES,
+			.code = V4L2_MBUS_FMT_UYVY8_2X8,
+			.field = V4L2_FIELD_INTERLACED,
+			.colorspace = V4L2_COLORSPACE_SMPTE170M,
+		},
+	},
+	/* Standard: need to add for additional standard */
+};
+
 struct tvp5150 {
 	struct v4l2_subdev sd;
 	struct v4l2_ctrl_handler hdl;
-
-	v4l2_std_id norm;	/* Current set standard */
+	struct media_pad pad;
+	struct v4l2_mbus_framefmt *format;
+	v4l2_std_id std_idx;
+	int norm;
 	u32 input;
 	u32 output;
 	int enable;
@@ -692,6 +761,45 @@ static int tvp5150_get_vbi(struct v4l2_subdev *sd,
 	return type;
 }
 
+/**
+ * tvp515x_query_current_std() : Query the current standard detected by TVP5151
+ * @sd: ptr to v4l2_subdev struct
+ *
+ * Returns the current standard detected by TVP5151, STD_INVALID if there is no
+ * standard detected.
+ */
+static int tvp515x_query_current_std(struct v4l2_subdev *sd)
+{
+	u8 std, std_status;
+
+	std = tvp5150_read(sd, TVP5150_VIDEO_STD);
+	if ((std & VIDEO_STD_MASK) == VIDEO_STD_AUTO_SWITCH_BIT)
+		/* use the standard status register */
+		std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
+	else
+		/* use the standard register itself */
+		std_status = std;
+
+	switch (std_status & VIDEO_STD_MASK) {
+	case VIDEO_STD_NTSC_MJ_BIT:
+	case VIDEO_STD_NTSC_MJ_BIT_AS:
+		return STD_NTSC_MJ;
+
+	case VIDEO_STD_PAL_BDGHIN_BIT:
+	case VIDEO_STD_PAL_BDGHIN_BIT_AS:
+		return STD_PAL_BDGHIN;
+
+	default:
+		return STD_INVALID;
+	}
+
+	return STD_INVALID;
+}
+
+/****************************************************************************
+			V4L2 subdev video operations
+ ****************************************************************************/
+
 static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
 {
 	struct tvp5150 *decoder = to_tvp5150(sd);
@@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
 	if (std == V4L2_STD_ALL) {
 		fmt = 0;	/* Autodetect mode */
 	} else if (std & V4L2_STD_NTSC_443) {
-		fmt = 0xa;
+		fmt = VIDEO_STD_NTSC_4_43_BIT;
 	} else if (std & V4L2_STD_PAL_M) {
-		fmt = 0x6;
+		fmt = VIDEO_STD_PAL_M_BIT;
 	} else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
-		fmt = 0x8;
+		fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
 	} else {
 		/* Then, test against generic ones */
 		if (std & V4L2_STD_NTSC)
-			fmt = 0x2;
+			fmt = VIDEO_STD_NTSC_MJ_BIT;
 		else if (std & V4L2_STD_PAL)
-			fmt = 0x4;
+			fmt = VIDEO_STD_PAL_BDGHIN_BIT;
 		else if (std & V4L2_STD_SECAM)
-			fmt = 0xc;
+			fmt = VIDEO_STD_SECAM_BIT;
 	}
 
 	v4l2_dbg(1, debug, sd, "Set video std register to %d.\n", fmt);
@@ -727,11 +835,26 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
 static int tvp5150_s_std(struct v4l2_subdev *sd, v4l2_std_id std)
 {
 	struct tvp5150 *decoder = to_tvp5150(sd);
+	int i;
+	int num_stds = ARRAY_SIZE(tvp515x_std_list);
 
 	if (decoder->norm == std)
 		return 0;
 
-	return tvp5150_set_std(sd, std);
+	for (i = 0; i < num_stds; i++)
+		if (std & tvp515x_std_list[i].standard.id)
+			break;
+
+	if ((i == num_stds) || (i == STD_INVALID))
+		return -EINVAL;
+
+	tvp5150_write(sd, TVP5150_VIDEO_STD, tvp515x_std_list[i].video_std);
+
+	decoder->norm = i;
+	decoder->norm = std;
+
+/*	return tvp5150_set_std(sd, std); */
+	return 0;
 }
 
 static int tvp5150_reset(struct v4l2_subdev *sd, u32 val)
@@ -778,6 +901,177 @@ static int tvp5150_s_ctrl(struct v4l2_ctrl *ctrl)
 	return -EINVAL;
 }
 
+static struct v4l2_mbus_framefmt *
+__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
+			 unsigned int pad, enum v4l2_subdev_format_whence which)
+{
+	switch (which) {
+	case V4L2_SUBDEV_FORMAT_TRY:
+		return v4l2_subdev_get_try_format(fh, pad);
+	case V4L2_SUBDEV_FORMAT_ACTIVE:
+		return tvp5150->format;
+	default:
+		return NULL;
+	}
+}
+
+static int tvp5150_get_pad_format(struct v4l2_subdev *subdev,
+			      struct v4l2_subdev_fh *fh,
+			      struct v4l2_subdev_format *format)
+{
+	struct tvp5150 *tvp5150 = to_tvp5150(subdev);
+
+	format->format = *__tvp5150_get_pad_format(tvp5150, fh, format->pad,
+						   format->which);
+
+	return 0;
+}
+
+static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
+			      struct v4l2_subdev_fh *fh,
+			      struct v4l2_subdev_format *format)
+{
+	struct tvp5150 *tvp5150 = to_tvp5150(subdev);
+	tvp5150->std_idx = STD_INVALID;
+
+	tvp5150->std_idx = tvp515x_query_current_std(subdev);
+	if (tvp5150->std_idx == STD_INVALID) {
+		v4l2_err(subdev, "Unable to query std\n");
+		return 0;
+	}
+
+	tvp5150->norm = tvp515x_std_list[tvp5150->std_idx].standard.id;
+
+	tvp5150->format = &tvp515x_std_list[tvp5150->std_idx].format;
+
+	format->format = *__tvp5150_get_pad_format(tvp5150, fh, format->pad,
+	format->which);
+
+	v4l2_info(subdev, "code=x%x width=%u height=%u colorspace=0x%x\n",
+			format->format.code, format->format.width,
+			format->format.height, format->format.colorspace);
+
+	return 0;
+}
+
+/**
+ * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
+ * @sd: pointer to standard V4L2 sub-device structure
+ * @f: pointer to the mediabus format structure
+ *
+ * Negotiates the image capture size and mediabus format.
+ */
+static int
+tvp515x_mbus_fmt(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt *f)
+{
+	struct tvp5150 *decoder = to_tvp5150(sd);
+
+	if (f == NULL)
+		return -EINVAL;
+
+	f = decoder->format;
+
+	v4l2_dbg(1, debug, sd, "MBUS_FMT: Width - %d, Height - %d\n",
+			f->width, f->height);
+	return 0;
+}
+
+/*
+ * tvp515x_s_stream() - V4L2 decoder i/f handler for s_stream
+ * @sd: pointer to standard V4L2 sub-device structure
+ * @enable: streaming enable or disable
+ *
+ * Sets streaming to enable or disable, if possible.
+ */
+static int tvp515x_s_stream(struct v4l2_subdev *subdev, int enable)
+{
+
+	/* Initializes TVP5150 to its default values */
+	/* # set PCLK (27MHz) */
+	tvp5150_write(subdev, TVP5150_CONF_SHARED_PIN, 0x00);
+
+	/* Output format: 8-bit ITU-R BT.656 with embedded syncs */
+	if (enable)
+		tvp5150_write(subdev, TVP5150_MISC_CTL, 0x09);
+	else
+		tvp5150_write(subdev, TVP5150_MISC_CTL, 0x00);
+
+	return 0;
+}
+
+
+/**
+ * tvp515x_enum_mbus_fmt() - V4L2 decoder interface handler for enum_mbus_fmt
+ * @sd: pointer to standard V4L2 sub-device structure
+ * @index: index of pixelcode to retrieve
+ * @code: receives the pixelcode
+ *
+ * Enumerates supported mediabus formats
+ */
+static int
+tvp515x_enum_mbus_fmt(struct v4l2_subdev *sd, unsigned index,
+					enum v4l2_mbus_pixelcode *code)
+{
+	if (index)
+		return -EINVAL;
+
+	*code = V4L2_MBUS_FMT_UYVY8_2X8;
+	return 0;
+}
+
+/**
+ * tvp515x_g_parm() - V4L2 decoder interface handler for g_parm
+ * @sd: pointer to standard V4L2 sub-device structure
+ * @a: pointer to standard V4L2 VIDIOC_G_PARM ioctl structure
+ *
+ * Returns the decoder's video CAPTURE parameters.
+ */
+static int
+tvp515x_g_parm(struct v4l2_subdev *sd, struct v4l2_streamparm *a)
+{
+	struct v4l2_captureparm *cparm;
+
+	if (a == NULL)
+		return -EINVAL;
+
+	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		/* only capture is supported */
+		return -EINVAL;
+
+	cparm = &a->parm.capture;
+	cparm->capability = V4L2_CAP_TIMEPERFRAME;
+	cparm->timeperframe = a->parm.capture.timeperframe;
+
+	return 0;
+}
+
+/**
+ * tvp515x_s_parm() - V4L2 decoder interface handler for s_parm
+ * @sd: pointer to standard V4L2 sub-device structure
+ * @a: pointer to standard V4L2 VIDIOC_S_PARM ioctl structure
+ *
+ * Configures the decoder to use the input parameters, if possible. If
+ * not possible, returns the appropriate error code.
+ */
+static int
+tvp515x_s_parm(struct v4l2_subdev *sd, struct v4l2_streamparm *a)
+{
+	struct v4l2_fract *timeperframe;
+
+	if (a == NULL)
+		return -EINVAL;
+
+	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		/* only capture is supported */
+		return -EINVAL;
+
+	timeperframe = &a->parm.capture.timeperframe;
+
+	return 0;
+}
+
+
+
 /****************************************************************************
 			I2C Command
  ****************************************************************************/
@@ -900,7 +1194,28 @@ static int tvp5150_g_tuner(struct v4l2_subdev *sd, struct v4l2_tuner *vt)
 	return 0;
 }
 
-/* ----------------------------------------------------------------------- */
+/****************************************************************************
+		    V4L2 subdev core operations
+ ****************************************************************************/
+
+static int tvp5150_open(struct v4l2_subdev *subdev, struct v4l2_subdev_fh *fh)
+{
+	struct tvp5150 *decoder = to_tvp5150(subdev);
+
+	decoder->std_idx = STD_INVALID;
+
+	decoder->std_idx = tvp515x_query_current_std(subdev);
+
+	if (decoder->std_idx == STD_INVALID) {
+		v4l2_err(subdev, "Unable to query std\n");
+		return 0;
+	}
+
+	decoder->format = (&(tvp515x_std_list[decoder->std_idx].format));
+	decoder->norm = tvp515x_std_list[decoder->std_idx].standard.id;
+
+	return 0;
+}
 
 static const struct v4l2_ctrl_ops tvp5150_ctrl_ops = {
 	.s_ctrl = tvp5150_s_ctrl,
@@ -924,12 +1239,24 @@ static const struct v4l2_subdev_core_ops tvp5150_core_ops = {
 #endif
 };
 
+static struct v4l2_subdev_file_ops tvp5150_subdev_file_ops = {
+	.open		= tvp5150_open,
+};
+
 static const struct v4l2_subdev_tuner_ops tvp5150_tuner_ops = {
 	.g_tuner = tvp5150_g_tuner,
 };
 
 static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
 	.s_routing = tvp5150_s_routing,
+	.s_stream = tvp515x_s_stream,
+	.enum_mbus_fmt = tvp515x_enum_mbus_fmt,
+	.g_mbus_fmt = tvp515x_mbus_fmt,
+	.try_mbus_fmt = tvp515x_mbus_fmt,
+	.s_mbus_fmt = tvp515x_mbus_fmt,
+	.g_parm = tvp515x_g_parm,
+	.s_parm = tvp515x_s_parm,
+	.s_std_output = tvp5150_s_std,
 };
 
 static const struct v4l2_subdev_vbi_ops tvp5150_vbi_ops = {
@@ -939,14 +1266,57 @@ static const struct v4l2_subdev_vbi_ops tvp5150_vbi_ops = {
 	.s_raw_fmt = tvp5150_s_raw_fmt,
 };
 
+static int tvp515x_enum_mbus_code(struct v4l2_subdev *subdev,
+				  struct v4l2_subdev_fh *fh,
+				  struct v4l2_subdev_mbus_code_enum *code)
+{
+	if (code->index >= ARRAY_SIZE(tvp515x_std_list))
+		return -EINVAL;
+
+	code->code = V4L2_MBUS_FMT_UYVY8_2X8;
+
+	return 0;
+}
+
+static int tvp515x_enum_frame_size(struct v4l2_subdev *subdev,
+				   struct v4l2_subdev_fh *fh,
+				   struct v4l2_subdev_frame_size_enum *fse)
+{
+	int current_std = STD_INVALID;
+
+	if (fse->code != V4L2_MBUS_FMT_UYVY8_2X8)
+		return -EINVAL;
+
+	/* query the current standard */
+	current_std = tvp515x_query_current_std(subdev);
+	if (current_std == STD_INVALID) {
+		v4l2_err(subdev, "Unable to query std\n");
+		return 0;
+	}
+
+	fse->min_width = tvp515x_std_list[current_std].format.width;
+	fse->min_height = tvp515x_std_list[current_std].format.height;
+	fse->max_width = fse->min_width;
+	fse->max_height = fse->min_height;
+	return 0;
+}
+
+static struct v4l2_subdev_pad_ops tvp5150_pad_ops = {
+	.enum_mbus_code = tvp515x_enum_mbus_code,
+	.enum_frame_size = tvp515x_enum_frame_size,
+	.get_fmt = tvp5150_get_pad_format,
+	.set_fmt = tvp5150_set_pad_format,
+};
+
 static const struct v4l2_subdev_ops tvp5150_ops = {
 	.core = &tvp5150_core_ops,
+	.file	= &tvp5150_subdev_file_ops,
 	.tuner = &tvp5150_tuner_ops,
 	.video = &tvp5150_video_ops,
 	.vbi = &tvp5150_vbi_ops,
+	.pad = &tvp5150_pad_ops,
 };
 
-
 /****************************************************************************
 			I2C Client & Driver
  ****************************************************************************/
@@ -957,6 +1327,7 @@ static int tvp5150_probe(struct i2c_client *c,
 	struct tvp5150 *core;
 	struct v4l2_subdev *sd;
 	u8 msb_id, lsb_id, msb_rom, lsb_rom;
+	int ret;
 
 	/* Check if the adapter supports the needed features */
 	if (!i2c_check_functionality(c->adapter,
@@ -992,6 +1363,7 @@ static int tvp5150_probe(struct i2c_client *c,
 		}
 	}
 
+	core->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
 	core->norm = V4L2_STD_ALL;	/* Default is autodetect */
 	core->input = TVP5150_COMPOSITE1;
 	core->enable = 1;
@@ -1017,6 +1389,12 @@ static int tvp5150_probe(struct i2c_client *c,
 
 	if (debug > 1)
 		tvp5150_log_status(sd);
+
+	core->pad.flags = MEDIA_PAD_FLAG_OUTPUT;
+	ret = media_entity_init(&core->sd.entity, 1, &core->pad, 0);
+	if (ret < 0)
+		kfree(core);
+
 	return 0;
 }
 
-- 
1.7.4.1

Re: [PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Gary Thomas]

On 2011-09-30 18:33, Javier Martinez Canillas wrote:
> Hello,
>
> The tvp5150 video decoder is usually used on a video pipeline with several
> video processing integrated circuits. So the driver has to be migrated to
> the new media device API to reflect this at the software level.
>
> Also the tvp5150 is able to detect what is the video standard at which
> the device is currently operating, so it makes sense to add video format
> detection in the driver as well as.
>
> This patch-set migrates the tvp5150 driver to the MCF and adds video format detection.
>

What is this patchset relative to?
Does it still handle the case of overscan? e.g. I typically capture from
an NTSC source using this device at 720x524.
Even if it does detect the signal shape (NTSC, PAL), doesn't one still need
to [externally] configure the pads for this shape?

Have you given any thought as to how the input (composite-A, composite-B or S-Video)
could be configured using the MCF?

Note: CC list trimmed to only the linux-media list.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Sat, Oct 1, 2011 at 3:34 PM, Gary Thomas <gary@mlbassoc.com> wrote:
> On 2011-09-30 18:33, Javier Martinez Canillas wrote:
>>
>> Hello,
>>
>> The tvp5150 video decoder is usually used on a video pipeline with several
>> video processing integrated circuits. So the driver has to be migrated to
>> the new media device API to reflect this at the software level.
>>
>> Also the tvp5150 is able to detect what is the video standard at which
>> the device is currently operating, so it makes sense to add video format
>> detection in the driver as well as.
>>
>> This patch-set migrates the tvp5150 driver to the MCF and adds video
>> format detection.
>>
>
> What is this patchset relative to?

Hello Gary,

Thank you, I'm a newbie with v4l2 in general and media controller
framework in particular so your comments and suggestions are highly
appreciated.

I'm working to have proper support for the tvp5151 video capture
connected through its parallel interface with our custom TI DM3730 ARM
OMAP board. I think it's better to show the code as early as possible
so I can have feedback from the community an see if I'm in the right
path or completely lost, that is this patch-set about :)

We hack a few bits of the ISP CCDC driver to support ITU-R BT656
interlaced data with embedded syncs video format and ported the
tvp5150 driver to the MCF so it can be detected as a sub-device and be
part of the OMAP ISP image processing pipeline (as a source pad).

We are configuring the graph like this:

./media-ctl -r -l '"tvp5150 2-005c":0->"OMAP3 ISP CCDC":0[1], "OMAP3
ISP CCDC":1->"OMAP3 ISP CCDC output":0[1]'

I thought (probably wrong for your comments) that since the tvp5150
can sense which signal shape is being transfer to it (NTSC/PAL/etc) we
can configure automatically the tvp5150 source pad frame format to
capture all the lines (not only the visible ones). And if user space
wants a different frame format we can process the image latter.

So we only have to configure the ISP CCDC input pad format to be
coherent with the tvp5150 output pad.

./media-ctl --set-format '"OMAP3 ISP CCDC":0 [UYVY 720x625]'

> Does it still handle the case of overscan? e.g. I typically capture from
> an NTSC source using this device at 720x524.

For the case of the overscan of if you want to crop the image I
thought to use either the CCDC (to copy less lines on the memory
output buffer) or the ISP resizer. But in that case one has to
manually configure a different pipeline including the resizer and set
the frame formats for each input and output pad (probably I'm wrong
with this approach too).

> Even if it does detect the signal shape (NTSC, PAL), doesn't one still need
> to [externally] configure the pads for this shape?
>

Yes, that is why I wanted to do the auto-detection for the tvp5151, so
we only have to manually configure the ISP components (or any other
hardware video processing pipeline entities, sorry for my
OMAP-specific comments).

> Have you given any thought as to how the input (composite-A, composite-B or
> S-Video)
> could be configured using the MCF?
>

I didn't know that the physical connection affected the video output
format, I thought that it was only a physical medium to carry the same
information, sorry if my comments are silly but I'm really newbie with
video in general.

> Note: CC list trimmed to only the linux-media list.
>

Thanks a lot, I just followed get_maintainer script suggestions.

Best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Enrico]

On Sat, Oct 1, 2011 at 5:55 PM, Javier Martinez Canillas
<martinez.javier@gmail.com> wrote:
> We hack a few bits of the ISP CCDC driver to support ITU-R BT656
> interlaced data with embedded syncs video format and ported the
> tvp5150 driver to the MCF so it can be detected as a sub-device and be
> part of the OMAP ISP image processing pipeline (as a source pad).

That was already posted on the list [1], there was some discussion but
i don't know what's the status/plan to get it into mainline.

And, as you can see in [2], don't expect many comments :D

[1]: http://www.spinics.net/lists/linux-media/msg37710.html
[2]: http://www.spinics.net/lists/linux-media/msg37116.html


>> Even if it does detect the signal shape (NTSC, PAL), doesn't one still need
>> to [externally] configure the pads for this shape?
>>
>
> Yes, that is why I wanted to do the auto-detection for the tvp5151, so
> we only have to manually configure the ISP components (or any other
> hardware video processing pipeline entities, sorry for my
> OMAP-specific comments).

Laurent was not very happy [3] about changing video formats out of the
driver control, so this should be discussed more.

[3]: http://www.spinics.net/lists/linux-omap/msg56983.html


> I didn't know that the physical connection affected the video output
> format, I thought that it was only a physical medium to carry the same
> information, sorry if my comments are silly but I'm really newbie with
> video in general.

I think you got it right, i haven't tested it but the output format
shouldn't be affected by the video source( if it stays pal/ntsc of
course). Maybe you will get only a different "active" video area so
only cropping will be affected.

Enrico

Re: [PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Sat, Oct 1, 2011 at 6:39 PM, Enrico <ebutera@users.berlios.de> wrote:
> On Sat, Oct 1, 2011 at 5:55 PM, Javier Martinez Canillas
> <martinez.javier@gmail.com> wrote:
>> We hack a few bits of the ISP CCDC driver to support ITU-R BT656
>> interlaced data with embedded syncs video format and ported the
>> tvp5150 driver to the MCF so it can be detected as a sub-device and be
>> part of the OMAP ISP image processing pipeline (as a source pad).
>
> That was already posted on the list [1], there was some discussion but
> i don't know what's the status/plan to get it into mainline.
>

Hello Enrico,

Yes, I saw it. That is why I didn't post our modifications to the ISP
CCDC driver. Our approach is very similar to the one followed by TI
(changing the CCDC output buffer every two VD0 interrupts) but we did
different a few things:

- decouple next buffer obtaining from last buffer releasing
- maintain two buffers (struct isp_buffer), current and last
- move move most of the logic to the VD1 interrupt since the ISP is
already busy while execution VD0 handler.

> And, as you can see in [2], don't expect many comments :D
>

Yes, now I saw that you already posted this, sorry for not doing a
correct mail archive browsing before posting the patch.

> [1]: http://www.spinics.net/lists/linux-media/msg37710.html
> [2]: http://www.spinics.net/lists/linux-media/msg37116.html
>
>
>>> Even if it does detect the signal shape (NTSC, PAL), doesn't one still need
>>> to [externally] configure the pads for this shape?
>>>
>>
>> Yes, that is why I wanted to do the auto-detection for the tvp5151, so
>> we only have to manually configure the ISP components (or any other
>> hardware video processing pipeline entities, sorry for my
>> OMAP-specific comments).
>
> Laurent was not very happy [3] about changing video formats out of the
> driver control, so this should be discussed more.
>
> [3]: http://www.spinics.net/lists/linux-omap/msg56983.html
>
>

Ok, I thought it was the right thing to do, my bad. Lets do it from
user-space then using the MCF.

>> I didn't know that the physical connection affected the video output
>> format, I thought that it was only a physical medium to carry the same
>> information, sorry if my comments are silly but I'm really newbie with
>> video in general.
>
> I think you got it right, i haven't tested it but the output format
> shouldn't be affected by the video source( if it stays pal/ntsc of
> course). Maybe you will get only a different "active" video area so
> only cropping will be affected.
>
> Enrico
>

Thanks and best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Enrico]

On Sat, Oct 1, 2011 at 7:27 PM, Javier Martinez Canillas
<martinez.javier@gmail.com> wrote:
> Yes, I saw it. That is why I didn't post our modifications to the ISP
> CCDC driver. Our approach is very similar to the one followed by TI
> (changing the CCDC output buffer every two VD0 interrupts) but we did
> different a few things:
>
> - decouple next buffer obtaining from last buffer releasing
> - maintain two buffers (struct isp_buffer), current and last
> - move move most of the logic to the VD1 interrupt since the ISP is
> already busy while execution VD0 handler.

If you think it is a better approach you can submit it for review,
right now there is no "clean" version supporting bt656 so yours can be
the one.


>>>> Even if it does detect the signal shape (NTSC, PAL), doesn't one still need
>>>> to [externally] configure the pads for this shape?
>>>>
>>>
>>> Yes, that is why I wanted to do the auto-detection for the tvp5151, so
>>> we only have to manually configure the ISP components (or any other
>>> hardware video processing pipeline entities, sorry for my
>>> OMAP-specific comments).
>>
>> Laurent was not very happy [3] about changing video formats out of the
>> driver control, so this should be discussed more.
>>
>> [3]: http://www.spinics.net/lists/linux-omap/msg56983.html
>>
>>
>
> Ok, I thought it was the right thing to do, my bad. Lets do it from
> user-space then using the MCF.

I see there is some ongoing discussion about a similar topic, so just
follow it and see how it turns out.

Enrico

Re: [PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Sat, Oct 1, 2011 at 7:46 PM, Enrico <ebutera@users.berlios.de> wrote:
> On Sat, Oct 1, 2011 at 7:27 PM, Javier Martinez Canillas
> <martinez.javier@gmail.com> wrote:
>> Yes, I saw it. That is why I didn't post our modifications to the ISP
>> CCDC driver. Our approach is very similar to the one followed by TI
>> (changing the CCDC output buffer every two VD0 interrupts) but we did
>> different a few things:
>>
>> - decouple next buffer obtaining from last buffer releasing
>> - maintain two buffers (struct isp_buffer), current and last
>> - move move most of the logic to the VD1 interrupt since the ISP is
>> already busy while execution VD0 handler.
>
> If you think it is a better approach you can submit it for review,
> right now there is no "clean" version supporting bt656 so yours can be
> the one.
>
>

Ok, I don't know if my approach is the "cleaner" version to support
bt656, but I will cleanup the code to be in a merge-able state and
send for review.

Something that I don't know if I got it right is configure the CCDC to
capture and odd numbers of lines (i.e: 625 lines from a PAL frame)
since the first and second sub-frames have different numbers of lines
(i.e: 313 and 312).

My solution was to reprogram the CCDC in the VD1 interrupt handler so
it can use the new configuration for the next sub-frame:

static void ispccdc_change_numlines(struct isp_device *isp, int numlines)
{
	isp_reg_writel(isp, ((numlines - 1) << ISPCCDC_VDINT_0_SHIFT) |
		       ((numlines * 2 / 3) << ISPCCDC_VDINT_1_SHIFT),
		       OMAP3_ISP_IOMEM_CCDC, ISPCCDC_VDINT);
	isp_reg_writel(isp, (numlines - 1)
		       << ISPCCDC_VERT_LINES_NLV_SHIFT,
		       OMAP3_ISP_IOMEM_CCDC, ISPCCDC_VERT_LINES);
}

Is this the right approach? because I didn't find in the TRM how to
configure the CCDC to generate the VD0 and VD1 interrupts with
variable frame vertical length.

>>>>> Even if it does detect the signal shape (NTSC, PAL), doesn't one still need
>>>>> to [externally] configure the pads for this shape?
>>>>>
>>>>
>>>> Yes, that is why I wanted to do the auto-detection for the tvp5151, so
>>>> we only have to manually configure the ISP components (or any other
>>>> hardware video processing pipeline entities, sorry for my
>>>> OMAP-specific comments).
>>>
>>> Laurent was not very happy [3] about changing video formats out of the
>>> driver control, so this should be discussed more.
>>>
>>> [3]: http://www.spinics.net/lists/linux-omap/msg56983.html
>>>
>>>
>>
>> Ok, I thought it was the right thing to do, my bad. Lets do it from
>> user-space then using the MCF.
>
> I see there is some ongoing discussion about a similar topic, so just
> follow it and see how it turns out.
>
> Enrico
>

Ok, I will. Thank you very much for your comments.

Best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Sakari Ailus]

Hi Javier,

Thanks for the patch! It's very interesting to see a driver for a video
decoder using the MC interface. Before this we've had just image sensors.

Javier Martinez Canillas wrote:
> The tvp5150 video decoder is usually used on a video pipeline with several
> video processing integrated circuits. So the driver has to be migrated to
> the new media device API to reflect this at the software level.
> 
> Also the tvp5150 is able to detect what is the video standard at which
> the device is currently operating, so it makes sense to add video format
> detection in the driver.
> 
> Signed-off-by: Javier Martinez Canillas <martinez.javier@gmail.com>
> ---
>  drivers/media/video/tvp5150.c |  400 +++++++++++++++++++++++++++++++++++++++--
>  1 files changed, 389 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/media/video/tvp5150.c b/drivers/media/video/tvp5150.c
> index e927d25..1c771f9 100644
> --- a/drivers/media/video/tvp5150.c
> +++ b/drivers/media/video/tvp5150.c
> @@ -13,6 +13,7 @@
>  #include <media/tvp5150.h>
>  #include <media/v4l2-chip-ident.h>
>  #include <media/v4l2-ctrls.h>
> +#include <media/v4l2-subdev.h>
>  
>  #include "tvp5150_reg.h"
>  
> @@ -25,11 +26,79 @@ static int debug;
>  module_param(debug, int, 0);
>  MODULE_PARM_DESC(debug, "Debug level (0-2)");
>  
> +/* enum tvp515x_std - enum for supported standards */
> +enum tvp515x_std {
> +	STD_PAL_BDGHIN = 0,
> +	STD_NTSC_MJ,
> +	STD_INVALID
> +};
> +
> +/**
> + * struct tvp515x_std_info - Structure to store standard informations
> + * @width: Line width in pixels
> + * @height:Number of active lines
> + * @video_std: Value to write in REG_VIDEO_STD register
> + * @standard: v4l2 standard structure information
> + */
> +struct tvp515x_std_info {
> +	u8 video_std;
> +	struct v4l2_standard standard;
> +	struct v4l2_mbus_framefmt format;
> +};
> +
> +/**
> + * Supported standards -
> + *
> + * Currently supports two standards only, need to add support for rest of the
> + * modes, like SECAM, etc...
> + */
> +static struct tvp515x_std_info tvp515x_std_list[] = {
> +	/* Standard: STD_NTSC_MJ */
> +	/* Standard: STD_PAL_BDGHIN */
> +	[STD_PAL_BDGHIN] = {
> +		.video_std = VIDEO_STD_PAL_BDGHIN_BIT,
> +		.standard = {
> +			.index = 1,
> +			.id = V4L2_STD_PAL,
> +			.name = "PAL",
> +			.frameperiod = {1, 25},
> +			.framelines = 625
> +		},
> +		.format = {
> +			.width = PAL_NUM_ACTIVE_PIXELS,
> +			.height = PAL_NUM_ACTIVE_LINES,
> +			.code = V4L2_MBUS_FMT_UYVY8_2X8,
> +			.field = V4L2_FIELD_INTERLACED,
> +			.colorspace = V4L2_COLORSPACE_SMPTE170M,
> +		},
> +	},
> +	[STD_NTSC_MJ] = {
> +		.video_std = VIDEO_STD_NTSC_MJ_BIT,
> +		.standard = {
> +			.index = 0,
> +			.id = V4L2_STD_NTSC,
> +			.name = "NTSC",
> +			.frameperiod = {1001, 30000},
> +			.framelines = 525
> +		},
> +		.format = {
> +			.width = NTSC_NUM_ACTIVE_PIXELS,
> +			.height = NTSC_NUM_ACTIVE_LINES,
> +			.code = V4L2_MBUS_FMT_UYVY8_2X8,
> +			.field = V4L2_FIELD_INTERLACED,
> +			.colorspace = V4L2_COLORSPACE_SMPTE170M,
> +		},
> +	},
> +	/* Standard: need to add for additional standard */
> +};
> +
>  struct tvp5150 {
>  	struct v4l2_subdev sd;
>  	struct v4l2_ctrl_handler hdl;
> -
> -	v4l2_std_id norm;	/* Current set standard */
> +	struct media_pad pad;
> +	struct v4l2_mbus_framefmt *format;
> +	v4l2_std_id std_idx;
> +	int norm;
>  	u32 input;
>  	u32 output;
>  	int enable;
> @@ -692,6 +761,45 @@ static int tvp5150_get_vbi(struct v4l2_subdev *sd,
>  	return type;
>  }
>  
> +/**
> + * tvp515x_query_current_std() : Query the current standard detected by TVP5151
> + * @sd: ptr to v4l2_subdev struct
> + *
> + * Returns the current standard detected by TVP5151, STD_INVALID if there is no
> + * standard detected.
> + */
> +static int tvp515x_query_current_std(struct v4l2_subdev *sd)
> +{
> +	u8 std, std_status;
> +
> +	std = tvp5150_read(sd, TVP5150_VIDEO_STD);
> +	if ((std & VIDEO_STD_MASK) == VIDEO_STD_AUTO_SWITCH_BIT)
> +		/* use the standard status register */
> +		std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
> +	else
> +		/* use the standard register itself */
> +		std_status = std;

Braces would be nice here.

> +	switch (std_status & VIDEO_STD_MASK) {
> +	case VIDEO_STD_NTSC_MJ_BIT:
> +	case VIDEO_STD_NTSC_MJ_BIT_AS:
> +		return STD_NTSC_MJ;
> +
> +	case VIDEO_STD_PAL_BDGHIN_BIT:
> +	case VIDEO_STD_PAL_BDGHIN_BIT_AS:
> +		return STD_PAL_BDGHIN;
> +
> +	default:
> +		return STD_INVALID;
> +	}
> +
> +	return STD_INVALID;

This return won't do anything.

> +}
> +
> +/****************************************************************************
> +			V4L2 subdev video operations
> + ****************************************************************************/
> +
>  static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
>  {
>  	struct tvp5150 *decoder = to_tvp5150(sd);
> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
>  	if (std == V4L2_STD_ALL) {
>  		fmt = 0;	/* Autodetect mode */
>  	} else if (std & V4L2_STD_NTSC_443) {
> -		fmt = 0xa;
> +		fmt = VIDEO_STD_NTSC_4_43_BIT;
>  	} else if (std & V4L2_STD_PAL_M) {
> -		fmt = 0x6;
> +		fmt = VIDEO_STD_PAL_M_BIT;
>  	} else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
> -		fmt = 0x8;
> +		fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
>  	} else {
>  		/* Then, test against generic ones */
>  		if (std & V4L2_STD_NTSC)
> -			fmt = 0x2;
> +			fmt = VIDEO_STD_NTSC_MJ_BIT;
>  		else if (std & V4L2_STD_PAL)
> -			fmt = 0x4;
> +			fmt = VIDEO_STD_PAL_BDGHIN_BIT;
>  		else if (std & V4L2_STD_SECAM)
> -			fmt = 0xc;
> +			fmt = VIDEO_STD_SECAM_BIT;
>  	}

Excellent! Less magic numbers...

>  	v4l2_dbg(1, debug, sd, "Set video std register to %d.\n", fmt);
> @@ -727,11 +835,26 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
>  static int tvp5150_s_std(struct v4l2_subdev *sd, v4l2_std_id std)
>  {
>  	struct tvp5150 *decoder = to_tvp5150(sd);
> +	int i;
> +	int num_stds = ARRAY_SIZE(tvp515x_std_list);
>  
>  	if (decoder->norm == std)
>  		return 0;
>  
> -	return tvp5150_set_std(sd, std);
> +	for (i = 0; i < num_stds; i++)
> +		if (std & tvp515x_std_list[i].standard.id)
> +			break;
> +
> +	if ((i == num_stds) || (i == STD_INVALID))
> +		return -EINVAL;
> +
> +	tvp5150_write(sd, TVP5150_VIDEO_STD, tvp515x_std_list[i].video_std);
> +
> +	decoder->norm = i;
> +	decoder->norm = std;
> +
> +/*	return tvp5150_set_std(sd, std); */
> +	return 0;
>  }
>  
>  static int tvp5150_reset(struct v4l2_subdev *sd, u32 val)
> @@ -778,6 +901,177 @@ static int tvp5150_s_ctrl(struct v4l2_ctrl *ctrl)
>  	return -EINVAL;
>  }
>  
> +static struct v4l2_mbus_framefmt *
> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
> +			 unsigned int pad, enum v4l2_subdev_format_whence which)
> +{
> +	switch (which) {
> +	case V4L2_SUBDEV_FORMAT_TRY:
> +		return v4l2_subdev_get_try_format(fh, pad);
> +	case V4L2_SUBDEV_FORMAT_ACTIVE:
> +		return tvp5150->format;
> +	default:
> +		return NULL;

Hmm. This will never happen, but is returning NULL the right thing to
do? An easy alternative is to just replace this with if (which may only
have either of the two values).

> +	}
> +}
> +
> +static int tvp5150_get_pad_format(struct v4l2_subdev *subdev,
> +			      struct v4l2_subdev_fh *fh,
> +			      struct v4l2_subdev_format *format)
> +{
> +	struct tvp5150 *tvp5150 = to_tvp5150(subdev);
> +
> +	format->format = *__tvp5150_get_pad_format(tvp5150, fh, format->pad,
> +						   format->which);
> +
> +	return 0;
> +}
> +
> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
> +			      struct v4l2_subdev_fh *fh,
> +			      struct v4l2_subdev_format *format)
> +{
> +	struct tvp5150 *tvp5150 = to_tvp5150(subdev);
> +	tvp5150->std_idx = STD_INVALID;

The above assignment will always be overwritten immediately.

> +	tvp5150->std_idx = tvp515x_query_current_std(subdev);
> +	if (tvp5150->std_idx == STD_INVALID) {
> +		v4l2_err(subdev, "Unable to query std\n");
> +		return 0;

Isn't this an error?

> +	}
> +
> +	tvp5150->norm = tvp515x_std_list[tvp5150->std_idx].standard.id;
> +
> +	tvp5150->format = &tvp515x_std_list[tvp5150->std_idx].format;
> +
> +	format->format = *__tvp5150_get_pad_format(tvp5150, fh, format->pad,
> +	format->which);
> +
> +	v4l2_info(subdev, "code=x%x width=%u height=%u colorspace=0x%x\n",
> +			format->format.code, format->format.width,
> +			format->format.height, format->format.colorspace);
> +
> +	return 0;
> +}
> +
> +/**
> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt

The name of the function is different.

> + * @sd: pointer to standard V4L2 sub-device structure
> + * @f: pointer to the mediabus format structure
> + *
> + * Negotiates the image capture size and mediabus format.
> + */
> +static int
> +tvp515x_mbus_fmt(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt *f)
> +{
> +	struct tvp5150 *decoder = to_tvp5150(sd);
> +
> +	if (f == NULL)
> +		return -EINVAL;
> +
> +	f = decoder->format;
> +
> +	v4l2_dbg(1, debug, sd, "MBUS_FMT: Width - %d, Height - %d\n",
> +			f->width, f->height);
> +	return 0;
> +}
> +
> +/*
> + * tvp515x_s_stream() - V4L2 decoder i/f handler for s_stream
> + * @sd: pointer to standard V4L2 sub-device structure
> + * @enable: streaming enable or disable
> + *
> + * Sets streaming to enable or disable, if possible.
> + */
> +static int tvp515x_s_stream(struct v4l2_subdev *subdev, int enable)
> +{
> +
> +	/* Initializes TVP5150 to its default values */
> +	/* # set PCLK (27MHz) */
> +	tvp5150_write(subdev, TVP5150_CONF_SHARED_PIN, 0x00);
> +
> +	/* Output format: 8-bit ITU-R BT.656 with embedded syncs */
> +	if (enable)
> +		tvp5150_write(subdev, TVP5150_MISC_CTL, 0x09);
> +	else
> +		tvp5150_write(subdev, TVP5150_MISC_CTL, 0x00);
> +
> +	return 0;
> +}
> +
> +
> +/**
> + * tvp515x_enum_mbus_fmt() - V4L2 decoder interface handler for enum_mbus_fmt
> + * @sd: pointer to standard V4L2 sub-device structure
> + * @index: index of pixelcode to retrieve
> + * @code: receives the pixelcode
> + *
> + * Enumerates supported mediabus formats
> + */
> +static int
> +tvp515x_enum_mbus_fmt(struct v4l2_subdev *sd, unsigned index,
> +					enum v4l2_mbus_pixelcode *code)
> +{
> +	if (index)
> +		return -EINVAL;
> +
> +	*code = V4L2_MBUS_FMT_UYVY8_2X8;
> +	return 0;
> +}
> +
> +/**
> + * tvp515x_g_parm() - V4L2 decoder interface handler for g_parm
> + * @sd: pointer to standard V4L2 sub-device structure
> + * @a: pointer to standard V4L2 VIDIOC_G_PARM ioctl structure
> + *
> + * Returns the decoder's video CAPTURE parameters.
> + */
> +static int
> +tvp515x_g_parm(struct v4l2_subdev *sd, struct v4l2_streamparm *a)
> +{
> +	struct v4l2_captureparm *cparm;
> +
> +	if (a == NULL)
> +		return -EINVAL;
> +
> +	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> +		/* only capture is supported */
> +		return -EINVAL;
> +
> +	cparm = &a->parm.capture;
> +	cparm->capability = V4L2_CAP_TIMEPERFRAME;
> +	cparm->timeperframe = a->parm.capture.timeperframe;
> +
> +	return 0;
> +}
> +
> +/**
> + * tvp515x_s_parm() - V4L2 decoder interface handler for s_parm
> + * @sd: pointer to standard V4L2 sub-device structure
> + * @a: pointer to standard V4L2 VIDIOC_S_PARM ioctl structure
> + *
> + * Configures the decoder to use the input parameters, if possible. If
> + * not possible, returns the appropriate error code.
> + */
> +static int
> +tvp515x_s_parm(struct v4l2_subdev *sd, struct v4l2_streamparm *a)
> +{
> +	struct v4l2_fract *timeperframe;
> +
> +	if (a == NULL)
> +		return -EINVAL;
> +
> +	if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
> +		/* only capture is supported */
> +		return -EINVAL;
> +
> +	timeperframe = &a->parm.capture.timeperframe;
> +
> +	return 0;
> +}
> +
> +
> +
>  /****************************************************************************
>  			I2C Command
>   ****************************************************************************/
> @@ -900,7 +1194,28 @@ static int tvp5150_g_tuner(struct v4l2_subdev *sd, struct v4l2_tuner *vt)
>  	return 0;
>  }
>  
> -/* ----------------------------------------------------------------------- */
> +/****************************************************************************
> +		    V4L2 subdev core operations
> + ****************************************************************************/
> +
> +static int tvp5150_open(struct v4l2_subdev *subdev, struct v4l2_subdev_fh *fh)
> +{
> +	struct tvp5150 *decoder = to_tvp5150(subdev);
> +
> +	decoder->std_idx = STD_INVALID;
> +
> +	decoder->std_idx = tvp515x_query_current_std(subdev);
> +
> +	if (decoder->std_idx == STD_INVALID) {
> +		v4l2_err(subdev, "Unable to query std\n");
> +		return 0;
> +	}
> +
> +	decoder->format = (&(tvp515x_std_list[decoder->std_idx].format));
> +	decoder->norm = tvp515x_std_list[decoder->std_idx].standard.id;
> +
> +	return 0;
> +}
>  
>  static const struct v4l2_ctrl_ops tvp5150_ctrl_ops = {
>  	.s_ctrl = tvp5150_s_ctrl,
> @@ -924,12 +1239,24 @@ static const struct v4l2_subdev_core_ops tvp5150_core_ops = {
>  #endif
>  };
>  
> +static struct v4l2_subdev_file_ops tvp5150_subdev_file_ops = {
> +	.open		= tvp5150_open,
> +};
> +
>  static const struct v4l2_subdev_tuner_ops tvp5150_tuner_ops = {
>  	.g_tuner = tvp5150_g_tuner,
>  };
>  
>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>  	.s_routing = tvp5150_s_routing,
> +	.s_stream = tvp515x_s_stream,
> +	.enum_mbus_fmt = tvp515x_enum_mbus_fmt,
> +	.g_mbus_fmt = tvp515x_mbus_fmt,
> +	.try_mbus_fmt = tvp515x_mbus_fmt,
> +	.s_mbus_fmt = tvp515x_mbus_fmt,
> +	.g_parm = tvp515x_g_parm,
> +	.s_parm = tvp515x_s_parm,
> +	.s_std_output = tvp5150_s_std,

Do we really need both video and pad format ops?

s_std should be added to pad ops so it would be available on the subdev
node.

>  };
>  
>  static const struct v4l2_subdev_vbi_ops tvp5150_vbi_ops = {
> @@ -939,14 +1266,57 @@ static const struct v4l2_subdev_vbi_ops tvp5150_vbi_ops = {
>  	.s_raw_fmt = tvp5150_s_raw_fmt,
>  };
>  
> +static int tvp515x_enum_mbus_code(struct v4l2_subdev *subdev,
> +				  struct v4l2_subdev_fh *fh,
> +				  struct v4l2_subdev_mbus_code_enum *code)
> +{
> +	if (code->index >= ARRAY_SIZE(tvp515x_std_list))
> +		return -EINVAL;
> +
> +	code->code = V4L2_MBUS_FMT_UYVY8_2X8;

If there's just one supported mbus code, non-zero code->index must
return -EINVAL.

> +	return 0;
> +}
> +
> +static int tvp515x_enum_frame_size(struct v4l2_subdev *subdev,
> +				   struct v4l2_subdev_fh *fh,
> +				   struct v4l2_subdev_frame_size_enum *fse)
> +{
> +	int current_std = STD_INVALID;

current_std is overwritten before it gets used.

> +	if (fse->code != V4L2_MBUS_FMT_UYVY8_2X8)
> +		return -EINVAL;
> +
> +	/* query the current standard */
> +	current_std = tvp515x_query_current_std(subdev);
> +	if (current_std == STD_INVALID) {
> +		v4l2_err(subdev, "Unable to query std\n");
> +		return 0;
> +	}

I wonder how the enum_frame_size and s_std are supposed to interact,
especially that I understand, after reading the discussion, the chip may
be used to force certain standard while the actual signal is different.

> +	fse->min_width = tvp515x_std_list[current_std].format.width;
> +	fse->min_height = tvp515x_std_list[current_std].format.height;
> +	fse->max_width = fse->min_width;
> +	fse->max_height = fse->min_height;
> +	return 0;
> +}
> +
> +static struct v4l2_subdev_pad_ops tvp5150_pad_ops = {
> +	.enum_mbus_code = tvp515x_enum_mbus_code,
> +	.enum_frame_size = tvp515x_enum_frame_size,
> +	.get_fmt = tvp5150_get_pad_format,
> +	.set_fmt = tvp5150_set_pad_format,
> +};
> +
>  static const struct v4l2_subdev_ops tvp5150_ops = {
>  	.core = &tvp5150_core_ops,
> +	.file	= &tvp5150_subdev_file_ops,
>  	.tuner = &tvp5150_tuner_ops,
>  	.video = &tvp5150_video_ops,
>  	.vbi = &tvp5150_vbi_ops,
> +	.pad = &tvp5150_pad_ops,
>  };
>  
> -
>  /****************************************************************************
>  			I2C Client & Driver
>   ****************************************************************************/
> @@ -957,6 +1327,7 @@ static int tvp5150_probe(struct i2c_client *c,
>  	struct tvp5150 *core;
>  	struct v4l2_subdev *sd;
>  	u8 msb_id, lsb_id, msb_rom, lsb_rom;
> +	int ret;
>  
>  	/* Check if the adapter supports the needed features */
>  	if (!i2c_check_functionality(c->adapter,
> @@ -992,6 +1363,7 @@ static int tvp5150_probe(struct i2c_client *c,
>  		}
>  	}
>  
> +	core->sd.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
>  	core->norm = V4L2_STD_ALL;	/* Default is autodetect */
>  	core->input = TVP5150_COMPOSITE1;
>  	core->enable = 1;
> @@ -1017,6 +1389,12 @@ static int tvp5150_probe(struct i2c_client *c,
>  
>  	if (debug > 1)
>  		tvp5150_log_status(sd);
> +
> +	core->pad.flags = MEDIA_PAD_FLAG_OUTPUT;
> +	ret = media_entity_init(&core->sd.entity, 1, &core->pad, 0);
> +	if (ret < 0)
> +		kfree(core);
> +
>  	return 0;
>  }
>  

-- 
Sakari Ailus
sakari.ailus@iki.fi

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus <sakari.ailus@iki.fi> wrote:
> Hi Javier,
>
> Thanks for the patch! It's very interesting to see a driver for a video
> decoder using the MC interface. Before this we've had just image sensors.
>

Hello Sakari,

Thanks for your comments.

> Javier Martinez Canillas wrote:
>> +             /* use the standard status register */
>> +             std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
>> +     else
>> +             /* use the standard register itself */
>> +             std_status = std;
>
> Braces would be nice here.
>

Ok.

>> +     switch (std_status & VIDEO_STD_MASK) {
>> +     case VIDEO_STD_NTSC_MJ_BIT:
>> +     case VIDEO_STD_NTSC_MJ_BIT_AS:
>> +             return STD_NTSC_MJ;
>> +
>> +     case VIDEO_STD_PAL_BDGHIN_BIT:
>> +     case VIDEO_STD_PAL_BDGHIN_BIT_AS:
>> +             return STD_PAL_BDGHIN;
>> +
>> +     default:
>> +             return STD_INVALID;
>> +     }
>> +
>> +     return STD_INVALID;
>
> This return won't do anything.
>

Yes, will clean this.

>> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
>>       if (std == V4L2_STD_ALL) {
>>               fmt = 0;        /* Autodetect mode */
>>       } else if (std & V4L2_STD_NTSC_443) {
>> -             fmt = 0xa;
>> +             fmt = VIDEO_STD_NTSC_4_43_BIT;
>>       } else if (std & V4L2_STD_PAL_M) {
>> -             fmt = 0x6;
>> +             fmt = VIDEO_STD_PAL_M_BIT;
>>       } else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
>> -             fmt = 0x8;
>> +             fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
>>       } else {
>>               /* Then, test against generic ones */
>>               if (std & V4L2_STD_NTSC)
>> -                     fmt = 0x2;
>> +                     fmt = VIDEO_STD_NTSC_MJ_BIT;
>>               else if (std & V4L2_STD_PAL)
>> -                     fmt = 0x4;
>> +                     fmt = VIDEO_STD_PAL_BDGHIN_BIT;
>>               else if (std & V4L2_STD_SECAM)
>> -                     fmt = 0xc;
>> +                     fmt = VIDEO_STD_SECAM_BIT;
>>       }
>
> Excellent! Less magic numbers...
>
>>
>> +static struct v4l2_mbus_framefmt *
>> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
>> +                      unsigned int pad, enum v4l2_subdev_format_whence which)
>> +{
>> +     switch (which) {
>> +     case V4L2_SUBDEV_FORMAT_TRY:
>> +             return v4l2_subdev_get_try_format(fh, pad);
>> +     case V4L2_SUBDEV_FORMAT_ACTIVE:
>> +             return tvp5150->format;
>> +     default:
>> +             return NULL;
>
> Hmm. This will never happen, but is returning NULL the right thing to
> do? An easy alternative is to just replace this with if (which may only
> have either of the two values).
>

Ok I'll cleanup, I was being a bit paranoid there :)

>> +
>> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
>> +                           struct v4l2_subdev_fh *fh,
>> +                           struct v4l2_subdev_format *format)
>> +{
>> +     struct tvp5150 *tvp5150 = to_tvp5150(subdev);
>> +     tvp5150->std_idx = STD_INVALID;
>
> The above assignment will always be overwritten immediately.
>

Yes, since tvp515x_query_current_std() already returns STD_INVALID on
error the assignment is not needed. Will change that.

>> +     tvp5150->std_idx = tvp515x_query_current_std(subdev);
>> +     if (tvp5150->std_idx == STD_INVALID) {
>> +             v4l2_err(subdev, "Unable to query std\n");
>> +             return 0;
>
> Isn't this an error?
>

Yes, I'll change to report the error to the caller.

>> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
>
> The name of the function is different.
>

Yes, I'll change that.

>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>>       .s_routing = tvp5150_s_routing,
>> +     .s_stream = tvp515x_s_stream,
>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>> +     .g_parm = tvp515x_g_parm,
>> +     .s_parm = tvp515x_s_parm,
>> +     .s_std_output = tvp5150_s_std,
>
> Do we really need both video and pad format ops?
>

Good question, I don't know. Can this device be used as a standalone
v4l2 device? Or is supposed to always be a part of a video streaming
pipeline as a sub-device with a source pad? Sorry if my questions are
silly but as I stated before, I'm a newbie with v4l2 and MCF.

> s_std should be added to pad ops so it would be available on the subdev
> node.
>

Ok, I'll add s_std operation.

>>
>> +static int tvp515x_enum_mbus_code(struct v4l2_subdev *subdev,
>> +                               struct v4l2_subdev_fh *fh,
>> +                               struct v4l2_subdev_mbus_code_enum *code)
>> +{
>> +     if (code->index >= ARRAY_SIZE(tvp515x_std_list))
>> +             return -EINVAL;
>> +
>> +     code->code = V4L2_MBUS_FMT_UYVY8_2X8;
>
> If there's just one supported mbus code, non-zero code->index must
> return -EINVAL.
>

Ok, I'll change that.

>> +     return 0;
>> +}
>> +
>> +static int tvp515x_enum_frame_size(struct v4l2_subdev *subdev,
>> +                                struct v4l2_subdev_fh *fh,
>> +                                struct v4l2_subdev_frame_size_enum *fse)
>> +{
>> +     int current_std = STD_INVALID;
>
> current_std is overwritten before it gets used.
>

Yes, same case here, will remove the assignment.

>> +     if (fse->code != V4L2_MBUS_FMT_UYVY8_2X8)
>> +             return -EINVAL;
>> +
>> +     /* query the current standard */
>> +     current_std = tvp515x_query_current_std(subdev);
>> +     if (current_std == STD_INVALID) {
>> +             v4l2_err(subdev, "Unable to query std\n");
>> +             return 0;
>> +     }
>
> I wonder how the enum_frame_size and s_std are supposed to interact,
> especially that I understand, after reading the discussion, the chip may
> be used to force certain standard while the actual signal is different.
>

Well my thought was that the application can select which standard
(i.e: V4L2_STD_PAL) and enum_frame_size will return the width and
height for that standard (i.e: 720x625 for PAL). Since that is what
the device is capturing.

Then a user-space application can configure the CCDC and RESIZER to
modify the format. Does this make sense to you? Or the user-space
application can select a different frame size at the sub-dev level
(tvp5151)?

>
> --
> Sakari Ailus
> sakari.ailus@iki.fi
>

Thanks a lot for your suggestions and comments.

Best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
> On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus <sakari.ailus@iki.fi> wrote:
>> Hi Javier,
>>
>> Thanks for the patch! It's very interesting to see a driver for a video
>> decoder using the MC interface. Before this we've had just image sensors.
>>
> 
> Hello Sakari,
> 
> Thanks for your comments.
> 
>> Javier Martinez Canillas wrote:
>>> +             /* use the standard status register */
>>> +             std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
>>> +     else
>>> +             /* use the standard register itself */
>>> +             std_status = std;
>>
>> Braces would be nice here.
>>
> 
> Ok.
> 
>>> +     switch (std_status & VIDEO_STD_MASK) {
>>> +     case VIDEO_STD_NTSC_MJ_BIT:
>>> +     case VIDEO_STD_NTSC_MJ_BIT_AS:
>>> +             return STD_NTSC_MJ;
>>> +
>>> +     case VIDEO_STD_PAL_BDGHIN_BIT:
>>> +     case VIDEO_STD_PAL_BDGHIN_BIT_AS:
>>> +             return STD_PAL_BDGHIN;
>>> +
>>> +     default:
>>> +             return STD_INVALID;
>>> +     }
>>> +
>>> +     return STD_INVALID;
>>
>> This return won't do anything.
>>
> 
> Yes, will clean this.
> 
>>> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
>>>       if (std == V4L2_STD_ALL) {
>>>               fmt = 0;        /* Autodetect mode */
>>>       } else if (std & V4L2_STD_NTSC_443) {
>>> -             fmt = 0xa;
>>> +             fmt = VIDEO_STD_NTSC_4_43_BIT;
>>>       } else if (std & V4L2_STD_PAL_M) {
>>> -             fmt = 0x6;
>>> +             fmt = VIDEO_STD_PAL_M_BIT;
>>>       } else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
>>> -             fmt = 0x8;
>>> +             fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
>>>       } else {
>>>               /* Then, test against generic ones */
>>>               if (std & V4L2_STD_NTSC)
>>> -                     fmt = 0x2;
>>> +                     fmt = VIDEO_STD_NTSC_MJ_BIT;
>>>               else if (std & V4L2_STD_PAL)
>>> -                     fmt = 0x4;
>>> +                     fmt = VIDEO_STD_PAL_BDGHIN_BIT;
>>>               else if (std & V4L2_STD_SECAM)
>>> -                     fmt = 0xc;
>>> +                     fmt = VIDEO_STD_SECAM_BIT;
>>>       }
>>
>> Excellent! Less magic numbers...
>>
>>>
>>> +static struct v4l2_mbus_framefmt *
>>> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
>>> +                      unsigned int pad, enum v4l2_subdev_format_whence which)
>>> +{
>>> +     switch (which) {
>>> +     case V4L2_SUBDEV_FORMAT_TRY:
>>> +             return v4l2_subdev_get_try_format(fh, pad);
>>> +     case V4L2_SUBDEV_FORMAT_ACTIVE:
>>> +             return tvp5150->format;
>>> +     default:
>>> +             return NULL;
>>
>> Hmm. This will never happen, but is returning NULL the right thing to
>> do? An easy alternative is to just replace this with if (which may only
>> have either of the two values).
>>
> 
> Ok I'll cleanup, I was being a bit paranoid there :)
> 
>>> +
>>> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
>>> +                           struct v4l2_subdev_fh *fh,
>>> +                           struct v4l2_subdev_format *format)
>>> +{
>>> +     struct tvp5150 *tvp5150 = to_tvp5150(subdev);
>>> +     tvp5150->std_idx = STD_INVALID;
>>
>> The above assignment will always be overwritten immediately.
>>
> 
> Yes, since tvp515x_query_current_std() already returns STD_INVALID on
> error the assignment is not needed. Will change that.
> 
>>> +     tvp5150->std_idx = tvp515x_query_current_std(subdev);
>>> +     if (tvp5150->std_idx == STD_INVALID) {
>>> +             v4l2_err(subdev, "Unable to query std\n");
>>> +             return 0;
>>
>> Isn't this an error?
>>
> 
> Yes, I'll change to report the error to the caller.
> 
>>> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
>>
>> The name of the function is different.
>>
> 
> Yes, I'll change that.
> 
>>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>>>       .s_routing = tvp5150_s_routing,
>>> +     .s_stream = tvp515x_s_stream,
>>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>>> +     .g_parm = tvp515x_g_parm,
>>> +     .s_parm = tvp515x_s_parm,
>>> +     .s_std_output = tvp5150_s_std,
>>
>> Do we really need both video and pad format ops?
>>
> 
> Good question, I don't know. Can this device be used as a standalone
> v4l2 device? Or is supposed to always be a part of a video streaming
> pipeline as a sub-device with a source pad? Sorry if my questions are
> silly but as I stated before, I'm a newbie with v4l2 and MCF.

The tvp5150 driver is used on some em28xx devices. It is nice to add auto-detection
code to the driver, but converting it to the media bus should be done with
enough care to not break support for the existing devices.

Also, as I've argued with Laurent before, the expected behavior is that the standards
format selection should be done via the video node, and not via the media 
controller node. The V4L2 API has enough support for all you need to do with the
video decoder, so there's no excuse to duplicate it with any other API.
The media controller API is there not to replace V4L2, but to complement
it where needed.

In the specific code of standards auto-detection, a few drivers currently support
this feature. They're (or should be) coded to do is:

If V4L2_STD_ALL is used, the driver should autodetect the video standard of the
currently tuned channel.

The detected standard can be returned to userspace via VIDIOC_G_STD.

If otherwise, another standard mask is sent to the driver via VIDIOC_S_STD,
the expected behavior is that the driver should select the standards detector
to conform with the desired mask. If an unsupported configuration is requested,
the driver should return the mask it actually used at the return of VIDIOC_S_STD
call.

For example, if V4L2_STD_NTSC_M_JP is used, the driver should disable the
auto-detector, and use NTSC/M with the Japanese audio standard. both S_STD
and G_STD will return V4L2_STD_NTSC_M_JP.
If V4L2_STD_MN is used and the driver can auto-detect between all those formats, 
the driver should detect if the standard is PAL or NTSC and detect between 
STD/M or STD/M (and the corresponding audio standards).

If an unsupported mask like V4L2_STD_PAL_J | V4L2_STD_NTSC_M_JP is used, the driver
should return a valid combination to S_STD (for example, returning V4L2_STD_PAL_J.

In any case, on V4L2_G_STD, if the driver can't detect what's the standard, it should
just return the current detection mask to userspace (instead of returning something 
like STD_INVALID).

I hope that helps,
Mauro.

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Hans Verkuil]

On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
> > On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus <sakari.ailus@iki.fi> wrote:
> >> Hi Javier,
> >>
> >> Thanks for the patch! It's very interesting to see a driver for a video
> >> decoder using the MC interface. Before this we've had just image sensors.
> >>
> > 
> > Hello Sakari,
> > 
> > Thanks for your comments.
> > 
> >> Javier Martinez Canillas wrote:
> >>> +             /* use the standard status register */
> >>> +             std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
> >>> +     else
> >>> +             /* use the standard register itself */
> >>> +             std_status = std;
> >>
> >> Braces would be nice here.
> >>
> > 
> > Ok.
> > 
> >>> +     switch (std_status & VIDEO_STD_MASK) {
> >>> +     case VIDEO_STD_NTSC_MJ_BIT:
> >>> +     case VIDEO_STD_NTSC_MJ_BIT_AS:
> >>> +             return STD_NTSC_MJ;
> >>> +
> >>> +     case VIDEO_STD_PAL_BDGHIN_BIT:
> >>> +     case VIDEO_STD_PAL_BDGHIN_BIT_AS:
> >>> +             return STD_PAL_BDGHIN;
> >>> +
> >>> +     default:
> >>> +             return STD_INVALID;
> >>> +     }
> >>> +
> >>> +     return STD_INVALID;
> >>
> >> This return won't do anything.
> >>
> > 
> > Yes, will clean this.
> > 
> >>> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
> >>>       if (std == V4L2_STD_ALL) {
> >>>               fmt = 0;        /* Autodetect mode */
> >>>       } else if (std & V4L2_STD_NTSC_443) {
> >>> -             fmt = 0xa;
> >>> +             fmt = VIDEO_STD_NTSC_4_43_BIT;
> >>>       } else if (std & V4L2_STD_PAL_M) {
> >>> -             fmt = 0x6;
> >>> +             fmt = VIDEO_STD_PAL_M_BIT;
> >>>       } else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
> >>> -             fmt = 0x8;
> >>> +             fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
> >>>       } else {
> >>>               /* Then, test against generic ones */
> >>>               if (std & V4L2_STD_NTSC)
> >>> -                     fmt = 0x2;
> >>> +                     fmt = VIDEO_STD_NTSC_MJ_BIT;
> >>>               else if (std & V4L2_STD_PAL)
> >>> -                     fmt = 0x4;
> >>> +                     fmt = VIDEO_STD_PAL_BDGHIN_BIT;
> >>>               else if (std & V4L2_STD_SECAM)
> >>> -                     fmt = 0xc;
> >>> +                     fmt = VIDEO_STD_SECAM_BIT;
> >>>       }
> >>
> >> Excellent! Less magic numbers...
> >>
> >>>
> >>> +static struct v4l2_mbus_framefmt *
> >>> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
> >>> +                      unsigned int pad, enum v4l2_subdev_format_whence which)
> >>> +{
> >>> +     switch (which) {
> >>> +     case V4L2_SUBDEV_FORMAT_TRY:
> >>> +             return v4l2_subdev_get_try_format(fh, pad);
> >>> +     case V4L2_SUBDEV_FORMAT_ACTIVE:
> >>> +             return tvp5150->format;
> >>> +     default:
> >>> +             return NULL;
> >>
> >> Hmm. This will never happen, but is returning NULL the right thing to
> >> do? An easy alternative is to just replace this with if (which may only
> >> have either of the two values).
> >>
> > 
> > Ok I'll cleanup, I was being a bit paranoid there :)
> > 
> >>> +
> >>> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
> >>> +                           struct v4l2_subdev_fh *fh,
> >>> +                           struct v4l2_subdev_format *format)
> >>> +{
> >>> +     struct tvp5150 *tvp5150 = to_tvp5150(subdev);
> >>> +     tvp5150->std_idx = STD_INVALID;
> >>
> >> The above assignment will always be overwritten immediately.
> >>
> > 
> > Yes, since tvp515x_query_current_std() already returns STD_INVALID on
> > error the assignment is not needed. Will change that.
> > 
> >>> +     tvp5150->std_idx = tvp515x_query_current_std(subdev);
> >>> +     if (tvp5150->std_idx == STD_INVALID) {
> >>> +             v4l2_err(subdev, "Unable to query std\n");
> >>> +             return 0;
> >>
> >> Isn't this an error?
> >>
> > 
> > Yes, I'll change to report the error to the caller.
> > 
> >>> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
> >>
> >> The name of the function is different.
> >>
> > 
> > Yes, I'll change that.
> > 
> >>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
> >>>       .s_routing = tvp5150_s_routing,
> >>> +     .s_stream = tvp515x_s_stream,
> >>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
> >>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
> >>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
> >>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
> >>> +     .g_parm = tvp515x_g_parm,
> >>> +     .s_parm = tvp515x_s_parm,
> >>> +     .s_std_output = tvp5150_s_std,
> >>
> >> Do we really need both video and pad format ops?
> >>
> > 
> > Good question, I don't know. Can this device be used as a standalone
> > v4l2 device? Or is supposed to always be a part of a video streaming
> > pipeline as a sub-device with a source pad? Sorry if my questions are
> > silly but as I stated before, I'm a newbie with v4l2 and MCF.
> 
> The tvp5150 driver is used on some em28xx devices. It is nice to add auto-detection
> code to the driver, but converting it to the media bus should be done with
> enough care to not break support for the existing devices.

So in other words, the tvp5150 driver needs both pad and non-pad ops.
Eventually all non-pad variants in subdev drivers should be replaced by the
pad variants so you don't have duplication of ops. But that will take a lot
more work.

> Also, as I've argued with Laurent before, the expected behavior is that the standards
> format selection should be done via the video node, and not via the media 
> controller node. The V4L2 API has enough support for all you need to do with the
> video decoder, so there's no excuse to duplicate it with any other API.

This is relevant for bridge drivers, not for subdev drivers.

> The media controller API is there not to replace V4L2, but to complement
> it where needed.

That will be a nice discussion during the workshop :-)

> In the specific code of standards auto-detection, a few drivers currently support
> this feature. They're (or should be) coded to do is:
> 
> If V4L2_STD_ALL is used, the driver should autodetect the video standard of the
> currently tuned channel.

Actually, this is optional. As per the spec:

"When the standard set is ambiguous drivers may return EINVAL or choose any of
the requested standards."

Nor does the spec say anything about doing an autodetect when STD_ALL is passed
in. Most drivers will just set the std to PAL or NTSC in this case.

If you want to autodetect, then use QUERYSTD. Applications cannot rely on drivers
to handle V4L2_STD_ALL the way you say.

> The detected standard can be returned to userspace via VIDIOC_G_STD.

No! G_STD always returns the current *selected* standard. Only QUERYSTD returns
the detected standard.

> 
> If otherwise, another standard mask is sent to the driver via VIDIOC_S_STD,
> the expected behavior is that the driver should select the standards detector
> to conform with the desired mask. If an unsupported configuration is requested,
> the driver should return the mask it actually used at the return of VIDIOC_S_STD
> call.

S_STD is a write-only ioctl, so the mask isn't updated.
 
> For example, if V4L2_STD_NTSC_M_JP is used, the driver should disable the
> auto-detector, and use NTSC/M with the Japanese audio standard. both S_STD
> and G_STD will return V4L2_STD_NTSC_M_JP.
> If V4L2_STD_MN is used and the driver can auto-detect between all those formats, 
> the driver should detect if the standard is PAL or NTSC and detect between 
> STD/M or STD/M (and the corresponding audio standards).
> 
> If an unsupported mask like V4L2_STD_PAL_J | V4L2_STD_NTSC_M_JP is used, the driver
> should return a valid combination to S_STD (for example, returning V4L2_STD_PAL_J.
> 
> In any case, on V4L2_G_STD, if the driver can't detect what's the standard, it should
> just return the current detection mask to userspace (instead of returning something 
> like STD_INVALID).

G_STD must always return the currently selected standard, never the detected standard.
That's QUERYSTD.

When the driver is first loaded it must pre-select a standard (usually in the probe
function), either hardcoded (NTSC or PAL), or by doing an initial autodetect. But
the standard should always be set to something. This allows you to start streaming
immediately.

Regards,

	Hans

> I hope that helps,
> Mauro.
> 

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Mon, Oct 3, 2011 at 8:30 AM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
>> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
>> >
>> > Yes, I'll change that.
>> >
>> >>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>> >>>       .s_routing = tvp5150_s_routing,
>> >>> +     .s_stream = tvp515x_s_stream,
>> >>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>> >>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>> >>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>> >>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>> >>> +     .g_parm = tvp515x_g_parm,
>> >>> +     .s_parm = tvp515x_s_parm,
>> >>> +     .s_std_output = tvp5150_s_std,
>> >>
>> >> Do we really need both video and pad format ops?
>> >>
>> >
>> > Good question, I don't know. Can this device be used as a standalone
>> > v4l2 device? Or is supposed to always be a part of a video streaming
>> > pipeline as a sub-device with a source pad? Sorry if my questions are
>> > silly but as I stated before, I'm a newbie with v4l2 and MCF.
>>
>> The tvp5150 driver is used on some em28xx devices. It is nice to add auto-detection
>> code to the driver, but converting it to the media bus should be done with
>> enough care to not break support for the existing devices.
>
> So in other words, the tvp5150 driver needs both pad and non-pad ops.
> Eventually all non-pad variants in subdev drivers should be replaced by the
> pad variants so you don't have duplication of ops. But that will take a lot
> more work.
>

Great, that was a doubt I had, thanks for the clarification.

>
>> In the specific code of standards auto-detection, a few drivers currently support
>> this feature. They're (or should be) coded to do is:
>>
>> If V4L2_STD_ALL is used, the driver should autodetect the video standard of the
>> currently tuned channel.
>
> Actually, this is optional. As per the spec:
>
> "When the standard set is ambiguous drivers may return EINVAL or choose any of
> the requested standards."
>
> Nor does the spec say anything about doing an autodetect when STD_ALL is passed
> in. Most drivers will just set the std to PAL or NTSC in this case.
>
> If you want to autodetect, then use QUERYSTD. Applications cannot rely on drivers
> to handle V4L2_STD_ALL the way you say.
>
>> The detected standard can be returned to userspace via VIDIOC_G_STD.
>
> No! G_STD always returns the current *selected* standard. Only QUERYSTD returns
> the detected standard.
>
>>
>> If otherwise, another standard mask is sent to the driver via VIDIOC_S_STD,
>> the expected behavior is that the driver should select the standards detector
>> to conform with the desired mask. If an unsupported configuration is requested,
>> the driver should return the mask it actually used at the return of VIDIOC_S_STD
>> call.
>
> S_STD is a write-only ioctl, so the mask isn't updated.
>
>> For example, if V4L2_STD_NTSC_M_JP is used, the driver should disable the
>> auto-detector, and use NTSC/M with the Japanese audio standard. both S_STD
>> and G_STD will return V4L2_STD_NTSC_M_JP.
>> If V4L2_STD_MN is used and the driver can auto-detect between all those formats,
>> the driver should detect if the standard is PAL or NTSC and detect between
>> STD/M or STD/M (and the corresponding audio standards).
>>
>> If an unsupported mask like V4L2_STD_PAL_J | V4L2_STD_NTSC_M_JP is used, the driver
>> should return a valid combination to S_STD (for example, returning V4L2_STD_PAL_J.
>>
>> In any case, on V4L2_G_STD, if the driver can't detect what's the standard, it should
>> just return the current detection mask to userspace (instead of returning something
>> like STD_INVALID).
>
> G_STD must always return the currently selected standard, never the detected standard.
> That's QUERYSTD.
>
> When the driver is first loaded it must pre-select a standard (usually in the probe
> function), either hardcoded (NTSC or PAL), or by doing an initial autodetect. But
> the standard should always be set to something. This allows you to start streaming
> immediately.
>
> Regards,
>
>        Hans
>
>> I hope that helps,
>> Mauro.
>>
>

Thanks Mauro and Hans for your comments.

I plan to work on the autodetect code and the issues called out by
Sakari and resubmit the patch, can you point me a driver that got
auto-detect the right way so I can use it as a reference?

Best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Hans,

On Monday 03 October 2011 08:30:25 Hans Verkuil wrote:
> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
> > Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
> > > On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus wrote:

[snip]

> > >>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
> > >>>  
> > >>>       .s_routing = tvp5150_s_routing,
> > >>> 
> > >>> +     .s_stream = tvp515x_s_stream,
> > >>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
> > >>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
> > >>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
> > >>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
> > >>> +     .g_parm = tvp515x_g_parm,
> > >>> +     .s_parm = tvp515x_s_parm,
> > >>> +     .s_std_output = tvp5150_s_std,
> > >> 
> > >> Do we really need both video and pad format ops?
> > > 
> > > Good question, I don't know. Can this device be used as a standalone
> > > v4l2 device? Or is supposed to always be a part of a video streaming
> > > pipeline as a sub-device with a source pad? Sorry if my questions are
> > > silly but as I stated before, I'm a newbie with v4l2 and MCF.
> > 
> > The tvp5150 driver is used on some em28xx devices. It is nice to add
> > auto-detection code to the driver, but converting it to the media bus
> > should be done with enough care to not break support for the existing
> > devices.
> 
> So in other words, the tvp5150 driver needs both pad and non-pad ops.
> Eventually all non-pad variants in subdev drivers should be replaced by the
> pad variants so you don't have duplication of ops. But that will take a lot
> more work.

What about replacing direct calls to non-pad operations with core V4L2 
functions that would use the subdev non-pad operation if available, and 
emulate if with the pad operation otherwise ? I think this would ease the 
transition, as subdev drivers could be ported to pad operations without 
worrying about the bridges that use them, and bridge drivers could be switched 
to the new wrappers with a simple search and replace.

> > Also, as I've argued with Laurent before, the expected behavior is that
> > the standards format selection should be done via the video node, and
> > not via the media controller node. The V4L2 API has enough support for
> > all you need to do with the video decoder, so there's no excuse to
> > duplicate it with any other API.
> 
> This is relevant for bridge drivers, not for subdev drivers.
> 
> > The media controller API is there not to replace V4L2, but to complement
> > it where needed.
> 
> That will be a nice discussion during the workshop :-)

I don't think we disagree on that, but we probably disagree on what it means 
:-)

> > In the specific code of standards auto-detection, a few drivers currently
> > support this feature. They're (or should be) coded to do is:
> > 
> > If V4L2_STD_ALL is used, the driver should autodetect the video standard
> > of the currently tuned channel.
> 
> Actually, this is optional. As per the spec:
> 
> "When the standard set is ambiguous drivers may return EINVAL or choose any
> of the requested standards."
> 
> Nor does the spec say anything about doing an autodetect when STD_ALL is
> passed in. Most drivers will just set the std to PAL or NTSC in this case.
> 
> If you want to autodetect, then use QUERYSTD. Applications cannot rely on
> drivers to handle V4L2_STD_ALL the way you say.
> 
> > The detected standard can be returned to userspace via VIDIOC_G_STD.
> 
> No! G_STD always returns the current *selected* standard. Only QUERYSTD
> returns the detected standard.
> 
> > If otherwise, another standard mask is sent to the driver via
> > VIDIOC_S_STD, the expected behavior is that the driver should select the
> > standards detector to conform with the desired mask. If an unsupported
> > configuration is requested, the driver should return the mask it
> > actually used at the return of VIDIOC_S_STD call.
> 
> S_STD is a write-only ioctl, so the mask isn't updated.
> 
> > For example, if V4L2_STD_NTSC_M_JP is used, the driver should disable the
> > auto-detector, and use NTSC/M with the Japanese audio standard. both
> > S_STD and G_STD will return V4L2_STD_NTSC_M_JP.
> > If V4L2_STD_MN is used and the driver can auto-detect between all those
> > formats, the driver should detect if the standard is PAL or NTSC and
> > detect between STD/M or STD/M (and the corresponding audio standards).
> > 
> > If an unsupported mask like V4L2_STD_PAL_J | V4L2_STD_NTSC_M_JP is used,
> > the driver should return a valid combination to S_STD (for example,
> > returning V4L2_STD_PAL_J.
> > 
> > In any case, on V4L2_G_STD, if the driver can't detect what's the
> > standard, it should just return the current detection mask to userspace
> > (instead of returning something like STD_INVALID).
> 
> G_STD must always return the currently selected standard, never the
> detected standard. That's QUERYSTD.
> 
> When the driver is first loaded it must pre-select a standard (usually in
> the probe function), either hardcoded (NTSC or PAL), or by doing an
> initial autodetect. But the standard should always be set to something.
> This allows you to start streaming immediately.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Mon, Oct 3, 2011 at 10:39 AM, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> Hi Hans,
>
> On Monday 03 October 2011 08:30:25 Hans Verkuil wrote:
>> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
>> > Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
>> > > On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus wrote:
>
> [snip]
>
>> > >>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>> > >>>
>> > >>>       .s_routing = tvp5150_s_routing,
>> > >>>
>> > >>> +     .s_stream = tvp515x_s_stream,
>> > >>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>> > >>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>> > >>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>> > >>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>> > >>> +     .g_parm = tvp515x_g_parm,
>> > >>> +     .s_parm = tvp515x_s_parm,
>> > >>> +     .s_std_output = tvp5150_s_std,
>> > >>
>> > >> Do we really need both video and pad format ops?
>> > >
>> > > Good question, I don't know. Can this device be used as a standalone
>> > > v4l2 device? Or is supposed to always be a part of a video streaming
>> > > pipeline as a sub-device with a source pad? Sorry if my questions are
>> > > silly but as I stated before, I'm a newbie with v4l2 and MCF.
>> >
>> > The tvp5150 driver is used on some em28xx devices. It is nice to add
>> > auto-detection code to the driver, but converting it to the media bus
>> > should be done with enough care to not break support for the existing
>> > devices.
>>
>> So in other words, the tvp5150 driver needs both pad and non-pad ops.
>> Eventually all non-pad variants in subdev drivers should be replaced by the
>> pad variants so you don't have duplication of ops. But that will take a lot
>> more work.
>
> What about replacing direct calls to non-pad operations with core V4L2
> functions that would use the subdev non-pad operation if available, and
> emulate if with the pad operation otherwise ? I think this would ease the
> transition, as subdev drivers could be ported to pad operations without
> worrying about the bridges that use them, and bridge drivers could be switched
> to the new wrappers with a simple search and replace.

Ok, that is a good solution. I'll do that. Implement V4L2 core
operations as wrappers of the subdev pad operations.

>
>> > Also, as I've argued with Laurent before, the expected behavior is that
>> > the standards format selection should be done via the video node, and
>> > not via the media controller node. The V4L2 API has enough support for
>> > all you need to do with the video decoder, so there's no excuse to
>> > duplicate it with any other API.
>>
>> This is relevant for bridge drivers, not for subdev drivers.
>>
>> > The media controller API is there not to replace V4L2, but to complement
>> > it where needed.
>>
>> That will be a nice discussion during the workshop :-)
>
> I don't think we disagree on that, but we probably disagree on what it means
> :-)
>
> --
> Regards,
>
> Laurent Pinchart
>

Laurent, I have a few questions about MCF and the OMAP3ISP driver if
you are so kind to answer.

1- User-space programs that are not MCF aware negotiate the format
with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
pad. But the real format is driven by the analog video format in the
source pad (i.e: tvp5151).

I modified the ISP driver to get the data format from the source pad
and set the format for each pad on the pipeline accordingly but I've
read from the documentation [1] that is not correct to propagate a
data format from source pads to sink pads, that the correct thing is
to do it from sink to source.

So, in this case an administrator has to externally configure the
format for each pad and to guarantee a coherent format on the whole
pipeline?. Or does exist a way to do this automatic?. i.e: The output
entity on the pipeline promotes the capabilities of the source pad so
applications can select a data format and this format gets propagated
all over the pipeline from the sink pad to the source?

[1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html

2- If the application want a different format that the default
provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
crop the image? I thought this can be made using the CCDC, copying
less lines to memory or the RESIZER if the application wants a bigger
image. What is the best approach for this?

3- When using embedded sync, CCDC doesn't have an external vertical
sync signal, so we have to manually configure when we want the VD0
interrupt to raise. This works for progressive frames, since each
frame has the same size but in the case of interlaced video,
sub-frames have different sizes (i.e: 313 and 312 vertical lines for
PAL).

What I did is to reconfigure the CCDC on the VD1 interrupt handler,
but I think this is more a hack than a clean solution. What do you
think is the best approach to solve this?

Best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Sakari Ailus]

On Sun, Oct 02, 2011 at 11:18:29PM +0200, Javier Martinez Canillas wrote:
> On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus <sakari.ailus@iki.fi> wrote:
> > Hi Javier,
> >
> > Thanks for the patch! It's very interesting to see a driver for a video
> > decoder using the MC interface. Before this we've had just image sensors.
> >
> 
> Hello Sakari,
> 
> Thanks for your comments.

Hi Javier,

You're welcome. You also got very good comments from others.

> > Javier Martinez Canillas wrote:
> >> +             /* use the standard status register */
> >> +             std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
> >> +     else
> >> +             /* use the standard register itself */
> >> +             std_status = std;
> >
> > Braces would be nice here.
> >
> 
> Ok.
> 
> >> +     switch (std_status & VIDEO_STD_MASK) {
> >> +     case VIDEO_STD_NTSC_MJ_BIT:
> >> +     case VIDEO_STD_NTSC_MJ_BIT_AS:
> >> +             return STD_NTSC_MJ;
> >> +
> >> +     case VIDEO_STD_PAL_BDGHIN_BIT:
> >> +     case VIDEO_STD_PAL_BDGHIN_BIT_AS:
> >> +             return STD_PAL_BDGHIN;
> >> +
> >> +     default:
> >> +             return STD_INVALID;
> >> +     }
> >> +
> >> +     return STD_INVALID;
> >
> > This return won't do anything.
> >
> 
> Yes, will clean this.
> 
> >> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
> >>       if (std == V4L2_STD_ALL) {
> >>               fmt = 0;        /* Autodetect mode */
> >>       } else if (std & V4L2_STD_NTSC_443) {
> >> -             fmt = 0xa;
> >> +             fmt = VIDEO_STD_NTSC_4_43_BIT;
> >>       } else if (std & V4L2_STD_PAL_M) {
> >> -             fmt = 0x6;
> >> +             fmt = VIDEO_STD_PAL_M_BIT;
> >>       } else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
> >> -             fmt = 0x8;
> >> +             fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
> >>       } else {
> >>               /* Then, test against generic ones */
> >>               if (std & V4L2_STD_NTSC)
> >> -                     fmt = 0x2;
> >> +                     fmt = VIDEO_STD_NTSC_MJ_BIT;
> >>               else if (std & V4L2_STD_PAL)
> >> -                     fmt = 0x4;
> >> +                     fmt = VIDEO_STD_PAL_BDGHIN_BIT;
> >>               else if (std & V4L2_STD_SECAM)
> >> -                     fmt = 0xc;
> >> +                     fmt = VIDEO_STD_SECAM_BIT;
> >>       }
> >
> > Excellent! Less magic numbers...
> >
> >>
> >> +static struct v4l2_mbus_framefmt *
> >> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
> >> +                      unsigned int pad, enum v4l2_subdev_format_whence which)
> >> +{
> >> +     switch (which) {
> >> +     case V4L2_SUBDEV_FORMAT_TRY:
> >> +             return v4l2_subdev_get_try_format(fh, pad);
> >> +     case V4L2_SUBDEV_FORMAT_ACTIVE:
> >> +             return tvp5150->format;
> >> +     default:
> >> +             return NULL;
> >
> > Hmm. This will never happen, but is returning NULL the right thing to
> > do? An easy alternative is to just replace this with if (which may only
> > have either of the two values).
> >
> 
> Ok I'll cleanup, I was being a bit paranoid there :)
> 
> >> +
> >> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
> >> +                           struct v4l2_subdev_fh *fh,
> >> +                           struct v4l2_subdev_format *format)
> >> +{
> >> +     struct tvp5150 *tvp5150 = to_tvp5150(subdev);
> >> +     tvp5150->std_idx = STD_INVALID;
> >
> > The above assignment will always be overwritten immediately.
> >
> 
> Yes, since tvp515x_query_current_std() already returns STD_INVALID on
> error the assignment is not needed. Will change that.
> 
> >> +     tvp5150->std_idx = tvp515x_query_current_std(subdev);
> >> +     if (tvp5150->std_idx == STD_INVALID) {
> >> +             v4l2_err(subdev, "Unable to query std\n");
> >> +             return 0;
> >
> > Isn't this an error?
> >
> 
> Yes, I'll change to report the error to the caller.

Thinking about this again, the error likely shouldn't be returned to the
user.

<URL:http://hverkuil.home.xs4all.nl/spec/media.html#vidioc-subdev-g-fmt>

Nonetheless, something should definitely be returned to the user. It might
be best to leave it unchanged.

> >> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
> >
> > The name of the function is different.
> >
> 
> Yes, I'll change that.
> 
> >>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
> >>       .s_routing = tvp5150_s_routing,
> >> +     .s_stream = tvp515x_s_stream,
> >> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
> >> +     .g_mbus_fmt = tvp515x_mbus_fmt,
> >> +     .try_mbus_fmt = tvp515x_mbus_fmt,
> >> +     .s_mbus_fmt = tvp515x_mbus_fmt,
> >> +     .g_parm = tvp515x_g_parm,
> >> +     .s_parm = tvp515x_s_parm,
> >> +     .s_std_output = tvp5150_s_std,
> >
> > Do we really need both video and pad format ops?
> >
> 
> Good question, I don't know. Can this device be used as a standalone
> v4l2 device? Or is supposed to always be a part of a video streaming
> pipeline as a sub-device with a source pad? Sorry if my questions are
> silly but as I stated before, I'm a newbie with v4l2 and MCF.

You got good comments from others to this.

I agree with Laurent, the right thing to do is to implement wrappers for the
video fmt ops so that the driver only needed to implement pad ops. This way
all the bridge drivers would work with your subdev driver.

> > s_std should be added to pad ops so it would be available on the subdev
> > node.
> >
> 
> Ok, I'll add s_std operation.
> 
> >>
> >> +static int tvp515x_enum_mbus_code(struct v4l2_subdev *subdev,
> >> +                               struct v4l2_subdev_fh *fh,
> >> +                               struct v4l2_subdev_mbus_code_enum *code)
> >> +{
> >> +     if (code->index >= ARRAY_SIZE(tvp515x_std_list))
> >> +             return -EINVAL;
> >> +
> >> +     code->code = V4L2_MBUS_FMT_UYVY8_2X8;
> >
> > If there's just one supported mbus code, non-zero code->index must
> > return -EINVAL.
> >
> 
> Ok, I'll change that.
> 
> >> +     return 0;
> >> +}
> >> +
> >> +static int tvp515x_enum_frame_size(struct v4l2_subdev *subdev,
> >> +                                struct v4l2_subdev_fh *fh,
> >> +                                struct v4l2_subdev_frame_size_enum *fse)
> >> +{
> >> +     int current_std = STD_INVALID;
> >
> > current_std is overwritten before it gets used.
> >
> 
> Yes, same case here, will remove the assignment.
> 
> >> +     if (fse->code != V4L2_MBUS_FMT_UYVY8_2X8)
> >> +             return -EINVAL;
> >> +
> >> +     /* query the current standard */
> >> +     current_std = tvp515x_query_current_std(subdev);
> >> +     if (current_std == STD_INVALID) {
> >> +             v4l2_err(subdev, "Unable to query std\n");
> >> +             return 0;
> >> +     }
> >
> > I wonder how the enum_frame_size and s_std are supposed to interact,
> > especially that I understand, after reading the discussion, the chip may
> > be used to force certain standard while the actual signal is different.
> >
> 
> Well my thought was that the application can select which standard
> (i.e: V4L2_STD_PAL) and enum_frame_size will return the width and
> height for that standard (i.e: 720x625 for PAL). Since that is what
> the device is capturing.
> 
> Then a user-space application can configure the CCDC and RESIZER to
> modify the format. Does this make sense to you? Or the user-space
> application can select a different frame size at the sub-dev level
> (tvp5151)?

That's a good question.

I think that if the receiver should output the format it can and try nothing
else.

The resizer may then be used to scale that. Then, g_std does return
framelines that corresponds to the original received video signal but I
don't think that's an issue. The user knows nevertheless that scaling is
being made since he must have configured that.

To support generic applications properly we do need a plugin for libv4l. We
have a few bits in place already, like libmediactl, libv4l2subdev and an
OMAP 3 plugin for libv4l.

Regards,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [PATCH 0/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Gary Thomas]

On 2011-10-01 10:39, Enrico wrote:
> On Sat, Oct 1, 2011 at 5:55 PM, Javier Martinez Canillas
> <martinez.javier@gmail.com>  wrote:
>> We hack a few bits of the ISP CCDC driver to support ITU-R BT656
>> interlaced data with embedded syncs video format and ported the
>> tvp5150 driver to the MCF so it can be detected as a sub-device and be
>> part of the OMAP ISP image processing pipeline (as a source pad).
>
> That was already posted on the list [1], there was some discussion but
> i don't know what's the status/plan to get it into mainline.
>
> And, as you can see in [2], don't expect many comments :D
>
> [1]: http://www.spinics.net/lists/linux-media/msg37710.html
> [2]: http://www.spinics.net/lists/linux-media/msg37116.html
>
>
>>> Even if it does detect the signal shape (NTSC, PAL), doesn't one still need
>>> to [externally] configure the pads for this shape?
>>>
>>
>> Yes, that is why I wanted to do the auto-detection for the tvp5151, so
>> we only have to manually configure the ISP components (or any other
>> hardware video processing pipeline entities, sorry for my
>> OMAP-specific comments).
>
> Laurent was not very happy [3] about changing video formats out of the
> driver control, so this should be discussed more.
>
> [3]: http://www.spinics.net/lists/linux-omap/msg56983.html
>
>
>> I didn't know that the physical connection affected the video output
>> format, I thought that it was only a physical medium to carry the same
>> information, sorry if my comments are silly but I'm really newbie with
>> video in general.
>
> I think you got it right, i haven't tested it but the output format
> shouldn't be affected by the video source( if it stays pal/ntsc of
> course). Maybe you will get only a different "active" video area so
> only cropping will be affected.

It's not so much the video output [shape], rather that the input source
can be selected and there does not seem to be a way to do that currently
using the MC framework.  I was thinking perhaps to have the driver have
3 different output pads and depending on which one you choose to link
up tells the driver how to configure the input.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Javier,

On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:

[snip]

> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
> you are so kind to answer.
> 
> 1- User-space programs that are not MCF aware negotiate the format
> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
> pad. But the real format is driven by the analog video format in the
> source pad (i.e: tvp5151).

That's not different from existing systems using digital sensors, where the 
format is driven by the sensor.

> I modified the ISP driver to get the data format from the source pad
> and set the format for each pad on the pipeline accordingly but I've
> read from the documentation [1] that is not correct to propagate a
> data format from source pads to sink pads, that the correct thing is
> to do it from sink to source.
> 
> So, in this case an administrator has to externally configure the
> format for each pad and to guarantee a coherent format on the whole
> pipeline?.

That's correct (except you don't need to be an administrator to do so :-)).

> Or does exist a way to do this automatic?. i.e: The output entity on the
> pipeline promotes the capabilities of the source pad so applications can
> select a data format and this format gets propagated all over the pipeline
> from the sink pad to the source?

It can be automated in userspace (through a libv4l plugin for instance), but 
it's really not the kernel's job to do so.

> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
> 
> 2- If the application want a different format that the default
> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
> crop the image? I thought this can be made using the CCDC, copying
> less lines to memory or the RESIZER if the application wants a bigger
> image. What is the best approach for this?

Cropping can be done in the resizer, and I will soon post patches that add 
cropping support in the preview engine (although that will be useless for the 
TVP5151, as the preview engine doesn't support YUV data). The CCDC supports 
cropping too, but that's not implemented in the driver yet.

> 3- When using embedded sync, CCDC doesn't have an external vertical
> sync signal, so we have to manually configure when we want the VD0
> interrupt to raise. This works for progressive frames, since each
> frame has the same size but in the case of interlaced video,
> sub-frames have different sizes (i.e: 313 and 312 vertical lines for
> PAL).
> 
> What I did is to reconfigure the CCDC on the VD1 interrupt handler,
> but I think this is more a hack than a clean solution. What do you
> think is the best approach to solve this?

I *really* wish the CCDC had an end of frame interrupt :-( I'm not sure if 
there's a non-hackish solution to this.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 03-10-2011 03:30, Hans Verkuil escreveu:
> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
>> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
>>> On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus <sakari.ailus@iki.fi> wrote:
>>>> Hi Javier,
>>>>
>>>> Thanks for the patch! It's very interesting to see a driver for a video
>>>> decoder using the MC interface. Before this we've had just image sensors.
>>>>
>>>
>>> Hello Sakari,
>>>
>>> Thanks for your comments.
>>>
>>>> Javier Martinez Canillas wrote:
>>>>> +             /* use the standard status register */
>>>>> +             std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
>>>>> +     else
>>>>> +             /* use the standard register itself */
>>>>> +             std_status = std;
>>>>
>>>> Braces would be nice here.
>>>>
>>>
>>> Ok.
>>>
>>>>> +     switch (std_status & VIDEO_STD_MASK) {
>>>>> +     case VIDEO_STD_NTSC_MJ_BIT:
>>>>> +     case VIDEO_STD_NTSC_MJ_BIT_AS:
>>>>> +             return STD_NTSC_MJ;
>>>>> +
>>>>> +     case VIDEO_STD_PAL_BDGHIN_BIT:
>>>>> +     case VIDEO_STD_PAL_BDGHIN_BIT_AS:
>>>>> +             return STD_PAL_BDGHIN;
>>>>> +
>>>>> +     default:
>>>>> +             return STD_INVALID;
>>>>> +     }
>>>>> +
>>>>> +     return STD_INVALID;
>>>>
>>>> This return won't do anything.
>>>>
>>>
>>> Yes, will clean this.
>>>
>>>>> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
>>>>>       if (std == V4L2_STD_ALL) {
>>>>>               fmt = 0;        /* Autodetect mode */
>>>>>       } else if (std & V4L2_STD_NTSC_443) {
>>>>> -             fmt = 0xa;
>>>>> +             fmt = VIDEO_STD_NTSC_4_43_BIT;
>>>>>       } else if (std & V4L2_STD_PAL_M) {
>>>>> -             fmt = 0x6;
>>>>> +             fmt = VIDEO_STD_PAL_M_BIT;
>>>>>       } else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
>>>>> -             fmt = 0x8;
>>>>> +             fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
>>>>>       } else {
>>>>>               /* Then, test against generic ones */
>>>>>               if (std & V4L2_STD_NTSC)
>>>>> -                     fmt = 0x2;
>>>>> +                     fmt = VIDEO_STD_NTSC_MJ_BIT;
>>>>>               else if (std & V4L2_STD_PAL)
>>>>> -                     fmt = 0x4;
>>>>> +                     fmt = VIDEO_STD_PAL_BDGHIN_BIT;
>>>>>               else if (std & V4L2_STD_SECAM)
>>>>> -                     fmt = 0xc;
>>>>> +                     fmt = VIDEO_STD_SECAM_BIT;
>>>>>       }
>>>>
>>>> Excellent! Less magic numbers...
>>>>
>>>>>
>>>>> +static struct v4l2_mbus_framefmt *
>>>>> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
>>>>> +                      unsigned int pad, enum v4l2_subdev_format_whence which)
>>>>> +{
>>>>> +     switch (which) {
>>>>> +     case V4L2_SUBDEV_FORMAT_TRY:
>>>>> +             return v4l2_subdev_get_try_format(fh, pad);
>>>>> +     case V4L2_SUBDEV_FORMAT_ACTIVE:
>>>>> +             return tvp5150->format;
>>>>> +     default:
>>>>> +             return NULL;
>>>>
>>>> Hmm. This will never happen, but is returning NULL the right thing to
>>>> do? An easy alternative is to just replace this with if (which may only
>>>> have either of the two values).
>>>>
>>>
>>> Ok I'll cleanup, I was being a bit paranoid there :)
>>>
>>>>> +
>>>>> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
>>>>> +                           struct v4l2_subdev_fh *fh,
>>>>> +                           struct v4l2_subdev_format *format)
>>>>> +{
>>>>> +     struct tvp5150 *tvp5150 = to_tvp5150(subdev);
>>>>> +     tvp5150->std_idx = STD_INVALID;
>>>>
>>>> The above assignment will always be overwritten immediately.
>>>>
>>>
>>> Yes, since tvp515x_query_current_std() already returns STD_INVALID on
>>> error the assignment is not needed. Will change that.
>>>
>>>>> +     tvp5150->std_idx = tvp515x_query_current_std(subdev);
>>>>> +     if (tvp5150->std_idx == STD_INVALID) {
>>>>> +             v4l2_err(subdev, "Unable to query std\n");
>>>>> +             return 0;
>>>>
>>>> Isn't this an error?
>>>>
>>>
>>> Yes, I'll change to report the error to the caller.
>>>
>>>>> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
>>>>
>>>> The name of the function is different.
>>>>
>>>
>>> Yes, I'll change that.
>>>
>>>>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>>>>>       .s_routing = tvp5150_s_routing,
>>>>> +     .s_stream = tvp515x_s_stream,
>>>>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>>>>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>>>>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>>>>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>>>>> +     .g_parm = tvp515x_g_parm,
>>>>> +     .s_parm = tvp515x_s_parm,
>>>>> +     .s_std_output = tvp5150_s_std,
>>>>
>>>> Do we really need both video and pad format ops?
>>>>
>>>
>>> Good question, I don't know. Can this device be used as a standalone
>>> v4l2 device? Or is supposed to always be a part of a video streaming
>>> pipeline as a sub-device with a source pad? Sorry if my questions are
>>> silly but as I stated before, I'm a newbie with v4l2 and MCF.
>>
>> The tvp5150 driver is used on some em28xx devices. It is nice to add auto-detection
>> code to the driver, but converting it to the media bus should be done with
>> enough care to not break support for the existing devices.
> 
> So in other words, the tvp5150 driver needs both pad and non-pad ops.
> Eventually all non-pad variants in subdev drivers should be replaced by the
> pad variants so you don't have duplication of ops. But that will take a lot
> more work.
> 
>> Also, as I've argued with Laurent before, the expected behavior is that the standards
>> format selection should be done via the video node, and not via the media 
>> controller node. The V4L2 API has enough support for all you need to do with the
>> video decoder, so there's no excuse to duplicate it with any other API.
> 
> This is relevant for bridge drivers, not for subdev drivers.
> 
>> The media controller API is there not to replace V4L2, but to complement
>> it where needed.
> 
> That will be a nice discussion during the workshop :-)
> 
>> In the specific code of standards auto-detection, a few drivers currently support
>> this feature. They're (or should be) coded to do is:
>>
>> If V4L2_STD_ALL is used, the driver should autodetect the video standard of the
>> currently tuned channel.
> 
> Actually, this is optional. As per the spec:
> 
> "When the standard set is ambiguous drivers may return EINVAL or choose any of
> the requested standards."

Yes. The entire auto-detection thing is optional. Several devices aren't capable of
auto-detecting standards. Btw, userspace support for standards auto-detection is
another chapter. I don't think they implement standards auto-detection.

There are even some applications (qv4l2 and tvtime, for example) that aren't capable of 
refreshing the maximum height when the standard changes.

> Nor does the spec say anything about doing an autodetect when STD_ALL is passed
> in. Most drivers will just set the std to PAL or NTSC in this case.

Auto-detection is device specific. That's why most applications don't handle it 
well (or don't even care on trying to do it). Btw, I did some tests with autodetection
on a device with a saa7115, and I found some bugs. Just sent a fix for it.

> If you want to autodetect, then use QUERYSTD. Applications cannot rely on drivers
> to handle V4L2_STD_ALL the way you say.
> 
>> The detected standard can be returned to userspace via VIDIOC_G_STD.
> 
> No! G_STD always returns the current *selected* standard. Only QUERYSTD returns
> the detected standard.

Yes, you're right. I should not try to answer emails when I'm lazy enough to not
look in to the code ;)

Anyway, the thing is that V4L2 API has enough support for auto-detection. There's
no need to duplicate stuff with MC API.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 03-10-2011 04:11, Javier Martinez Canillas escreveu:
> On Mon, Oct 3, 2011 at 8:30 AM, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
>>> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
>>>>
>>>> Yes, I'll change that.
>>>>
>>>>>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>>>>>>       .s_routing = tvp5150_s_routing,
>>>>>> +     .s_stream = tvp515x_s_stream,
>>>>>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>>>>>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>>>>>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>>>>>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>>>>>> +     .g_parm = tvp515x_g_parm,
>>>>>> +     .s_parm = tvp515x_s_parm,
>>>>>> +     .s_std_output = tvp5150_s_std,
>>>>>
>>>>> Do we really need both video and pad format ops?
>>>>>
>>>>
>>>> Good question, I don't know. Can this device be used as a standalone
>>>> v4l2 device? Or is supposed to always be a part of a video streaming
>>>> pipeline as a sub-device with a source pad? Sorry if my questions are
>>>> silly but as I stated before, I'm a newbie with v4l2 and MCF.
>>>
>>> The tvp5150 driver is used on some em28xx devices. It is nice to add auto-detection
>>> code to the driver, but converting it to the media bus should be done with
>>> enough care to not break support for the existing devices.
>>
>> So in other words, the tvp5150 driver needs both pad and non-pad ops.
>> Eventually all non-pad variants in subdev drivers should be replaced by the
>> pad variants so you don't have duplication of ops. But that will take a lot
>> more work.

Why doing it at pad level? It makes no sense. After selecting the pipeline/input
all pad's that handle analog TV formats should be changed at the same time: 
video decoder, audio decoder, video enhancer/filters, etc.

> Great, that was a doubt I had, thanks for the clarification.
> 
>>
>>> In the specific code of standards auto-detection, a few drivers currently support
>>> this feature. They're (or should be) coded to do is:
>>>
>>> If V4L2_STD_ALL is used, the driver should autodetect the video standard of the
>>> currently tuned channel.
>>
>> Actually, this is optional. As per the spec:
>>
>> "When the standard set is ambiguous drivers may return EINVAL or choose any of
>> the requested standards."
>>
>> Nor does the spec say anything about doing an autodetect when STD_ALL is passed
>> in. Most drivers will just set the std to PAL or NTSC in this case.
>>
>> If you want to autodetect, then use QUERYSTD. Applications cannot rely on drivers
>> to handle V4L2_STD_ALL the way you say.
>>
>>> The detected standard can be returned to userspace via VIDIOC_G_STD.
>>
>> No! G_STD always returns the current *selected* standard. Only QUERYSTD returns
>> the detected standard.
>>
>>>
>>> If otherwise, another standard mask is sent to the driver via VIDIOC_S_STD,
>>> the expected behavior is that the driver should select the standards detector
>>> to conform with the desired mask. If an unsupported configuration is requested,
>>> the driver should return the mask it actually used at the return of VIDIOC_S_STD
>>> call.
>>
>> S_STD is a write-only ioctl, so the mask isn't updated.
>>
>>> For example, if V4L2_STD_NTSC_M_JP is used, the driver should disable the
>>> auto-detector, and use NTSC/M with the Japanese audio standard. both S_STD
>>> and G_STD will return V4L2_STD_NTSC_M_JP.
>>> If V4L2_STD_MN is used and the driver can auto-detect between all those formats,
>>> the driver should detect if the standard is PAL or NTSC and detect between
>>> STD/M or STD/M (and the corresponding audio standards).
>>>
>>> If an unsupported mask like V4L2_STD_PAL_J | V4L2_STD_NTSC_M_JP is used, the driver
>>> should return a valid combination to S_STD (for example, returning V4L2_STD_PAL_J.
>>>
>>> In any case, on V4L2_G_STD, if the driver can't detect what's the standard, it should
>>> just return the current detection mask to userspace (instead of returning something
>>> like STD_INVALID).
>>
>> G_STD must always return the currently selected standard, never the detected standard.
>> That's QUERYSTD.
>>
>> When the driver is first loaded it must pre-select a standard (usually in the probe
>> function), either hardcoded (NTSC or PAL), or by doing an initial autodetect. But
>> the standard should always be set to something. This allows you to start streaming
>> immediately.
>>
>> Regards,
>>
>>        Hans
>>
>>> I hope that helps,
>>> Mauro.
>>>
>>
> 
> Thanks Mauro and Hans for your comments.
> 
> I plan to work on the autodetect code and the issues called out by
> Sakari and resubmit the patch, can you point me a driver that got
> auto-detect the right way so I can use it as a reference?

The saa7115 driver implements it right. I've reviewed its code and tested it with
a real device.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Sakari Ailus]

On Mon, Oct 03, 2011 at 03:53:54PM -0300, Mauro Carvalho Chehab wrote:
> Em 03-10-2011 03:30, Hans Verkuil escreveu:
> > On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
> >> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
> >>> On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus <sakari.ailus@iki.fi> wrote:
> >>>> Hi Javier,
> >>>>
> >>>> Thanks for the patch! It's very interesting to see a driver for a video
> >>>> decoder using the MC interface. Before this we've had just image sensors.
> >>>>
> >>>
> >>> Hello Sakari,
> >>>
> >>> Thanks for your comments.
> >>>
> >>>> Javier Martinez Canillas wrote:
> >>>>> +             /* use the standard status register */
> >>>>> +             std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
> >>>>> +     else
> >>>>> +             /* use the standard register itself */
> >>>>> +             std_status = std;
> >>>>
> >>>> Braces would be nice here.
> >>>>
> >>>
> >>> Ok.
> >>>
> >>>>> +     switch (std_status & VIDEO_STD_MASK) {
> >>>>> +     case VIDEO_STD_NTSC_MJ_BIT:
> >>>>> +     case VIDEO_STD_NTSC_MJ_BIT_AS:
> >>>>> +             return STD_NTSC_MJ;
> >>>>> +
> >>>>> +     case VIDEO_STD_PAL_BDGHIN_BIT:
> >>>>> +     case VIDEO_STD_PAL_BDGHIN_BIT_AS:
> >>>>> +             return STD_PAL_BDGHIN;
> >>>>> +
> >>>>> +     default:
> >>>>> +             return STD_INVALID;
> >>>>> +     }
> >>>>> +
> >>>>> +     return STD_INVALID;
> >>>>
> >>>> This return won't do anything.
> >>>>
> >>>
> >>> Yes, will clean this.
> >>>
> >>>>> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
> >>>>>       if (std == V4L2_STD_ALL) {
> >>>>>               fmt = 0;        /* Autodetect mode */
> >>>>>       } else if (std & V4L2_STD_NTSC_443) {
> >>>>> -             fmt = 0xa;
> >>>>> +             fmt = VIDEO_STD_NTSC_4_43_BIT;
> >>>>>       } else if (std & V4L2_STD_PAL_M) {
> >>>>> -             fmt = 0x6;
> >>>>> +             fmt = VIDEO_STD_PAL_M_BIT;
> >>>>>       } else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
> >>>>> -             fmt = 0x8;
> >>>>> +             fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
> >>>>>       } else {
> >>>>>               /* Then, test against generic ones */
> >>>>>               if (std & V4L2_STD_NTSC)
> >>>>> -                     fmt = 0x2;
> >>>>> +                     fmt = VIDEO_STD_NTSC_MJ_BIT;
> >>>>>               else if (std & V4L2_STD_PAL)
> >>>>> -                     fmt = 0x4;
> >>>>> +                     fmt = VIDEO_STD_PAL_BDGHIN_BIT;
> >>>>>               else if (std & V4L2_STD_SECAM)
> >>>>> -                     fmt = 0xc;
> >>>>> +                     fmt = VIDEO_STD_SECAM_BIT;
> >>>>>       }
> >>>>
> >>>> Excellent! Less magic numbers...
> >>>>
> >>>>>
> >>>>> +static struct v4l2_mbus_framefmt *
> >>>>> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
> >>>>> +                      unsigned int pad, enum v4l2_subdev_format_whence which)
> >>>>> +{
> >>>>> +     switch (which) {
> >>>>> +     case V4L2_SUBDEV_FORMAT_TRY:
> >>>>> +             return v4l2_subdev_get_try_format(fh, pad);
> >>>>> +     case V4L2_SUBDEV_FORMAT_ACTIVE:
> >>>>> +             return tvp5150->format;
> >>>>> +     default:
> >>>>> +             return NULL;
> >>>>
> >>>> Hmm. This will never happen, but is returning NULL the right thing to
> >>>> do? An easy alternative is to just replace this with if (which may only
> >>>> have either of the two values).
> >>>>
> >>>
> >>> Ok I'll cleanup, I was being a bit paranoid there :)
> >>>
> >>>>> +
> >>>>> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
> >>>>> +                           struct v4l2_subdev_fh *fh,
> >>>>> +                           struct v4l2_subdev_format *format)
> >>>>> +{
> >>>>> +     struct tvp5150 *tvp5150 = to_tvp5150(subdev);
> >>>>> +     tvp5150->std_idx = STD_INVALID;
> >>>>
> >>>> The above assignment will always be overwritten immediately.
> >>>>
> >>>
> >>> Yes, since tvp515x_query_current_std() already returns STD_INVALID on
> >>> error the assignment is not needed. Will change that.
> >>>
> >>>>> +     tvp5150->std_idx = tvp515x_query_current_std(subdev);
> >>>>> +     if (tvp5150->std_idx == STD_INVALID) {
> >>>>> +             v4l2_err(subdev, "Unable to query std\n");
> >>>>> +             return 0;
> >>>>
> >>>> Isn't this an error?
> >>>>
> >>>
> >>> Yes, I'll change to report the error to the caller.
> >>>
> >>>>> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
> >>>>
> >>>> The name of the function is different.
> >>>>
> >>>
> >>> Yes, I'll change that.
> >>>
> >>>>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
> >>>>>       .s_routing = tvp5150_s_routing,
> >>>>> +     .s_stream = tvp515x_s_stream,
> >>>>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
> >>>>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
> >>>>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
> >>>>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
> >>>>> +     .g_parm = tvp515x_g_parm,
> >>>>> +     .s_parm = tvp515x_s_parm,
> >>>>> +     .s_std_output = tvp5150_s_std,
> >>>>
> >>>> Do we really need both video and pad format ops?
> >>>>
> >>>
> >>> Good question, I don't know. Can this device be used as a standalone
> >>> v4l2 device? Or is supposed to always be a part of a video streaming
> >>> pipeline as a sub-device with a source pad? Sorry if my questions are
> >>> silly but as I stated before, I'm a newbie with v4l2 and MCF.
> >>
> >> The tvp5150 driver is used on some em28xx devices. It is nice to add auto-detection
> >> code to the driver, but converting it to the media bus should be done with
> >> enough care to not break support for the existing devices.
> > 
> > So in other words, the tvp5150 driver needs both pad and non-pad ops.
> > Eventually all non-pad variants in subdev drivers should be replaced by the
> > pad variants so you don't have duplication of ops. But that will take a lot
> > more work.
> > 
> >> Also, as I've argued with Laurent before, the expected behavior is that the standards
> >> format selection should be done via the video node, and not via the media 
> >> controller node. The V4L2 API has enough support for all you need to do with the
> >> video decoder, so there's no excuse to duplicate it with any other API.
> > 
> > This is relevant for bridge drivers, not for subdev drivers.
> > 
> >> The media controller API is there not to replace V4L2, but to complement
> >> it where needed.
> > 
> > That will be a nice discussion during the workshop :-)
> > 
> >> In the specific code of standards auto-detection, a few drivers currently support
> >> this feature. They're (or should be) coded to do is:
> >>
> >> If V4L2_STD_ALL is used, the driver should autodetect the video standard of the
> >> currently tuned channel.
> > 
> > Actually, this is optional. As per the spec:
> > 
> > "When the standard set is ambiguous drivers may return EINVAL or choose any of
> > the requested standards."
> 
> Yes. The entire auto-detection thing is optional. Several devices aren't capable of
> auto-detecting standards. Btw, userspace support for standards auto-detection is
> another chapter. I don't think they implement standards auto-detection.
> 
> There are even some applications (qv4l2 and tvtime, for example) that aren't capable of 
> refreshing the maximum height when the standard changes.
> 
> > Nor does the spec say anything about doing an autodetect when STD_ALL is passed
> > in. Most drivers will just set the std to PAL or NTSC in this case.
> 
> Auto-detection is device specific. That's why most applications don't handle it 
> well (or don't even care on trying to do it). Btw, I did some tests with autodetection
> on a device with a saa7115, and I found some bugs. Just sent a fix for it.
> 
> > If you want to autodetect, then use QUERYSTD. Applications cannot rely on drivers
> > to handle V4L2_STD_ALL the way you say.
> > 
> >> The detected standard can be returned to userspace via VIDIOC_G_STD.
> > 
> > No! G_STD always returns the current *selected* standard. Only QUERYSTD returns
> > the detected standard.
> 
> Yes, you're right. I should not try to answer emails when I'm lazy enough to not
> look in to the code ;)
> 
> Anyway, the thing is that V4L2 API has enough support for auto-detection. There's
> no need to duplicate stuff with MC API.

It's not MC API but V4L2 subdev API. No-one has proposed to add video
standard awareness to the Media controller API. There's no reason to export
a video node in video decoder drivers... but I guess you didn't mean that.

Would implementing ENUM/G/S_STD make sense for the camera ISP driver, that
has nothing to do with any video standard? If you have two video decoders
connected to your system, then which one should the ioctls be redirected to?
What if there's a sensor and a video decoder? And how could the user know
about this?

It's the same old issues again... let's discuss this in the Multimedia
summit.

Cheers,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 03-10-2011 06:53, Javier Martinez Canillas escreveu:
> On Mon, Oct 3, 2011 at 10:39 AM, Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
>> Hi Hans,
>>
>> On Monday 03 October 2011 08:30:25 Hans Verkuil wrote:
>>> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
>>>> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
>>>>> On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus wrote:
>>
>> [snip]
>>
>>>>>>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>>>>>>>
>>>>>>>       .s_routing = tvp5150_s_routing,
>>>>>>>
>>>>>>> +     .s_stream = tvp515x_s_stream,
>>>>>>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>>>>>>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>>>>>>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>>>>>>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>>>>>>> +     .g_parm = tvp515x_g_parm,
>>>>>>> +     .s_parm = tvp515x_s_parm,
>>>>>>> +     .s_std_output = tvp5150_s_std,
>>>>>>
>>>>>> Do we really need both video and pad format ops?
>>>>>
>>>>> Good question, I don't know. Can this device be used as a standalone
>>>>> v4l2 device? Or is supposed to always be a part of a video streaming
>>>>> pipeline as a sub-device with a source pad? Sorry if my questions are
>>>>> silly but as I stated before, I'm a newbie with v4l2 and MCF.
>>>>
>>>> The tvp5150 driver is used on some em28xx devices. It is nice to add
>>>> auto-detection code to the driver, but converting it to the media bus
>>>> should be done with enough care to not break support for the existing
>>>> devices.
>>>
>>> So in other words, the tvp5150 driver needs both pad and non-pad ops.
>>> Eventually all non-pad variants in subdev drivers should be replaced by the
>>> pad variants so you don't have duplication of ops. But that will take a lot
>>> more work.
>>
>> What about replacing direct calls to non-pad operations with core V4L2
>> functions that would use the subdev non-pad operation if available, and
>> emulate if with the pad operation otherwise ? I think this would ease the
>> transition, as subdev drivers could be ported to pad operations without
>> worrying about the bridges that use them, and bridge drivers could be switched
>> to the new wrappers with a simple search and replace.
> 
> Ok, that is a good solution. I'll do that. Implement V4L2 core
> operations as wrappers of the subdev pad operations.

As I said, I can't see _any_ reason why setting a format would be needed
at pad level. Patches shouldn't increase driver/core and userspace complexity
for nothing.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 03-10-2011 08:53, Laurent Pinchart escreveu:
> Hi Javier,
> 
> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
> 
> [snip]
> 
>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
>> you are so kind to answer.
>>
>> 1- User-space programs that are not MCF aware negotiate the format
>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>> pad. But the real format is driven by the analog video format in the
>> source pad (i.e: tvp5151).
> 
> That's not different from existing systems using digital sensors, where the 
> format is driven by the sensor.
> 
>> I modified the ISP driver to get the data format from the source pad
>> and set the format for each pad on the pipeline accordingly but I've
>> read from the documentation [1] that is not correct to propagate a
>> data format from source pads to sink pads, that the correct thing is
>> to do it from sink to source.
>>
>> So, in this case an administrator has to externally configure the
>> format for each pad and to guarantee a coherent format on the whole
>> pipeline?.
> 
> That's correct (except you don't need to be an administrator to do so :-)).

NACK.

When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs that
are handling the video/audio standard should be changed, in order to obey the
V4L2 ioctl. This is what happens with all other drivers since the beginning
of the V4L1 API. There's no reason to change it, and such change would be
a regression.

>> Or does exist a way to do this automatic?. i.e: The output entity on the
>> pipeline promotes the capabilities of the source pad so applications can
>> select a data format and this format gets propagated all over the pipeline
>> from the sink pad to the source?
> 
> It can be automated in userspace (through a libv4l plugin for instance), but 
> it's really not the kernel's job to do so.

It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
userspace plugin.

>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>
>> 2- If the application want a different format that the default
>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
>> crop the image? I thought this can be made using the CCDC, copying
>> less lines to memory or the RESIZER if the application wants a bigger
>> image. What is the best approach for this?

Not sure if I understood your question, but maybe you're mixing two different
concepts here.

If the application wants a different image resolution, it will use S_FMT.
In this case, what userspace expects is that the driver will scale, 
if supported, or return -EINVAL otherwise.

Cropping should be done via S_CROP (or, after accepted upstream, via the 
S_SELECTION ioctl).

> Cropping can be done in the resizer, and I will soon post patches that add 
> cropping support in the preview engine (although that will be useless for the 
> TVP5151, as the preview engine doesn't support YUV data). The CCDC supports 
> cropping too, but that's not implemented in the driver yet.
> 
>> 3- When using embedded sync, CCDC doesn't have an external vertical
>> sync signal, so we have to manually configure when we want the VD0
>> interrupt to raise. This works for progressive frames, since each
>> frame has the same size but in the case of interlaced video,
>> sub-frames have different sizes (i.e: 313 and 312 vertical lines for
>> PAL).
>>
>> What I did is to reconfigure the CCDC on the VD1 interrupt handler,
>> but I think this is more a hack than a clean solution. What do you
>> think is the best approach to solve this?
> 
> I *really* wish the CCDC had an end of frame interrupt :-( I'm not sure if 
> there's a non-hackish solution to this.
> 

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 03-10-2011 16:01, Sakari Ailus escreveu:
> On Mon, Oct 03, 2011 at 03:53:54PM -0300, Mauro Carvalho Chehab wrote:
>> Em 03-10-2011 03:30, Hans Verkuil escreveu:
>>> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
>>>> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
>>>>> On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus <sakari.ailus@iki.fi> wrote:
>>>>>> Hi Javier,
>>>>>>
>>>>>> Thanks for the patch! It's very interesting to see a driver for a video
>>>>>> decoder using the MC interface. Before this we've had just image sensors.
>>>>>>
>>>>>
>>>>> Hello Sakari,
>>>>>
>>>>> Thanks for your comments.
>>>>>
>>>>>> Javier Martinez Canillas wrote:
>>>>>>> +             /* use the standard status register */
>>>>>>> +             std_status = tvp5150_read(sd, TVP5150_STATUS_REG_5);
>>>>>>> +     else
>>>>>>> +             /* use the standard register itself */
>>>>>>> +             std_status = std;
>>>>>>
>>>>>> Braces would be nice here.
>>>>>>
>>>>>
>>>>> Ok.
>>>>>
>>>>>>> +     switch (std_status & VIDEO_STD_MASK) {
>>>>>>> +     case VIDEO_STD_NTSC_MJ_BIT:
>>>>>>> +     case VIDEO_STD_NTSC_MJ_BIT_AS:
>>>>>>> +             return STD_NTSC_MJ;
>>>>>>> +
>>>>>>> +     case VIDEO_STD_PAL_BDGHIN_BIT:
>>>>>>> +     case VIDEO_STD_PAL_BDGHIN_BIT_AS:
>>>>>>> +             return STD_PAL_BDGHIN;
>>>>>>> +
>>>>>>> +     default:
>>>>>>> +             return STD_INVALID;
>>>>>>> +     }
>>>>>>> +
>>>>>>> +     return STD_INVALID;
>>>>>>
>>>>>> This return won't do anything.
>>>>>>
>>>>>
>>>>> Yes, will clean this.
>>>>>
>>>>>>> @@ -704,19 +812,19 @@ static int tvp5150_set_std(struct v4l2_subdev *sd, v4l2_std_id std)
>>>>>>>       if (std == V4L2_STD_ALL) {
>>>>>>>               fmt = 0;        /* Autodetect mode */
>>>>>>>       } else if (std & V4L2_STD_NTSC_443) {
>>>>>>> -             fmt = 0xa;
>>>>>>> +             fmt = VIDEO_STD_NTSC_4_43_BIT;
>>>>>>>       } else if (std & V4L2_STD_PAL_M) {
>>>>>>> -             fmt = 0x6;
>>>>>>> +             fmt = VIDEO_STD_PAL_M_BIT;
>>>>>>>       } else if (std & (V4L2_STD_PAL_N | V4L2_STD_PAL_Nc)) {
>>>>>>> -             fmt = 0x8;
>>>>>>> +             fmt = VIDEO_STD_PAL_COMBINATION_N_BIT;
>>>>>>>       } else {
>>>>>>>               /* Then, test against generic ones */
>>>>>>>               if (std & V4L2_STD_NTSC)
>>>>>>> -                     fmt = 0x2;
>>>>>>> +                     fmt = VIDEO_STD_NTSC_MJ_BIT;
>>>>>>>               else if (std & V4L2_STD_PAL)
>>>>>>> -                     fmt = 0x4;
>>>>>>> +                     fmt = VIDEO_STD_PAL_BDGHIN_BIT;
>>>>>>>               else if (std & V4L2_STD_SECAM)
>>>>>>> -                     fmt = 0xc;
>>>>>>> +                     fmt = VIDEO_STD_SECAM_BIT;
>>>>>>>       }
>>>>>>
>>>>>> Excellent! Less magic numbers...
>>>>>>
>>>>>>>
>>>>>>> +static struct v4l2_mbus_framefmt *
>>>>>>> +__tvp5150_get_pad_format(struct tvp5150 *tvp5150, struct v4l2_subdev_fh *fh,
>>>>>>> +                      unsigned int pad, enum v4l2_subdev_format_whence which)
>>>>>>> +{
>>>>>>> +     switch (which) {
>>>>>>> +     case V4L2_SUBDEV_FORMAT_TRY:
>>>>>>> +             return v4l2_subdev_get_try_format(fh, pad);
>>>>>>> +     case V4L2_SUBDEV_FORMAT_ACTIVE:
>>>>>>> +             return tvp5150->format;
>>>>>>> +     default:
>>>>>>> +             return NULL;
>>>>>>
>>>>>> Hmm. This will never happen, but is returning NULL the right thing to
>>>>>> do? An easy alternative is to just replace this with if (which may only
>>>>>> have either of the two values).
>>>>>>
>>>>>
>>>>> Ok I'll cleanup, I was being a bit paranoid there :)
>>>>>
>>>>>>> +
>>>>>>> +static int tvp5150_set_pad_format(struct v4l2_subdev *subdev,
>>>>>>> +                           struct v4l2_subdev_fh *fh,
>>>>>>> +                           struct v4l2_subdev_format *format)
>>>>>>> +{
>>>>>>> +     struct tvp5150 *tvp5150 = to_tvp5150(subdev);
>>>>>>> +     tvp5150->std_idx = STD_INVALID;
>>>>>>
>>>>>> The above assignment will always be overwritten immediately.
>>>>>>
>>>>>
>>>>> Yes, since tvp515x_query_current_std() already returns STD_INVALID on
>>>>> error the assignment is not needed. Will change that.
>>>>>
>>>>>>> +     tvp5150->std_idx = tvp515x_query_current_std(subdev);
>>>>>>> +     if (tvp5150->std_idx == STD_INVALID) {
>>>>>>> +             v4l2_err(subdev, "Unable to query std\n");
>>>>>>> +             return 0;
>>>>>>
>>>>>> Isn't this an error?
>>>>>>
>>>>>
>>>>> Yes, I'll change to report the error to the caller.
>>>>>
>>>>>>> + * tvp515x_mbus_fmt_cap() - V4L2 decoder interface handler for try/s/g_mbus_fmt
>>>>>>
>>>>>> The name of the function is different.
>>>>>>
>>>>>
>>>>> Yes, I'll change that.
>>>>>
>>>>>>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
>>>>>>>       .s_routing = tvp5150_s_routing,
>>>>>>> +     .s_stream = tvp515x_s_stream,
>>>>>>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
>>>>>>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
>>>>>>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
>>>>>>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
>>>>>>> +     .g_parm = tvp515x_g_parm,
>>>>>>> +     .s_parm = tvp515x_s_parm,
>>>>>>> +     .s_std_output = tvp5150_s_std,
>>>>>>
>>>>>> Do we really need both video and pad format ops?
>>>>>>
>>>>>
>>>>> Good question, I don't know. Can this device be used as a standalone
>>>>> v4l2 device? Or is supposed to always be a part of a video streaming
>>>>> pipeline as a sub-device with a source pad? Sorry if my questions are
>>>>> silly but as I stated before, I'm a newbie with v4l2 and MCF.
>>>>
>>>> The tvp5150 driver is used on some em28xx devices. It is nice to add auto-detection
>>>> code to the driver, but converting it to the media bus should be done with
>>>> enough care to not break support for the existing devices.
>>>
>>> So in other words, the tvp5150 driver needs both pad and non-pad ops.
>>> Eventually all non-pad variants in subdev drivers should be replaced by the
>>> pad variants so you don't have duplication of ops. But that will take a lot
>>> more work.
>>>
>>>> Also, as I've argued with Laurent before, the expected behavior is that the standards
>>>> format selection should be done via the video node, and not via the media 
>>>> controller node. The V4L2 API has enough support for all you need to do with the
>>>> video decoder, so there's no excuse to duplicate it with any other API.
>>>
>>> This is relevant for bridge drivers, not for subdev drivers.
>>>
>>>> The media controller API is there not to replace V4L2, but to complement
>>>> it where needed.
>>>
>>> That will be a nice discussion during the workshop :-)
>>>
>>>> In the specific code of standards auto-detection, a few drivers currently support
>>>> this feature. They're (or should be) coded to do is:
>>>>
>>>> If V4L2_STD_ALL is used, the driver should autodetect the video standard of the
>>>> currently tuned channel.
>>>
>>> Actually, this is optional. As per the spec:
>>>
>>> "When the standard set is ambiguous drivers may return EINVAL or choose any of
>>> the requested standards."
>>
>> Yes. The entire auto-detection thing is optional. Several devices aren't capable of
>> auto-detecting standards. Btw, userspace support for standards auto-detection is
>> another chapter. I don't think they implement standards auto-detection.
>>
>> There are even some applications (qv4l2 and tvtime, for example) that aren't capable of 
>> refreshing the maximum height when the standard changes.
>>
>>> Nor does the spec say anything about doing an autodetect when STD_ALL is passed
>>> in. Most drivers will just set the std to PAL or NTSC in this case.
>>
>> Auto-detection is device specific. That's why most applications don't handle it 
>> well (or don't even care on trying to do it). Btw, I did some tests with autodetection
>> on a device with a saa7115, and I found some bugs. Just sent a fix for it.
>>
>>> If you want to autodetect, then use QUERYSTD. Applications cannot rely on drivers
>>> to handle V4L2_STD_ALL the way you say.
>>>
>>>> The detected standard can be returned to userspace via VIDIOC_G_STD.
>>>
>>> No! G_STD always returns the current *selected* standard. Only QUERYSTD returns
>>> the detected standard.
>>
>> Yes, you're right. I should not try to answer emails when I'm lazy enough to not
>> look in to the code ;)
>>
>> Anyway, the thing is that V4L2 API has enough support for auto-detection. There's
>> no need to duplicate stuff with MC API.
> 
> It's not MC API but V4L2 subdev API. No-one has proposed to add video
> standard awareness to the Media controller API. There's no reason to export
> a video node in video decoder drivers... but I guess you didn't mean that.
> 
> Would implementing ENUM/G/S_STD make sense for the camera ISP driver, that
> has nothing to do with any video standard?

This is an old discussion, and we never agreed on that. Some webcam drivers 
implement those ioctls. Others don't. Both cases are compliant with the
current specs. In the past, several userspace applications used to abort if those
ioctl's weren't implemented, but I think that this were fixed already there.

As I said, we should define a per-device type profile in order to enforce that
all devices of the same type will do the same. We'll need man power to fix the
ones that aren't compliant, and solve the userspace issues. Volunteers needed.

There's one point to bold on it: devices that can have either an analog input
or a digital input will likely need to implement ENUM/G/S_STD for both, as
userspace applications may fail, if the ioctl's are disabled depending on the
type of input. We had to implement them on several drivers, due to that.

> If you have two video decoders
> connected to your system, then which one should the ioctls be redirected to?
> What if there's a sensor and a video decoder? And how could the user know
> about this?

When an input is selected (either via the V4L2 way - S_INPUT or via the MC/subdev
way, there's just one video decoder or sensor associated to the corresponding
V4L2 node).

> It's the same old issues again... let's discuss this in the Multimedia
> summit.

We can discuss more at the summit, but we should start discussing it here, as
otherwise we may not be able to go into a consensus there, due to the limited
amount of time we would have for each topic.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Mauro,

On Monday 03 October 2011 21:06:14 Mauro Carvalho Chehab wrote:
> Em 03-10-2011 06:53, Javier Martinez Canillas escreveu:
> > On Mon, Oct 3, 2011 at 10:39 AM, Laurent Pinchart wrote:
> >> On Monday 03 October 2011 08:30:25 Hans Verkuil wrote:
> >>> On Monday, October 03, 2011 04:17:06 Mauro Carvalho Chehab wrote:
> >>>> Em 02-10-2011 18:18, Javier Martinez Canillas escreveu:
> >>>>> On Sun, Oct 2, 2011 at 6:30 PM, Sakari Ailus wrote:
> >> [snip]
> >> 
> >>>>>>>  static const struct v4l2_subdev_video_ops tvp5150_video_ops = {
> >>>>>>>  
> >>>>>>>       .s_routing = tvp5150_s_routing,
> >>>>>>> 
> >>>>>>> +     .s_stream = tvp515x_s_stream,
> >>>>>>> +     .enum_mbus_fmt = tvp515x_enum_mbus_fmt,
> >>>>>>> +     .g_mbus_fmt = tvp515x_mbus_fmt,
> >>>>>>> +     .try_mbus_fmt = tvp515x_mbus_fmt,
> >>>>>>> +     .s_mbus_fmt = tvp515x_mbus_fmt,
> >>>>>>> +     .g_parm = tvp515x_g_parm,
> >>>>>>> +     .s_parm = tvp515x_s_parm,
> >>>>>>> +     .s_std_output = tvp5150_s_std,
> >>>>>> 
> >>>>>> Do we really need both video and pad format ops?
> >>>>> 
> >>>>> Good question, I don't know. Can this device be used as a standalone
> >>>>> v4l2 device? Or is supposed to always be a part of a video streaming
> >>>>> pipeline as a sub-device with a source pad? Sorry if my questions are
> >>>>> silly but as I stated before, I'm a newbie with v4l2 and MCF.
> >>>> 
> >>>> The tvp5150 driver is used on some em28xx devices. It is nice to add
> >>>> auto-detection code to the driver, but converting it to the media bus
> >>>> should be done with enough care to not break support for the existing
> >>>> devices.
> >>> 
> >>> So in other words, the tvp5150 driver needs both pad and non-pad ops.
> >>> Eventually all non-pad variants in subdev drivers should be replaced by
> >>> the pad variants so you don't have duplication of ops. But that will
> >>> take a lot more work.
> >> 
> >> What about replacing direct calls to non-pad operations with core V4L2
> >> functions that would use the subdev non-pad operation if available, and
> >> emulate if with the pad operation otherwise ? I think this would ease
> >> the transition, as subdev drivers could be ported to pad operations
> >> without worrying about the bridges that use them, and bridge drivers
> >> could be switched to the new wrappers with a simple search and replace.
> > 
> > Ok, that is a good solution. I'll do that. Implement V4L2 core
> > operations as wrappers of the subdev pad operations.
> 
> As I said, I can't see _any_ reason why setting a format would be needed
> at pad level. Patches shouldn't increase driver/core and userspace
> complexity for nothing.

Sorry ? We already have format setting at the pad level, and that's used by 
drivers and applications. It's one key feature of the V4L2/MC API.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Mauro,

On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
> > On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
> > 
> > [snip]
> > 
> >> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
> >> you are so kind to answer.
> >> 
> >> 1- User-space programs that are not MCF aware negotiate the format
> >> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
> >> pad. But the real format is driven by the analog video format in the
> >> source pad (i.e: tvp5151).
> > 
> > That's not different from existing systems using digital sensors, where
> > the format is driven by the sensor.
> > 
> >> I modified the ISP driver to get the data format from the source pad
> >> and set the format for each pad on the pipeline accordingly but I've
> >> read from the documentation [1] that is not correct to propagate a
> >> data format from source pads to sink pads, that the correct thing is
> >> to do it from sink to source.
> >> 
> >> So, in this case an administrator has to externally configure the
> >> format for each pad and to guarantee a coherent format on the whole
> >> pipeline?.
> > 
> > That's correct (except you don't need to be an administrator to do so
> > :-)).
> 
> NACK.

Double NACK :-D

> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
> that are handling the video/audio standard should be changed, in order to
> obey the V4L2 ioctl. This is what happens with all other drivers since the
> beginning of the V4L1 API. There's no reason to change it, and such change
> would be a regression.

The same could have been told for the format API:

"When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs that 
are handling the video format should be changed, in order to obey the V4L2 
ioctl. This is what happens with all other drivers since the beginning of the 
V4L1 API. There's no reason to change it, and such change would be a 
regression."

But we've introduced a pad-level format API. I don't see any reason to treat 
standard differently.

> >> Or does exist a way to do this automatic?. i.e: The output entity on the
> >> pipeline promotes the capabilities of the source pad so applications can
> >> select a data format and this format gets propagated all over the
> >> pipeline from the sink pad to the source?
> > 
> > It can be automated in userspace (through a libv4l plugin for instance),
> > but it's really not the kernel's job to do so.
> 
> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
> userspace plugin.

And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are 
standards different ?

> >> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
> >> 
> >> 2- If the application want a different format that the default
> >> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
> >> crop the image? I thought this can be made using the CCDC, copying
> >> less lines to memory or the RESIZER if the application wants a bigger
> >> image. What is the best approach for this?
> 
> Not sure if I understood your question, but maybe you're mixing two
> different concepts here.
> 
> If the application wants a different image resolution, it will use S_FMT.
> In this case, what userspace expects is that the driver will scale,
> if supported, or return -EINVAL otherwise.

With the OMAP3 ISP, which is I believe what Javier was asking about, the 
application will set the format on the OMAP3 ISP resizer input and output pads 
to configure scaling.

> Cropping should be done via S_CROP (or, after accepted upstream, via the
> S_SELECTION ioctl).
> 
> > Cropping can be done in the resizer, and I will soon post patches that
> > add cropping support in the preview engine (although that will be
> > useless for the TVP5151, as the preview engine doesn't support YUV
> > data). The CCDC supports cropping too, but that's not implemented in the
> > driver yet.
> > 
> >> 3- When using embedded sync, CCDC doesn't have an external vertical
> >> sync signal, so we have to manually configure when we want the VD0
> >> interrupt to raise. This works for progressive frames, since each
> >> frame has the same size but in the case of interlaced video,
> >> sub-frames have different sizes (i.e: 313 and 312 vertical lines for
> >> PAL).
> >> 
> >> What I did is to reconfigure the CCDC on the VD1 interrupt handler,
> >> but I think this is more a hack than a clean solution. What do you
> >> think is the best approach to solve this?
> > 
> > I *really* wish the CCDC had an end of frame interrupt :-( I'm not sure
> > if there's a non-hackish solution to this.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 03-10-2011 18:44, Laurent Pinchart escreveu:
> Hi Mauro,
> 
> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
>>>
>>> [snip]
>>>
>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
>>>> you are so kind to answer.
>>>>
>>>> 1- User-space programs that are not MCF aware negotiate the format
>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>>>> pad. But the real format is driven by the analog video format in the
>>>> source pad (i.e: tvp5151).
>>>
>>> That's not different from existing systems using digital sensors, where
>>> the format is driven by the sensor.
>>>
>>>> I modified the ISP driver to get the data format from the source pad
>>>> and set the format for each pad on the pipeline accordingly but I've
>>>> read from the documentation [1] that is not correct to propagate a
>>>> data format from source pads to sink pads, that the correct thing is
>>>> to do it from sink to source.
>>>>
>>>> So, in this case an administrator has to externally configure the
>>>> format for each pad and to guarantee a coherent format on the whole
>>>> pipeline?.
>>>
>>> That's correct (except you don't need to be an administrator to do so
>>> :-)).
>>
>> NACK.
> 
> Double NACK :-D
> 
>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
>> that are handling the video/audio standard should be changed, in order to
>> obey the V4L2 ioctl. This is what happens with all other drivers since the
>> beginning of the V4L1 API. There's no reason to change it, and such change
>> would be a regression.
> 
> The same could have been told for the format API:
> 
> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs that 
> are handling the video format should be changed, in order to obey the V4L2 
> ioctl. This is what happens with all other drivers since the beginning of the 
> V4L1 API. There's no reason to change it, and such change would be a 
> regression."
> 
> But we've introduced a pad-level format API. I don't see any reason to treat 
> standard differently.

Neither do I. The pad-level API should not replace the V4L2 API for standard,
for controls and/or for format settings.

>>>> Or does exist a way to do this automatic?. i.e: The output entity on the
>>>> pipeline promotes the capabilities of the source pad so applications can
>>>> select a data format and this format gets propagated all over the
>>>> pipeline from the sink pad to the source?
>>>
>>> It can be automated in userspace (through a libv4l plugin for instance),
>>> but it's really not the kernel's job to do so.
>>
>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
>> userspace plugin.
> 
> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are 
> standards different ?

All v4l media devices have sub-devices with either tv decoders or sensors connected
into a sink. The sink implements the /dev/video? node.
When an ioctl is sent to the v4l node, the sensors/decoders are controlled
to implement whatever is requested: video standards, formats etc.

Changing it would be a major regression. If OMAP3 is not doing the right thing,
it should be fixed, and not the opposite.

The MC/subdev API is there to fill the blanks, e. g. to handle cases where the
same function could be implemented on two different places of the pipeline, e. g.
when both the sensor and the bridge could do scaling, and userspace wants to
explicitly use one, or the other, but it were never meant to replace the V4L2
functionality.

> 
>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>>>
>>>> 2- If the application want a different format that the default
>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
>>>> crop the image? I thought this can be made using the CCDC, copying
>>>> less lines to memory or the RESIZER if the application wants a bigger
>>>> image. What is the best approach for this?
>>
>> Not sure if I understood your question, but maybe you're mixing two
>> different concepts here.
>>
>> If the application wants a different image resolution, it will use S_FMT.
>> In this case, what userspace expects is that the driver will scale,
>> if supported, or return -EINVAL otherwise.
> 
> With the OMAP3 ISP, which is I believe what Javier was asking about, the 
> application will set the format on the OMAP3 ISP resizer input and output pads 
> to configure scaling.

The V4L2 API doesn't tell where a function like scaler will be implemented.
So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
V4L2 call is sent.

I'm ok if you want to offer the possibility of doing scale on the other parts
of the pipeline, as a bonus, via the MC/subdev API, but the absolute minimum
requirement is to implement it via the V4L2 API.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

Hello,

Reading the last emails I understand that still isn't a consensus on
the way this has to be made. If it has to be implemented at the video
device node level or at the sub-device level. And if it has to be made
in kernel or user-space.

On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab
<mchehab@infradead.org> wrote:
> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
>> Hi Mauro,
>>
>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
>>>>
>>>> [snip]
>>>>
>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
>>>>> you are so kind to answer.
>>>>>
>>>>> 1- User-space programs that are not MCF aware negotiate the format
>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>>>>> pad. But the real format is driven by the analog video format in the
>>>>> source pad (i.e: tvp5151).
>>>>
>>>> That's not different from existing systems using digital sensors, where
>>>> the format is driven by the sensor.
>>>>
>>>>> I modified the ISP driver to get the data format from the source pad
>>>>> and set the format for each pad on the pipeline accordingly but I've
>>>>> read from the documentation [1] that is not correct to propagate a
>>>>> data format from source pads to sink pads, that the correct thing is
>>>>> to do it from sink to source.
>>>>>
>>>>> So, in this case an administrator has to externally configure the
>>>>> format for each pad and to guarantee a coherent format on the whole
>>>>> pipeline?.
>>>>
>>>> That's correct (except you don't need to be an administrator to do so
>>>> :-)).
>>>
>>> NACK.
>>
>> Double NACK :-D
>>
>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
>>> that are handling the video/audio standard should be changed, in order to
>>> obey the V4L2 ioctl. This is what happens with all other drivers since the
>>> beginning of the V4L1 API. There's no reason to change it, and such change
>>> would be a regression.
>>
>> The same could have been told for the format API:
>>
>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs that
>> are handling the video format should be changed, in order to obey the V4L2
>> ioctl. This is what happens with all other drivers since the beginning of the
>> V4L1 API. There's no reason to change it, and such change would be a
>> regression."
>>
>> But we've introduced a pad-level format API. I don't see any reason to treat
>> standard differently.
>
> Neither do I. The pad-level API should not replace the V4L2 API for standard,
> for controls and/or for format settings.
>
>>>>> Or does exist a way to do this automatic?. i.e: The output entity on the
>>>>> pipeline promotes the capabilities of the source pad so applications can
>>>>> select a data format and this format gets propagated all over the
>>>>> pipeline from the sink pad to the source?
>>>>
>>>> It can be automated in userspace (through a libv4l plugin for instance),
>>>> but it's really not the kernel's job to do so.
>>>
>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
>>> userspace plugin.
>>
>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are
>> standards different ?
>
> All v4l media devices have sub-devices with either tv decoders or sensors connected
> into a sink. The sink implements the /dev/video? node.
> When an ioctl is sent to the v4l node, the sensors/decoders are controlled
> to implement whatever is requested: video standards, formats etc.
>
> Changing it would be a major regression. If OMAP3 is not doing the right thing,
> it should be fixed, and not the opposite.
>

That is the approach we took, we hack the isp v4l2 device driver
(ispvideo) to bypass the ioctls to the sub-device that as the source
pad (tvp5151 in our case, but it could be a sensor as well). So, for
example the VIDIOC_S_STD ioctl handler looks like this (I post a
simplified version of the code, just to give an idea):

static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id *norm)
{
    struct isp_video *video = video_drvdata(file);
    struct v4l2_subdev *sink_subdev;
    struct v4l2_subdev *source_subdev;

    sink_subdev = isp_video_remote_subdev(video, NULL);
    sink_pad = &sink_subdev->entity.pads[0];
    source_pad = media_entity_remote_source(sink_pad);
    source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
    v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
}

So applications interact with the /dev/video? via V4L2 ioctls whose
handlers call the sub-dev functions. Is that what you propose?

> The MC/subdev API is there to fill the blanks, e. g. to handle cases where the
> same function could be implemented on two different places of the pipeline, e. g.
> when both the sensor and the bridge could do scaling, and userspace wants to
> explicitly use one, or the other, but it were never meant to replace the V4L2
> functionality.
>
>>
>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>>>>
>>>>> 2- If the application want a different format that the default
>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
>>>>> crop the image? I thought this can be made using the CCDC, copying
>>>>> less lines to memory or the RESIZER if the application wants a bigger
>>>>> image. What is the best approach for this?
>>>
>>> Not sure if I understood your question, but maybe you're mixing two
>>> different concepts here.
>>>
>>> If the application wants a different image resolution, it will use S_FMT.
>>> In this case, what userspace expects is that the driver will scale,
>>> if supported, or return -EINVAL otherwise.
>>
>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
>> application will set the format on the OMAP3 ISP resizer input and output pads
>> to configure scaling.
>

Yes, that was my question about. But still is not clear to me if
configuring the ISP resizer input and output pads with different frame
sizes automatically means that I have to do the scale or this has to
be configured using a S_FMT ioctl to the /dev/video? node.

Basically what I don't know is when I have to modify the pipeline
graph inside the ISP driver and when this has to be made from
user-space via MCF.

> The V4L2 API doesn't tell where a function like scaler will be implemented.
> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
> V4L2 call is sent.
>

I don't think I can do the cropping and scaling in the tvp5151 driver
since this is a dumb device, it only spits bytes via its parallel
interface. The actual buffer is inside the ISP.

> Regards,
> Mauro
>

Best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 03-10-2011 19:37, Javier Martinez Canillas escreveu:
> Hello,
> 
> Reading the last emails I understand that still isn't a consensus on
> the way this has to be made.

True.

> If it has to be implemented at the video
> device node level or at the sub-device level. And if it has to be made
> in kernel or user-space.

For now, I propose you to just add/improve the auto-detection on the
existing callbacks. We need to reach a consensus before working at the pad level.

> On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab
> <mchehab@infradead.org> wrote:
>> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
>>> Hi Mauro,
>>>
>>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
>>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
>>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
>>>>>
>>>>> [snip]
>>>>>
>>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
>>>>>> you are so kind to answer.
>>>>>>
>>>>>> 1- User-space programs that are not MCF aware negotiate the format
>>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>>>>>> pad. But the real format is driven by the analog video format in the
>>>>>> source pad (i.e: tvp5151).
>>>>>
>>>>> That's not different from existing systems using digital sensors, where
>>>>> the format is driven by the sensor.
>>>>>
>>>>>> I modified the ISP driver to get the data format from the source pad
>>>>>> and set the format for each pad on the pipeline accordingly but I've
>>>>>> read from the documentation [1] that is not correct to propagate a
>>>>>> data format from source pads to sink pads, that the correct thing is
>>>>>> to do it from sink to source.
>>>>>>
>>>>>> So, in this case an administrator has to externally configure the
>>>>>> format for each pad and to guarantee a coherent format on the whole
>>>>>> pipeline?.
>>>>>
>>>>> That's correct (except you don't need to be an administrator to do so
>>>>> :-)).
>>>>
>>>> NACK.
>>>
>>> Double NACK :-D
>>>
>>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
>>>> that are handling the video/audio standard should be changed, in order to
>>>> obey the V4L2 ioctl. This is what happens with all other drivers since the
>>>> beginning of the V4L1 API. There's no reason to change it, and such change
>>>> would be a regression.
>>>
>>> The same could have been told for the format API:
>>>
>>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs that
>>> are handling the video format should be changed, in order to obey the V4L2
>>> ioctl. This is what happens with all other drivers since the beginning of the
>>> V4L1 API. There's no reason to change it, and such change would be a
>>> regression."
>>>
>>> But we've introduced a pad-level format API. I don't see any reason to treat
>>> standard differently.
>>
>> Neither do I. The pad-level API should not replace the V4L2 API for standard,
>> for controls and/or for format settings.
>>
>>>>>> Or does exist a way to do this automatic?. i.e: The output entity on the
>>>>>> pipeline promotes the capabilities of the source pad so applications can
>>>>>> select a data format and this format gets propagated all over the
>>>>>> pipeline from the sink pad to the source?
>>>>>
>>>>> It can be automated in userspace (through a libv4l plugin for instance),
>>>>> but it's really not the kernel's job to do so.
>>>>
>>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
>>>> userspace plugin.
>>>
>>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are
>>> standards different ?
>>
>> All v4l media devices have sub-devices with either tv decoders or sensors connected
>> into a sink. The sink implements the /dev/video? node.
>> When an ioctl is sent to the v4l node, the sensors/decoders are controlled
>> to implement whatever is requested: video standards, formats etc.
>>
>> Changing it would be a major regression. If OMAP3 is not doing the right thing,
>> it should be fixed, and not the opposite.
>>
> 
> That is the approach we took, we hack the isp v4l2 device driver
> (ispvideo) to bypass the ioctls to the sub-device that as the source
> pad (tvp5151 in our case, but it could be a sensor as well). So, for
> example the VIDIOC_S_STD ioctl handler looks like this (I post a
> simplified version of the code, just to give an idea):
> 
> static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id *norm)
> {
>     struct isp_video *video = video_drvdata(file);
>     struct v4l2_subdev *sink_subdev;
>     struct v4l2_subdev *source_subdev;
> 
>     sink_subdev = isp_video_remote_subdev(video, NULL);
>     sink_pad = &sink_subdev->entity.pads[0];
>     source_pad = media_entity_remote_source(sink_pad);
>     source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
>     v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
> }
> 
> So applications interact with the /dev/video? via V4L2 ioctls whose
> handlers call the sub-dev functions. Is that what you propose?

Something like that. For example:

static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id *norm)
{
	/* Do some sanity test/resolution adjustments, etc */

	v4l2_device_call_all(&dev->v4l2_dev, 0, core, s_std, dev->norm);

	return 0;
}

It should be noticed that:

1) a full standard auto-detection is only possible at the V4L2 level, as a 
standard is a composition of audio and video carriers. I intend to work tomorrow
to add audio auto-detection to pvrusb2;

2) a full s_std should not only adjust the TV demod pad, but also, the audio pad
and the tuner pad (if the analog input comes from a tuner).

>> The MC/subdev API is there to fill the blanks, e. g. to handle cases where the
>> same function could be implemented on two different places of the pipeline, e. g.
>> when both the sensor and the bridge could do scaling, and userspace wants to
>> explicitly use one, or the other, but it were never meant to replace the V4L2
>> functionality.
>>
>>>
>>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>>>>>
>>>>>> 2- If the application want a different format that the default
>>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
>>>>>> crop the image? I thought this can be made using the CCDC, copying
>>>>>> less lines to memory or the RESIZER if the application wants a bigger
>>>>>> image. What is the best approach for this?
>>>>
>>>> Not sure if I understood your question, but maybe you're mixing two
>>>> different concepts here.
>>>>
>>>> If the application wants a different image resolution, it will use S_FMT.
>>>> In this case, what userspace expects is that the driver will scale,
>>>> if supported, or return -EINVAL otherwise.
>>>
>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
>>> application will set the format on the OMAP3 ISP resizer input and output pads
>>> to configure scaling.
>>
> 
> Yes, that was my question about. But still is not clear to me if
> configuring the ISP resizer input and output pads with different frame
> sizes automatically means that I have to do the scale or this has to
> be configured using a S_FMT ioctl to the /dev/video? node.
> 
> Basically what I don't know is when I have to modify the pipeline
> graph inside the ISP driver and when this has to be made from
> user-space via MCF.

In the specific case of analog inputs, In general, better results are obtained
if the scaling is done at the analog demod, as it can play with the pixel
sampling rate, and obtaining a better result than a decimation filter. Not
all demods accept this through.

Anyway, S_FMT is expected to work at the /dev/video? node. Laurent will likely
argument against, but, again, allowing to control the scaling on a different
place is a bonus of the MC/subdev API, but it should never replace the S_FMT
V4L2 call.

>> The V4L2 API doesn't tell where a function like scaler will be implemented.
>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
>> V4L2 call is sent.
>>
> 
> I don't think I can do the cropping and scaling in the tvp5151 driver
> since this is a dumb device, it only spits bytes via its parallel
> interface. The actual buffer is inside the ISP.

I wrote the tvp5150 driver from scratch a long time ago. Can't remember all
details anymore. As far as I can remember, I don't think it has scaling. Also,
its sampler seems to use a fixed pixel clock rate.

It does support crop by adjusting the blank registers. Probably there's a 
limit to the range that cropping can be done, but, in general, it should be enough
to remove the black lines from the borders (and the teletext/cc info).

I can't think on any advantage of doing cropping at tvp5150. So, you can just
implement it where you're more comfortable with.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Hans Verkuil]

On Tuesday, October 04, 2011 07:31:27 Mauro Carvalho Chehab wrote:
> Em 03-10-2011 19:37, Javier Martinez Canillas escreveu:
> > Hello,
> > 
> > Reading the last emails I understand that still isn't a consensus on
> > the way this has to be made.
> 
> True.
> 
> > If it has to be implemented at the video
> > device node level or at the sub-device level. And if it has to be made
> > in kernel or user-space.
> 
> For now, I propose you to just add/improve the auto-detection on the
> existing callbacks. We need to reach a consensus before working at the pad level.

I agree.

> > On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab
> > <mchehab@infradead.org> wrote:
> >> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
> >>> Hi Mauro,
> >>>
> >>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
> >>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
> >>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
> >>>>>
> >>>>> [snip]
> >>>>>
> >>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
> >>>>>> you are so kind to answer.
> >>>>>>
> >>>>>> 1- User-space programs that are not MCF aware negotiate the format
> >>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
> >>>>>> pad. But the real format is driven by the analog video format in the
> >>>>>> source pad (i.e: tvp5151).
> >>>>>
> >>>>> That's not different from existing systems using digital sensors, where
> >>>>> the format is driven by the sensor.
> >>>>>
> >>>>>> I modified the ISP driver to get the data format from the source pad
> >>>>>> and set the format for each pad on the pipeline accordingly but I've
> >>>>>> read from the documentation [1] that is not correct to propagate a
> >>>>>> data format from source pads to sink pads, that the correct thing is
> >>>>>> to do it from sink to source.
> >>>>>>
> >>>>>> So, in this case an administrator has to externally configure the
> >>>>>> format for each pad and to guarantee a coherent format on the whole
> >>>>>> pipeline?.
> >>>>>
> >>>>> That's correct (except you don't need to be an administrator to do so
> >>>>> :-)).
> >>>>
> >>>> NACK.
> >>>
> >>> Double NACK :-D
> >>>
> >>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
> >>>> that are handling the video/audio standard should be changed, in order to
> >>>> obey the V4L2 ioctl. This is what happens with all other drivers since the
> >>>> beginning of the V4L1 API. There's no reason to change it, and such change
> >>>> would be a regression.
> >>>
> >>> The same could have been told for the format API:
> >>>
> >>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs that
> >>> are handling the video format should be changed, in order to obey the V4L2
> >>> ioctl. This is what happens with all other drivers since the beginning of the
> >>> V4L1 API. There's no reason to change it, and such change would be a
> >>> regression."
> >>>
> >>> But we've introduced a pad-level format API. I don't see any reason to treat
> >>> standard differently.
> >>
> >> Neither do I. The pad-level API should not replace the V4L2 API for standard,
> >> for controls and/or for format settings.

Remember we are talking about the subdev driver here. It makes no sense to
have both a s_mbus_fmt video op and a set_fmt pad op, which both do the same
thing. Bridge drivers should be adapted to use set_fmt only, so we can drop
s_mbus_fmt.

BTW, the names 'set_fmt' and 'get_fmt' are very confusing to me. I always think
these refer to v4l2_format. Can we please rename this to g/s_mbus_fmt? And
set/get_crop to s/g_crop? This for consistent naming.

When it comes to S_STD I don't see the need for a pad version of this. It is
an op that is used to configure subdevs to handle a specific standard. If you
are configuring the pipelines manually, then after calling S_STD you have to
set up the mbus formats yourself.

Of course, I can generate scenarios where you would need to set the standard
through the subdev node (i.e. two video receivers connected to a SoC, one
receiving PAL, the other receiving NTSC, and both streams composed into a single
new stream that's DMA-ed to memory), but frankly, those scenarios are very
contrived :-)

The preset ioctls would be more realistic since I know that a scenario like the
one above does exist for e.g. HDMI inputs, where each can receive a different
format.

In that case the preset ioctls would have to be exposed to the subdev nodes,
allowing you to set it up for each HDMI receiver independently. But you still
do not need pads to do this since this is a subdev-level operation, not pad-level.

> >>
> >>>>>> Or does exist a way to do this automatic?. i.e: The output entity on the
> >>>>>> pipeline promotes the capabilities of the source pad so applications can
> >>>>>> select a data format and this format gets propagated all over the
> >>>>>> pipeline from the sink pad to the source?
> >>>>>
> >>>>> It can be automated in userspace (through a libv4l plugin for instance),
> >>>>> but it's really not the kernel's job to do so.
> >>>>
> >>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
> >>>> userspace plugin.
> >>>
> >>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are
> >>> standards different ?
> >>
> >> All v4l media devices have sub-devices with either tv decoders or sensors connected
> >> into a sink. The sink implements the /dev/video? node.
> >> When an ioctl is sent to the v4l node, the sensors/decoders are controlled
> >> to implement whatever is requested: video standards, formats etc.
> >>
> >> Changing it would be a major regression. If OMAP3 is not doing the right thing,
> >> it should be fixed, and not the opposite.
> >>
> > 
> > That is the approach we took, we hack the isp v4l2 device driver
> > (ispvideo) to bypass the ioctls to the sub-device that as the source
> > pad (tvp5151 in our case, but it could be a sensor as well). So, for
> > example the VIDIOC_S_STD ioctl handler looks like this (I post a
> > simplified version of the code, just to give an idea):
> > 
> > static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id *norm)
> > {
> >     struct isp_video *video = video_drvdata(file);
> >     struct v4l2_subdev *sink_subdev;
> >     struct v4l2_subdev *source_subdev;
> > 
> >     sink_subdev = isp_video_remote_subdev(video, NULL);
> >     sink_pad = &sink_subdev->entity.pads[0];
> >     source_pad = media_entity_remote_source(sink_pad);
> >     source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
> >     v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
> > }
> > 
> > So applications interact with the /dev/video? via V4L2 ioctls whose
> > handlers call the sub-dev functions. Is that what you propose?
> 
> Something like that. For example:
> 
> static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id *norm)
> {
> 	/* Do some sanity test/resolution adjustments, etc */
> 
> 	v4l2_device_call_all(&dev->v4l2_dev, 0, core, s_std, dev->norm);
> 
> 	return 0;
> }
> 
> It should be noticed that:
> 
> 1) a full standard auto-detection is only possible at the V4L2 level, as a 
> standard is a composition of audio and video carriers. I intend to work tomorrow
> to add audio auto-detection to pvrusb2;
> 
> 2) a full s_std should not only adjust the TV demod pad, but also, the audio pad
> and the tuner pad (if the analog input comes from a tuner).
> 
> >> The MC/subdev API is there to fill the blanks, e. g. to handle cases where the
> >> same function could be implemented on two different places of the pipeline, e. g.
> >> when both the sensor and the bridge could do scaling, and userspace wants to
> >> explicitly use one, or the other, but it were never meant to replace the V4L2
> >> functionality.
> >>
> >>>
> >>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
> >>>>>>
> >>>>>> 2- If the application want a different format that the default
> >>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
> >>>>>> crop the image? I thought this can be made using the CCDC, copying
> >>>>>> less lines to memory or the RESIZER if the application wants a bigger
> >>>>>> image. What is the best approach for this?
> >>>>
> >>>> Not sure if I understood your question, but maybe you're mixing two
> >>>> different concepts here.
> >>>>
> >>>> If the application wants a different image resolution, it will use S_FMT.
> >>>> In this case, what userspace expects is that the driver will scale,
> >>>> if supported, or return -EINVAL otherwise.
> >>>
> >>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
> >>> application will set the format on the OMAP3 ISP resizer input and output pads
> >>> to configure scaling.
> >>
> > 
> > Yes, that was my question about. But still is not clear to me if
> > configuring the ISP resizer input and output pads with different frame
> > sizes automatically means that I have to do the scale or this has to
> > be configured using a S_FMT ioctl to the /dev/video? node.
> > 
> > Basically what I don't know is when I have to modify the pipeline
> > graph inside the ISP driver and when this has to be made from
> > user-space via MCF.
> 
> In the specific case of analog inputs, In general, better results are obtained
> if the scaling is done at the analog demod, as it can play with the pixel
> sampling rate, and obtaining a better result than a decimation filter. Not
> all demods accept this through.
> 
> Anyway, S_FMT is expected to work at the /dev/video? node. Laurent will likely
> argument against, but, again, allowing to control the scaling on a different
> place is a bonus of the MC/subdev API, but it should never replace the S_FMT
> V4L2 call.

I agree with Mauro, up to a point. One way or another drivers should support S_FMT
through the video nodes. But if someone starts programming the pipeline through
subdev nodes, then I think it is reasonable that trying to call S_FMT would return
EBUSY or something similar.

It has been suggested that S_FMT can be implemented for such systems as a libv4l2
plugin, but I would like to see code doing this first as I am not convinced that
that's the best way to do it.

Regards,

	Hans

> >> The V4L2 API doesn't tell where a function like scaler will be implemented.
> >> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
> >> V4L2 call is sent.
> >>
> > 
> > I don't think I can do the cropping and scaling in the tvp5151 driver
> > since this is a dumb device, it only spits bytes via its parallel
> > interface. The actual buffer is inside the ISP.
> 
> I wrote the tvp5150 driver from scratch a long time ago. Can't remember all
> details anymore. As far as I can remember, I don't think it has scaling. Also,
> its sampler seems to use a fixed pixel clock rate.
> 
> It does support crop by adjusting the blank registers. Probably there's a 
> limit to the range that cropping can be done, but, in general, it should be enough
> to remove the black lines from the borders (and the teletext/cc info).
> 
> I can't think on any advantage of doing cropping at tvp5150. So, you can just
> implement it where you're more comfortable with.
> 
> Regards,
> Mauro
> 

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Tue, Oct 4, 2011 at 7:31 AM, Mauro Carvalho Chehab
<mchehab@infradead.org> wrote:
>
> For now, I propose you to just add/improve the auto-detection on the
> existing callbacks. We need to reach a consensus before working at the pad level.
>

Ok, I'll do that using saa7115 driver as a reference as you suggested.

>> On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab
>> <mchehab@infradead.org> wrote:
>>> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
>>>>>> It can be automated in userspace (through a libv4l plugin for instance),
>>>>>> but it's really not the kernel's job to do so.
>>>>>
>>>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
>>>>> userspace plugin.
>>>>
>>>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are
>>>> standards different ?
>>>
>>> All v4l media devices have sub-devices with either tv decoders or sensors connected
>>> into a sink. The sink implements the /dev/video? node.
>>> When an ioctl is sent to the v4l node, the sensors/decoders are controlled
>>> to implement whatever is requested: video standards, formats etc.
>>>
>>> Changing it would be a major regression. If OMAP3 is not doing the right thing,
>>> it should be fixed, and not the opposite.
>>>
>>
>> That is the approach we took, we hack the isp v4l2 device driver
>> (ispvideo) to bypass the ioctls to the sub-device that as the source
>> pad (tvp5151 in our case, but it could be a sensor as well). So, for
>> example the VIDIOC_S_STD ioctl handler looks like this (I post a
>> simplified version of the code, just to give an idea):
>>
>> static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id *norm)
>> {
>>     struct isp_video *video = video_drvdata(file);
>>     struct v4l2_subdev *sink_subdev;
>>     struct v4l2_subdev *source_subdev;
>>
>>     sink_subdev = isp_video_remote_subdev(video, NULL);
>>     sink_pad = &sink_subdev->entity.pads[0];
>>     source_pad = media_entity_remote_source(sink_pad);
>>     source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
>>     v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
>> }
>>
>> So applications interact with the /dev/video? via V4L2 ioctls whose
>> handlers call the sub-dev functions. Is that what you propose?
>
> Something like that. For example:
>
> static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id *norm)
> {
>        /* Do some sanity test/resolution adjustments, etc */
>
>        v4l2_device_call_all(&dev->v4l2_dev, 0, core, s_std, dev->norm);
>
>        return 0;
> }
>
> It should be noticed that:
>
> 1) a full standard auto-detection is only possible at the V4L2 level, as a
> standard is a composition of audio and video carriers. I intend to work tomorrow
> to add audio auto-detection to pvrusb2;
>
> 2) a full s_std should not only adjust the TV demod pad, but also, the audio pad
> and the tuner pad (if the analog input comes from a tuner).
>

Ok, got it. Thanks.

>>> The MC/subdev API is there to fill the blanks, e. g. to handle cases where the
>>> same function could be implemented on two different places of the pipeline, e. g.
>>> when both the sensor and the bridge could do scaling, and userspace wants to
>>> explicitly use one, or the other, but it were never meant to replace the V4L2
>>> functionality.
>>>
>>>>
>>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
>>>> application will set the format on the OMAP3 ISP resizer input and output pads
>>>> to configure scaling.
>>>
>>
>> Yes, that was my question about. But still is not clear to me if
>> configuring the ISP resizer input and output pads with different frame
>> sizes automatically means that I have to do the scale or this has to
>> be configured using a S_FMT ioctl to the /dev/video? node.
>>
>> Basically what I don't know is when I have to modify the pipeline
>> graph inside the ISP driver and when this has to be made from
>> user-space via MCF.
>
> In the specific case of analog inputs, In general, better results are obtained
> if the scaling is done at the analog demod, as it can play with the pixel
> sampling rate, and obtaining a better result than a decimation filter. Not
> all demods accept this through.
>
> Anyway, S_FMT is expected to work at the /dev/video? node. Laurent will likely
> argument against, but, again, allowing to control the scaling on a different
> place is a bonus of the MC/subdev API, but it should never replace the S_FMT
> V4L2 call.
>
>>> The V4L2 API doesn't tell where a function like scaler will be implemented.
>>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
>>> V4L2 call is sent.
>>>
>>
>> I don't think I can do the cropping and scaling in the tvp5151 driver
>> since this is a dumb device, it only spits bytes via its parallel
>> interface. The actual buffer is inside the ISP.
>
> I wrote the tvp5150 driver from scratch a long time ago. Can't remember all
> details anymore. As far as I can remember, I don't think it has scaling. Also,
> its sampler seems to use a fixed pixel clock rate.
>
> It does support crop by adjusting the blank registers. Probably there's a
> limit to the range that cropping can be done, but, in general, it should be enough
> to remove the black lines from the borders (and the teletext/cc info).
>
> I can't think on any advantage of doing cropping at tvp5150. So, you can just
> implement it where you're more comfortable with.
>
> Regards,
> Mauro
>

Once everyone agree on how G/S_FMT, G/S_STD, etc selection has to be
made and in which level (V4L2 device node, sub-dev, pad, etc) I want
to fix everything that has to be fix on the ISP driver, we don't have
the resources nor the will to maintain an ISP driver fork as an
out-of-tree project :)

Also, I volunteer to do any cleanup/modification/migration that
existing V4L2 drivers will require, as I told you before I'm a newbie
both on V4L2 and MCF but I have some free slots that I can dedicate to
this.

Best regards,

-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 04-10-2011 04:03, Hans Verkuil escreveu:
> On Tuesday, October 04, 2011 07:31:27 Mauro Carvalho Chehab wrote:
>> Em 03-10-2011 19:37, Javier Martinez Canillas escreveu:
>>> Hello,
>>>
>>> Reading the last emails I understand that still isn't a consensus on
>>> the way this has to be made.
>>
>> True.
>>
>>> If it has to be implemented at the video
>>> device node level or at the sub-device level. And if it has to be made
>>> in kernel or user-space.
>>
>> For now, I propose you to just add/improve the auto-detection on the
>> existing callbacks. We need to reach a consensus before working at the pad level.
> 
> I agree.
> 
>>> On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab
>>> <mchehab@infradead.org> wrote:
>>>> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
>>>>> Hi Mauro,
>>>>>
>>>>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
>>>>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
>>>>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
>>>>>>>
>>>>>>> [snip]
>>>>>>>
>>>>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
>>>>>>>> you are so kind to answer.
>>>>>>>>
>>>>>>>> 1- User-space programs that are not MCF aware negotiate the format
>>>>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>>>>>>>> pad. But the real format is driven by the analog video format in the
>>>>>>>> source pad (i.e: tvp5151).
>>>>>>>
>>>>>>> That's not different from existing systems using digital sensors, where
>>>>>>> the format is driven by the sensor.
>>>>>>>
>>>>>>>> I modified the ISP driver to get the data format from the source pad
>>>>>>>> and set the format for each pad on the pipeline accordingly but I've
>>>>>>>> read from the documentation [1] that is not correct to propagate a
>>>>>>>> data format from source pads to sink pads, that the correct thing is
>>>>>>>> to do it from sink to source.
>>>>>>>>
>>>>>>>> So, in this case an administrator has to externally configure the
>>>>>>>> format for each pad and to guarantee a coherent format on the whole
>>>>>>>> pipeline?.
>>>>>>>
>>>>>>> That's correct (except you don't need to be an administrator to do so
>>>>>>> :-)).
>>>>>>
>>>>>> NACK.
>>>>>
>>>>> Double NACK :-D
>>>>>
>>>>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
>>>>>> that are handling the video/audio standard should be changed, in order to
>>>>>> obey the V4L2 ioctl. This is what happens with all other drivers since the
>>>>>> beginning of the V4L1 API. There's no reason to change it, and such change
>>>>>> would be a regression.
>>>>>
>>>>> The same could have been told for the format API:
>>>>>
>>>>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs that
>>>>> are handling the video format should be changed, in order to obey the V4L2
>>>>> ioctl. This is what happens with all other drivers since the beginning of the
>>>>> V4L1 API. There's no reason to change it, and such change would be a
>>>>> regression."
>>>>>
>>>>> But we've introduced a pad-level format API. I don't see any reason to treat
>>>>> standard differently.
>>>>
>>>> Neither do I. The pad-level API should not replace the V4L2 API for standard,
>>>> for controls and/or for format settings.
> 
> Remember we are talking about the subdev driver here. It makes no sense to
> have both a s_mbus_fmt video op and a set_fmt pad op, which both do the same
> thing. Bridge drivers should be adapted to use set_fmt only, so we can drop
> s_mbus_fmt.

Agreed.

> BTW, the names 'set_fmt' and 'get_fmt' are very confusing to me. I always think
> these refer to v4l2_format. Can we please rename this to g/s_mbus_fmt? And
> set/get_crop to s/g_crop? This for consistent naming.
> 
> When it comes to S_STD I don't see the need for a pad version of this. It is
> an op that is used to configure subdevs to handle a specific standard. If you
> are configuring the pipelines manually, then after calling S_STD you have to
> set up the mbus formats yourself.
> 
> Of course, I can generate scenarios where you would need to set the standard
> through the subdev node (i.e. two video receivers connected to a SoC, one
> receiving PAL, the other receiving NTSC, and both streams composed into a single
> new stream that's DMA-ed to memory), but frankly, those scenarios are very
> contrived :-)

Even a very rare scenario like the above could be easily implemented with V4L2 API,
using 3 video nodes, one for each receiver, and another for the composed stream.
A device like ivtv or cx25821 probably can do that already, maybe with a few
adjustments at the driver.
 
> The preset ioctls would be more realistic since I know that a scenario like the
> one above does exist for e.g. HDMI inputs, where each can receive a different
> format.
> 
> In that case the preset ioctls would have to be exposed to the subdev nodes,
> allowing you to set it up for each HDMI receiver independently. But you still
> do not need pads to do this since this is a subdev-level operation, not pad-level.
> 
>>>>
>>>>>>>> Or does exist a way to do this automatic?. i.e: The output entity on the
>>>>>>>> pipeline promotes the capabilities of the source pad so applications can
>>>>>>>> select a data format and this format gets propagated all over the
>>>>>>>> pipeline from the sink pad to the source?
>>>>>>>
>>>>>>> It can be automated in userspace (through a libv4l plugin for instance),
>>>>>>> but it's really not the kernel's job to do so.
>>>>>>
>>>>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to any
>>>>>> userspace plugin.
>>>>>
>>>>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are
>>>>> standards different ?
>>>>
>>>> All v4l media devices have sub-devices with either tv decoders or sensors connected
>>>> into a sink. The sink implements the /dev/video? node.
>>>> When an ioctl is sent to the v4l node, the sensors/decoders are controlled
>>>> to implement whatever is requested: video standards, formats etc.
>>>>
>>>> Changing it would be a major regression. If OMAP3 is not doing the right thing,
>>>> it should be fixed, and not the opposite.
>>>>
>>>
>>> That is the approach we took, we hack the isp v4l2 device driver
>>> (ispvideo) to bypass the ioctls to the sub-device that as the source
>>> pad (tvp5151 in our case, but it could be a sensor as well). So, for
>>> example the VIDIOC_S_STD ioctl handler looks like this (I post a
>>> simplified version of the code, just to give an idea):
>>>
>>> static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id *norm)
>>> {
>>>     struct isp_video *video = video_drvdata(file);
>>>     struct v4l2_subdev *sink_subdev;
>>>     struct v4l2_subdev *source_subdev;
>>>
>>>     sink_subdev = isp_video_remote_subdev(video, NULL);
>>>     sink_pad = &sink_subdev->entity.pads[0];
>>>     source_pad = media_entity_remote_source(sink_pad);
>>>     source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
>>>     v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
>>> }
>>>
>>> So applications interact with the /dev/video? via V4L2 ioctls whose
>>> handlers call the sub-dev functions. Is that what you propose?
>>
>> Something like that. For example:
>>
>> static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id *norm)
>> {
>> 	/* Do some sanity test/resolution adjustments, etc */
>>
>> 	v4l2_device_call_all(&dev->v4l2_dev, 0, core, s_std, dev->norm);
>>
>> 	return 0;
>> }
>>
>> It should be noticed that:
>>
>> 1) a full standard auto-detection is only possible at the V4L2 level, as a 
>> standard is a composition of audio and video carriers. I intend to work tomorrow
>> to add audio auto-detection to pvrusb2;
>>
>> 2) a full s_std should not only adjust the TV demod pad, but also, the audio pad
>> and the tuner pad (if the analog input comes from a tuner).
>>
>>>> The MC/subdev API is there to fill the blanks, e. g. to handle cases where the
>>>> same function could be implemented on two different places of the pipeline, e. g.
>>>> when both the sensor and the bridge could do scaling, and userspace wants to
>>>> explicitly use one, or the other, but it were never meant to replace the V4L2
>>>> functionality.
>>>>
>>>>>
>>>>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>>>>>>>
>>>>>>>> 2- If the application want a different format that the default
>>>>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
>>>>>>>> crop the image? I thought this can be made using the CCDC, copying
>>>>>>>> less lines to memory or the RESIZER if the application wants a bigger
>>>>>>>> image. What is the best approach for this?
>>>>>>
>>>>>> Not sure if I understood your question, but maybe you're mixing two
>>>>>> different concepts here.
>>>>>>
>>>>>> If the application wants a different image resolution, it will use S_FMT.
>>>>>> In this case, what userspace expects is that the driver will scale,
>>>>>> if supported, or return -EINVAL otherwise.
>>>>>
>>>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
>>>>> application will set the format on the OMAP3 ISP resizer input and output pads
>>>>> to configure scaling.
>>>>
>>>
>>> Yes, that was my question about. But still is not clear to me if
>>> configuring the ISP resizer input and output pads with different frame
>>> sizes automatically means that I have to do the scale or this has to
>>> be configured using a S_FMT ioctl to the /dev/video? node.
>>>
>>> Basically what I don't know is when I have to modify the pipeline
>>> graph inside the ISP driver and when this has to be made from
>>> user-space via MCF.
>>
>> In the specific case of analog inputs, In general, better results are obtained
>> if the scaling is done at the analog demod, as it can play with the pixel
>> sampling rate, and obtaining a better result than a decimation filter. Not
>> all demods accept this through.
>>
>> Anyway, S_FMT is expected to work at the /dev/video? node. Laurent will likely
>> argument against, but, again, allowing to control the scaling on a different
>> place is a bonus of the MC/subdev API, but it should never replace the S_FMT
>> V4L2 call.
> 
> I agree with Mauro, up to a point. One way or another drivers should support S_FMT
> through the video nodes. But if someone starts programming the pipeline through
> subdev nodes, then I think it is reasonable that trying to call S_FMT would return
> EBUSY or something similar.
> 
> It has been suggested that S_FMT can be implemented for such systems as a libv4l2
> plugin, but I would like to see code doing this first as I am not convinced that
> that's the best way to do it.
> 
> Regards,
> 
> 	Hans
> 
>>>> The V4L2 API doesn't tell where a function like scaler will be implemented.
>>>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
>>>> V4L2 call is sent.
>>>>
>>>
>>> I don't think I can do the cropping and scaling in the tvp5151 driver
>>> since this is a dumb device, it only spits bytes via its parallel
>>> interface. The actual buffer is inside the ISP.
>>
>> I wrote the tvp5150 driver from scratch a long time ago. Can't remember all
>> details anymore. As far as I can remember, I don't think it has scaling. Also,
>> its sampler seems to use a fixed pixel clock rate.
>>
>> It does support crop by adjusting the blank registers. Probably there's a 
>> limit to the range that cropping can be done, but, in general, it should be enough
>> to remove the black lines from the borders (and the teletext/cc info).
>>
>> I can't think on any advantage of doing cropping at tvp5150. So, you can just
>> implement it where you're more comfortable with.
>>
>> Regards,
>> Mauro
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Mauro,

On Monday 03 October 2011 23:56:06 Mauro Carvalho Chehab wrote:
> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
> > On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
> >> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
> >>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
> >>> 
> >>> [snip]
> >>> 
> >>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
> >>>> you are so kind to answer.
> >>>> 
> >>>> 1- User-space programs that are not MCF aware negotiate the format
> >>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
> >>>> pad. But the real format is driven by the analog video format in the
> >>>> source pad (i.e: tvp5151).
> >>> 
> >>> That's not different from existing systems using digital sensors, where
> >>> the format is driven by the sensor.
> >>> 
> >>>> I modified the ISP driver to get the data format from the source pad
> >>>> and set the format for each pad on the pipeline accordingly but I've
> >>>> read from the documentation [1] that is not correct to propagate a
> >>>> data format from source pads to sink pads, that the correct thing is
> >>>> to do it from sink to source.
> >>>> 
> >>>> So, in this case an administrator has to externally configure the
> >>>> format for each pad and to guarantee a coherent format on the whole
> >>>> pipeline?.
> >>> 
> >>> That's correct (except you don't need to be an administrator to do so
> >>> 
> >>> :-)).
> >> 
> >> NACK.
> > 
> > Double NACK :-D
> > 
> >> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
> >> that are handling the video/audio standard should be changed, in order
> >> to obey the V4L2 ioctl. This is what happens with all other drivers
> >> since the beginning of the V4L1 API. There's no reason to change it,
> >> and such change would be a regression.
> > 
> > The same could have been told for the format API:
> > 
> > "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs
> > that are handling the video format should be changed, in order to obey
> > the V4L2 ioctl. This is what happens with all other drivers since the
> > beginning of the V4L1 API. There's no reason to change it, and such
> > change would be a regression."
> > 
> > But we've introduced a pad-level format API. I don't see any reason to
> > treat standard differently.
> 
> Neither do I. The pad-level API should not replace the V4L2 API for
> standard, for controls and/or for format settings.

The pad-level API doesn't replace the V4L2 API, it complements it. I'm of 
course not advocating modifying any driver to replace V4L2 ioctls by direct 
subdev access. However, the pad-level API needs to be exposed to userspace, as 
some harware simply can't be configured through a video node only.

As Hans also mentionned in his reply, the pad-level API is made of two parts: 
an in-kernel API made of subdev operations, and a userspace API accessed 
through ioctls. As the userspace API is needed for some devices, the kernel 
API needs to be implemented by drivers. We should phase out the non pad-level 
format operations in favor of pad-level operations, as the former can be 
implemented using the later. That has absolutely no influence on the userspace 
API.

> >>>> Or does exist a way to do this automatic?. i.e: The output entity on
> >>>> the pipeline promotes the capabilities of the source pad so
> >>>> applications can select a data format and this format gets propagated
> >>>> all over the pipeline from the sink pad to the source?
> >>> 
> >>> It can be automated in userspace (through a libv4l plugin for
> >>> instance), but it's really not the kernel's job to do so.
> >> 
> >> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to
> >> any userspace plugin.
> > 
> > And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are
> > standards different ?
> 
> All v4l media devices have sub-devices with either tv decoders or sensors
> connected into a sink. The sink implements the /dev/video? node.
> When an ioctl is sent to the v4l node, the sensors/decoders are controlled
> to implement whatever is requested: video standards, formats etc.

But that simply doesn't work out with the OMAP3 ISP or similar hardware. If 
applications only set the format on the video node the driver has no way to 
know how to configure the individual subdevs in the pipeline. When more than a 
handful of subdevs are involved, the approach simply doesn't scale.

On the other hand, when the TV decoder is connected to a bridge that is only 
capable of transferring the data to memory, there is no reason to require 
applications to configure subdevs directly. In that case the bridge driver can 
configure the subdevs itself.

> Changing it would be a major regression. If OMAP3 is not doing the right
> thing, it should be fixed, and not the opposite.
> 
> The MC/subdev API is there to fill the blanks, e. g. to handle cases where
> the same function could be implemented on two different places of the
> pipeline, e. g. when both the sensor and the bridge could do scaling, and
> userspace wants to explicitly use one, or the other, but it were never
> meant to replace the V4L2 functionality.

Isn't that what I've been saying ? :-)

> >>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
> >>>> 
> >>>> 2- If the application want a different format that the default
> >>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
> >>>> crop the image? I thought this can be made using the CCDC, copying
> >>>> less lines to memory or the RESIZER if the application wants a bigger
> >>>> image. What is the best approach for this?
> >> 
> >> Not sure if I understood your question, but maybe you're mixing two
> >> different concepts here.
> >> 
> >> If the application wants a different image resolution, it will use
> >> S_FMT. In this case, what userspace expects is that the driver will
> >> scale, if supported, or return -EINVAL otherwise.
> > 
> > With the OMAP3 ISP, which is I believe what Javier was asking about, the
> > application will set the format on the OMAP3 ISP resizer input and output
> > pads to configure scaling.
> 
> The V4L2 API doesn't tell where a function like scaler will be implemented.
> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
> V4L2 call is sent.

By rolling a dice ? :-)

> I'm ok if you want to offer the possibility of doing scale on the other
> parts of the pipeline, as a bonus, via the MC/subdev API, but the absolute
> minimum requirement is to implement it via the V4L2 API.

The absolute minimum requirement is to make the driver usable by applications 
using the V4L2 API in a crippled mode with only a subdev of features 
available. For the OMAP3 ISP that means full-size capture, with no scaling.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Javier,

On Tuesday 04 October 2011 00:37:27 Javier Martinez Canillas wrote:
> Hello,
> 
> Reading the last emails I understand that still isn't a consensus on
> the way this has to be made. If it has to be implemented at the video
> device node level or at the sub-device level. And if it has to be made
> in kernel or user-space.
> 
> On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab wrote:
> > Em 03-10-2011 18:44, Laurent Pinchart escreveu:
> >> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
> >>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
> >>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:

[snip]

> >> With the OMAP3 ISP, which is I believe what Javier was asking about, the
> >> application will set the format on the OMAP3 ISP resizer input and
> >> output pads to configure scaling.
> 
> Yes, that was my question about. But still is not clear to me if
> configuring the ISP resizer input and output pads with different frame
> sizes automatically means that I have to do the scale or this has to
> be configured using a S_FMT ioctl to the /dev/video? node.

The resizer is completely controlled through the formats at its sink and 
source pads. It will scale the image to achieve what is configured at its 
source pad (with x1/4..x4 limits in the zoom ratio).

> Basically what I don't know is when I have to modify the pipeline graph
> inside the ISP driver and when this has to be made from user-space via MCF.

The pipeline needs to be configured before you start video capture. This means 
setting the links according to your use case, and configuring the formats on 
pads. You will then be able to use a pure V4L2 application to capture video.

> > The V4L2 API doesn't tell where a function like scaler will be
> > implemented. So, it is fine to implement it at tvp5151 or at the omap3
> > resizer, when a V4L2 call is sent.
> 
> I don't think I can do the cropping and scaling in the tvp5151 driver
> since this is a dumb device, it only spits bytes via its parallel
> interface. The actual buffer is inside the ISP.

Cropping might be possible (I'm not too familiar with the tvp5151), but 
scaling indeed isn't. That doesn't matter much though.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Hans,

On Tuesday 04 October 2011 09:03:59 Hans Verkuil wrote:
> On Tuesday, October 04, 2011 07:31:27 Mauro Carvalho Chehab wrote:
> > Em 03-10-2011 19:37, Javier Martinez Canillas escreveu:
> > > On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab wrote:
> > >> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
> > >>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
> > >>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
> > >>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
> > >>>>> 
> > >>>>> [snip]
> > >>>>> 
> > >>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver
> > >>>>>> if you are so kind to answer.
> > >>>>>> 
> > >>>>>> 1- User-space programs that are not MCF aware negotiate the format
> > >>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
> > >>>>>> pad. But the real format is driven by the analog video format in
> > >>>>>> the source pad (i.e: tvp5151).
> > >>>>> 
> > >>>>> That's not different from existing systems using digital sensors,
> > >>>>> where the format is driven by the sensor.
> > >>>>> 
> > >>>>>> I modified the ISP driver to get the data format from the source
> > >>>>>> pad and set the format for each pad on the pipeline accordingly
> > >>>>>> but I've read from the documentation [1] that is not correct to
> > >>>>>> propagate a data format from source pads to sink pads, that the
> > >>>>>> correct thing is to do it from sink to source.
> > >>>>>> 
> > >>>>>> So, in this case an administrator has to externally configure the
> > >>>>>> format for each pad and to guarantee a coherent format on the
> > >>>>>> whole pipeline?.
> > >>>>> 
> > >>>>> That's correct (except you don't need to be an administrator to do
> > >>>>> so
> > >>>>> 
> > >>>>> :-)).
> > >>>> 
> > >>>> NACK.
> > >>> 
> > >>> Double NACK :-D
> > >>> 
> > >>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the
> > >>>> subdevs that are handling the video/audio standard should be
> > >>>> changed, in order to obey the V4L2 ioctl. This is what happens with
> > >>>> all other drivers since the beginning of the V4L1 API. There's no
> > >>>> reason to change it, and such change would be a regression.
> > >>> 
> > >>> The same could have been told for the format API:
> > >>> 
> > >>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the
> > >>> subdevs that are handling the video format should be changed, in
> > >>> order to obey the V4L2 ioctl. This is what happens with all other
> > >>> drivers since the beginning of the V4L1 API. There's no reason to
> > >>> change it, and such change would be a regression."
> > >>> 
> > >>> But we've introduced a pad-level format API. I don't see any reason
> > >>> to treat standard differently.
> > >> 
> > >> Neither do I. The pad-level API should not replace the V4L2 API for
> > >> standard, for controls and/or for format settings.
> 
> Remember we are talking about the subdev driver here. It makes no sense to
> have both a s_mbus_fmt video op and a set_fmt pad op, which both do the
> same thing. Bridge drivers should be adapted to use set_fmt only, so we
> can drop s_mbus_fmt.
> 
> BTW, the names 'set_fmt' and 'get_fmt' are very confusing to me. I always
> think these refer to v4l2_format. Can we please rename this to
> g/s_mbus_fmt?

Shouldn't we do it the other way around, renaming the g/s_mbus_fmt video 
operations to g/s_fmt, now that we only have mbus formats left ? The operation 
names would be shorter.

> And set/get_crop to s/g_crop? This for consistent naming.

set/get_crop will be replaced with set/get_selection (or s/g_selection if you 
like that better :-)).

> When it comes to S_STD I don't see the need for a pad version of this. It
> is an op that is used to configure subdevs to handle a specific standard.
> If you are configuring the pipelines manually, then after calling S_STD
> you have to set up the mbus formats yourself.
> 
> Of course, I can generate scenarios where you would need to set the
> standard through the subdev node (i.e. two video receivers connected to a
> SoC, one receiving PAL, the other receiving NTSC, and both streams
> composed into a single new stream that's DMA-ed to memory), but frankly,
> those scenarios are very contrived :-)

Unless I don't understand this correctly, I think those two paragraphs are not 
related.

Regarding the pad-level standard operations, you're probably right. Subdevs 
that can handle two or more analog streams at the same time would need that, 
but we probably won't need to support them any time soon (if ever). So we 
could keep the subdev-level standard operations, with the implicit (or 
explicit) rule that they apply to the currently selected input of the subdev 
(I suppose TV decoders with an input mux are not uncommon).

Regarding controlling standards directly on subdevs, I think that's the way to 
go for complex pipelines, but that doesn't require pad-level standard 
operations.

> The preset ioctls would be more realistic since I know that a scenario like
> the one above does exist for e.g. HDMI inputs, where each can receive a
> different format.
> 
> In that case the preset ioctls would have to be exposed to the subdev
> nodes, allowing you to set it up for each HDMI receiver independently. But
> you still do not need pads to do this since this is a subdev-level
> operation, not pad-level.
>
> > >>>>>> Or does exist a way to do this automatic?. i.e: The output entity
> > >>>>>> on the pipeline promotes the capabilities of the source pad so
> > >>>>>> applications can select a data format and this format gets
> > >>>>>> propagated all over the pipeline from the sink pad to the source?
> > >>>>> 
> > >>>>> It can be automated in userspace (through a libv4l plugin for
> > >>>>> instance), but it's really not the kernel's job to do so.
> > >>>> 
> > >>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left
> > >>>> to any userspace plugin.
> > >>> 
> > >>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why
> > >>> are standards different ?
> > >> 
> > >> All v4l media devices have sub-devices with either tv decoders or
> > >> sensors connected into a sink. The sink implements the /dev/video?
> > >> node.
> > >> When an ioctl is sent to the v4l node, the sensors/decoders are
> > >> controlled to implement whatever is requested: video standards,
> > >> formats etc.
> > >> 
> > >> Changing it would be a major regression. If OMAP3 is not doing the
> > >> right thing, it should be fixed, and not the opposite.
> > > 
> > > That is the approach we took, we hack the isp v4l2 device driver
> > > (ispvideo) to bypass the ioctls to the sub-device that as the source
> > > pad (tvp5151 in our case, but it could be a sensor as well). So, for
> > > example the VIDIOC_S_STD ioctl handler looks like this (I post a
> > > simplified version of the code, just to give an idea):
> > > 
> > > static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id
> > > *norm) {
> > > 
> > >     struct isp_video *video = video_drvdata(file);
> > >     struct v4l2_subdev *sink_subdev;
> > >     struct v4l2_subdev *source_subdev;
> > >     
> > >     sink_subdev = isp_video_remote_subdev(video, NULL);
> > >     sink_pad = &sink_subdev->entity.pads[0];
> > >     source_pad = media_entity_remote_source(sink_pad);
> > >     source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
> > >     v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
> > > 
> > > }
> > > 
> > > So applications interact with the /dev/video? via V4L2 ioctls whose
> > > handlers call the sub-dev functions. Is that what you propose?
> > 
> > Something like that. For example:
> > 
> > static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id *norm)
> > {
> > 
> > 	/* Do some sanity test/resolution adjustments, etc */
> > 	
> > 	v4l2_device_call_all(&dev->v4l2_dev, 0, core, s_std, dev->norm);
> > 	
> > 	return 0;
> > 
> > }
> > 
> > It should be noticed that:
> > 
> > 1) a full standard auto-detection is only possible at the V4L2 level, as
> > a standard is a composition of audio and video carriers. I intend to
> > work tomorrow to add audio auto-detection to pvrusb2;
> > 
> > 2) a full s_std should not only adjust the TV demod pad, but also, the
> > audio pad and the tuner pad (if the analog input comes from a tuner).
> > 
> > >> The MC/subdev API is there to fill the blanks, e. g. to handle cases
> > >> where the same function could be implemented on two different places
> > >> of the pipeline, e. g. when both the sensor and the bridge could do
> > >> scaling, and userspace wants to explicitly use one, or the other, but
> > >> it were never meant to replace the V4L2 functionality.
> > >> 
> > >>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
> > >>>>>> 
> > >>>>>> 2- If the application want a different format that the default
> > >>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have
> > >>>>>> to crop the image? I thought this can be made using the CCDC,
> > >>>>>> copying less lines to memory or the RESIZER if the application
> > >>>>>> wants a bigger image. What is the best approach for this?
> > >>>> 
> > >>>> Not sure if I understood your question, but maybe you're mixing two
> > >>>> different concepts here.
> > >>>> 
> > >>>> If the application wants a different image resolution, it will use
> > >>>> S_FMT. In this case, what userspace expects is that the driver will
> > >>>> scale, if supported, or return -EINVAL otherwise.
> > >>> 
> > >>> With the OMAP3 ISP, which is I believe what Javier was asking about,
> > >>> the application will set the format on the OMAP3 ISP resizer input
> > >>> and output pads to configure scaling.
> > > 
> > > Yes, that was my question about. But still is not clear to me if
> > > configuring the ISP resizer input and output pads with different frame
> > > sizes automatically means that I have to do the scale or this has to
> > > be configured using a S_FMT ioctl to the /dev/video? node.
> > > 
> > > Basically what I don't know is when I have to modify the pipeline
> > > graph inside the ISP driver and when this has to be made from
> > > user-space via MCF.
> > 
> > In the specific case of analog inputs, In general, better results are
> > obtained if the scaling is done at the analog demod, as it can play with
> > the pixel sampling rate, and obtaining a better result than a decimation
> > filter. Not all demods accept this through.
> > 
> > Anyway, S_FMT is expected to work at the /dev/video? node. Laurent will
> > likely argument against, but, again, allowing to control the scaling on
> > a different place is a bonus of the MC/subdev API, but it should never
> > replace the S_FMT V4L2 call.
> 
> I agree with Mauro, up to a point. One way or another drivers should
> support S_FMT through the video nodes.

When there's a one-to-one relationship between the video device (in the 
hardware sense) and the video node, that's pretty normal. However, when a 
video device is made of a complex pipeline and several video nodes, the 
problem becomes much more complex and mapping a "simple" operations such as 
S_FMT to hardware configuration becomes a policy decision.

> But if someone starts programming the pipeline through subdev nodes, then I
> think it is reasonable that trying to call S_FMT would return EBUSY or
> something similar.
>
> It has been suggested that S_FMT can be implemented for such systems as a
> libv4l2 plugin, but I would like to see code doing this first as I am not
> convinced that that's the best way to do it.

Sakari, what's the status of the OMAP3 ISP libv4l plugin ?

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 05-10-2011 17:08, Laurent Pinchart escreveu:
> Hi Mauro,
>
> On Monday 03 October 2011 23:56:06 Mauro Carvalho Chehab wrote:
>> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
>>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
>>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
>>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
>>>>>
>>>>> [snip]
>>>>>
>>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver if
>>>>>> you are so kind to answer.
>>>>>>
>>>>>> 1- User-space programs that are not MCF aware negotiate the format
>>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>>>>>> pad. But the real format is driven by the analog video format in the
>>>>>> source pad (i.e: tvp5151).
>>>>>
>>>>> That's not different from existing systems using digital sensors, where
>>>>> the format is driven by the sensor.
>>>>>
>>>>>> I modified the ISP driver to get the data format from the source pad
>>>>>> and set the format for each pad on the pipeline accordingly but I've
>>>>>> read from the documentation [1] that is not correct to propagate a
>>>>>> data format from source pads to sink pads, that the correct thing is
>>>>>> to do it from sink to source.
>>>>>>
>>>>>> So, in this case an administrator has to externally configure the
>>>>>> format for each pad and to guarantee a coherent format on the whole
>>>>>> pipeline?.
>>>>>
>>>>> That's correct (except you don't need to be an administrator to do so
>>>>>
>>>>> :-)).
>>>>
>>>> NACK.
>>>
>>> Double NACK :-D
>>>
>>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the subdevs
>>>> that are handling the video/audio standard should be changed, in order
>>>> to obey the V4L2 ioctl. This is what happens with all other drivers
>>>> since the beginning of the V4L1 API. There's no reason to change it,
>>>> and such change would be a regression.
>>>
>>> The same could have been told for the format API:
>>>
>>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the subdevs
>>> that are handling the video format should be changed, in order to obey
>>> the V4L2 ioctl. This is what happens with all other drivers since the
>>> beginning of the V4L1 API. There's no reason to change it, and such
>>> change would be a regression."
>>>
>>> But we've introduced a pad-level format API. I don't see any reason to
>>> treat standard differently.
>>
>> Neither do I. The pad-level API should not replace the V4L2 API for
>> standard, for controls and/or for format settings.
>
> The pad-level API doesn't replace the V4L2 API, it complements it. I'm of
> course not advocating modifying any driver to replace V4L2 ioctls by direct
> subdev access. However, the pad-level API needs to be exposed to userspace, as
> some harware simply can't be configured through a video node only.
>
> As Hans also mentionned in his reply, the pad-level API is made of two parts:
> an in-kernel API made of subdev operations, and a userspace API accessed
> through ioctls. As the userspace API is needed for some devices, the kernel
> API needs to be implemented by drivers. We should phase out the non pad-level
> format operations in favor of pad-level operations, as the former can be
> implemented using the later. That has absolutely no influence on the userspace
> API.

What I'm seeing is that:
	- the drivers that are implementing the MC/pad API's aren't
compatible with generic V4L2 applications;
	- there's no way to write a generic application that works with all
drivers, even implementing MC/pad API's there as each driver is taking different
a approach on how to map things at the different API's, and pipeline configuration
is device-dependent;
	- there's no effort to publish patches to libv4l to match the changes
at the kernel driver.

At the time I tried to test the Samsung drivers, I failed to see how this would
work, e. g. how an application like tvtime or xawtv could be used with such
devices. I don't have here any omap3 hardware with cameras or tv tuners, but I
don't see any patches at libv4l that allows using V4L2 to communicate with an
omap3 hardware.

So, in practice, there's no device abstraction that would expose those
hardware to userspace in a manner that the application developer could write
a generic application that would work for all hardware.

I'm fine on providing raw interfaces, just like we have for some types of device
(like the block raw interfaces used by CD-ROM drivers) as a bonus, but this should
never replace an API where an application developed by a third party could work
with all media hardware, without needing hardware specific details.

So, let's take a more pragmatic approach: please provide me means where I test those
drivers with a real hardware, using generic applications (xawtv, tvtime, camorama,
qv4l2, etc), and being able to double check that they will behave just like any
other existing driver.

>>>>>> Or does exist a way to do this automatic?. i.e: The output entity on
>>>>>> the pipeline promotes the capabilities of the source pad so
>>>>>> applications can select a data format and this format gets propagated
>>>>>> all over the pipeline from the sink pad to the source?
>>>>>
>>>>> It can be automated in userspace (through a libv4l plugin for
>>>>> instance), but it's really not the kernel's job to do so.
>>>>
>>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left to
>>>> any userspace plugin.
>>>
>>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why are
>>> standards different ?
>>
>> All v4l media devices have sub-devices with either tv decoders or sensors
>> connected into a sink. The sink implements the /dev/video? node.
>> When an ioctl is sent to the v4l node, the sensors/decoders are controlled
>> to implement whatever is requested: video standards, formats etc.
>
> But that simply doesn't work out with the OMAP3 ISP or similar hardware. If
> applications only set the format on the video node the driver has no way to
> know how to configure the individual subdevs in the pipeline. When more than a
> handful of subdevs are involved, the approach simply doesn't scale.
>
> On the other hand, when the TV decoder is connected to a bridge that is only
> capable of transferring the data to memory, there is no reason to require
> applications to configure subdevs directly. In that case the bridge driver can
> configure the subdevs itself.

For good or for bad, an analog TV standard weren't mapped on V4L on its canonical
form. One single bitmask carries 3 different types of information inside:
	- the analog TV monochromatic standard:
		for 50 Hz line frequency: /A /B /D /K, ...
		for 60 Hz line frequency: /M /60
	- the analog TV chroma standard: PAL, NTSC, SECAM;
	- the audio standard: MTS/BTSC, EIA-J, NICAM, A2.

Due to that, any operation that touches on it should not be done at PAD level, as
the entire pipeline needs to know when something has changed there.

For example, a S_STD call needs to change the tuner saw filters (as the analog
monochromatic standard will need to adjust to the channel bandwidth, and, on some
digital tuners, to the audio and chroma subcarrier frequencies), configure the
tuner IF, configure the demod to work with the same IF as the tuner, set the number
of lines at the analog video demod and at the audio demod, etc.

In particular, when a format is changed between 50Hz and 60Hz standards, the
driver needs to validate the selected format (S_FMT), as most hardware scallers
don't support using 576 lines with 60Hz standards (as the maximum number of
lines provided by those standards are 480 lines).

The standards detection logic also need to merge information from the audio demod
and the video demod, in order to distinguish between standards like NTSC_M,
NTSC_M_JP and NTSC_M_KR, where the only difference is at the audio subcarrier
(there are some similar examples for PAL standards as well).

The support for those at the kernel level is as easy as broadcasting the message
to all subdevs, via the V4L2 "service bus". Only the drivers that hook into the
s_std/g_std/querystd ops will handle such request.

So, a PAD level support for it doesn't make sense, as it would duplicate a
functionality from kernelspace into userspace, and make harder to tune userspace
to work with all hardware, as userspace will never know what devices need to
be accessed when a video standard is changed.

>
>> Changing it would be a major regression. If OMAP3 is not doing the right
>> thing, it should be fixed, and not the opposite.
>>
>> The MC/subdev API is there to fill the blanks, e. g. to handle cases where
>> the same function could be implemented on two different places of the
>> pipeline, e. g. when both the sensor and the bridge could do scaling, and
>> userspace wants to explicitly use one, or the other, but it were never
>> meant to replace the V4L2 functionality.
>
> Isn't that what I've been saying ? :-)
>
>>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>>>>>
>>>>>> 2- If the application want a different format that the default
>>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have to
>>>>>> crop the image? I thought this can be made using the CCDC, copying
>>>>>> less lines to memory or the RESIZER if the application wants a bigger
>>>>>> image. What is the best approach for this?
>>>>
>>>> Not sure if I understood your question, but maybe you're mixing two
>>>> different concepts here.
>>>>
>>>> If the application wants a different image resolution, it will use
>>>> S_FMT. In this case, what userspace expects is that the driver will
>>>> scale, if supported, or return -EINVAL otherwise.
>>>
>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
>>> application will set the format on the OMAP3 ISP resizer input and output
>>> pads to configure scaling.
>>
>> The V4L2 API doesn't tell where a function like scaler will be implemented.
>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
>> V4L2 call is sent.
>
> By rolling a dice ? :-)

By using good sense. I never had a case where I had doubts about where the
scaling should be implemented on the drivers I've coded. For omap3/tvp5151, the
decision is also clear: it should be done at the bridge (omap3) resizer, as the
demod doesn't support scaling.

>> I'm ok if you want to offer the possibility of doing scale on the other
>> parts of the pipeline, as a bonus, via the MC/subdev API, but the absolute
>> minimum requirement is to implement it via the V4L2 API.
>
> The absolute minimum requirement is to make the driver usable by applications
> using the V4L2 API in a crippled mode with only a subdev of features
> available. For the OMAP3 ISP that means full-size capture, with no scaling.

As far as I can remember, all the existing drivers that have analog TV support
allows scaling. The worse case are the em2800 devices (the first series,
manufactured before 2005 - I never had one of those - newer em28xx supports
full scaling) and tm5600/tm60x0 devices. On both, the device can only scale between
full and half lines and full and half columns. All of them were already discontinued.

All modern drivers that support analog TV I'm aware does allow scaling and cropping.
and this is one of the things I check when I receive a new analog TV driver: if scaling
API (S_FMT) is properly written, otherwise, I nack the driver.
So, If OMAP3 can't support scaling at all, then you're right. Otherwise, scaling
support via S_FMT needs to be implemented.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 05-10-2011 17:54, Laurent Pinchart escreveu:
> Hi Hans,
>
> On Tuesday 04 October 2011 09:03:59 Hans Verkuil wrote:
>> On Tuesday, October 04, 2011 07:31:27 Mauro Carvalho Chehab wrote:
>>> Em 03-10-2011 19:37, Javier Martinez Canillas escreveu:
>>>> On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab wrote:
>>>>> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
>>>>>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
>>>>>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
>>>>>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
>>>>>>>>
>>>>>>>> [snip]
>>>>>>>>
>>>>>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver
>>>>>>>>> if you are so kind to answer.
>>>>>>>>>
>>>>>>>>> 1- User-space programs that are not MCF aware negotiate the format
>>>>>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>>>>>>>>> pad. But the real format is driven by the analog video format in
>>>>>>>>> the source pad (i.e: tvp5151).
>>>>>>>>
>>>>>>>> That's not different from existing systems using digital sensors,
>>>>>>>> where the format is driven by the sensor.
>>>>>>>>
>>>>>>>>> I modified the ISP driver to get the data format from the source
>>>>>>>>> pad and set the format for each pad on the pipeline accordingly
>>>>>>>>> but I've read from the documentation [1] that is not correct to
>>>>>>>>> propagate a data format from source pads to sink pads, that the
>>>>>>>>> correct thing is to do it from sink to source.
>>>>>>>>>
>>>>>>>>> So, in this case an administrator has to externally configure the
>>>>>>>>> format for each pad and to guarantee a coherent format on the
>>>>>>>>> whole pipeline?.
>>>>>>>>
>>>>>>>> That's correct (except you don't need to be an administrator to do
>>>>>>>> so
>>>>>>>>
>>>>>>>> :-)).
>>>>>>>
>>>>>>> NACK.
>>>>>>
>>>>>> Double NACK :-D
>>>>>>
>>>>>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the
>>>>>>> subdevs that are handling the video/audio standard should be
>>>>>>> changed, in order to obey the V4L2 ioctl. This is what happens with
>>>>>>> all other drivers since the beginning of the V4L1 API. There's no
>>>>>>> reason to change it, and such change would be a regression.
>>>>>>
>>>>>> The same could have been told for the format API:
>>>>>>
>>>>>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the
>>>>>> subdevs that are handling the video format should be changed, in
>>>>>> order to obey the V4L2 ioctl. This is what happens with all other
>>>>>> drivers since the beginning of the V4L1 API. There's no reason to
>>>>>> change it, and such change would be a regression."
>>>>>>
>>>>>> But we've introduced a pad-level format API. I don't see any reason
>>>>>> to treat standard differently.
>>>>>
>>>>> Neither do I. The pad-level API should not replace the V4L2 API for
>>>>> standard, for controls and/or for format settings.
>>
>> Remember we are talking about the subdev driver here. It makes no sense to
>> have both a s_mbus_fmt video op and a set_fmt pad op, which both do the
>> same thing. Bridge drivers should be adapted to use set_fmt only, so we
>> can drop s_mbus_fmt.
>>
>> BTW, the names 'set_fmt' and 'get_fmt' are very confusing to me. I always
>> think these refer to v4l2_format. Can we please rename this to
>> g/s_mbus_fmt?
>
> Shouldn't we do it the other way around, renaming the g/s_mbus_fmt video
> operations to g/s_fmt, now that we only have mbus formats left ? The operation
> names would be shorter.
>
>> And set/get_crop to s/g_crop? This for consistent naming.
>
> set/get_crop will be replaced with set/get_selection (or s/g_selection if you
> like that better :-)).
>
>> When it comes to S_STD I don't see the need for a pad version of this. It
>> is an op that is used to configure subdevs to handle a specific standard.
>> If you are configuring the pipelines manually, then after calling S_STD
>> you have to set up the mbus formats yourself.
>>
>> Of course, I can generate scenarios where you would need to set the
>> standard through the subdev node (i.e. two video receivers connected to a
>> SoC, one receiving PAL, the other receiving NTSC, and both streams
>> composed into a single new stream that's DMA-ed to memory), but frankly,
>> those scenarios are very contrived :-)
>
> Unless I don't understand this correctly, I think those two paragraphs are not
> related.
>
> Regarding the pad-level standard operations, you're probably right. Subdevs
> that can handle two or more analog streams at the same time would need that,
> but we probably won't need to support them any time soon (if ever). So we
> could keep the subdev-level standard operations, with the implicit (or
> explicit) rule that they apply to the currently selected input of the subdev
> (I suppose TV decoders with an input mux are not uncommon).

Switching from one input to the other doesn't switch the TV format (except,
of course, if autodetection is enabled). The driver should keep the last settings
to the new input for resolution, format, fourcc and image controls.

> Regarding controlling standards directly on subdevs, I think that's the way to
> go for complex pipelines, but that doesn't require pad-level standard
> operations.
>
>> The preset ioctls would be more realistic since I know that a scenario like
>> the one above does exist for e.g. HDMI inputs, where each can receive a
>> different format.
>>
>> In that case the preset ioctls would have to be exposed to the subdev
>> nodes, allowing you to set it up for each HDMI receiver independently. But
>> you still do not need pads to do this since this is a subdev-level
>> operation, not pad-level.
>>
>>>>>>>>> Or does exist a way to do this automatic?. i.e: The output entity
>>>>>>>>> on the pipeline promotes the capabilities of the source pad so
>>>>>>>>> applications can select a data format and this format gets
>>>>>>>>> propagated all over the pipeline from the sink pad to the source?
>>>>>>>>
>>>>>>>> It can be automated in userspace (through a libv4l plugin for
>>>>>>>> instance), but it's really not the kernel's job to do so.
>>>>>>>
>>>>>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left
>>>>>>> to any userspace plugin.
>>>>>>
>>>>>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why
>>>>>> are standards different ?
>>>>>
>>>>> All v4l media devices have sub-devices with either tv decoders or
>>>>> sensors connected into a sink. The sink implements the /dev/video?
>>>>> node.
>>>>> When an ioctl is sent to the v4l node, the sensors/decoders are
>>>>> controlled to implement whatever is requested: video standards,
>>>>> formats etc.
>>>>>
>>>>> Changing it would be a major regression. If OMAP3 is not doing the
>>>>> right thing, it should be fixed, and not the opposite.
>>>>
>>>> That is the approach we took, we hack the isp v4l2 device driver
>>>> (ispvideo) to bypass the ioctls to the sub-device that as the source
>>>> pad (tvp5151 in our case, but it could be a sensor as well). So, for
>>>> example the VIDIOC_S_STD ioctl handler looks like this (I post a
>>>> simplified version of the code, just to give an idea):
>>>>
>>>> static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id
>>>> *norm) {
>>>>
>>>>      struct isp_video *video = video_drvdata(file);
>>>>      struct v4l2_subdev *sink_subdev;
>>>>      struct v4l2_subdev *source_subdev;
>>>>
>>>>      sink_subdev = isp_video_remote_subdev(video, NULL);
>>>>      sink_pad =&sink_subdev->entity.pads[0];
>>>>      source_pad = media_entity_remote_source(sink_pad);
>>>>      source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
>>>>      v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
>>>>
>>>> }
>>>>
>>>> So applications interact with the /dev/video? via V4L2 ioctls whose
>>>> handlers call the sub-dev functions. Is that what you propose?
>>>
>>> Something like that. For example:
>>>
>>> static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id *norm)
>>> {
>>>
>>> 	/* Do some sanity test/resolution adjustments, etc */
>>> 	
>>> 	v4l2_device_call_all(&dev->v4l2_dev, 0, core, s_std, dev->norm);
>>> 	
>>> 	return 0;
>>>
>>> }
>>>
>>> It should be noticed that:
>>>
>>> 1) a full standard auto-detection is only possible at the V4L2 level, as
>>> a standard is a composition of audio and video carriers. I intend to
>>> work tomorrow to add audio auto-detection to pvrusb2;
>>>
>>> 2) a full s_std should not only adjust the TV demod pad, but also, the
>>> audio pad and the tuner pad (if the analog input comes from a tuner).
>>>
>>>>> The MC/subdev API is there to fill the blanks, e. g. to handle cases
>>>>> where the same function could be implemented on two different places
>>>>> of the pipeline, e. g. when both the sensor and the bridge could do
>>>>> scaling, and userspace wants to explicitly use one, or the other, but
>>>>> it were never meant to replace the V4L2 functionality.
>>>>>
>>>>>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>>>>>>>>
>>>>>>>>> 2- If the application want a different format that the default
>>>>>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have
>>>>>>>>> to crop the image? I thought this can be made using the CCDC,
>>>>>>>>> copying less lines to memory or the RESIZER if the application
>>>>>>>>> wants a bigger image. What is the best approach for this?
>>>>>>>
>>>>>>> Not sure if I understood your question, but maybe you're mixing two
>>>>>>> different concepts here.
>>>>>>>
>>>>>>> If the application wants a different image resolution, it will use
>>>>>>> S_FMT. In this case, what userspace expects is that the driver will
>>>>>>> scale, if supported, or return -EINVAL otherwise.
>>>>>>
>>>>>> With the OMAP3 ISP, which is I believe what Javier was asking about,
>>>>>> the application will set the format on the OMAP3 ISP resizer input
>>>>>> and output pads to configure scaling.
>>>>
>>>> Yes, that was my question about. But still is not clear to me if
>>>> configuring the ISP resizer input and output pads with different frame
>>>> sizes automatically means that I have to do the scale or this has to
>>>> be configured using a S_FMT ioctl to the /dev/video? node.
>>>>
>>>> Basically what I don't know is when I have to modify the pipeline
>>>> graph inside the ISP driver and when this has to be made from
>>>> user-space via MCF.
>>>
>>> In the specific case of analog inputs, In general, better results are
>>> obtained if the scaling is done at the analog demod, as it can play with
>>> the pixel sampling rate, and obtaining a better result than a decimation
>>> filter. Not all demods accept this through.
>>>
>>> Anyway, S_FMT is expected to work at the /dev/video? node. Laurent will
>>> likely argument against, but, again, allowing to control the scaling on
>>> a different place is a bonus of the MC/subdev API, but it should never
>>> replace the S_FMT V4L2 call.
>>
>> I agree with Mauro, up to a point. One way or another drivers should
>> support S_FMT through the video nodes.
>
> When there's a one-to-one relationship between the video device (in the
> hardware sense) and the video node, that's pretty normal. However, when a
> video device is made of a complex pipeline and several video nodes, the
> problem becomes much more complex and mapping a "simple" operations such as
> S_FMT to hardware configuration becomes a policy decision.

One video node should "master" the operation. A V4L2 application will open just
one device to control the stuff (as a general rule, I'd say that it will open
the sink node). A V4L2 ioctl applied into it should be propagated along the
pipeline for the ioctl to happen. This is what happens with all other devices,
and this is what a V4L2 userspace application expects.

>> But if someone starts programming the pipeline through subdev nodes, then I
>> think it is reasonable that trying to call S_FMT would return EBUSY or
>> something similar.
>>
>> It has been suggested that S_FMT can be implemented for such systems as a
>> libv4l2 plugin, but I would like to see code doing this first as I am not
>> convinced that that's the best way to do it.
>
> Sakari, what's the status of the OMAP3 ISP libv4l plugin ?
>

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Javier Martinez Canillas]

On Wed, Oct 5, 2011 at 11:48 PM, Mauro Carvalho Chehab
<mchehab@infradead.org> wrote:
> Em 05-10-2011 17:54, Laurent Pinchart escreveu:
>>
>> Hi Hans,
>>
>> On Tuesday 04 October 2011 09:03:59 Hans Verkuil wrote:
>>>
>>> On Tuesday, October 04, 2011 07:31:27 Mauro Carvalho Chehab wrote:
>>>>
>>>> Em 03-10-2011 19:37, Javier Martinez Canillas escreveu:
>>>>>
>>>>> On Mon, Oct 3, 2011 at 11:56 PM, Mauro Carvalho Chehab wrote:
>>>>>>
>>>>>> Em 03-10-2011 18:44, Laurent Pinchart escreveu:
>>>>>>>
>>>>>>> On Monday 03 October 2011 21:16:45 Mauro Carvalho Chehab wrote:
>>>>>>>>
>>>>>>>> Em 03-10-2011 08:53, Laurent Pinchart escreveu:
>>>>>>>>>
>>>>>>>>> On Monday 03 October 2011 11:53:44 Javier Martinez Canillas wrote:
>>>>>>>>>
>>>>>>>>> [snip]
>>>>>>>>>
>>>>>>>>>> Laurent, I have a few questions about MCF and the OMAP3ISP driver
>>>>>>>>>> if you are so kind to answer.
>>>>>>>>>>
>>>>>>>>>> 1- User-space programs that are not MCF aware negotiate the format
>>>>>>>>>> with the V4L2 device (i.e: OMAP3 ISP CCDC output), which is a sink
>>>>>>>>>> pad. But the real format is driven by the analog video format in
>>>>>>>>>> the source pad (i.e: tvp5151).
>>>>>>>>>
>>>>>>>>> That's not different from existing systems using digital sensors,
>>>>>>>>> where the format is driven by the sensor.
>>>>>>>>>
>>>>>>>>>> I modified the ISP driver to get the data format from the source
>>>>>>>>>> pad and set the format for each pad on the pipeline accordingly
>>>>>>>>>> but I've read from the documentation [1] that is not correct to
>>>>>>>>>> propagate a data format from source pads to sink pads, that the
>>>>>>>>>> correct thing is to do it from sink to source.
>>>>>>>>>>
>>>>>>>>>> So, in this case an administrator has to externally configure the
>>>>>>>>>> format for each pad and to guarantee a coherent format on the
>>>>>>>>>> whole pipeline?.
>>>>>>>>>
>>>>>>>>> That's correct (except you don't need to be an administrator to do
>>>>>>>>> so
>>>>>>>>>
>>>>>>>>> :-)).
>>>>>>>>
>>>>>>>> NACK.
>>>>>>>
>>>>>>> Double NACK :-D
>>>>>>>
>>>>>>>> When userspace sends a VIDIOC_S_STD ioctl to the sink node, the
>>>>>>>> subdevs that are handling the video/audio standard should be
>>>>>>>> changed, in order to obey the V4L2 ioctl. This is what happens with
>>>>>>>> all other drivers since the beginning of the V4L1 API. There's no
>>>>>>>> reason to change it, and such change would be a regression.
>>>>>>>
>>>>>>> The same could have been told for the format API:
>>>>>>>
>>>>>>> "When userspace sends a VIDIOC_S_FMT ioctl to the sink node, the
>>>>>>> subdevs that are handling the video format should be changed, in
>>>>>>> order to obey the V4L2 ioctl. This is what happens with all other
>>>>>>> drivers since the beginning of the V4L1 API. There's no reason to
>>>>>>> change it, and such change would be a regression."
>>>>>>>
>>>>>>> But we've introduced a pad-level format API. I don't see any reason
>>>>>>> to treat standard differently.
>>>>>>
>>>>>> Neither do I. The pad-level API should not replace the V4L2 API for
>>>>>> standard, for controls and/or for format settings.
>>>
>>> Remember we are talking about the subdev driver here. It makes no sense
>>> to
>>> have both a s_mbus_fmt video op and a set_fmt pad op, which both do the
>>> same thing. Bridge drivers should be adapted to use set_fmt only, so we
>>> can drop s_mbus_fmt.
>>>
>>> BTW, the names 'set_fmt' and 'get_fmt' are very confusing to me. I always
>>> think these refer to v4l2_format. Can we please rename this to
>>> g/s_mbus_fmt?
>>
>> Shouldn't we do it the other way around, renaming the g/s_mbus_fmt video
>> operations to g/s_fmt, now that we only have mbus formats left ? The
>> operation
>> names would be shorter.
>>
>>> And set/get_crop to s/g_crop? This for consistent naming.
>>
>> set/get_crop will be replaced with set/get_selection (or s/g_selection if
>> you
>> like that better :-)).
>>
>>> When it comes to S_STD I don't see the need for a pad version of this. It
>>> is an op that is used to configure subdevs to handle a specific standard.
>>> If you are configuring the pipelines manually, then after calling S_STD
>>> you have to set up the mbus formats yourself.
>>>
>>> Of course, I can generate scenarios where you would need to set the
>>> standard through the subdev node (i.e. two video receivers connected to a
>>> SoC, one receiving PAL, the other receiving NTSC, and both streams
>>> composed into a single new stream that's DMA-ed to memory), but frankly,
>>> those scenarios are very contrived :-)
>>
>> Unless I don't understand this correctly, I think those two paragraphs are
>> not
>> related.
>>
>> Regarding the pad-level standard operations, you're probably right.
>> Subdevs
>> that can handle two or more analog streams at the same time would need
>> that,
>> but we probably won't need to support them any time soon (if ever). So we
>> could keep the subdev-level standard operations, with the implicit (or
>> explicit) rule that they apply to the currently selected input of the
>> subdev
>> (I suppose TV decoders with an input mux are not uncommon).
>
> Switching from one input to the other doesn't switch the TV format (except,
> of course, if autodetection is enabled). The driver should keep the last
> settings
> to the new input for resolution, format, fourcc and image controls.
>
>> Regarding controlling standards directly on subdevs, I think that's the
>> way to
>> go for complex pipelines, but that doesn't require pad-level standard
>> operations.
>>
>>> The preset ioctls would be more realistic since I know that a scenario
>>> like
>>> the one above does exist for e.g. HDMI inputs, where each can receive a
>>> different format.
>>>
>>> In that case the preset ioctls would have to be exposed to the subdev
>>> nodes, allowing you to set it up for each HDMI receiver independently.
>>> But
>>> you still do not need pads to do this since this is a subdev-level
>>> operation, not pad-level.
>>>
>>>>>>>>>> Or does exist a way to do this automatic?. i.e: The output entity
>>>>>>>>>> on the pipeline promotes the capabilities of the source pad so
>>>>>>>>>> applications can select a data format and this format gets
>>>>>>>>>> propagated all over the pipeline from the sink pad to the source?
>>>>>>>>>
>>>>>>>>> It can be automated in userspace (through a libv4l plugin for
>>>>>>>>> instance), but it's really not the kernel's job to do so.
>>>>>>>>
>>>>>>>> It is a kernel job to handle VIDIOC_S_STD, and not a task to be left
>>>>>>>> to any userspace plugin.
>>>>>>>
>>>>>>> And VIDIOC_S_FMT is handled by userspace for the OMAP3 ISP today. Why
>>>>>>> are standards different ?
>>>>>>
>>>>>> All v4l media devices have sub-devices with either tv decoders or
>>>>>> sensors connected into a sink. The sink implements the /dev/video?
>>>>>> node.
>>>>>> When an ioctl is sent to the v4l node, the sensors/decoders are
>>>>>> controlled to implement whatever is requested: video standards,
>>>>>> formats etc.
>>>>>>
>>>>>> Changing it would be a major regression. If OMAP3 is not doing the
>>>>>> right thing, it should be fixed, and not the opposite.
>>>>>
>>>>> That is the approach we took, we hack the isp v4l2 device driver
>>>>> (ispvideo) to bypass the ioctls to the sub-device that as the source
>>>>> pad (tvp5151 in our case, but it could be a sensor as well). So, for
>>>>> example the VIDIOC_S_STD ioctl handler looks like this (I post a
>>>>> simplified version of the code, just to give an idea):
>>>>>
>>>>> static int isp_video_s_std(struct file *file, void *fh, v4l2_std_id
>>>>> *norm) {
>>>>>
>>>>>     struct isp_video *video = video_drvdata(file);
>>>>>     struct v4l2_subdev *sink_subdev;
>>>>>     struct v4l2_subdev *source_subdev;
>>>>>
>>>>>     sink_subdev = isp_video_remote_subdev(video, NULL);
>>>>>     sink_pad =&sink_subdev->entity.pads[0];
>>>>>     source_pad = media_entity_remote_source(sink_pad);
>>>>>     source_subdev = media_entity_to_v4l2_subdev(source_pad->entity);
>>>>>     v4l2_subdev_call(source_subdev, core, s_std, NULL, norm);
>>>>>
>>>>> }
>>>>>
>>>>> So applications interact with the /dev/video? via V4L2 ioctls whose
>>>>> handlers call the sub-dev functions. Is that what you propose?
>>>>
>>>> Something like that. For example:
>>>>
>>>> static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id
>>>> *norm)
>>>> {
>>>>
>>>>        /* Do some sanity test/resolution adjustments, etc */
>>>>
>>>>        v4l2_device_call_all(&dev->v4l2_dev, 0, core, s_std, dev->norm);
>>>>
>>>>        return 0;
>>>>
>>>> }
>>>>
>>>> It should be noticed that:
>>>>
>>>> 1) a full standard auto-detection is only possible at the V4L2 level, as
>>>> a standard is a composition of audio and video carriers. I intend to
>>>> work tomorrow to add audio auto-detection to pvrusb2;
>>>>
>>>> 2) a full s_std should not only adjust the TV demod pad, but also, the
>>>> audio pad and the tuner pad (if the analog input comes from a tuner).
>>>>
>>>>>> The MC/subdev API is there to fill the blanks, e. g. to handle cases
>>>>>> where the same function could be implemented on two different places
>>>>>> of the pipeline, e. g. when both the sensor and the bridge could do
>>>>>> scaling, and userspace wants to explicitly use one, or the other, but
>>>>>> it were never meant to replace the V4L2 functionality.
>>>>>>
>>>>>>>>>> [1]: http://linuxtv.org/downloads/v4l-dvb-apis/subdev.html
>>>>>>>>>>
>>>>>>>>>> 2- If the application want a different format that the default
>>>>>>>>>> provided by the tvp5151, (i.e: 720x576 for PAL), where do I have
>>>>>>>>>> to crop the image? I thought this can be made using the CCDC,
>>>>>>>>>> copying less lines to memory or the RESIZER if the application
>>>>>>>>>> wants a bigger image. What is the best approach for this?
>>>>>>>>
>>>>>>>> Not sure if I understood your question, but maybe you're mixing two
>>>>>>>> different concepts here.
>>>>>>>>
>>>>>>>> If the application wants a different image resolution, it will use
>>>>>>>> S_FMT. In this case, what userspace expects is that the driver will
>>>>>>>> scale, if supported, or return -EINVAL otherwise.
>>>>>>>
>>>>>>> With the OMAP3 ISP, which is I believe what Javier was asking about,
>>>>>>> the application will set the format on the OMAP3 ISP resizer input
>>>>>>> and output pads to configure scaling.
>>>>>
>>>>> Yes, that was my question about. But still is not clear to me if
>>>>> configuring the ISP resizer input and output pads with different frame
>>>>> sizes automatically means that I have to do the scale or this has to
>>>>> be configured using a S_FMT ioctl to the /dev/video? node.
>>>>>
>>>>> Basically what I don't know is when I have to modify the pipeline
>>>>> graph inside the ISP driver and when this has to be made from
>>>>> user-space via MCF.
>>>>
>>>> In the specific case of analog inputs, In general, better results are
>>>> obtained if the scaling is done at the analog demod, as it can play with
>>>> the pixel sampling rate, and obtaining a better result than a decimation
>>>> filter. Not all demods accept this through.
>>>>
>>>> Anyway, S_FMT is expected to work at the /dev/video? node. Laurent will
>>>> likely argument against, but, again, allowing to control the scaling on
>>>> a different place is a bonus of the MC/subdev API, but it should never
>>>> replace the S_FMT V4L2 call.
>>>
>>> I agree with Mauro, up to a point. One way or another drivers should
>>> support S_FMT through the video nodes.
>>
>> When there's a one-to-one relationship between the video device (in the
>> hardware sense) and the video node, that's pretty normal. However, when a
>> video device is made of a complex pipeline and several video nodes, the
>> problem becomes much more complex and mapping a "simple" operations such
>> as
>> S_FMT to hardware configuration becomes a policy decision.
>
> One video node should "master" the operation. A V4L2 application will open
> just
> one device to control the stuff (as a general rule, I'd say that it will
> open
> the sink node). A V4L2 ioctl applied into it should be propagated along the
> pipeline for the ioctl to happen. This is what happens with all other
> devices,
> and this is what a V4L2 userspace application expects.
>

What about a hybrid solution? We can set the links from user-space and
kernel-space.

If the application wants to change the current format (S_FMT) the
driver can modify the graph and change the pipeline to include the
resizer or any other entity that can offer the service asked by the
application.

Or in the other hand the user setting the pipeline decides what
capabilities the video device support. If the user doesn't include the
resizer for example then an ioctl call S_FMT to the sink node will
return -ENOIOCTLCMD or -EINVAL.

But then we will need a way to associate V4L2 ioctl actions with media
entities or pads, to know if someone in the pipeline (or someone not
in the pipeline but registered as a sub-device) can provide the
service that the application needs.

>>> But if someone starts programming the pipeline through subdev nodes, then
>>> I
>>> think it is reasonable that trying to call S_FMT would return EBUSY or
>>> something similar.
>>>
>>> It has been suggested that S_FMT can be implemented for such systems as a
>>> libv4l2 plugin, but I would like to see code doing this first as I am not
>>> convinced that that's the best way to do it.
>>
>> Sakari, what's the status of the OMAP3 ISP libv4l plugin ?
>>
>
>



-- 
Javier Martínez Canillas
(+34) 682 39 81 69
Barcelona, Spain

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Sakari Ailus]

Hi Mauro,

On Wed, Oct 05, 2011 at 06:41:27PM -0300, Mauro Carvalho Chehab wrote:
> Em 05-10-2011 17:08, Laurent Pinchart escreveu:
[clip]
> >The pad-level API doesn't replace the V4L2 API, it complements it. I'm of
> >course not advocating modifying any driver to replace V4L2 ioctls by direct
> >subdev access. However, the pad-level API needs to be exposed to userspace, as
> >some harware simply can't be configured through a video node only.
> >
> >As Hans also mentionned in his reply, the pad-level API is made of two parts:
> >an in-kernel API made of subdev operations, and a userspace API accessed
> >through ioctls. As the userspace API is needed for some devices, the kernel
> >API needs to be implemented by drivers. We should phase out the non pad-level
> >format operations in favor of pad-level operations, as the former can be
> >implemented using the later. That has absolutely no influence on the userspace
> >API.
> 
> What I'm seeing is that:
> 	- the drivers that are implementing the MC/pad API's aren't
> compatible with generic V4L2 applications;

This is currently true up to a certain degree; you'll need to configure such
devices using media-ctl at least. Even then, such embedded systems do often
have automatic exposure and white balance algorithms above the V4L2 (often
proprietary implementations).

To be really useful for a regular user, such algorithms would also be
needed.

The two problems are separate but they still both need to be resolved to be
able to use general purpose V4L2 applications on such systems in a
meaningful way.

> 	- there's no way to write a generic application that works with all
> drivers, even implementing MC/pad API's there as each driver is taking different
> a approach on how to map things at the different API's, and pipeline configuration
> is device-dependent;

The pipeline configuration is device specific but the interfaces are not.
Thus it's possible to write a generic plugin which configures the device.

The static configuration can be kept in a text file in the first phase and
later on hints may be added for synamic configuration on e.g. where digital
gain, scaling and cropping should be performed and which resolutions to
advertise in enum_framesizes.

But we do not have such plugin yet.

> 	- there's no effort to publish patches to libv4l to match the changes
> at the kernel driver.

I'd prefer concentrating all efforts towards a single, generic plugin. As
noted before, a plugin for OMAP 3 ISP exists, but I don't think it does
anything a generic plugin couldn't do, so it hasn't been merged.

I'm working on patches to move the text-based pipeline configuration to
libmediactl and libv4l2subdev and will post them to the list in the coming
few days, among with a few other improvements. This is one of the first
required steps towards such generic plugin.

Unfortunately I haven't been able to use much of my time on this; help would
indeed be appreciated from any interested party.

[clip]

> I'm fine on providing raw interfaces, just like we have for some types of device
> (like the block raw interfaces used by CD-ROM drivers) as a bonus, but this should
> never replace an API where an application developed by a third party could work
> with all media hardware, without needing hardware specific details.

I agree.

[clip]

> >>>>If the application wants a different image resolution, it will use
> >>>>S_FMT. In this case, what userspace expects is that the driver will
> >>>>scale, if supported, or return -EINVAL otherwise.
> >>>
> >>>With the OMAP3 ISP, which is I believe what Javier was asking about, the
> >>>application will set the format on the OMAP3 ISP resizer input and output
> >>>pads to configure scaling.
> >>
> >>The V4L2 API doesn't tell where a function like scaler will be implemented.
> >>So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
> >>V4L2 call is sent.
> >
> >By rolling a dice ? :-)
> 
> By using good sense. I never had a case where I had doubts about where the
> scaling should be implemented on the drivers I've coded. For omap3/tvp5151, the
> decision is also clear: it should be done at the bridge (omap3) resizer, as the
> demod doesn't support scaling.

Good sense in this case would require knowledge of the tv tuner in the
camera ISP driver. One could also, and actually does, connect sensors which
can do scaling in two steps to the same ISP. How does the ISP driver now
know where to do scaling?

OMAP 3 ISP as such is a relatively simple one; there are ISPs which have two
scalers and ability to write the image to memory also in between the two.

I don't think a driver, be that tv tuner / sensor or ISP driver, has enough
information to perform the pipeline configuration which is mandatory to
implement the standard V4L2 ioctls for e.g. format setting and cropping. We
actually had such an implementation of the OMAP 3 ISP driver around 2009
(but without MC or V4L2 subdevs, still the underlying technical issues are
the same) and it was a disaster. The patches were posted to linux-media a
few times back then if you're interested. It was evident something had to be
done, and now we have the Media controller and V4L2 subdev user space APIs.

I want to reiterate that I'm very much in favour of supporting generic V4L2
applications on these devices. The additional support that these application
need on top of the drivers requires making policy decisions that
applications intended for embedded devices (such as Fcam) often run on
embedded devices need to make themselves. To support both using the same
drivers, these policy decisions must be taken in the user space, not in the
kernel. In my opinion libv4l is in perfect spot to fill this gap.

Kind regards,

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Sakari Ailus]

On Mon, Oct 03, 2011 at 11:39:31PM +0200, Laurent Pinchart wrote:
> Sorry ? We already have format setting at the pad level, and that's used by 
> drivers and applications. It's one key feature of the V4L2/MC API.

A tiny additional note: this feature is actuall necessary since e.g. OMAP 3
ISP CCDC provides images of different size at different outputs; one to
memory and one to another ISP block.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Sakari Ailus]

On Mon, Oct 03, 2011 at 04:36:44PM -0300, Mauro Carvalho Chehab wrote:
> Em 03-10-2011 16:01, Sakari Ailus escreveu:
> > On Mon, Oct 03, 2011 at 03:53:54PM -0300, Mauro Carvalho Chehab wrote:
> >> Yes, you're right. I should not try to answer emails when I'm lazy enough to not
> >> look in to the code ;)
> >>
> >> Anyway, the thing is that V4L2 API has enough support for auto-detection. There's
> >> no need to duplicate stuff with MC API.
> > 
> > It's not MC API but V4L2 subdev API. No-one has proposed to add video
> > standard awareness to the Media controller API. There's no reason to export
> > a video node in video decoder drivers... but I guess you didn't mean that.
> > 
> > Would implementing ENUM/G/S_STD make sense for the camera ISP driver, that
> > has nothing to do with any video standard?
> 
> This is an old discussion, and we never agreed on that. Some webcam drivers 
> implement those ioctls. Others don't. Both cases are compliant with the
> current specs. In the past, several userspace applications used to abort if those
> ioctl's weren't implemented, but I think that this were fixed already there.
> 
> As I said, we should define a per-device type profile in order to enforce that
> all devices of the same type will do the same. We'll need man power to fix the
> ones that aren't compliant, and solve the userspace issues. Volunteers needed.
> 
> There's one point to bold on it: devices that can have either an analog input
> or a digital input will likely need to implement ENUM/G/S_STD for both, as
> userspace applications may fail, if the ioctl's are disabled depending on the
> type of input. We had to implement them on several drivers, due to that.

My disguised question behind this was actually that would a driver need to
implement an ioctl that has no relevance to the driver itself at all but
only to support another driver, yet the first driver might not have enough
information to properly implement it?

It may be sometimes necessary but I would like to avoid that if possible
since it complicates even more drivers which already are very complex.

> > If you have two video decoders
> > connected to your system, then which one should the ioctls be redirected to?
> > What if there's a sensor and a video decoder? And how could the user know
> > about this?
> 
> When an input is selected (either via the V4L2 way - S_INPUT or via the MC/subdev
> way, there's just one video decoder or sensor associated to the corresponding
> V4L2 node).
> 
> > It's the same old issues again... let's discuss this in the Multimedia
> > summit.
> 
> We can discuss more at the summit, but we should start discussing it here, as
> otherwise we may not be able to go into a consensus there, due to the limited
> amount of time we would have for each topic.

Sounds good to me, but sometimes face-to-face discussion just is not
replaceable.

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi	jabber/XMPP/Gmail: sailus@retiisi.org.uk

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 05-10-2011 20:14, Sakari Ailus escreveu:
> Hi Mauro,
>
> On Wed, Oct 05, 2011 at 06:41:27PM -0300, Mauro Carvalho Chehab wrote:
>> Em 05-10-2011 17:08, Laurent Pinchart escreveu:
> [clip]
>>> The pad-level API doesn't replace the V4L2 API, it complements it. I'm of
>>> course not advocating modifying any driver to replace V4L2 ioctls by direct
>>> subdev access. However, the pad-level API needs to be exposed to userspace, as
>>> some harware simply can't be configured through a video node only.
>>>
>>> As Hans also mentionned in his reply, the pad-level API is made of two parts:
>>> an in-kernel API made of subdev operations, and a userspace API accessed
>>> through ioctls. As the userspace API is needed for some devices, the kernel
>>> API needs to be implemented by drivers. We should phase out the non pad-level
>>> format operations in favor of pad-level operations, as the former can be
>>> implemented using the later. That has absolutely no influence on the userspace
>>> API.
>>
>> What I'm seeing is that:
>> 	- the drivers that are implementing the MC/pad API's aren't
>> compatible with generic V4L2 applications;
>
> This is currently true up to a certain degree; you'll need to configure such
> devices using media-ctl at least. Even then, such embedded systems do often
> have automatic exposure and white balance algorithms above the V4L2 (often
> proprietary implementations).
>
> To be really useful for a regular user, such algorithms would also be
> needed.
>
> The two problems are separate but they still both need to be resolved to be
> able to use general purpose V4L2 applications on such systems in a
> meaningful way.

That's true, and the MC/pad API's were added to offer support for those proprietary
plugins via libv4l.

That's fine, but the thing is that some developers seem to think that only the streaming
should be done via the V4L2 API, and all the rest via pad configs, while others have
different views. Also, I couldn't see a consense about input selection on drivers that
implement MC: they could either implement S_INPUT, create one V4L device node for each
input, or create just one devnode and let userspace (somehow) to discover the valid
inputs and set the pipelines via MC/pad.

With all those complex and different scenarios, writing a generic plugin/application
that would cover all possible alternatives with a mix of V4L/MC/pad API's would be
extremely complex, and will take years to do it right.

In other words, we need to confine the possible solutions to the ones that will
actually be supported.

>> 	- there's no way to write a generic application that works with all
>> drivers, even implementing MC/pad API's there as each driver is taking different
>> a approach on how to map things at the different API's, and pipeline configuration
>> is device-dependent;
>
> The pipeline configuration is device specific but the interfaces are not.
> Thus it's possible to write a generic plugin which configures the device.
>
> The static configuration can be kept in a text file in the first phase and
> later on hints may be added for synamic configuration on e.g. where digital
> gain, scaling and cropping should be performed and which resolutions to
> advertise in enum_framesizes.

Assuming that all drivers do the same, this would be an alternative for such
plugin.

> But we do not have such plugin yet.

As several SoC developers showed on all opportunities we've met, the complexity
for those devices is very high. The practical effect of adding a kernel driver
without the corresponding required library support is that we ended by
merging incomplete drivers and nobody, except for the manufacturers/developers
whose submitted them could actually make an userspace application to work
with. That would be ok if the drivers would be at /staging, but they aren't.
This situation should be fixed as soon as possible.

I'm seriously considering to not send the patches we have on our tree for those
drivers upstream while we don't have a solution for that.

>> 	- there's no effort to publish patches to libv4l to match the changes
>> at the kernel driver.
>
> I'd prefer concentrating all efforts towards a single, generic plugin. As
> noted before, a plugin for OMAP 3 ISP exists, but I don't think it does
> anything a generic plugin couldn't do, so it hasn't been merged.
>
> I'm working on patches to move the text-based pipeline configuration to
> libmediactl and libv4l2subdev and will post them to the list in the coming
> few days, among with a few other improvements. This is one of the first
> required steps towards such generic plugin.

That would be great, but it won't solve, as more mess is being proposed each
day. One thing is to set the pipelines. Another thing is to not support
things like S_FMT/S_STD/... ioctl's. The user knows what format he wants,
and that's what S_FMT/S_STD tells to the driver. the extra formats at the
pipeline could be handled by a policy either at the driver or at a libv4l,
although I don't think that a generic libv4l plugin would be capable of
doing it (see more about that bellow).

> Unfortunately I haven't been able to use much of my time on this; help would
> indeed be appreciated from any interested party.
>
> [clip]
>
>> I'm fine on providing raw interfaces, just like we have for some types of device
>> (like the block raw interfaces used by CD-ROM drivers) as a bonus, but this should
>> never replace an API where an application developed by a third party could work
>> with all media hardware, without needing hardware specific details.
>
> I agree.
>
> [clip]
>
>>>>>> If the application wants a different image resolution, it will use
>>>>>> S_FMT. In this case, what userspace expects is that the driver will
>>>>>> scale, if supported, or return -EINVAL otherwise.
>>>>>
>>>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
>>>>> application will set the format on the OMAP3 ISP resizer input and output
>>>>> pads to configure scaling.
>>>>
>>>> The V4L2 API doesn't tell where a function like scaler will be implemented.
>>>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
>>>> V4L2 call is sent.
>>>
>>> By rolling a dice ? :-)
>>
>> By using good sense. I never had a case where I had doubts about where the
>> scaling should be implemented on the drivers I've coded. For omap3/tvp5151, the
>> decision is also clear: it should be done at the bridge (omap3) resizer, as the
>> demod doesn't support scaling.
>
> Good sense in this case would require knowledge of the tv tuner in the
> camera ISP driver. One could also, and actually does, connect sensors which
> can do scaling in two steps to the same ISP. How does the ISP driver now
> know where to do scaling?

Does it actually make sense to allow scaling on both? In the case of the
generic plugin you're writing, would it do scale on both? If not, how
would it decide where to scale? Moving the problem to userspace won't solve
it, and would require userspace to take some hard decisions based on the specific
hardware designs. In practice, it means that, every time a hardware is updated
(a new sensor is added, etc), both the plugin and the driver will need to be
touched, at the same time. This also means that the userspace plugin will
be dependent on an specific kernel version, which would be a real mess to
the distributors to package the libv4l plugin.

Btw, we have some devices that support scaling on both tv decoder and bridge.
As not all tv decoders have scaling, the decision were to implement scaling
it only at the bridge. This was as good as doing it at the sensor (or even
better, as some bridges have temporal/spacial decimation filtering, with
provides a better decimation than a pure spacial filtering, and can reduce
the quantization noise).

> OMAP 3 ISP as such is a relatively simple one; there are ISPs which have two
> scalers and ability to write the image to memory also in between the two.
>
> I don't think a driver, be that tv tuner / sensor or ISP driver, has enough
> information to perform the pipeline configuration which is mandatory to
> implement the standard V4L2 ioctls for e.g. format setting and cropping. We
> actually had such an implementation of the OMAP 3 ISP driver around 2009
> (but without MC or V4L2 subdevs, still the underlying technical issues are
> the same) and it was a disaster. The patches were posted to linux-media a
> few times back then if you're interested. It was evident something had to be
> done, and now we have the Media controller and V4L2 subdev user space APIs.

As I said before, you're just transfering the problem from the kernel to
something else, without actually solving it.

> I want to reiterate that I'm very much in favour of supporting generic V4L2
> applications on these devices. The additional support that these application
> need on top of the drivers requires making policy decisions that
> applications intended for embedded devices (such as Fcam) often run on
> embedded devices need to make themselves. To support both using the same
> drivers, these policy decisions must be taken in the user space, not in the
> kernel. In my opinion libv4l is in perfect spot to fill this gap.
>
> Kind regards,
>

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 05-10-2011 20:41, Sakari Ailus escreveu:
> On Mon, Oct 03, 2011 at 04:36:44PM -0300, Mauro Carvalho Chehab wrote:
>> Em 03-10-2011 16:01, Sakari Ailus escreveu:
>>> On Mon, Oct 03, 2011 at 03:53:54PM -0300, Mauro Carvalho Chehab wrote:
>>>> Yes, you're right. I should not try to answer emails when I'm lazy enough to not
>>>> look in to the code ;)
>>>>
>>>> Anyway, the thing is that V4L2 API has enough support for auto-detection. There's
>>>> no need to duplicate stuff with MC API.
>>>
>>> It's not MC API but V4L2 subdev API. No-one has proposed to add video
>>> standard awareness to the Media controller API. There's no reason to export
>>> a video node in video decoder drivers... but I guess you didn't mean that.
>>>
>>> Would implementing ENUM/G/S_STD make sense for the camera ISP driver, that
>>> has nothing to do with any video standard?
>>
>> This is an old discussion, and we never agreed on that. Some webcam drivers
>> implement those ioctls. Others don't. Both cases are compliant with the
>> current specs. In the past, several userspace applications used to abort if those
>> ioctl's weren't implemented, but I think that this were fixed already there.
>>
>> As I said, we should define a per-device type profile in order to enforce that
>> all devices of the same type will do the same. We'll need man power to fix the
>> ones that aren't compliant, and solve the userspace issues. Volunteers needed.
>>
>> There's one point to bold on it: devices that can have either an analog input
>> or a digital input will likely need to implement ENUM/G/S_STD for both, as
>> userspace applications may fail, if the ioctl's are disabled depending on the
>> type of input. We had to implement them on several drivers, due to that.
>
> My disguised question behind this was actually that would a driver need to
> implement an ioctl that has no relevance to the driver itself at all but
> only to support another driver, yet the first driver might not have enough
> information to properly implement it?

This is done a lot at the V4L2 drivers: basically, the code at the bridge
driver just forwards the request to the sub-devices, when it doesn't know
what to do, and return the information back to the userspace, acting like
a bridge between the userspace and the sub-devices.

For example, this is what the usbvision[1] driver does for all control ioctl's:

[1] usbvision is not an example of a modern driver. It is for some old
generations of USB 1.1 media devices. I'm just using it here as it contains
one of the simplest implementations among the drivers we have, as most of
the work is done by the saa7115 driver.

static int vidioc_s_ctrl(struct file *file, void *priv,
				struct v4l2_control *ctrl)
{
	struct usb_usbvision *usbvision = video_drvdata(file);

	call_all(usbvision, core, s_ctrl, ctrl);
	return 0;
}

In other words, it just forwards the call to the tv decoder or to the sensor
(the same driver is used with both webcams and tv decoders).

The implementation for S_STD is a little more complex, as it also sets the
input at the video muxer. Even so, it is trivial:

static int vidioc_s_std(struct file *file, void *priv, v4l2_std_id *id)
{
	struct usb_usbvision *usbvision = video_drvdata(file);

	usbvision->tvnorm_id = *id;

	call_all(usbvision, core, s_std, usbvision->tvnorm_id);
	/* propagate the change to the decoder */
	usbvision_muxsel(usbvision, usbvision->ctl_input);

	return 0;
}

> It may be sometimes necessary but I would like to avoid that if possible
> since it complicates even more drivers which already are very complex.

As you can see from the two above examples, a code that will just bridge
the call to the subdevs is trivial to implement, and won't affect much
the drivers complexity.

On the other hand, Implementing the same at userspace can be much more complex, as userspace
will need to know some details about the device. For example, userspace
would need to know what nodes are affected by a command like S_STD, and
what are the requirements for each pad, to avoid putting the device into
an unsupported configuration.

>>> If you have two video decoders
>>> connected to your system, then which one should the ioctls be redirected to?
>>> What if there's a sensor and a video decoder? And how could the user know
>>> about this?
>>
>> When an input is selected (either via the V4L2 way - S_INPUT or via the MC/subdev
>> way, there's just one video decoder or sensor associated to the corresponding
>> V4L2 node).
>>
>>> It's the same old issues again... let's discuss this in the Multimedia
>>> summit.
>>
>> We can discuss more at the summit, but we should start discussing it here, as
>> otherwise we may not be able to go into a consensus there, due to the limited
>> amount of time we would have for each topic.
>
> Sounds good to me, but sometimes face-to-face discussion just is not
> replaceable.
>

We've scheduled some time for discussing it there, and we may schedule more discussions
a about that if needed during the rest of the week.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Hans Verkuil]

On Thursday, October 06, 2011 02:32:33 Mauro Carvalho Chehab wrote:
> Em 05-10-2011 20:14, Sakari Ailus escreveu:
> > Hi Mauro,
> >
> > On Wed, Oct 05, 2011 at 06:41:27PM -0300, Mauro Carvalho Chehab wrote:
> >> Em 05-10-2011 17:08, Laurent Pinchart escreveu:
> > [clip]
> >>> The pad-level API doesn't replace the V4L2 API, it complements it. I'm of
> >>> course not advocating modifying any driver to replace V4L2 ioctls by direct
> >>> subdev access. However, the pad-level API needs to be exposed to userspace, as
> >>> some harware simply can't be configured through a video node only.
> >>>
> >>> As Hans also mentionned in his reply, the pad-level API is made of two parts:
> >>> an in-kernel API made of subdev operations, and a userspace API accessed
> >>> through ioctls. As the userspace API is needed for some devices, the kernel
> >>> API needs to be implemented by drivers. We should phase out the non pad-level
> >>> format operations in favor of pad-level operations, as the former can be
> >>> implemented using the later. That has absolutely no influence on the userspace
> >>> API.
> >>
> >> What I'm seeing is that:
> >> 	- the drivers that are implementing the MC/pad API's aren't
> >> compatible with generic V4L2 applications;
> >
> > This is currently true up to a certain degree; you'll need to configure such
> > devices using media-ctl at least. Even then, such embedded systems do often
> > have automatic exposure and white balance algorithms above the V4L2 (often
> > proprietary implementations).
> >
> > To be really useful for a regular user, such algorithms would also be
> > needed.
> >
> > The two problems are separate but they still both need to be resolved to be
> > able to use general purpose V4L2 applications on such systems in a
> > meaningful way.
> 
> That's true, and the MC/pad API's were added to offer support for those proprietary
> plugins via libv4l.
> 
> That's fine, but the thing is that some developers seem to think that only the streaming
> should be done via the V4L2 API, and all the rest via pad configs, while others have
> different views.

The original design always assumed that there would be a default initial setup
that would allow the use of e.g. tvtime to get at least some sort of halfway
decent video from the device. Unfortunately, that never materialized :-(

> Also, I couldn't see a consense about input selection on drivers that
> implement MC: they could either implement S_INPUT, create one V4L device node for each
> input, or create just one devnode and let userspace (somehow) to discover the valid
> inputs and set the pipelines via MC/pad.

I don't follow. I haven't seen any MC driver yet that uses S_INPUT as that
generally doesn't make sense. But for a device like tvp5150 we probably need
it since the tvp5150 has multiple inputs. This is actually an interesting
challenge how to implement this as this is platform-level knowledge. I suspect
that the only way to do this that is sufficiently generic is to model this
with MC links.

> With all those complex and different scenarios, writing a generic plugin/application
> that would cover all possible alternatives with a mix of V4L/MC/pad API's would be
> extremely complex, and will take years to do it right.

I suspect so as well.

> In other words, we need to confine the possible solutions to the ones that will
> actually be supported.

Yes.

> >> 	- there's no way to write a generic application that works with all
> >> drivers, even implementing MC/pad API's there as each driver is taking different
> >> a approach on how to map things at the different API's, and pipeline configuration
> >> is device-dependent;
> >
> > The pipeline configuration is device specific but the interfaces are not.
> > Thus it's possible to write a generic plugin which configures the device.
> >
> > The static configuration can be kept in a text file in the first phase and
> > later on hints may be added for synamic configuration on e.g. where digital
> > gain, scaling and cropping should be performed and which resolutions to
> > advertise in enum_framesizes.
> 
> Assuming that all drivers do the same, this would be an alternative for such
> plugin.
> 
> > But we do not have such plugin yet.
> 
> As several SoC developers showed on all opportunities we've met, the complexity
> for those devices is very high. The practical effect of adding a kernel driver
> without the corresponding required library support is that we ended by
> merging incomplete drivers and nobody, except for the manufacturers/developers
> whose submitted them could actually make an userspace application to work
> with. That would be ok if the drivers would be at /staging, but they aren't.
> This situation should be fixed as soon as possible.
> 
> I'm seriously considering to not send the patches we have on our tree for those
> drivers upstream while we don't have a solution for that.

I think that's a bit overkill, I would go with not accepting new submissions
until this is sorted. Just my opinion, though.

> >> 	- there's no effort to publish patches to libv4l to match the changes
> >> at the kernel driver.
> >
> > I'd prefer concentrating all efforts towards a single, generic plugin. As
> > noted before, a plugin for OMAP 3 ISP exists, but I don't think it does
> > anything a generic plugin couldn't do, so it hasn't been merged.
> >
> > I'm working on patches to move the text-based pipeline configuration to
> > libmediactl and libv4l2subdev and will post them to the list in the coming
> > few days, among with a few other improvements. This is one of the first
> > required steps towards such generic plugin.

All these libraries are on Laurent's site. Can we please move it to linuxtv?

Mauro, wouldn't it be a good idea to create a media-utils.git and merge
v4l-utils, dvb-apps and these new media utils/libs in there?

> That would be great, but it won't solve, as more mess is being proposed each
> day. One thing is to set the pipelines. Another thing is to not support
> things like S_FMT/S_STD/... ioctl's. The user knows what format he wants,
> and that's what S_FMT/S_STD tells to the driver. the extra formats at the
> pipeline could be handled by a policy either at the driver or at a libv4l,
> although I don't think that a generic libv4l plugin would be capable of
> doing it (see more about that bellow).
< 
> > Unfortunately I haven't been able to use much of my time on this; help would
> > indeed be appreciated from any interested party.
> >
> > [clip]
> >
> >> I'm fine on providing raw interfaces, just like we have for some types of device
> >> (like the block raw interfaces used by CD-ROM drivers) as a bonus, but this should
> >> never replace an API where an application developed by a third party could work
> >> with all media hardware, without needing hardware specific details.
> >
> > I agree.
> >
> > [clip]
> >
> >>>>>> If the application wants a different image resolution, it will use
> >>>>>> S_FMT. In this case, what userspace expects is that the driver will
> >>>>>> scale, if supported, or return -EINVAL otherwise.
> >>>>>
> >>>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
> >>>>> application will set the format on the OMAP3 ISP resizer input and output
> >>>>> pads to configure scaling.
> >>>>
> >>>> The V4L2 API doesn't tell where a function like scaler will be implemented.
> >>>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
> >>>> V4L2 call is sent.
> >>>
> >>> By rolling a dice ? :-)
> >>
> >> By using good sense. I never had a case where I had doubts about where the
> >> scaling should be implemented on the drivers I've coded. For omap3/tvp5151, the
> >> decision is also clear: it should be done at the bridge (omap3) resizer, as the
> >> demod doesn't support scaling.
> >
> > Good sense in this case would require knowledge of the tv tuner in the
> > camera ISP driver. One could also, and actually does, connect sensors which
> > can do scaling in two steps to the same ISP. How does the ISP driver now
> > know where to do scaling?
> 
> Does it actually make sense to allow scaling on both? In the case of the
> generic plugin you're writing, would it do scale on both? If not, how
> would it decide where to scale? Moving the problem to userspace won't solve
> it, and would require userspace to take some hard decisions based on the specific
> hardware designs. In practice, it means that, every time a hardware is updated
> (a new sensor is added, etc), both the plugin and the driver will need to be
> touched, at the same time. This also means that the userspace plugin will
> be dependent on an specific kernel version, which would be a real mess to
> the distributors to package the libv4l plugin.
> 
> Btw, we have some devices that support scaling on both tv decoder and bridge.
> As not all tv decoders have scaling, the decision were to implement scaling
> it only at the bridge. This was as good as doing it at the sensor (or even
> better, as some bridges have temporal/spacial decimation filtering, with
> provides a better decimation than a pure spacial filtering, and can reduce
> the quantization noise).

The subdevs still need to support scaling since there are bridges that do
not do any scaling (e.g. the Conexant bridges in ivtv and cx18).

> > OMAP 3 ISP as such is a relatively simple one; there are ISPs which have two
> > scalers and ability to write the image to memory also in between the two.
> >
> > I don't think a driver, be that tv tuner / sensor or ISP driver, has enough
> > information to perform the pipeline configuration which is mandatory to
> > implement the standard V4L2 ioctls for e.g. format setting and cropping. We
> > actually had such an implementation of the OMAP 3 ISP driver around 2009
> > (but without MC or V4L2 subdevs, still the underlying technical issues are
> > the same) and it was a disaster. The patches were posted to linux-media a
> > few times back then if you're interested. It was evident something had to be
> > done, and now we have the Media controller and V4L2 subdev user space APIs.
> 
> As I said before, you're just transfering the problem from the kernel to
> something else, without actually solving it.

I agree with Mauro here. The reason it was a disaster in the omap driver is
that they tried to use the V4L2 API as the main API. That truly doesn't work.
But what I want to see is just an initial default pipeline that allows you
to test video with a standard application. Nothing fancy. That should be
easy on omap3: you have effectively three inputs (two serial, one parallel),
and you can create a default pipeline from there. Once userspace starts
manually messing around with the pipeline you can drop support it, that's
fine by me.

Whether or not you include a scaler in the default pipeline is optional
as far as I am concerned.

I fear that plugins just don't cut it. And even if they do, that it will be
very easy for the plugin and the corresponding driver to go out of sync.

Note that we could well have both: a simple default pipeline, and a more
complex plugin that gives more functionality/better quality.

But I remain very skeptical about having just a plugin.

Regards,

	Hans

> > I want to reiterate that I'm very much in favour of supporting generic V4L2
> > applications on these devices. The additional support that these application
> > need on top of the drivers requires making policy decisions that
> > applications intended for embedded devices (such as Fcam) often run on
> > embedded devices need to make themselves. To support both using the same
> > drivers, these policy decisions must be taken in the user space, not in the
> > kernel. In my opinion libv4l is in perfect spot to fill this gap.
> >
> > Kind regards,
> >
> 

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Hans Verkuil]

On Thursday, October 06, 2011 09:09:26 Hans Verkuil wrote:
> On Thursday, October 06, 2011 02:32:33 Mauro Carvalho Chehab wrote:
> > Em 05-10-2011 20:14, Sakari Ailus escreveu:
> > > Hi Mauro,
> > >
> > > On Wed, Oct 05, 2011 at 06:41:27PM -0300, Mauro Carvalho Chehab wrote:
> > >> Em 05-10-2011 17:08, Laurent Pinchart escreveu:
> > > [clip]
> > >>> The pad-level API doesn't replace the V4L2 API, it complements it. I'm of
> > >>> course not advocating modifying any driver to replace V4L2 ioctls by direct
> > >>> subdev access. However, the pad-level API needs to be exposed to userspace, as
> > >>> some harware simply can't be configured through a video node only.
> > >>>
> > >>> As Hans also mentionned in his reply, the pad-level API is made of two parts:
> > >>> an in-kernel API made of subdev operations, and a userspace API accessed
> > >>> through ioctls. As the userspace API is needed for some devices, the kernel
> > >>> API needs to be implemented by drivers. We should phase out the non pad-level
> > >>> format operations in favor of pad-level operations, as the former can be
> > >>> implemented using the later. That has absolutely no influence on the userspace
> > >>> API.
> > >>
> > >> What I'm seeing is that:
> > >> 	- the drivers that are implementing the MC/pad API's aren't
> > >> compatible with generic V4L2 applications;
> > >
> > > This is currently true up to a certain degree; you'll need to configure such
> > > devices using media-ctl at least. Even then, such embedded systems do often
> > > have automatic exposure and white balance algorithms above the V4L2 (often
> > > proprietary implementations).
> > >
> > > To be really useful for a regular user, such algorithms would also be
> > > needed.
> > >
> > > The two problems are separate but they still both need to be resolved to be
> > > able to use general purpose V4L2 applications on such systems in a
> > > meaningful way.
> > 
> > That's true, and the MC/pad API's were added to offer support for those proprietary
> > plugins via libv4l.
> > 
> > That's fine, but the thing is that some developers seem to think that only the streaming
> > should be done via the V4L2 API, and all the rest via pad configs, while others have
> > different views.
> 
> The original design always assumed that there would be a default initial setup
> that would allow the use of e.g. tvtime to get at least some sort of halfway
> decent video from the device. Unfortunately, that never materialized :-(
> 
> > Also, I couldn't see a consense about input selection on drivers that
> > implement MC: they could either implement S_INPUT, create one V4L device node for each
> > input, or create just one devnode and let userspace (somehow) to discover the valid
> > inputs and set the pipelines via MC/pad.
> 
> I don't follow. I haven't seen any MC driver yet that uses S_INPUT as that
> generally doesn't make sense. But for a device like tvp5150 we probably need
> it since the tvp5150 has multiple inputs. This is actually an interesting
> challenge how to implement this as this is platform-level knowledge. I suspect
> that the only way to do this that is sufficiently generic is to model this
> with MC links.
> 
> > With all those complex and different scenarios, writing a generic plugin/application
> > that would cover all possible alternatives with a mix of V4L/MC/pad API's would be
> > extremely complex, and will take years to do it right.
> 
> I suspect so as well.
> 
> > In other words, we need to confine the possible solutions to the ones that will
> > actually be supported.
> 
> Yes.
> 
> > >> 	- there's no way to write a generic application that works with all
> > >> drivers, even implementing MC/pad API's there as each driver is taking different
> > >> a approach on how to map things at the different API's, and pipeline configuration
> > >> is device-dependent;
> > >
> > > The pipeline configuration is device specific but the interfaces are not.
> > > Thus it's possible to write a generic plugin which configures the device.
> > >
> > > The static configuration can be kept in a text file in the first phase and
> > > later on hints may be added for synamic configuration on e.g. where digital
> > > gain, scaling and cropping should be performed and which resolutions to
> > > advertise in enum_framesizes.
> > 
> > Assuming that all drivers do the same, this would be an alternative for such
> > plugin.
> > 
> > > But we do not have such plugin yet.
> > 
> > As several SoC developers showed on all opportunities we've met, the complexity
> > for those devices is very high. The practical effect of adding a kernel driver
> > without the corresponding required library support is that we ended by
> > merging incomplete drivers and nobody, except for the manufacturers/developers
> > whose submitted them could actually make an userspace application to work
> > with. That would be ok if the drivers would be at /staging, but they aren't.
> > This situation should be fixed as soon as possible.
> > 
> > I'm seriously considering to not send the patches we have on our tree for those
> > drivers upstream while we don't have a solution for that.
> 
> I think that's a bit overkill, I would go with not accepting new submissions
> until this is sorted. Just my opinion, though.
> 
> > >> 	- there's no effort to publish patches to libv4l to match the changes
> > >> at the kernel driver.
> > >
> > > I'd prefer concentrating all efforts towards a single, generic plugin. As
> > > noted before, a plugin for OMAP 3 ISP exists, but I don't think it does
> > > anything a generic plugin couldn't do, so it hasn't been merged.
> > >
> > > I'm working on patches to move the text-based pipeline configuration to
> > > libmediactl and libv4l2subdev and will post them to the list in the coming
> > > few days, among with a few other improvements. This is one of the first
> > > required steps towards such generic plugin.
> 
> All these libraries are on Laurent's site. Can we please move it to linuxtv?
> 
> Mauro, wouldn't it be a good idea to create a media-utils.git and merge
> v4l-utils, dvb-apps and these new media utils/libs in there?
> 
> > That would be great, but it won't solve, as more mess is being proposed each
> > day. One thing is to set the pipelines. Another thing is to not support
> > things like S_FMT/S_STD/... ioctl's. The user knows what format he wants,
> > and that's what S_FMT/S_STD tells to the driver. the extra formats at the
> > pipeline could be handled by a policy either at the driver or at a libv4l,
> > although I don't think that a generic libv4l plugin would be capable of
> > doing it (see more about that bellow).
> < 
> > > Unfortunately I haven't been able to use much of my time on this; help would
> > > indeed be appreciated from any interested party.
> > >
> > > [clip]
> > >
> > >> I'm fine on providing raw interfaces, just like we have for some types of device
> > >> (like the block raw interfaces used by CD-ROM drivers) as a bonus, but this should
> > >> never replace an API where an application developed by a third party could work
> > >> with all media hardware, without needing hardware specific details.
> > >
> > > I agree.
> > >
> > > [clip]
> > >
> > >>>>>> If the application wants a different image resolution, it will use
> > >>>>>> S_FMT. In this case, what userspace expects is that the driver will
> > >>>>>> scale, if supported, or return -EINVAL otherwise.
> > >>>>>
> > >>>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
> > >>>>> application will set the format on the OMAP3 ISP resizer input and output
> > >>>>> pads to configure scaling.
> > >>>>
> > >>>> The V4L2 API doesn't tell where a function like scaler will be implemented.
> > >>>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
> > >>>> V4L2 call is sent.
> > >>>
> > >>> By rolling a dice ? :-)
> > >>
> > >> By using good sense. I never had a case where I had doubts about where the
> > >> scaling should be implemented on the drivers I've coded. For omap3/tvp5151, the
> > >> decision is also clear: it should be done at the bridge (omap3) resizer, as the
> > >> demod doesn't support scaling.
> > >
> > > Good sense in this case would require knowledge of the tv tuner in the
> > > camera ISP driver. One could also, and actually does, connect sensors which
> > > can do scaling in two steps to the same ISP. How does the ISP driver now
> > > know where to do scaling?
> > 
> > Does it actually make sense to allow scaling on both? In the case of the
> > generic plugin you're writing, would it do scale on both? If not, how
> > would it decide where to scale? Moving the problem to userspace won't solve
> > it, and would require userspace to take some hard decisions based on the specific
> > hardware designs. In practice, it means that, every time a hardware is updated
> > (a new sensor is added, etc), both the plugin and the driver will need to be
> > touched, at the same time. This also means that the userspace plugin will
> > be dependent on an specific kernel version, which would be a real mess to
> > the distributors to package the libv4l plugin.
> > 
> > Btw, we have some devices that support scaling on both tv decoder and bridge.
> > As not all tv decoders have scaling, the decision were to implement scaling
> > it only at the bridge. This was as good as doing it at the sensor (or even
> > better, as some bridges have temporal/spacial decimation filtering, with
> > provides a better decimation than a pure spacial filtering, and can reduce
> > the quantization noise).
> 
> The subdevs still need to support scaling since there are bridges that do
> not do any scaling (e.g. the Conexant bridges in ivtv and cx18).
> 
> > > OMAP 3 ISP as such is a relatively simple one; there are ISPs which have two
> > > scalers and ability to write the image to memory also in between the two.
> > >
> > > I don't think a driver, be that tv tuner / sensor or ISP driver, has enough
> > > information to perform the pipeline configuration which is mandatory to
> > > implement the standard V4L2 ioctls for e.g. format setting and cropping. We
> > > actually had such an implementation of the OMAP 3 ISP driver around 2009
> > > (but without MC or V4L2 subdevs, still the underlying technical issues are
> > > the same) and it was a disaster. The patches were posted to linux-media a
> > > few times back then if you're interested. It was evident something had to be
> > > done, and now we have the Media controller and V4L2 subdev user space APIs.
> > 
> > As I said before, you're just transfering the problem from the kernel to
> > something else, without actually solving it.
> 
> I agree with Mauro here. The reason it was a disaster in the omap driver is
> that they tried to use the V4L2 API as the main API. That truly doesn't work.
> But what I want to see is just an initial default pipeline that allows you
> to test video with a standard application. Nothing fancy. That should be
> easy on omap3: you have effectively three inputs (two serial, one parallel),
> and you can create a default pipeline from there. Once userspace starts
> manually messing around with the pipeline you can drop support it, that's
> fine by me.
> 
> Whether or not you include a scaler in the default pipeline is optional
> as far as I am concerned.
> 
> I fear that plugins just don't cut it. And even if they do, that it will be
> very easy for the plugin and the corresponding driver to go out of sync.
> 
> Note that we could well have both: a simple default pipeline, and a more
> complex plugin that gives more functionality/better quality.
> 
> But I remain very skeptical about having just a plugin.

I want to add one more thing: right now the MC is only used on specialized
hardware, but I'm sure we will start to see such SoC devices on laptops, etc.
in the future running generic linux distros. There we will need to support
generic applications as well.

Regards,

	Hans

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 06-10-2011 04:23, Hans Verkuil escreveu:
> On Thursday, October 06, 2011 09:09:26 Hans Verkuil wrote:
>> On Thursday, October 06, 2011 02:32:33 Mauro Carvalho Chehab wrote:
>>> Em 05-10-2011 20:14, Sakari Ailus escreveu:
>>>> Hi Mauro,
>>>>
>>>> On Wed, Oct 05, 2011 at 06:41:27PM -0300, Mauro Carvalho Chehab wrote:
>>>>> Em 05-10-2011 17:08, Laurent Pinchart escreveu:
>>>> [clip]
>>>>>> The pad-level API doesn't replace the V4L2 API, it complements it. I'm of
>>>>>> course not advocating modifying any driver to replace V4L2 ioctls by direct
>>>>>> subdev access. However, the pad-level API needs to be exposed to userspace, as
>>>>>> some harware simply can't be configured through a video node only.
>>>>>>
>>>>>> As Hans also mentionned in his reply, the pad-level API is made of two parts:
>>>>>> an in-kernel API made of subdev operations, and a userspace API accessed
>>>>>> through ioctls. As the userspace API is needed for some devices, the kernel
>>>>>> API needs to be implemented by drivers. We should phase out the non pad-level
>>>>>> format operations in favor of pad-level operations, as the former can be
>>>>>> implemented using the later. That has absolutely no influence on the userspace
>>>>>> API.
>>>>>
>>>>> What I'm seeing is that:
>>>>> 	- the drivers that are implementing the MC/pad API's aren't
>>>>> compatible with generic V4L2 applications;
>>>>
>>>> This is currently true up to a certain degree; you'll need to configure such
>>>> devices using media-ctl at least. Even then, such embedded systems do often
>>>> have automatic exposure and white balance algorithms above the V4L2 (often
>>>> proprietary implementations).
>>>>
>>>> To be really useful for a regular user, such algorithms would also be
>>>> needed.
>>>>
>>>> The two problems are separate but they still both need to be resolved to be
>>>> able to use general purpose V4L2 applications on such systems in a
>>>> meaningful way.
>>>
>>> That's true, and the MC/pad API's were added to offer support for those proprietary
>>> plugins via libv4l.
>>>
>>> That's fine, but the thing is that some developers seem to think that only the streaming
>>> should be done via the V4L2 API, and all the rest via pad configs, while others have
>>> different views.
>>
>> The original design always assumed that there would be a default initial setup
>> that would allow the use of e.g. tvtime to get at least some sort of halfway
>> decent video from the device. Unfortunately, that never materialized :-(

Yes. We need to concentrate on how to fix this gap during our meeting at the Workshop days.

>>
>>> Also, I couldn't see a consense about input selection on drivers that
>>> implement MC: they could either implement S_INPUT, create one V4L device node for each
>>> input, or create just one devnode and let userspace (somehow) to discover the valid
>>> inputs and set the pipelines via MC/pad.
>>
>> I don't follow. I haven't seen any MC driver yet that uses S_INPUT as that
>> generally doesn't make sense. But for a device like tvp5150 we probably need
>> it since the tvp5150 has multiple inputs. This is actually an interesting
>> challenge how to implement this as this is platform-level knowledge. I suspect
>> that the only way to do this that is sufficiently generic is to model this
>> with MC links.

$ git grep s_input drivers/media/video/s5p-*
drivers/media/video/s5p-fimc/fimc-capture.c:      .vidioc_s_input                 = fimc_cap_s_input,

The current code does nothing, but take a look at what was there before changeset 3e002182.

 From the discussions we had at the pull request that s_input code were being changed/removed,
It became clear to me that omap3 drivers took one direction, and s5p drivers took another
direction, in terms on how to associate the V4L2 device nodes with the IP blocks.

>>> With all those complex and different scenarios, writing a generic plugin/application
>>> that would cover all possible alternatives with a mix of V4L/MC/pad API's would be
>>> extremely complex, and will take years to do it right.
>>
>> I suspect so as well.
>>
>>> In other words, we need to confine the possible solutions to the ones that will
>>> actually be supported.
>>
>> Yes.
>>
>>>>> 	- there's no way to write a generic application that works with all
>>>>> drivers, even implementing MC/pad API's there as each driver is taking different
>>>>> a approach on how to map things at the different API's, and pipeline configuration
>>>>> is device-dependent;
>>>>
>>>> The pipeline configuration is device specific but the interfaces are not.
>>>> Thus it's possible to write a generic plugin which configures the device.
>>>>
>>>> The static configuration can be kept in a text file in the first phase and
>>>> later on hints may be added for synamic configuration on e.g. where digital
>>>> gain, scaling and cropping should be performed and which resolutions to
>>>> advertise in enum_framesizes.
>>>
>>> Assuming that all drivers do the same, this would be an alternative for such
>>> plugin.
>>>
>>>> But we do not have such plugin yet.
>>>
>>> As several SoC developers showed on all opportunities we've met, the complexity
>>> for those devices is very high. The practical effect of adding a kernel driver
>>> without the corresponding required library support is that we ended by
>>> merging incomplete drivers and nobody, except for the manufacturers/developers
>>> whose submitted them could actually make an userspace application to work
>>> with. That would be ok if the drivers would be at /staging, but they aren't.
>>> This situation should be fixed as soon as possible.
>>>
>>> I'm seriously considering to not send the patches we have on our tree for those
>>> drivers upstream while we don't have a solution for that.
>>
>> I think that's a bit overkill, I would go with not accepting new submissions
>> until this is sorted. Just my opinion, though.

I'm planning to take another review on those patches before sending them out,
during the next merge window, in order to figure out if they aren't adding more
gaps between what's desired and what was actually implemented on them, before
taking such decision.

>>
>>>>> 	- there's no effort to publish patches to libv4l to match the changes
>>>>> at the kernel driver.
>>>>
>>>> I'd prefer concentrating all efforts towards a single, generic plugin. As
>>>> noted before, a plugin for OMAP 3 ISP exists, but I don't think it does
>>>> anything a generic plugin couldn't do, so it hasn't been merged.
>>>>
>>>> I'm working on patches to move the text-based pipeline configuration to
>>>> libmediactl and libv4l2subdev and will post them to the list in the coming
>>>> few days, among with a few other improvements. This is one of the first
>>>> required steps towards such generic plugin.
>>
>> All these libraries are on Laurent's site. Can we please move it to linuxtv?

Yes, please.

>>
>> Mauro, wouldn't it be a good idea to create a media-utils.git and merge
>> v4l-utils, dvb-apps and these new media utils/libs in there?

I like that idea. I remember that some dvb people argued against when we
first come to it, but I think that merging both is the right thing to do.

In any case, libmediactl/libv4l2subdev should, IMHO, be part of the v4l-utils.

I suggest to open a separate thread for this subject.

For stable distros, merging packages are painful, as their policies may forbid
package source removal. So, maybe it makes sense to have something like:
"./configure --disable-[feature]" in order to allow them to keep maintaining
separate sources for separate parts of a media-utils tree. That also means
that a "--disable-libv4l" would force libv4l-aware applications to be built
statically linked.

>>> That would be great, but it won't solve, as more mess is being proposed each
>>> day. One thing is to set the pipelines. Another thing is to not support
>>> things like S_FMT/S_STD/... ioctl's. The user knows what format he wants,
>>> and that's what S_FMT/S_STD tells to the driver. the extra formats at the
>>> pipeline could be handled by a policy either at the driver or at a libv4l,
>>> although I don't think that a generic libv4l plugin would be capable of
>>> doing it (see more about that bellow).
>> <
>>>> Unfortunately I haven't been able to use much of my time on this; help would
>>>> indeed be appreciated from any interested party.
>>>>
>>>> [clip]
>>>>
>>>>> I'm fine on providing raw interfaces, just like we have for some types of device
>>>>> (like the block raw interfaces used by CD-ROM drivers) as a bonus, but this should
>>>>> never replace an API where an application developed by a third party could work
>>>>> with all media hardware, without needing hardware specific details.
>>>>
>>>> I agree.
>>>>
>>>> [clip]
>>>>
>>>>>>>>> If the application wants a different image resolution, it will use
>>>>>>>>> S_FMT. In this case, what userspace expects is that the driver will
>>>>>>>>> scale, if supported, or return -EINVAL otherwise.
>>>>>>>>
>>>>>>>> With the OMAP3 ISP, which is I believe what Javier was asking about, the
>>>>>>>> application will set the format on the OMAP3 ISP resizer input and output
>>>>>>>> pads to configure scaling.
>>>>>>>
>>>>>>> The V4L2 API doesn't tell where a function like scaler will be implemented.
>>>>>>> So, it is fine to implement it at tvp5151 or at the omap3 resizer, when a
>>>>>>> V4L2 call is sent.
>>>>>>
>>>>>> By rolling a dice ? :-)
>>>>>
>>>>> By using good sense. I never had a case where I had doubts about where the
>>>>> scaling should be implemented on the drivers I've coded. For omap3/tvp5151, the
>>>>> decision is also clear: it should be done at the bridge (omap3) resizer, as the
>>>>> demod doesn't support scaling.
>>>>
>>>> Good sense in this case would require knowledge of the tv tuner in the
>>>> camera ISP driver. One could also, and actually does, connect sensors which
>>>> can do scaling in two steps to the same ISP. How does the ISP driver now
>>>> know where to do scaling?
>>>
>>> Does it actually make sense to allow scaling on both? In the case of the
>>> generic plugin you're writing, would it do scale on both? If not, how
>>> would it decide where to scale? Moving the problem to userspace won't solve
>>> it, and would require userspace to take some hard decisions based on the specific
>>> hardware designs. In practice, it means that, every time a hardware is updated
>>> (a new sensor is added, etc), both the plugin and the driver will need to be
>>> touched, at the same time. This also means that the userspace plugin will
>>> be dependent on an specific kernel version, which would be a real mess to
>>> the distributors to package the libv4l plugin.
>>>
>>> Btw, we have some devices that support scaling on both tv decoder and bridge.
>>> As not all tv decoders have scaling, the decision were to implement scaling
>>> it only at the bridge. This was as good as doing it at the sensor (or even
>>> better, as some bridges have temporal/spacial decimation filtering, with
>>> provides a better decimation than a pure spacial filtering, and can reduce
>>> the quantization noise).
>>
>> The subdevs still need to support scaling since there are bridges that do
>> not do any scaling (e.g. the Conexant bridges in ivtv and cx18).

Yes, sure. But the thing is: there's not much sense to offer both ways to
userspace for the same device, at the same time[1], as doing that will just
reduce the image quality. So, just one scaling will be used in practice. Only
a hardware-aware code will know what's the better scaling.

[1] The only exception I can see where two scalers could be used in practice is
if the same input being directed to two different outputs, e. g. a "stored stream"
and a scaled "preview stream". But, in this case, the scale that the preview stream
will use will not be the sensor's/tv demod scaling. So, again, the decision about
where the scaling will be is clear.

>>
>>>> OMAP 3 ISP as such is a relatively simple one; there are ISPs which have two
>>>> scalers and ability to write the image to memory also in between the two.
>>>>
>>>> I don't think a driver, be that tv tuner / sensor or ISP driver, has enough
>>>> information to perform the pipeline configuration which is mandatory to
>>>> implement the standard V4L2 ioctls for e.g. format setting and cropping. We
>>>> actually had such an implementation of the OMAP 3 ISP driver around 2009
>>>> (but without MC or V4L2 subdevs, still the underlying technical issues are
>>>> the same) and it was a disaster. The patches were posted to linux-media a
>>>> few times back then if you're interested. It was evident something had to be
>>>> done, and now we have the Media controller and V4L2 subdev user space APIs.
>>>
>>> As I said before, you're just transfering the problem from the kernel to
>>> something else, without actually solving it.
>>
>> I agree with Mauro here. The reason it was a disaster in the omap driver is
>> that they tried to use the V4L2 API as the main API. That truly doesn't work.
>> But what I want to see is just an initial default pipeline that allows you
>> to test video with a standard application. Nothing fancy. That should be
>> easy on omap3: you have effectively three inputs (two serial, one parallel),
>> and you can create a default pipeline from there. Once userspace starts
>> manually messing around with the pipeline you can drop support it, that's
>> fine by me.
>>
>> Whether or not you include a scaler in the default pipeline is optional
>> as far as I am concerned.

I think that such default pipeline should include a scaler, especially if the
sensor(s)/demod(s) on such pipeline don't have it.

>> I fear that plugins just don't cut it. And even if they do, that it will be
>> very easy for the plugin and the corresponding driver to go out of sync.
>>
>> Note that we could well have both: a simple default pipeline, and a more
>> complex plugin that gives more functionality/better quality.
>>
>> But I remain very skeptical about having just a plugin.

Agreed.

>
> I want to add one more thing: right now the MC is only used on specialized
> hardware, but I'm sure we will start to see such SoC devices on laptops, etc.
> in the future running generic linux distros. There we will need to support
> generic applications as well.

I have the same feeling. Newer tablet devices with hdmi output will likely replace
netbooks and even may even replace mass-market desktops. They're all based on SoC
chips, and I bet will want to run a powerful Linux distribution on them for such type
of usage.

Regards,
Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Laurent Pinchart]

Hi Mauro,

On Thursday 06 October 2011 03:41:50 Mauro Carvalho Chehab wrote:
> Em 05-10-2011 20:41, Sakari Ailus escreveu:
> > On Mon, Oct 03, 2011 at 04:36:44PM -0300, Mauro Carvalho Chehab wrote:
> >> Em 03-10-2011 16:01, Sakari Ailus escreveu:
> >>
> >>> It's the same old issues again... let's discuss this in the Multimedia
> >>> summit.
> >> 
> >> We can discuss more at the summit, but we should start discussing it
> >> here, as otherwise we may not be able to go into a consensus there, due
> >> to the limited amount of time we would have for each topic.
> > 
> > Sounds good to me, but sometimes face-to-face discussion just is not
> > replaceable.
> 
> We've scheduled some time for discussing it there, and we may schedule more
> discussions a about that if needed during the rest of the week.

This is clearly a hot topic, and I believe there are some basic 
misunderstandings (probably on all sides) that would be much easier to solve 
with a face to face meeting. The kernel summit is only a couple of weeks away, 
what about taking a bit of psychological rest until then ? :-)

-- 
Regards,

Laurent Pinchart

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Hans Verkuil]

On Thursday 06 October 2011 13:51:15 Mauro Carvalho Chehab wrote:
> Em 06-10-2011 04:23, Hans Verkuil escreveu:
> > On Thursday, October 06, 2011 09:09:26 Hans Verkuil wrote:
> >> On Thursday, October 06, 2011 02:32:33 Mauro Carvalho Chehab wrote:
> >>> Also, I couldn't see a consense about input selection on drivers that
> >>> implement MC: they could either implement S_INPUT, create one V4L
> >>> device node for each input, or create just one devnode and let
> >>> userspace (somehow) to discover the valid inputs and set the pipelines
> >>> via MC/pad.
> >> 
> >> I don't follow. I haven't seen any MC driver yet that uses S_INPUT as
> >> that generally doesn't make sense. But for a device like tvp5150 we
> >> probably need it since the tvp5150 has multiple inputs. This is
> >> actually an interesting challenge how to implement this as this is
> >> platform-level knowledge. I suspect that the only way to do this that
> >> is sufficiently generic is to model this with MC links.
> 
> $ git grep s_input drivers/media/video/s5p-*
> drivers/media/video/s5p-fimc/fimc-capture.c:      .vidioc_s_input          
>       = fimc_cap_s_input,
> 
> The current code does nothing, but take a look at what was there before
> changeset 3e002182.
> 
>  From the discussions we had at the pull request that s_input code were
> being changed/removed, It became clear to me that omap3 drivers took one
> direction, and s5p drivers took another direction, in terms on how to
> associate the V4L2 device nodes with the IP blocks.

>From what I remember the s5p drivers converged/are converging on the same
approach as omap3. I don't believe there is any discussion anymore on what is 
the correct method.

> >> All these libraries are on Laurent's site. Can we please move it to
> >> linuxtv?
> 
> Yes, please.
> 
> >> Mauro, wouldn't it be a good idea to create a media-utils.git and merge
> >> v4l-utils, dvb-apps and these new media utils/libs in there?
> 
> I like that idea. I remember that some dvb people argued against when we
> first come to it, but I think that merging both is the right thing to do.
> 
> In any case, libmediactl/libv4l2subdev should, IMHO, be part of the
> v4l-utils.
> 
> I suggest to open a separate thread for this subject.
> 
> For stable distros, merging packages are painful, as their policies may
> forbid package source removal. So, maybe it makes sense to have something
> like: "./configure --disable-[feature]" in order to allow them to keep
> maintaining separate sources for separate parts of a media-utils tree.
> That also means that a "--disable-libv4l" would force libv4l-aware
> applications to be built statically linked.

Seems very complicated to me. But I'll start a separate thread for this.

...

> >> Whether or not you include a scaler in the default pipeline is optional
> >> as far as I am concerned.
> 
> I think that such default pipeline should include a scaler, especially if
> the sensor(s)/demod(s) on such pipeline don't have it.

That would be the ideal situation, yes, but for now I'd be happy just to get a 
picture out of an SoC :-)

Regards,

	Hans

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Mauro Carvalho Chehab]

Em 06-10-2011 09:06, Hans Verkuil escreveu:
> On Thursday 06 October 2011 13:51:15 Mauro Carvalho Chehab wrote:
>> Em 06-10-2011 04:23, Hans Verkuil escreveu:
>>> On Thursday, October 06, 2011 09:09:26 Hans Verkuil wrote:
>>>> On Thursday, October 06, 2011 02:32:33 Mauro Carvalho Chehab wrote:
>>>>> Also, I couldn't see a consense about input selection on drivers that
>>>>> implement MC: they could either implement S_INPUT, create one V4L
>>>>> device node for each input, or create just one devnode and let
>>>>> userspace (somehow) to discover the valid inputs and set the pipelines
>>>>> via MC/pad.
>>>>
>>>> I don't follow. I haven't seen any MC driver yet that uses S_INPUT as
>>>> that generally doesn't make sense. But for a device like tvp5150 we
>>>> probably need it since the tvp5150 has multiple inputs. This is
>>>> actually an interesting challenge how to implement this as this is
>>>> platform-level knowledge. I suspect that the only way to do this that
>>>> is sufficiently generic is to model this with MC links.
>>
>> $ git grep s_input drivers/media/video/s5p-*
>> drivers/media/video/s5p-fimc/fimc-capture.c:      .vidioc_s_input
>>        = fimc_cap_s_input,
>>
>> The current code does nothing, but take a look at what was there before
>> changeset 3e002182.
>>
>>   From the discussions we had at the pull request that s_input code were
>> being changed/removed, It became clear to me that omap3 drivers took one
>> direction, and s5p drivers took another direction, in terms on how to
>> associate the V4L2 device nodes with the IP blocks.
>
>  From what I remember the s5p drivers converged/are converging on the same
> approach as omap3. I don't believe there is any discussion anymore on what is
> the correct method.

With also doesn't support generic apps. Not sure if this is an improvement.

Two drivers fully following the API specs can use different and incompatible
approaches from the POV of an userspace application, as now there are several
different ways of implementing the very same thing, and those SoC drivers
are not doing what was agreed when we decided to add the MC API: properly implement
the V4L2 API, to allow existing applications to use them.

>>>> All these libraries are on Laurent's site. Can we please move it to
>>>> linuxtv?
>>
>> Yes, please.
>>
>>>> Mauro, wouldn't it be a good idea to create a media-utils.git and merge
>>>> v4l-utils, dvb-apps and these new media utils/libs in there?
>>
>> I like that idea. I remember that some dvb people argued against when we
>> first come to it, but I think that merging both is the right thing to do.
>>
>> In any case, libmediactl/libv4l2subdev should, IMHO, be part of the
>> v4l-utils.
>>
>> I suggest to open a separate thread for this subject.
>>
>> For stable distros, merging packages are painful, as their policies may
>> forbid package source removal. So, maybe it makes sense to have something
>> like: "./configure --disable-[feature]" in order to allow them to keep
>> maintaining separate sources for separate parts of a media-utils tree.
>> That also means that a "--disable-libv4l" would force libv4l-aware
>> applications to be built statically linked.
>
> Seems very complicated to me. But I'll start a separate thread for this.

It is not that complicated, but it will require some time for doing that. Autoconf
stripts are capable of doing things like that, but time is required to add
those patches into it.

>
> ...
>
>>>> Whether or not you include a scaler in the default pipeline is optional
>>>> as far as I am concerned.
>>
>> I think that such default pipeline should include a scaler, especially if
>> the sensor(s)/demod(s) on such pipeline don't have it.
>
> That would be the ideal situation, yes, but for now I'd be happy just to get a
> picture out of an SoC :-)

:)

Mauro

Re: [PATCH 3/3] [media] tvp5150: Migrate to media-controller framework and add video format detection

[Sylwester Nawrocki]

On 10/06/2011 02:06 PM, Hans Verkuil wrote:
> On Thursday 06 October 2011 13:51:15 Mauro Carvalho Chehab wrote:
>> Em 06-10-2011 04:23, Hans Verkuil escreveu:
>>> On Thursday, October 06, 2011 09:09:26 Hans Verkuil wrote:
>>>> On Thursday, October 06, 2011 02:32:33 Mauro Carvalho Chehab wrote:
>>>>> Also, I couldn't see a consense about input selection on drivers that
>>>>> implement MC: they could either implement S_INPUT, create one V4L
>>>>> device node for each input, or create just one devnode and let
>>>>> userspace (somehow) to discover the valid inputs and set the pipelines
>>>>> via MC/pad.
>>>>
>>>> I don't follow. I haven't seen any MC driver yet that uses S_INPUT as
>>>> that generally doesn't make sense. But for a device like tvp5150 we
>>>> probably need it since the tvp5150 has multiple inputs. This is
>>>> actually an interesting challenge how to implement this as this is
>>>> platform-level knowledge. I suspect that the only way to do this that
>>>> is sufficiently generic is to model this with MC links.
>>
>> $ git grep s_input drivers/media/video/s5p-*
>> drivers/media/video/s5p-fimc/fimc-capture.c:      .vidioc_s_input          
>>       = fimc_cap_s_input,
>>
>> The current code does nothing, but take a look at what was there before
>> changeset 3e002182.

Hmm, now I believe it was a mistake to originally push a half-usable driver
to mainline. It seems like the rule push early/push often just didn't work
out here. Perhaps this driver should have been merged to staging at first
place.

>>
>>  From the discussions we had at the pull request that s_input code were
>> being changed/removed, It became clear to me that omap3 drivers took one
>> direction, and s5p drivers took another direction, in terms on how to
>> associate the V4L2 device nodes with the IP blocks.

Before the MC existed in mainline the s5p capture driver hard coded some
policies in the kernel and only a part of functionality was exposed to user
space. E.g. camera could only work with instance 0 of the IP and only
parallel interface was supported. That's why s_input could be usable there.
The OMAP3 ISP driver have taken the right approach from the beginning.

> 
> From what I remember the s5p drivers converged/are converging on the same
> approach as omap3. I don't believe there is any discussion anymore on what is 
> the correct method.

Yes, plus the s5p driver creates default data paths so it possible to just
open a video node and start streaming. 
The original interface is retained, except the s_input behaviour. Which I
believe is not any meaningful issue to the applications, since they can be
coded to support one video node with two inputs or two video nodes with one
input, with any kernel version. Everything can be detected, enumerated and set
using plain old V4L2 capture interface.

>>>> All these libraries are on Laurent's site. Can we please move it to
>>>> linuxtv?
>>
>> Yes, please.
>>
>>>> Mauro, wouldn't it be a good idea to create a media-utils.git and merge
>>>> v4l-utils, dvb-apps and these new media utils/libs in there?
>>
>> I like that idea. I remember that some dvb people argued against when we
>> first come to it, but I think that merging both is the right thing to do.
>>
>> In any case, libmediactl/libv4l2subdev should, IMHO, be part of the
>> v4l-utils.
>>
>> I suggest to open a separate thread for this subject.
>>
>> For stable distros, merging packages are painful, as their policies may
>> forbid package source removal. So, maybe it makes sense to have something
>> like: "./configure --disable-[feature]" in order to allow them to keep
>> maintaining separate sources for separate parts of a media-utils tree.
>> That also means that a "--disable-libv4l" would force libv4l-aware
>> applications to be built statically linked.
> 
> Seems very complicated to me. But I'll start a separate thread for this.
> 
> ...
> 
>>>> Whether or not you include a scaler in the default pipeline is optional
>>>> as far as I am concerned.
>>
>> I think that such default pipeline should include a scaler, especially if
>> the sensor(s)/demod(s) on such pipeline don't have it.
> 
> That would be the ideal situation, yes, but for now I'd be happy just to get a 
> picture out of an SoC :-)

I agree. 

> 
> Regards,
> 
> 	Hans


Regards,
-- 
Sylwester Nawrocki
Samsung Poland R&D Center