* [PATCH v5 0/3] Fixes for colorspace logic in exynos-gsc and s5p-mfc drivers @ 2017-02-21 19:20 Thibault Saunier 2017-02-21 19:20 ` [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided Thibault Saunier ` (2 more replies) 0 siblings, 3 replies; 15+ messages in thread From: Thibault Saunier @ 2017-02-21 19:20 UTC (permalink / raw) To: linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, Thibault Saunier, linux-arm-kernel, Ulf Hansson, Hans Verkuil, Andrzej Hajda, Jeongtae Park, Kyungmin Park, Kamil Debski Hello, This patchset fixes a few issues on the colorspace logic for the exynos-gsc and s5p-mfc drivers. We now handle the colorspace in those drivers, and make sure to respect user setting if possible. We also now set the 'v4l2_pix_format:field' if userspace passed ANY, avoiding GStreamer spamming error at us about the driver not following the standard. This is the fifth version of the patch serie. Best regards, Thibault Saunier Changes in v5: - Squash commit to always use output colorspace on the capture side inside this one - Fix typo in commit message - Just adapt the field and never error out. Changes in v4: - Reword commit message to better back our assumptions on specifications - Set the colorspace only if the user passed V4L2_COLORSPACE_DEFAULT, in all other cases just use what userspace provided. Changes in v3: - Do not check values in the g_fmt functions as Andrzej explained in previous review - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' - Do not check values in the g_fmt functions as Andrzej explained in previous review - Set colorspace if user passed V4L2_COLORSPACE_DEFAULT in - Do not check values in the g_fmt functions as Andrzej explained in previous review Changes in v2: - Fix a silly build error that slipped in while rebasing the patches Thibault Saunier (3): [media] exynos-gsc: Use user configured colorspace if provided [media] s5p-mfc: Set colorspace in VIDIO_{G,TRY}_FMT if DEFAULT provided [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt drivers/media/platform/exynos-gsc/gsc-core.c | 20 +++++++++++++++----- drivers/media/platform/exynos-gsc/gsc-core.h | 1 + drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 17 +++++++++++++++++ 3 files changed, 33 insertions(+), 5 deletions(-) -- 2.11.1 ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided 2017-02-21 19:20 [PATCH v5 0/3] Fixes for colorspace logic in exynos-gsc and s5p-mfc drivers Thibault Saunier @ 2017-02-21 19:20 ` Thibault Saunier 2017-02-22 2:19 ` Hans Verkuil 2017-02-21 19:20 ` [PATCH v5 2/3] [media] s5p-mfc: Set colorspace in VIDIO_{G,TRY}_FMT if DEFAULT provided Thibault Saunier 2017-02-21 19:20 ` [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt Thibault Saunier 2 siblings, 1 reply; 15+ messages in thread From: Thibault Saunier @ 2017-02-21 19:20 UTC (permalink / raw) To: linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, Thibault Saunier, linux-arm-kernel, Ulf Hansson, Hans Verkuil Use colorspace provided by the user as we are only doing scaling and color encoding conversion, we won't be able to transform the colorspace itself and the colorspace won't mater in that operation. Also always use output colorspace on the capture side. Start using 576p as a threashold to compute the colorspace. The media documentation says that the V4L2_COLORSPACE_SMPTE170M colorspace should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV. But drivers don't agree on the display resolution that should be used as a threshold. >From EIA CEA 861B about colorimetry for various resolutions: - 5.1 480p, 480i, 576p, 576i, 240p, and 288p The color space used by the 480-line, 576-line, 240-line, and 288-line formats will likely be based on SMPTE 170M [1]. - 5.2 1080i, 1080p, and 720p The color space used by the high definition formats will likely be based on ITU-R BT.709-4 This indicates that in the case that userspace does not specify what colorspace should be used, we should use 576p as a threshold to set V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_SMPTE170M. Even if it is only 'likely' and not a requirement it is the best guess we can make. The stream should have been encoded with the information and userspace has to pass it to the driver if it is not the case, otherwise we won't be able to handle it properly anyhow. Also, check for the resolution in G_FMT instead unconditionally setting the V4L2_COLORSPACE_REC709 colorspace. Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com> Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> Reviewed-by: Andrzej Hajda <a.hajda@samsung.com> --- Changes in v5: - Squash commit to always use output colorspace on the capture side inside this one - Fix typo in commit message Changes in v4: - Reword commit message to better back our assumptions on specifications Changes in v3: - Do not check values in the g_fmt functions as Andrzej explained in previous review - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' Changes in v2: None drivers/media/platform/exynos-gsc/gsc-core.c | 20 +++++++++++++++----- drivers/media/platform/exynos-gsc/gsc-core.h | 1 + 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c b/drivers/media/platform/exynos-gsc/gsc-core.c index 59a634201830..772599de8c13 100644 --- a/drivers/media/platform/exynos-gsc/gsc-core.c +++ b/drivers/media/platform/exynos-gsc/gsc-core.c @@ -454,6 +454,7 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) } else { min_w = variant->pix_min->target_rot_dis_w; min_h = variant->pix_min->target_rot_dis_h; + pix_mp->colorspace = ctx->out_colorspace; } pr_debug("mod_x: %d, mod_y: %d, max_w: %d, max_h = %d", @@ -472,10 +473,15 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) pix_mp->num_planes = fmt->num_planes; - if (pix_mp->width >= 1280) /* HD */ - pix_mp->colorspace = V4L2_COLORSPACE_REC709; - else /* SD */ - pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ + pix_mp->colorspace = V4L2_COLORSPACE_REC709; + else /* SD */ + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; + } + + if (V4L2_TYPE_IS_OUTPUT(f->type)) + ctx->out_colorspace = pix_mp->colorspace; for (i = 0; i < pix_mp->num_planes; ++i) { struct v4l2_plane_pix_format *plane_fmt = &pix_mp->plane_fmt[i]; @@ -519,9 +525,13 @@ int gsc_g_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) pix_mp->height = frame->f_height; pix_mp->field = V4L2_FIELD_NONE; pix_mp->pixelformat = frame->fmt->pixelformat; - pix_mp->colorspace = V4L2_COLORSPACE_REC709; pix_mp->num_planes = frame->fmt->num_planes; + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ + pix_mp->colorspace = V4L2_COLORSPACE_REC709; + else /* SD */ + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; + for (i = 0; i < pix_mp->num_planes; ++i) { pix_mp->plane_fmt[i].bytesperline = (frame->f_width * frame->fmt->depth[i]) / 8; diff --git a/drivers/media/platform/exynos-gsc/gsc-core.h b/drivers/media/platform/exynos-gsc/gsc-core.h index 696217e9af66..715d9c9d8d30 100644 --- a/drivers/media/platform/exynos-gsc/gsc-core.h +++ b/drivers/media/platform/exynos-gsc/gsc-core.h @@ -376,6 +376,7 @@ struct gsc_ctx { struct v4l2_ctrl_handler ctrl_handler; struct gsc_ctrls gsc_ctrls; bool ctrls_rdy; + enum v4l2_colorspace out_colorspace; }; void gsc_set_prefbuf(struct gsc_dev *gsc, struct gsc_frame *frm); -- 2.11.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided 2017-02-21 19:20 ` [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided Thibault Saunier @ 2017-02-22 2:19 ` Hans Verkuil 2017-02-22 13:05 ` Thibault Saunier 0 siblings, 1 reply; 15+ messages in thread From: Hans Verkuil @ 2017-02-22 2:19 UTC (permalink / raw) To: Thibault Saunier, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Hans Verkuil On 02/21/2017 11:20 AM, Thibault Saunier wrote: > Use colorspace provided by the user as we are only doing scaling and > color encoding conversion, we won't be able to transform the colorspace > itself and the colorspace won't mater in that operation. > > Also always use output colorspace on the capture side. > > Start using 576p as a threashold to compute the colorspace. > The media documentation says that the V4L2_COLORSPACE_SMPTE170M colorspace > should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV. But drivers > don't agree on the display resolution that should be used as a threshold. > > From EIA CEA 861B about colorimetry for various resolutions: > > - 5.1 480p, 480i, 576p, 576i, 240p, and 288p > The color space used by the 480-line, 576-line, 240-line, and 288-line > formats will likely be based on SMPTE 170M [1]. > - 5.2 1080i, 1080p, and 720p > The color space used by the high definition formats will likely be > based on ITU-R BT.709-4 > > This indicates that in the case that userspace does not specify what > colorspace should be used, we should use 576p as a threshold to set > V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_SMPTE170M. Even if it is > only 'likely' and not a requirement it is the best guess we can make. > > The stream should have been encoded with the information and userspace > has to pass it to the driver if it is not the case, otherwise we won't be > able to handle it properly anyhow. > > Also, check for the resolution in G_FMT instead unconditionally setting > the V4L2_COLORSPACE_REC709 colorspace. > > Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com> > Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> > Reviewed-by: Andrzej Hajda <a.hajda@samsung.com> > > --- > > Changes in v5: > - Squash commit to always use output colorspace on the capture side > inside this one > - Fix typo in commit message > > Changes in v4: > - Reword commit message to better back our assumptions on specifications > > Changes in v3: > - Do not check values in the g_fmt functions as Andrzej explained in previous review > - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' > > Changes in v2: None > > drivers/media/platform/exynos-gsc/gsc-core.c | 20 +++++++++++++++----- > drivers/media/platform/exynos-gsc/gsc-core.h | 1 + > 2 files changed, 16 insertions(+), 5 deletions(-) > > diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c b/drivers/media/platform/exynos-gsc/gsc-core.c > index 59a634201830..772599de8c13 100644 > --- a/drivers/media/platform/exynos-gsc/gsc-core.c > +++ b/drivers/media/platform/exynos-gsc/gsc-core.c > @@ -454,6 +454,7 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) > } else { > min_w = variant->pix_min->target_rot_dis_w; > min_h = variant->pix_min->target_rot_dis_h; > + pix_mp->colorspace = ctx->out_colorspace; > } > > pr_debug("mod_x: %d, mod_y: %d, max_w: %d, max_h = %d", > @@ -472,10 +473,15 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) > > pix_mp->num_planes = fmt->num_planes; > > - if (pix_mp->width >= 1280) /* HD */ > - pix_mp->colorspace = V4L2_COLORSPACE_REC709; > - else /* SD */ > - pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; > + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { > + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ I'd use || instead of && here. Ditto for the next patch. > + pix_mp->colorspace = V4L2_COLORSPACE_REC709; > + else /* SD */ > + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; > + } Are you sure this is in fact how it is used? If the source of the video is the sensor (camera), then the colorspace is typically SRGB. I'm not really sure you should guess here. All a mem2mem driver should do is to pass the colorspace etc. information from the output to the capture side, and not invent things. That simply makes no sense. We may have to update the documentation or v4l2-compliance test if this is an issue. The more I think about this, the more I think we shouldn't do this guessing game. Regards, Hans > + > + if (V4L2_TYPE_IS_OUTPUT(f->type)) > + ctx->out_colorspace = pix_mp->colorspace; > > for (i = 0; i < pix_mp->num_planes; ++i) { > struct v4l2_plane_pix_format *plane_fmt = &pix_mp->plane_fmt[i]; > @@ -519,9 +525,13 @@ int gsc_g_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) > pix_mp->height = frame->f_height; > pix_mp->field = V4L2_FIELD_NONE; > pix_mp->pixelformat = frame->fmt->pixelformat; > - pix_mp->colorspace = V4L2_COLORSPACE_REC709; > pix_mp->num_planes = frame->fmt->num_planes; > > + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ > + pix_mp->colorspace = V4L2_COLORSPACE_REC709; > + else /* SD */ > + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; > + > for (i = 0; i < pix_mp->num_planes; ++i) { > pix_mp->plane_fmt[i].bytesperline = (frame->f_width * > frame->fmt->depth[i]) / 8; > diff --git a/drivers/media/platform/exynos-gsc/gsc-core.h b/drivers/media/platform/exynos-gsc/gsc-core.h > index 696217e9af66..715d9c9d8d30 100644 > --- a/drivers/media/platform/exynos-gsc/gsc-core.h > +++ b/drivers/media/platform/exynos-gsc/gsc-core.h > @@ -376,6 +376,7 @@ struct gsc_ctx { > struct v4l2_ctrl_handler ctrl_handler; > struct gsc_ctrls gsc_ctrls; > bool ctrls_rdy; > + enum v4l2_colorspace out_colorspace; > }; > > void gsc_set_prefbuf(struct gsc_dev *gsc, struct gsc_frame *frm); > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided 2017-02-22 2:19 ` Hans Verkuil @ 2017-02-22 13:05 ` Thibault Saunier 2017-02-22 18:06 ` Hans Verkuil 0 siblings, 1 reply; 15+ messages in thread From: Thibault Saunier @ 2017-02-22 13:05 UTC (permalink / raw) To: Hans Verkuil, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Hans Verkuil Hello, On 02/21/2017 11:19 PM, Hans Verkuil wrote: > > On 02/21/2017 11:20 AM, Thibault Saunier wrote: >> Use colorspace provided by the user as we are only doing scaling and >> color encoding conversion, we won't be able to transform the colorspace >> itself and the colorspace won't mater in that operation. >> >> Also always use output colorspace on the capture side. >> >> Start using 576p as a threashold to compute the colorspace. >> The media documentation says that the V4L2_COLORSPACE_SMPTE170M colorspace >> should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV. But drivers >> don't agree on the display resolution that should be used as a threshold. >> >> From EIA CEA 861B about colorimetry for various resolutions: >> >> - 5.1 480p, 480i, 576p, 576i, 240p, and 288p >> The color space used by the 480-line, 576-line, 240-line, and 288-line >> formats will likely be based on SMPTE 170M [1]. >> - 5.2 1080i, 1080p, and 720p >> The color space used by the high definition formats will likely be >> based on ITU-R BT.709-4 >> >> This indicates that in the case that userspace does not specify what >> colorspace should be used, we should use 576p as a threshold to set >> V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_SMPTE170M. Even if it is >> only 'likely' and not a requirement it is the best guess we can make. >> >> The stream should have been encoded with the information and userspace >> has to pass it to the driver if it is not the case, otherwise we won't be >> able to handle it properly anyhow. >> >> Also, check for the resolution in G_FMT instead unconditionally setting >> the V4L2_COLORSPACE_REC709 colorspace. >> >> Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com> >> Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> >> Reviewed-by: Andrzej Hajda <a.hajda@samsung.com> >> >> --- >> >> Changes in v5: >> - Squash commit to always use output colorspace on the capture side >> inside this one >> - Fix typo in commit message >> >> Changes in v4: >> - Reword commit message to better back our assumptions on specifications >> >> Changes in v3: >> - Do not check values in the g_fmt functions as Andrzej explained in previous review >> - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' >> >> Changes in v2: None >> >> drivers/media/platform/exynos-gsc/gsc-core.c | 20 +++++++++++++++----- >> drivers/media/platform/exynos-gsc/gsc-core.h | 1 + >> 2 files changed, 16 insertions(+), 5 deletions(-) >> >> diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c b/drivers/media/platform/exynos-gsc/gsc-core.c >> index 59a634201830..772599de8c13 100644 >> --- a/drivers/media/platform/exynos-gsc/gsc-core.c >> +++ b/drivers/media/platform/exynos-gsc/gsc-core.c >> @@ -454,6 +454,7 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) >> } else { >> min_w = variant->pix_min->target_rot_dis_w; >> min_h = variant->pix_min->target_rot_dis_h; >> + pix_mp->colorspace = ctx->out_colorspace; >> } >> >> pr_debug("mod_x: %d, mod_y: %d, max_w: %d, max_h = %d", >> @@ -472,10 +473,15 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) >> >> pix_mp->num_planes = fmt->num_planes; >> >> - if (pix_mp->width >= 1280) /* HD */ >> - pix_mp->colorspace = V4L2_COLORSPACE_REC709; >> - else /* SD */ >> - pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >> + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { >> + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ > I'd use || instead of && here. Ditto for the next patch. > >> + pix_mp->colorspace = V4L2_COLORSPACE_REC709; >> + else /* SD */ >> + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >> + } > Are you sure this is in fact how it is used? If the source of the video > is the sensor (camera), then the colorspace is typically SRGB. I'm not > really sure you should guess here. All a mem2mem driver should do is to > pass the colorspace etc. information from the output to the capture side, > and not invent things. That simply makes no sense. This is not a camera device but a colorspace conversion device, in many cases the info will not be passed by the userspace, basically because the info is not encoded in the media stream (this often happens). I am not inventing here, just making sure we use the most likely value in case none was provided (and if none was provided it should always be correct given that the encoded stream was not broken). In the case the source is a camera and then we use the colorspace converter then the info should copied from the camera to the transcoding node (by userspace) so there should be no problem. What I am doing here is what is done in other drivers. Regards, Thibault > > We may have to update the documentation or v4l2-compliance test if this > is an issue. The more I think about this, the more I think we shouldn't > do this guessing game. > > Regards, > > Hans > >> + >> + if (V4L2_TYPE_IS_OUTPUT(f->type)) >> + ctx->out_colorspace = pix_mp->colorspace; >> >> for (i = 0; i < pix_mp->num_planes; ++i) { >> struct v4l2_plane_pix_format *plane_fmt = &pix_mp->plane_fmt[i]; >> @@ -519,9 +525,13 @@ int gsc_g_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) >> pix_mp->height = frame->f_height; >> pix_mp->field = V4L2_FIELD_NONE; >> pix_mp->pixelformat = frame->fmt->pixelformat; >> - pix_mp->colorspace = V4L2_COLORSPACE_REC709; >> pix_mp->num_planes = frame->fmt->num_planes; >> >> + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ >> + pix_mp->colorspace = V4L2_COLORSPACE_REC709; >> + else /* SD */ >> + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >> + >> for (i = 0; i < pix_mp->num_planes; ++i) { >> pix_mp->plane_fmt[i].bytesperline = (frame->f_width * >> frame->fmt->depth[i]) / 8; >> diff --git a/drivers/media/platform/exynos-gsc/gsc-core.h b/drivers/media/platform/exynos-gsc/gsc-core.h >> index 696217e9af66..715d9c9d8d30 100644 >> --- a/drivers/media/platform/exynos-gsc/gsc-core.h >> +++ b/drivers/media/platform/exynos-gsc/gsc-core.h >> @@ -376,6 +376,7 @@ struct gsc_ctx { >> struct v4l2_ctrl_handler ctrl_handler; >> struct gsc_ctrls gsc_ctrls; >> bool ctrls_rdy; >> + enum v4l2_colorspace out_colorspace; >> }; >> >> void gsc_set_prefbuf(struct gsc_dev *gsc, struct gsc_frame *frm); >> ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided 2017-02-22 13:05 ` Thibault Saunier @ 2017-02-22 18:06 ` Hans Verkuil 2017-02-22 18:57 ` Thibault Saunier 0 siblings, 1 reply; 15+ messages in thread From: Hans Verkuil @ 2017-02-22 18:06 UTC (permalink / raw) To: Thibault Saunier, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Hans Verkuil On 02/22/2017 05:05 AM, Thibault Saunier wrote: > Hello, > > > On 02/21/2017 11:19 PM, Hans Verkuil wrote: >> >> On 02/21/2017 11:20 AM, Thibault Saunier wrote: >>> Use colorspace provided by the user as we are only doing scaling and >>> color encoding conversion, we won't be able to transform the colorspace >>> itself and the colorspace won't mater in that operation. >>> >>> Also always use output colorspace on the capture side. >>> >>> Start using 576p as a threashold to compute the colorspace. >>> The media documentation says that the V4L2_COLORSPACE_SMPTE170M colorspace >>> should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV. But drivers >>> don't agree on the display resolution that should be used as a threshold. >>> >>> From EIA CEA 861B about colorimetry for various resolutions: >>> >>> - 5.1 480p, 480i, 576p, 576i, 240p, and 288p >>> The color space used by the 480-line, 576-line, 240-line, and 288-line >>> formats will likely be based on SMPTE 170M [1]. >>> - 5.2 1080i, 1080p, and 720p >>> The color space used by the high definition formats will likely be >>> based on ITU-R BT.709-4 >>> >>> This indicates that in the case that userspace does not specify what >>> colorspace should be used, we should use 576p as a threshold to set >>> V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_SMPTE170M. Even if it is >>> only 'likely' and not a requirement it is the best guess we can make. >>> >>> The stream should have been encoded with the information and userspace >>> has to pass it to the driver if it is not the case, otherwise we won't be >>> able to handle it properly anyhow. >>> >>> Also, check for the resolution in G_FMT instead unconditionally setting >>> the V4L2_COLORSPACE_REC709 colorspace. >>> >>> Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com> >>> Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> >>> Reviewed-by: Andrzej Hajda <a.hajda@samsung.com> >>> >>> --- >>> >>> Changes in v5: >>> - Squash commit to always use output colorspace on the capture side >>> inside this one >>> - Fix typo in commit message >>> >>> Changes in v4: >>> - Reword commit message to better back our assumptions on specifications >>> >>> Changes in v3: >>> - Do not check values in the g_fmt functions as Andrzej explained in previous review >>> - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' >>> >>> Changes in v2: None >>> >>> drivers/media/platform/exynos-gsc/gsc-core.c | 20 +++++++++++++++----- >>> drivers/media/platform/exynos-gsc/gsc-core.h | 1 + >>> 2 files changed, 16 insertions(+), 5 deletions(-) >>> >>> diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c b/drivers/media/platform/exynos-gsc/gsc-core.c >>> index 59a634201830..772599de8c13 100644 >>> --- a/drivers/media/platform/exynos-gsc/gsc-core.c >>> +++ b/drivers/media/platform/exynos-gsc/gsc-core.c >>> @@ -454,6 +454,7 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) >>> } else { >>> min_w = variant->pix_min->target_rot_dis_w; >>> min_h = variant->pix_min->target_rot_dis_h; >>> + pix_mp->colorspace = ctx->out_colorspace; >>> } >>> pr_debug("mod_x: %d, mod_y: %d, max_w: %d, max_h = %d", >>> @@ -472,10 +473,15 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) >>> pix_mp->num_planes = fmt->num_planes; >>> - if (pix_mp->width >= 1280) /* HD */ >>> - pix_mp->colorspace = V4L2_COLORSPACE_REC709; >>> - else /* SD */ >>> - pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >>> + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { >>> + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ >> I'd use || instead of && here. Ditto for the next patch. >> >>> + pix_mp->colorspace = V4L2_COLORSPACE_REC709; >>> + else /* SD */ >>> + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >>> + } >> Are you sure this is in fact how it is used? If the source of the video >> is the sensor (camera), then the colorspace is typically SRGB. I'm not >> really sure you should guess here. All a mem2mem driver should do is to >> pass the colorspace etc. information from the output to the capture side, >> and not invent things. That simply makes no sense. > > This is not a camera device but a colorspace conversion device, in many Not really, this is a color encoding conversion device. I.e. it only affects the Y'CbCr encoding and quantization range. The colorspace (aka chromaticities) and transfer function remain unchanged. In fact, I suspect (correct me if I am wrong) that it only converts between RGB, Y'CbCr 601 and Y'CbCr 709 encodings, where RGB is full range and the Y'CbCr formats are limited range. If you pass in limited range RGB it will probably do the wrong thing as I don't seen any quantization checks in the code. So the colorspace and xfer_func fields remain unchanged by this driver. If you want to do this really correctly, then you need to do more. I don't have time right now to go into this in-depth, but I will try to get back to this on Monday. I am thinking of documenting this as part of the V4L2 colorspace documentation. This stuff is complex and if you don't know the rules then it can be hard to implement correctly. > cases the info will not be passed by the userspace, basically because the info > is not encoded in the media stream (this often happens). I am not inventing here, > just making sure we use the most likely value in case none was provided (and if none > was provided it should always be correct given that the encoded stream was not broken). > > In the case the source is a camera and then we use the colorspace converter then the info > should copied from the camera to the transcoding node (by userspace) so there should be > no problem. > > What I am doing here is what is done in other drivers. Most (all?) other mem2mem drivers do not do color encoding conversion, they just copy all the colorspace info from output to capture. If there are other m2m drivers that do this, then I doubt they do the right thing. They are probably right for most cases, but not all. Regards, Hans ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided 2017-02-22 18:06 ` Hans Verkuil @ 2017-02-22 18:57 ` Thibault Saunier 2017-02-22 20:03 ` Nicolas Dufresne 0 siblings, 1 reply; 15+ messages in thread From: Thibault Saunier @ 2017-02-22 18:57 UTC (permalink / raw) To: Hans Verkuil, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Hans Verkuil On 02/22/2017 03:06 PM, Hans Verkuil wrote: > > On 02/22/2017 05:05 AM, Thibault Saunier wrote: >> Hello, >> >> >> On 02/21/2017 11:19 PM, Hans Verkuil wrote: >>> On 02/21/2017 11:20 AM, Thibault Saunier wrote: >>>> Use colorspace provided by the user as we are only doing scaling and >>>> color encoding conversion, we won't be able to transform the colorspace >>>> itself and the colorspace won't mater in that operation. >>>> >>>> Also always use output colorspace on the capture side. >>>> >>>> Start using 576p as a threashold to compute the colorspace. >>>> The media documentation says that the V4L2_COLORSPACE_SMPTE170M colorspace >>>> should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV. But drivers >>>> don't agree on the display resolution that should be used as a threshold. >>>> >>>> From EIA CEA 861B about colorimetry for various resolutions: >>>> >>>> - 5.1 480p, 480i, 576p, 576i, 240p, and 288p >>>> The color space used by the 480-line, 576-line, 240-line, and 288-line >>>> formats will likely be based on SMPTE 170M [1]. >>>> - 5.2 1080i, 1080p, and 720p >>>> The color space used by the high definition formats will likely be >>>> based on ITU-R BT.709-4 >>>> >>>> This indicates that in the case that userspace does not specify what >>>> colorspace should be used, we should use 576p as a threshold to set >>>> V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_SMPTE170M. Even if it is >>>> only 'likely' and not a requirement it is the best guess we can make. >>>> >>>> The stream should have been encoded with the information and userspace >>>> has to pass it to the driver if it is not the case, otherwise we won't be >>>> able to handle it properly anyhow. >>>> >>>> Also, check for the resolution in G_FMT instead unconditionally setting >>>> the V4L2_COLORSPACE_REC709 colorspace. >>>> >>>> Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com> >>>> Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> >>>> Reviewed-by: Andrzej Hajda <a.hajda@samsung.com> >>>> >>>> --- >>>> >>>> Changes in v5: >>>> - Squash commit to always use output colorspace on the capture side >>>> inside this one >>>> - Fix typo in commit message >>>> >>>> Changes in v4: >>>> - Reword commit message to better back our assumptions on specifications >>>> >>>> Changes in v3: >>>> - Do not check values in the g_fmt functions as Andrzej explained in previous review >>>> - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' >>>> >>>> Changes in v2: None >>>> >>>> drivers/media/platform/exynos-gsc/gsc-core.c | 20 +++++++++++++++----- >>>> drivers/media/platform/exynos-gsc/gsc-core.h | 1 + >>>> 2 files changed, 16 insertions(+), 5 deletions(-) >>>> >>>> diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c b/drivers/media/platform/exynos-gsc/gsc-core.c >>>> index 59a634201830..772599de8c13 100644 >>>> --- a/drivers/media/platform/exynos-gsc/gsc-core.c >>>> +++ b/drivers/media/platform/exynos-gsc/gsc-core.c >>>> @@ -454,6 +454,7 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) >>>> } else { >>>> min_w = variant->pix_min->target_rot_dis_w; >>>> min_h = variant->pix_min->target_rot_dis_h; >>>> + pix_mp->colorspace = ctx->out_colorspace; >>>> } >>>> pr_debug("mod_x: %d, mod_y: %d, max_w: %d, max_h = %d", >>>> @@ -472,10 +473,15 @@ int gsc_try_fmt_mplane(struct gsc_ctx *ctx, struct v4l2_format *f) >>>> pix_mp->num_planes = fmt->num_planes; >>>> - if (pix_mp->width >= 1280) /* HD */ >>>> - pix_mp->colorspace = V4L2_COLORSPACE_REC709; >>>> - else /* SD */ >>>> - pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >>>> + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { >>>> + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ >>> I'd use || instead of && here. Ditto for the next patch. >>> >>>> + pix_mp->colorspace = V4L2_COLORSPACE_REC709; >>>> + else /* SD */ >>>> + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >>>> + } >>> Are you sure this is in fact how it is used? If the source of the video >>> is the sensor (camera), then the colorspace is typically SRGB. I'm not >>> really sure you should guess here. All a mem2mem driver should do is to >>> pass the colorspace etc. information from the output to the capture side, >>> and not invent things. That simply makes no sense. >> This is not a camera device but a colorspace conversion device, in many > Not really, this is a color encoding conversion device. I.e. it only affects > the Y'CbCr encoding and quantization range. The colorspace (aka chromaticities) > and transfer function remain unchanged. Well, right, sorry I am talking in GStreamer terminlogy where what you call colorspace is called colorimetry, and colorspace is what I am talking about here. > In fact, I suspect (correct me if I am wrong) that it only converts between > RGB, Y'CbCr 601 and Y'CbCr 709 encodings, where RGB is full range and the Y'CbCr > formats are limited range. That sounds correct. > If you pass in limited range RGB it will probably do the wrong thing as I don't > seen any quantization checks in the code. > > So the colorspace and xfer_func fields remain unchanged by this driver. > > If you want to do this really correctly, then you need to do more. I don't have > time right now to go into this in-depth, but I will try to get back to this on > Monday. I am thinking of documenting this as part of the V4L2 colorspace > documentation. This stuff is complex and if you don't know the rules then it > can be hard to implement correctly. Here I am just making sure that we set the colorspace (v4l2 terminology) if the case the user is passing DEFAULT (meaning he does not know what it should be and lets the driver set it, this will happen in the case where the information was not contain in the source, which means the value has to be guessed, and here I am simply doing an educated guess relying on what the previously named standard suggests. I would be happy to read more information about that subject and will try to fix remaining suggested points once I have a better understanding of the whole concepts and problems in that driver, but I still think that this patch is correct for what it is aiming at fixing. Regards, Thibault Saunier >> cases the info will not be passed by the userspace, basically because the info >> is not encoded in the media stream (this often happens). I am not inventing here, >> just making sure we use the most likely value in case none was provided (and if none >> was provided it should always be correct given that the encoded stream was not broken). >> >> In the case the source is a camera and then we use the colorspace converter then the info >> should copied from the camera to the transcoding node (by userspace) so there should be >> no problem. >> >> What I am doing here is what is done in other drivers. > Most (all?) other mem2mem drivers do not do color encoding conversion, they just copy > all the colorspace info from output to capture. > > If there are other m2m drivers that do this, then I doubt they do the right thing. > They are probably right for most cases, but not all. > > Regards, > > Hans ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided 2017-02-22 18:57 ` Thibault Saunier @ 2017-02-22 20:03 ` Nicolas Dufresne 2017-02-22 20:08 ` Thibault Saunier 0 siblings, 1 reply; 15+ messages in thread From: Nicolas Dufresne @ 2017-02-22 20:03 UTC (permalink / raw) To: Thibault Saunier, Hans Verkuil, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Hans Verkuil [-- Attachment #1: Type: text/plain, Size: 9960 bytes --] Le mercredi 22 février 2017 à 15:57 -0300, Thibault Saunier a écrit : > On 02/22/2017 03:06 PM, Hans Verkuil wrote: > > > > On 02/22/2017 05:05 AM, Thibault Saunier wrote: > > > Hello, > > > > > > > > > On 02/21/2017 11:19 PM, Hans Verkuil wrote: > > > > On 02/21/2017 11:20 AM, Thibault Saunier wrote: > > > > > Use colorspace provided by the user as we are only doing > > > > > scaling and > > > > > color encoding conversion, we won't be able to transform the > > > > > colorspace > > > > > itself and the colorspace won't mater in that operation. > > > > > > > > > > Also always use output colorspace on the capture side. > > > > > > > > > > Start using 576p as a threashold to compute the colorspace. > > > > > The media documentation says that the > > > > > V4L2_COLORSPACE_SMPTE170M colorspace > > > > > should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV. > > > > > But drivers > > > > > don't agree on the display resolution that should be used as > > > > > a threshold. > > > > > > > > > > From EIA CEA 861B about colorimetry for various > > > > > resolutions: > > > > > > > > > > - 5.1 480p, 480i, 576p, 576i, 240p, and 288p > > > > > The color space used by the 480-line, 576-line, 240- > > > > > line, and 288-line > > > > > formats will likely be based on SMPTE 170M [1]. > > > > > - 5.2 1080i, 1080p, and 720p > > > > > The color space used by the high definition formats > > > > > will likely be > > > > > based on ITU-R BT.709-4 > > > > > > > > > > This indicates that in the case that userspace does not > > > > > specify what > > > > > colorspace should be used, we should use 576p as a threshold > > > > > to set > > > > > V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_SMPTE170M. > > > > > Even if it is > > > > > only 'likely' and not a requirement it is the best guess we > > > > > can make. > > > > > > > > > > The stream should have been encoded with the information and > > > > > userspace > > > > > has to pass it to the driver if it is not the case, otherwise > > > > > we won't be > > > > > able to handle it properly anyhow. > > > > > > > > > > Also, check for the resolution in G_FMT instead > > > > > unconditionally setting > > > > > the V4L2_COLORSPACE_REC709 colorspace. > > > > > > > > > > Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.c > > > > > om> > > > > > Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung > > > > > .com> > > > > > Reviewed-by: Andrzej Hajda <a.hajda@samsung.com> > > > > > > > > > > --- > > > > > > > > > > Changes in v5: > > > > > - Squash commit to always use output colorspace on the > > > > > capture side > > > > > inside this one > > > > > - Fix typo in commit message > > > > > > > > > > Changes in v4: > > > > > - Reword commit message to better back our assumptions on > > > > > specifications > > > > > > > > > > Changes in v3: > > > > > - Do not check values in the g_fmt functions as Andrzej > > > > > explained in previous review > > > > > - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' > > > > > > > > > > Changes in v2: None > > > > > > > > > > drivers/media/platform/exynos-gsc/gsc-core.c | 20 > > > > > +++++++++++++++----- > > > > > drivers/media/platform/exynos-gsc/gsc-core.h | 1 + > > > > > 2 files changed, 16 insertions(+), 5 deletions(-) > > > > > > > > > > diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c > > > > > b/drivers/media/platform/exynos-gsc/gsc-core.c > > > > > index 59a634201830..772599de8c13 100644 > > > > > --- a/drivers/media/platform/exynos-gsc/gsc-core.c > > > > > +++ b/drivers/media/platform/exynos-gsc/gsc-core.c > > > > > @@ -454,6 +454,7 @@ int gsc_try_fmt_mplane(struct gsc_ctx > > > > > *ctx, struct v4l2_format *f) > > > > > } else { > > > > > min_w = variant->pix_min->target_rot_dis_w; > > > > > min_h = variant->pix_min->target_rot_dis_h; > > > > > + pix_mp->colorspace = ctx->out_colorspace; > > > > > } > > > > > pr_debug("mod_x: %d, mod_y: %d, max_w: %d, max_h = > > > > > %d", > > > > > @@ -472,10 +473,15 @@ int gsc_try_fmt_mplane(struct gsc_ctx > > > > > *ctx, struct v4l2_format *f) > > > > > pix_mp->num_planes = fmt->num_planes; > > > > > - if (pix_mp->width >= 1280) /* HD */ > > > > > - pix_mp->colorspace = V4L2_COLORSPACE_REC709; > > > > > - else /* SD */ > > > > > - pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; > > > > > + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { > > > > > + if (pix_mp->width > 720 && pix_mp->height > 576) /* > > > > > HD */ > > > > > > > > I'd use || instead of && here. Ditto for the next patch. > > > > > > > > > + pix_mp->colorspace = V4L2_COLORSPACE_REC709; > > > > > + else /* SD */ > > > > > + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; > > > > > + } > > > > > > > > Are you sure this is in fact how it is used? If the source of > > > > the video > > > > is the sensor (camera), then the colorspace is typically SRGB. > > > > I'm not > > > > really sure you should guess here. All a mem2mem driver should > > > > do is to > > > > pass the colorspace etc. information from the output to the > > > > capture side, > > > > and not invent things. That simply makes no sense. > > > > > > This is not a camera device but a colorspace conversion device, > > > in many > > > > Not really, this is a color encoding conversion device. I.e. it > > only affects > > the Y'CbCr encoding and quantization range. The colorspace (aka > > chromaticities) > > and transfer function remain unchanged. > > Well, right, sorry I am talking in GStreamer terminlogy where what > you call > colorspace is called colorimetry, and colorspace is what I am > talking > about here. > > > In fact, I suspect (correct me if I am wrong) that it only converts > > between > > RGB, Y'CbCr 601 and Y'CbCr 709 encodings, where RGB is full range > > and the Y'CbCr > > formats are limited range. > > That sounds correct. > > > If you pass in limited range RGB it will probably do the wrong > > thing as I don't > > seen any quantization checks in the code. > > > > So the colorspace and xfer_func fields remain unchanged by this > > driver. > > > > If you want to do this really correctly, then you need to do more. > > I don't have > > time right now to go into this in-depth, but I will try to get back > > to this on > > Monday. I am thinking of documenting this as part of the V4L2 > > colorspace > > documentation. This stuff is complex and if you don't know the > > rules then it > > can be hard to implement correctly. > > Here I am just making sure that we set the colorspace (v4l2 > terminology) > if the case the user is passing DEFAULT (meaning he does not know > what > it should be and lets the driver set it, this will happen in the > case > where the > information was not contain in the source, which means the value has > to be > guessed, and here I am simply doing an educated guess relying on what > the > previously named standard suggests. > > I would be happy to read more information about that subject and will > try to fix remaining suggested points once I have a better > understanding > of the whole concepts and problems in that driver, but I still think > that this > patch is correct for what it is aiming at fixing. > > Regards, > > Thibault Saunier > > > > cases the info will not be passed by the userspace, basically > > > because the info > > > is not encoded in the media stream (this often happens). I am not > > > inventing here, > > > just making sure we use the most likely value in case none was > > > provided (and if none > > > was provided it should always be correct given that the encoded > > > stream was not broken). > > > > > > In the case the source is a camera and then we use the colorspace > > > converter then the info > > > should copied from the camera to the transcoding node (by > > > userspace) so there should be > > > no problem. > > > > > > What I am doing here is what is done in other drivers. > > > > Most (all?) other mem2mem drivers do not do color encoding > > conversion, they just copy > > all the colorspace info from output to capture. > > > > If there are other m2m drivers that do this, then I doubt they do > > the right thing. > > They are probably right for most cases, but not all. > > > > Regards, > > > > Hans I think Hans is right about the colorspace here. Unlike the field for interlace modes, there it is no mandatory requirement in the spec for the driver to change DEFAULT into something else. If you don't know the input colorspace, then you don't known the output. Even in GStreamer, this would be best, since the only guessing will happen at the display, or in some conversion to RGB. Though, if you don't support a specific value, then the driver should change it. Where I don't agree though, is that an m2m color converter will always copy blindly that colorspace. Even though the support is limited, sometime it may change, this need to be sorted case by case. I have rarely seen such a driver keep the limited range when turning buffers into RGB data. But changing the colorspace is complicated, since it's slightly vague concept. It will in fact imply a specific set of v4l2_xfer_func, v4l2_ycbcr_encoding and v4l2_quantization (ignoring hsv, as it does not apply). If you have a limited driver, that only target one of these 3 items, you will likely keep the colorspace as-is, and override the specific functions. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided 2017-02-22 20:03 ` Nicolas Dufresne @ 2017-02-22 20:08 ` Thibault Saunier 0 siblings, 0 replies; 15+ messages in thread From: Thibault Saunier @ 2017-02-22 20:08 UTC (permalink / raw) To: Nicolas Dufresne, Hans Verkuil, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Hans Verkuil On 02/22/2017 05:03 PM, Nicolas Dufresne wrote: > Le mercredi 22 février 2017 à 15:57 -0300, Thibault Saunier a écrit : >> On 02/22/2017 03:06 PM, Hans Verkuil wrote: >>> On 02/22/2017 05:05 AM, Thibault Saunier wrote: >>>> Hello, >>>> >>>> >>>> On 02/21/2017 11:19 PM, Hans Verkuil wrote: >>>>> On 02/21/2017 11:20 AM, Thibault Saunier wrote: >>>>>> Use colorspace provided by the user as we are only doing >>>>>> scaling and >>>>>> color encoding conversion, we won't be able to transform the >>>>>> colorspace >>>>>> itself and the colorspace won't mater in that operation. >>>>>> >>>>>> Also always use output colorspace on the capture side. >>>>>> >>>>>> Start using 576p as a threashold to compute the colorspace. >>>>>> The media documentation says that the >>>>>> V4L2_COLORSPACE_SMPTE170M colorspace >>>>>> should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV. >>>>>> But drivers >>>>>> don't agree on the display resolution that should be used as >>>>>> a threshold. >>>>>> >>>>>> From EIA CEA 861B about colorimetry for various >>>>>> resolutions: >>>>>> >>>>>> - 5.1 480p, 480i, 576p, 576i, 240p, and 288p >>>>>> The color space used by the 480-line, 576-line, 240- >>>>>> line, and 288-line >>>>>> formats will likely be based on SMPTE 170M [1]. >>>>>> - 5.2 1080i, 1080p, and 720p >>>>>> The color space used by the high definition formats >>>>>> will likely be >>>>>> based on ITU-R BT.709-4 >>>>>> >>>>>> This indicates that in the case that userspace does not >>>>>> specify what >>>>>> colorspace should be used, we should use 576p as a threshold >>>>>> to set >>>>>> V4L2_COLORSPACE_REC709 instead of V4L2_COLORSPACE_SMPTE170M. >>>>>> Even if it is >>>>>> only 'likely' and not a requirement it is the best guess we >>>>>> can make. >>>>>> >>>>>> The stream should have been encoded with the information and >>>>>> userspace >>>>>> has to pass it to the driver if it is not the case, otherwise >>>>>> we won't be >>>>>> able to handle it properly anyhow. >>>>>> >>>>>> Also, check for the resolution in G_FMT instead >>>>>> unconditionally setting >>>>>> the V4L2_COLORSPACE_REC709 colorspace. >>>>>> >>>>>> Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.c >>>>>> om> >>>>>> Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung >>>>>> .com> >>>>>> Reviewed-by: Andrzej Hajda <a.hajda@samsung.com> >>>>>> >>>>>> --- >>>>>> >>>>>> Changes in v5: >>>>>> - Squash commit to always use output colorspace on the >>>>>> capture side >>>>>> inside this one >>>>>> - Fix typo in commit message >>>>>> >>>>>> Changes in v4: >>>>>> - Reword commit message to better back our assumptions on >>>>>> specifications >>>>>> >>>>>> Changes in v3: >>>>>> - Do not check values in the g_fmt functions as Andrzej >>>>>> explained in previous review >>>>>> - Added 'Reviewed-by: Andrzej Hajda <a.hajda@samsung.com>' >>>>>> >>>>>> Changes in v2: None >>>>>> >>>>>> drivers/media/platform/exynos-gsc/gsc-core.c | 20 >>>>>> +++++++++++++++----- >>>>>> drivers/media/platform/exynos-gsc/gsc-core.h | 1 + >>>>>> 2 files changed, 16 insertions(+), 5 deletions(-) >>>>>> >>>>>> diff --git a/drivers/media/platform/exynos-gsc/gsc-core.c >>>>>> b/drivers/media/platform/exynos-gsc/gsc-core.c >>>>>> index 59a634201830..772599de8c13 100644 >>>>>> --- a/drivers/media/platform/exynos-gsc/gsc-core.c >>>>>> +++ b/drivers/media/platform/exynos-gsc/gsc-core.c >>>>>> @@ -454,6 +454,7 @@ int gsc_try_fmt_mplane(struct gsc_ctx >>>>>> *ctx, struct v4l2_format *f) >>>>>> } else { >>>>>> min_w = variant->pix_min->target_rot_dis_w; >>>>>> min_h = variant->pix_min->target_rot_dis_h; >>>>>> + pix_mp->colorspace = ctx->out_colorspace; >>>>>> } >>>>>> pr_debug("mod_x: %d, mod_y: %d, max_w: %d, max_h = >>>>>> %d", >>>>>> @@ -472,10 +473,15 @@ int gsc_try_fmt_mplane(struct gsc_ctx >>>>>> *ctx, struct v4l2_format *f) >>>>>> pix_mp->num_planes = fmt->num_planes; >>>>>> - if (pix_mp->width >= 1280) /* HD */ >>>>>> - pix_mp->colorspace = V4L2_COLORSPACE_REC709; >>>>>> - else /* SD */ >>>>>> - pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >>>>>> + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { >>>>>> + if (pix_mp->width > 720 && pix_mp->height > 576) /* >>>>>> HD */ >>>>> I'd use || instead of && here. Ditto for the next patch. >>>>> >>>>>> + pix_mp->colorspace = V4L2_COLORSPACE_REC709; >>>>>> + else /* SD */ >>>>>> + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; >>>>>> + } >>>>> Are you sure this is in fact how it is used? If the source of >>>>> the video >>>>> is the sensor (camera), then the colorspace is typically SRGB. >>>>> I'm not >>>>> really sure you should guess here. All a mem2mem driver should >>>>> do is to >>>>> pass the colorspace etc. information from the output to the >>>>> capture side, >>>>> and not invent things. That simply makes no sense. >>>> This is not a camera device but a colorspace conversion device, >>>> in many >>> Not really, this is a color encoding conversion device. I.e. it >>> only affects >>> the Y'CbCr encoding and quantization range. The colorspace (aka >>> chromaticities) >>> and transfer function remain unchanged. >> Well, right, sorry I am talking in GStreamer terminlogy where what >> you call >> colorspace is called colorimetry, and colorspace is what I am >> talking >> about here. >> >>> In fact, I suspect (correct me if I am wrong) that it only converts >>> between >>> RGB, Y'CbCr 601 and Y'CbCr 709 encodings, where RGB is full range >>> and the Y'CbCr >>> formats are limited range. >> That sounds correct. >> >>> If you pass in limited range RGB it will probably do the wrong >>> thing as I don't >>> seen any quantization checks in the code. >>> >>> So the colorspace and xfer_func fields remain unchanged by this >>> driver. >>> >>> If you want to do this really correctly, then you need to do more. >>> I don't have >>> time right now to go into this in-depth, but I will try to get back >>> to this on >>> Monday. I am thinking of documenting this as part of the V4L2 >>> colorspace >>> documentation. This stuff is complex and if you don't know the >>> rules then it >>> can be hard to implement correctly. >> Here I am just making sure that we set the colorspace (v4l2 >> terminology) >> if the case the user is passing DEFAULT (meaning he does not know >> what >> it should be and lets the driver set it, this will happen in the >> case >> where the >> information was not contain in the source, which means the value has >> to be >> guessed, and here I am simply doing an educated guess relying on what >> the >> previously named standard suggests. >> >> I would be happy to read more information about that subject and will >> try to fix remaining suggested points once I have a better >> understanding >> of the whole concepts and problems in that driver, but I still think >> that this >> patch is correct for what it is aiming at fixing. >> >> Regards, >> >> Thibault Saunier >> >>>> cases the info will not be passed by the userspace, basically >>>> because the info >>>> is not encoded in the media stream (this often happens). I am not >>>> inventing here, >>>> just making sure we use the most likely value in case none was >>>> provided (and if none >>>> was provided it should always be correct given that the encoded >>>> stream was not broken). >>>> >>>> In the case the source is a camera and then we use the colorspace >>>> converter then the info >>>> should copied from the camera to the transcoding node (by >>>> userspace) so there should be >>>> no problem. >>>> >>>> What I am doing here is what is done in other drivers. >>> Most (all?) other mem2mem drivers do not do color encoding >>> conversion, they just copy >>> all the colorspace info from output to capture. >>> >>> If there are other m2m drivers that do this, then I doubt they do >>> the right thing. >>> They are probably right for most cases, but not all. >>> >>> Regards, >>> >>> Hans > I think Hans is right about the colorspace here. Unlike the field for > interlace modes, there it is no mandatory requirement in the spec for > the driver to change DEFAULT into something else. If you don't know the > input colorspace, then you don't known the output. Even in GStreamer, > this would be best, since the only guessing will happen at the display, > or in some conversion to RGB. Though, if you don't support a specific > value, then the driver should change it. Well the documentation, even if not really precise states: V4L2_COLORSPACE_DEFAULT: The default colorspace. This can be used by applications to let the driver fill in the colorspace. Which is why I supposed the driver should fill it in, also this is what other drivers do too. Regards, Thibault > Where I don't agree though, is that an m2m color converter will always > copy blindly that colorspace. Even though the support is limited, > sometime it may change, this need to be sorted case by case. I have > rarely seen such a driver keep the limited range when turning buffers > into RGB data. > > But changing the colorspace is complicated, since it's slightly vague > concept. It will in fact imply a specific set of v4l2_xfer_func, > v4l2_ycbcr_encoding and v4l2_quantization (ignoring hsv, as it does not > apply). If you have a limited driver, that only target one of these 3 > items, you will likely keep the colorspace as-is, and override the > specific functions. ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v5 2/3] [media] s5p-mfc: Set colorspace in VIDIO_{G,TRY}_FMT if DEFAULT provided 2017-02-21 19:20 [PATCH v5 0/3] Fixes for colorspace logic in exynos-gsc and s5p-mfc drivers Thibault Saunier 2017-02-21 19:20 ` [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided Thibault Saunier @ 2017-02-21 19:20 ` Thibault Saunier 2017-02-23 13:42 ` Andrzej Hajda 2017-02-21 19:20 ` [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt Thibault Saunier 2 siblings, 1 reply; 15+ messages in thread From: Thibault Saunier @ 2017-02-21 19:20 UTC (permalink / raw) To: linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, Thibault Saunier, linux-arm-kernel, Ulf Hansson, Andrzej Hajda, Jeongtae Park, Kyungmin Park, Kamil Debski The media documentation says that the V4L2_COLORSPACE_SMPTE170M colorspace should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV but the driver didn't set the colorimetry when userspace provided V4L2_COLORSPACE_DEFAULT. Use 576p display resolution as a threshold to set this as suggested by EIA CEA 861B. Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> --- Changes in v5: None Changes in v4: - Set the colorspace only if the user passed V4L2_COLORSPACE_DEFAULT, in all other cases just use what userspace provided. Changes in v3: - Do not check values in the g_fmt functions as Andrzej explained in previous review - Set colorspace if user passed V4L2_COLORSPACE_DEFAULT in Changes in v2: None drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c index 367ef8e8dbf0..0976c3e0a5ce 100644 --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c @@ -354,6 +354,11 @@ static int vidioc_g_fmt(struct file *file, void *priv, struct v4l2_format *f) pix_mp->plane_fmt[0].sizeimage = ctx->luma_size; pix_mp->plane_fmt[1].bytesperline = ctx->buf_width; pix_mp->plane_fmt[1].sizeimage = ctx->chroma_size; + + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ + pix_mp->colorspace = V4L2_COLORSPACE_REC709; + else /* SD */ + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { /* This is run on OUTPUT The buffer contains compressed image @@ -378,6 +383,7 @@ static int vidioc_g_fmt(struct file *file, void *priv, struct v4l2_format *f) static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) { struct s5p_mfc_dev *dev = video_drvdata(file); + struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; struct s5p_mfc_fmt *fmt; mfc_debug(2, "Type is %d\n", f->type); @@ -405,6 +411,14 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) mfc_err("Unsupported format by this MFC version.\n"); return -EINVAL; } + + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { + if (pix_mp->width > 720 && + pix_mp->height > 576) /* HD */ + pix_mp->colorspace = V4L2_COLORSPACE_REC709; + else /* SD */ + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; + } } return 0; -- 2.11.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v5 2/3] [media] s5p-mfc: Set colorspace in VIDIO_{G,TRY}_FMT if DEFAULT provided 2017-02-21 19:20 ` [PATCH v5 2/3] [media] s5p-mfc: Set colorspace in VIDIO_{G,TRY}_FMT if DEFAULT provided Thibault Saunier @ 2017-02-23 13:42 ` Andrzej Hajda 0 siblings, 0 replies; 15+ messages in thread From: Andrzej Hajda @ 2017-02-23 13:42 UTC (permalink / raw) To: Thibault Saunier, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Jeongtae Park, Kyungmin Park, Kamil Debski On 21.02.2017 20:20, Thibault Saunier wrote: > The media documentation says that the V4L2_COLORSPACE_SMPTE170M colorspace > should be used for SDTV and V4L2_COLORSPACE_REC709 for HDTV but the driver > didn't set the colorimetry when userspace provided > V4L2_COLORSPACE_DEFAULT. > > Use 576p display resolution as a threshold to set this as suggested by > EIA CEA 861B. > > Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> > > --- > > Changes in v5: None > Changes in v4: > - Set the colorspace only if the user passed V4L2_COLORSPACE_DEFAULT, in > all other cases just use what userspace provided. > > Changes in v3: > - Do not check values in the g_fmt functions as Andrzej explained in previous review > - Set colorspace if user passed V4L2_COLORSPACE_DEFAULT in > > Changes in v2: None > > drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 14 ++++++++++++++ > 1 file changed, 14 insertions(+) > > diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > index 367ef8e8dbf0..0976c3e0a5ce 100644 > --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > @@ -354,6 +354,11 @@ static int vidioc_g_fmt(struct file *file, void *priv, struct v4l2_format *f) > pix_mp->plane_fmt[0].sizeimage = ctx->luma_size; > pix_mp->plane_fmt[1].bytesperline = ctx->buf_width; > pix_mp->plane_fmt[1].sizeimage = ctx->chroma_size; > + > + if (pix_mp->width > 720 && pix_mp->height > 576) /* HD */ > + pix_mp->colorspace = V4L2_COLORSPACE_REC709; > + else /* SD */ > + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; Again, it seems much more complicated for decoder, I suppose colorspace field should be filled according to decoded information from the header of byte stream. It looks like MFC does not parse stream header for such info. So it should be done either by the driver, either by userspace, if userspace is able to get such info. For example in H.264 this info is encoded in VUI header [1], I do not know about other codecs. I guess the best solution for now is to not touch this field unless you can retrieve this info from MFC. [1]: https://github.com/copiousfreetime/mp4parser/blob/master/isoparser/src/main/java/com/googlecode/mp4parser/h264/model/SeqParameterSet.java#L227 Regards Andrzej > } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { > /* This is run on OUTPUT > The buffer contains compressed image > @@ -378,6 +383,7 @@ static int vidioc_g_fmt(struct file *file, void *priv, struct v4l2_format *f) > static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) > { > struct s5p_mfc_dev *dev = video_drvdata(file); > + struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; > struct s5p_mfc_fmt *fmt; > > mfc_debug(2, "Type is %d\n", f->type); > @@ -405,6 +411,14 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) > mfc_err("Unsupported format by this MFC version.\n"); > return -EINVAL; > } > + > + if (pix_mp->colorspace == V4L2_COLORSPACE_DEFAULT) { > + if (pix_mp->width > 720 && > + pix_mp->height > 576) /* HD */ > + pix_mp->colorspace = V4L2_COLORSPACE_REC709; > + else /* SD */ > + pix_mp->colorspace = V4L2_COLORSPACE_SMPTE170M; > + } > } > > return 0; ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt 2017-02-21 19:20 [PATCH v5 0/3] Fixes for colorspace logic in exynos-gsc and s5p-mfc drivers Thibault Saunier 2017-02-21 19:20 ` [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided Thibault Saunier 2017-02-21 19:20 ` [PATCH v5 2/3] [media] s5p-mfc: Set colorspace in VIDIO_{G,TRY}_FMT if DEFAULT provided Thibault Saunier @ 2017-02-21 19:20 ` Thibault Saunier 2017-02-22 9:29 ` Andrzej Hajda 2 siblings, 1 reply; 15+ messages in thread From: Thibault Saunier @ 2017-02-21 19:20 UTC (permalink / raw) To: linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, Thibault Saunier, linux-arm-kernel, Ulf Hansson, Andrzej Hajda, Jeongtae Park, Kyungmin Park, Kamil Debski It is required by the standard that the field order is set by the driver. Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> --- Changes in v5: - Just adapt the field and never error out. Changes in v4: None Changes in v3: - Do not check values in the g_fmt functions as Andrzej explained in previous review Changes in v2: - Fix a silly build error that slipped in while rebasing the patches drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c index 0976c3e0a5ce..44ed2afe0780 100644 --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c @@ -386,6 +386,9 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; struct s5p_mfc_fmt *fmt; + if (f->fmt.pix.field == V4L2_FIELD_ANY) + f->fmt.pix.field = V4L2_FIELD_NONE; + mfc_debug(2, "Type is %d\n", f->type); if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { fmt = find_format(f, MFC_FMT_DEC); -- 2.11.1 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt 2017-02-21 19:20 ` [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt Thibault Saunier @ 2017-02-22 9:29 ` Andrzej Hajda 2017-02-22 13:10 ` Thibault Saunier 0 siblings, 1 reply; 15+ messages in thread From: Andrzej Hajda @ 2017-02-22 9:29 UTC (permalink / raw) To: Thibault Saunier, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Jeongtae Park, Kyungmin Park, Kamil Debski On 21.02.2017 20:20, Thibault Saunier wrote: > It is required by the standard that the field order is set by the > driver. > > Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> > > --- > > Changes in v5: > - Just adapt the field and never error out. > > Changes in v4: None > Changes in v3: > - Do not check values in the g_fmt functions as Andrzej explained in previous review > > Changes in v2: > - Fix a silly build error that slipped in while rebasing the patches > > drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > index 0976c3e0a5ce..44ed2afe0780 100644 > --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > @@ -386,6 +386,9 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) > struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; > struct s5p_mfc_fmt *fmt; > > + if (f->fmt.pix.field == V4L2_FIELD_ANY) > + f->fmt.pix.field = V4L2_FIELD_NONE; > + In previous version the only supported field type was NONE, here you support everything. If the only supported format is none you should set 'field' unconditionally to NONE, nothing more. Regards Andrzej > mfc_debug(2, "Type is %d\n", f->type); > if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { > fmt = find_format(f, MFC_FMT_DEC); ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt 2017-02-22 9:29 ` Andrzej Hajda @ 2017-02-22 13:10 ` Thibault Saunier 2017-02-22 14:42 ` Nicolas Dufresne 2017-02-23 11:17 ` Andrzej Hajda 0 siblings, 2 replies; 15+ messages in thread From: Thibault Saunier @ 2017-02-22 13:10 UTC (permalink / raw) To: Andrzej Hajda, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Jeongtae Park, Kyungmin Park, Kamil Debski Hello, On 02/22/2017 06:29 AM, Andrzej Hajda wrote: > On 21.02.2017 20:20, Thibault Saunier wrote: >> It is required by the standard that the field order is set by the >> driver. >> >> Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> >> >> --- >> >> Changes in v5: >> - Just adapt the field and never error out. >> >> Changes in v4: None >> Changes in v3: >> - Do not check values in the g_fmt functions as Andrzej explained in previous review >> >> Changes in v2: >> - Fix a silly build error that slipped in while rebasing the patches >> >> drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c >> index 0976c3e0a5ce..44ed2afe0780 100644 >> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c >> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c >> @@ -386,6 +386,9 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) >> struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; >> struct s5p_mfc_fmt *fmt; >> >> + if (f->fmt.pix.field == V4L2_FIELD_ANY) >> + f->fmt.pix.field = V4L2_FIELD_NONE; >> + > In previous version the only supported field type was NONE, here you > support everything. > If the only supported format is none you should set 'field' > unconditionally to NONE, nothing more. Afaict we support 2 things: 1. NONE 2. INTERLACE Until now we were not checking what was supported or not and basically ignoring that info, this patch keeps the old behaviour making sure to be compliant. I had a doubt and was pondering doing: ``` diff + if (f->fmt.pix.field != V4L2_FIELD_INTERLACED) + f->fmt.pix.field = V4L2_FIELD_NONE; + ``` instead, it is probably more correct, do you think it is what should be done here? Regards, Thibault > > Regards > Andrzej > >> mfc_debug(2, "Type is %d\n", f->type); >> if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { >> fmt = find_format(f, MFC_FMT_DEC); > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt 2017-02-22 13:10 ` Thibault Saunier @ 2017-02-22 14:42 ` Nicolas Dufresne 2017-02-23 11:17 ` Andrzej Hajda 1 sibling, 0 replies; 15+ messages in thread From: Nicolas Dufresne @ 2017-02-22 14:42 UTC (permalink / raw) To: Thibault Saunier, Andrzej Hajda, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Jeongtae Park, Kyungmin Park, Kamil Debski [-- Attachment #1: Type: text/plain, Size: 2492 bytes --] Le mercredi 22 février 2017 à 10:10 -0300, Thibault Saunier a écrit : > Hello, > > On 02/22/2017 06:29 AM, Andrzej Hajda wrote: > > On 21.02.2017 20:20, Thibault Saunier wrote: > > > It is required by the standard that the field order is set by the > > > driver. > > > > > > Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com > > > > > > > > > > --- > > > > > > Changes in v5: > > > - Just adapt the field and never error out. > > > > > > Changes in v4: None > > > Changes in v3: > > > - Do not check values in the g_fmt functions as Andrzej explained > > > in previous review > > > > > > Changes in v2: > > > - Fix a silly build error that slipped in while rebasing the > > > patches > > > > > > drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > > > b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > > > index 0976c3e0a5ce..44ed2afe0780 100644 > > > --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > > > +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c > > > @@ -386,6 +386,9 @@ static int vidioc_try_fmt(struct file *file, > > > void *priv, struct v4l2_format *f) > > > struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; > > > struct s5p_mfc_fmt *fmt; > > > > > > + if (f->fmt.pix.field == V4L2_FIELD_ANY) > > > + f->fmt.pix.field = V4L2_FIELD_NONE; > > > + > > > > In previous version the only supported field type was NONE, here > > you > > support everything. > > If the only supported format is none you should set 'field' > > unconditionally to NONE, nothing more. > > Afaict we support 2 things: > > 1. NONE > 2. INTERLACE > > Until now we were not checking what was supported or not and > basically > ignoring that info, this patch > keeps the old behaviour making sure to be compliant. > > I had a doubt and was pondering doing: > > ``` diff > > + if (f->fmt.pix.field != V4L2_FIELD_INTERLACED) > + f->fmt.pix.field = V4L2_FIELD_NONE; > + > This looks better to me. > ``` > > instead, it is probably more correct, do you think it is what should > be > done here? > > Regards, > > Thibault > > > > > Regards > > Andrzej > > > > > mfc_debug(2, "Type is %d\n", f->type); > > > if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { > > > fmt = find_format(f, MFC_FMT_DEC); > > [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt 2017-02-22 13:10 ` Thibault Saunier 2017-02-22 14:42 ` Nicolas Dufresne @ 2017-02-23 11:17 ` Andrzej Hajda 1 sibling, 0 replies; 15+ messages in thread From: Andrzej Hajda @ 2017-02-23 11:17 UTC (permalink / raw) To: Thibault Saunier, linux-kernel Cc: Mauro Carvalho Chehab, Marek Szyprowski, Kukjin Kim, Mauro Carvalho Chehab, Nicolas Dufresne, Andi Shyti, linux-media, Shuah Khan, Javier Martinez Canillas, linux-samsung-soc, Krzysztof Kozlowski, Inki Dae, Sylwester Nawrocki, linux-arm-kernel, Ulf Hansson, Jeongtae Park, Kyungmin Park, Kamil Debski On 22.02.2017 14:10, Thibault Saunier wrote: > Hello, > > On 02/22/2017 06:29 AM, Andrzej Hajda wrote: >> On 21.02.2017 20:20, Thibault Saunier wrote: >>> It is required by the standard that the field order is set by the >>> driver. >>> >>> Signed-off-by: Thibault Saunier <thibault.saunier@osg.samsung.com> >>> >>> --- >>> >>> Changes in v5: >>> - Just adapt the field and never error out. >>> >>> Changes in v4: None >>> Changes in v3: >>> - Do not check values in the g_fmt functions as Andrzej explained in previous review >>> >>> Changes in v2: >>> - Fix a silly build error that slipped in while rebasing the patches >>> >>> drivers/media/platform/s5p-mfc/s5p_mfc_dec.c | 3 +++ >>> 1 file changed, 3 insertions(+) >>> >>> diff --git a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c >>> index 0976c3e0a5ce..44ed2afe0780 100644 >>> --- a/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c >>> +++ b/drivers/media/platform/s5p-mfc/s5p_mfc_dec.c >>> @@ -386,6 +386,9 @@ static int vidioc_try_fmt(struct file *file, void *priv, struct v4l2_format *f) >>> struct v4l2_pix_format_mplane *pix_mp = &f->fmt.pix_mp; >>> struct s5p_mfc_fmt *fmt; >>> >>> + if (f->fmt.pix.field == V4L2_FIELD_ANY) >>> + f->fmt.pix.field = V4L2_FIELD_NONE; >>> + >> In previous version the only supported field type was NONE, here you >> support everything. >> If the only supported format is none you should set 'field' >> unconditionally to NONE, nothing more. > Afaict we support 2 things: > > 1. NONE > 2. INTERLACE > > Until now we were not checking what was supported or not and basically > ignoring that info, this patch > keeps the old behaviour making sure to be compliant. > > I had a doubt and was pondering doing: > > ``` diff > > + if (f->fmt.pix.field != V4L2_FIELD_INTERLACED) > + f->fmt.pix.field = V4L2_FIELD_NONE; > + > > ``` > > instead, it is probably more correct, do you think it is what should be > done here? I have realized I have forgot that it is not simple mem2mem device, it is decoder. It is feed with compressed data which is just byte stream, the only valid field value on output side is V4L2_FIELD_NONE. About possible interlacing we could only say in case of capture side. And in this case there are multiple questions: - how MFC decodes stream with interlaced content? - is it possible to convince it to decode it as user asks? - does it perform (de-)interlacing? - what is implemented in the driver? After answering above questions we will be able to say how fmt.pix.field should be treated on capture side. And similar question we can pose in case of encoder. So simple patch and so many doubts. Regards Andrzej > > Regards, > > Thibault > >> Regards >> Andrzej >> >>> mfc_debug(2, "Type is %d\n", f->type); >>> if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { >>> fmt = find_format(f, MFC_FMT_DEC); > -- > To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2017-02-23 13:53 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-02-21 19:20 [PATCH v5 0/3] Fixes for colorspace logic in exynos-gsc and s5p-mfc drivers Thibault Saunier 2017-02-21 19:20 ` [PATCH v5 1/3] [media] exynos-gsc: Use user configured colorspace if provided Thibault Saunier 2017-02-22 2:19 ` Hans Verkuil 2017-02-22 13:05 ` Thibault Saunier 2017-02-22 18:06 ` Hans Verkuil 2017-02-22 18:57 ` Thibault Saunier 2017-02-22 20:03 ` Nicolas Dufresne 2017-02-22 20:08 ` Thibault Saunier 2017-02-21 19:20 ` [PATCH v5 2/3] [media] s5p-mfc: Set colorspace in VIDIO_{G,TRY}_FMT if DEFAULT provided Thibault Saunier 2017-02-23 13:42 ` Andrzej Hajda 2017-02-21 19:20 ` [PATCH v5 3/3] [media] s5p-mfc: Check and set 'v4l2_pix_format:field' field in try_fmt Thibault Saunier 2017-02-22 9:29 ` Andrzej Hajda 2017-02-22 13:10 ` Thibault Saunier 2017-02-22 14:42 ` Nicolas Dufresne 2017-02-23 11:17 ` Andrzej Hajda
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).