CTM and other color related properties

CTM and other color related properties

[Jyri Sarha]

Referring to this discussion:
https://patchwork.kernel.org/patch/9546905/

Since the discussion, has there been any planning/work been done about
the CTM2 API?

We would need for omap drm (for DSS5 and DSS6) a similar matrix API
for two purposes. However, neither of them is an exact match to the
CTM property.

1. CRTC specific Color Phase Rotation matrix is pretty close to CTM
concept, but it is applied after the gamma correction. However, there
is an optional static full-range or limited-range post-offset vector
and with it the CTM can also be used to convert the RGB output to a
YCbCr display output.

2. Plane specific Color Space Conversion matrix and pre-offset vector
is for YUV to RGB conversion. For customization purposes we would like
to expose this 3x3 matrix and the 3-element offset vector to user
space. So in general this is almost the same thing at the previous, but
for reverse conversion.

So when adding a CTM2 property blob, I would also vote for adding
pre- and post-offset vectors. Then a CSC would simply be a
combination off CTM and either a pre- or post- offset vector or maybe
both, depending on whether the block provides a conversion from RGB to
YUV, the other way around, or both.

Then it is a question whether the offset vectors should be absolute or
or relative to the bit depth of RGB components. A relative, with enough
precision, would be the most generic choice but it would leave a lot of
work to the driver code in many cases.

For convenience there could also be a standard enum for selecting
either custom coefficients or predefined standard conversion
(Full-range, ITU-R BT.601, and ITU-R BT.709 at least).

In general the color space conversion arithmetic are described well
on this web page:
http://www.equasys.de/colorconversion.html

Best regards,
Jyri

CTM and other color related properties

[Jyri Sarha]

Referring to this discussion:
https://patchwork.kernel.org/patch/9546905/

Since the discussion, has there been any planning/work been done about
the CTM2 API?

We would need for omap drm (for DSS5 and DSS6) a similar matrix API
for two purposes. However, neither of them is an exact match to the
CTM property.

1. CRTC specific Color Phase Rotation matrix is pretty close to CTM
concept, but it is applied after the gamma correction. However, there
is an optional static full-range or limited-range post-offset vector
and with it the CTM can also be used to convert the RGB output to a
YCbCr display output.

2. Plane specific Color Space Conversion matrix and pre-offset vector
is for YUV to RGB conversion. For customization purposes we would like
to expose this 3x3 matrix and the 3-element offset vector to user
space. So in general this is almost the same thing at the previous, but
for reverse conversion.

So when adding a CTM2 property blob, I would also vote for adding
pre- and post-offset vectors. Then a CSC would simply be a
combination off CTM and either a pre- or post- offset vector or maybe
both, depending on whether the block provides a conversion from RGB to
YUV, the other way around, or both.

Then it is a question whether the offset vectors should be absolute or
or relative to the bit depth of RGB components. A relative, with enough
precision, would be the most generic choice but it would leave a lot of
work to the driver code in many cases.

For convenience there could also be a standard enum for selecting
either custom coefficients or predefined standard conversion
(Full-range, ITU-R BT.601, and ITU-R BT.709 at least).

In general the color space conversion arithmetic are described well
on this web page:
http://www.equasys.de/colorconversion.html

Best regards,
Jyri
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: CTM and other color related properties

[Ville Syrjälä]

On Wed, Mar 29, 2017 at 10:23:54AM +0300, Jyri Sarha wrote:
> Referring to this discussion:
> https://patchwork.kernel.org/patch/9546905/
> 
> Since the discussion, has there been any planning/work been done about
> the CTM2 API?
> 
> We would need for omap drm (for DSS5 and DSS6) a similar matrix API
> for two purposes. However, neither of them is an exact match to the
> CTM property.
> 
> 1. CRTC specific Color Phase Rotation matrix is pretty close to CTM
> concept, but it is applied after the gamma correction. However, there
> is an optional static full-range or limited-range post-offset vector
> and with it the CTM can also be used to convert the RGB output to a
> YCbCr display output.

Having a post-gamma csc is defintiely the right way to do it. In our
case we don't have that in the current hardware :( All we have is
degamma+csc+gamma, so this complicates things quite a bit when the user
wants to apply ctm and/or gamma and we also need to use the csc either
for rgb->ycbcr or rgb full->limited range conversion. ATM we don't do
ycbcr output (but Shashank has plans) and it looks like our code for
dealing with the rgb full->limited range conversion is totally bogus if
there's a user specified ctm or gamma.

So I think what we want is a degamma->csc->gamma->csc type of
pipeline, where each driver can obviously select which parts of the
pipeline they actually can support.

> 
> 2. Plane specific Color Space Conversion matrix and pre-offset vector
> is for YUV to RGB conversion. For customization purposes we would like
> to expose this 3x3 matrix and the 3-element offset vector to user
> space. So in general this is almost the same thing at the previous, but
> for reverse conversion.

Yeah, for planes I think we want a csc->degamma->csc->gamma type of
pipeline. Again not all hardware can do it all so some of it will be
optional. And on a lot of hardware some of these are totally fixed
function blocks, so eg. exposing a fully programmable matrix may not
always make sense.

We did discuss this on the list recently:
https://lists.freedesktop.org/archives/dri-devel/2017-January/131175.html
https://lists.freedesktop.org/archives/dri-devel/2017-March/135854.html

> 
> So when adding a CTM2 property blob, I would also vote for adding
> pre- and post-offset vectors.

Indeed. I was actually thinking that wouldn't it be cool if the hw
actually had a 4x4 matrix so that you could do the translations purely
with the matrix itself. But I've never actually seen that in any
hardware, so exposing the pre/post offsets separately seems like the
better plan.

> Then a CSC would simply be a
> combination off CTM and either a pre- or post- offset vector or maybe
> both, depending on whether the block provides a conversion from RGB to
> YUV, the other way around, or both.
> 
> Then it is a question whether the offset vectors should be absolute or
> or relative to the bit depth of RGB components. A relative, with enough
> precision, would be the most generic choice but it would leave a lot of
> work to the driver code in many cases.

The actual depth of the data going through the matrix is hardware
dependant anyway, so I don't think absolute values will really work.

> 
> For convenience there could also be a standard enum for selecting
> either custom coefficients or predefined standard conversion
> (Full-range, ITU-R BT.601, and ITU-R BT.709 at least).
> 
> In general the color space conversion arithmetic are described well
> on this web page:
> http://www.equasys.de/colorconversion.html
> 
> Best regards,
> Jyri

-- 
Ville Syrjälä
Intel OTC

Re: CTM and other color related properties

[Ville Syrjälä]

On Wed, Mar 29, 2017 at 10:23:54AM +0300, Jyri Sarha wrote:
> Referring to this discussion:
> https://patchwork.kernel.org/patch/9546905/
> 
> Since the discussion, has there been any planning/work been done about
> the CTM2 API?
> 
> We would need for omap drm (for DSS5 and DSS6) a similar matrix API
> for two purposes. However, neither of them is an exact match to the
> CTM property.
> 
> 1. CRTC specific Color Phase Rotation matrix is pretty close to CTM
> concept, but it is applied after the gamma correction. However, there
> is an optional static full-range or limited-range post-offset vector
> and with it the CTM can also be used to convert the RGB output to a
> YCbCr display output.

Having a post-gamma csc is defintiely the right way to do it. In our
case we don't have that in the current hardware :( All we have is
degamma+csc+gamma, so this complicates things quite a bit when the user
wants to apply ctm and/or gamma and we also need to use the csc either
for rgb->ycbcr or rgb full->limited range conversion. ATM we don't do
ycbcr output (but Shashank has plans) and it looks like our code for
dealing with the rgb full->limited range conversion is totally bogus if
there's a user specified ctm or gamma.

So I think what we want is a degamma->csc->gamma->csc type of
pipeline, where each driver can obviously select which parts of the
pipeline they actually can support.

> 
> 2. Plane specific Color Space Conversion matrix and pre-offset vector
> is for YUV to RGB conversion. For customization purposes we would like
> to expose this 3x3 matrix and the 3-element offset vector to user
> space. So in general this is almost the same thing at the previous, but
> for reverse conversion.

Yeah, for planes I think we want a csc->degamma->csc->gamma type of
pipeline. Again not all hardware can do it all so some of it will be
optional. And on a lot of hardware some of these are totally fixed
function blocks, so eg. exposing a fully programmable matrix may not
always make sense.

We did discuss this on the list recently:
https://lists.freedesktop.org/archives/dri-devel/2017-January/131175.html
https://lists.freedesktop.org/archives/dri-devel/2017-March/135854.html

> 
> So when adding a CTM2 property blob, I would also vote for adding
> pre- and post-offset vectors.

Indeed. I was actually thinking that wouldn't it be cool if the hw
actually had a 4x4 matrix so that you could do the translations purely
with the matrix itself. But I've never actually seen that in any
hardware, so exposing the pre/post offsets separately seems like the
better plan.

> Then a CSC would simply be a
> combination off CTM and either a pre- or post- offset vector or maybe
> both, depending on whether the block provides a conversion from RGB to
> YUV, the other way around, or both.
> 
> Then it is a question whether the offset vectors should be absolute or
> or relative to the bit depth of RGB components. A relative, with enough
> precision, would be the most generic choice but it would leave a lot of
> work to the driver code in many cases.

The actual depth of the data going through the matrix is hardware
dependant anyway, so I don't think absolute values will really work.

> 
> For convenience there could also be a standard enum for selecting
> either custom coefficients or predefined standard conversion
> (Full-range, ITU-R BT.601, and ITU-R BT.709 at least).
> 
> In general the color space conversion arithmetic are described well
> on this web page:
> http://www.equasys.de/colorconversion.html
> 
> Best regards,
> Jyri

-- 
Ville Syrjälä
Intel OTC
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel