AMD display drivers handling DRM CRTC color mgmt props

AMD display drivers handling DRM CRTC color mgmt props

[Melissa Wen]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

Hi all,

I'm examining how DRM color management properties (degamma, ctm, gamma)
are applied to AMD display drivers. As far I could understand thanks
Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
per-plane color correction features:

* - Input gamma LUT (de-normalized)
* - Input CSC (normalized)
* - Surface degamma LUT (normalized)
* - Surface CSC (normalized)
* - Surface regamma LUT (normalized)
* - Output CSC (normalized)
                             
so DM is "adapting" those DRM per-CRTC properties to fit into three of
these color correction stages, which I guess are the surface stages:

* - Surface degamma LUT (normalized)
* - Surface CSC (normalized)
* - Surface regamma LUT (normalized)

I'm trying to understand what this mapping is doing. A comment mentions
that is not possible to do these color corrections after blending, so,
the same color correction pipe is performed on every plane before
blending?  (is the surface the plane?) Does this adaptation affect the
expected output?  Moreover, is there something that I misunderstood? :)

That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
step of color correction, I don't see how to accommodate it in the
mapping above, but I see DC already supports programming 3D LUT on DPP.
Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
the 3D LUT will be at the end of the color correction pipeline? Is there
anything I need to worry about mapping DRM 3D LUT support? Or any
advice?

Thanks in advance,

Melissa

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

AMD display drivers handling DRM CRTC color mgmt props

[Melissa Wen]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

Hi all,

I'm examining how DRM color management properties (degamma, ctm, gamma)
are applied to AMD display drivers. As far I could understand thanks
Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
per-plane color correction features:

* - Input gamma LUT (de-normalized)
* - Input CSC (normalized)
* - Surface degamma LUT (normalized)
* - Surface CSC (normalized)
* - Surface regamma LUT (normalized)
* - Output CSC (normalized)
                             
so DM is "adapting" those DRM per-CRTC properties to fit into three of
these color correction stages, which I guess are the surface stages:

* - Surface degamma LUT (normalized)
* - Surface CSC (normalized)
* - Surface regamma LUT (normalized)

I'm trying to understand what this mapping is doing. A comment mentions
that is not possible to do these color corrections after blending, so,
the same color correction pipe is performed on every plane before
blending?  (is the surface the plane?) Does this adaptation affect the
expected output?  Moreover, is there something that I misunderstood? :)

That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
step of color correction, I don't see how to accommodate it in the
mapping above, but I see DC already supports programming 3D LUT on DPP.
Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
the 3D LUT will be at the end of the color correction pipeline? Is there
anything I need to worry about mapping DRM 3D LUT support? Or any
advice?

Thanks in advance,

Melissa

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: AMD display drivers handling DRM CRTC color mgmt props

[Harry Wentland]

[-- Attachment #1: Type: text/plain, Size: 3387 bytes --]



On 2022-04-21 10:37, Melissa Wen wrote:
> Hi all,
> 
> I'm examining how DRM color management properties (degamma, ctm, gamma)
> are applied to AMD display drivers. As far I could understand thanks
> Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
> per-plane color correction features:
> 

DC programs some of the color correction features pre-blending but
DRM/KMS has not per-plane color correction properties.

See this series from Uma Shankar for an RFC on how to introduce those 
properties for 1D LUTs and CSC matrix:
https://patchwork.freedesktop.org/series/90826/

Bhanuprakash has a series of IGT tests for these properties: 
https://patchwork.freedesktop.org/series/96895/

I've rebased these on amd-staging-drm-next and maintain a kernel and IGT 
branch with these patches:
https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr

We've had many discussions with Weston guys on this. In order to merge 
the kernel properties we need a canonical userspace implementation that 
are using it. Weston guys are working towards that but if you want to 
suggest a different userspace to serve as that vehicle I'd be all ears. :)

Note that in order to show this all working we also need a Wayland 
Protocol update.

See
https://gitlab.freedesktop.org/pq/color-and-hdr
https://gitlab.freedesktop.org/swick/wayland-protocols
https://gitlab.freedesktop.org/wayland/weston/-/issues/467

> * - Input gamma LUT (de-normalized)
> * - Input CSC (normalized)
> * - Surface degamma LUT (normalized)
> * - Surface CSC (normalized)
> * - Surface regamma LUT (normalized)
> * - Output CSC (normalized)
>                               
> so DM is "adapting" those DRM per-CRTC properties to fit into three of
> these color correction stages, which I guess are the surface stages:
> 
> * - Surface degamma LUT (normalized)
> * - Surface CSC (normalized)
> * - Surface regamma LUT (normalized)
> 
> I'm trying to understand what this mapping is doing. A comment mentions
> that is not possible to do these color corrections after blending, so,
> the same color correction pipe is performed on every plane before
> blending?  (is the surface the plane?) Does this adaptation affect the
> expected output?  Moreover, is there something that I misunderstood? :)
> 

What's possible to do before and after blending has changed quite a bit
between DCN generations. We program the CRTC Gamma and CTM after 
blending. See attached picture for a view relating the color bits 
between the DRM interface, DC interface and DCN 3.0 HW blocks.

> That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last

Where do you see 3D LUT support in DRM? Is there a new proposal that 
I've missed?

I'm thinking of putting a 3D LUT proposal together but haven't gotten 
around to it yet. We'll want a pre-blending 3D LUT, and possible a 
programmable post-blending one as well.

Thanks,
Harry

> step of color correction, I don't see how to accommodate it in the
> mapping above, but I see DC already supports programming 3D LUT on DPP.
> Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
> the 3D LUT will be at the end of the color correction pipeline? Is there
> anything I need to worry about mapping DRM 3D LUT support? Or any
> advice?
> 
> Thanks in advance,
> 
> Melissa

[-- Attachment #2: dcn3_cm_drm_current.drawio.png --]
[-- Type: image/png, Size: 336117 bytes --]

Re: AMD display drivers handling DRM CRTC color mgmt props

[Melissa Wen]

[-- Attachment #1: Type: text/plain, Size: 5091 bytes --]

On 04/21, Harry Wentland wrote:
> 
> 
> On 2022-04-21 10:37, Melissa Wen wrote:
> > Hi all,
> > 
> > I'm examining how DRM color management properties (degamma, ctm, gamma)
> > are applied to AMD display drivers. As far I could understand thanks
> > Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
> > per-plane color correction features:
> > 
Hi Harry,

Wow, thanks so much for all details!
> 
> DC programs some of the color correction features pre-blending but
> DRM/KMS has not per-plane color correction properties.
> 
> See this series from Uma Shankar for an RFC on how to introduce those
> properties for 1D LUTs and CSC matrix:
> https://patchwork.freedesktop.org/series/90826/
> 
> Bhanuprakash has a series of IGT tests for these properties:
> https://patchwork.freedesktop.org/series/96895/
> 
> I've rebased these on amd-staging-drm-next and maintain a kernel and IGT
> branch with these patches:
> https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr
> 
> We've had many discussions with Weston guys on this. In order to merge the
> kernel properties we need a canonical userspace implementation that are
> using it. Weston guys are working towards that but if you want to suggest a
> different userspace to serve as that vehicle I'd be all ears. :)
> 
> Note that in order to show this all working we also need a Wayland Protocol
> update.
> 
> See
> https://gitlab.freedesktop.org/pq/color-and-hdr
> https://gitlab.freedesktop.org/swick/wayland-protocols
> https://gitlab.freedesktop.org/wayland/weston/-/issues/467

So, I've followed these discussions (until the issue on naming) because
initially I considered it addresses our current goals for color
correction. But after some discussions, what we are targeting is a 3D
LUT after blending (per-CRTC). I found past proposals on dri-devel
[1][2] to extend the DRM CRTC color management properties, but they
didn't move forward and were never applied.

> 
> > * - Input gamma LUT (de-normalized)
> > * - Input CSC (normalized)
> > * - Surface degamma LUT (normalized)
> > * - Surface CSC (normalized)
> > * - Surface regamma LUT (normalized)
> > * - Output CSC (normalized)
> > so DM is "adapting" those DRM per-CRTC properties to fit into three of
> > these color correction stages, which I guess are the surface stages:
> > 
> > * - Surface degamma LUT (normalized)
> > * - Surface CSC (normalized)
> > * - Surface regamma LUT (normalized)
> > 
> > I'm trying to understand what this mapping is doing. A comment mentions
> > that is not possible to do these color corrections after blending, so,
> > the same color correction pipe is performed on every plane before
> > blending?  (is the surface the plane?) Does this adaptation affect the
> > expected output?  Moreover, is there something that I misunderstood? :)
> > 
> 
> What's possible to do before and after blending has changed quite a bit
> between DCN generations. We program the CRTC Gamma and CTM after blending.
> See attached picture for a view relating the color bits between the DRM
> interface, DC interface and DCN 3.0 HW blocks.

This picture is really enlightening, thanks!
You said it changes between generations, therefore, I can't consider the
DCN 2.x family follow the same mapping, right? If so, can you share the
main differences for a DCN 2.x regarding per-CRTC properties?

> 
> > That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
> 
> Where do you see 3D LUT support in DRM? Is there a new proposal that I've
> missed?

So, it's exactly what I aim to work: a proposal to add 3D LUT to the
current range of DRM per-CRTC color properties. But I also need to
understand how this property will be mapped to AMD display once it
exists in the DRM framework.

One of the things that caught my attention after seeing the attached
picture is the position of 3D LUT. I was expecting to see the 3D LUT
correction after gamma correction. Is this position a particularity of
DCN 3.0 (that varies between hw) or was I expecting a wrong color
correction pipeline at all?

Melissa

[1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
[2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
> 
> I'm thinking of putting a 3D LUT proposal together but haven't gotten around
> to it yet. We'll want a pre-blending 3D LUT, and possible a programmable
> post-blending one as well.
> 
> Thanks,
> Harry
> 
> > step of color correction, I don't see how to accommodate it in the
> > mapping above, but I see DC already supports programming 3D LUT on DPP.
> > Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
> > the 3D LUT will be at the end of the color correction pipeline? Is there
> > anything I need to worry about mapping DRM 3D LUT support? Or any
> > advice?
> > 
> > Thanks in advance,
> > 
> > Melissa



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: AMD display drivers handling DRM CRTC color mgmt props

[Harry Wentland]

[-- Attachment #1: Type: text/plain, Size: 6302 bytes --]



On 2022-04-21 15:20, Melissa Wen wrote:
> On 04/21, Harry Wentland wrote:
>>
>>
>> On 2022-04-21 10:37, Melissa Wen wrote:
>>> Hi all,
>>>
>>> I'm examining how DRM color management properties (degamma, ctm, gamma)
>>> are applied to AMD display drivers. As far I could understand thanks
>>> Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
>>> per-plane color correction features:
>>>
> Hi Harry,
> 
> Wow, thanks so much for all details!
>>
>> DC programs some of the color correction features pre-blending but
>> DRM/KMS has not per-plane color correction properties.
>>
>> See this series from Uma Shankar for an RFC on how to introduce those
>> properties for 1D LUTs and CSC matrix:
>> https://patchwork.freedesktop.org/series/90826/
>>
>> Bhanuprakash has a series of IGT tests for these properties:
>> https://patchwork.freedesktop.org/series/96895/
>>
>> I've rebased these on amd-staging-drm-next and maintain a kernel and IGT
>> branch with these patches:
>> https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
>> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr
>>
>> We've had many discussions with Weston guys on this. In order to merge the
>> kernel properties we need a canonical userspace implementation that are
>> using it. Weston guys are working towards that but if you want to suggest a
>> different userspace to serve as that vehicle I'd be all ears. :)
>>
>> Note that in order to show this all working we also need a Wayland Protocol
>> update.
>>
>> See
>> https://gitlab.freedesktop.org/pq/color-and-hdr
>> https://gitlab.freedesktop.org/swick/wayland-protocols
>> https://gitlab.freedesktop.org/wayland/weston/-/issues/467
> 
> So, I've followed these discussions (until the issue on naming) because
> initially I considered it addresses our current goals for color
> correction. But after some discussions, what we are targeting is a 3D
> LUT after blending (per-CRTC). I found past proposals on dri-devel
> [1][2] to extend the DRM CRTC color management properties, but they
> didn't move forward and were never applied.
> 

They're stuck in limbo until we have an upstream userspace
implementation that's making use of them.

>>
>>> * - Input gamma LUT (de-normalized)
>>> * - Input CSC (normalized)
>>> * - Surface degamma LUT (normalized)
>>> * - Surface CSC (normalized)
>>> * - Surface regamma LUT (normalized)
>>> * - Output CSC (normalized)
>>> so DM is "adapting" those DRM per-CRTC properties to fit into three of
>>> these color correction stages, which I guess are the surface stages:
>>>
>>> * - Surface degamma LUT (normalized)
>>> * - Surface CSC (normalized)
>>> * - Surface regamma LUT (normalized)
>>>
>>> I'm trying to understand what this mapping is doing. A comment mentions
>>> that is not possible to do these color corrections after blending, so,
>>> the same color correction pipe is performed on every plane before
>>> blending?  (is the surface the plane?) Does this adaptation affect the
>>> expected output?  Moreover, is there something that I misunderstood? :)
>>>
>>
>> What's possible to do before and after blending has changed quite a bit
>> between DCN generations. We program the CRTC Gamma and CTM after blending.
>> See attached picture for a view relating the color bits between the DRM
>> interface, DC interface and DCN 3.0 HW blocks.
> 
> This picture is really enlightening, thanks!
> You said it changes between generations, therefore, I can't consider the
> DCN 2.x family follow the same mapping, right? If so, can you share the
> main differences for a DCN 2.x regarding per-CRTC properties?
> 

See attached diagram for DCN 2.0.

>>
>>> That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
>>
>> Where do you see 3D LUT support in DRM? Is there a new proposal that I've
>> missed?
> 
> So, it's exactly what I aim to work: a proposal to add 3D LUT to the
> current range of DRM per-CRTC color properties. But I also need to
> understand how this property will be mapped to AMD display once it
> exists in the DRM framework.
> 

Ah, nice to see. :)

> One of the things that caught my attention after seeing the attached
> picture is the position of 3D LUT. I was expecting to see the 3D LUT
> correction after gamma correction. Is this position a particularity of
> DCN 3.0 (that varies between hw) or was I expecting a wrong color
> correction pipeline at all?
> 

Before DCN 3.0 there was no 3D LUT after blending.

Note in the diagram that our HW (and DC interface) have a Shaper LUT
available before the 3D LUT. You could expose if you want to shape your
content post-blending before applying the 3D LUT.

The 3D LUT is most effective when it's in non-linear space. Currently
DRM has no way to specify a way for drm_plane to be linearized (see 
notes (1) and (2)) so it is assumed that you're blending in non-linear 
space and therefore your pixels would already be non-linear going into 
your 3D LUT.

(1) unless you use the drm_plane PWL API that was proposed
(2) amdgpu_dm is currently setting the drm_crtc degamma LUT on the
     DC plane. This might lead to unexpected behavior when using
     multiple planes (though I believe gamescope is making use of
     this behavior).

Have you looked at [1] yet? It might provide a good example on how to
define a 3D LUT. For AMD HW you'll want a 17x17x17 LUT.

[1] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html

Harry

> Melissa
> 
> [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
> [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
>>
>> I'm thinking of putting a 3D LUT proposal together but haven't gotten around
>> to it yet. We'll want a pre-blending 3D LUT, and possible a programmable
>> post-blending one as well.
>>
>> Thanks,
>> Harry
>>
>>> step of color correction, I don't see how to accommodate it in the
>>> mapping above, but I see DC already supports programming 3D LUT on DPP.
>>> Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
>>> the 3D LUT will be at the end of the color correction pipeline? Is there
>>> anything I need to worry about mapping DRM 3D LUT support? Or any
>>> advice?
>>>
>>> Thanks in advance,
>>>
>>> Melissa
> 
> 

[-- Attachment #2: dcn2_cm_drm_current.drawio.png --]
[-- Type: image/png, Size: 326980 bytes --]

Re: AMD display drivers handling DRM CRTC color mgmt props

[Melissa Wen]

[-- Attachment #1: Type: text/plain, Size: 7856 bytes --]

On 04/21, Harry Wentland wrote:
> 
> 
> On 2022-04-21 15:20, Melissa Wen wrote:
> > On 04/21, Harry Wentland wrote:
> > > 
> > > 
> > > On 2022-04-21 10:37, Melissa Wen wrote:
> > > > Hi all,
> > > > 
> > > > I'm examining how DRM color management properties (degamma, ctm, gamma)
> > > > are applied to AMD display drivers. As far I could understand thanks
> > > > Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
> > > > per-plane color correction features:
> > > > 
> > Hi Harry,
> > 
> > Wow, thanks so much for all details!
> > > 
> > > DC programs some of the color correction features pre-blending but
> > > DRM/KMS has not per-plane color correction properties.
> > > 
> > > See this series from Uma Shankar for an RFC on how to introduce those
> > > properties for 1D LUTs and CSC matrix:
> > > https://patchwork.freedesktop.org/series/90826/
> > > 
> > > Bhanuprakash has a series of IGT tests for these properties:
> > > https://patchwork.freedesktop.org/series/96895/
> > > 
> > > I've rebased these on amd-staging-drm-next and maintain a kernel and IGT
> > > branch with these patches:
> > > https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
> > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr
> > > 
> > > We've had many discussions with Weston guys on this. In order to merge the
> > > kernel properties we need a canonical userspace implementation that are
> > > using it. Weston guys are working towards that but if you want to suggest a
> > > different userspace to serve as that vehicle I'd be all ears. :)
> > > 
> > > Note that in order to show this all working we also need a Wayland Protocol
> > > update.
> > > 
> > > See
> > > https://gitlab.freedesktop.org/pq/color-and-hdr
> > > https://gitlab.freedesktop.org/swick/wayland-protocols
> > > https://gitlab.freedesktop.org/wayland/weston/-/issues/467
> > 
> > So, I've followed these discussions (until the issue on naming) because
> > initially I considered it addresses our current goals for color
> > correction. But after some discussions, what we are targeting is a 3D
> > LUT after blending (per-CRTC). I found past proposals on dri-devel
> > [1][2] to extend the DRM CRTC color management properties, but they
> > didn't move forward and were never applied.
> > 
> 
> They're stuck in limbo until we have an upstream userspace
> implementation that's making use of them.

Yes... afaiu, the basic requirements for all of these changes are IGT
tests + open userspace usage, right?

> 
> > > 
> > > > * - Input gamma LUT (de-normalized)
> > > > * - Input CSC (normalized)
> > > > * - Surface degamma LUT (normalized)
> > > > * - Surface CSC (normalized)
> > > > * - Surface regamma LUT (normalized)
> > > > * - Output CSC (normalized)
> > > > so DM is "adapting" those DRM per-CRTC properties to fit into three of
> > > > these color correction stages, which I guess are the surface stages:
> > > > 
> > > > * - Surface degamma LUT (normalized)
> > > > * - Surface CSC (normalized)
> > > > * - Surface regamma LUT (normalized)
> > > > 
> > > > I'm trying to understand what this mapping is doing. A comment mentions
> > > > that is not possible to do these color corrections after blending, so,
> > > > the same color correction pipe is performed on every plane before
> > > > blending?  (is the surface the plane?) Does this adaptation affect the
> > > > expected output?  Moreover, is there something that I misunderstood? :)
> > > > 
> > > 
> > > What's possible to do before and after blending has changed quite a bit
> > > between DCN generations. We program the CRTC Gamma and CTM after blending.
> > > See attached picture for a view relating the color bits between the DRM
> > > interface, DC interface and DCN 3.0 HW blocks.
> > 
> > This picture is really enlightening, thanks!
> > You said it changes between generations, therefore, I can't consider the
> > DCN 2.x family follow the same mapping, right? If so, can you share the
> > main differences for a DCN 2.x regarding per-CRTC properties?
> > 
> 
> See attached diagram for DCN 2.0.

Thanks again!

> 
> > > 
> > > > That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
> > > 
> > > Where do you see 3D LUT support in DRM? Is there a new proposal that I've
> > > missed?
> > 
> > So, it's exactly what I aim to work: a proposal to add 3D LUT to the
> > current range of DRM per-CRTC color properties. But I also need to
> > understand how this property will be mapped to AMD display once it
> > exists in the DRM framework.
> > 
> 
> Ah, nice to see. :)
> 
> > One of the things that caught my attention after seeing the attached
> > picture is the position of 3D LUT. I was expecting to see the 3D LUT
> > correction after gamma correction. Is this position a particularity of
> > DCN 3.0 (that varies between hw) or was I expecting a wrong color
> > correction pipeline at all?
> > 
> 
> Before DCN 3.0 there was no 3D LUT after blending.
>
By comparing these diagrams, I'm curious: in case we have a per-CRTC 3D
LUT support on DRM, DCN 2.0 generations would initially map this
property as a pre-blending property on DPP (currently the same approach
for CTM, for example), right? But after we also have a per-plane color
management property, those per-CRTC property would be ignored? And how
about degamma for both generations? No problem if there isn't an answer
yet (many if's), but it may help me to think of a more generic solution.

> Note in the diagram that our HW (and DC interface) have a Shaper LUT
> available before the 3D LUT. You could expose if you want to shape your
> content post-blending before applying the 3D LUT.
> 
> The 3D LUT is most effective when it's in non-linear space. Currently
> DRM has no way to specify a way for drm_plane to be linearized (see notes
> (1) and (2)) so it is assumed that you're blending in non-linear space and
> therefore your pixels would already be non-linear going into your 3D LUT.
> 
> (1) unless you use the drm_plane PWL API that was proposed
> (2) amdgpu_dm is currently setting the drm_crtc degamma LUT on the
>     DC plane. This might lead to unexpected behavior when using
>     multiple planes (though I believe gamescope is making use of
>     this behavior).

Thanks for raising these points. In fact, I was considering unexpected
behavior when I saw this DRM <-> DC mapping.
> 
> Have you looked at [1] yet? It might provide a good example on how to
> define a 3D LUT. For AMD HW you'll want a 17x17x17 LUT.
> 
> [1] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html

Not yet, but it seems helpful. I'll take as a reference... until now,
I've only examined details on DC drivers.

Thanks,

Melissa

> 
> Harry
> 
> > Melissa
> > 
> > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
> > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
> > > 
> > > I'm thinking of putting a 3D LUT proposal together but haven't gotten around
> > > to it yet. We'll want a pre-blending 3D LUT, and possible a programmable
> > > post-blending one as well.
> > > 
> > > Thanks,
> > > Harry
> > > 
> > > > step of color correction, I don't see how to accommodate it in the
> > > > mapping above, but I see DC already supports programming 3D LUT on DPP.
> > > > Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
> > > > the 3D LUT will be at the end of the color correction pipeline? Is there
> > > > anything I need to worry about mapping DRM 3D LUT support? Or any
> > > > advice?
> > > > 
> > > > Thanks in advance,
> > > > 
> > > > Melissa
> > 
> > 



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: AMD display drivers handling DRM CRTC color mgmt props

[Harry Wentland]

On 2022-04-22 10:28, Melissa Wen wrote:
> On 04/21, Harry Wentland wrote:
>>
>>
>> On 2022-04-21 15:20, Melissa Wen wrote:
>>> On 04/21, Harry Wentland wrote:
>>>>
>>>>
>>>> On 2022-04-21 10:37, Melissa Wen wrote:
>>>>> Hi all,
>>>>>
>>>>> I'm examining how DRM color management properties (degamma, ctm, gamma)
>>>>> are applied to AMD display drivers. As far I could understand thanks
>>>>> Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
>>>>> per-plane color correction features:
>>>>>
>>> Hi Harry,
>>>
>>> Wow, thanks so much for all details!
>>>>
>>>> DC programs some of the color correction features pre-blending but
>>>> DRM/KMS has not per-plane color correction properties.
>>>>
>>>> See this series from Uma Shankar for an RFC on how to introduce those
>>>> properties for 1D LUTs and CSC matrix:
>>>> https://patchwork.freedesktop.org/series/90826/
>>>>
>>>> Bhanuprakash has a series of IGT tests for these properties:
>>>> https://patchwork.freedesktop.org/series/96895/
>>>>
>>>> I've rebased these on amd-staging-drm-next and maintain a kernel and IGT
>>>> branch with these patches:
>>>> https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
>>>> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr
>>>>
>>>> We've had many discussions with Weston guys on this. In order to merge the
>>>> kernel properties we need a canonical userspace implementation that are
>>>> using it. Weston guys are working towards that but if you want to suggest a
>>>> different userspace to serve as that vehicle I'd be all ears. :)
>>>>
>>>> Note that in order to show this all working we also need a Wayland Protocol
>>>> update.
>>>>
>>>> See
>>>> https://gitlab.freedesktop.org/pq/color-and-hdr
>>>> https://gitlab.freedesktop.org/swick/wayland-protocols
>>>> https://gitlab.freedesktop.org/wayland/weston/-/issues/467
>>>
>>> So, I've followed these discussions (until the issue on naming) because
>>> initially I considered it addresses our current goals for color
>>> correction. But after some discussions, what we are targeting is a 3D
>>> LUT after blending (per-CRTC). I found past proposals on dri-devel
>>> [1][2] to extend the DRM CRTC color management properties, but they
>>> didn't move forward and were never applied.
>>>
>>
>> They're stuck in limbo until we have an upstream userspace
>> implementation that's making use of them.
> 
> Yes... afaiu, the basic requirements for all of these changes are IGT
> tests + open userspace usage, right?
> 

Correct. See [1] and [2].

[1] https://www.kernel.org/doc/html/latest/gpu/drm-kms.html#requirements
[2] 
https://www.kernel.org/doc/html/latest/gpu/drm-uapi.html#open-source-userspace-requirements

>>
>>>>
>>>>> * - Input gamma LUT (de-normalized)
>>>>> * - Input CSC (normalized)
>>>>> * - Surface degamma LUT (normalized)
>>>>> * - Surface CSC (normalized)
>>>>> * - Surface regamma LUT (normalized)
>>>>> * - Output CSC (normalized)
>>>>> so DM is "adapting" those DRM per-CRTC properties to fit into three of
>>>>> these color correction stages, which I guess are the surface stages:
>>>>>
>>>>> * - Surface degamma LUT (normalized)
>>>>> * - Surface CSC (normalized)
>>>>> * - Surface regamma LUT (normalized)
>>>>>
>>>>> I'm trying to understand what this mapping is doing. A comment mentions
>>>>> that is not possible to do these color corrections after blending, so,
>>>>> the same color correction pipe is performed on every plane before
>>>>> blending?  (is the surface the plane?) Does this adaptation affect the
>>>>> expected output?  Moreover, is there something that I misunderstood? :)
>>>>>
>>>>
>>>> What's possible to do before and after blending has changed quite a bit
>>>> between DCN generations. We program the CRTC Gamma and CTM after blending.
>>>> See attached picture for a view relating the color bits between the DRM
>>>> interface, DC interface and DCN 3.0 HW blocks.
>>>
>>> This picture is really enlightening, thanks!
>>> You said it changes between generations, therefore, I can't consider the
>>> DCN 2.x family follow the same mapping, right? If so, can you share the
>>> main differences for a DCN 2.x regarding per-CRTC properties?
>>>
>>
>> See attached diagram for DCN 2.0.
> 
> Thanks again!
> 
>>
>>>>
>>>>> That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
>>>>
>>>> Where do you see 3D LUT support in DRM? Is there a new proposal that I've
>>>> missed?
>>>
>>> So, it's exactly what I aim to work: a proposal to add 3D LUT to the
>>> current range of DRM per-CRTC color properties. But I also need to
>>> understand how this property will be mapped to AMD display once it
>>> exists in the DRM framework.
>>>
>>
>> Ah, nice to see. :)
>>
>>> One of the things that caught my attention after seeing the attached
>>> picture is the position of 3D LUT. I was expecting to see the 3D LUT
>>> correction after gamma correction. Is this position a particularity of
>>> DCN 3.0 (that varies between hw) or was I expecting a wrong color
>>> correction pipeline at all?
>>>
>>
>> Before DCN 3.0 there was no 3D LUT after blending.
>>
> By comparing these diagrams, I'm curious: in case we have a per-CRTC 3D
> LUT support on DRM, DCN 2.0 generations would initially map this
> property as a pre-blending property on DPP (currently the same approach
> for CTM, for example), right? But after we also have a per-plane color
> management property, those per-CRTC property would be ignored? And how
> about degamma for both generations? No problem if there isn't an answer
> yet (many if's), but it may help me to think of a more generic solution.
> 

We'll need to define what the expectations are for the API 
implementations in DRM drivers, as well as for the implementing userspace.

The way I think about this it might make sense to introduce a 3D LUT on 
a drm_plane, as well as on drm_crtc and a driver exposes whatever HW 
supports. Userspace can then figure out what it can use based on driver 
support.

It should be possible to use pre-blending 3D LUTs to accomplish the same 
as a post-blending 3D LUT, but we might need the ability to linearize 
before blending, but after applying the 3D LUT, and de-linearize after 
blending again. Something like this:

3dlut > linearize (1dlut) > blend > de-linearize (1dlut) > output

Instead of this for the post-blending 3dlut:

linearize (1dlut) > blend > de-linearize (1dlut) > 3dlut > output

Though it depends a bit on the color model in the compositor or 
rendering app.

>> Note in the diagram that our HW (and DC interface) have a Shaper LUT
>> available before the 3D LUT. You could expose if you want to shape your
>> content post-blending before applying the 3D LUT.
>>
>> The 3D LUT is most effective when it's in non-linear space. Currently
>> DRM has no way to specify a way for drm_plane to be linearized (see notes
>> (1) and (2)) so it is assumed that you're blending in non-linear space and
>> therefore your pixels would already be non-linear going into your 3D LUT.
>>
>> (1) unless you use the drm_plane PWL API that was proposed
>> (2) amdgpu_dm is currently setting the drm_crtc degamma LUT on the
>>      DC plane. This might lead to unexpected behavior when using
>>      multiple planes (though I believe gamescope is making use of
>>      this behavior).
> 
> Thanks for raising these points. In fact, I was considering unexpected
> behavior when I saw this DRM <-> DC mapping. >>
>> Have you looked at [1] yet? It might provide a good example on how to
>> define a 3D LUT. For AMD HW you'll want a 17x17x17 LUT.
>>
>> [1] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html
> 
> Not yet, but it seems helpful. I'll take as a reference... until now,
> I've only examined details on DC drivers.
> 

Sounds great.

Harry

> Thanks,
> 
> Melissa
> 
>>
>> Harry
>>
>>> Melissa
>>>
>>> [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
>>> [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
>>>>
>>>> I'm thinking of putting a 3D LUT proposal together but haven't gotten around
>>>> to it yet. We'll want a pre-blending 3D LUT, and possible a programmable
>>>> post-blending one as well.
>>>>
>>>> Thanks,
>>>> Harry
>>>>
>>>>> step of color correction, I don't see how to accommodate it in the
>>>>> mapping above, but I see DC already supports programming 3D LUT on DPP.
>>>>> Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
>>>>> the 3D LUT will be at the end of the color correction pipeline? Is there
>>>>> anything I need to worry about mapping DRM 3D LUT support? Or any
>>>>> advice?
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> Melissa
>>>
>>>
> 
> 

Re: AMD display drivers handling DRM CRTC color mgmt props

[Melissa Wen]

[-- Attachment #1: Type: text/plain, Size: 10842 bytes --]

On 04/22, Harry Wentland wrote:
> 
> 
> On 2022-04-22 10:28, Melissa Wen wrote:
> > On 04/21, Harry Wentland wrote:
> > > 
> > > 
> > > On 2022-04-21 15:20, Melissa Wen wrote:
> > > > On 04/21, Harry Wentland wrote:
> > > > > 
> > > > > 
> > > > > On 2022-04-21 10:37, Melissa Wen wrote:
> > > > > > Hi all,
> > > > > > 
> > > > > > I'm examining how DRM color management properties (degamma, ctm, gamma)
> > > > > > are applied to AMD display drivers. As far I could understand thanks
> > > > > > Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
> > > > > > per-plane color correction features:
> > > > > > 
> > > > Hi Harry,
> > > > 
> > > > Wow, thanks so much for all details!
> > > > > 
> > > > > DC programs some of the color correction features pre-blending but
> > > > > DRM/KMS has not per-plane color correction properties.
> > > > > 
> > > > > See this series from Uma Shankar for an RFC on how to introduce those
> > > > > properties for 1D LUTs and CSC matrix:
> > > > > https://patchwork.freedesktop.org/series/90826/
> > > > > 
> > > > > Bhanuprakash has a series of IGT tests for these properties:
> > > > > https://patchwork.freedesktop.org/series/96895/
> > > > > 
> > > > > I've rebased these on amd-staging-drm-next and maintain a kernel and IGT
> > > > > branch with these patches:
> > > > > https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
> > > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr
> > > > > 
> > > > > We've had many discussions with Weston guys on this. In order to merge the
> > > > > kernel properties we need a canonical userspace implementation that are
> > > > > using it. Weston guys are working towards that but if you want to suggest a
> > > > > different userspace to serve as that vehicle I'd be all ears. :)
> > > > > 
> > > > > Note that in order to show this all working we also need a Wayland Protocol
> > > > > update.
> > > > > 
> > > > > See
> > > > > https://gitlab.freedesktop.org/pq/color-and-hdr
> > > > > https://gitlab.freedesktop.org/swick/wayland-protocols
> > > > > https://gitlab.freedesktop.org/wayland/weston/-/issues/467
> > > > 
> > > > So, I've followed these discussions (until the issue on naming) because
> > > > initially I considered it addresses our current goals for color
> > > > correction. But after some discussions, what we are targeting is a 3D
> > > > LUT after blending (per-CRTC). I found past proposals on dri-devel
> > > > [1][2] to extend the DRM CRTC color management properties, but they
> > > > didn't move forward and were never applied.
> > > > 
> > > 
> > > They're stuck in limbo until we have an upstream userspace
> > > implementation that's making use of them.
> > 
> > Yes... afaiu, the basic requirements for all of these changes are IGT
> > tests + open userspace usage, right?
> > 
> 
> Correct. See [1] and [2].
> 
> [1] https://www.kernel.org/doc/html/latest/gpu/drm-kms.html#requirements
> [2] https://www.kernel.org/doc/html/latest/gpu/drm-uapi.html#open-source-userspace-requirements
> 
> > > 
> > > > > 
> > > > > > * - Input gamma LUT (de-normalized)
> > > > > > * - Input CSC (normalized)
> > > > > > * - Surface degamma LUT (normalized)
> > > > > > * - Surface CSC (normalized)
> > > > > > * - Surface regamma LUT (normalized)
> > > > > > * - Output CSC (normalized)
> > > > > > so DM is "adapting" those DRM per-CRTC properties to fit into three of
> > > > > > these color correction stages, which I guess are the surface stages:
> > > > > > 
> > > > > > * - Surface degamma LUT (normalized)
> > > > > > * - Surface CSC (normalized)
> > > > > > * - Surface regamma LUT (normalized)
> > > > > > 
> > > > > > I'm trying to understand what this mapping is doing. A comment mentions
> > > > > > that is not possible to do these color corrections after blending, so,
> > > > > > the same color correction pipe is performed on every plane before
> > > > > > blending?  (is the surface the plane?) Does this adaptation affect the
> > > > > > expected output?  Moreover, is there something that I misunderstood? :)
> > > > > > 
> > > > > 
> > > > > What's possible to do before and after blending has changed quite a bit
> > > > > between DCN generations. We program the CRTC Gamma and CTM after blending.
> > > > > See attached picture for a view relating the color bits between the DRM
> > > > > interface, DC interface and DCN 3.0 HW blocks.
> > > > 
> > > > This picture is really enlightening, thanks!
> > > > You said it changes between generations, therefore, I can't consider the
> > > > DCN 2.x family follow the same mapping, right? If so, can you share the
> > > > main differences for a DCN 2.x regarding per-CRTC properties?
> > > > 
> > > 
> > > See attached diagram for DCN 2.0.
> > 
> > Thanks again!
> > 
> > > 
> > > > > 
> > > > > > That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
> > > > > 
> > > > > Where do you see 3D LUT support in DRM? Is there a new proposal that I've
> > > > > missed?
> > > > 
> > > > So, it's exactly what I aim to work: a proposal to add 3D LUT to the
> > > > current range of DRM per-CRTC color properties. But I also need to
> > > > understand how this property will be mapped to AMD display once it
> > > > exists in the DRM framework.
> > > > 
> > > 
> > > Ah, nice to see. :)
> > > 
> > > > One of the things that caught my attention after seeing the attached
> > > > picture is the position of 3D LUT. I was expecting to see the 3D LUT
> > > > correction after gamma correction. Is this position a particularity of
> > > > DCN 3.0 (that varies between hw) or was I expecting a wrong color
> > > > correction pipeline at all?
> > > > 
> > > 
> > > Before DCN 3.0 there was no 3D LUT after blending.
> > > 
> > By comparing these diagrams, I'm curious: in case we have a per-CRTC 3D
> > LUT support on DRM, DCN 2.0 generations would initially map this
> > property as a pre-blending property on DPP (currently the same approach
> > for CTM, for example), right? But after we also have a per-plane color
> > management property, those per-CRTC property would be ignored? And how
> > about degamma for both generations? No problem if there isn't an answer
> > yet (many if's), but it may help me to think of a more generic solution.
> > 
> 
> We'll need to define what the expectations are for the API implementations
> in DRM drivers, as well as for the implementing userspace.
> 
> The way I think about this it might make sense to introduce a 3D LUT on a
> drm_plane, as well as on drm_crtc and a driver exposes whatever HW supports.
> Userspace can then figure out what it can use based on driver support.
> 
> It should be possible to use pre-blending 3D LUTs to accomplish the same as
> a post-blending 3D LUT, but we might need the ability to linearize before
> blending, but after applying the 3D LUT, and de-linearize after blending
> again. Something like this:
> 
> 3dlut > linearize (1dlut) > blend > de-linearize (1dlut) > output
> 
> Instead of this for the post-blending 3dlut:
> 
> linearize (1dlut) > blend > de-linearize (1dlut) > 3dlut > output
> 
> Though it depends a bit on the color model in the compositor or rendering
> app.

Sorry for the delay in reply, and thanks for explaining this behavior.

So, this topic of some DRM CRTC properties being programmed as
pre-blending came to my mind again when looking at the diagrams you
kindly shared before. Although DCN 2.0 doesn't have a post-blending CTM,
I see DCN 3.0 supports CTM pre and post blending (Gamut Remap), right?
But I also see stream->gamut_remap_matrix linked to DPP gamut remap in
the diagram.

AFAIU, CTM property on DCN 3.0 takes the same path from 2.0 and is only
managed by dpp_set_gamut_remap. I think CTM mapping to
stream->gamut_remap_matrix can be reprogrammed to mpc->set_gamut_remap
for DCN 3.0. I wonder if I can work on this reconnection to avoid
unexpected behavior (wiring DRM CRTC CTM to MPC instead of the current
DPP programming). 
So, is there any reason to not wired CTM to MPC on the DCN 3.0 driver?
Or, again, am I missing some part of the code about it?

Thanks,

Melissa

> 
> > > Note in the diagram that our HW (and DC interface) have a Shaper
> > > LUT available before the 3D LUT. You could expose if you want to
> > > shape your content post-blending before applying the 3D LUT.
> > > 
> > > The 3D LUT is most effective when it's in non-linear space. Currently
> > > DRM has no way to specify a way for drm_plane to be linearized (see notes
> > > (1) and (2)) so it is assumed that you're blending in non-linear space and
> > > therefore your pixels would already be non-linear going into your 3D LUT.
> > > 
> > > (1) unless you use the drm_plane PWL API that was proposed
> > > (2) amdgpu_dm is currently setting the drm_crtc degamma LUT on the
> > >      DC plane. This might lead to unexpected behavior when using
> > >      multiple planes (though I believe gamescope is making use of
> > >      this behavior).
> > 
> > Thanks for raising these points. In fact, I was considering unexpected
> > behavior when I saw this DRM <-> DC mapping. >>
> > > Have you looked at [1] yet? It might provide a good example on how to
> > > define a 3D LUT. For AMD HW you'll want a 17x17x17 LUT.
> > > 
> > > [1] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html
> > 
> > Not yet, but it seems helpful. I'll take as a reference... until now,
> > I've only examined details on DC drivers.
> > 
> 
> Sounds great.
> 
> Harry
> 
> > Thanks,
> > 
> > Melissa
> > 
> > > 
> > > Harry
> > > 
> > > > Melissa
> > > > 
> > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
> > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
> > > > > 
> > > > > I'm thinking of putting a 3D LUT proposal together but haven't gotten around
> > > > > to it yet. We'll want a pre-blending 3D LUT, and possible a programmable
> > > > > post-blending one as well.
> > > > > 
> > > > > Thanks,
> > > > > Harry
> > > > > 
> > > > > > step of color correction, I don't see how to accommodate it in the
> > > > > > mapping above, but I see DC already supports programming 3D LUT on DPP.
> > > > > > Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
> > > > > > the 3D LUT will be at the end of the color correction pipeline? Is there
> > > > > > anything I need to worry about mapping DRM 3D LUT support? Or any
> > > > > > advice?
> > > > > > 
> > > > > > Thanks in advance,
> > > > > > 
> > > > > > Melissa
> > > > 
> > > > 
> > 
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]