Re: AMD display drivers handling DRM CRTC color mgmt props

[Harry Wentland <harry.wentland@amd.com>]

On 2022-04-22 10:28, Melissa Wen wrote:
> On 04/21, Harry Wentland wrote:
>>
>>
>> On 2022-04-21 15:20, Melissa Wen wrote:
>>> On 04/21, Harry Wentland wrote:
>>>>
>>>>
>>>> On 2022-04-21 10:37, Melissa Wen wrote:
>>>>> Hi all,
>>>>>
>>>>> I'm examining how DRM color management properties (degamma, ctm, gamma)
>>>>> are applied to AMD display drivers. As far I could understand thanks
>>>>> Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
>>>>> per-plane color correction features:
>>>>>
>>> Hi Harry,
>>>
>>> Wow, thanks so much for all details!
>>>>
>>>> DC programs some of the color correction features pre-blending but
>>>> DRM/KMS has not per-plane color correction properties.
>>>>
>>>> See this series from Uma Shankar for an RFC on how to introduce those
>>>> properties for 1D LUTs and CSC matrix:
>>>> https://patchwork.freedesktop.org/series/90826/
>>>>
>>>> Bhanuprakash has a series of IGT tests for these properties:
>>>> https://patchwork.freedesktop.org/series/96895/
>>>>
>>>> I've rebased these on amd-staging-drm-next and maintain a kernel and IGT
>>>> branch with these patches:
>>>> https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
>>>> https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr
>>>>
>>>> We've had many discussions with Weston guys on this. In order to merge the
>>>> kernel properties we need a canonical userspace implementation that are
>>>> using it. Weston guys are working towards that but if you want to suggest a
>>>> different userspace to serve as that vehicle I'd be all ears. :)
>>>>
>>>> Note that in order to show this all working we also need a Wayland Protocol
>>>> update.
>>>>
>>>> See
>>>> https://gitlab.freedesktop.org/pq/color-and-hdr
>>>> https://gitlab.freedesktop.org/swick/wayland-protocols
>>>> https://gitlab.freedesktop.org/wayland/weston/-/issues/467
>>>
>>> So, I've followed these discussions (until the issue on naming) because
>>> initially I considered it addresses our current goals for color
>>> correction. But after some discussions, what we are targeting is a 3D
>>> LUT after blending (per-CRTC). I found past proposals on dri-devel
>>> [1][2] to extend the DRM CRTC color management properties, but they
>>> didn't move forward and were never applied.
>>>
>>
>> They're stuck in limbo until we have an upstream userspace
>> implementation that's making use of them.
> 
> Yes... afaiu, the basic requirements for all of these changes are IGT
> tests + open userspace usage, right?
> 

Correct. See [1] and [2].

[1] https://www.kernel.org/doc/html/latest/gpu/drm-kms.html#requirements
[2] 
https://www.kernel.org/doc/html/latest/gpu/drm-uapi.html#open-source-userspace-requirements

>>
>>>>
>>>>> * - Input gamma LUT (de-normalized)
>>>>> * - Input CSC (normalized)
>>>>> * - Surface degamma LUT (normalized)
>>>>> * - Surface CSC (normalized)
>>>>> * - Surface regamma LUT (normalized)
>>>>> * - Output CSC (normalized)
>>>>> so DM is "adapting" those DRM per-CRTC properties to fit into three of
>>>>> these color correction stages, which I guess are the surface stages:
>>>>>
>>>>> * - Surface degamma LUT (normalized)
>>>>> * - Surface CSC (normalized)
>>>>> * - Surface regamma LUT (normalized)
>>>>>
>>>>> I'm trying to understand what this mapping is doing. A comment mentions
>>>>> that is not possible to do these color corrections after blending, so,
>>>>> the same color correction pipe is performed on every plane before
>>>>> blending?  (is the surface the plane?) Does this adaptation affect the
>>>>> expected output?  Moreover, is there something that I misunderstood? :)
>>>>>
>>>>
>>>> What's possible to do before and after blending has changed quite a bit
>>>> between DCN generations. We program the CRTC Gamma and CTM after blending.
>>>> See attached picture for a view relating the color bits between the DRM
>>>> interface, DC interface and DCN 3.0 HW blocks.
>>>
>>> This picture is really enlightening, thanks!
>>> You said it changes between generations, therefore, I can't consider the
>>> DCN 2.x family follow the same mapping, right? If so, can you share the
>>> main differences for a DCN 2.x regarding per-CRTC properties?
>>>
>>
>> See attached diagram for DCN 2.0.
> 
> Thanks again!
> 
>>
>>>>
>>>>> That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
>>>>
>>>> Where do you see 3D LUT support in DRM? Is there a new proposal that I've
>>>> missed?
>>>
>>> So, it's exactly what I aim to work: a proposal to add 3D LUT to the
>>> current range of DRM per-CRTC color properties. But I also need to
>>> understand how this property will be mapped to AMD display once it
>>> exists in the DRM framework.
>>>
>>
>> Ah, nice to see. :)
>>
>>> One of the things that caught my attention after seeing the attached
>>> picture is the position of 3D LUT. I was expecting to see the 3D LUT
>>> correction after gamma correction. Is this position a particularity of
>>> DCN 3.0 (that varies between hw) or was I expecting a wrong color
>>> correction pipeline at all?
>>>
>>
>> Before DCN 3.0 there was no 3D LUT after blending.
>>
> By comparing these diagrams, I'm curious: in case we have a per-CRTC 3D
> LUT support on DRM, DCN 2.0 generations would initially map this
> property as a pre-blending property on DPP (currently the same approach
> for CTM, for example), right? But after we also have a per-plane color
> management property, those per-CRTC property would be ignored? And how
> about degamma for both generations? No problem if there isn't an answer
> yet (many if's), but it may help me to think of a more generic solution.
> 

We'll need to define what the expectations are for the API 
implementations in DRM drivers, as well as for the implementing userspace.

The way I think about this it might make sense to introduce a 3D LUT on 
a drm_plane, as well as on drm_crtc and a driver exposes whatever HW 
supports. Userspace can then figure out what it can use based on driver 
support.

It should be possible to use pre-blending 3D LUTs to accomplish the same 
as a post-blending 3D LUT, but we might need the ability to linearize 
before blending, but after applying the 3D LUT, and de-linearize after 
blending again. Something like this:

3dlut > linearize (1dlut) > blend > de-linearize (1dlut) > output

Instead of this for the post-blending 3dlut:

linearize (1dlut) > blend > de-linearize (1dlut) > 3dlut > output

Though it depends a bit on the color model in the compositor or 
rendering app.

>> Note in the diagram that our HW (and DC interface) have a Shaper LUT
>> available before the 3D LUT. You could expose if you want to shape your
>> content post-blending before applying the 3D LUT.
>>
>> The 3D LUT is most effective when it's in non-linear space. Currently
>> DRM has no way to specify a way for drm_plane to be linearized (see notes
>> (1) and (2)) so it is assumed that you're blending in non-linear space and
>> therefore your pixels would already be non-linear going into your 3D LUT.
>>
>> (1) unless you use the drm_plane PWL API that was proposed
>> (2) amdgpu_dm is currently setting the drm_crtc degamma LUT on the
>>      DC plane. This might lead to unexpected behavior when using
>>      multiple planes (though I believe gamescope is making use of
>>      this behavior).
> 
> Thanks for raising these points. In fact, I was considering unexpected
> behavior when I saw this DRM <-> DC mapping. >>
>> Have you looked at [1] yet? It might provide a good example on how to
>> define a 3D LUT. For AMD HW you'll want a 17x17x17 LUT.
>>
>> [1] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html
> 
> Not yet, but it seems helpful. I'll take as a reference... until now,
> I've only examined details on DC drivers.
> 

Sounds great.

Harry

> Thanks,
> 
> Melissa
> 
>>
>> Harry
>>
>>> Melissa
>>>
>>> [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
>>> [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
>>>>
>>>> I'm thinking of putting a 3D LUT proposal together but haven't gotten around
>>>> to it yet. We'll want a pre-blending 3D LUT, and possible a programmable
>>>> post-blending one as well.
>>>>
>>>> Thanks,
>>>> Harry
>>>>
>>>>> step of color correction, I don't see how to accommodate it in the
>>>>> mapping above, but I see DC already supports programming 3D LUT on DPP.
>>>>> Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
>>>>> the 3D LUT will be at the end of the color correction pipeline? Is there
>>>>> anything I need to worry about mapping DRM 3D LUT support? Or any
>>>>> advice?
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> Melissa
>>>
>>>
> 
>