Re: AMD display drivers handling DRM CRTC color mgmt props

[Melissa Wen <mwen@igalia.com>]

[-- Attachment #1: Type: text/plain, Size: 10842 bytes --]

On 04/22, Harry Wentland wrote:
> 
> 
> On 2022-04-22 10:28, Melissa Wen wrote:
> > On 04/21, Harry Wentland wrote:
> > > 
> > > 
> > > On 2022-04-21 15:20, Melissa Wen wrote:
> > > > On 04/21, Harry Wentland wrote:
> > > > > 
> > > > > 
> > > > > On 2022-04-21 10:37, Melissa Wen wrote:
> > > > > > Hi all,
> > > > > > 
> > > > > > I'm examining how DRM color management properties (degamma, ctm, gamma)
> > > > > > are applied to AMD display drivers. As far I could understand thanks
> > > > > > Nicholas documentation on amdgpu_dm/amdgpu_dm_color, DC drivers have
> > > > > > per-plane color correction features:
> > > > > > 
> > > > Hi Harry,
> > > > 
> > > > Wow, thanks so much for all details!
> > > > > 
> > > > > DC programs some of the color correction features pre-blending but
> > > > > DRM/KMS has not per-plane color correction properties.
> > > > > 
> > > > > See this series from Uma Shankar for an RFC on how to introduce those
> > > > > properties for 1D LUTs and CSC matrix:
> > > > > https://patchwork.freedesktop.org/series/90826/
> > > > > 
> > > > > Bhanuprakash has a series of IGT tests for these properties:
> > > > > https://patchwork.freedesktop.org/series/96895/
> > > > > 
> > > > > I've rebased these on amd-staging-drm-next and maintain a kernel and IGT
> > > > > branch with these patches:
> > > > > https://gitlab.freedesktop.org/hwentland/linux/-/tree/color-and-hdr
> > > > > https://gitlab.freedesktop.org/hwentland/igt-gpu-tools/-/tree/color-and-hdr
> > > > > 
> > > > > We've had many discussions with Weston guys on this. In order to merge the
> > > > > kernel properties we need a canonical userspace implementation that are
> > > > > using it. Weston guys are working towards that but if you want to suggest a
> > > > > different userspace to serve as that vehicle I'd be all ears. :)
> > > > > 
> > > > > Note that in order to show this all working we also need a Wayland Protocol
> > > > > update.
> > > > > 
> > > > > See
> > > > > https://gitlab.freedesktop.org/pq/color-and-hdr
> > > > > https://gitlab.freedesktop.org/swick/wayland-protocols
> > > > > https://gitlab.freedesktop.org/wayland/weston/-/issues/467
> > > > 
> > > > So, I've followed these discussions (until the issue on naming) because
> > > > initially I considered it addresses our current goals for color
> > > > correction. But after some discussions, what we are targeting is a 3D
> > > > LUT after blending (per-CRTC). I found past proposals on dri-devel
> > > > [1][2] to extend the DRM CRTC color management properties, but they
> > > > didn't move forward and were never applied.
> > > > 
> > > 
> > > They're stuck in limbo until we have an upstream userspace
> > > implementation that's making use of them.
> > 
> > Yes... afaiu, the basic requirements for all of these changes are IGT
> > tests + open userspace usage, right?
> > 
> 
> Correct. See [1] and [2].
> 
> [1] https://www.kernel.org/doc/html/latest/gpu/drm-kms.html#requirements
> [2] https://www.kernel.org/doc/html/latest/gpu/drm-uapi.html#open-source-userspace-requirements
> 
> > > 
> > > > > 
> > > > > > * - Input gamma LUT (de-normalized)
> > > > > > * - Input CSC (normalized)
> > > > > > * - Surface degamma LUT (normalized)
> > > > > > * - Surface CSC (normalized)
> > > > > > * - Surface regamma LUT (normalized)
> > > > > > * - Output CSC (normalized)
> > > > > > so DM is "adapting" those DRM per-CRTC properties to fit into three of
> > > > > > these color correction stages, which I guess are the surface stages:
> > > > > > 
> > > > > > * - Surface degamma LUT (normalized)
> > > > > > * - Surface CSC (normalized)
> > > > > > * - Surface regamma LUT (normalized)
> > > > > > 
> > > > > > I'm trying to understand what this mapping is doing. A comment mentions
> > > > > > that is not possible to do these color corrections after blending, so,
> > > > > > the same color correction pipe is performed on every plane before
> > > > > > blending?  (is the surface the plane?) Does this adaptation affect the
> > > > > > expected output?  Moreover, is there something that I misunderstood? :)
> > > > > > 
> > > > > 
> > > > > What's possible to do before and after blending has changed quite a bit
> > > > > between DCN generations. We program the CRTC Gamma and CTM after blending.
> > > > > See attached picture for a view relating the color bits between the DRM
> > > > > interface, DC interface and DCN 3.0 HW blocks.
> > > > 
> > > > This picture is really enlightening, thanks!
> > > > You said it changes between generations, therefore, I can't consider the
> > > > DCN 2.x family follow the same mapping, right? If so, can you share the
> > > > main differences for a DCN 2.x regarding per-CRTC properties?
> > > > 
> > > 
> > > See attached diagram for DCN 2.0.
> > 
> > Thanks again!
> > 
> > > 
> > > > > 
> > > > > > That said, if the DRM color mgmt supports per-CRTC 3D LUT as the last
> > > > > 
> > > > > Where do you see 3D LUT support in DRM? Is there a new proposal that I've
> > > > > missed?
> > > > 
> > > > So, it's exactly what I aim to work: a proposal to add 3D LUT to the
> > > > current range of DRM per-CRTC color properties. But I also need to
> > > > understand how this property will be mapped to AMD display once it
> > > > exists in the DRM framework.
> > > > 
> > > 
> > > Ah, nice to see. :)
> > > 
> > > > One of the things that caught my attention after seeing the attached
> > > > picture is the position of 3D LUT. I was expecting to see the 3D LUT
> > > > correction after gamma correction. Is this position a particularity of
> > > > DCN 3.0 (that varies between hw) or was I expecting a wrong color
> > > > correction pipeline at all?
> > > > 
> > > 
> > > Before DCN 3.0 there was no 3D LUT after blending.
> > > 
> > By comparing these diagrams, I'm curious: in case we have a per-CRTC 3D
> > LUT support on DRM, DCN 2.0 generations would initially map this
> > property as a pre-blending property on DPP (currently the same approach
> > for CTM, for example), right? But after we also have a per-plane color
> > management property, those per-CRTC property would be ignored? And how
> > about degamma for both generations? No problem if there isn't an answer
> > yet (many if's), but it may help me to think of a more generic solution.
> > 
> 
> We'll need to define what the expectations are for the API implementations
> in DRM drivers, as well as for the implementing userspace.
> 
> The way I think about this it might make sense to introduce a 3D LUT on a
> drm_plane, as well as on drm_crtc and a driver exposes whatever HW supports.
> Userspace can then figure out what it can use based on driver support.
> 
> It should be possible to use pre-blending 3D LUTs to accomplish the same as
> a post-blending 3D LUT, but we might need the ability to linearize before
> blending, but after applying the 3D LUT, and de-linearize after blending
> again. Something like this:
> 
> 3dlut > linearize (1dlut) > blend > de-linearize (1dlut) > output
> 
> Instead of this for the post-blending 3dlut:
> 
> linearize (1dlut) > blend > de-linearize (1dlut) > 3dlut > output
> 
> Though it depends a bit on the color model in the compositor or rendering
> app.

Sorry for the delay in reply, and thanks for explaining this behavior.

So, this topic of some DRM CRTC properties being programmed as
pre-blending came to my mind again when looking at the diagrams you
kindly shared before. Although DCN 2.0 doesn't have a post-blending CTM,
I see DCN 3.0 supports CTM pre and post blending (Gamut Remap), right?
But I also see stream->gamut_remap_matrix linked to DPP gamut remap in
the diagram.

AFAIU, CTM property on DCN 3.0 takes the same path from 2.0 and is only
managed by dpp_set_gamut_remap. I think CTM mapping to
stream->gamut_remap_matrix can be reprogrammed to mpc->set_gamut_remap
for DCN 3.0. I wonder if I can work on this reconnection to avoid
unexpected behavior (wiring DRM CRTC CTM to MPC instead of the current
DPP programming). 
So, is there any reason to not wired CTM to MPC on the DCN 3.0 driver?
Or, again, am I missing some part of the code about it?

Thanks,

Melissa

> 
> > > Note in the diagram that our HW (and DC interface) have a Shaper
> > > LUT available before the 3D LUT. You could expose if you want to
> > > shape your content post-blending before applying the 3D LUT.
> > > 
> > > The 3D LUT is most effective when it's in non-linear space. Currently
> > > DRM has no way to specify a way for drm_plane to be linearized (see notes
> > > (1) and (2)) so it is assumed that you're blending in non-linear space and
> > > therefore your pixels would already be non-linear going into your 3D LUT.
> > > 
> > > (1) unless you use the drm_plane PWL API that was proposed
> > > (2) amdgpu_dm is currently setting the drm_crtc degamma LUT on the
> > >      DC plane. This might lead to unexpected behavior when using
> > >      multiple planes (though I believe gamescope is making use of
> > >      this behavior).
> > 
> > Thanks for raising these points. In fact, I was considering unexpected
> > behavior when I saw this DRM <-> DC mapping. >>
> > > Have you looked at [1] yet? It might provide a good example on how to
> > > define a 3D LUT. For AMD HW you'll want a 17x17x17 LUT.
> > > 
> > > [1] http://intel.github.io/libva/structVAProcFilterParameterBuffer3DLUT.html
> > 
> > Not yet, but it seems helpful. I'll take as a reference... until now,
> > I've only examined details on DC drivers.
> > 
> 
> Sounds great.
> 
> Harry
> 
> > Thanks,
> > 
> > Melissa
> > 
> > > 
> > > Harry
> > > 
> > > > Melissa
> > > > 
> > > > [1] https://lore.kernel.org/all/20201221015730.28333-1-laurent.pinchart+renesas@ideasonboard.com/
> > > > [2] https://github.com/vsyrjala/linux/commit/4d28e8ddf2a076f30f9e5bdc17cbb4656fe23e69
> > > > > 
> > > > > I'm thinking of putting a 3D LUT proposal together but haven't gotten around
> > > > > to it yet. We'll want a pre-blending 3D LUT, and possible a programmable
> > > > > post-blending one as well.
> > > > > 
> > > > > Thanks,
> > > > > Harry
> > > > > 
> > > > > > step of color correction, I don't see how to accommodate it in the
> > > > > > mapping above, but I see DC already supports programming 3D LUT on DPP.
> > > > > > Once DRM has the 3D LUT interface and DM mapped it as a DPP property,
> > > > > > the 3D LUT will be at the end of the color correction pipeline? Is there
> > > > > > anything I need to worry about mapping DRM 3D LUT support? Or any
> > > > > > advice?
> > > > > > 
> > > > > > Thanks in advance,
> > > > > > 
> > > > > > Melissa
> > > > 
> > > > 
> > 
> > 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]